Page 267 - Kaleidoscope Academic Conference Proceedings 2024
P. 267

Innovation and Digital Transformation for a Sustainable World




            Table 7 – Top seven deployers of the AI systems in AIID  Table 9 – Sharing of incident data by AIAAIC and AIID


            Deployer of AI system  incidents      %age         Data            AIAAIC              AIID
            tesla                     39           6%          sharing
            facebook                  36           6%          Format        Available as a   Weekly snapshots
            google                    28           4%                        Google Sheet.    of the database in
            unknown                   23           4%                                         JSON, MongoDB,
            amazon                    21           3%                                          and CSV format
            openai                    20           3%          Information  Contributor details      -
            cruise                    12           2%          not           are not public.
                                                               accessible   Harm data is only
                                                                              accessible to
            Table 8 – Top seven countries of the incidents in AIAAIC
                                                                           premium members.
                                                               APIs              None              None
            Countries              Incidents      %age
            USA                       424         46.9%       exchange. Secondly, the captured data is generally insufficient
            UK                        59          6.5%        for assessing the severity and proper categorization of the
            China                     53          5.9%        incidents.
            USA; Global               26          2.9%
            Global                    21          2.3%        Recommendation 3:   Standardise AI-incident database
            India                     21          2.3%        structures: Standardising the fields of AI-incident databases
            Canada                    18          2.0%        will ensure that the collected data has sufficient granularity
                                                              required for analysis. It will also facilitate interoperability,
                                                              data exchange, and ease of aggregating data from multiple
           risks and hinder efforts to develop inclusive and equitable
                                                              databases.
           solutions.
                                                              5.4 Inadequate motive to report incidents
           Inference: The AI-incident databases may suffer from the
           biases of the submitters or the reviewers related to attributes
                                                              Observation: As indicated in Table 1, incident reporting
           such as their political leanings, gender, minority groups,
                                                              in both databases is voluntary and lacks incentives.
           countries, and so on. Further, different individuals classify
                                                              Without legal mandates or rewards, reporting relies on
           the incidents and their harms in distinct ways, depending on
                                                              reporters’ discretion and motivation, potentially resulting in
           their exposure, capabilities, and understanding, which may
                                                              underreporting.
           lead to inconsistencies and misclassification.
                                                              Inference: Fears of data privacy breaches may discourage
           Recommendation 2:   Define guidelines for AI-incident  reporting, leading to incomplete or underreported AI
           database quality audits: Formulate procedures to regularly  incidents.  Without transparent and privacy-protective
           audit the AI-incident databases for consistency, checking for  reporting mechanisms, stakeholders may hesitate to disclose
           misreporting, misclassification, reported incidents meeting  incidents, hampering the effectiveness of incident databases.
           the defined criteria, and so on.                   Additionally, fragmentation among databases complicates
                                                              data collection and analysis, impeding comprehensive risk
           5.3 Insufficient and incompatible data fields      understanding and response.
           Observation:  Table 3 compares the columns available  Recommendation 4:  Develop regulatory and policy
           in the two databases, showing that only six fields are  frameworks for AI-incident reporting: Make sector-specific
           compatible between the two datasets, while the remaining  legal provisions to mandate or encourage AI-incident
           are incompatible. Secondly, these databases do not have  reporting. Global standards organizations such as ITU should
           enough detailed data fields needed for thorough analysis,  develop standardized regulatory and policy frameworks for
           like identifying the causes, context, and impact of reported  AI-incident reporting to enable consistency across nations.
           incidents. AIID does not have fields to capture impacted
           sectors (Table 6), impacted countries (Table 8), and so on.  5.5 Narrow base of the incidents reported
           On the other hand, AIAAIC does not capture details of the
           harmed (or nearly harmed) parties the way AIID does, such  Observation: Though the incident reporting is open to the
           as Facebook users, minority groups, patients, and so on.  public, only a few individuals report the incidents. Table 4
                                                              indicates that just four individuals, excluding the anonymous
           Inference: Different and incompatible structures of AI  ones, have reported half of the incidents in AIID. Further,
           incident databases make aggregating data from multiple  the top sources of the reports submitted to AIID are from
           databases difficult, limit interoperability, and restrict data  American or European newspapers, as detailed in Table 5.




                                                          – 223 –
   262   263   264   265   266   267   268   269   270   271   272