Page 265 - Kaleidoscope Academic Conference Proceedings 2024
P. 265

Innovation and Digital Transformation for a Sustainable World




                         3.  METHODOLOGY                          Table 1 – Incident reporting in AIAAIC and AIID

           The study adopted the following methodology:
                                                                                     AIAAIC       AIID
            1. Executed an exhaustive search and literature review to  What can be reported  Incidents and  Real-world
               discover AI incident repositories.                                    controversies  harms or
                                                                                     driven by and  near harms
            2. Isolated four potential repositories: AIAAIC, AIID,                   relating to  caused by AI
               AILD, and AVID. Given that AILD focuses on AI related                 AI.          systems.
               legal aspects and AVID emphasizes identifying AI  Incidents reported (as  905      657
               system vulnerabilities, shortlisted the two open-access  on 05-05-2024)
               repositories, AIID and AIAAIC, for further scrutiny.  Who can report  Anyone       Anyone
                                                               incidents
            3. Examined the policies, scope, reporting procedures, and  Submissions reviewed  Yes  Yes
               review mechanisms of the AIID and AIAAIC databases  before publishing?
               to comprehend their operational frameworks.     Nature of reporting   Voluntary    Voluntary
                                                               Incentive for reporting  None      None
            4. Submitted an incident to each database to discern their
               reporting protocols and procedural intricacies.
                                                                 Table 2 – Snapshot of Incidents reported in AIAAIC
            5. Retrieved and scrutinized publicly available data from
               both databases to evaluate their content and structure.  AIAAIC ID# Headline               Ref.
                                                              AIAAIC1449   Adobe trained Firefly AI model on  [28]
            6. Investigated the repositories to pinpoint gaps in           competitor images
               standardization across various dimensions, including:  AIAAIC1439  OpenAI scrapes YouTube to train  [29]
               incident reporting protocols, quality control, data         GPT-4
               interoperability, comprehensiveness of data, contributor  AIAAIC1414  Leonardo AI generates celebrity  [30]
               and  source  diversity,  sector-specific  coverage,         non-consensual porn images
               geographical coverage, and data sharing protocols.  AIAAIC1395  Scientific journals publish papers  [31]
                                                                           with AI-generated introductions
            7. Tabulated observations and inferred key insights based  AIAAIC1368  Microsoft Copilot generates fake  [32]
               on the conducted analysis.                                  Putin comments on Navalny death
                                                              AIAAIC1356   ChatGPT ’goes crazy’, speaks   [33]
            8. Formulated  recommendations  for  standardization           gibberish
               activities to address identified gaps and enhance the
               effectiveness of AI incident reporting practices.
                                                              4.4 Contributors to the Databases
                             4. RESULTS
                                                              Table 4 lists the top seven submitters of the published
                                                              incidents in AIID. They reported more than 70% of all the
           This section presents the observations and results of the study.
                                                              incidents in AIID. AIAAIC does not have data fields to
           The next section analyses and draws inferences from them.
                                                              capture this data.
           4.1 Incident reporting
                                                              4.5 Sources of the reports submitted to the databases
           Table 1 provides the basics of incident reporting in AIAAIC
                                                              Table 5 provides details of the top seven source domains of
           and AIID. Both have similar processes for incident reporting,
                                                              the reports submitted to AIID. AIAAIC does not have data
           though their scopes are slightly different.
                                                              fields to capture this data.
           4.2  AI-Incident snapshot                          4.6 Sector Coverage

           Sample incidents reported in AIAAIC, shortlisted for  Table 6 details the top seven sectors of the incidents reported
           analysis, are listed in Table 2. These were extracted for  in AIAAIC. While AIID does not have data fields to capture
           analysis by filtering on the criteria “Occurred” = “2024” and  this data, Table 7 provides details of the top seven deployers
           “Country(ies)” = “Global”.                         of the AI systems with incidents reported in AIID.

           4.3  Interoperability and data sharing             4.7 Geographical coverage

           Table 3 compares the data fields available in the two  Table 8 lists the top seven countries related to the geographic
           databases. They have different data structures.    origin and/or primary extent of the incidents reported in




                                                          – 221 –
   260   261   262   263   264   265   266   267   268   269   270