Page 266 - Kaleidoscope Academic Conference Proceedings 2024

2024 ITU Kaleidoscope Academic Conference

Table 3 – Comparison of data fields available in AIID and Table 5 – Top seven source-domains of the reports in AIID
AIAAIC
Source domain Reports
Fields available Fields available Fields available theguardian.com 143
in both AIID and only in AIID only in AIAAIC theverge.com 95
AIAAIC nytimes.com 94
Type; washingtonpost.com 71
Released (year); wired.com 69
Country(ies); vice.com 54
Incident ID; Sector(s); reuters.com 53
Title/ Headline; System name(s); bbc.com 53
Description; Technology(ies);
Alleged harmed or
Occurrence date; Purpose(s);
nearly or nearly Table 6 – Top seven sectors of the incidents in AIAAIC
System deployer; Media trigger(s);
harmed parties
System developer; Issue(s);
Transparency;
Sectors Incidents %age
External harms;
Media/entertainment/sports/arts 193 21.3%
Internal harms
Automotive 86 9.5%
Politics 75 8.3%
Table 4 – Top seven submitters of the incidents in AIID Technology 60 6.6%
Education 58 6.4%
Banking/financial services 40 4.4%
Submitters Incidents %age
Business/professional services 35 3.9%
Daniel Atherton 149 23%
Anonymous 96 15%
Khoa Lam 93 14% infringement, but qualifying them as AI-incidents will depend
Ingrid Dickinson CSET 49 7% on the definition of AI-incident. Similarly, incident id
Roman Yampolskiy 29 4% AIAAIC1395 [31] at s.no. 4 in Table (2) relates to the
AIAAIC 25 4% ethics of the authors and the screening processes followed
by the journals and does not meet the AI-incident definition
Kate Perkins 21 3%
provided by OECD [15]. Also, it is challenging to determine
the severity of the incidents based on the information available
AIAAIC. While AIID does not have data fields to capture in both databases.
this data, as indicated in Table 7, the incidents reported in
AIID are predominantly related to AI systems developed by Inference: One significant gap is the absence of standardized
American companies. definitions and taxonomies related to AI incidents and AI
harms. It becomes challenging to compare and analyze
4.8 Data sharing incidents across different domains and jurisdictions without
consistent guidelines for categorizing incidents, their harms,
Table 9 outlines the formats available for downloading and severity levels.
incident data from the two databases and the limitations on
accessible data. Recommendation 1:Standardise AI-incident and AI-harms
taxonomies: Develop standard taxonomies for AI-incidents
5. GAP ANALYSIS AND RECOMMENDATIONS and AI-harms based on domain, severity, root causes,
and impact on SDGs to enable consistent classification
This section analyses the results to identify gaps in and analysis of AI-incidents across different sectors and
existing AI-incident reporting mechanisms and recommends jurisdictions, facilitating benchmarking and trend analysis.
areas for standardization and policy initiatives. These
recommendations aim to address observed gaps, enabling 5.2 Bias, inconsistencies, and misclassification
meticulous AI-incident reporting and contributing to the
achievement of the UN SDGs. Observation: As mentioned in the previous paragraph,
three of the incidents cited in Table 2 [28], [29], and
5.1 Lack of definitions and taxonomies [31] may not qualify as AI incidents depending on the
definition considered. The reporting of incidents, their
Observation: There is a lack of consistency in qualifying review, classification as incidents, and assessing their harm
the reported events as incidents. The AIAAIC incidents quotients being manual are prone to biases and capabilities
with ids AIAAIC1449 [28] and AIAAIC1439 [29] cited in of the individuals involved. Biases and inconsistencies
Table 2 relate to ethical practices and possible copyright in incident reporting can skew perceptions of AI-related

– 222 –