[AI Ethics Seminar EP.2] Research aimed at properly understanding the risks of AI

Why do we study AI ethics? We believe that AI ethics research is essential to the development of AI that benefits people, so we’re always thinking about AI ethics throughout the entire AI lifecycle to make sure there are no side effects of the technology that we haven't considered.  Our AI ethics research process involves a diverse group of people. Our AI Ethics Seminar brings together AI researchers, business developers, lawyers, UI/UX designers, data scientists, and AI education planners. They analyze notable AI ethics research and recent trends, sharing their insights at the seminar. These discussions are then incorporated into our AI ethics activities.  This post will highlight key topics discussed at this year's AI Ethics Seminar. We hope that the concerns and insights we share will help advance AI ethics.

Behind the seemingly endless possibilities of AI technology, there are those who talk about the harms and dangers of AI, but everyone has a different idea of the risks they haven't experienced. In order to specifically prepare for future AI risks without exaggerating or minimizing them too much, we need experience and knowledge of the real risks of AI. At this AI Ethics Seminar, recent research and efforts to understand the “real risks and harms from AI” were introduced, and attendees discussed how we can prepare for them.

From Databases to Classification Systems: 
Efforts to Systematically Organize AI Risks 

First, here are a few attempts to gather a sense of what problems AI is causing around the world. It started with channels like the AI Incident Database (AIID), where people voluntarily record AI incidents. This database is still operational, but with so many incidents happening around the world, it became difficult to keep track of them all, so the OECD created the AI Incidents Monitor (AIM), a tool to detect AI incidents in real time. These case study tools document AI-related incidents, providing researchers and policymakers with the knowledge they need to define AI risks and harms, as well as to understand and manage these risks.

From 2014 to October 2024, there were approximately 13,574 incidents attributed to AI, as reported by the OECD. The number of incidents is rising rapidly, as is the interest in AI. Compared to 2023, the number of incidents in 2024 increased by 139%. The incident records provide a sense of the nature of AI risks. The incidents recorded last month were reported to be related to privacy (111), human rights (86), information security (148), democracy (39), safety (59), fairness (46), and malfunction (91), respectively. Incident records also record information about the severity of the incident (whether it resulted in physical damage, such as death or injury), the stakeholders (companies, general users, vulnerable groups, etc.), and more.

Image 1. AI incident database^[1]

Image 2. AI incident monitoring system (OECD)^[2]

Previously, incident records were often categorized under different systems by various organizations or institutions. However, recently, there have been numerous efforts to organize these records under a unified classification system. For example, some are based on the scope of the incident's harmful effects and related ethical values (NIST AI Risk Framework, 2022), the physical, social, and psychological forms of the incident's harm (EPIC, 2023), the behavior of AI service users (Ferrara, 2024), and the type of AI model (Weidinger, 2022). This post will introduce some of the studies that were impressive.

1. Defining and Categorizing AI Risks (MIT)

To define AI risks, MIT built the AI Risk Repository, a database of 777 AI risks. The study divided AI risks into two categories: Causal Taxonomy, based on the cause of the AI risk, and Domain Taxonomy, based on the field of damage.

1. Causal Taxonomy: This classifies how, when, and why the AI risk occurred, by who created them (human/AI itself), time (before/after deployment), and intent (intentional/unintentional). For example, it distinguishes between risks that are caused by human error in manipulating the system and risks that could be caused by flaws in the AI system itself.

2. Domain Taxonomy: AI risks are categorized into seven high-level areas (e.g., misinformation) and low-level areas (false information). In addition, they are divided into areas such as malicious use and misuse, human-computer interaction, social interaction, economic and environmental impacts, and the safety of AI systems.

Looking at the causal classification, AI risks are often attributed to humans (34%), but the AI system itself (51%) is the source of the majority of incidents. When analyzing the intentionality of the incidents, we found that they were similarly split between intentional (35%) and unintentional (37%). When classified by time of occurrence, the majority of incidents occurred after the system was deployed (65%). 

When classified by domain, the most discussed risks were AI system safety (76%), socioeconomic and environmental harm (73%), and discrimination and harm (71%), while human-computer interaction (41%) and misinformation (44%) were discussed less frequently. Certain risk sub-areas, such as unfair discrimination (8%) and AI goal conflicts (8%), were discussed more frequently, while AI rights (less than 1%) and competitive dynamics (1%) were discussed less frequently.

Image 3. Causal/domain classification of AI risks, AI Risk Repository^[3]

The AI Risk Repository is currently open and free for anyone to use. How can these AI risk classifications be utilized? Policymakers can use them to assess and oversee the risks of AI for policy decisions. Academic researchers can use them to create educational materials on AI risks and identify new areas of research. Companies can use them to identify new, previously undocumented risks, assess existing risks internally, and consider how to mitigate them.

2. Risks Arising from Generative AI (EPIC)

The Electronic Privacy Information Center (EPIC) is a nonprofit organization established to protect privacy and related fundamental rights in the digital environment and has been active for 30 years. EPIC states that discussions about the issues arising from generative AI are fundamentally no different from those concerning privacy, transparency, and fairness in the existing digital environment.

EPIC has released a report^[5] highlighting the problems that are arising from the use of generative AI among other AI technologies. The report calls for a focus on the problems at hand, rather than the future problems of generative AI causing humanity to go extinct. This is because we need to look at who is actually being harmed and what that harm is before we can think about what we can do to fix it. The report focuses on four areas where urgent intervention is needed.

EPIC ranked Endangering Elections as the number one risk in the report. AI-generated misinformation and disinformation is having a significant impact on elections, including a recent robocall with AI-generated audio that mimicked President Biden’s voice and told New Hampshire residents not to vote in the primary. We've also seen a noticeable increase in AI-generated content from Russia and China targeting the U.S. presidential election. False content, particularly in non-English languages, is harder to detect, making election interference easier. Voice phishing is also becoming a problem during elections, with impersonators of candidates, political parties, and interest groups reaching out to voters to solicit donations.

The second problem the report cited was full-blown data and privacy breaches (Eroding Privacy). AI models are driven by a belief in the “scaling law and “maximalist data use,” which means that the basic principles of data privacy - data minimization and purpose limitation - are not being followed. Data collected from web scraping often includes revenge porn, child sexual abuse material (CSAM), illegal personal information, and more. Data-related incidents are reportedly occurring at an average rate of 158 per 10,000 corporate users per month, including source code exposure (22 incidents), regulated data (18 incidents), intellectual property (4 incidents), and posts containing passwords and keys (4 incidents).

After the privacy breaches caused by data collection, the functional and qualitative degradation of data (Data Degradation) is another issue. The internet is flooded with AI-generated news that is full of errors and plagiarism. The interplay of AI-generated content overflow and indiscriminate data collection means that data can no longer be relied upon to serve as a source of knowledge and information for human society. In fact, low-quality, fraudulent books generated by AI are flooding Amazon and confusing consumers into making the wrong purchases.

The report also pointed to the negative effects of content licensing schemes, the proliferation of misinformation, privacy and data security, intellectual property rights violations, exacerbating the climate crisis, devaluing labor, discriminating against marginalized groups, and strengthening market power and monopolies. It states specifically what companies and governments developing technologies should do to prevent adverse effects of technology adoption.

3. A Case Study of the Misuse of Generative AI (Google)

Google reviewed and classified nearly 200 cases of intentional misuse of generative AI by individuals or organizations. It excluded harm caused by structural/functional limitations of the AI and focused only on risks created intentionally by users. Based on 191 cases (from January 2023 to March 2024, including media reports and social media searches on platforms like X and Reddit), it classified potential motives and misuse strategies.

Image 4. The most common AI misuse tactics^[4]

Let's first take a look at how generative AI was used in these harm cases. The most common tactic used was impersonating a person or falsifying information, followed by scaling and amplification, and sockpuppeting, which involves creating fake profiles to shape public opinion. 

What's interesting about this misuse of AI is that it uses basic generative AI functions. There were only a few instances where attacks that require a high level of technical expertise were used, as the basic functions of generative AI were sufficient to enable impersonation, information falsification, and amplification. Impersonation, information falsification, and amplification are problems that predate the advent of AI, but they are rapidly growing and worsening as AI is utilized, not only because people can be more sophisticated, but also because it is cheap and easy to use.

So what's causing this damage, and what are the purposes for which people have abused AI? The number one purpose was to manipulate public opinion. Generative AI-manipulated content has been used effectively to create countless rumors and conspiracy theories, mostly related to wars, social unrest, and economic crises. Voice files impersonating President Biden and manipulated images of Hawaiian wildfires were systematically spread to drive political division. 

The next purpose was fraud and monetization. Internet search results are also being flooded with articles written by generative AI. Google has even announced new search policies to combat the flood of low-quality AI content that is automatically generated to generate ad revenue. A number of sexualized content using generative AI has also been traded for financial gain. There have been a growing number of cases of AI-generated images or video calls using video being used to extort money from victims by impersonating co-workers or supervisors, or phishing emails that convincingly mimic an organization's trademark or logo.

To combat the growing misuse of AI, it’s important to consider the ease with which generative AI is accessible to a large number of users and to consider how to address the purpose of their use. Sometimes, directly educating AI users directly can have a vaccine-like effect. For example, to prevent the spread of manipulated information, research has shown that simply having people watch a short video that teaches them how to recognize manipulated content improves their ability to identify trustworthy content.

How LG AI Research Is Working to Prevent AI Harm

LG AI Research also monitors existing cases of AI harm and strives to prevent future problems. Examining and discussing actual cases of harm with employees through the AI Ethics Seminar serves as education to prevent harm. 

In addition, existing cases of AI harm are reflected in LG AI Research's AI ethical impact assessment. The AI ethical impact assessment is a process that discusses social and ethical issues that may arise from the AI projects LG AI Research is working on, considering the characteristics of the AI projects. Since 2024, LG AI Research has been conducting AI ethical impact assessments for all ongoing research projects, with direct participation of project planners and researchers in the discussion process.  Through this, we aim to directly diagnose possible social and ethical impacts in the entire AI lifecycle and prevent anticipated problems.

In this year’s ongoing AI ethical impact assessment, we comprehensively reviewed potential issues by stakeholder impact, data-related issues, model development process, and service operation/deployment areas. As a result, we found that data-related issues tend to be the most easily identified. As the performance and reliability of AI systems are highly dependent on the quality of data, data-related issues must be carefully reviewed. At LG AI Research, issues such as data copyright and personal information are systematically managed through separate data compliance procedures. In addition, each researcher makes continuous efforts to ensure that data is representative and up-to-date.

Image 5. Types of potential problems managed by the LG AI Research AI ethical impact assessment

The AI ethical impact assessment provides an opportunity for researchers to think beyond the performance of AI technology and consider the important societal implications of its future use. It allows them to recognize potential problems and consider whether any of them can be addressed during the research stage. A record of anticipated problems and their resolution can help researchers move more quickly when real-world problems arise because they've thought about how to respond during the research stage.

Based on the results of the AI ethical impact assessment, LG AI Research is also conducting various studies to understand and prevent AI risk cases. We apply safety filters to AI services, study generative AI content detection to prevent misuse of generative AI, and propose technical solutions such as technology to delete privacy-invasive data. 
 
It's important to understand what AI risks are currently occurring. This makes it possible to focus on the problems that are real and need to be addressed first, not threats that don't exist. Existing research analyzing risk cases has helped us recognize the kinds of harm and incidents that AI can cause and it’s expected that more research will be conducted on ways to solve each problem. Organizations can create policies and conduct research to proactively identify and prevent potential impacts of AI. LG AI Research will also endeavor to manage the issues identified by AI ethical impact assessments and take preventive measures to share them with society. Also we will enhance the reliability and stability of AI systems through policies and research that prevent real-world harm. 

AI Ethics Seminar Series

#1. [AI Ethics Seminar EP.1] Red Team Research Trends and Applications for Generative AI
#3. [AI Ethics Seminar EP.3] Introduction to Key AI Toolkits for Designing Trustworthy AI Experiences and Practical Application Guide
#4. [AI Ethics Seminar EP.4] Building a Responsible AI Society through AI Agents
#5. [AI Ethics Seminar EP.5] AI Ethics from a UI/UX Designer's Perspective