[AI Ethics Seminar 2025 EP.1] How AI Is Changing Human Critical Thinking

The Journey Toward Trustworthy AI: Introducing LG AI Research’s “AI Ethics Seminar”  Why does LG AI Research study AI ethics? Because we believe that AI ethics research is essential to developing AI that truly benefits people. Technology is ultimately created by humans, and as AI continues to advance, responsible development and use are just as important as technological innovation itself.  Throughout the entire AI lifecycle, we engage diverse members of our organization in open discussions and practices around AI ethics. At our AI Ethics Seminar, AI researchers, business developers, UI/UX designers, data scientists, and AI policy planners come together to explore ethical challenges in AI. The seminar serves as a forum for participants to deepen their understanding of the real value of AI ethics and to discuss how to apply ethical principles in their day-to-day work.  We hope discussions on AI ethics will spread beyond our organization and help build a broader ecosystem of trustworthy AI. In this post, we’ll introduce the key topics discussed at the 2025 AI Ethics Seminar as part of our ongoing series on responsible AI.

 

Generative AI is revolutionizing nearly every field, from writing and data analysis to coding, maximizing productivity. While this enables us to work faster and more accurately, it also raises a question: As our reliance on AI grows, are we losing opportunities to think for ourselves?

Recent studies suggest that AI is reshaping not only how humans think, but also who leads the thinking process. This article focuses on how critical thinking is evolving in the age of AI, and introduces how LG AI Research's ChatEXAONE incorporates these changes into its design philosophy.

1. The evolution of critical thinking: How much should we trust AI?

In a paper by Lee et al. (2025)^[1], researchers from Carnegie Mellon University and Microsoft analyzed changes in the critical thinking of 319 knowledge workers when using generative AI. The results were noteworthy.

1. The higher the reliance in AI, the less effort was made to think independently.

2. Those who had high confidence in their own judgment maintained critical thinking even when using AI.

 

The research team describes this shift as a transition from a “Doer” to a “Steward.” Whereas humans previously defined and solved problems, AI now drafts solutions, while humans review and refine them. In other words, the focus of critical thinking has shifted from creating information to verifying and adjusting it.

However, this shift carries one significant risk. As reliance in AI accumulates over time, there is a danger of falling into over-reliance, where we become too quick to accept AI’s results and skip verification altogether.

2. The key to balanced collaboration: What is “appropriate reliance?”

Microsoft Research's Passi et al.(2024)^[2] proposed the concept of “appropriate reliance” to address this issue. This concept refers to a balanced state of reliance where AI is neither blindly trusted nor unconditionally distrusted.

1. CAIR(Correct AI Reliance): The percentage of people who rely on AI when AI is right

2. CSR(Correct Self-Reliance): The percentage of people who rely on themselves when AI is wrong

 

The higher both of these factors are, the more users can be assessed as appropriately relying on the AI. The Passi research team presented this as a composite metric called AoR (Appropriateness of Reliance).
True AI utilization should be assessed not only by accuracy, but also by the quality of collaboration between humans and AI to achieve better results together.

Furthermore, this study emphasizes that generative AI responses occupy a “gray area” where they are partially correct and partially incorrect. Therefore, users should go beyond simply reviewing AI’s responses and assess how trustworthy the AI is in each context, adjusting their level of reliance accordingly.

3. Four factors that cause over-reliance in AI

The Passi study identified four primary factors that cause users to over-rely on AI. These factors provide important implications for AI system design.

1. Difference in expertise: Beginners lacking verified knowledge tend to trust AI answers outright, while experts possess a higher ability to cross-check results.

2. Conversation style: Multi-turn conversation structures, involving multiple exchanges with AI, help reduce overreliance.

3. Task characteristics: In coding, review steps are easily skipped, while in writing, the “anchoring effect,” where users become influenced by AI's initial suggestions, frequently occurs.

4. Verification cost: AI's fluent speech and rapid speed induce an illusion of accuracy in users, causing them to skip verification steps. This aligns with the psychological tendency to reduce time and effort spent on verification, ultimately reinforcing over-reliance.

 

Ultimately, it is crucial to design systems that users can easily verify and that help them correctly understand how AI operates.

4. Three key strategies for AI design

The Passi research team proposed three design principles for AI to support users' critical thinking. These strategies embody the principle that AI should act as a facilitator rather than a substitute for human thought.

1. Explanations to assist in verifying AI responses

1. Rather than providing lengthy explanations for why AI arrived at an answer, it should present evidence that allows users to judge its correctness for themselves.

2. For example, when AI was designed to critique its own results, users were 50% more effective at detecting errors.

2. AI response uncertainty notifications

1. When AI indicates its accuracy, such as “This may not be accurate,” users interpret the results more cautiously.

2. However, excessive expressions of uncertainty can undermine trustworthiness itself, making balance crucial.

3. Questions that prompt users to think

1. When AI poses questions like “Are there any counterexamples to this conclusion?” instead of definitive answers, users are prompted to reconsider rather than accept the results at face value.

2. This design is intended to foster critical thinking without imposing excessive cognitive load.

 

5. Critical Questions for Designing and Using AI Systems

Based on the research findings, the following questions must be examined when designing or using AI systems.

1. Is verification easy? Is sufficient evidence provided to quickly check and compare AI responses?

2. Does it help users understand the limitations of AI? Can users clearly recognize that results may not always be perfect answers?

3. Is it designed for user context? Does interaction with AI vary depending on the situation, such as between beginners and experts, or simple queries and in-depth analysis?

4. Do humans and AI together produce better results? Does actual work performance and problem-solving capability improve when humans and AI collaborate, rather than relying solely on AI?

 

These four questions focus not on “How smart have we made AI?” but on “How well does AI assist human thinking?”

6. ChatEXAONE: Revealing how AI thinks

Image 1. ChatEXAONE's Deep Research Mode

ChatEXAONE is an AI conversational assistant developed based on EXAONE, LG AI Research’s large-scale language model (LLM). It significantly enhances work convenience and efficiency through diverse capabilities such as real-time web-based Q&A, document-based Q&A, and coding assistance.

In particular, its advanced Reasoning and Deep Research features enable the system to gather relevant information, analyze problems and contexts, verify reasoning paths, identify potential errors, and consider alternative approaches before drawing conclusions—all while transparently disclosing AI’s thought process at each step.

This feature helps users perceive AI not as a mere tool for answers, but as a partner that thinks and grows with them. By revealing AI's reasoning process step by step, ChatEXAONE allows users to review the evidence and logic behind each conclusion, refining and expanding upon AI’s decisions. Through this transparency, users are encouraged to critically assess AI’s reasoning—rather than accept results—and to form their own judgments.

Not an era of thinking less, but an era of thinking differently

Critical thinking in the age of AI cannot be sustained by individual effort alone. It begins with technology and design that empower people to think critically and reflect. ChatEXAONE goes beyond merely providing answers, offering an environment where users can naturally review and make their own judgments through an interaction structure that transparently reveals AI's thought processes and reasoning.

This approach shows that AI is evolving not as a tool that replaces human thought, but as a partner designed to inspire critical thinking. LG AI Research embodies this philosophy through ChatEXAONE, pioneering a future where humans and AI learn, grow, and create intellectual synergy together.

Try ChatEXAONE Beta →

 

AI Ethics Seminar 2025 Series

#2. [AI Ethics Seminar 2025 EP.1] How AI Is Changing Human Critical Thinking