Korea's Flagship AI, Completed with Proprietary Technology: K-EXAONE

Globally, artificial intelligence (AI) technology is advancing rapidly, with leading AI nations, particularly the United States and China, engaged in fierce competition, especially in the field of large language models (LLMs). These nations are developing massive AI models with parameters ranging from hundreds of billions to trillions, continuously enhancing their performance.

So what about the situation in Korea? Korea is also striving to reach world-class levels in the AI field, but currently lacks core infrastructure such as dedicated AI data centers and high-performance AI chips. Consequently, it has focused primarily on developing relatively small-scale AI models (at the level of billions of parameters) up to now.

However, today's AI has evolved beyond mere technology to become critical infrastructure underpinning a nation's economy, industry, and security, emerging as a core element of national competitiveness. Therefore, the time has come for a national AI model with world-class performance.

Amid this trend, the Korean government is promoting its “Proprietary AI Foundation Model Project” to support core resources such as high-performance GPUs, data, and talent for the development of the AI industry. LG AI Research is actively participating in this project and has developed a large-scale AI model called K-EXAONE. This post introduces K-AI and K-EXAONE, developed using LG AI Research's proprietary technology.

1. Reducing learning and inference costs by ensuring efficiency

The size of an LLM is primarily determined by the number of parameters in the model. While a larger number of parameters enables the model to learn more complex patterns and achieve higher performance, it also requires significantly more computing resources and time. The LG AI Research consortium explored approaches focused on improving performance through efficiency to overcome these limitations. To this end, we established a strategy unique to K-EXAONE, and the specific methods will be examined below. 

1.1 Mixture-of-Experts Model Structure
While existing EXAONE models were based on a dense architecture, K-EXAONE adopts a Mixture-of-Experts (MoE) structure to maximize the model's scalability and efficiency. Specifically, K-EXAONE implements an MoE design utilizing 128 experts. It activates the top 8 experts and one shared expert, enabling a total of 9 experts to operate simultaneously. Consequently, while the total number of parameters reaches 236 billion, only about 10%, or 23 billion parameters, are actually activated, significantly enhancing resource efficiency.

1.2 Multi-Token Prediction
Existing language models use autoregressive decoding. In other words, they predict one token at a time and incorporate the result into the next prediction. While this approach is intuitive, it has limitations in terms of inference speed and efficiency.

In contrast, K-EXAONE incorporates a Multi-Token Prediction (MTP) module, designed to predict the next token and the token after that simultaneously. This enables K-EXAONE to achieve approximately 1.5 times faster inference speed compared to existing models, allowing users to receive responses more quickly.

1.3 Tokenizer: Providing more information with fewer tokens
If a single token can hold more characters, the number of tokens required to express the same sentence decreases, resulting in improved overall efficiency.

The existing EXAONE series utilized approximately 100,000 vocabulary items, but K-EXAONE has expanded this to approximately 150,000. This enables the simultaneous processing of a wider variety of words and expressions, resulting in outstanding efficiency across diverse languages and specialized fields (e.g., science, technology, mathematics, code, etc.).

Additionally, K-EXAONE adopts the SuperBPE strategy, grouping frequently occurring word combinations into a single token. For example, it treats “New York,” which consists of two tokens, as a single token like “New_York.” This is called a Superword Token, accounting for approximately 20% of the K-EXAONE lexicon. As a result, K-EXAONE's tokenizer achieved an average efficiency improvement of approximately 30% compared to the existing EXAONE, measured in bytes per token. 

2. Learning methods to achieve global-level performance 

2.1 Pre-learning
The pre-learning of K-EXAONE consists of three stages where the model sequentially learns foundational knowledge, specialized knowledge, and reasoning capabilities. Notably, in the third stage, data including thinking trajectories is synthesized and utilized for learning to enhance reasoning capabilities.

This approach enables the model to learn step-by-step reasoning processes, rather than merely outputting the correct answer. In other words, it helps the model understand the logical flow it should follow and the intermediate steps it must take when solving problems. This learning method plays a crucial role in significantly enhancing the model's reasoning capabilities during subsequent post-learning stages.

2.2 Context Length Extension
K-EXAONE supports long context lengths capable of processing up to approximately 260,000 tokens, achieving this through a two-step process of progressively extending the context length.

In the first extension phase, the context length was extended from approximately 8,000 tokens to about 33,000 tokens, and in the second phase, up to approximately 260,000 tokens. A critical point is that using only long-context data for learning can degrade performance on short contexts. To address this, a rehearsal dataset containing high-quality short-context samples was employed alongside the extended context lengths during learning.

To verify that the context length was extending properly, the Needle-In-A-Haystack (NIAH) test was utilized. This test evaluates the ability to find specific information within a long context, and it was performed periodically during the learning process to validate the model's long-context comprehension capability.

2.3 Post-learning
Pre-learning is similar to reading textbooks and building background knowledge. At this stage, the model reads various texts, learning language structure, word meanings, and general knowledge. Post-learning, on the other hand, is the process of learning how to solve problems or understand user commands and perform tasks based on that knowledge.

K-EXAONE's post-learning process consists of three stages, as follows.

(1) Supervised Fine-Tuning (SFT): Learning to understand and responding to a variety of instructions
As we transition from the era of generative AI to the era of agentic AI, teaching AI to use various tools has become a critical challenge. To achieve this, environments where AI can directly connect and interact with real-world tools and APIs must be created, a process that is costly and time-consuming.

Therefore, K-EXAONE created a virtual tool usage environment instead of real-world tools to generate data. For example, it virtually created coding tools or common tool invocation scenarios, and also set verifiable success criteria for each task. Through this created environment, K-EXAONE was able to learn hundreds of realistic and verifiable tool usage tasks.

(2) Reinforcement Learning (RL): Improved reasoning ability
Reinforcement learning is a method where a model learns by receiving rewards. It's similar to how a child receives praise from their parents when they complete a puzzle, and strives to do better through that praise (reward). AI learns in the same way: it receives a reward when it gets the correct answer and receives no reward when it gets it wrong.

K-EXAONE has conducted reinforcement learning across diverse fields including math, coding, science, and following instructions. It uses verifiable rewards during the learning process, meaning the rewards can be objectively verified. For example, when solving a coding problem, the system awards 1 point if the code works correctly and 0 points if an error occurs.

K-EXAONE uses the newly designed AGAPO (Asymmetric Sampling and Global Advantage Policy Optimization) algorithm instead of the existing GRPO (Group Relative Policy Optimization) algorithm. The existing GRPO excludes samples from the learning process if all responses are either correct or incorrect, judging that such samples are not helpful for learning. In contrast, AGAPO utilizes even samples where all responses are incorrect. It does so by assigning a small negative reward to such samples, thereby guiding the model to avoid erroneous inference paths. This approach enables the model to learn from its mistakes and perform more accurate inferences.

(3) Preference Alignment: Adjusting the model's response style to match human preferences
K-EXAONE has significantly enhanced its logical reasoning abilities in fields such as mathematics, coding, and science by understanding diverse instructions through large-scale supervised and reinforcement learning. However, as AI being merely smart is insufficient, a final step is required to enable users to interact more familiarly, safely, and naturally.

To achieve this, K-EXAONE undergoes a preference learning phase, where the model is further refined to better align with human preferences.

K-EXAONE employs a new method called GrouPER (Group-wise SimPER), which advances the existing SimPER algorithm. GrouPER learns by comparing multiple responses simultaneously. For example, it generates several responses for a single question, evaluates which response is more appropriate, and then incorporates this into the learning process. This method of evaluating samples in groups helps stabilize learning by offsetting noise from individual samples, compared to learning by evaluating each sample individually.

The preference score for each response is evaluated based on two criteria:

1. Rule-based evaluation: Evaluated using clear criteria such as "Is this a safe response?" and "Did you follow the instructions?"
2. Rubric-based evaluation: Evaluated using criteria such as creativity, logic, and naturalness

After that, it determines which response is more appropriate on a group basis and updates the model based on the results.

3. Performance Evaluation Results

Evaluating an LLM's performance is not simply about how well it gets the right answers. What matters is determining how effectively this model can be utilized in the diverse situations we actually encounter.

The reason it is difficult to evaluate an LLM's performance with just one test is that each test (benchmark) is often tailored to a specific task. Therefore, to comprehensively evaluate an LLM's performance, tests must be performed using benchmarks selected from diverse domains.

K-EXAONE underwent performance evaluations across several domains, including ▲knowledge ▲mathematics ▲coding ▲agent tool usage ▲following instructions ▲understanding long-term context ▲Korean and multilingual capabilities ▲safety.

To validate the performance of K-EXAONE, the models selected as comparators are the latest model in the EXAONE series, the EXAONE 4.0 32B model, and the open-weight models demonstrating the best performance in their class: gpt-oss-120b (hereafter GPT), Qwen3-235B-A22B-Thinking-2507 (hereafter Qwen3), DeepSeek-V3.2 (hereafter DeepSeek).

K-EXAONE demonstrated outstanding competitiveness across diverse fields including knowledge, mathematics, and coding. It particularly achieved excellent results on the knowledge benchmark MMLU-Pro, proving its academic knowledge comprehension and reasoning abilities, and outperformed GPT and Qwen3 on the mathematics benchmark AIME 2025. In the coding domain on LiveCodeBench v6, it demonstrated improved performance over Qwen3 and DeepSeek, enhancing its potential for practical application in real coding tasks.

K-EXAONE demonstrated excellent performance on the τ²-Bench, an agent tool usage domain benchmark, proving its tool selection and information exploration capabilities in environments requiring multi-step interactions. Furthermore, K-EXAONE achieved respectable scores on the IFBench instruction-following benchmark, the Korean-language KoBALT benchmark, and the MMMLU used for multilingual evaluation, proving its stable performance.

Finally, it demonstrated the best performance in the KGC-Safety benchmark for evaluating safety. This achievement reflects robustness against various harmful prompts while simultaneously meeting Korean sociocultural contexts and global ethical standards, showing that K-EXAONE can effectively mitigate risks when handling sensitive queries without compromising performance in general tasks.

K-EXAONE has demonstrated the world-class caliber of Korean AI technology by ranking 7^th globally and 1^st in Korea among open-weights models on the Artificial Analysis Intelligence Index, a global AI benchmark. This achievement goes beyond a high score, this milestone signifies that Korea is staking its claim to AI sovereignty and carving out a unique strategic position in the global market.

K-EXAONE has become a stepping stone for South Korea's AI technology to leap forward to the next level. It enhanced efficiency by introducing a mixed expert structure, multi-token prediction technology, and an advanced tokenizer, and through multi-stage pre-learning and context length extension utilizing rehearsal datasets, it achieved the capability to process up to approximately 260,000 tokens.

Furthermore, through a post-learning process comprising large-scale supervised learning, reinforcement learning, and preference learning, it has secured high accuracy and reliable safety across diverse tasks. Beyond mere technical achievements, it has demonstrated its potential as a model that solves real-world problems, supports better decision-making, and drives innovation when applied to actual industries and research fields.

We will continue our ongoing AI research and development, ensuring K-EXAONE positively impacts the AI ecosystem, and as Korea's leading national AI representative in the global market, we will lead the nation's AI competitiveness.

K-EXAONE Hugging Face  →

K-EXAONE Friendli AI  →