Seonghwan Kim
2025.05.29
2025 LLM Trends: from FM to AI Agent
1. Introduction
In recent years, large language models (LLMs) have experienced explosive growth. Beyond being assistants that answer simple questions for users, they now demonstrate advanced reasoning capabilities, solving Ph.D-level questions or difficult problems at the level of a math competition. With this advancement in reasoning ability, they are leveraging a variety of tools, such as searching, computer manipulation, API calls, or code execution to perform complex tasks on behalf of users, greatly improving usability.
With these developments in LLM, LG AI Research has also contributed to the advancement of LLM technology by releasing the EXAONE 3.0, EXAONE 3.5, and EXAONE Deep[1,2,3] models, which have achieved leading performance in various benchmarks. We are continuing our research to develop models with even better performance and enhanced usability.
This post will take a look at some of the recent highlights from the Foundation Model to Post-training and Agents, and discuss where LLMs are headed in the future.
With these developments in LLM, LG AI Research has also contributed to the advancement of LLM technology by releasing the EXAONE 3.0, EXAONE 3.5, and EXAONE Deep[1,2,3] models, which have achieved leading performance in various benchmarks. We are continuing our research to develop models with even better performance and enhanced usability.
This post will take a look at some of the recent highlights from the Foundation Model to Post-training and Agents, and discuss where LLMs are headed in the future.
2. Foundation Models: The Core of LLM Advancement
Foundation Model refers to a pre-trained model based on a large dataset, most notably a language model trained to predict the next token from massive web data using a Transformer. Recently, foundation models have become a core component of a wide array of LLM applications.
The performance and architecture of the foundation model have a significant impact on all subsequent training stages. Once a model architecture and training method have been determined at the initial design stage, modifying them later is challenging. Thus, it is crucial to carefully consider their implications not only for pre-training but also for subsequent post-training and inference stages when designing a foundation model. In addition, pre-training a foundation model is typically the most time-consuming and expensive stage of the entire training process, so an efficient training methodology is important.
Pre-training of a foundation model mainly involves scaling the size of the model and the training data according to the scaling law[9, 10]. However, determining what data to use and which model architectures to adopt within limited resources remains a crucial challenge. Next, we will discuss the Mixture-of-Experts (MoE) model and FP8 training method for efficiently scaling model size.
The performance and architecture of the foundation model have a significant impact on all subsequent training stages. Once a model architecture and training method have been determined at the initial design stage, modifying them later is challenging. Thus, it is crucial to carefully consider their implications not only for pre-training but also for subsequent post-training and inference stages when designing a foundation model. In addition, pre-training a foundation model is typically the most time-consuming and expensive stage of the entire training process, so an efficient training methodology is important.
Pre-training of a foundation model mainly involves scaling the size of the model and the training data according to the scaling law[9, 10]. However, determining what data to use and which model architectures to adopt within limited resources remains a crucial challenge. Next, we will discuss the Mixture-of-Experts (MoE) model and FP8 training method for efficiently scaling model size.
Mixture-of-Experts for Efficient Large-Scale Model Training
Various architectures are being studied for more efficient training of foundation models. A recently notable architecture is the Mixture-of-Experts (MoE). MoE models are efficient architectures that enable large-scale model training at relatively low computational cost.
In an MoE architecture, the Feed-Forward Network (FFN) layer of a traditional transformer is replaced by an MoE layer. This MoE layer consists of multiple FFNs, termed 'experts,' and selectively activates only a subset of these experts for each input token to perform computation. From a computational perspective, transformers that use a large FFN layer are called dense models, while MoEs that selectively use only a few experts are called sparse models.
Compared to dense models with similar computational budgets or numbers of active parameters, MoE models can support larger total parameter counts. While MoE models may exhibit slightly lower performance than dense models of equivalent size under similar training conditions (primarily due to training instability), they can compensate for this by leveraging their computational efficiency to process substantially more training tokens. MoE models typically offer faster training and inference speeds, and greater computational efficiency since fewer parameters are actively used at any given time.
Research on MoE architectures has primarily focused on techniques for reliable and effective training. Key areas include optimizing the number and size of experts, as well as enhancing training stability and effectiveness through methods such as using auxiliary losses to promote balanced token routing to experts and preventing token-dropping. Examples of these techniques can be found in models like Switch Transformer[4], Mixtral[5], and Megablocks[6].
DeepSeek proposed DeepSeekMoE [7], which was recently applied to the DeepSeek-V3 [8] model. DeepSeekMoE is characterized by fine-grained expert segmentation and shared experts. The fine-grained expert segmentation approach encourages each expert to learn knowledge that is more specialized to their particular field, while the common knowledge that is generally needed is handled by the shared expert, allowing them to learn more efficiently in their respective areas of expertise and common knowledge.
In an MoE architecture, the Feed-Forward Network (FFN) layer of a traditional transformer is replaced by an MoE layer. This MoE layer consists of multiple FFNs, termed 'experts,' and selectively activates only a subset of these experts for each input token to perform computation. From a computational perspective, transformers that use a large FFN layer are called dense models, while MoEs that selectively use only a few experts are called sparse models.
Compared to dense models with similar computational budgets or numbers of active parameters, MoE models can support larger total parameter counts. While MoE models may exhibit slightly lower performance than dense models of equivalent size under similar training conditions (primarily due to training instability), they can compensate for this by leveraging their computational efficiency to process substantially more training tokens. MoE models typically offer faster training and inference speeds, and greater computational efficiency since fewer parameters are actively used at any given time.
Research on MoE architectures has primarily focused on techniques for reliable and effective training. Key areas include optimizing the number and size of experts, as well as enhancing training stability and effectiveness through methods such as using auxiliary losses to promote balanced token routing to experts and preventing token-dropping. Examples of these techniques can be found in models like Switch Transformer[4], Mixtral[5], and Megablocks[6].
DeepSeek proposed DeepSeekMoE [7], which was recently applied to the DeepSeek-V3 [8] model. DeepSeekMoE is characterized by fine-grained expert segmentation and shared experts. The fine-grained expert segmentation approach encourages each expert to learn knowledge that is more specialized to their particular field, while the common knowledge that is generally needed is handled by the shared expert, allowing them to learn more efficiently in their respective areas of expertise and common knowledge.
Image 1. Basic MoE model and DeepSeekMoE model architecture. [7]FP8 Training: Enhancing Speed and Efficiency
Low-precision training is gaining traction to make LLMs faster and more efficient. Currently, many models are trained in BF16 (Bfloat16), a 16-bit floating-point format, but techniques for training at even lower precisions, such as FP8, are becoming increasingly important.
The precision used for LLM training has evolved from the initial 32 bits (float32) to 16 bits like BF16, and more recently, there has been active research into applying even lower precision beyond 8 bits (FP8). Using FP8 precision theoretically offers up to a 2x computational speedup and reduced memory usage. Furthermore, it enhances training efficiency by decreasing data communication between GPUs and nodes—a significant bottleneck in large-scale model training.
However, there is a downside to lowering precision, as it can negatively impact model performance by reducing training stability. Therefore, the key to low-precision training techniques is to train reliably while minimizing this performance degradation.
The precision used for LLM training has evolved from the initial 32 bits (float32) to 16 bits like BF16, and more recently, there has been active research into applying even lower precision beyond 8 bits (FP8). Using FP8 precision theoretically offers up to a 2x computational speedup and reduced memory usage. Furthermore, it enhances training efficiency by decreasing data communication between GPUs and nodes—a significant bottleneck in large-scale model training.
However, there is a downside to lowering precision, as it can negatively impact model performance by reducing training stability. Therefore, the key to low-precision training techniques is to train reliably while minimizing this performance degradation.
Mixed Precision Framework
For FP8 training, DeepSeek-V3 proposed a mixed precision framework [8]. This strategically employs FP8 for computationally intensive modules and FP32 or BF16 precision for numerical stability to prevent performance degradation.
In the figure below, the General Matrix Multiplication (GEMM) operation part in yellow is composed of three operations: Fprop (forward pass), Dgrad (activation backward pass), and Wgrad (weight backward pass), all of which are executed at FP8 precision. This design provides theoretical up to 2x speedups compared to the original BF16, while also lowering memory usage.
However, despite the advantages of FP8, some operators require high precision due to their numerical sensitivity. A fine-grained analysis identifies specific operations like embeddings, output heads, MoE gating modules, normalization, and attention, which are maintained at higher precision (e.g., BF16/FP32) to ensure model stability. A number of other strategies are also used, such as fine-grained quantization, to improve the accuracy of training.
In the figure below, the General Matrix Multiplication (GEMM) operation part in yellow is composed of three operations: Fprop (forward pass), Dgrad (activation backward pass), and Wgrad (weight backward pass), all of which are executed at FP8 precision. This design provides theoretical up to 2x speedups compared to the original BF16, while also lowering memory usage.
However, despite the advantages of FP8, some operators require high precision due to their numerical sensitivity. A fine-grained analysis identifies specific operations like embeddings, output heads, MoE gating modules, normalization, and attention, which are maintained at higher precision (e.g., BF16/FP32) to ensure model stability. A number of other strategies are also used, such as fine-grained quantization, to improve the accuracy of training.
Image 2. Mixed Precision Framework used in DeepSeek-V3 model. [8]3. Aligning Models with Human Preferences: Instruction Tuning and Reasoning Models
Pre-trained models are trained on large amounts of web data and generate text reflecting the distribution of web data. Consequently, their responses are often not aligned with human preferences and may fail to address user requests appropriately. Therefore, post-training is required to give the model the ability to generate answers that align with human preferences, solve more difficult problems, or utilize a variety of tools.
In the post-training process, data from different domains and tasks are collected and synthesized, filtered, and then subjected to supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF). The following sections will discuss recent post-training methods, from instruction data generation to reasoning models.
In the post-training process, data from different domains and tasks are collected and synthesized, filtered, and then subjected to supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF). The following sections will discuss recent post-training methods, from instruction data generation to reasoning models.
Generating high-quality instruction data
To instill a variety of abilities into the model, the most important step is creation of high-quality instruction tuning data from a variety of domains and tasks. For the EXAONE 3.5 model, the process of obtaining such high-quality data begins with preprocessing a large web corpus to categorize data into domains like mathematics, code, and science. A knowledge taxonomy is extracted from the categorized data, and various instruction data is generated according to the extracted taxonomy. This generated data subsequently undergoes an evolutionary process to progressively increase its complexity and difficulty. This process aims to enhance the model's capabilities and ultimately yields higher-quality data. SFT is performed based on the collected and generated data.
Image 3. Instruction data generation pipeline of EXAONE 3.5 model. [2]Learning Human Preferences
With high-quality instruction data, SFT models learn to follow user instructions and acquire diverse skills and response styles. However, while SFT models are trained to imitate high-quality generated data, they may not generate responses that consider crucial aspects such as helpfulness, harmlessness, and accuracy. Therefore, an RLHF process is needed to align model with human preferences.
The EXAONE 3.5 model employs a dedicated process for data generation and training in preference learning. This data generation process utilizes multiple response generators to produce a set of candidate responses. A reward model assigns a score to each response. Subsequently, the response with the highest score is labeled 'chosen', and the response with the lowest score is labeled 'rejected'. These selections are used to construct the preference data. This preference data is used in a two-stage preference learning approach to train the SFT model to generate highly human-preferred responses and to maximize performance.
The EXAONE 3.5 model employs a dedicated process for data generation and training in preference learning. This data generation process utilizes multiple response generators to produce a set of candidate responses. A reward model assigns a score to each response. Subsequently, the response with the highest score is labeled 'chosen', and the response with the lowest score is labeled 'rejected'. These selections are used to construct the preference data. This preference data is used in a two-stage preference learning approach to train the SFT model to generate highly human-preferred responses and to maximize performance.
Image 4. Preference Learning training pipeline of the EXAONE 3.5 model. [2]GRPO and Verifiable Reward
Online reinforcement learning (RL) stands out as a powerful methodology for the fine-grained tuning and alignment of large language models, primarily due to its capacity to directly leverage reward signals. Notable methods include Proximal Policy Optimization (PPO) [13] and Group Relative Policy Optimization (GRPO) [14], which was used to train the DeepSeek R1 model.
GRPO can be considered a PPO variant, sharing fundamental aspects of PPO's learning process and objectives. The main differences from PPO are that GRPO removes the value model which is used in the advantage calculation, and GRPO replaces PPO’s advantage with group normalization by the group average and group standard deviation among the response groups. Also, to avoid computational complexity, the KL penalty is applied as an addition to the final loss instead of applying it token by token as in PPO.
By removing the value model, GRPO reduces the number of models requiring training. This reduction enhances learning stability and resource efficiency in RL. Furthermore, GRPO has demonstrated effectiveness in both general and reasoning tasks.
GRPO can be considered a PPO variant, sharing fundamental aspects of PPO's learning process and objectives. The main differences from PPO are that GRPO removes the value model which is used in the advantage calculation, and GRPO replaces PPO’s advantage with group normalization by the group average and group standard deviation among the response groups. Also, to avoid computational complexity, the KL penalty is applied as an addition to the final loss instead of applying it token by token as in PPO.
By removing the value model, GRPO reduces the number of models requiring training. This reduction enhances learning stability and resource efficiency in RL. Furthermore, GRPO has demonstrated effectiveness in both general and reasoning tasks.
Image 5. GRPO's objective. [14]
Image 6. Illustration of PPO and GRPO. [14]
The importance of Reinforcement Learning with Verifiable Rewards (RLVR) [15] is growing. Verifiable rewards typically rely on rule-based reward functions that determine outcomes such as simple correct answer matching or successful code execution. Verifiable rewards offer several advantages. Firstly, they make reward hacking difficult because the rewards are verifiable and straightforward (e.g., +1 for a correct answer and -1 for an incorrect one, based on the rule-based function's output). Secondly, this approach (or RLVR) is efficient in terms of both training speed and GPU memory, as it does not require an additional, trainable reward model during RL training.
Image 7. Training flow of RL using Verifiable Reward. [15]Enhancing Deep Reasoning Capabilities
Reasoning model refers to a model that solves problems in various strategies such as reflection, backtracking, verification, and subgoal-setting in the chain-of-thought (CoT) process, mirroring human thought processes. Reasoning models are significant as they have the potential to advance human development by tackling previously unsolved problems and by demonstrating capabilities to solve complex tasks that challenge even human experts.
Reasoning models are trained using large-scale reinforcement learning. A characteristic of reasoning models is that their performance on complex tasks tends to improve with more elaborate or 'deeper' reasoning, a process that utilizes greater computational resources (test-time compute) during problem-solving. The figure below shows how OpenAI o1[16] improves accuracy on the American Invitational Mathematics Examination (AIME), a high-stakes math test, as test-time compute is gradually increased.
Reasoning models are trained using large-scale reinforcement learning. A characteristic of reasoning models is that their performance on complex tasks tends to improve with more elaborate or 'deeper' reasoning, a process that utilizes greater computational resources (test-time compute) during problem-solving. The figure below shows how OpenAI o1[16] improves accuracy on the American Invitational Mathematics Examination (AIME), a high-stakes math test, as test-time compute is gradually increased.
Image 8. Graph showing AIME performance improvement as test-time compute increases. [16]Since the o1 model, many researchers have tried different methods to achieve o1-level reasoning capabilities, but have not been able to achieve the same performance as the o1 model. However, DeepSeek R1[17] showed that reasoning models can be trained using simple rule-based rewards and large-scale reinforcement learning. R1 showed that during large-scale RL training, the number of tokens in the model's reasoning process gradually increases, and the model's reasoning ability improves accordingly.
Image 9. Graph showing performance and response length increasing with RL training of the DeepSeek R1 model. [17]An interesting phenomenon observed during RL training of a reasoning model is when the model being trained solves a problem and then revisits its initial problem-solving process. These points in the model's reasoning where it revisits or switches between different problem-solving strategies are called “aha moments,” and as these “aha moments” occur, the model's reasoning ability improves. The strength and beauty of RL lies in the fact that problem-solving strategies are not directly taught during the RL process, but rather the model autonomously explores and develops more advanced strategies.
Image 10. The “Aha Moment” of a reasoning model. [17]LG AI Research released the EXAONE Deep model in March 2025, which is based on the EXAONE 3.5 Instruct model and specialized in math, code, and science. The EXAONE Deep model comprises models with 2.4B, 7.8B, and 32B sizes. The Deep models demonstrated significantly superior performance compared to similarly sized models at 2.4B and 7.8B, and the 32B model showed performance comparable to that of the much larger DeepSeek R1 model, despite being only about 5% its size.
We trained the EXAONE Deep model in the following order: SFT, DPO, and GRPO. First, we collected and generated high-difficulty math, code, and science data, comprising of 1.6 million SFT data. For DPO training data, we generated multiple responses similar to the preference data generation pipeline used in the previous EXAONE 3.5 model, filtered the data through a verification process, and trained using the SimPER [23] algorithm, which is a DPO variant. Finally, we adapted the GRPO algorithm to train our reasoning model to maximize the performance of the model.
Image 11. Performance of the EXAONE Deep model. [3]
4. The Rise of AI Agents
AI agents are rapidly emerging as a pivotal area of advancement. Agents solve complex problems by autonomously utilizing external tools for a given goal. They are the culmination of many years of AI technologies, including reasoning, multimodal understanding, tool-use, and reinforcement learning (RL). Recently, with the emergence of advanced reasoning models, standardization of tool-use methods, and improvements in image understanding, agents have been attracting attention for their potential to broaden the scope of LLM and bring innovation to various fields.
What is ‘AI Agent’?
The definition of an agent varies across companies, but in general, an agent can be thought of as a system that interacts with its environment to process a given task, using a variety of tools to perform complex tasks. What distinguishes them from existing assistants is that they have a high degree of autonomy, allowing the model to make independent decisions to solve problems rather than following a set procedure.
Image 12. Autonomous Agent. [18]Tool Use via Function Calling
Function Calling is how an LLM uses tools. Tools are needed to extend the functionality of the LLM beyond text generation to access up-to-date information, as well as to process user requests by accessing computations or external services. To enable an LLM to utilize tools via text-based interaction, developers define a function for use as input to the LLM, and the model analyzes the natural language input and processes it by generating a structured output to call a pre-defined function, including the necessary arguments. Importantly, the model itself does not execute functions directly. Rather, the LLM provides the structured output needed for function calling, and the function execution environment handles the actual function and passes the results back to the LLM.
Examples of Utilization
- For questions like “What’s the weather like in Seoul?”, an LLM can use get_current_weather(location:string, unit: 'celsius'| 'fahrenheit'), a function that calls the API to get the current weather information.
- The LLM can also use the send_email(to:string, body:string) function to send an email on behalf of the user for questions such as “Send Minji an email about going to the movies next Saturday”
Standardizing Model Interactions: The Model Context Protocol (MCP)
The Model Context Protocol (MCP)[19] is a new standard for connecting AI models to specific systems (content repositories, business tools, development environments). MCP aims to enable AI models to connect to various data sources and services like “USB ports,” allowing them to interact in a standardized way without having to write custom code each time.
Standardized protocols are needed so that when AI assistants connect with different systems to generate responses, each system can be implemented differently or may use legacy systems, so that the interface for connecting to each system is unified into a single protocol so that the AI system can access the system in a stable manner.
The figure below illustrates the client-server architecture in MCP, where a host that wants to access an external data source or service communicates with an MCP server via the MCP protocol as an MCP client. These protocols allow developers to expose data through the MCP server, and application developers to access it through the MCP client, creating a secure, scalable, two-way communication architecture.
Standardized protocols are needed so that when AI assistants connect with different systems to generate responses, each system can be implemented differently or may use legacy systems, so that the interface for connecting to each system is unified into a single protocol so that the AI system can access the system in a stable manner.
The figure below illustrates the client-server architecture in MCP, where a host that wants to access an external data source or service communicates with an MCP server via the MCP protocol as an MCP client. These protocols allow developers to expose data through the MCP server, and application developers to access it through the MCP client, creating a secure, scalable, two-way communication architecture.
Image 13. Data flow in MCP. [24]Directly manipulating a computer with Computer-Using Agent (CUA)
Computer-Using Agent (CUA)[20, 21] is the ability of a model to interact directly with a computer's graphical user interface (GUI) to perform tasks. This computer-use is based on years of research that integrates a model's reasoning ability, multimodal comprehension, and tool use. The model understands the visual information on the screen much like a human perceives a computer screen, and manipulates various applications through actions such as mouse clicks, keyboard input, and scrolling to complete a given task.
Similar to the previous agent operation flow, CUA follows a repeated loop of perception, reasoning, and action. When a user requests a task, the agent perceives a screenshot of the computer, goes through a reasoning process to determine the next action, such as moving the mouse, typing on the keyboard, logging in, or solving a CAPTCHA, and then goes back to the perception stage and repeats the loop until the task is completed.
Similar to the previous agent operation flow, CUA follows a repeated loop of perception, reasoning, and action. When a user requests a task, the agent perceives a screenshot of the computer, goes through a reasoning process to determine the next action, such as moving the mouse, typing on the keyboard, logging in, or solving a CAPTCHA, and then goes back to the perception stage and repeats the loop until the task is completed.
Image 14. Operation of computer-using agents. [20]5. Conclusion
From the foundation models that are the basis of the LLMs, to advances in reasoning models that make models smarter, to agents that act on their own, we've covered the highlights. We are at another turning point in this rapidly evolving field, where models' ability to reason, multimodal understanding, and tool use are coming together in a rapidly changing landscape.
The Era of Experience[22] argues that learning based on existing large-scale human data is gradually reaching its limits, and that for AI to achieve superhuman capabilities beyond human intelligence, it needs to learn from the experiential data that models themselves generate as they interact with their environment. In this Era of Experience, reinforcement learning is likely to play a major role, with models utilizing their enhanced multimodal capabilities and learning from rich information provided by diverse environments.
However, the inner workings of AI models are still a black box, and there are concerns about unpredictable side effects as humans become more reliant on models and autonomous agents. Therefore, research on the safety and alignment of models will become more important in the future.
LG AI Research will continue to research models that are safe and reliable, along with advancements in LLM technology, based on the technological prowess we have built through EXAONE 3.0, EXAONE 3.5, and EXAONE Deep models.
The Era of Experience[22] argues that learning based on existing large-scale human data is gradually reaching its limits, and that for AI to achieve superhuman capabilities beyond human intelligence, it needs to learn from the experiential data that models themselves generate as they interact with their environment. In this Era of Experience, reinforcement learning is likely to play a major role, with models utilizing their enhanced multimodal capabilities and learning from rich information provided by diverse environments.
However, the inner workings of AI models are still a black box, and there are concerns about unpredictable side effects as humans become more reliant on models and autonomous agents. Therefore, research on the safety and alignment of models will become more important in the future.
LG AI Research will continue to research models that are safe and reliable, along with advancements in LLM technology, based on the technological prowess we have built through EXAONE 3.0, EXAONE 3.5, and EXAONE Deep models.
- 참고
[1] LG AI Research. "EXAONE 3.0 7.8 B Instruction Tuned Language Model." arXiv preprint arXiv:2408.03541 (2024).
[2] LG AI Research. "EXAONE 3.5: Series of Large Language Models for Real-world Use Cases." arXiv preprint arXiv:2412.04862 (2024).
[3] LG AI Research. "EXAONE Deep: Reasoning Enhanced Language Models." arXiv preprint arXiv:2503.12524 (2025).
[4] Fedus, William, Barret Zoph, and Noam Shazeer. "Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity." Journal of Machine Learning Research 23.120 (2022): 1-39.
[5] Jiang, Albert Q., et al. "Mixtral of experts." arXiv preprint arXiv:2401.04088 (2024).
[6] Gale, Trevor, et al. "Megablocks: Efficient sparse training with mixture-of-experts." Proceedings of Machine Learning and Systems 5 (2023): 288-304.
[7] Dai, Damai, et al. "Deepseekmoe: Towards ultimate expert specialization in mixture-of-experts language models." arXiv preprint arXiv:2401.06066 (2024).
[8] Liu, Aixin, et al. "Deepseek-v3 technical report." arXiv preprint arXiv:2412.19437 (2024).
[9] Kaplan, Jared, et al. "Scaling laws for neural language models." arXiv preprint arXiv:2001.08361 (2020).
[10] Hoffmann, Jordan, et al. "Training compute-optimal large language models." arXiv preprint arXiv:2203.15556 (2022).
[11] Rafailov, Rafael, et al. "Direct preference optimization: Your language model is secretly a reward model." Advances in Neural Information Processing Systems 36 (2023): 53728-53741.
[12] Meng, Yu, Mengzhou Xia, and Danqi Chen. "Simpo: Simple preference optimization with a reference-free reward." Advances in Neural Information Processing Systems 37 (2024): 124198-124235.
[13] Schulman, John, et al. "Proximal policy optimization algorithms." arXiv preprint arXiv:1707.06347 (2017).
[14] Shao, Zhihong, et al. "Deepseekmath: Pushing the limits of mathematical reasoning in open language models." arXiv preprint arXiv:2402.03300 (2024).
[15] Lambert, Nathan, et al. "Tulu 3: Pushing frontiers in open language model post-training." arXiv preprint arXiv:2411.15124 (2024).
[16] OpenAI. "Learning to Reason with LLMs." OpenAI, March 28, 2024. https://openai.com/index/learning-to-reason-with-llms/.
[17] DeepSeek AI. "Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning." arXiv preprint arXiv:2501.12948 (2025).
[18] Anthropic. “Building effective agents” Dec 19, 2024. https://www.anthropic.com/engineering/building-effective-agents.
[19] Anthropic. “Introducing the Model Context Protocol” Nov 26, 2024. https://www.anthropic.com/news/model-context-protocol.
[20] OpenAI. "Computer-Using Agent" OpenAI, January 23, 2025. https://openai.com/index/computer-using-agent/.
[21] Anthropic. “Introducing computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku” Oct 23, 2024. https://www.anthropic.com/news/3-5-models-and-computer-use
[22] Silver, David, and Richard S. Sutton. "Welcome to the Era of Experience." Google AI (2025).
[23] Xiao, Teng, et al. "SimPER: A Minimalist Approach to Preference Alignment without Hyperparameters." arXiv preprint arXiv:2502.00883 (2025).
[24] Anthropic. “MCP Introduction”, 2024. https://modelcontextprotocol.io/introduction.