Free-form text—such as product reviews, customer surveys, and employee feedback—often contains multiple ideas within a single document. Topic modeling is the task of automatically discovering the latent topics discussed across large collections of text, enabling organizations to understand what people are actually talking about at scale.
As one of the foundational techniques in natural language processing (NLP), topic modeling powers a wide range of real-world applications, from identifying customer concerns in millions of reviews to organizing survey responses and summarizing large document collections for data-driven decision making. From the classical Latent Dirichlet Allocation (LDA) to recent approaches leveraging Large Language Models (LLMs), topic modeling has remained an active area of research for decades.
In this article, we introduce "From Documents to Segments: A Contextual Reformulation for Topic Assignment," a paper presented at ACL 2026 by researchers at LG AI Research.[1] Rather than assigning a topic to an entire document, the paper proposes assigning topics to segments—smaller semantic units within the document. Although conceptually simple, extensive experiments demonstrate that this reformulation fundamentally changes how topic modeling should be approached.
A Practical Challenge: The Model Learns Two Distributions Well—but Breaks Down at the Corpus Level
To understand the motivation behind this work, it is helpful to revisit what topic modeling actually learns.
At its core, topic modeling estimates two probability distributions.
The first is the topic-to-word distribution (), which captures the words most strongly associated with each topic. For example, a PRICE topic is likely to assign high probability to words such as cost, value, cheap, pricey, and worth.
The second is the document-to-topic distribution (), which represents the mixture of topics contained within each document. A single review, for instance, may simultaneously discuss FOOD, SERVICE, and PRICE in different proportions.
This formulation is one of the greatest strengths of topic modeling: from the outset, it naturally represents documents as mixtures of multiple topics rather than forcing them into a single category.
The challenge arises in the next step.
In practice, what analysts often want to answer is a corpus-level question:
What topics exist across the entire corpus, and how prevalent is each topic?
Surprisingly, answering this question typically requires an additional heuristic step. Since each document contains multiple topics, conventional topic modeling methods first assign one representative topic to every document and then aggregate those assignments across the corpus.
In other words, documents that naturally contain a mixture of topics are compressed into a single label simply to estimate corpus-level topic proportions—for example, FOOD (45%), SERVICE (35%), and PRICE (20%).
This forced assignment introduces a fundamental source of error.
Consider the sentence:
"Tasty food, slow service, a bit pricey."
The sentence clearly discusses food, service, and price simultaneously. However, if the entire document must be assigned only one representative topic, it may ultimately be labeled as FOOD.
As a result, the service- and price-related content is implicitly counted as part of the FOOD topic when corpus statistics are aggregated, distorting the estimated topic distribution.
The paper refers to this phenomenon as topic contamination.
Topic contamination also affects downstream applications. For example, when searching for documents related to FOOD, retrieved documents may contain only one or two sentences about food while the majority of the content discusses unrelated topics such as service or pricing. The assigned topic no longer faithfully represents the actual semantic content of the document.
In summary, although topic modeling is explicitly designed to represent documents as mixtures of topics, conventional corpus-level analysis ultimately collapses those mixtures into one topic per document. It is precisely this forced aggregation step that introduces systematic distortion.
Through practical applications such as customer review analysis and survey analytics, researchers at LG AI Research repeatedly encountered this limitation. Rather than attempting to improve the assignment algorithm itself, they questioned a more fundamental assumption:
What if the problem lies not in the model, but in the unit of topic assignment?
Image 1. Topic modeling learns two probability distributions—the topic-to-word distribution (φ) and the document-to-topic distribution (θ).
However, estimating topic prevalence at the corpus level typically requires assigning each document to a single representative topic.
As a result, a document such as "Tasty food, slow service, a bit pricey.", which naturally contains multiple topics, may be assigned only to FOOD,
causing topic contamination by incorrectly attributing service- and price-related content to the FOOD topic. [1]
LG AI Research's Approach: Shifting the Unit of Topic Assignment from Documents to Segments
The central idea behind this work is both simple and intuitive. If topic contamination stems from assigning a single topic to an entire document—a unit that is simply too coarse—then the solution is to redefine the unit of topic assignment itself.
Instead of assigning topics at the document level, the paper proposes assigning them to segments: short spans of text, such as sentences or clauses, that each express a coherent semantic idea. The authors refer to this framework as Segment-based Topic Allocation (SBTA).
Whereas conventional Document-based Topic Allocation (DBTA) treats an entire document as the atomic unit for topic assignment, SBTA identifies and groups only the segments that are relevant to a given topic. This finer-grained formulation substantially reduces topic contamination and produces topic clusters that are both more semantically coherent and more interpretable.
By redefining the fundamental unit of topic assignment—from documents to segments—SBTA enables corpus-level topic analysis that more faithfully reflects the actual semantic structure of the underlying text.
Image 2. Document-based Topic Allocation (DBTA, left) retrieves entire documents when identifying a topic such as PRICE,
even if only a small portion of the document is actually related to pricing.
In contrast, Segment-based Topic Allocation (SBTA, right) extracts and groups only the text segments
that are semantically relevant to the target topic, resulting in more precise topic assignment, higher semantic purity, and greater interpretability. [1]
Why Segment-based Topic Allocation Matters
Moving from document-level to segment-level topic assignment offers three key advantages.
First, it improves topic purity. Because topics are assigned only to semantically coherent segments, unrelated content is no longer aggregated under the same topic, substantially reducing topic contamination.
Second, it improves interpretability. Traditional topic models typically represent a topic as a distribution over words, requiring users to infer its semantic meaning. In contrast, segments are complete semantic units that can be read directly, making it much easier to understand why a particular topic has been assigned.
Third, it aligns more naturally with real-world applications. In practice, analysts are often interested in retrieving only the text relevant to a specific topic—for example, identifying comments related to compensation in employee surveys or extracting only pricing-related feedback from customer reviews. These tasks require locating topic-specific passages rather than classifying entire documents, making segment-level topic assignment a much better fit for downstream use cases.
Building a New Benchmark: SemEval-STM
Redefining the problem also requires a new benchmark for evaluation.
Existing topic modeling datasets provide topic labels only at the document level, making them unsuitable for evaluating segment-level topic assignment. To address this limitation, the authors introduce SemEval-STM (Segment-based Topic Modeling), a new benchmark constructed from the SemEval-2016 Aspect-Based Sentiment Analysis (ABSA) dataset.
The key insight is that review aspects—such as PRICE, SERVICE, and FOOD—naturally correspond to topics in topic modeling.
The dataset construction process consists of two stages. First, an LLM (OpenAI o3-mini) was used to identify topic-relevant text segments within each review. The extracted segments were then manually verified, refined, merged, and reannotated by the authors to produce high-quality segment-level topic labels.
The resulting benchmark transforms the original document-level annotations into a segment-level representation, where each segment is associated with a single topic while preserving the multi-topic nature of the original documents.
Experimental Results: Cleaner Topic Structures and Better Topic Modeling
The authors conducted a series of experiments on SemEval-STM to validate both the quality of the proposed benchmark and the effectiveness of Segment-based Topic Allocation (SBTA).
First, they evaluated clustering quality using six widely adopted metrics, including the Davies–Bouldin (DB) Index, Calinski–Harabasz (CH) Index, and Silhouette Score. Across every evaluation metric and every domain, SBTA consistently produced topic clusters that were more compact and better separated than those generated by conventional Document-based Topic Allocation (DBTA).
To further evaluate how well each method captures the underlying topic structure of a corpus, the authors performed a Shuffle Test. In this experiment, documents (for DBTA) or segments (for SBTA) assigned to each topic are randomly shuffled, and the resulting performance degradation is measured. If a model has learned meaningful topic structures, disrupting those structures should significantly reduce performance.
The results clearly support this hypothesis. After shuffling, SBTA exhibited a substantial performance drop across all datasets, indicating that the original topic assignments captured meaningful semantic structure that was destroyed by randomization.
In contrast, DBTA showed only modest degradation—and in some datasets, performance even improved after shuffling. This suggests that document-level topic assignments failed to establish strong topic structures in the first place, leaving relatively little meaningful structure to disrupt.
Taken together, these findings demonstrate that SBTA learns the latent topic organization of a corpus far more effectively than conventional document-level approaches.
Finally, the authors benchmarked a broad range of topic modeling methods on SemEval-STM, including traditional approaches such as LDA and BERTopic, as well as LLM-based methods built on GPT, Claude, Gemini, Llama, and DeepSeek.
Across both label-free and label-based evaluation metrics, LLM-based topic modeling combined with SBTA consistently outperformed traditional methods. Interestingly, no single LLM dominated across all domains. Smaller models achieved the strongest performance in the Laptop domain, while GPT-family models performed best on Restaurant reviews, suggesting that optimal model selection remains domain dependent.
Image 3. Performance comparison on the SemEval-STM benchmark (Laptop domain; scores normalized across evaluation metrics).
LLM-based topic modeling with Segment-based Topic Allocation (SBTA)—including LLaMA and DeepSeek—
consistently outperforms traditional topic modeling methods such as LDA and BERTopic across all evaluation metrics,
including F1, Purity, Adjusted Rand Index (ARI), Normalized Mutual Information (NMI), and the Calinski–Harabasz (CH) Index. [1]
Segment Intrusion: A Human-Centered Evaluation for Topic Coherence
To evaluate whether the discovered topics are not only quantitatively strong but also semantically coherent to human readers, the authors introduce Segment Intrusion, a new evaluation task that extends the classic Word Intrusion benchmark from individual words to text segments.
The idea is straightforward. Given a set of segments assigned to the same topic, one or more intruder segments that belong to a different topic are inserted into the group. Human annotators—or LLMs—are then asked to identify the intruders. The easier it is to detect these out-of-place segments, the more semantically coherent the topic cluster is considered to be.
To systematically evaluate the task, the authors designed four experimental settings by varying two factors:
-
Number of intruders: one or two intruder segments.
-
Difficulty level: easy or hard. In the hard setting, the intruder is sampled from a different topic within the same domain, making it substantially more difficult to distinguish from the target topic than an intruder drawn from another domain.
Experimental results show that both human evaluators and LLMs experience noticeable performance degradation under the more challenging settings. Moreover, most current LLMs still fall short of human performance, suggesting that Segment Intrusion provides a realistic and appropriately challenging benchmark for evaluating topic coherence.
Importantly, this evaluation is only possible because the unit of analysis has shifted from words to segments. Unlike the traditional Word Intrusion task, which requires evaluators to infer a topic from a list of isolated keywords, Segment Intrusion presents complete semantic units. As a result, evaluators can identify intruders more naturally and intuitively, making the evaluation process both more reliable and easier to interpret.
Image 4. Segment Intrusion extends the classic Word Intrusion task from individual words to semantically coherent text segments.
In the traditional Word Intrusion setting (①), evaluators identify an out-of-place word (e.g., "pizza") that does not belong to the topic.
In Segment Intrusion (②), evaluators instead identify an intruder segment within a group of segments assigned to the same topic.
For example, among segments labeled PRICE, the sentence "The battery lasts all day." — which discusses DESIGN rather than pricing—is correctly identified as the intruder. By operating on complete semantic units rather than isolated words, Segment Intrusion provides a more intuitive and reliable way to evaluate topic coherence. [1]
Image 5. F1 scores on the Segment Intrusion benchmark.
As the task becomes more challenging—from SI-E (Segment Intrusion–Easy) to DI-H (Domain Intrusion–Hard)—performance consistently declines for both human evaluators and all evaluated LLMs.
While current frontier models demonstrate strong capabilities, most still fall short of human-level performance, particularly under the most difficult evaluation settings. These results highlight the increasing difficulty of distinguishing semantically similar topics and demonstrate that Segment Intrusion provides a challenging and discriminative benchmark for evaluating topic coherence. [1]
What’s Next
This work revisits one of the most fundamental assumptions in topic modeling: what should be the unit of topic assignment? By reformulating the problem around segments rather than entire documents, the authors demonstrate that topic modeling can better capture the semantic structure of real-world text while improving both interpretability and clustering quality.
At LG AI Research, we believe this formulation extends far beyond product reviews and benchmark datasets. Many real-world text collections—including employee feedback, customer surveys, support tickets, and large document repositories—naturally contain multiple intertwined topics within a single document. Segment-based Topic Allocation provides a more faithful representation of these complex texts and opens the door to more precise downstream analysis.
Looking ahead, we plan to further develop SBTA into a complete LLM-powered topic modeling pipeline. While this work focuses on assigning topics to text segments, future research will integrate this capability with automated topic discovery, enabling an end-to-end system that can both generate meaningful topic candidates and accurately assign them to the relevant segments.
Finally, the newly introduced SemEval-STM benchmark and the Segment Intrusion evaluation protocol provide a standardized foundation for future research on segment-level topic modeling. We hope these resources will support both academic and industrial researchers in developing the next generation of more interpretable and practically useful topic modeling systems.
Hoonsang Yoon
2026.08.14