[AAAI-25] Enhanced Anomaly/Out-of-Distribution Detection with Foundation Models

The Association for the Advancement of Artificial Intelligence (AAAI 2025), held from February 25 to March 4, 2025, in Philadelphia, Pennsylvania, USA, was a venue for researchers and industry experts from around the world to share the latest trends and innovative research in AI technology. This year's AAAI focused on research presentations on the technological advancement of AI and its applications in real-world industries, especially those that effectively utilize foundation models to solve existing industry challenges.

One of the key challenges common to many industries is the ability to accurately detect anomalies or out-of-distribution (OOD) data that has not been encountered during the learning process. LG AI Research's Data Intelligence (DI) Lab presented “Diffusion-based Semantic Outlier Generation via Nuisance Awareness for Out-of-Distribution Detection” at the AAAI main track poster session, which effectively approached this problem.

This blog post will analyze the key findings of the research published by LG AI Research along with the major development trends in the field of anomaly detection/OOD detection.

LG AI Research, “Diffusion-based Semantic Outlier Generation via Nuisance Awareness for Out-of-Distribution Detection (SONA)^[1]”

Out-of-distribution (OOD) detection is the problem of detecting whether a given sample belongs to the in-distribution (ID) used for learning or not. One of the existing OOD detection methods, the outlier exposure technique, utilizes externally obtained OOD samples to help the model learn to distinguish between IDs and OODs. However, this method is limited by the cost of collecting external OOD samples and the difficulty in accurately distinguishing between samples that are similar to IDs, such as Near-OODs. Recent work has attempted to use diffusion models to generate OOD samples in realistic pixel-space, but these approaches suffer from unstable performance as the quality of the generated samples is highly dependent on the target. Therefore, a new approach is needed that can accurately detect near-OOD samples by taking into account the detailed semantic features of identity samples.

SONA (Semantic Outlier generation via Nuisance Awareness) is a new outlier generation framework for accurate near-OOD to far-OOD detection, the core of which is to use ID images directly in a diffusion model. As shown in the image below, SONA consists of two main parts: synthesizing SONA outliers and training an OOD classifier with SONA outliers.

Image 1. SONA Framework Overview^[1]

First, the outlier synthesis process feeds noise into the ID image up to a random timestep $\mathrm{T̃}$ before feeding the ID image into the diffusion model, so that the noise corruption effect is more concentrated on the semantic part of the ID. Then, to give different effects to the semantic and nuisance parts of the original ID image, the difference between the conditional and unconditional noise estimation values is utilized to mask them. If the difference is large, the part that contains many features of the condition is masked as semantic, which has few redundant features with the unconditional term. On the other hand, if the difference is small, the part that shares many redundant features with the unconditional term is masked as a nuisance.

Based on this masked information, SONA guidance applies the following three effects:

1. It removes the original semantic information while preserving the nuisance information as much as possible.

2. It selects a random OOD semantic.

3. The selected OOD semantic is then mixed with the existing ID semantic.

 

Our proposed SONA guidance more effectively addresses the quality sensitivity of outlier generation due to differences in $\mathrm{T̃}$ values, which has been mentioned as a limitation in previous studies.

Furthermore, this paper proposes a new loss term to help OOD classifiers learn SONA outliers more effectively. The first term of loss performs multi-class classification with ID class labels, and the second term trains the classifier to distinguish between ID and OOD samples. Finally, the third term minimizes the mutual information of the semantic information of ID and OOD samples to make the classifier learn to focus more on the semantic differences.

$\begin{array}{c}\mathcal{L} = {\mathbb{E}}_{(\mathbf{x},y)~D_{\mathrm{ID}}}\left[{\mathcal{L}}_{\mathrm{CE}}\left(g\right(f\left(\mathbf{x}\right)),y)\right]\\ +\beta {\mathbb{E}}_{{\mathbf{x}}_{\mathrm{OOD}}~D_{\text{OOD }}}\left[{\mathcal{L}}_{\text{OE }}\left(g\left(f\left({\mathbf{x}}_{\text{OOD }}\right)\right)\right)\right]\\ +{\mathcal{L}}_{\mathrm{MI}}\left(f\left(\mathbf{x}\right), f\left({\mathbf{x}}_{\text{OOD }}\right)\right)\end{array}$

 

An example of the generated SONA images can be seen below. You can see that the SONA outliers show much more sophisticated semantic differences compared to the existing utilized outliers.

Image 2. Visual comparison results between SONA and existing outlier generation methodologies^[1]

The experiments in this paper demonstrate the SOTA performance of SONA. Compared to various existing OOD detection methodologies, we show the performance of SOTA on both near-OOD and far-OOD. In particular, while existing methodologies show severe performance degradation on near-OOD datasets, SONA still maintains high performance on near-OOD datasets, demonstrating the effectiveness of the framework.

Image 3. Performance comparison between SONA and existing methods on near-OOD and far-OOD datasets in ImageNet200^[1] (Evaluation metric: AUROC)

Notable Detection-Related Research at AAAI-25

1) LogicAD: Explainable Anomaly Detection Via VLM-Based Text Feature Extraction^[2]

This paper argues that the problem of image anomaly detection (AD) in industrial settings can be divided into structural anomalies (SA) and logical anomalies (LA). Structural anomalies are those that manifest as localized regional features, such as defects, scratches, and damage, while logical anomalies are complex forms of anomalies that require more global and abstract judgments, such as misplaced, missing, or inconsistent counts of objects. However, existing AD methodologies are mostly visual feature-based, relying on pre-trained vision encoders to learn patterns in normal data to determine anomalies.

However, these methods mostly focus on local visual features and are limited in making logical anomaly judgments and results. In particular, LA requires a higher level of semantic understanding as it requires understanding and reasoning about the regular configuration of the normal state. As a result, the potential of anomaly detection based on Vision-Language Models (VLMs), especially autoregressive multimodal Vision Language Models (AVLMs), has recently gained attention. To overcome these limitations, LogicAD proposes an explainable logical anomaly detection framework based on text feature extraction and logic reasoning. The core idea is to leverage the image-to-text mapping capabilities of AVLMs, generate logical text descriptions from images, and use an automated theorem prover (ATP) to determine whether a given input image violates normal rules.

Here is a complete overview of the paper. LogicAD consists of three core components :

1. Text feature extraction, which generates logically interpretable text information from images

2. Format embedding, which embeds the generated text information into a formalized structure and calculates similarity to determine anomalies

3. Logic reasoner, which converts text information into logical specifications and performs formal logic-based reasoning to determine anomalies more precisely and in an explainable way

 

Image 4. LogicAD Framework Overview^[2]

LogicAD's preprocessing starts by extracting explanatory text from the visual features present in the input image. To do this, it uses Grounding DINO to automatically extract regions of interest (ROIs) in the image and feeds each ROI and the entire image into an AVLM to generate text describing those regions. However, simple text generation doesn't provide enough detail to make logical anomaly judgments, so LogicAD introduces a guided chain-of-thought (CoT) prompting method. For example, it constructs question-like prompts that directly elicit the information needed to make specific and logical judgments, such as “How many slots are on the left?,” “What color cable is connected to each slot?,” or “Do the relative positions (top/middle/bottom) of two slots match?,” to guide the AVLM to go beyond simple descriptions and generate structure- and relation-driven explanations. The generated explanations are generated multiple times to reduce the variability of the AVLM, vectorized using a text-embedding-3-large embedding model, and then filtered using a local outlier factor (LOF) to remove low-quality or semantically inconsistent explanations. This process results in logic-based text features that are semantically consistent and accurate.

These text features are utilized in two ways. One is an embedding-based comparison method, and the other is a formal logic-based inference method. In the former case, the texts generated by AVLM are normalized to JSON-based format through LLM, embeddings are created for each of the normal and query images, and an anomaly score is obtained by calculating cosine similarity. This embedding similarity is used for score-based anomaly judgment, and the lower the similarity, the more likely the anomaly is. While this method has the advantage of fast evaluation, it has limitations in explaining underlying anomaly causes or precisely detecting complex logic errors. LogicAD uses a parallel logical reasoning method to perform more in-depth anomaly judgment.

In formal logic-based reasoning, an image description in natural language is converted into a formal logical representation, which is then used to determine if the image violates the normal rules of logic using a theorem prover (Prover9). For example, if the description “There is an apple and a necklace on the left” contradicts the normal rule “There should only be one or the other on the left,” LogicAD can logically prove this and determine that the image is an anomaly. This process does not just determine whether an anomaly exists, it also identifies what information caused the conflict, enabling explainable anomaly detection. To increase the accuracy of this logical reasoning, LogicAD is designed to reliably handle missing information and ambiguous situations by utilizing a variety of logic rules, including rules for distinguishing between objects, uniqueness conditions, limiting the number of objects that can appear, and assigning default values.

LogicAD has demonstrated its effectiveness by performing well on a variety of anomaly detection benchmarks. In particular, the paper highlights experiments based on the MVTec LOCO AD dataset, demonstrating that LogicAD significantly outperforms traditional visual feature-based methodologies.

Image 5. Performance comparison between LogicAD and existing methodologies in MVTec LOCO AD^[2] (Evaluation metric: AUROC, F1-Max)

2) Uniformaly: Towards Task-agnostic Unified Framework for Visual Anomaly Detection^[3]

UniFormaly was motivated by the need for a task-agnostic and unified anomaly detection framework. In real-world environments, anomaly detection is required in various forms, such as defect detection, semantic anomaly detection, multi-class anomaly detection, and anomaly clustering, but existing studies have designed separate model structures and learning strategies for each task, which are not easily transferable to other tasks. In particular, as anomaly detection mostly operates in an unsupervised learning environment that utilizes only normal data, existing methods that require model redesign or fine-tuning for each task are inefficient in terms of practicality and scalability. Therefore, this research focuses on a unified representation of self-supervised vision transformers and proposes UniFormaly, aiming for a single framework that can be applied to any task without further modification. To this end, three key elements are incorporated into the design:

1. A robust representation that can be applied to a variety of tasks is utilized

2. Background noise is removed and foreground-centric information is incorporated into learning

3. A flexible scoring scheme that can capture different distributions of anomalies is introduced

 

An overview of the paper is shown below, and the overall framework consists of three main steps. First, normal images from the training set are taken as input, patch-level representations are extracted, and a foreground-centered memory bank is built based on this. Then, for inference, patch-level features are extracted for the test image in the same way, and anomaly scores are calculated based on the similarity with normal patches in the memory. This structure is similar to the existing memory-based anomaly detection structure, but UniFormaly introduces two new methods to it, which greatly improve its robustness and generalizability.

Image 6. Uniformaly Framework Overview^[3]

Back Patch Masking (BPM) was introduced to solve the problem of meaningless background patches affecting the memory bank or anomaly score. While existing methods include all patches in memory, resulting in unnecessary noise, UniFormaly selects only those patches that correspond to the foreground based on the self-attention map generated by ViT's [CLS] token. After smoothing out areas with high attention scores, a threshold is applied to create a binary mask, which allows us to store only foreground patches in memory and utilize them for scoring. This reduces memory usage and allows us to focus on meaningful anomaly detection.

The second technique, Top k-ratio Feature Matching, compensates for the limitations of the existing max aggregation method. The max method relied on a single patch and was sensitive to noise or failed to capture semantic anomalies. Accordingly, UniFormaly averages the scores of the top k% of foreground patches with the highest anomaly score and uses them as the final anomaly score. This method reduces the impact of noise, increases robustness by being able to cover a wide range of anomaly patterns, and can also be effectively utilized for multi-class anomalies or anomaly clustering.

UniFormaly has demonstrated stable and consistent performance compared to existing methodologies across a variety of anomaly detection tasks. In defect detection in particular, UniFormaly achieved higher accuracy and lower variance than existing SOTA techniques on representative industrial datasets, demonstrating that it is capable of reliable anomaly detection in real-world manufacturing environments. In semantic anomaly detection, it also performed well on a variety of datasets.

Image 7. Comparison of defect detection performance in MVTecAD^[3] (Evaluation metric: AUROC)

Image 8. Comparison of semantic anomaly detection performance on various datasets^[3] (Evaluation metric: AUROC)

Future Plans

LG AI Research's Data Intelligence Lab has been conducting research across various modalities, centering on anomaly and OOD detection. We have developed robust anomaly detection techniques not only for unstructured data such as text and images, but also for structured data such as tabular, time-series, and graphs, which play an important role in real-world industries. In the future, we plan to further enhance our anomaly/OOD detection capabilities and develop foundation models targeted at this task. Based on this technology foundation, LG AI Research will provide reliable anomaly detection solutions in various domains and expand the value of AI in real-world applications.

AAAI-25 Series

EP.1 [AAAI-25] Study on Denoising Distillation Model Using 3D Coordinates
EP.2 [AAAI-25] A Novel Augmentation Technique in Truncated SVD-Based Representation Space for Improving Tabular Data Prediction Performance
EP.3 [AAAI-25] Study on Efficient Time Series Forecasting Model Learning Strategies That Alleviate Variable Redundancy
EP.4 [AAAI-25] ImagePiece: A Novel Content-Aware Re-Tokenization Technique for Semanticizing Tokens in Efficient ViT