[ICML 2021] Part 2: Generative Models for OOD Detection in the ICML 2021

The International Conference on Machine Learning 2021 (ICML 2021) is a large-scale international conference that attracts around 70,000 participants each year. This year marks the 38th iteration. It is also one of the most influential conferences in the field of artificial intelligence with thesis acceptance rate of 20% and impact factor of 6.99. Here is a summary of the meaningful research papers shared at the conference, picked by three researchers from LG AI Research who participated in the ICML 2021 held online from July 18 to 24. - Part 1: Neural Combinatorial Optimization in ICML 2021 - Kanghoon Lee from Di Lab - Part 2: Generative Models for OOD Detection in the ICML 2021 - Hyemin Jung from Di Lab - Part 3: Spatio-temporal Time-series Forecasting in ICML 2021 - Juhyun Lyu from Di Lab

The Data Intelligence Lab of LG AI Research studies representation learning and distribution estimates of the given data, which are used to solve a variety of problems related to LG affiliates, such as anomaly detection, generation, prediction, and recommendation.

The ICML 2021 hosted six sessions on representation learning and four on deep generative models, indicating that research on these fields is a trend that is in the limelight recently. Below is a summary of what was discussed in the deep generative model sessions:

- Generative models for various types of data such as video, diagram, 3D image, audio, and time series data, as well as existing image data
- Academic papers addressing how to get over the difficulties of training energy-based models
- Academic papers addressing how generative models give high correlations to out-of-distribution data
- Academic papers addressing how to improve the structure of the existing generative models and make them trained

In this content, I summarized two of the papers that performed out-of-distribution (OOD) detection with generative models.

1. A paper that identified why a generative model failed to conduct OOD detection and made up for the failure to improve the performance of OOD detection 2. A paper that improved the modeling power of an energy-based model to enhance the performance of OOD detection

Hierarchical VAEs Knows What They Don’t Know^[1]

Out-of-distribution detection using generative models and related issues

As a generative model estimates the distribution in which the given data is sampled, it can use the generative model to perform OOD detection tasks. That is, if the likelihood that the model assigns for the given input data is low, the data is determined to be OOD.

According to some recent studies, however, generative models that well estimate the distribution of the given data sometimes assign a high likelihood to OOD data.

This paper explains of the phenomena from the perspective of Hierarchical VAE, suggesting a new type of OOD detection score and making SOTA records in OOD detection based on the suggested score.

Hierarchical VAE

Let’s take a look at the model used in this paper. The Hierarchical VAE is a model in which stochastic layers are constructed in the form of conditions. The existing VAEs consist of single latent variables, a structure that restricts estimation of complex input distributions.

The Hierarchical VAE was suggested to overcome such restriction; it was made by adding a hierarchy of latent variables to the basic VAE. There are various types of Hierarchical VAE, and the most common generative and inference models are as follows:



This paper performed experiments with the above (simple-type) and a more complicated Hierarchical VAEs.

What is appropriate feature level for OOD detection?

When data is input to pass through the layers, a deep learning model learns the data from the low-level feature (information such as edge) to high-level feature (semantic information). This paper identified that the low-level feature was well generalized in several datasets, as in the experiment below. There are two Hierarchical VAEs models: Model 1 is an estimation of Fashion MNIST data and Model 2 an estimation of MNIST data.

The first row of the figure below is the visualization of calculating the correlations of the features from the layers of each model after inputting the Fashion MNIST data to the Model 1, which learned the Fashion MNIST data (in-distribution data) and to the Model 2, which learned the MNIST data (out-distribution data).

The second row of the figure below is the visualization of calculating the correlations of the features by layers after inputting the MNIST data to each of the models in the same way. In other words, the figure below displays the correlations by steps of in-distribution and out-distribution.



The results of the correlations show that the low-level features from low layers have high correlations with in-distribution data and out-distribution data while the high-level features have low correlations with them, indicating that low-level features are difficult to distinguish in-distribution from out-distribution. Therefore, other datasets can be determined to be in-distribution in terms of the low-level features and to be rather different from each other in terms of the high-level features.

Problems of the existing VAE objective loss

The previous experiments showed that the low-level feature might be difficult to tell the difference between in-distribution and out-distribution. However, if a model is trained according to the existing ELBO equation, the low-level features dominate the loss equation, thereby affecting much influence. Below is the ELBO equation suitable for Hierarchical VAE:



The second term is the KL divergence term. The summation, the first part of the second term, is the sum of the log ratio values of z from each layer. These values increase more as the dimension of z increases. Thus, in Hierarchical VAE having a bottleneck structure, the high-level latent feature z has low log ratio value, affecting less influence on loss. In other words, if a model is trained according to the existing ELBO equation, the low-level features dominate.

Why a generative model fails in OOD detection and how to overcome it

High-level features should be used for OOD detection, but in the existing ELBO equation, low-level features dominate with their considerable influence on learning. This is why Hierarchical VAE fails in OOD detection no matter how well it estimates likelihood.

In order to solve the problem, this paper makes the model learn using the loss suggested in a previous study.^[2] The suggested loss equation is as follows:



When a model is trained according to the suggested loss equation, it can calculate the loss using z above or below the specified level (k), which prevents the lowest latent variables from dominating. The above equation can be expressed as follows, which uses the data space likelihood function.


When a detection is performed by the learned model, the detection can have issues if the generative model assigns a high likelihood to OOD. To make up for it, this paper suggests using likelihood ratio as the score of the OOD detection.



The value of likelihood ratio, LLR^>k(x), measures the divergence at latent space as the data space likelihood function, logp_θ(x) term, disappears.

Experiment results

Two experiments were performed in this paper: MNIST was input to a model trained with Fashion-MNIST and SVHN was input to a model trained with CIFAR10. The model trained with the loss suggested in the paper showed better performance than the existing models in terms of log likelihood. The results are as shown in the table below.

 

In addition, the detection using the likelihood ratio score suggested in the paper showed better performance than the existing methods. The results are as shown in the table below:

(Left: object detection from Fashion MNIST to MNIST; Right: object detection from CIFAR10 to SVHN)

Consideration

I had believed that a generative model with good performance would smoothly deal with downstream problems such as OOD detection tasks. However, this paper taught me to discard the thought that a good-performance model can simply improve the performance of relative tasks, for different features should be applied to distribution estimate and to OOD detection. It is needed to recognize what information is required for a task I want to address and verify and improve on whether the information is sufficiently trained through the existing generative model structure and objective function.


Improved Contrastive Divergence Training of Energy-Based Model^[3]

Limitations in energy-based models and the existing learning methods

The likelihood of input x can be expressed as an energy function as follows:



An energy-based model (EBM) is NN approximation of this function, a model that uses the energy of the input as the output when data is input to it.

Training of the EBM is that the energy of the training data is lowered and the parameter θ of the model is updated by calculating the log gradient with respect to input x of the above objective function. However, since it is difficult to calculate the log gradient, a contrastive divergence objective is suggested in terms of the approximation of the objective as follows:

 

When a model is trained with the above contrastive divergence objective, the energy of the training data is lowered and the energy of the data generated and sampled by the model increases. This way of training is simple, with issues of instability during the training. The existing prior studies show limitations in model structures and the number of layers due to the training instability, with a shortage of modeling power and robustness of a model.

This paper suggests how to solve the training instability and identifies that the model trained with the method showed improvements in robustness, generation quality, and OOD detection.

New contrastive divergence objective function

This section explains the method suggested in this paper. Below is a negative gradient of a contrastive divergence objective:

 

Most previous studies performed training only using the gradients of the first and second terms and ignored the third term (KL divergence term) because the term was difficult to be estimated and empirically negligible. However, this paper argues that ignoring the KL divergence term can lead to inaccurate optimization, suggesting a new joint loss equation for optimizing a contrastive divergence including the KL divergence term. The gradients of this loss equation were shown to be equal to the gradients of the existing contrastive divergence objective.

 

The L_CD is the existing contrastive loss, assigning low energy to the real samples while high energy to the generated samples of the model. If a model is trained only with the term, it falls into an adversarial mode, making sampling difficult. The difficulty can be prevented when the L_KL is used. This paper suggests a method of estimating the L_KL term, which can decrease the energy of the generated samples in the model and increase the entropy.

Additional structures and the entire frame

In addition to the new loss equation, this paper suggests a sampling after data augment, in order to prevent only fixed images from being sampled due to a collapse during sampling. Moreover, in order for the energy function to focus on both low and high resolutions, this paper suggests a structure that receives images of different resolutions, outputs energy from each of the images, and adds the outputs for use. The entire frame is as shown below:



Experiment results

Experiments of image generation and OOD detection were performed as suggested in this paper. For the image generation, the Inception score increased while the FID decreased, as shown in the results below:



For the OOD detection, when a model trained with the CIFAR10 was input by other datasets, the detection (AUROC) was improved.

consideration

The energy-based model is one of the methods of data likelihood modeling, superior in generality and simplicity compared to other methods. Because of its difficulty in training, however, many studies of the model have been focusing on how to deal with the difficulty. The main contribution of this paper, in this context, is to suggest how the training instability can be addressed.

As mentioned for the first paper, the performance of a generative model is not always proportional to the performance of downstream tasks, such as OOD detection. Although solving the training issues of the EBM is significant, it can be important to verify and improve on whether the structure and objective function of an EBM are appropriate for the OOD detection.


Conclusion

The Deep Generative Model sessions of the ICML 2021 showed that there are several attempts to model a variety of data such as diagrams, 3D data, video data, cloud points, as well as image data. Improvements in the structure and objective function of generative models were on the increase in order to enhance the performance of OOD detection, beyond the existing goal of enhancing the performance of image generation tasks. In addition, many studies were presented to facilitate the training stability of the EBMs.

In line with this trend, we should identify generative models appropriate for various types of data, develop generative models and training methods to enhance the performance of OOD detection, and study how distribution is estimated in terms of an EBM.


ICML 2021 Review ② DiLab Hyemin Jung  (Link)