A Day with SNU AI Talents at the 2022 AIIS Autumn Retreat

The 2022 AIIS Autumn Retreat was held on November 4 at the Seoul National University (SNU) Faculty Club. The Artificial Intelligence Institute Seoul National University (AIIS) Retreat that is held every spring and fall, is an event where the outstanding staff of SNU and companies leading the AI industry gather at one place to share the latest research trends and interact with each other. This year's AIIS Autumn Retreat was joined by LG AI Research, LG Innotek, and LG Energy Solution to communicate with the AI talents. Other companies such as NAVER, Kakao Enterprise, CJ, Megazone Cloud, and GC Biopharma also participated in the event.

The event kicked off with introductions of member companies conducting AI research and education through agreements with AIIS. It was followed by various programs such as sharing the latest AI research outcomes conducted by AIIS and companies, oral sessions, and poster sessions.

In the opening address, AIIS Director Byoungtak Zhang explained the meaning of the event to participants by saying, “It is very meaningful to continue partnerships with outstanding companies. We can achieve growth when we create synergy by combining the strengths and competitiveness of each person in the rapidly evolving society of today. I hope that this AIIS Retreat will serve as an opportunity for synergy to all of you.”

Introductions of member companies of AIIS followed the opening remarks. LG AI Research, CJ, Kakao Enterprise, Megazone Cloud, and NAVER took part in introducing the member companies.

Woohyung Lim, Applied AI Research Lab Leader from LG AI Research, made a presentation introducing the AI projects that are solving various problems within the LG Group, EXAONE, the super-giant AI of LG, and the AI research environment and education programs.

Lim explained about the researches being conducted by LG AI Research by saying, “LG AI Research is focusing on using AI to solve the issues of affiliates. AI is being applied in a wide range of areas including predicting EV battery life, developing new materials, vision inspection of manufacturing lines, circuit design, and document summary for R&D throughout the LG Group. We are also conducting fundamental research such as reinforcement learning, representation learning, transfer learning, and explainable AI.”

He also added, “We created AI Business Strategy organization to find difficult problems that can be solved with AI and come up with creative methods to solve this. Research directions are being considered with partner companies through the Expert AI Alliance that are joined by not only LG Group affiliates, but other companies as well. We are also holding the AI Hackathon with LG AI Graduate School.”

In the oral session that followed member company introductions, Byoungjip Kim, a researcher at the LG AI Research Fundamental Research Lab, introduced recent research papers.

[NeurIPS 2022] Transferring Pre-trained Multimodal Representations with Cross-modal Similarity Matching by Byoungjip Kim from the Fundamental Research Lab (Link)

In this paper, the efficient BeamCLIP method that transfers the pre-trained large-scale multi-modal model to a small target model was proposed. Kim, who is the author of this paper, explained, “Zero-shot transfer has good performance, but in general, it requires a huge amount of data and computing resources for pre-learning a large-scale multi-modal model. In order to make improvements to this, cross-modal similarity matching (CSM) was implemented for unsupervised transfer, thereby having a student model learn the representation of the teacher model. When using this methodology, it is possible to create representation in small models equivalent to that of big models just by following the teacher model well.”

He also added, “In the case of CLIP ResNet 50, zero-shot performance was found to be about 59%. In order to gain such results, you need about 400 million pages of data, but when using the teacher model, similar performance can be obtained with just about 3% of the data. Instead of having to learn the data as big as 400 million pages, all you have to do is bring the representation that was trained by the large model.”

 

Grass field next to the SNU Faculty Club where the poster session was held

After the oral presentations indoors, the poster session was held at the grass field outside the SNU Faculty Club where LG AI Research introduced papers accepted in this year’s ICLR and NeurIPS.

[ICLR 2022] Towards Continual Knowledge Learning of Language Models by Joel Jang from the Fundamental Research Lab (Link)

The first paper introduced by LG AI Research in the poster session was on research where the large-scale language model learns new knowledge without forgetting existing knowledge. Joel Jang from the Fundamental Research Lab stated, “Large-scale models forget prior knowledge when learning new knowledge. This problem was solved using the new continual learning (CL) technique of continual knowledge learning (CKL). In fact, when applying the CKL, it was found that the large-scale language model maintained performance even after fine tuning. New benchmark and metrics were developed to measure performance maintenance capacities for the three sub-conditions: preserved knowledge, updated knowledge, and newly acquired knowledge.”

 

[NeurIPS 2022] Transformers meet Stochastic Block Models: Attention with Data-Adaptive Sparsity and Cost by Sungjun Cho from the Fundamental Research Lab (Link)

 

Sungjun Cho from the Fundamental Research Lab introducing a paper in the poster session

 

The second paper at the poster session presents new technologies that improve transformers. Sungjun Cho from the Fundamental Research Lab said, “When the sequence length gets longer, it starts using more memory, thereby making it difficult to model long phrases or whole documents during language modeling. Although many solutions were proposed to solve this problem, there lies a common problem of lacking data adaptability that adjusts attention according to data, which became the reason for carrying out this research.”

Cho added, “Using new attention mechanisms presented in the paper, the amount of attention required for data can be freely adjusted, and the amount of required attention can be made proportional to time and memory cost."

 

Speaking with SNU students visiting the LG booth

 

In addition, LG operated a total of two booths at the SNU Faculty Club where the AIIS Retreat was held to interact with participating researchers. Hiring staff of LG AI Research, LG Innotek, and LG Energy Solution were present at the LG booth to introduce the company to SNU students and talk about employment opportunities. 


The 2022 AIIS Retreat held on a fresh autumn day was an opportunity for researchers at LG AI Research, hiring personnel of LG Group affiliates, and excellent students researching AI at SNU to closely interact with one another. At this event, LG AI Research explained about the company through introductions of member companies, introduced papers accepted by the world’s top-tier AI conferences including NeurIPS in the oral and poster sessions to prove its research capacities, and operated a job booth to communicate with AI talents. LG AI Research plans to continue to meet with AI talents that will become leaders of Korea by continuously participating in industry-academic exchange events such as the AIIS Retreat.