LG AI Research Presents Impressive Results at CVPR 2023

The world’s largest computer vision conference, CVPR (Computer Vision and Pattern Recognition) 2023, is held in Vancouver, Canada from June 18 to 22 (local time). At CVPR 2023, LG AI Research publicly unveil its ‘Captioning AI’ for the first time and demonstrate the technology to researchers from around the world who visit the LG booth. The institute also plans to host a workshop on ‘Zero-shot Image Captioning’ and various other activities.

We will take a look at the Captioning AI that will bring changes to the image search market, and the technologies and visions presented by LG AI Research at CVPR 2023.

Captioning AI: Understanding and Describing Images Like Humans

Captioning AI is a generative AI model that takes images, such as photos or illustrations, as input and outputs sentences or keywords describing the content. Like humans, it can understand and infer images based on previously learned data and accurately describe images it has never seen before.

As Image Captioning technology becomes more advanced, it can train and classify large-scale image and text data and generate captions by connecting image features and text when new images are inputted. This technology is called Zero-shot Image Captioning, which has no limitations in object classification categories and can describe the entire image’s understanding, including context and relationships between objects, as text, similar to human visual perception.

Captioning AI can generate metadata, which can be used for image searches. Depending on the length and quantity of the text, on average, it generates five sentences and ten keywords within ten seconds. When the image range is expanded to 10,000 images, it can complete the task in less than two days, making it possible to build a customized image search and management system quickly. It is expected to help improve the efficiency and productivity of businesses that need to manage a large number of images.

Collaborating with Shutterstock to Enhance Technical and Ethical Maturity

Captioning AI was developed through a collaboration between LG AI Research and Shutterstock, a global platform company that adds hundreds of thousands of visual content, such as images and videos, daily. Shutterstock has experts with extensive experience in content analysis and processing, as well as a vast amount of know-how in image captioning, such as sentence length and expression methods suitable for image classification and search. Through close cooperation with Shutterstock, from data training to service development, the completeness of Captioning AI has been enhanced. In particular, copyright transparency of training data and AI ethics verification, such as bias and sexuality, was conducted to develop a practical and reliable AI model.

Sejal Amin, CTO of Shutterstock, emphasized that “Shutterstock’s partnership with LG continues to show progress as we explore an image-to-text model pilot for enterprise customers. Through this AI image keyword and captioning technology, we aim to help our customers further automate the non-creative parts of the creative process so they can spend more time in creative flow.”

‘NICE’ Workshop on Zero-shot Image Captioning

On the morning of June 18 (local time), the first day of CVPR, LG AI Research, Seoul National University AI Graduate School, and Shutterstock held a workshop on zero-shot image captioning titled ‘NICE: New Frontiers for Zero-shot Image Captioning Evaluation.’

The workshop, which started with a welcoming speech by Prof. Kyoung Mu Lee of Seoul National University, featured in-depth discussions on the latest research trends, future prospects, and the impact of technology on society regarding AI ethics by global experts in the field of Image Captioning. Following this, Honglak Lee, Chief Scientist of AI at LG AI Research, introduced the ongoing research on Image Captioning at LG AI Research, including previous studies on captioning technology.

An awards ceremony for the ‘LG Global AI Challenge’ held in the first half of the year was also held during the workshop. A total of 142 research teams participated in the challenge, which evaluated the image understanding capabilities of self-developed AI models. Participants from Nanjing University of Science and Technology and KAIST, who ranked first and second in the challenge, presented their research results at the workshop. The top-ranked team was able to achieve a high score by analyzing the provided dataset and conducting fine-tuning.

Seunghwan Kim, Leader of Vision Lab at LG AI Research, said, “This workshop is even more meaningful as it is linked to the first commercial service, ‘Captioning AI,’” and added, “We plan to continue pursuing the development of new evaluation metrics and research on new technologies to secure global research leadership in the field of image captioning by establishing an organic cooperation system with various partners.”

Showcasing LG Group’s Latest AI Technologies and Networking with Global AI Talents

During the conference, LG AI Research, along with major LG affiliates such as LG Electronics, LG Innotek, LG Energy Solution, and LG Uplus, will seek to secure global AI talent. To this end, a networking event called ‘LG AI Day’ will be held on the 19th for master’s and doctoral students participating in the conference, and from the 20th for three days, the latest AI demos from each subsidiary will be held at the LG booth, along with job consultations.

LG Electronics will introduce AI technologies that visually implement changes in the freshness of food in refrigerators and the status of food in ovens according to cooking processes, as well as a driver monitoring system that detects driver inattention, such as drowsiness, by recognizing the driver’s face and gaze based on vision inspection technology. LG Innotek will introduce digital twin technology that allows testing in a digital space before mass-producing actual products. LG Energy Solution will present Anomaly Detection, a vision-based inspection technology that detects defects in the hundreds of thousands of battery cells produced daily. LG Uplus will showcase AI technology that extracts metadata representing various information such as characters, actions, locations, situations, and texts in video scenes, enabling customers to easily find their desired scenes in media content.

The release of Captioning AI, hosting the NICE workshop, and other events at CVPR 2023 will be an opportunity for LG AI Research to prove its world-class AI technology and capabilities. Moving forward, LG AI Research will actively engage in exchanges through a global research network, continuously recruit outstanding AI talents, and strive to become a research institution leading the development of global AI technology.

▶ CVPR 2023 NICE Workshop Official Website (Link)