Ahra Jo
2025.05.07
Sehyun Chun
2025.05.07
Jiye Kim
2025.05.07
Release of MolMole, an AI That Understands Chemical Molecular Structural Formula Information from Documents
We encounter tons of documents every day. We make sense of the information in these documents, including the diagrams, images, and graphs used alongside the text, and use them to understand the deeper meaning behind the document.
“If AI could understand different forms of data in documents, how would our lives change?”
LG has been researching Deep Document Understanding (DDU) technology with the goal of creating an AI that understands every document in the world. LG's DDU technology not only understands text, graphs, and tables in general documents, but also complex molecular structural formulas and reaction formulas in chemical papers and patents.
Image 1. LG's DDU Technology [1]We're excited to unveil the latest achievement of LG's Deep Documnet Understanding (DDU) technology—a specialized model for the field of chemistry, MolMole. The MolMole model has demonstrated outstanding performance by achieving SOTA against major competing models. The technical report of the MolMole model provides a technical description of the model and performance evaluation results.
Along with the publication of this research, LG has also built our own benchmark dataset to measure the performance of DDU technology in the chemical field. Until now, there have been benchmarks to evaluate the performance of single models such as Optical Chemical Structure Recognition (OCSR), but there has been no benchmark to evaluate the performance of extracting molecular structural formulas from full PDF documents in the context of real-world chemists. We wanted to build a benchmark to measure the performance of our models to activate the AI ecosystem, and we will be releasing our own benchmark dataset later this year. We hope that our attempt will spark discussions and technical exchanges among researchers of DDU technologies.
Along with the publication of this research, LG has also built our own benchmark dataset to measure the performance of DDU technology in the chemical field. Until now, there have been benchmarks to evaluate the performance of single models such as Optical Chemical Structure Recognition (OCSR), but there has been no benchmark to evaluate the performance of extracting molecular structural formulas from full PDF documents in the context of real-world chemists. We wanted to build a benchmark to measure the performance of our models to activate the AI ecosystem, and we will be releasing our own benchmark dataset later this year. We hope that our attempt will spark discussions and technical exchanges among researchers of DDU technologies.
MolMole, DDU Specialized for the Chemical Field
Chemical patents and papers contain tons of molecular structural formula information, but most of it has not been converted into machine-understandable data. What would happen if AI could recognize and convert molecular structural structures in chemical documents into data?
First and foremost, a large database of molecular structural formulas can be built, and researchers can search for chemical reactions in the database and perform the large-scale analysis required for their research. AI technology can help chemists work more efficiently and support breakthrough discoveries in chemistry.
First and foremost, a large database of molecular structural formulas can be built, and researchers can search for chemical reactions in the database and perform the large-scale analysis required for their research. AI technology can help chemists work more efficiently and support breakthrough discoveries in chemistry.
High difficulty of molecular structural formula recognition technology
Extracting molecular structure and reaction data from complex documents is challenging, as patents and papers contain chemical information in many different forms and have unstructured features that make it impossible to define rules. The layout of the documents is also complex, making it very challenging to develop AI to recognize them accurately.
Extracting molecular structure and reaction data from complex documents is challenging, as patents and papers contain chemical information in many different forms and have unstructured features that make it impossible to define rules. The layout of the documents is also complex, making it very challenging to develop AI to recognize them accurately.
MolMole's unique features and technologies
An important feature of MolMole is that it has been implemented in a framework that reflects the needs of real-world researchers. Existing DDU technologies in the chemical field are limited to converting molecular structural formula images as input values. However, in the real world of research, patents, papers, and other materials exist in the form of PDF documents. Other AI models that require images in PDF documents to be cropped and entered as input values are obviously less useful. Therefore, MolMole was implemented so that PDF documents can be entered as input values as they are. When you input a PDF document into MolMole, it recognizes and provides chemical data such as molecular structures at once.
The MolMole model consists of three modules: ViDetect, ViReact, and ViMore, which are responsible for extracting molecular structural formula domains from documents and recognizing the composition of molecular structures and chemical reactions.
An important feature of MolMole is that it has been implemented in a framework that reflects the needs of real-world researchers. Existing DDU technologies in the chemical field are limited to converting molecular structural formula images as input values. However, in the real world of research, patents, papers, and other materials exist in the form of PDF documents. Other AI models that require images in PDF documents to be cropped and entered as input values are obviously less useful. Therefore, MolMole was implemented so that PDF documents can be entered as input values as they are. When you input a PDF document into MolMole, it recognizes and provides chemical data such as molecular structures at once.
The MolMole model consists of three modules: ViDetect, ViReact, and ViMore, which are responsible for extracting molecular structural formula domains from documents and recognizing the composition of molecular structures and chemical reactions.
Image 2. MolMole model pipeline[1]- ViDetect: Extracting molecular structure domains from documents
The ViDetect module plays a critical role in precisely detecting molecular structure regions within PDF documents by identifying them with bounding boxes. Unlike general object detection tasks, even the omission of a single atom can result in an entirely different molecule—demanding exceptionally high precision and accuracy. To meet this challenge, ViDetect leverages advanced object detection techniques to ensure every component of a molecular structure is captured without omission. - ViReact: Extracting structured response information from documents
Chemical reaction diagrams typically consist of three key components: reactants, reaction conditions, and products. The ViReact module is designed to identify the position of each component within a reaction diagram and assign the appropriate classification to each region. Given that reaction diagrams can range from simple single-line formats to more complex tree and graph structures, accurately extracting this information poses a significant technical challenge. ViReact addresses this with a robust approach to structured information extraction, enabling precise parsing of a wide variety of reaction formats. - Convert molecular structures into SMILES, InChI, and Mol formats
The ViMore module recognizes molecular structures within documents and analyzes the atoms and their bonding relationships to convert them into standard chemical representations such as SMILES, InChI, and Mol formats. Given that missing even a single atom or bond can lead to an entirely different molecular interpretation, the module is engineered to achieve a high level of precision.
Real-world constraints were also carefully considered during development. Patent documents, for instance, are often available only as scanned images, which can introduce significant noise and hinder recognition accuracy. ViMore addresses this challenge with specialized techniques designed to improve performance in noisy, scan-based document environments.
New Tech, New Benchmark
Alongside our research on MolMole, we also focused on setting a new standard for DDU. We built our own benchmark dataset to measure the performance of DDU technology in the chemical field. As described earlier, the technical need and importance were high, but the practical implementation was challenging, and there was a lack of credible benchmarks. While benchmarks in the OCSR field existed, they were limited in that they did not measure recognition and conversion performance on a PDF document-level basis.
We at LG have built a new customized dataset that takes into account usability in real-world research settings, and evaluated its performance against major models that provide chemical information recognition technology at the PDF document level. LG’s own benchmark dataset will be made available to researchers later this year.
We at LG have built a new customized dataset that takes into account usability in real-world research settings, and evaluated its performance against major models that provide chemical information recognition technology at the PDF document level. LG’s own benchmark dataset will be made available to researchers later this year.
Key Performance
ViMore module achieves SOTA on authoritative OCSR benchmarks
Four benchmarks are typically used to evaluate OCSR performance: CLEF, JPO, UOB, and USPTO. JPO consists of images extracted from Japanese patent documents, and is a challenging dataset that is primarily low-resolution and irregularly structured. USPTO consists of images extracted from US patent documents, which often have complex structures in the images. CLEF and UOB are characterized by having relatively clear images.
ViMore, LG's molecular structural formula conversion model, achieved SOTA in three benchmarks, outperforming the existing SOTA deep learning models, DECIMER Image Transformer, MolScribe, and MolGrapher. This proves that LG's model has higher molecular structural formula conversion accuracy than existing models. In particular, LG's model outperformed other models in JPO, a dataset with a high level of difficulty. It has higher or similar performance to existing SOTA models in other datasets, demonstrating its universal applicability and excellent accuracy performance.
ViMore, LG's molecular structural formula conversion model, achieved SOTA in three benchmarks, outperforming the existing SOTA deep learning models, DECIMER Image Transformer, MolScribe, and MolGrapher. This proves that LG's model has higher molecular structural formula conversion accuracy than existing models. In particular, LG's model outperformed other models in JPO, a dataset with a high level of difficulty. It has higher or similar performance to existing SOTA models in other datasets, demonstrating its universal applicability and excellent accuracy performance.
Image 3. OCSR performance on public benchmarks.InChI and SMILES refer to exact match accuracy based on InChI keys and SMILES strings, respectively[1].
LG's model outperforms key competitors in its own benchmarks
ViDetect & ViMore
We also evaluated the integrated performance of ViDetect, the molecular structural formula domain detection module, and ViMore, the molecular structural formula conversion module, using our own benchmark dataset. As this is an integrated performance evaluation, it is important that both the detection and conversion of molecular structural formula domains perform well. If one module performs poorly, the overall performance will be lower.
The benchmark dataset built by LG separates patents and papers, and evaluates 300 pages of patents and 250 pages of papers, respectively. This allows us to reflect the different characteristics of patents and papers. Patents often contain low-resolution scanned images, with figures in different formats. Molecular structure formulas are often densely packed on a single page, or the molecular structural formulas are in tables, etc. This makes molecular structure recognition difficult. Papers, on the other hand, tend to be higher resolution and clearer, but they often contain color in their molecular structural formulas and backgrounds, which can make recognition difficult. Another strength of the benchmark dataset built by LG is that by categorizing patents and papers in this way, we were able to secure test sets from as many different domains as possible.
In conclusion, LG's ViDetect & ViMore outperformed the existing competing models, Decimer Segmentation & Image Transformer and MolDetect & MolScribe, in Precision Recall. This means the most reliable end-to-end performance in real-world model usage environments.
The benchmark dataset built by LG separates patents and papers, and evaluates 300 pages of patents and 250 pages of papers, respectively. This allows us to reflect the different characteristics of patents and papers. Patents often contain low-resolution scanned images, with figures in different formats. Molecular structure formulas are often densely packed on a single page, or the molecular structural formulas are in tables, etc. This makes molecular structure recognition difficult. Papers, on the other hand, tend to be higher resolution and clearer, but they often contain color in their molecular structural formulas and backgrounds, which can make recognition difficult. Another strength of the benchmark dataset built by LG is that by categorizing patents and papers in this way, we were able to secure test sets from as many different domains as possible.
In conclusion, LG's ViDetect & ViMore outperformed the existing competing models, Decimer Segmentation & Image Transformer and MolDetect & MolScribe, in Precision Recall. This means the most reliable end-to-end performance in real-world model usage environments.
Image 4. Combined performance of molecule detection to conversion on Patents and Articles[1]. ViReact
We also evaluated the performance of reactive detection with LG's benchmark dataset, and found that it outperformed the existing competing models, ReactionDataExtractor2.0 and RxnScribe, in terms of precision recall.
Image 5. Reaction parsing performance on Patents and Articles[1].Changes to Come
As mentioned above, the commercialization of chemical field-specific DDU models like MolMole will eventually lead to the creation of a large database of molecular structural formulas. We expect this to help chemists research more efficiently and support innovative scientific discoveries.
In addition, there has been recent research on high-performance inference models that understand mathematical logic and infer scientific concepts. It is expected that core information in the chemical field can be understood through DDU technology and utilized for training enhanced inference models. Please look forward to the changes LG is making with AI technology!
In addition, there has been recent research on high-performance inference models that understand mathematical logic and infer scientific concepts. It is expected that core information in the chemical field can be understood through DDU technology and utilized for training enhanced inference models. Please look forward to the changes LG is making with AI technology!
▶ LG’s DDU, Unlock the Power of AI (YouTube)
- 참고
- [1] Sehyun Chun, et al. "MolMole: Molecule Mining from Scientific Literature." arXiv:2505.03777 (2025).