NeurIPS Conference (2022)
Rodrigo Hormazabal, Changyoung Park, Soonyoung Lee, Sehui Han, Yeonsik Jo, Jaewan Lee, Ahra Jo, Seunghwan Kim, Jaegul Choo (KAIST AI), Moontae Lee, Honglak Lee
Abstract
Chemical structural information in scientific documents is typically depicted in the form of 2D images, which cannot be directly recognized and parsed by chemical database search engines. Optical Chemical Structure Recognition (OCSR) is a task that deals with this translation from images to chemical structures, and has been thoroughly researched for decades. Traditional rule-based methods follow a framework based on the detection of molecular components in an image (i.e., atoms and bonds) and the subsequent reconstruction of the graph structure. Recently, neural architectures analog to image captioning have been explored to solve this task, yet they still show to be very data inefficient, using millions of examples just to perform on par with traditional rule-based methods. Looking to motivate and benchmark new approaches based on atomic-level entities detection and graph reconstruction, We present CEDe, a unique collection of datasets manually curated by experts. These datasets combined contain more than 700,000 atom-level entity labels with their corresponding bounding boxes, which constitute all the necessary information for complete chemical graph reconstruction, including atom supergroups, stereochemistry and charge related annotations. We present experiments that show how baseline approaches using this framework can achieve performances on par with or better than image captioning-like models, even with 100x fewer training examples. In addition, a large synthetic dataset, as well as the code used for its creation, is released in order to explore transfer-learning techniques that could help these architectures perform better under low-data regimes, where the diversity in chemical structure rendering styles can pose a challenging problem.