CVPR 2022
Bumsoo Kim, Jonghwan Mun(Kakao Brain), Kyoung-Woon On(Kakao Brain), Minchul Shin(Kakao Brain), Junhyun Lee (Korea University), Eun-Sol Kim(Hanyang University)
Abstract
Human-Object Interaction (HOI) detection is the task of identifying a set of hhuman, object, interactioni triplets from an image. Recent work proposed transformer encoderdecoder architectures that successfully eliminated the need for many hand-designed components in HOI detection through end-to-end training. However, they are limited to single-scale feature resolution, providing suboptimal performance in scenes containing humans, objects, and their interactions with vastly different scales and distances. To tackle this problem, we propose a Multi-Scale TRansformer (MSTR) for HOI detection powered by two novel HOIaware deformable attention modules called Dual-Entity attention and Entity-conditioned Context attention. While existing deformable attention comes at a huge cost in HOI detection performance, our proposed attention modules of MSTR learn to effectively attend to sampling points that are essential to identify interactions. In experiments, we achieve the new state-of-the-art performance on two HOI detection benchmarks