Object Affordances Graph Network for Action Recognition

Haoliang Tan (Xi'an Jiaotong University), Le Wang (Xi'an Jiaotong University), Qilin Zhang (HERE Technologies), Zhanning Gao (Alibaba Group), Nanning Zheng (Xi'an Jiaotong University), Gang Hua (Wormpex AI Research)

Abstract
Human actions often involve interactions with objects, and such action possibilities of objects were termed ``affordances'' in human-computer interaction (HCI) literature. To facilitate action recognition with object affordances, we propose the Object Affordances Graph (OAG), which cast human-object interaction cues into video representations via an iterative refinement procedure. With the spatio-temporal co-occurrences between human and objects captured, the Object Affordances Graph Network (OAGN) is subsequently proposed. To provide a fair evaluation of the role that object affordances could play on human action recognition, we have assembled a new dataset with additional annotated object bounding boxes to account for human-object interactions. Multiple experiments on this proposed Object-Charades dataset verify the value of including object affordances in human action recognition, specifically via the proposed OAGN, which outperforms existing state-of-the-art affordance-less action recognition methods.

DOI
10.5244/C.33.8
https://dx.doi.org/10.5244/C.33.8

Files
Paper (PDF)

BibTeX