Abstract: Human activity recognition is a challenging high-level vision task, for which multiple factors, such as subject, object, and their diverse interactions, have to be considered and modeled.