Social Scene Understanding from Social Cameras

Summary

This thesis presents a framework to develop social cognition by perceiving social signals, modeling the relationship between them, and predicting social behaviors, and leverages social cameras to reconstruct three types of social signals: gaze movement, body motion, and general scene motion.

Keywords

Social cognition, Motor cognition, Social heuristics, Social competence, Social relation

References

The primary position of the eyes, the resetting saccade, and the transverse visual head plane. Head movements around the cervical joints.R. S. Jampel, Dian X. Shi — 1992
An Invitation to 3-D VisionYi Ma, Stefano Soatto, Jana Košecká — 2004
Calibration-Free Eye Gaze Direction Detection with Gaussian ProcessesBasilio Noris, Karim Benmachiche, Aude Billard — 2008
Shifts in selective visual attention: towards the underlying neural circuitry.Christof Koch, Shimon Ullman — 1985
A polynomial case of unconstrained zero-one quadratic optimizationKim Allemand, Komei Fukuda, Thomas M. Liebling — 2001
Multiple View Geometry in Computer VisionBernhard P. Wrobel — 2001
A computer algorithm for reconstructing a scene from two projectionsH. C. Longuet–Higgins — 1981
Greedy and Local Search Heuristics for Unconstrained Binary Quadratic ProgrammingPeter Merz, Bernd Freisleben — 2002
"Here's looking at you, kid". Detecting people looking at each other in videosManuel J. Marín‐Jiménez, Andrew Zisserman, Vittorio Ferrari — 2011
Estimating 3D point-of-regard and visualizing gaze trajectories under natural head movementsKentaro Takemura, Yuji Kohashi, Tsuyoshi Suenaga — 2010
A Closed-Form Solution to Non-Rigid Shape and Motion RecoveryJing Xiao, Jinxiang Chai, Takeo Kanade — 2004
A Study of Parts-Based Object Class Detection Using Complete GraphsMartin Bergtholdt, Jörg Hendrik Kappes, Stefan Schmidt — 2010
A general method for the point of regard estimation in 3D spaceFiora Pirri, Matia Pizzoli, Alessandro Rudi — 2011
Reconstructing 3D trajectories of independently moving objects using generic constraintsKemal Egemen Özden, K. Cornelis, Luc Van Eycken — 2004
Determining the gaze of faces in imagesAndrew H. Gee, Roberto Cipolla — 1994
A solvable case of quadratic 0-1 programmingFrancisco Barahona — 1986
Temporal causality for the analysis of visual eventsK. Prabhakar, Sangmin Oh, Ping Wang — 2010
MAP Visibility Estimation for Large-Scale Dynamic 3D ReconstructionHanbyul Joo, Hyun Soo Park, Yaser Sheikh — 2014
Social Role Discovery in Human EventsVignesh Ramanathan, Bangpeng Yao, Li Fei-Fei — 2013
Study on eye gaze estimationJiangang Wang, Eric Sung — 2002

Cited by

Interest Detection in Image, Video and Multiple Videos: Model and ApplicationsYuewei Lin — 2016
Co-Interest Person Detection from Multiple Wearable Camera VideosYuewei Lin, Kareem Ezzeldeen, Youjie Zhou — 2015
Understanding Human Behaviors from Wearable CamerasH. Park — 2015

Related papers

Vision for Social Robots: Human Perception and Pose EstimationMatteo Ruggero Ronchi — 2020
Following Gaze Across ViewsAdrià Recasens, Carl Vondrick, Aditya Khosla — 2016
When Computer Vision Gazes at CognitionTao Gao, Daniel Harari, Joshua B. Tenenbaum — 2014
Inferring the Intentions and Attentions of Agents from VideosDan Xie — 2016
From simple innate biases to complex visual conceptsShimon Ullman, Daniel Harari, Nimrod Dorfman — 2012
Looking for Semantic Similarity: What a Vector-Space Model of Semantics Can Tell Us About Attention in Real-World ScenesTaylor R. Hayes, John M. Henderson — 2021
Looking Beyond a Clever Narrative: Visual Context and Attention are Primary Drivers of Affect in Video AdvertisementsAbhinav Shukla, Harish Katti, Mohan Kankanhalli — 2018
Egocentric Hand Track and Object-Based Human Action RecognitionGeorgios Kapidis, Ronald Poppe, Elsbeth van Dam — 2019