Research Group
Associate Professor, Artificial Intelligence Research Institute
Shenzhen MSU-BIT University
Our group studies embodied affective intelligence — enabling AI systems to understand what people do, infer how they feel and why, and actively perceive and interact with humans in dynamic environments. Our research builds from human action understanding to affective understanding and ultimately to embodied affective intelligence.
I received my Bachelor’s Degree in Automation in 2015 and my Ph.D. in Software Engineering in 2021, both from the South China University of Technology, under the supervision of Prof. Mingkui Tan.
Featured Research
A Controllable Interactive 3D Simulation Benchmark for Embodied Affective Perception
AffectSim moves affective computing beyond fixed recordings by placing emotion-expressive human behavior in controllable, replayable 3D environments. It enables us to study not only what an agent perceives, but also where it observes from and how active sensing changes affective understanding.
Research Vision
Our earlier and continuing work asks how machines can reliably understand what people do over time. We study temporal action localization, video grounding, long-video understanding, and robust and efficient video models — providing the perceptual foundation for reasoning about human affect.
Featured Work
Models action proposals as a graph and reasons over their contextual and semantic relations to improve temporal action localization.
Introduced graph-based proposal–proposal relation modeling for temporal action localization.
Extends relation-aware action localization into a general graph module that captures temporal and semantic dependencies among action units.
A plug-and-play graph module that works across both two-stage and one-stage TAL detectors.
Systematically studies how temporal corruptions break modern action detectors and develops a strong baseline for more robust temporal action detection.
First benchmark dedicated to temporal robustness in temporal action detection.
Selected work across temporal localization, grounding, long video, robustness, and efficiency.
Behavior is observable; affect is latent. We study how machines can move beyond recognizing visible actions to infer affective states from body behavior, temporal context, multimodal evidence, and the events behind observable expressions — including cases where appearance alone can be misleading.
Featured Work
An agentic framework that seeks, verifies, and explains affective moments in long videos under vague natural-language queries.
Introduces VQAU, a new long-video affective understanding task driven by vague user queries.
A mechanistic study of where affective adaptation is encoded inside multimodal foundation models.
Identifies the FFN gate projection as a central architectural locus of affective adaptation across models and tasks.
An agentic multimodal framework that integrates textual, acoustic, and visual evidence and explicitly deliberates across modalities for depression risk assessment.
Transforms multimodal depression assessment from implicit feature fusion into explicit evidence deliberation.
Additional work on affective behavior, multimodal affect, and psychological-state understanding.
Real-world affect is not always visible from a fixed viewpoint. We therefore study agents that can actively observe, move, seek evidence, and interact to reduce uncertainty about human affect. This direction connects affective computing with embodied perception, simulation, robot learning, and interactive intelligence.
Featured & Supporting Work
A simulation benchmark in which affective behavior can be re-observed under controllable viewpoints and agent-controlled sensing.
Introduces a controllable interactive 3D benchmark for studying embodied affective perception through active observation.
Learning robot behavior by turning visual demonstrations into reward functions, connecting video understanding with embodied action.
Generates executable reward functions from demonstration videos, bridging visual understanding and robot skill learning.
Co-evolving reward functions and policies for language-instructed embodied skill acquisition.
Jointly co-evolves reward functions and policies from language instructions to reduce manual reward engineering.
Additional work in embodied perception, navigation, and robot learning.
Selected Applications
Beyond our main research trajectory, we are interested in applying multimodal perception and learning to selected healthcare and human-centered problems.
Adapting foundation segmentation models to noisy, temporally evolving cardiac ultrasound video.
Paper
Combining flexible sensing with intelligent recognition for human motion and interaction.
PaperPublications
Publications are organized around the research trajectory of human action → affect → embodiment, with additional work in multimodal AI and human-centered applications.
This page highlights publications most relevant to the research story above. For the complete and continuously updated record, please see Google Scholar or DBLP.
Updates
