MIT 6.S094: Deep Learning for Human Sensing

Real-world data is crucial for training supervised learning methods. Data collection and annotation are often the most challenging and critical parts of developing successful deep learning systems, more so than the algorithms themselves.

The primary human imperfections discussed are distraction (texting, eating, secondary tasks), drunk driving, and drowsy driving. These factors significantly contribute to motor vehicle crashes and fatalities.

The human-centered approach involves collaboration between humans and AI to improve safety, alleviate distraction, and bring attention back to the road. It contrasts with the 'full autonomy' approach, which aims to remove the human driver entirely.

The dataset uses instrumented vehicles with multiple cameras focused on the driver's face and body, as well as external cameras for scene perception. Data is synchronized with GPS and car data, capturing billions of video frames.

Pedestrian detection involves identifying the full body of a human in images or videos. Early methods used sliding windows with features like HOG, while modern approaches leverage convolutional neural networks (CNNs) like R-CNN for more intelligent candidate generation and classification.

Body pose estimation involves identifying the positions of key body joints. In driving, it's important for understanding driver alignment (e.g., relative to dummy positions for safety testing), general movement, and detecting activities like hands-on-wheel or smartphone usage.

Glance classification determines if a driver is looking 'on-road' or 'off-road' (or into specific regions). Unlike geometric gaze estimation, it's framed as a machine learning classification problem, focusing on broad regions rather than precise XYZ coordinates.

Cognitive load is assessed by analyzing eye dynamics, including blink patterns and pupil diameter. While pupil diameter is standard in controlled lab settings, real-world conditions require focusing on blink dynamics and eye movement, often using 3D convolutional neural networks.

Full autonomy is likely decades away. In the interim, humans will remain integral to the driving process, requiring cars to understand and interact with drivers. This necessitates robust human-robot interaction focused on communication, trust, and mutual understanding of mental states.

The lecture mentions upcoming courses on Artificial General Intelligence (AGI), Introduction to Deep Learning, and the Global Business of AI and Robotics. Competitions like 'Deep Traffic' and 'Deep Crash' are also highlighted.