In this work, we introduce a computer vision system that performs automatic facial and body behavior analysis and recognition using RGB-D camera. From RGB video, the system performs facial landmark tracking, which automatically determines the locations of fiducial facial points near major facial components. From depth video, the system extracts skeleton joint positions. Given tracking results, the system performs human eye gaze tracking, head pose estimation, facial expression recognition and gesture recognition.