Dynamic Facial Tracking
For this projected, we used open source libraries in openCV to detect faces, and then implemented optical flow algorithms to track the movement of the facial features over the course of a video.
Some of the largest challenges of this project were related to detecting faces in different conditions. Faces in certain videos (e.g. faces are oriented toward the camera) were much easier to detect. Once faces were detected in the first frame of a video, following the movement of the face throughout the video is also a delicate balance of tuning how sensitive the optical flow function is to movement between frames.
The video of the cast of Stranger Things being interviewed by Jimmy Fallon is a great example. as the interviewees rapidly jump up and down, there are points where the optical flow I implemented can't follow the motion of the face between frames and "loses" the face for a second.