Final Project: Enhanced Augmented Reality
For this project, students were required to create a simple markerless augmented reality system, similar to the assignment below, but without the corner markers and produce a white paper detailing our experiences with the technique.
To accomplish this task, I used an image tracker system which uses a combination of image recognition to identify a target and then search through input frames for positive identification.
Each frame, a number of steps are completed including feature extraction and matching, homography estimation and refinement, and pose estimation based on a known camera calibration.
A few of the computer vision methods used in the project are ORB feature detection, KNN feature matching using a brute force matcher with cross correlation, homography estimation using RANSAC, SolvePNP() with Levenberg-Marquardt optimization, and conversion of the rotation matrix to a vector using Rodrigues.
In this project, students were required to build a basic augmented reality system using corner markers and handle some real life cases such as camera rotation/skew as well as video artifacts and noise.
To start, we injected basic still frames into the video with a calculated homography to ensure the process worked correctly but the extra credit version extended this technique by interleaving frames from a video into the marker locations.
To achieve this, I used non-local means denoising to remove any static noise and then Gaussian blurred the result to remove circle artifacts.
Harris corners were used to estimate marker locations and those results were fed into KMeans with a K = 4 to identify the corners.
These four points were then used to compute a homography matrix using least squares and the injected image was then inverse warped into the final frame to avoid holes or strange artifacts in the output.
Motion Detection and Optical Flow
In this assignment we implemented standard and hierarchical versions of optical flow using image pyramids, based on the Lucas Kanade algorithm and papers.
For extra credit, we visualized the algorithm results with quiver plots overlayed on video frames.
Traffic Lights and Signs
For the traffic light project, students were tasked with detecting traffic lights and signs in simulated images. Later, these detectors were used with real images to evaulate their efficacy in real world scenarios.
Object Tracking and Pedestrian Detection
In this project, a variety of scenarios were presented for students to track pedestrians on the street and detect patches in presidential debates captured from TV.
This final project implemented used a variety of techniques to detect faces in common datasets such as Yalefaces and Faces94.