Computer Vision

Final Project: Enhanced Augmented Reality

For this project, students were required to create a simple markerless augmented reality system, similar to the assignment below, but without the corner markers and produce a white paper detailing our experiences with the technique.

To accomplish this task, I used an image tracker system which uses a combination of image recognition to identify a target and then search through input frames for positive identification.

Each frame, a number of steps are completed including feature extraction and matching, homography estimation and refinement, and pose estimation based on a known camera calibration.

A few of the computer vision methods used in the project are ORB feature detection, KNN feature matching using a brute force matcher with cross correlation, homography estimation using RANSAC, SolvePNP() with Levenberg-Marquardt optimization, and conversion of the rotation matrix to a vector using Rodrigues.

Algorithms: ORB, BF Matcher, KNN Matching, RANSAC, findHomography, warpPerspective, perspectiveTransform, camera calibration, pose estimation, solvePNP, Rodrigues

Papers: ORB: an efficient alternative to SIFT or SURF (2011), Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography (1980)

Augmented Reality

In this project, students were required to build a basic augmented reality system using corner markers and handle some real life cases such as camera rotation/skew as well as video artifacts and noise.

To start, we injected basic still frames into the video with a calculated homography to ensure the process worked correctly but the extra credit version extended this technique by interleaving frames from a video into the marker locations.

To achieve this, I used non-local means denoising to remove any static noise and then Gaussian blurred the result to remove circle artifacts.

Harris corners were used to estimate marker locations and those results were fed into KMeans with a K = 4 to identify the corners.

These four points were then used to compute a homography matrix using least squares and the injected image was then inverse warped into the final frame to avoid holes or strange artifacts in the output.

Algorithms: Fast non-local means denoising, median blur, gaussian blur, dilate, Harris Corners, kmeans, homography estimation, corner detection

Papers: A Combined Corner and Edge Detector (1988)

Motion Detection and Optical Flow

In this assignment we implemented standard and hierarchical versions of optical flow using image pyramids, based on the Lucas Kanade algorithm and papers.

For extra credit, we visualized the algorithm results with quiver plots overlayed on video frames.

Algorithms: Sobel, Sobel Operator, Optical Flow, Hierarchical Lucas Kanade

Traffic Lights and Signs

For the traffic light project, students were tasked with detecting traffic lights and signs in simulated images. Later, these detectors were used with real images to evaulate their efficacy in real world scenarios.

Algorithms: Hough Lines, Hough Lines Probablistic, Hough Circles, Canny Edge Detection, Hough Transform

Object Tracking and Pedestrian Detection

In this project, a variety of scenarios were presented for students to track pedestrians on the street and detect patches in presidential debates captured from TV.

Algorithms: Kalman Filter | code, Particle Filters | code

Classification

This final project implemented used a variety of techniques to detect faces in common datasets such as Yalefaces and Faces94.

Algorithms: Principal Component Analysis, Boosting, Haar Features, Viola Jones, Hidden Markov Models

Papers: Rapid Object Detection using a Boosted Cascade of Simple Features (2001)