As part of a course in designing autonomous robots, Saumeel Desai and I built a car capable of navigating roads using computer vision and sensor fusion. The car used a Raspberry Pi camera to detect roads and an echo sensor to avoid obstacles. An onboard IMU and GPS sensor were used to estimate heading and position, and encoders were used to estimate the cars velocity. Computation was performed using an Arduino Mega and a Raspberry Pi 3.
Over the course of this project I incorporated concepts from control theory, including complementary filters, state estimators and PID control. I also implemented path planning concepts such as Djikstra's algorithm to determine points on a map that could be navigated to based on GPS position.
My favorite part of this project was getting to experiment with various computer vision strategies to robustly detect roads. What ultimately worked best for me was converting images to HSV space, thresholding based on the "color of the road" and using tuned edge and line detectors. Using this I was able to get the linear offset and heading of the road, which informed navigation control of the car.