Bookmark and Share
Realtime Tracking With a Pan-Tilt Camera

Leave comment
Created by Blake Foster, Rui Wang, and Erik Learned-Miller

Parts List (links to parts included)

The human eye is amazingly adept at tracking moving objects. The process is so natural to humans that it happens without any conscious effort. While this remarkable ability depends in part on the human brain's immense processing power, the fast response of the extraocular muscles and the eyeball's light weight are also vital. Even a small point and shoot camera mounted on a servo is typically too heavy to move with the agility of the human eye. How, then, can we give a computer the ability to track movement?

The camera

We recently discovered a small first person view (FPV) camera intended for use on model airplanes. The camera is mounted on servo motors which can aim the camera with two degrees of freedom. The entire assembly weighs only 32 grams, only slightly more than a typical human eyeball. Coupled with a GPU-based tracking algorithm, the FPV camera allows the computer to track nearly anything we place in front of it with remarkable stability.

The Camera

We used a 420-line pan-tilt camera manufactured by Fat Shark. The camera is mounted on two servos motors, which allow for about 170� of rotation on the yaw axis and 90� of rotation on the pitch axis. The camera produces composite video in PAL format. An NTSC version of the camera is available as well, but it was out of stock when we ordered our parts.

The power supply connected to the camera through the voltage regulator circuit.
A schematic of the voltage regulator circuit.

Because the video transmitter requires the 12 volts, we power the camera with a 12V rechargeable Lithium battery. We use a voltage regulator to provide 5 volts for the camera and the servos. We added capacitors before and after the regulator to eliminate any voltage fluctuations.

Both the video and servo cables from the camera connect to headers on the voltage regulator circuit, which provide the regulated 5V power supply. The servo control signals and the video output are passed through to a second set of headers, which connect to the frame grabber and the Arduino, respectively. The video output header additionally provides a 12V power supply for the video transmitter.

Digitizing the Video

We used a USB frame-grabber manufactured by StarTech to read the video into the host PC. The frame grabber supports both NTSC and PAL composite video, so the NTSC camera could be used without any hardware changes. We used a video cable sold by Digital Products Company to connect the frame grabber to the video output header on the voltage regulator circuit. The cable also has a power jack, which provides 12 volts to the video transmitter.

The camera connected to the frame grabber.

The frame-grabber provides 640 x 480 interlaced video at 25 FPS. For efficiency, we downsample the video to half resolution for tracking. Our downsampling filter discards the even lines to eliminate errors due to combing artifacts. We display the video at full resolution, after eliminating combing artifacts with a standard deinterlacing filter.

The Arduino connected to the servos.

Controlling the Servos

We used an Arduino Diecimila to generate the control signal for the servos. The Arduino receives the desired pulse widths for the servos over its serial port. Each pulse width is encoded as a 16-bit integer, with 1 bit reserved to select one of the two servos. We use the servo library included with the Arduino software to generate the PWM signals.

The camera connected to the video transmitter and the RF link receiver.
The host PC connected to the video receiver and the RF link transmitter.

Wireless Operation

We can eliminate the wired connections to the host PC with a wireless transmitter and receiver for the video, and a wireless RF link for the servos.

The RF Link

The wireless RF link transmits the servo angles digitally, with a range of up to 500 feet. Both the transmitter and receiver connect to Arduinos running the Virtual Wire library. The transmitter Arduino (connected to the host PC) broadcasts each 2-byte angle, followed by a byte of all zeros to keep the transmitter and receiver in sync. The receiver Arduino updates the servo angles when an angle is transmitted correctly (that is, all 3 bytes are received). Because Virtual Wire is incompatible with the Arduino's servo library, we use the Software Servo library to control the servos.

Wireless Video

The wireless video transmitter broadcasts NTSC or PAL video with a range of up to 500 meters. The transmitter plugs directly into the RCA jack from the camera. The 12 volt line on the video output header powers the transmitter. The RCA jack on the receiver plugs directly into the frame grabber. The receiver can be powered by a generic 12 volt power adapter.



Comments:
kanad gutte (from India) August 20, 2010, 7:55 am
Hey..congrats..its very good project..
i m going to make it..can you send me more details
about coding,software using & connections..since i m just starting now with such project kind of work ..
my email address is..
kanadaditya@yahoo.co.in

thank you
frank (from italy) April 8, 2013, 4:21 pm
hello have link for download arduino software project ? best regards from italyfrankcotecorsaire@gmail.com
frank (from italy) April 8, 2013, 4:27 pm
cotecorsaire@gmail.com  this is my mail sorry for error..:-))
Steve April 21, 2013, 3:24 am
Hi, there very interesting project. I happened accross this project on another website which included a link to the software source which appeared to be broken. Is this source still available or has it been removed intentionally? Thanks steve302@adam.com.au
William Trigos (from Colombia) May 8, 2013, 4:54 pm
www.uis.edu.co
hello

first of all I would like felicitarls for their great achievements in this project so interesting. I am currently undertaking the creation of a vision system on arduino as a classroom project with undergraduate students in engineering, and I would seem interesting to evaluate and learn about its development because I think a good implementation.unfortunately the source code no available on the homepage of the project. I can access it, but for the educational purpose that would give that code to give a new approach to classes. thanks malliwi88@hotmail.com
Tom Kaz (from NJ) July 31, 2013, 4:42 pm
Hello: a similar inquiry was sent by me (and others I see) to understand how to obtain the software. Await your reply.
Tom Kaz July 31, 2013, 5:02 pm
email: tjkaz@erols.com, 201-920-3500
Jason Roberts (from Canada) August 2, 2013, 11:36 pm
Hullo Blake,Thanks for sharing your project. I like some of the guys above am dying to try out this project. If it is OK with you could you please share the supporting source code. Cheers JR. My email address is jason.robertsiii@gmail.com
Kerry Jackson September 17, 2013, 12:33 am
Thanks for sharing your project. If it is OK with you, could you please share the supporting source code. My email address is kerryjackson@netscape.net
Anonymous September 20, 2013, 10:47 am
Hi, thanks for sharing your project. Would you please share the source code too?. My email address is sainer.s@gmail.com
Nick Kiriazis October 29, 2013, 10:56 am
Hi congrats for your very good project. We are trying to do something similar to this but the camera will be in a flying quadcopter. Could you please share the source code. My e-mail address is xristos35@yahoo.gr. Thank you for sharing your project.
Vinayak April 3, 2014, 9:37 am
Please, it would be great if you could send me the code for this project. I am also working in tracking using GPU..this code would be of great study in my research..My mail id is nagelivs1981@gmail.com waiting for your kind reply..
nor6 November 25, 2014, 6:06 am
Hi, thanks for sharing your project. Would you please share the source code too?. My email address is nor@katch.ne.jp
Anonymous March 1, 2015, 1:04 pm
Thanks for sharing your project. If it is ok with you, could you please share the supporting source code. My email address is nkolder@gmail.com
ben (from singapore) April 2, 2015, 2:26 am
Hi, good job. Could you please share the source code? my email address is yunbensg@gmail.com. Many thanks.
syir March 5, 2016, 11:49 pm
Hi, congrats for your impressive project. I'm trying to do something similar to this but using multi camera for multi point tracking. Could you please share the hardware connection,software used and the source code. My e-mail address is basyir19@gmail.com . thanks..^_^.
Mihail September 2, 2016, 3:38 am
Hello. I've not ever seen before perfect like this. Really want to make adopt my project.Can you please share source code? Thanks really. ramdonardo@gmail.com
mehmet December 14, 2016, 8:00 am
Hey..congrats..its very good project..
i m going to make it..can you send me more details
about coding,software using & connections..since i m just starting now with such project kind of work ..
my email address is..                     mmtevrn13@gmail.com
Leave comment

Name (optional):

Location (optional):

Website (optional):

Comment:

Characters remaining: 600


Please enter the code that appears in the above image to verify that you are human: