python logo

Car tracking with cascades


Python hosting: Host, run, and code Python in the cloud!
Car Tracking with OpenCV Car Tracking with OpenCV

In this tutorial we will look at vehicle tracking using haar features. We have a haar cascade file trained on cars.

The program will detect regions of interest, classify them as cars and show rectangles around them.

Related course:
Master Computer Vision with OpenCV

Detecting with cascades

Lets start with the basic cascade detection program:

#! /usr/bin/python

import cv2

face_cascade = cv2.CascadeClassifier('cars.xml')
vc = cv2.VideoCapture('road.avi')

if vc.isOpened():
rval , frame = vc.read()
else:
rval = False

while rval:
rval, frame = vc.read()

# car detection.
cars = face_cascade.detectMultiScale(frame, 1.1, 2)

ncars = 0
for (x,y,w,h) in cars:
cv2.rectangle(frame,(x,y),(x+w,y+h),(0,0,255),2)
ncars = ncars + 1

# show result
cv2.imshow("Result",frame)
cv2.waitKey(1);
vc.release()

This will detect cars in the screen but also noise and the screen will be jittering sometimes. To avoid all of these, we have to improve our car tracking algorithm.  We decided to come up with a simple solution.

Related course:
Master Computer Vision with OpenCV

Car tracking algorithm


For every frame:
  • Detect potential regions of interest
  • Filter detected regions based on vertical,horizontal similarity
  • If its a new region, add to the collection
  • Clear collection every 30 frames

Removing false positives
The mean square error function is used to remove false positives. We compare vertical and horizontal sides of the images. If the difference is to large or to small it cannot be a car.

ROI detection
A car may not be detected in every frame. If a new car is detected, its added to the collection.
We keep this collection for 30 frames, then clear it.

#!/usr/bin/python

import cv2
import numpy as np

def diffUpDown(img):
# compare top and bottom size of the image
# 1. cut image in two
# 2. flip the top side
# 3. resize to same size
# 4. compare difference
height, width, depth = img.shape
half = height/2
top = img[0:half, 0:width]
bottom = img[half:half+half, 0:width]
top = cv2.flip(top,1)
bottom = cv2.resize(bottom, (32, 64))
top = cv2.resize(top, (32, 64))
return ( mse(top,bottom) )

def diffLeftRight(img):
# compare left and right size of the image
# 1. cut image in two
# 2. flip the right side
# 3. resize to same size
# 4. compare difference
height, width, depth = img.shape
half = width/2
left = img[0:height, 0:half]
right = img[0:height, half:half + half-1]
right = cv2.flip(right,1)
left = cv2.resize(left, (32, 64))
right = cv2.resize(right, (32, 64))
return ( mse(left,right) )

def mse(imageA, imageB):
err = np.sum((imageA.astype("float") - imageB.astype("float")) ** 2)
err /= float(imageA.shape[0] * imageA.shape[1])
return err

def isNewRoi(rx,ry,rw,rh,rectangles):
for r in rectangles:
if abs(r[0] - rx) < 40 and abs(r[1] - ry) < 40:
return False
return True

def detectRegionsOfInterest(frame, cascade):
scaleDown = 2
frameHeight, frameWidth, fdepth = frame.shape
# Resize
frame = cv2.resize(frame, (frameWidth/scaleDown, frameHeight/scaleDown))
frameHeight, frameWidth, fdepth = frame.shape

# haar detection.
cars = cascade.detectMultiScale(frame, 1.2, 1)

newRegions = []
minY = int(frameHeight*0.3)

# iterate regions of interest
for (x,y,w,h) in cars:
roi = [x,y,w,h]
roiImage = frame[y:y+h, x:x+w]

carWidth = roiImage.shape[0]
if y > minY:
diffX = diffLeftRight(roiImage)
diffY = round(diffUpDown(roiImage))
if diffX > 1600 and diffX < 3000 and diffY > 12000:
rx,ry,rw,rh = roi
newRegions.append( [rx*scaleDown,ry*scaleDown,rw*scaleDown,rh*scaleDown] )

return newRegions

def detectCars(filename):
rectangles = []
cascade = cv2.CascadeClassifier('cars.xml')
vc = cv2.VideoCapture(filename)

if vc.isOpened():
rval , frame = vc.read()
else:
rval = False

roi = [0,0,0,0]
frameCount = 0

while rval:
rval, frame = vc.read()
frameHeight, frameWidth, fdepth = frame.shape

newRegions = detectRegionsOfInterest(frame, cascade)
for region in newRegions:
if isNewRoi(region[0],region[1],region[2],region[3],rectangles):
rectangles.append(region)

for r in rectangles:
cv2.rectangle(frame,(r[0],r[1]),(r[0]+r[2],r[1]+r[3]),(0,0,255),3)

frameCount = frameCount + 1
if frameCount > 30:
frameCount = 0
rectangles = []

# show result
cv2.imshow("Result",frame)
cv2.waitKey(1);
vc.release()

detectCars('road.avi')

Final notes
The cascades are not  rotation invariant, scale and translation invariant. In addition, Detecting vehicles with haar cascades may work reasonably well, but there is gain with other algorithms (salient points).

You may like:

Download Computer Vision Examples + Course

Back





Leave a Reply:




Gandalf Mon, 03 Aug 2015

Good day sir
The algorithm looks quite good and I want to do something similar. May I ask what you used to train the classifier. Did you use some database or did you collect all the training images yourself?

Frank Mon, 03 Aug 2015

hi, in this case the classifier was already trained with a car image database. If you want to train your own classifier, there are many image databases available freely on computer vision / image processing websites.

Frank Mon, 03 Aug 2015

Good day, in this case the classifier was already trained with a car image database. If you want to train your own classifier, there are many image databases available freely on computer vision / image processing websites.

Vijay Shah Thu, 05 Nov 2015

Dear Frank,
This is Vijay Shah from High Tech Security Servies, India. We have installed CCTV cameras on highways. Can you give us an application to count & classify vehicles ? We are ready to pay for your development charges.

Frank Mon, 09 Nov 2015

Hi Vijay, drop me a mail at [email protected]

Kundan Kumar Sun, 21 Feb 2016

Hi, I have to track human,cars and buses in the video and count each of them.
Could you please help me how to do that and also how to trained xml for human and buses.
Thanks

Frank Thu, 25 Feb 2016

There are different techniques to track them. Cascades are one way to track these. OpenCV comes with a bunch of trained cascades. You can find some tutorials here: http://stackoverflow.com/qu...

Jordi Truji Sat, 27 Feb 2016

I'm trying to run your code but I get the following error:
[python]
line 12, in
cars = face_cascade.detectMultiScale(frame, 1.1, 2)
cv2.error: /root/opencv/opencv/modules/core/src/array.cpp:2494: error: (-206) Unrecognized or unsupported array type in function cvGetMat
[/python]
Is it about the array type of frames?

Frank Sun, 28 Feb 2016

Does it occur on every frame?

The function cvGetMat converts arrays into a Mat. Mat is a datastructure used by cv2 to represent a matrix.
I think there is a file loading error. Do you have cars.xml and video2.avi in the same directory? Make sure the face_cascade and vc are loaded. If loading goes wrong, throw an error.

Supriya Sun, 06 Mar 2016

if x > fwidth*0.4 and x < fwidth*0.5 and y > fheight*0.25 and w > 30 and h > 30: their is syntax error in this cmd

Saiganesh Menon Sun, 06 Mar 2016

[python]
File "car_detection_original_refinement_new_1.py">car_detection_original_refi...", line 27
if x < fwidth*0.4 and x < fwidth*0.5 and y > fheight*0.25 and w > 30 and h > 30:
^
SyntaxError: invalid syntax
[/python]
how do is solve this.
great post btw
cheers

Frank Sun, 06 Mar 2016

Try the updated code, make sure the indention is correct. Be sure that you have the cascading file (cars3.xml)
The original repository is no longer available, please check the new link to the cascade file from github.

Frank Sun, 06 Mar 2016

Try the updated code. Do you have the cascading file (cars.xml) ?

Will Wed, 16 Mar 2016

Hello Frank,
Would you mind to share which database you used to train the classifier?
Thank you.

Frank Wed, 16 Mar 2016

Hi Will,
The classifier is trained using an image database containing cars and non-cars. You can find many of such databases online.
Some datasets:
<ul>
<li>http://ai.stanford.edu/~jkr...
<li>http://archive.ics.uci.edu/...
<li>http://cvlab.epfl.ch/data/p...
</ul>

techie Mon, 27 Jun 2016

Hi..How can I track particular car in consecutive frames where I have number of and changing cars in each frame? I need to track and then do some processing on particular detected object.

techie Tue, 28 Jun 2016

Hi..I am not getting the way you have calculated ROI using these 0.4 & 0.5 multiplications. Can you please share the logic behind it?

Frank Sun, 03 Jul 2016

These are simply boundary cases selected chance, you could leave them out. Basically we state that x and y of the top left of the ROI should be in the top left region. Then the sizes of the ROI are checked.

Frank Sun, 03 Jul 2016

First extract some features (color, structure or salient points) from the target car.
Then detect all cars using cascades and determine if it has these features.

victor Thu, 07 Jul 2016

Nice. Very good and excellent tutorial.
However, it seems that the cascade can only detect the car right in front of the camera but not on the right/left track.

Frank Fri, 15 Jul 2016

Correct, cascades can be trained for certain objects. You could simple download another cascade file from the web, or train yourself.

A left/right cascade:
https://github.com/abhi-kumar/CAR-DETECTION/blob/master/checkcas.xml

Dataset from University of Illinois
http://cogcomp.cs.illinois.edu/Data/Car/

vinod Thu, 11 Aug 2016

i tried to run this code , its not showing anything , i mean its not showing output , i don't have any errors

Frank Thu, 11 Aug 2016

Did you download the video + cascade file? Which Python version did you use? Do you have cv2 installed?

Thodsapon Hunsanon Fri, 10 Mar 2017

[python]
Traceback (most recent call last):
File "C:\Users\ExCITE-LAPTOP\Desktop\Car Tracking\carTracking\detect.py", line 113, in
detectCars('road.avi')
File "C:\Users\ExCITE-LAPTOP\Desktop\Car Tracking\carTracking\detect.py">detect.py", line 93, in detectCars
frameHeight, frameWidth, fdepth = frame.shape
AttributeError: 'NoneType' object has no attribute 'shape'
[/python]
How can I resolve this error? Thanks in Advance

Frank Sat, 11 Mar 2017

Do you have the files? (road.avi, cars.xml)

Priyanka Kochhar Sun, 19 Mar 2017

Another interesting approach is to use OpenCV HOG + SVM classifier to detect vehicles. Check out this blog:
https://medium.com/@priya.dwivedi/automatic-vehicle-detection-for-self-driving-cars-8d98c086b161#.6x2k4szf5

Volodymyr Rykhva Sun, 19 Mar 2017

hi:) Thanks for posting this tut.It helps me to understand a lot. But I got a problem with improved version of your car detection script. The basic script works well (emm...I meant it shows red rectangle when a car is in focus) and your solution doesn't show this red frame. What could be the issue here? Thanks

Frank Mon, 20 Mar 2017

Is it the same video? The array rectangles is probably empty, inside the loop of detectCars(), check using
[python]
print(len(rectangles))
[/python]

Volodymyr Rykhva Sun, 26 Mar 2017

yeap, it's actually empty for some reasons:(

Frank Mon, 27 Mar 2017

try changing the method isNewRoi, that decides if its added to to the list. The values are set to 40, which is just set by a test. Try changing the values

Saeed Iqbal Mon, 15 May 2017

Dear Sir,

I am facing the below error:
Unable to stop the stream: Inappropriate ioctl for device

Frank Mon, 15 May 2017

Hi Saeed, are you using a webcam or video file?

Sébastien Turrel Sun, 28 May 2017

Frank did you create the xml yourself ?
my openCv training on my Debian serv always stop after stage 0 :(
btw, when i run your last script with Roi the error is :
[python]
frame = cv2.resize(frame, (frameWidth / scaleDown, frameHeight / scaleDown))
TypeError: integer argument expected, got float
[/python]
can you help me with that my friend ?

Frank Mon, 29 May 2017

I don't recall, has been a while. Do you have video capture? Try to show the verify these variables contain the right values using debugger or print function. It mentions a type error, cast frameWidth/scaleDown and the other one to integers using int().

Sébastien Turrel Mon, 29 May 2017

this code is very interresting my friend.
when i do :
frame = cv2.resize(frame, (int(frameWidth / scaleDown), int(frameHeight / scaleDown)))
frameHeight, frameWidth, fdepth = frame.shape
it s say :
left = img[0:height, 0:half]
TypeError: slice indices must be integers or None or have an __index__ method
is it possible to you to write it again or something ? i am just discovering haar cascade and it's very fascinating.

Frank Wed, 31 May 2017

try with both Python 2 and Python 3. Can you display the data beforehand?

Sébastien Turrel Tue, 06 Jun 2017

Still here my friend ? :)

Frank Fri, 09 Jun 2017

Hi, has been a few busy days. I've no problem running the code from the zip file.
I'm using Python 2.7.13 on Manjaro Linux.
Did you download the zip file? It has an attached video and haar file.

Sébastien Turrel Fri, 09 Jun 2017

i did it my friend ;) maybe it's just because i am using Python 3.5 ;)

Sébastien Turrel Sun, 11 Jun 2017

yeah i did it, but i am running Python 3.5