Car tracking with cascades


Car Tracking with OpenCV
Car Tracking with OpenCV
In this tutorial we will look at vehicle tracking using haar features. We have a haar cascade file trained on cars.

The program will detect regions of interest, classify them as cars and show rectangles around them.

Download Code + Video + Cascade file

Detecting with cascades

Lets start with the basic cascade detection program:

#! /usr/bin/python
 
import cv2
 
face_cascade = cv2.CascadeClassifier('cars.xml')
vc = cv2.VideoCapture('road.avi')
 
if vc.isOpened():
    rval , frame = vc.read()
else:
    rval = False
 
while rval:
    rval, frame = vc.read()
 
    # car detection.
    cars = face_cascade.detectMultiScale(frame, 1.1, 2)
 
    ncars = 0
    for (x,y,w,h) in cars:
        cv2.rectangle(frame,(x,y),(x+w,y+h),(0,0,255),2)
        ncars = ncars + 1
 
    # show result
    cv2.imshow("Result",frame)
    cv2.waitKey(1);
vc.release()

This will detect cars in the screen but also noise and the screen will be jittering sometimes. To avoid all of these, we have to improve our car tracking algorithm.  We decided to come up with a simple solution.

Car tracking algorithm

For every frame:

  • Detect potential regions of interest
  • Filter detected regions based on vertical,horizontal similarity
  • If its a new region, add to the collection
  • Clear collection every 30 frames

 
Removing false positives
The mean square error function is used to remove false positives. We compare vertical and horizontal sides of the images. If the difference is to large or to small it cannot be a car.

ROI detection
A car may not be detected in every frame. If a new car is detected, its added to the collection.
We keep this collection for 30 frames, then clear it.

#! /usr/bin/python
 
import cv2
import numpy as np
 
def diffUpDown(img):
    # compare top and bottom size of the image
    # 1. cut image in two
    # 2. flip the top side
    # 3. resize to same size
    # 4. compare difference  
    height, width, depth = img.shape
    half = height/2
    top = img[0:half, 0:width]
    bottom = img[half:half+half, 0:width]
    top = cv2.flip(top,1)
    bottom = cv2.resize(bottom, (32, 64)) 
    top = cv2.resize(top, (32, 64))  
    return ( mse(top,bottom) )
 
 
def diffLeftRight(img):
    # compare left and right size of the image
    # 1. cut image in two
    # 2. flip the right side
    # 3. resize to same size
    # 4. compare difference  
    height, width, depth = img.shape
    half = width/2
    left = img[0:height, 0:half]
    right = img[0:height, half:half + half-1]
    right = cv2.flip(right,1)
    left = cv2.resize(left, (32, 64)) 
    right = cv2.resize(right, (32, 64))  
    return ( mse(left,right) )
 
 
def mse(imageA, imageB):
    err = np.sum((imageA.astype("float") - imageB.astype("float")) ** 2)
    err /= float(imageA.shape[0] * imageA.shape[1])
    return err
 
def isNewRoi(rx,ry,rw,rh,rectangles):
    for r in rectangles:
        if abs(r[0] - rx) < 40 and abs(r[1] - ry) < 40:
           return False  
    return True
 
def detectRegionsOfInterest(frame, cascade):
    scaleDown = 2
    frameHeight, frameWidth, fdepth = frame.shape 
 
    # Resize
    frame = cv2.resize(frame, (frameWidth/scaleDown, frameHeight/scaleDown)) 
    frameHeight, frameWidth, fdepth = frame.shape 
 
    # haar detection.
    cars = cascade.detectMultiScale(frame, 1.2, 1)
 
    newRegions = []
    minY = int(frameHeight*0.3)
 
    # iterate regions of interest
    for (x,y,w,h) in cars:
            roi = [x,y,w,h]
            roiImage = frame[y:y+h, x:x+w]   
 
            carWidth = roiImage.shape[0]
            if y > minY:
                diffX = diffLeftRight(roiImage)
                diffY = round(diffUpDown(roiImage))
 
                if diffX > 1600 and diffX < 3000 and diffY > 12000:
                    rx,ry,rw,rh = roi
                    newRegions.append( [rx*scaleDown,ry*scaleDown,rw*scaleDown,rh*scaleDown] )
 
    return newRegions
 
 
def detectCars(filename):
    rectangles = []
    cascade = cv2.CascadeClassifier('cars.xml')
    vc = cv2.VideoCapture(filename)
 
    if vc.isOpened():
        rval , frame = vc.read()
    else:
        rval = False
 
    roi = [0,0,0,0]
    frameCount = 0
 
    while rval:
        rval, frame = vc.read()
        frameHeight, frameWidth, fdepth = frame.shape 
 
        newRegions = detectRegionsOfInterest(frame, cascade)
        for region in newRegions:
            if isNewRoi(region[0],region[1],region[2],region[3],rectangles):
                rectangles.append(region)
 
        for r in rectangles:
            cv2.rectangle(frame,(r[0],r[1]),(r[0]+r[2],r[1]+r[3]),(0,0,255),3) 
 
        frameCount = frameCount + 1
        if frameCount > 30: 
            frameCount = 0
            rectangles = []
 
        # show result
        cv2.imshow("Result",frame)
        cv2.waitKey(1);
    vc.release()
 
detectCars('road.avi')

Final notes
The cascades are not  rotation invariant, scale and translation invariant. In addition, Detecting vehicles with haar cascades may work reasonably well, but there is gain with other algorithms (salient points).

You may like:

 
Download Code + Video + Cascade file


23 thoughts on “Car tracking with cascades

  1. Supriya - March 6, 2016

    if x > fwidth*0.4 and x < fwidth*0.5 and y > fheight*0.25 and w > 30 and h > 30: their is syntax error in this cmd

    1. Frank - March 6, 2016

      Try the updated code. Do you have the cascading file (cars.xml) ?

  2. Jordi Truji - February 27, 2016

    I’m trying to run your code but I get the following error:

    line 12, in 
        cars = face_cascade.detectMultiScale(frame, 1.1, 2)
    cv2.error: /root/opencv/opencv/modules/core/src/array.cpp:2494: error: (-206) Unrecognized or unsupported array type in function cvGetMat

    Is it about the array type of frames?

    1. Frank - February 28, 2016

      Does it occur on every frame?

      The function cvGetMat converts arrays into a Mat. Mat is a datastructure used by cv2 to represent a matrix.
      I think there is a file loading error. Do you have cars.xml and video2.avi in the same directory? Make sure the face_cascade and vc are loaded. If loading goes wrong, throw an error.

  3. Kundan Kumar - February 21, 2016

    Hi, I have to track human,cars and buses in the video and count each of them.
    Could you please help me how to do that and also how to trained xml for human and buses.
    Thanks

    1. Frank - February 25, 2016

      There are different techniques to track them. Cascades are one way to track these. OpenCV comes with a bunch of trained cascades. You can find some tutorials here: http://stackoverflow.com/questions/2000816/how-to-create-haar-cascade-xml-for-using-with-opencv

  4. Vijay Shah - November 5, 2015

    Dear Frank,
    This is Vijay Shah from High Tech Security Servies, India. We have installed CCTV cameras on highways. Can you give us an application to count & classify vehicles ? We are ready to pay for your development charges.

  5. Gandalf - August 3, 2015

    Good day sir
    The algorithm looks quite good and I want to do something similar. May I ask what you used to train the classifier. Did you use some database or did you collect all the training images yourself?

    1. Frank - August 3, 2015

      hi, in this case the classifier was already trained with a car image database. If you want to train your own classifier, there are many image databases available freely on computer vision / image processing websites.

      1. Will - March 16, 2016

        Hello Frank,
        Would you mind to share which database you used to train the classifier?
        Thank you.

        1. Frank - March 16, 2016

          Hi Will,
          The classifier is trained using an image database containing cars and non-cars. You can find many of such databases online.
          Some datasets:

    2. Frank - August 3, 2015

      Good day, in this case the classifier was already trained with a car image database. If you want to train your own classifier, there are many image databases available freely on computer vision / image processing websites.