Car tracking with cascades

Car Tracking with OpenCV
Car Tracking with OpenCV
In this tutorial we will look at vehicle tracking using haar features. We have a haar cascade file trained on cars.

The program will detect regions of interest, classify them as cars and show rectangles around them.

Related course:

Detecting with cascades

Lets start with the basic cascade detection program:

#! /usr/bin/python
import cv2
face_cascade = cv2.CascadeClassifier('cars.xml')
vc = cv2.VideoCapture('road.avi')
if vc.isOpened():
    rval , frame =
    rval = False
while rval:
    rval, frame =
    # car detection.
    cars = face_cascade.detectMultiScale(frame, 1.1, 2)
    ncars = 0
    for (x,y,w,h) in cars:
        ncars = ncars + 1
    # show result

This will detect cars in the screen but also noise and the screen will be jittering sometimes. To avoid all of these, we have to improve our car tracking algorithm.  We decided to come up with a simple solution.

Car tracking algorithm

For every frame:

  • Detect potential regions of interest
  • Filter detected regions based on vertical,horizontal similarity
  • If its a new region, add to the collection
  • Clear collection every 30 frames

Removing false positives
The mean square error function is used to remove false positives. We compare vertical and horizontal sides of the images. If the difference is to large or to small it cannot be a car.

ROI detection
A car may not be detected in every frame. If a new car is detected, its added to the collection.
We keep this collection for 30 frames, then clear it.

#! /usr/bin/python
import cv2
import numpy as np
def diffUpDown(img):
    # compare top and bottom size of the image
    # 1. cut image in two
    # 2. flip the top side
    # 3. resize to same size
    # 4. compare difference  
    height, width, depth = img.shape
    half = height/2
    top = img[0:half, 0:width]
    bottom = img[half:half+half, 0:width]
    top = cv2.flip(top,1)
    bottom = cv2.resize(bottom, (32, 64)) 
    top = cv2.resize(top, (32, 64))  
    return ( mse(top,bottom) )
def diffLeftRight(img):
    # compare left and right size of the image
    # 1. cut image in two
    # 2. flip the right side
    # 3. resize to same size
    # 4. compare difference  
    height, width, depth = img.shape
    half = width/2
    left = img[0:height, 0:half]
    right = img[0:height, half:half + half-1]
    right = cv2.flip(right,1)
    left = cv2.resize(left, (32, 64)) 
    right = cv2.resize(right, (32, 64))  
    return ( mse(left,right) )
def mse(imageA, imageB):
    err = np.sum((imageA.astype("float") - imageB.astype("float")) ** 2)
    err /= float(imageA.shape[0] * imageA.shape[1])
    return err
def isNewRoi(rx,ry,rw,rh,rectangles):
    for r in rectangles:
        if abs(r[0] - rx) < 40 and abs(r[1] - ry) < 40:
           return False  
    return True
def detectRegionsOfInterest(frame, cascade):
    scaleDown = 2
    frameHeight, frameWidth, fdepth = frame.shape 
    # Resize
    frame = cv2.resize(frame, (frameWidth/scaleDown, frameHeight/scaleDown)) 
    frameHeight, frameWidth, fdepth = frame.shape 
    # haar detection.
    cars = cascade.detectMultiScale(frame, 1.2, 1)
    newRegions = []
    minY = int(frameHeight*0.3)
    # iterate regions of interest
    for (x,y,w,h) in cars:
            roi = [x,y,w,h]
            roiImage = frame[y:y+h, x:x+w]   
            carWidth = roiImage.shape[0]
            if y > minY:
                diffX = diffLeftRight(roiImage)
                diffY = round(diffUpDown(roiImage))
                if diffX > 1600 and diffX < 3000 and diffY > 12000:
                    rx,ry,rw,rh = roi
                    newRegions.append( [rx*scaleDown,ry*scaleDown,rw*scaleDown,rh*scaleDown] )
    return newRegions
def detectCars(filename):
    rectangles = []
    cascade = cv2.CascadeClassifier('cars.xml')
    vc = cv2.VideoCapture(filename)
    if vc.isOpened():
        rval , frame =
        rval = False
    roi = [0,0,0,0]
    frameCount = 0
    while rval:
        rval, frame =
        frameHeight, frameWidth, fdepth = frame.shape 
        newRegions = detectRegionsOfInterest(frame, cascade)
        for region in newRegions:
            if isNewRoi(region[0],region[1],region[2],region[3],rectangles):
        for r in rectangles:
        frameCount = frameCount + 1
        if frameCount > 30: 
            frameCount = 0
            rectangles = []
        # show result

Final notes
The cascades are not  rotation invariant, scale and translation invariant. In addition, Detecting vehicles with haar cascades may work reasonably well, but there is gain with other algorithms (salient points).

You may like:

Download Code + Video + Cascade file

Face detection in Google Hangouts video

32 thoughts on “Car tracking with cascades

  1. Saeed Iqbal - May 15, 2017

    Dear Sir,

    I am facing the below error:
    Unable to stop the stream: Inappropriate ioctl for device

    1. Frank - May 15, 2017

      Hi Saeed, are you using a webcam or video file?

  2. Volodymyr Rykhva - March 19, 2017

    hi:) Thanks for posting this tut.It helps me to understand a lot. But I got a problem with improved version of your car detection script. The basic script works well (emm…I meant it shows red rectangle when a car is in focus) and your solution doesn’t show this red frame. What could be the issue here? Thanks

    1. Frank - March 20, 2017

      Is it the same video? The array rectangles is probably empty, inside the loop of detectCars(), check using

      1. Volodymyr Rykhva - March 26, 2017

        yeap, it’s actually empty for some reasons:(

        1. Frank - March 27, 2017

          try changing the method isNewRoi, that decides if its added to to the list. The values are set to 40, which is just set by a test. Try changing the values

  3. Thodsapon Hunsanon - March 10, 2017
    Traceback (most recent call last):
      File "C:\Users\ExCITE-LAPTOP\Desktop\Car Tracking\carTracking\", line 113, in 
      File "C:\Users\ExCITE-LAPTOP\Desktop\Car Tracking\carTracking\", line 93, in detectCars
        frameHeight, frameWidth, fdepth = frame.shape
    AttributeError: 'NoneType' object has no attribute 'shape'

    How can I resolve this error? Thanks in Advance

    1. Frank - March 11, 2017

      Do you have the files? (road.avi, cars.xml)