Car tracking with cascades


Car Tracking with OpenCV
Car Tracking with OpenCV
In this tutorial we will look at vehicle tracking using haar features. We have a haar cascade file trained on cars.

The program will detect regions of interest, classify them as cars and show rectangles around them.

Related courses:

Detecting with cascades

Lets start with the basic cascade detection program:

#! /usr/bin/python
 
import cv2
 
face_cascade = cv2.CascadeClassifier('cars.xml')
vc = cv2.VideoCapture('road.avi')
 
if vc.isOpened():
    rval , frame = vc.read()
else:
    rval = False
 
while rval:
    rval, frame = vc.read()
 
    # car detection.
    cars = face_cascade.detectMultiScale(frame, 1.1, 2)
 
    ncars = 0
    for (x,y,w,h) in cars:
        cv2.rectangle(frame,(x,y),(x+w,y+h),(0,0,255),2)
        ncars = ncars + 1
 
    # show result
    cv2.imshow("Result",frame)
    cv2.waitKey(1);
vc.release()

This will detect cars in the screen but also noise and the screen will be jittering sometimes. To avoid all of these, we have to improve our car tracking algorithm.  We decided to come up with a simple solution.

Car tracking algorithm

For every frame:

  • Detect potential regions of interest
  • Filter detected regions based on vertical,horizontal similarity
  • If its a new region, add to the collection
  • Clear collection every 30 frames

 
Removing false positives
The mean square error function is used to remove false positives. We compare vertical and horizontal sides of the images. If the difference is to large or to small it cannot be a car.

ROI detection
A car may not be detected in every frame. If a new car is detected, its added to the collection.
We keep this collection for 30 frames, then clear it.

#! /usr/bin/python
 
import cv2
import numpy as np
 
def diffUpDown(img):
    # compare top and bottom size of the image
    # 1. cut image in two
    # 2. flip the top side
    # 3. resize to same size
    # 4. compare difference  
    height, width, depth = img.shape
    half = height/2
    top = img[0:half, 0:width]
    bottom = img[half:half+half, 0:width]
    top = cv2.flip(top,1)
    bottom = cv2.resize(bottom, (32, 64)) 
    top = cv2.resize(top, (32, 64))  
    return ( mse(top,bottom) )
 
 
def diffLeftRight(img):
    # compare left and right size of the image
    # 1. cut image in two
    # 2. flip the right side
    # 3. resize to same size
    # 4. compare difference  
    height, width, depth = img.shape
    half = width/2
    left = img[0:height, 0:half]
    right = img[0:height, half:half + half-1]
    right = cv2.flip(right,1)
    left = cv2.resize(left, (32, 64)) 
    right = cv2.resize(right, (32, 64))  
    return ( mse(left,right) )
 
 
def mse(imageA, imageB):
    err = np.sum((imageA.astype("float") - imageB.astype("float")) ** 2)
    err /= float(imageA.shape[0] * imageA.shape[1])
    return err
 
def isNewRoi(rx,ry,rw,rh,rectangles):
    for r in rectangles:
        if abs(r[0] - rx) < 40 and abs(r[1] - ry) < 40:
           return False  
    return True
 
def detectRegionsOfInterest(frame, cascade):
    scaleDown = 2
    frameHeight, frameWidth, fdepth = frame.shape 
 
    # Resize
    frame = cv2.resize(frame, (frameWidth/scaleDown, frameHeight/scaleDown)) 
    frameHeight, frameWidth, fdepth = frame.shape 
 
    # haar detection.
    cars = cascade.detectMultiScale(frame, 1.2, 1)
 
    newRegions = []
    minY = int(frameHeight*0.3)
 
    # iterate regions of interest
    for (x,y,w,h) in cars:
            roi = [x,y,w,h]
            roiImage = frame[y:y+h, x:x+w]   
 
            carWidth = roiImage.shape[0]
            if y > minY:
                diffX = diffLeftRight(roiImage)
                diffY = round(diffUpDown(roiImage))
 
                if diffX > 1600 and diffX < 3000 and diffY > 12000:
                    rx,ry,rw,rh = roi
                    newRegions.append( [rx*scaleDown,ry*scaleDown,rw*scaleDown,rh*scaleDown] )
 
    return newRegions
 
 
def detectCars(filename):
    rectangles = []
    cascade = cv2.CascadeClassifier('cars.xml')
    vc = cv2.VideoCapture(filename)
 
    if vc.isOpened():
        rval , frame = vc.read()
    else:
        rval = False
 
    roi = [0,0,0,0]
    frameCount = 0
 
    while rval:
        rval, frame = vc.read()
        frameHeight, frameWidth, fdepth = frame.shape 
 
        newRegions = detectRegionsOfInterest(frame, cascade)
        for region in newRegions:
            if isNewRoi(region[0],region[1],region[2],region[3],rectangles):
                rectangles.append(region)
 
        for r in rectangles:
            cv2.rectangle(frame,(r[0],r[1]),(r[0]+r[2],r[1]+r[3]),(0,0,255),3) 
 
        frameCount = frameCount + 1
        if frameCount > 30: 
            frameCount = 0
            rectangles = []
 
        # show result
        cv2.imshow("Result",frame)
        cv2.waitKey(1);
    vc.release()
 
detectCars('road.avi')

Final notes
The cascades are not  rotation invariant, scale and translation invariant. In addition, Detecting vehicles with haar cascades may work reasonably well, but there is gain with other algorithms (salient points).

You may like:

 
Download Code + Video + Cascade file

40 thoughts on “Car tracking with cascades

  1. Frank - May 31, 2017

    try with both Python 2 and Python 3. Can you display the data beforehand?

  2. Sébastien Turrel - May 28, 2017

    Frank did you create the xml yourself ?
    my openCv training on my Debian serv always stop after stage 0 🙁
    btw, when i run your last script with Roi the error is :

        frame = cv2.resize(frame, (frameWidth / scaleDown, frameHeight / scaleDown))
    TypeError: integer argument expected, got float

    can you help me with that my friend ?

    1. Frank - May 29, 2017

      I don’t recall, has been a while. Do you have video capture? Try to show the verify these variables contain the right values using debugger or print function. It mentions a type error, cast frameWidth/scaleDown and the other one to integers using int().

      1. Sébastien Turrel - May 29, 2017

        this code is very interresting my friend.
        when i do :
        frame = cv2.resize(frame, (int(frameWidth / scaleDown), int(frameHeight / scaleDown)))
        frameHeight, frameWidth, fdepth = frame.shape
        it s say :
        left = img[0:height, 0:half]
        TypeError: slice indices must be integers or None or have an __index__ method
        is it possible to you to write it again or something ? i am just discovering haar cascade and it’s very fascinating.

      2. Sébastien Turrel - June 6, 2017

        Still here my friend ? 🙂

        1. Frank - June 9, 2017

          Hi, has been a few busy days. I’ve no problem running the code from the zip file.
          I’m using Python 2.7.13 on Manjaro Linux.
          Did you download the zip file? It has an attached video and haar file.

          1. Sébastien Turrel - June 11, 2017

            yeah i did it, but i am running Python 3.5

  3. Saeed Iqbal - May 15, 2017

    Dear Sir,

    I am facing the below error:
    Unable to stop the stream: Inappropriate ioctl for device

    1. Frank - May 15, 2017

      Hi Saeed, are you using a webcam or video file?

    2. Sébastien Turrel - June 9, 2017

      i did it my friend 😉 maybe it’s just because i am using Python 3.5 😉

  4. Volodymyr Rykhva - March 19, 2017

    hi:) Thanks for posting this tut.It helps me to understand a lot. But I got a problem with improved version of your car detection script. The basic script works well (emm…I meant it shows red rectangle when a car is in focus) and your solution doesn’t show this red frame. What could be the issue here? Thanks

    1. Frank - March 20, 2017

      Is it the same video? The array rectangles is probably empty, inside the loop of detectCars(), check using

      print(len(rectangles))
      1. Volodymyr Rykhva - March 26, 2017

        yeap, it’s actually empty for some reasons:(

        1. Frank - March 27, 2017

          try changing the method isNewRoi, that decides if its added to to the list. The values are set to 40, which is just set by a test. Try changing the values

  5. Priyanka Kochhar - March 19, 2017

    Another interesting approach is to use OpenCV HOG + SVM classifier to detect vehicles. Check out this blog:
    https://medium.com/@priya.dwivedi/automatic-vehicle-detection-for-self-driving-cars-8d98c086b161#.6x2k4szf5

  6. Thodsapon Hunsanon - March 10, 2017
    Traceback (most recent call last):
      File "C:\Users\ExCITE-LAPTOP\Desktop\Car Tracking\carTracking\detect.py", line 113, in 
        detectCars('road.avi')
      File "C:\Users\ExCITE-LAPTOP\Desktop\Car Tracking\carTracking\detect.py", line 93, in detectCars
        frameHeight, frameWidth, fdepth = frame.shape
    AttributeError: 'NoneType' object has no attribute 'shape'

    How can I resolve this error? Thanks in Advance

    1. Frank - March 11, 2017

      Do you have the files? (road.avi, cars.xml)

  7. vinod - August 11, 2016

    i tried to run this code , its not showing anything , i mean its not showing output , i don’t have any errors

    1. Frank - August 11, 2016

      Did you download the video + cascade file? Which Python version did you use? Do you have cv2 installed?

  8. victor - July 7, 2016

    Nice. Very good and excellent tutorial.
    However, it seems that the cascade can only detect the car right in front of the camera but not on the right/left track.

    1. Frank - July 15, 2016

      Correct, cascades can be trained for certain objects. You could simple download another cascade file from the web, or train yourself.

      A left/right cascade:
      https://github.com/abhi-kumar/CAR-DETECTION/blob/master/checkcas.xml

      Dataset from University of Illinois
      http://cogcomp.cs.illinois.edu/Data/Car/