Object detection with OpenCV

An interesting usage of the classification algorithms is object detection in live images. In this tutorial we will demonstrate how to detect a simple object using the open source library OpenCV. After a short description of OpenCV, we will see the steps needed to produce a model file using the OpenCV Cascade Classifier. This step is also called training the model. The model file will be then used to detect an object in the live images taken from a USB web camera. The model creation and the object detection code were tested on a laptop running Ubuntu 14.04 LTS. Basic C/C++ programming knowledge is required for this tutorial. A USB web camera is also needed.

OpenCV Library

OpenCV (Open Source Computer Vision) is a open source library which contains function for real-time image processing. The library is written in C++ and distributed under BSD License. It can be run under Linux, Windows, OS X, iOS, Android and has wrappers for Python, Java, C# and Ruby. It can be used for: object detection, motion detection, augmented reality, facial recognition or gesture recognition.

For this tutorial you will need to install OpenCV onUbuntu: OpenCV install on Ubuntu.

Trainig the model

In this section we will create a XML model file, which will be later used to detect the objects. For no special reason I choose a brezel (engl. prezel) as detection subject. The model creation process involves gathering a large amount of images containing the subject – we will call these positive images. We will also need a large a large amount of images not containing at all the searched object- negative images. Using the two folders and some description files we will run the OpenCV Cascade Classifier algorithm in order to produce the model file.

These are actual steps:

Step 1: I started by taking 201 JPG images of the brezel using a mobile phone.

Step 2: I installed GIMP for Ubuntu then I added David’s Batch Processor to the installation using the command:

Step 3:  I created a batch process to convert the original images to grayscale and to resize them to 60 x 80 pixels. These 201 images will be used in the next steps to produce additional training images

Step 4: I took the grayscale images and applied the following transformations:upside-down (+201 images), blur 3.0 (+201 images), brightness +0.5 (+201 images),  brightness -0.5 (+201 images), contrast +0.5 (+201 images), contrast -0.5 (+201 images). I ended having 1407 positive images.

Step 5: Created a folder named positives and moved all the images to this folder.

Step 6: Created the positives.txt file. This is a descriptor file for the positives folder. Each line in this file contains the path to a positive image, the number of objects (brezels) in this image, the coordinates of the bounding rectangle containing the object.

Step 7: Downloaded the negative image files from this tutorial. Moved all negative images in a folder called negatives.

Step 8: Created the negatives.txt file. This is a descriptor file for the negatives folder.

Step 9: Prepare a training dataset from 1200 positive images,  using the following command:

Step 10: Train the model using this commands

Depending on the hardware the training can last a few hours. On my computer I got the model after 3:41 hours:

The resulting model file can be found found in the data folder and is named cascade.xml. I took this file and renamed it to brezel.xml. As a reference you can have a look at my trained model available in this archive brezel.zip.

More on training the model can be found here: http://docs.opencv.org/doc/user_guide/ug_traincascade.html

Using the trained model to detect objects

In this section we will see how to create a simple C++ Project, add the needed libraries to the project, add the model and do the actual object detection. As development tool I used Eclipse Luna (4.4.0) with CDT (C/C++ Development Tools).

Following, are the steps needed to setup and run the project:

Step1: Start Eclipse and create a new project: File > New >  Other > C++ Project.

opencv_1_new_project

Step2: Name the project BrezelDetectionProject, the press  Finish.

opencv_2_project_name

Step3: Right click on the project and choose to create a new folder named src.

opencv_3_src_folder_creation

Step 4: Right click the src folder and create a new .cpp file, BrezelDetector.cpp.

opencv_4_source_file_creation

Step 5: Add the following code to the .cpp file.

Step 6: The project does not know where to find the OpenCV libraries. To fix that go to Project > Properties. In the C/C++ Build section, go to Settings, in the right pane select Tool Settings. In the GCC C++ Compiler section go to Includes and add to Include paths (-l) the folder where OpenCV is installed. For me the the path is /usr/include/opencv.

opencv_5_include_opencv_path

Step 7: Go to GCC C++ Linker section,  Library search path (-L) and add here the path to the OpenCV libaries. For me the path is /usr/lib/x86_64-linux-gnu/.

Under in Libraries(-l) add the following libraries one by one: opencv_core, opencv_objdetect, opencv_highgui.

opencv_6_linker_libraries

Step 8: Add your xml model file to the src folder. The final project structure should look like this:

opencv_7_project_structure

Step 9: Attach an USB web camera and run the program. The program can be stopped using the ESC key.

You can see results in the following short video:

Conclusion

We saw how we can use OpenCV to train a model for object detection and how to use this model to detect objects in live images. This is just a small use case of this computer vision library. Generally speaking, the computer vision has a wide application field and can handle areas like

  • Controlling processes industrial robots;
  • Navigation – autonomous vehicle or mobile robots;
  • Detecting events for visual surveillance or people counting;
  • Organizing information – indexing databases of images and image sequences;
  • Modelling objects or environments – medical image analysis or topographical modelling;
  • Interaction – input to a device for computer-human interaction
  • Automatic inspection – in manufacturing applications.

Interested to find out the latest in the computer vision field? You may be interested also in the following article: http://technobium.com/computer-vision-next-generation/

References

http://docs.opencv.org/modules/objdetect/doc/cascade_classification.html

http://docs.opencv.org/doc/user_guide/ug_traincascade.html

http://docs.opencv.org/doc/tutorials/introduction/linux_eclipse/linux_eclipse.html

http://note.sonots.com/SciSoftware/haartraining.html

 

 

22 Comments

Leave a Reply to James Cancel reply

Your email address will not be published. Required fields are marked *