Segmentation and feature extraction. Contours and blob detection.

Dec 20, 2010 by damiles

Posted under: OpenCV, Tutorials

In BasiOCR tutorial i explain how to preprocess, extract features and clasify a handwritten number, and a lot of people ask me how to segment an image where contains several numbers or objects.

In this tutorial I want explain how to segment an image and detect each object inside image, in this tutorial we can detect objects from plate (the numbers) or each object draw into a paper.

We can define three steps to do this task: preprocess image, find countours, and calculate bounding rect. With this three steps we have each object separatly and then we can use each object to classify it with basicocr sample.

For preprocessing task we use:

Smooth filter to simple noise elimination.
Threshold image to get binary image
Morphologic filter, erode and dilate to eliminate noise.

Once we have preprocessed our input image we look for the contours in our binary image with cvFindContours, and optimize its with approxpoly.


//Search countours in preprocesed image
 cvFindContours( img_contornos, storage, &contour, sizeof(CvContour),
 CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE, cvPoint(0, 0) );
 //Optimize contours, reduce points
 contourLow=cvApproxPoly(contour, sizeof(CvContour), storage,CV_POLY_APPROX_DP,1,1);

Then for each searched contour we calculate the bounding rect.


//For each contour found
 for( ; contourLow != 0; contourLow = contourLow->h_next )
 {

...

CvRect rect;

rect=cvBoundingRect(contourLow, NULL);

We can now draw his contour and/or bounding rect.


pt1.x = rect.x;
 pt2.x = (rect.x+rect.width);
 pt1.y = rect.y;
 pt2.y = (rect.y+rect.height);
 cvRectangle(imagen_color, pt1,pt2, color, 1, 8, 0);
//or
cvDrawContours( imagen_color, contourLow, color, color, -1, 0, 8, cvPoint(0,0) );

Now we have each object differenciate, now we can use this to get characteristics to classificate, for example the 7 invariant hu moments.


CvMoments moments;
CvHuMoments humoments;

//First calculate object moments
 cvMoments(contourLow, &moments, 0);
 //Now calculate hu moments
 cvGetHuMoments(&moments, &humoments);

Download source

31 Comments + Add Comment

piponazo December 20, 2010 at 5:51 pm

Very interesting topic .
damiles December 20, 2010 at 6:00 pm

I know piponazo you have a good opencv background, and this tutorial does not give you anything new. But a lot of people don’t know that with countours don’t need a specific function for blob detection.

Remember that there is a specific function for blob detection for image sequence (videos) that it’s more robust and omptimised to videos.
kayce December 28, 2010 at 12:29 am

Hi Damiles,

Good tutorial regarding the segmentation! I was wondering how you would segment the characters if they were touching each other. For example, non-segmented handwriting. Thanks and keep up the good work.

- Kayce

P.S Merry Christmas and happy new year!
damiles December 28, 2010 at 11:07 am

Hi Kayce, this tutorial only can be applied to handwritten separate characters, when two characters are touched this tutorial can be applied. For touched handwritten characters segmentation is more complex, and it’s no easy task. The segments depend if is italic handwritten, normal handwritten etc… but anyone can start to create the horizontal histogram of image characters and segment in low values of histogram.

See: http://blog.damiles.com/resources/text.jpg
kayce December 30, 2010 at 5:03 am

Thanks for the quick response Damiles. Can you give any insight at how to construct the histogram please? Thanks.

-Kayce
kayce January 2, 2011 at 6:14 pm

Hey Damiles,

Nevermind, I figured out the histogram thing! Once again, thank you for suggesting this approach. It helped a lot!

-Kayce
Nadeeshani January 5, 2011 at 2:43 am

Hello. Thx for your tutorial. It’s great. I have a question. How can we use those selected features for training?
meenu January 7, 2011 at 2:44 pm

thnx for your valuable input
Jira January 19, 2011 at 11:38 am

Very Good tutorial. thank you
jokk February 9, 2011 at 5:55 pm

Hi, this is a nice tutorial. I’m working on OCR project for graduation project. I’ll try to follow this tutorial. Hope I can make some good work
rick February 22, 2011 at 4:45 pm

Hey i tried running your code in vc++ and i get errors like:
error C2275: ‘CvMemStorage’ : illegal use of this type as an expression
damiles February 22, 2011 at 5:14 pm

Please paste all error in http://pasteall.org/ and send me the link to see all errors.

Regards.
pj March 1, 2011 at 10:38 am

I tried the program with 3 images downloaded and got out of memory error. But when I gave your images it ran perfectly.Why is it so?
damiles March 1, 2011 at 10:47 am

Uops! please send me the images to test!. david at artresnet dot com
PJ March 1, 2011 at 11:18 am

Hi

Me and a couple of friends were trying to get working implementation of OCR as a project

but we’ve run into a few stumbling blocks

I was wondering if you could give us a hand as to how to combine the segmentation and feature extraction module with the basic OCR module?

awaiting your reply

thank you

regards
PJ
johnflan April 2, 2011 at 6:55 pm

Hi David,

Excellent article, but I wonder would you speak a little more about how you would classify each of the symbols. Do you store a reference contour for each character? or do you use moments?
Raveena S Pai June 30, 2011 at 11:09 am

hi,

Hi, this is a nice tutorial. am a post graduate student and this tutorial help me a lot to do am academic project..it is based on pattern recognition..i think this would be a good reference material..
manha August 9, 2011 at 8:59 pm

thank you for the good tutorial, keep up the excellent work
Paulo October 10, 2011 at 10:39 pm

Hello Damiles !

Congratulations for a very good tutorial.

Do you know how can I get this segmented characters and “transform” in a String ?
damiles October 11, 2011 at 12:55 pm

See OCR tutorial
Huu Nhan November 2, 2011 at 8:29 am

hi damiles,

Could you fix your source link, I can not download it

Thank so much !
whyzee November 3, 2011 at 10:29 pm

Thank you so much for the tutorial. Really appreciate the work….. this is a great resource for beginners.
peace November 11, 2011 at 1:13 pm

Hello , thanks for your tutorials. I am new bee for OpenCV and do you use kinect camera with opencv ?
Which OS is best for working with opencv ,kinect?
jose November 13, 2011 at 10:46 pm

I do not know to speak english but i will try. When I compile main.c of seg_tutorial i get many errors

—— Build started: Project: seg_tutorial, Configuration: Debug Win32 ——
main.c
c:\opencv2.2\include\opencv2\legacy\blobtrack.hpp(82): error C2061: syntax error : identifier ‘CvVSModule’
c:\opencv2.2\include\opencv2\legacy\blobtrack.hpp(82): error C2059: syntax error : ‘;’
.
.
.
c:\opencv2.2\include\opencv2\legacy\blobtrack.hpp(620): error C2143: syntax error : missing ‘;’ before ‘*’
c:\opencv2.2\include\opencv2\legacy\blobtrack.hpp(620): error C2059: syntax error : ‘)’
c:\opencv2.2\include\opencv2\legacy\blobtrack.hpp(620): fatal error C1003: error count exceeds 100; stopping compilation
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

Do you know how can I get resolve it ?

Very Good tutorial. thank you.
mahdi November 20, 2011 at 9:09 pm

hi
i have any problem for segmentation!
do you can help me?
damiles November 22, 2011 at 10:09 am

What do you need? Please send me email via contact with your cuestion
damiles November 22, 2011 at 10:12 am

Hola Jose, it seems you are compiling with c and no c++, wich compiler you are using?
damiles November 22, 2011 at 10:13 am

I never work with kinect, but it’s microsoft device then sure there are a lot of support for windows, I remember there are support for linux too then select the os you are more confortable.
vebby December 4, 2011 at 10:18 am

Hi damiles,
Good tutorial regarding the segmentation! I
can you share me your source code transform blob detection to string??
help me,please.
thank you
andyvebby December 5, 2011 at 3:46 am

hi, damiles..
Thank you so much for the tutorial.
how can i transforming character from blob detection to string?
help me, please
thanks
febin February 9, 2012 at 8:11 am

hi damiles,
i did the character segmentation using above algorithm…
thnx for the tutorial…
is there some way to save the segmented characters as different images…
so that i can use it for neural network training