Powered by Blogger.

Monday, December 20, 2010

Tag: ,

Segmentation and feature extraction. Contours and blob detection.

In BasiOCR tutorial i explain  how to preprocess, extract features and clasify a handwritten number, and a lot of people ask me how to segment an image where contains several numbers or objects.

In this tutorial I want explain how to segment an image and detect each object inside image, in this tutorial we can detect objects from plate (the numbers) or each object draw into a paper.



We can define three steps to do this task: preprocess image, find countours, and calculate bounding rect. With this three steps we have each object separatly and then we can use each object to classify it with basicocr sample.

For preprocessing task we use:

  1. Smooth filter to simple noise elimination.

  2. Threshold image to get binary image

  3. Morphologic filter, erode and dilate to eliminate noise.




Once we have preprocessed our input image we look for the contours in our binary image with cvFindContours, and optimize its with approxpoly.

[c]

//Search countours in preprocesed image
cvFindContours( img_contornos, storage, &contour, sizeof(CvContour),
CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE, cvPoint(0, 0) );
//Optimize contours, reduce points
contourLow=cvApproxPoly(contour, sizeof(CvContour), storage,CV_POLY_APPROX_DP,1,1);

[/c]

Then for each searched contour we calculate the bounding rect.

[c]

//For each contour found
for( ; contourLow != 0; contourLow = contourLow->h_next )
{

...

CvRect rect;

rect=cvBoundingRect(contourLow, NULL);

[/c]

We can now draw his contour and/or bounding rect.

[c]

pt1.x = rect.x;
pt2.x = (rect.x+rect.width);
pt1.y = rect.y;
pt2.y = (rect.y+rect.height);
cvRectangle(imagen_color, pt1,pt2, color, 1, 8, 0);
//or
cvDrawContours( imagen_color, contourLow, color, color, -1, 0, 8, cvPoint(0,0) );

[/c]


Now we have each object differenciate, now we can use this to get characteristics to classificate, for example  the 7 invariant hu moments.

[c]

CvMoments moments;
CvHuMoments humoments;

//First calculate object moments
cvMoments(contourLow, &moments, 0);
//Now calculate hu moments
cvGetHuMoments(&moments, &humoments);

[/c]

Download source from GitHub

About David Millán Escrivá

David completed his studies in Universidad Politecnica de Valencia in IT with a Master's degree in artificial intelligence, computer graphics, and pattern recognition, focusing on pattern recognition and Computer Vision. David has more than 15 years of experience in IT, with more than ten years of experience in Computer Vision, computer graphics, and pattern recognition, working on different projects and start-ups, applying his knowledge of Computer Vision, optical character recognition, and augmented reality. Co-Author of two OpenCV books and reviewer of few more.

74 comments:

  1. I know piponazo you have a good opencv background, and this tutorial does not give you anything new. But a lot of people don't know that with countours don't need a specific function for blob detection.

    Remember that there is a specific function for blob detection for image sequence (videos) that it's more robust and omptimised to videos.

    ReplyDelete
  2. Hi Damiles,

    Good tutorial regarding the segmentation! I was wondering how you would segment the characters if they were touching each other. For example, non-segmented handwriting. Thanks and keep up the good work.

    - Kayce

    P.S Merry Christmas and happy new year!

    ReplyDelete
  3. Hi Kayce, this tutorial only can be applied to handwritten separate characters, when two characters are touched this tutorial can be applied. For touched handwritten characters segmentation is more complex, and it's no easy task. The segments depend if is italic handwritten, normal handwritten etc... but anyone can start to create the horizontal histogram of image characters and segment in low values of histogram.

    See: http://blog.damiles.com/resources/text.jpg

    ReplyDelete
  4. Thanks for the quick response Damiles. Can you give any insight at how to construct the histogram please? Thanks.

    -Kayce

    ReplyDelete
  5. Hey Damiles,

    Nevermind, I figured out the histogram thing! Once again, thank you for suggesting this approach. It helped a lot!

    -Kayce

    ReplyDelete
  6. Hello. Thx for your tutorial. It's great. I have a question. How can we use those selected features for training?

    ReplyDelete
  7. thnx for your valuable input

    ReplyDelete
  8. Very Good tutorial. thank you

    ReplyDelete
  9. Hi, this is a nice tutorial. I'm working on OCR project for graduation project. I'll try to follow this tutorial. Hope I can make some good work :)

    ReplyDelete
  10. Hey i tried running your code in vc++ and i get errors like:
    error C2275: 'CvMemStorage' : illegal use of this type as an expression

    ReplyDelete
  11. Please paste all error in http://pasteall.org/ and send me the link to see all errors.

    Regards.

    ReplyDelete
  12. I tried the program with 3 images downloaded and got out of memory error. But when I gave your images it ran perfectly.Why is it so?

    ReplyDelete
  13. Uops! please send me the images to test!. david at artresnet dot com

    ReplyDelete
  14. Hi


    Me and a couple of friends were trying to get working implementation of OCR as a project

    but we've run into a few stumbling blocks

    I was wondering if you could give us a hand as to how to combine the segmentation and feature extraction module with the basic OCR module?

    awaiting your reply

    thank you

    regards
    PJ

    ReplyDelete
  15. Hello

    Very interesting blog !
    Actually,I am wondering how to output the plate number into a char* : 9283GKJ.
    Indeed, I would like to know can you can go from the hu.moments to a char*

    Thanks ^^

    ReplyDelete
  16. hi,
    sir i m doing my project in DIP using opencv, i have to track human motion in video... i want to capture 1st frame of every second... how can i do that?????
    regards.

    ReplyDelete
  17. Hello, I am trying to use OPEN CV for DOT CODE Recognition on tyre image. I am not sure if it needs both OCR and Blob detection to use. Please let me know how to start up

    Also please give me a hand with creating a project for image segmentation and blob detection.

    Thanks,
    Prashanthi Anjuri

    ReplyDelete
  18. Hi David,

    Excellent article, but I wonder would you speak a little more about how you would classify each of the symbols. Do you store a reference contour for each character? or do you use moments?

    ReplyDelete
  19. good post :)

    I have been reading "learning opencv -o'reilly press" form past week and have got basic idea on structure of image ,matrix..
    i know what the function does but cant figure out in which order to use them.

    what will be the strategy ??
    if i have to read the text which will be pointed by users finger using cam input

    i have done little detection but it is not so accurate

    ==>> http://www.youtube.com/watch?v=_xLwlOx-EyU

    ReplyDelete
  20. hi,

    Hi, this is a nice tutorial. am a post graduate student and this tutorial help me a lot to do am academic project..it is based on pattern recognition..i think this would be a good reference material..

    ReplyDelete
  21. hi .... thank u about this useful tutorial....
    .in my graduation project i have to detect boxes and balls and triangles in a track , i already detect all objects and take contour and draw rectangle around every object. the problem now is that my objects are 3D and i want to see if they match one of the 3 shapes or it is another object that dont match any shape......all of this with only ONE picture for each object in data base.
    so what kind of matching techniques can help me as i know that hu moments only for 2D.....but remember ONE object ONLY in the data base :) thanks in advance

    ReplyDelete
  22. hey rick your error about
    "C2275: ‘CvMemStorage’ : illegal use of this type as an expression"
    is solved by just declare all variable below at main function
    like :
    int main( int argc, char** argv )
    {
    CvMemStorage* storage = NULL ;
    CvRect rect;
    CvPoint pt1, pt2;
    CvScalar avg;
    CvScalar avgStd;

    ReplyDelete
  23. Hi,

    If there's anyone that would like to make some money and could create me a solution for this captcha (http://www.barafranca.com/im-no-cheater/) please contact me @ o3script[at]gmail.com

    Thanks

    ReplyDelete
  24. thank you for the good tutorial, keep up the excellent work

    ReplyDelete
  25. Hi I got same error of Cvmemstorage and have posted the code as per told.
    Here is the link.
    http://www.pasteall.org/24005


    I have Visual Studio 2008 with opencv 2.1.

    Hope to get reply soon.

    Deepak

    ReplyDelete
  26. what if the license plate image is tilted?

    ReplyDelete
  27. Hello Damiles !

    Congratulations for a very good tutorial.

    Do you know how can I get this segmented characters and "transform" in a String ?

    ReplyDelete
  28. hi damiles,

    Could you fix your source link, I can not download it :(

    Thank so much !

    ReplyDelete
  29. Thank you so much for the tutorial. Really appreciate the work..... this is a great resource for beginners.

    ReplyDelete
  30. Hello , thanks for your tutorials. I am new bee for OpenCV and do you use kinect camera with opencv ?
    Which OS is best for working with opencv ,kinect?

    ReplyDelete
  31. I do not know to speak english but i will try. When I compile main.c of seg_tutorial i get many errors

    ------ Build started: Project: seg_tutorial, Configuration: Debug Win32 ------
    main.c
    c:\opencv2.2\include\opencv2\legacy\blobtrack.hpp(82): error C2061: syntax error : identifier 'CvVSModule'
    c:\opencv2.2\include\opencv2\legacy\blobtrack.hpp(82): error C2059: syntax error : ';'
    .
    .
    .
    c:\opencv2.2\include\opencv2\legacy\blobtrack.hpp(620): error C2143: syntax error : missing ';' before '*'
    c:\opencv2.2\include\opencv2\legacy\blobtrack.hpp(620): error C2059: syntax error : ')'
    c:\opencv2.2\include\opencv2\legacy\blobtrack.hpp(620): fatal error C1003: error count exceeds 100; stopping compilation
    ========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========


    Do you know how can I get resolve it ?


    Very Good tutorial. thank you.

    ReplyDelete
  32. hi
    i have any problem for segmentation!
    do you can help me?

    ReplyDelete
  33. What do you need? Please send me email via contact with your cuestion

    ReplyDelete
  34. Hola Jose, it seems you are compiling with c and no c++, wich compiler you are using?

    ReplyDelete
  35. I never work with kinect, but it's microsoft device then sure there are a lot of support for windows, I remember there are support for linux too then select the os you are more confortable.

    ReplyDelete
  36. Hi damiles,
    Good tutorial regarding the segmentation! I
    can you share me your source code transform blob detection to string??
    help me,please.
    thank you

    ReplyDelete
  37. hi, damiles..
    Thank you so much for the tutorial.
    how can i transforming character from blob detection to string?
    help me, please
    thanks

    ReplyDelete
  38. hey..nice tuts uve got here..ive problem with calculating hu moments...does image size affect hu moments?...n after segmenting characters for eg:here 9 and 2 wht if i get most of hu moments zero...shud i change the datatype.....can the very less value of hu moments affect classification ..for eg:h1 for 9=0.000065,h1 for 2=0.000034.....can i separate out 9 and 2...and can i use hu moments as feature vectors for ann algo or ml?..pls help....

    ReplyDelete
  39. The image for preprocessing is missing

    ReplyDelete
  40. Do you have the code that used for the pre-processing?

    ReplyDelete
  41. hi, damiles
    i segmented the characters using the above algorithm, thanks for the tutorial.
    i have a doubt.
    is there some way to save the segmented characters as different images, so that i can use it for training the neural network.

    ReplyDelete
  42. Hi,
    Good work, thanks. I applied your work to my project which is some kind of a palm recognition project. I have to compare two palm image (the lines in our palms, i have to compare and choose the similar one to template palm). I believe i can do this with using your work. But, is there any suggestion to make my work easier.

    Thanks in advance.

    ReplyDelete
  43. hi damiles,
    i did the character segmentation using above algorithm...
    thnx for the tutorial...
    is there some way to save the segmented characters as different images...
    so that i can use it for neural network training

    ReplyDelete
  44. to febin: Hi, ROI (Region of interest) is what you looking for.

    ReplyDelete
  45. Hi damiles,

    Thanks for your great tutorial, Can you suggest some approaches for this problem,
    My problem description is like that I have 26 alphabets(A-Z and a-z) and ten digits(0-9) references and I have to compare it with the hand drawn alphabets and digits to check whether the user draws it in a correct way or not. I needed the similarity score as well. Any suggestion from your side, I have to do it in opencv,

    ReplyDelete
  46. Hi Damiles,

    I have the similar problem of context which is like I have to compare the 26 alphabets and and 0-9 hand drawn digits with that of the absolute reference and to find out how much is the user draw it correctly, The image could be rotated, scaled or skewed. Note that I have to do it with Binary Images, can you kindly suggest an algorithm for it...... I previously did some template matching using Normalized cross correlation. But Normalized cross correlation would not be so much effective in this case due to invariance in template.

    ReplyDelete
  47. [...] Segmentation and feature extraction. Contours and blob detection. 【http://blog.damiles.com/2010/12/segmentation-and-feature-extraction-contours-and-blob-detection/】 [...]

    ReplyDelete
  48. hi ,i can't download the source can you send me by mail please? it's mail mail sossarah69@gmail.com
    thank you i'm really need this toturial

    ReplyDelete
  49. The links to download code are broken all across this website. Great tutorials, any chance the links will be fixed?

    ReplyDelete
  50. Hi, I wanted to look at your code but the weblink seems not working as it gives me 404 error. Is there any change you can send me the source? Thanks in advance

    ReplyDelete
  51. Hi, can you upload the source code again?
    Can't download, the page is off
    Thanks, your blog are helping me a lot.

    ReplyDelete
  52. Hi Damiles,
    If seems your download link is broken. Any chance reposting the source?

    ReplyDelete
  53. Could you please upload the source files again ,

    Thanks

    ReplyDelete
  54. Hi Damiles,
    There is a error in downloading the source code.Can you email me the source code??
    Thnx

    ReplyDelete
  55. hi Damiles,
    This is a very good tutorial...,
    Can u please share the source code?, I am not able to download it from the link u have given above.

    ReplyDelete
  56. hello, please i need your code source !!! help

    ReplyDelete
  57. will this be applicable to italics font?

    ReplyDelete
  58. hi damiles,
    thank you!
    the source link is borken, i got a 404 :(

    ReplyDelete
  59. Sory bu stupid question but i am so new at opencv.I cant understand the codes.All codes are there. Where is main one. If ı collect them, ı can reach succesfull.

    Sorry again but i am realy confused..

    ReplyDelete
  60. please send for me code ! thank all !

    ReplyDelete
  61. Hi Damiles ,

    Thank you it's a very interesting topic ,
    Can you please send me the downloadable source code .
    regards.

    ReplyDelete
  62. this is a nice tutorial, thanks. however, I am stuck in the point after finding all contours. I was able to achieve that, but am not sure how can I extract the numbers to classify them. would you mind explain that to me. Also I am unable to download the source code

    Thanks

    ReplyDelete
  63. Hi , could you fix project source code link..
    thk

    ReplyDelete
  64. hi damiles, i need your help, can you teach me?, how can i use Humoments, i learn to use cvBoundingRect, and it´s help me a lot, but now i want to use humoments.

    ReplyDelete
  65. Hey, you used to write fantastic, but the last several posts have been kinda boring?I miss your tremendous writings. Past few posts are just a little bit out of track! come on!

    ReplyDelete
  66. Hello Damiles,
    I'm a new learner OpenCV, in your source code demo, have'nt the extraction character from Image,
    Could you please send to me a source code demo for extraction character

    Thank Damiles

    ReplyDelete
  67. nice topic n really greate from damiles ,Thanks

    ReplyDelete
  68. Hi,
    Your blog is so good.
    How to construct a Classifier for character recognition?
    Say SVM or neural networks for example.

    ReplyDelete
  69. can you explain it in c# :)

    ReplyDelete
  70. I need to know if anyone had done this on android opencv sdk.

    ReplyDelete