Powered by Blogger.

Saturday, January 16, 2010

Tag: , , , , , , , ,

Segmentation & object detection by color.

In this tutorial i go to explain how to image segmentation or detect objects byred color, in this case by red color.

This task is simple, but there are some things we must known.

Now i go to explain and get a demo code for segmentation, how to determine if each image pixel is red or no, and then, i go to explain how we  can detect object, it's similar but with diferent concept.

In first moment, we can decide get the red channel of image  and get the values, if it's a higher value then we have a red pixel otherwise no.

This is incorrect. Why?, imagine we have a pixel and in his red channel have a 255 value, this is the higher value, then we decide this is a red pixel, but if in green and blue channel have 255 value too then we have a white pixel instead a red pixel. Then how we determine if a pixel is red or no?

Ok, we go to get some color values and determine how we decide if is red or no.



Then we can decide that higher red value, and  lower values of blue and green can be defined as red color. Moreover the green and blue values don't have a higher difference between his values.

We go to construct simple graph with some values to see better.



We see in this graph that the red value is higher than a max value and there are a min value for green and blue.

The min and max we go to name as blue green threshold and red threshold (bg_threshold, r_threshold).

This code compute for each pixel if it's red or no, and create a black/white image, this is our segmentation image, and with this image we can use to label each object, we use a cvFindContours to retreive the contours of each object detected on our image.

[c language="++"]
for(y=0;yheight; y++)
{
red=((uchar*)(img->imageData + img->widthStep*y))[x*3+2];
green=((uchar*)(img->imageData + img->widthStep*y))[x*3+1];
blue=((uchar*)(img->imageData + img->widthStep*y))[x*3];
uchar* temp_ptr=&((uchar*)(img_result_threshold->imageData + img_result_threshold->widthStep*y))[x];

if((red>threshold)&&(green < BG_threshold) && (blue < BG_threshold) && (abs(green-blue)< BG_diff)){
temp_ptr[0]=255;//White to greater of threshold
}else{
temp_ptr[0]=0;//Black other
}
}
}

cvMorphologyEx(img_result_threshold, img_morph, img_temp, NULL, CV_MOP_CLOSE, 6);

cvNamedWindow( "Threshold", 1 );
cvShowImage( "Threshold", img_morph );

cvFindContours( img_morph, storage, &contour, sizeof(CvContour), CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE, cvPoint(0,0) );

for( ; contour != 0; contour = contour->h_next )
{
CvScalar color = CV_RGB( rand()&255, rand()&255, rand()&255 );
cvDrawContours( img, contour, color, color, -1, 3, 8, cvPoint(0,0) );
}
[/c]

And this is the result, whe can see in the image a color border with detected red objects, the system detect two objects. The notebook and the notebook corner

Demo Source

This is perfect to image segmentation by color, but imagine we have objects ofone image, and we want get each object and know if this object have a predominant red color or no.

The theory is same but applied to the object histogram, and use the median to determine the thresholds



But with this we don't have enough, because we can get a gray image as we can see in the second histogram, then we need use another parameter more, the dispersion of each channel.



We can see that in this first histogram we can ensure this object have a predominant red, and in the second histogram no.

About David Millán Escrivá

David completed his studies in Universidad Politecnica de Valencia in IT with a Master's degree in artificial intelligence, computer graphics, and pattern recognition, focusing on pattern recognition and Computer Vision. David has more than 15 years of experience in IT, with more than ten years of experience in Computer Vision, computer graphics, and pattern recognition, working on different projects and start-ups, applying his knowledge of Computer Vision, optical character recognition, and augmented reality. Co-Author of two OpenCV books and reviewer of few more.

65 comments:

  1. I think that it's better convert the image to Lab space color and calculate the color distance.

    ReplyDelete
  2. Hi aclis, of course, there are better color segmentation. I want explain the more simply color segmentation in this tutorial. There are better color segmentation as this paper and another papers.

    The purpose of my blog is not explain the better solution, i want create basic approximations to introduce people to OpenCV and his functions, and basics of Computer vision.

    Regards David.

    ReplyDelete
  3. Hi,
    Thanks for your documentation, but i can not use the given code , can you send me as full version of it ? or put it somewhere in your blog?

    ReplyDelete
  4. Noctis there are a demo source in the post with a full version, under the image capture.

    ReplyDelete
  5. act,i need full version of the code. so pl. if possible, send me the code on my mail.i will be very thankful to u.........

    ReplyDelete
  6. awinash, there are a demo with full version under image capture.

    ReplyDelete
  7. First of all this is one of a great tutorial for newbee on open cv, Thanks alot. I would like to know can we use the same segmentation on human skin? I have read on many places hsv is best to provide skin color.?

    ReplyDelete
  8. Thanks, of course, HSV is better for color tasks.

    ReplyDelete
  9. it means i just need to convert the rgb to hsv and also i need to have skin colors store in shape of threshold like you did for red color? OR do i need more steps for it? Once again thanks for your reply.

    ReplyDelete
  10. You can start with basic threshold and basic task, but SURE you need more steps!

    ReplyDelete
  11. I tried something which is written in some online document that i have to do these following things.

    RGB to HSV color space
    Then HSV 's H should be between 6 and 38 range.

    Then make all other things as black while pixels between that range should be 255 so that result would come black and white.

    Then use filtering like dilate ,erode and smooth gaussian. But it doesnt work... can you give me any tip what i am doing mistake in it?


    IplImage* img = cvLoadImage("1.jpg");
    cvNamedWindow("Example1", CV_WINDOW_AUTOSIZE );

    //cvErode( img, img, NULL, 1);
    //cvDilate( img, img, NULL, 1);


    IplImage* hsv = cvCreateImage( cvGetSize(img), IPL_DEPTH_8U, 3 );



    cvCvtColor( img, hsv, CV_BGR2HSV );

    for(int x=0; xheight; x++)
    {
    for(int y=0;ywidth; y++)
    {
    float H=(float)((uchar*)(hsv->imageData + hsv->widthStep*x))[y*3+2];
    float S=(float)((uchar*)(hsv->imageData + hsv->widthStep*x))[y*3+1];
    float V=(float)((uchar*)(hsv->imageData + hsv->widthStep*x))[y*3];

    if(( V>6 && V0.23 && HimageData + hsv->widthStep*x))[y*3+2] =255;
    ((uchar*)(hsv->imageData + hsv->widthStep*x))[y*3+1] =255;
    ((uchar*)(hsv->imageData + hsv->widthStep*x))[y*3] =255;
    }
    else
    {
    ((uchar*)(hsv->imageData + hsv->widthStep*x))[y*3+2] =0;
    ((uchar*)(hsv->imageData + hsv->widthStep*x))[y*3+1] =0;
    ((uchar*)(hsv->imageData + hsv->widthStep*x))[y*3] =0;
    }


    }
    }


    cvDilate( hsv, hsv, NULL, 1);
    cvErode( hsv, hsv, NULL, 1);
    cvSmooth( hsv, hsv, CV_MEDIAN, 3, 3 );



    cvShowImage("Example1", hsv );
    cvWaitKey(0);
    cvReleaseImage( &img );
    cvDestroyWindow("Example1");

    ReplyDelete
  12. I think you have the error in conditional statement. I don't see correctly this statmente, i don't understand it.

    ReplyDelete
  13. Hi Damiles,

    I am very new using OpenCv. I've compiled the source code from demo source but it still doesn't work. I am confused why the name of the picture is not written in the source code? I don't know yet what argv [0], argv [1], etc means?

    Thanks in advanced!

    ReplyDelete
  14. Hi, this is a basic concept in console applications, the argv[0] is the app name and argv[1] is the first parameter send to app argv[2] the second ...

    Then when you execute the application, under console, you call ./my_app myimage.jpg and you are passing myimage.jpg as variable stored in argv[1].

    Regards. Damiles.

    ReplyDelete
  15. wow, thanks for the fast response. Sorry for the late reply :) Actually I've compiled it and succesfully result an application file. Then I put the image in the same folder with app file. But, it still doesn't work. I compiled it using Dev-C++.

    Actually, what I have to do to make the source code function well,

    thanks so much Damiles.

    ReplyDelete
  16. hy man..thx u for ur help us :) pls if u know any thing about facial motion capture (steps,links)pls hel me ...send to my e-mail......Great thanks

    ReplyDelete
  17. You can use opencv lkdemo.c sample. This is good starting point.

    ReplyDelete
  18. Thanks a lot for this post. I am a bit confused why the 3 is added to the index for the array. Obviously it's because 3+2 is r,, 3+1 is g, etc, but how would I have known that? Can you point me to the reference?

    ReplyDelete
  19. Hi c1ee, the 3 is no added to the index, its multiplied.

    The input image is 3 chanel (r,g,b) and the data is stored in a width x height x numchannels data vector, then for index 0 we get the 0 data vector for blue, 1 data vector for green and 2 for red.
    For index 1 of image we get 3 data vector position for blue, 4 for green and ....

    The formula is [xposition * num_channels + channel_selector] where channel_selector is 0 for blue, 1 for green and 2 for red.

    Regards.

    ReplyDelete
  20. Thanks for the post.
    Hey, I would like to do the same segmentation for Yellow color, can you please tell me what changes I need to make in the source code to do that.

    ReplyDelete
  21. You can use it for yellow searching red and green colors are maxim values.

    But the example is more simple and basic, and it's an introduction to this segmentation class, you must use this but using for example hsv instead rgb color model.

    Regards.

    ReplyDelete
  22. Im working on extracting text from an image can u plz help me out in this respect

    ReplyDelete
  23. hi, I don't know if i can help you but, ask me ;)

    ReplyDelete
  24. Is there any built in function in Opencv exist to create image from a weighted matrix.

    ReplyDelete
  25. If you have a matrix with data you can represent this as image simply.

    Create image with same size of your matrix and with 1 channel to get gray image and set the matris as the image data

    ReplyDelete
  26. I have a project that is to segment a text image by projecting horizontal lines to obtain separate there after I segemte each line to have separate characters and then I make the Knowledge
    I do just binarization and I could not do the segmentation
    I need your support
    it's urgent:((

    ReplyDelete
  27. Hi I need ur help.I want to extract text and pattern from an image.I have already done preprocessing steps using Opencv like gray and binary image conversion and contour detection.
    Wht should be the next step for Text block detection??plz guide me

    ReplyDelete
  28. What is your problem exactly? you can't segment each character from your binary data?

    ReplyDelete
  29. The next step for your blck detection is use cvMinAreaRect2 or cvFitEllipse2 to get each block. then you can classificate each block independent.

    ReplyDelete
  30. thnx for ur reply .........
    but I want to know how to extact each character from a word.
    For eg LIFE
    I want to read 'L' 'I' 'F' 'E' individually.

    plz tell me the algo for this

    ReplyDelete
  31. meenu, if you have the contours, you don't need any more, if you want each box you can use cvMinAreaRect2 or cvBoundingRect for each contour you found.

    Please take me 2 days and post a new tutorial about this tasks.

    ReplyDelete
  32. Thnx for ur help.
    I m eagerly waiting for ur tutorial.

    Meanwhile if u can just elaborate on the use of cvMinAreaRect2 .As I understood abt cvMinAreaRect2 use is:
    it requires first param as const CvArr* points which will be the image coordinate or point.
    My doubt is everytime the image changes we need to change this parameter.
    is it possible for you to explain in code the use of cvMinAreaRect2 for each block independent classification.
    Thnx a lot for ur instant response.

    I really liked ur tutorials.They are really helpful

    ReplyDelete
  33. Hi,Your post is great Can you help me to find orange object ?

    ReplyDelete
  34. milad get the RGB value of orange color you want select and then modify the tutorial. Regards.

    ReplyDelete
  35. To all the over commentors. Blogs can be a lot far better to study should you can keep Your feedback basic and to the level. No-one likes to read giant comments when the concept may be conveyed utilizing a not as long remark

    ReplyDelete
  36. Sorry for my english but, what do you want said? I don't undersand...

    ReplyDelete
  37. Hi
    may i know what the formula mean?
    red = [x*3+2], green = [x*3+1], blue = [x*3];
    i don't understand why need to add 2 for red, add 1 for green and for blue it do not required add any value..

    And the formula of
    uchar* temp_ptr=&((uchar*)(img_result_threshold->imageData + img_result_threshold->widthStep*y))[x];

    May i know what the formula mean?

    ReplyDelete
  38. this formula: uchar* temp_ptr=&((uchar*)(img_result_threshold->imageData + img_result_threshold->widthStep*y))[x]
    is for get the pointer of data(pixels) vector.

    This vector is organized in BGR format, this is [B1 G1 R1 B2 R2 G2 .... Bn Gn Rn] where have mxnxnumcolors data if image is m width, n height and numcolors.
    Then if you want get the x, y pixel the use this formula, but you get the B1, if you want get the other values only need sum 1 or 2 for Green and Red respectly.

    Regards

    ReplyDelete
  39. Hi . . . i need to help in separating objects based on its color,
    say for example if i have objects of red, green and blue color in the input image, i need to separate all these objects. my input image format is RGB. images will be captured through a USB
    web-camera.

    ReplyDelete
  40. Hi Damiles,
    Great job for posting these tutorials and especially answering people - this is really appreciated by all of us.

    I want to know your approach on detecting/segmenting & classifying road markings ( speed limit, priority, arrow left,right, etc). How would you do it? What would be a good color space for segmenting this and a good classifying algorithm ?

    ReplyDelete
  41. Hi Damiles, the tutorial is great. I want that if I underline in a (real) paper, it should be stored in the electronic version. Can u help how to go about it.

    Regards

    ReplyDelete
  42. Hi

    the download link is not availbale :( Could you please fix it,

    Thanks so much ! ^ ^

    ReplyDelete
  43. Hi, ur code looks great and almost same as want i want now to start with my project work. But i'm not able to get to the 'Demo source' page as it says 'Page not found'. Can you send the source code to me through mail plz.

    ReplyDelete
  44. hi
    how can i identify each number from a string of numbers
    eg: let my string be '1235'
    i need to get the result as '1', '2', '3', '5'
    Note: i am capturing the string as image through webcam.
    so first i take a picture of the string, save as a png file. now i need to do the segmentation...
    need help ......

    ReplyDelete
  45. Please, visit this tutorial: http://blog.damiles.com/2010/12/segmentation-and-feature-extraction-contours-and-blob-detection/

    ReplyDelete
  46. thank you very much...
    can u pls help me in implementing the same in python....
    i am working on it but there r few errors like ''cvScalar.val[0]' values cannot be retrived', "sizeof(CvContour) gives TypeError: this type has no size, and so on.
    pls help me.....

    ReplyDelete
  47. http://www.pasteall.org/27434/python

    i have pasted the code i wrote in python
    can you help me with it.

    ReplyDelete
  48. Hi Damiles,
    i need to extract text from a complex image ...
    text need not be aligned ... it can be scattered...
    i need to segment those text characters alone..
    how can i do this....(the basic idea)

    ReplyDelete
  49. Hello Damiles,
    I am using OpenCV 2.1 & VS2008. In that i am able to capture an image but i can't process it in real time. By using cvSaveImage it holds image but some memory problem is showing, how do I carry out it?
    Please Reply, Thank you.

    ReplyDelete
  50. Dear Sir,
    would u please give me the c/c++ source code of a real time tennis ball identification code?i need it badly.please help.

    ReplyDelete
  51. what should be the value of threshold, BG_threshold and BG_diff??
    thanks :D

    ReplyDelete
  52. Hi Damiles,
    i have problem about it . It dont detect red color but It has output is black window . Can u help me about that ,plz

    ReplyDelete
  53. i have working on color segmentation so plz send me code for color base segmentation using k-Mean Cluster
    ii)using Histogram

    ReplyDelete
  54. What are the values of threshold, BG_threshold and BG_diff that we put in order to get what u have got above in the image "best case"? thanks a lot for this useful tutorial

    ReplyDelete
  55. Hi Celine, the BG threshold and BG_diff are very different depending object to select, and this tutorial is no the best approach for efficient Color segementation, it is only a basic concept.

    ReplyDelete
  56. Hy color is very much effected by lightning condition .
    even if you convert rgb to hsv and then segment ta color object .
    Then problem is still same. hue is change as lightning condition change .
    so better for real time application, fix the background lightning condition.
    Could any one please tell me how to fix the background lightning condition ?
    Thanks

    ReplyDelete
  57. Hy even then color is depended or effected by lightning condition.
    if you convert rgb to hsv , then hue get change if lightning condition change .
    so better to fix the background lightning condition.
    Could any one please tell me how to fix the background thanks

    ReplyDelete
  58. thanks for tutorial. could you update link of code, please. cannot download it.

    ReplyDelete
  59. hi,
    very good, but "Demo Source" link is mistake, i require it... :(
    plz, correct.

    ReplyDelete
  60. why do you use BG_diff?

    ReplyDelete
  61. why do you use the BG_diff in segmentation method?

    ReplyDelete
  62. You can send Demo source to me?
    Email: muakhoc90@gmail.com
    Thanks for share

    ReplyDelete
  63. Offtopic: why I can see only 51st comment? Where are previous 50 comments?

    ReplyDelete
  64. The article is very interesting but the source code link is broken. Could you be so nice and repair it? I will be thankful for giving me possibility to study your example code. Thanks in advance.

    ReplyDelete