Powered by Blogger.

Friday, November 14, 2008

Tag: , , , , , , ,

The basic patter recognition and classification with openCV

In this tutorial we go to introduce to the pattern recognitions basics in openCV.

In pattern recognitions before we can classificate an element we need train our system. We go to train with 3 class with 100 samples each one.

Then we create 2 matrix trainData and traninClasses with 300 samples:
int train_sample_count = 300;
CvRNG rng_state = cvRNG(-1);
CvMat* trainData = cvCreateMat( train_sample_count, 2, CV_32FC1 );
CvMat* trainClasses = cvCreateMat( train_sample_count, 1, CV_32FC1 );

Where our 100 first data are class 1, the range 100 to 200 are class 2 and last 100 data are class 3, then we set our trainClasses matrix data:
cvGetRows( trainClasses, &trainClasses1, 0, 100 );
cvSet( &trainClasses1, cvScalar(1) );

cvGetRows( trainClasses, &trainClasses2, 100, 200 );
cvSet( &trainClasses2, cvScalar(2) );

cvGetRows( trainClasses, &trainClasses3, 200, 300 );
cvSet( &trainClasses3, cvScalar(3) );

cvGetRows function set a CvMat structure (matrix structure) with selected rows of source matrix.
cvGetRows( srcMatrix, destMatrix, first_row, last_row );

And set all rows to dest class witn cvSet, this function set all values of matrix to a scalar
cvSet( matrix, cv_scalar_value );

We go to use in this tutorial a random normalized sample in 2D (2D gaussian mixture)  that we generate in openCV with cvRandArr function. For create data sample we need create to cvRandArr, one for x values and other for y values.
//Train sample class 1
cvGetRows( trainData, &trainData1, 0, 100 );
cvGetCol( &trainData1, &colData1x, 0);
cvGetCol( &trainData1, &colData1y, 1);
cvRandArr( &rng_state, &colData1x, CV_RAND_NORMAL, cvScalar(200), cvScalar(50) );
cvRandArr( &rng_state, &colData1y, CV_RAND_NORMAL, cvScalar(200), cvScalar(50) );
//Train sample class 2
cvGetRows( trainData, &trainData2, 100, 200 );
cvGetCol( &trainData2, &colData2x, 0);
cvGetCol( &trainData2, &colData2y, 1);
cvRandArr( &rng_state, &colData2x, CV_RAND_NORMAL, cvScalar(300), cvScalar(50) );
cvRandArr( &rng_state, &colData2y, CV_RAND_NORMAL, cvScalar(300), cvScalar(50) );
//Train sample class 3
cvGetRows( trainData, &trainData3, 200, 300 );
cvGetCol( &trainData3, &colData3x, 0);
cvGetCol( &trainData3, &colData3y, 1);
cvRandArr( &rng_state, &colData3x, CV_RAND_NORMAL, cvScalar(100), cvScalar(30) );
cvRandArr( &rng_state, &colData3y, CV_RAND_NORMAL, cvScalar(400), cvScalar(30) );

Now we can train our classifier. In this basic tutorial, we go to use the k nearest neighbour classifier.

This classifier is the most simplest but it's used in more cases. This algorithm classify an object by a majority vote of its nearest neighbors.

Then we use CvKNearest openCV class
CvKNearest knn( trainData, trainClasses, 0, false, K );

Where first parameter is train data, second the classes, and last parameter is the nomber of k-neighbour value with maximum value is 32.

Now we have in knn instance a k-nn classifier. Now if we want know in wich class is classified point only need call knn.find_nearest and return the nearest class.

Then we go to classify a values for range [0,0] to [500,500] this our classifier, this is the result:

Red points: Train data of class 1.
Green points: Train data of class 2.
Blue points: Train data of class 3.

Red area: classify as class 1 with more than 5 neighbours in class 1
Light red area: classify as class 1 with less than 5 neighbours in class 1
Green area: classify as class 2 with more than 5 neighbours in class 2
Light green area: classify as class 2 with less than 5 neighbours in class 2
Blue area: classify as class 3 with more than 5 neighbours in class 3
Light blue area: classify as class 2 with less than 5 neighbours in class 2
Orange area: don't classify



Demo source

About David Millán Escrivá

David completed his studies in Universidad Politecnica de Valencia in IT with a Master's degree in artificial intelligence, computer graphics, and pattern recognition, focusing on pattern recognition and Computer Vision. David has more than 15 years of experience in IT, with more than ten years of experience in Computer Vision, computer graphics, and pattern recognition, working on different projects and start-ups, applying his knowledge of Computer Vision, optical character recognition, and augmented reality. Co-Author of two OpenCV books and reviewer of few more.

84 comments:

  1. [...] do it, we go to use all we learn in before tutorials, we go to use a simple basic painter and the basic pattern recognition and classification with openCV [...]

    ReplyDelete
  2. Good tutorial, we definetely need to explore more in depth the machine learning part inside openCV ;)

    ReplyDelete
  3. thank you for your good explaination

    im new at these,
    and im trying to understand how it works at opencv about image processing (in universal)
    i heard about pre-processing and processing on image processing.
    and im confuse about that

    ReplyDelete
  4. Hi nico.
    Thanks for your comment.

    Imagine you have a set of images where sizes are distint, then you want all images with same width and height, this is the pre-processing part, when you adapt your images.

    Next you must get some features of images, for example, pixels, or a image histogram, and then you analyze your image with this features (pixels, histogram, ...) this is the processing part, when you work with data you need, a lot of times the image pixels no are the data you work.

    ReplyDelete
  5. This tutorial about ocr recognition can be a good introduction to image calsify: http://blog.damiles.com/?p=93

    Regards. David.

    ReplyDelete
  6. Thank you i saw the code i understood the over all concept behind the classification. But i am not able to get the declarataion for CvKNearest and the find_nearest. I got the rest of the code and have written my own version of it . Currently i am using kmeans insted of KNN classifier. If you can sort that out i will be done with my a major part of my project.

    ReplyDelete
  7. it's simply, with CvKNearest knn(...) you initialize the CvKnearest class with knn variable and with train data, and with knn.find_nearest(...); you can classify a data. knn.find_nearest returns the class the data is classified, and in nearest you have a "k" neighbour values the returned value the data is classified is the more neighbours you have.
    Example if in nearest you have this values with k=6 {3,3,2,3,6,3} you data is classified in class 3, 1 and 6 but you have the class resutl 3 because you have 4/k more proxim of class 3, 1/k of class 2 and 1/k of class 6 then the class 3 is have more neighbours than others.

    ReplyDelete
  8. Sir,
    I understood the program. But when i tried to implement the same in my code i am being showed an error. I am working on visual studio express edition.

    ReplyDelete
  9. Please, send me the error...

    ReplyDelete
  10. Sir,
    It is working now. But the trainclasses are all the same.
    I can mail the code. It is some what customized. Can you give me your mailid.

    ReplyDelete
  11. Sir,
    I sent the code. Please go through them.

    ReplyDelete
  12. Sir,
    Is there a facility in opencv to append a matrix to another.

    ReplyDelete
  13. Sir,
    In opencv
    for image resizing we have cvPyrDown but its uses interpolation wile resizing. I need to get only the samples with out interpolation. Like in matalb
    sub=im(1:interval:r,1:interval:c);

    ReplyDelete
  14. Sir,
    I solved teh resizing with out interpolation.

    IplImage * subsample(IplImage * input, unsigned int intval_r, unsigned int intval_c) {
    CvMat *subrows=cvCreateMat(input->width/intval_r,input->height,cvGetElemType(input));

    //cout<<"One"<height, intval_r);
    //cvShowImage("main",&subrows);
    //printf("st:%d:\n",input->depth);
    //cout<<"two"<width, subrows->height,cvGetElemType(input));
    cvTranspose(subrows, temp);

    //cout<<"three"<height/intval_r),(temp->width/intval_c),cvGetElemType(input));
    cvGetRows(temp, out, 1, temp->height, intval_c);
    CvMat * out2=cvCreateMat(out->width,out->height,cvGetElemType(input));
    cvTranspose(out, out2);

    //cout<<"four"<height;
    sz.width=out2->width;

    IplImage *dst=cvCreateImage(sz,input->depth,input->nChannels);
    cvGetImage(out2,dst);
    //dst=&hdr;
    SImage src_dst(&dst);
    src_dst.display("SAMPLED");
    cvWaitKey();
    return dst;

    }

    ReplyDelete
  15. Can you help me with the Knn classifier problem...?

    ReplyDelete
  16. I understood the program now. Thank you. But o dont know the value of trainClasses. You set the values to 1,2 and 3. I dont have that information about the pixels in the image. Can you suggest me any mehtod to follow now

    ReplyDelete
  17. First. What do you want classify? Faces, pixels, cars, objects, characters, ...?
    Second, what properties of objects do you want use?, Pixels, contours...

    ReplyDelete
  18. Please, you must verify that trainData is a floating matrix data.

    ReplyDelete
  19. i can't understand that. how can i verify??
    and dado1, dado2 and dado3 are matrix of float (float **dado1,...)
    i'm new with opencv, this matrix structure isn't that simple for me.
    thanks

    ReplyDelete
  20. Please post the "trainData" variable definition.

    ReplyDelete
  21. I have posted some parts of my code at this link:
    http://codepad.org/uI24XAg7
    thanks

    ReplyDelete
  22. I see the code and i think it's correctly, you sure need debug your app line by line.

    ReplyDelete
  23. I am working on retinal images where i have to identify exudates based on pixel properties. A collection of pixel information is stored and then classification is done.
    The collection is like this
    G values
    sum of squares of R and G
    in a window of 3x3 centered on a pixel
    max value in that window
    min value in that window
    mean value in that window
    entropy of that window

    These are the pixel values used for classification.
    G - green channel
    R - Red channel

    ReplyDelete
  24. ok, i will try to do it,
    i'm using VC++ how can i see the data of a CvMat structure??
    i saw only some information, but i can't find the real data.
    thank for all.

    ReplyDelete
  25. I use linux and gdb or ddd, i can't help you in this. Sorry.

    ReplyDelete
  26. hi Damiles, Thank you i got the problem in my program. ITs working now.

    ReplyDelete
  27. hi Damiles, it is nice post. but, can you share your pattern samples ?

    ReplyDelete
  28. In this tutorial andol, the pattern samples are random numbers i generate in the code the class 1 samples are:

    cvRandArr( &rng_state, &colData1x, CV_RAND_NORMAL, cvScalar(200), cvScalar(50) );
    cvRandArr( &rng_state, &colData1y, CV_RAND_NORMAL, cvScalar(200), cvScalar(50) );

    are random numbers with normal distribution that position are 200, 200 with a width,height of 50.

    ReplyDelete
  29. Thanks for your contribution. I'm stating to research in the Action Recognition research area, and to start with easy examples of classification like this is very useful ;).

    ReplyDelete
  30. Hi damiles,

    I am doing a research on the facial expression detection for my university and really stuck with it. what i try to do is matching images from the webcam with some labelled images and then decide the emotion. Can you give me some suggestion about it? Which algorithm is the most suitable? And with which algorithm can i expand the database (labelled images) for the future use ?

    I am only a 2 nd student so be mercy if these are silly questions :D. thank you very much

    ReplyDelete
  31. hi bubuzzz, i don't know most suitable algorithm, you must look for paper about it.

    You only need know all patter recognition is:

    1. Processing image
    2. Extract image features
    3. Normalize features
    4. Compare the normalized features with your database of features and classificate it.
    5. If you can classificate get user input for classificate and add it to database if you consider to add.

    Regards david.

    ReplyDelete
  32. great.....................:)

    ReplyDelete
  33. hi,

    I want classify the star shape from image using knn. can you please tell me the how to get the traindata and trainclass information from image.

    thanks in advance
    Manivasakan

    ReplyDelete
  34. if you go to classify only the star then you have only one class, for traindata you can get N images of stars and cvMoments to get the statistical data.

    ReplyDelete
  35. thank you very much for reply.

    How to assign trainclass values from source image and traindata values from star image?

    can you give some code sample?

    Thanks
    Manivasakan.

    ReplyDelete
  36. Hi Damiles,
    Its a nice post you have in here. It allowed be to have a start with processing with videos. I was looking for the procedure to create a Video Frame with random numbers.Can u help?

    ReplyDelete
  37. Hi,

    I need recognize the red objects from webcam stream

    what can i do ? like algorithm classes or comments for it ?

    ReplyDelete
  38. if you need diference the red objects is very simply, only need chek an histogram of red channel pixels, and do threshold to get as white a objects with red or no.

    It's simply, get webcam image, for each pixel if red component of pixel > threshold then this is a red object pixel.

    Later you can give a 8-connected algorithm to detect diferent red objects in image.

    ReplyDelete
  39. if that is like simple can you give me code for it ? it will be very usefull for me (: thx for fast reply !

    ReplyDelete
  40. Burak, this is your task, i can't create your work. Instead, i go to create a tutorial with demo code to this tasks.

    Please Wait 1 day please.

    ReplyDelete
  41. that will be enough for me .

    i check lots of website blog or email grp but none of them answering anything . thx again i will w8

    ReplyDelete
  42. Hi Damiles,
    I am new with OpenCV. I would like to recognize some kind of shape like rounded rectangle, circle,trapezium,... Could you please tell me how I can do it with OpenCV?Thanks.

    ReplyDelete
  43. Hi KhaNguyen, i read your email, but i can't respond... For this task try to retrive the contours and get a aprox poly of its, and with moments and angles detect the object Type.

    ReplyDelete
  44. If lots of train data, the KNearest constructer will takes longer time to train the class for each time when the program start to run. Is it possible to save the trained result into a file, when the program start to run each time, it just read the train result from the saved file and applied to KNearest object ithout trainning to save the start up time?

    ReplyDelete
  45. Yes of course you can save this data into a file to load in other sessions... To do this there are opencv functions.

    ReplyDelete
  46. Hi Damiles,

    Thanks for your reply. So, if i train the system like basicOCR for shape object. Is it possible to do that?
    Thanks.

    ReplyDelete
  47. Hi KhaNguyen, of course, you can train and classify with any vector you want, you can train and clasify for example with a vector with data area, perimeter and radius or any value you want.

    ReplyDelete
  48. Hi,I like your blog, I am interested in OpenCv for machine learning and I tried your example code. But I found linker problem with the ml function. I am using Borland Builder c++ and OpenCb version 1.1. Someone have some suggestion for that problem ?
    Thanks.

    ReplyDelete
  49. hi, if you have link problem, you must check you have the libraries set in your configuration project. Check how is set in VC++ (http://opencv.willowgarage.com/wiki/VisualC%2B%2B)

    Regards

    ReplyDelete
  50. Hello, I need used opencv with netbeans of java for industrial parts detection by color and form.
    Thanks for your help.
    Sorry, i'm not speak english well

    ReplyDelete
  51. Hi, you need port or wrapper for use opencv in java or only need compile your C/C++ projects with Netbeans.

    There are many opencv ports and wrappers you can use. Look for in google and sure you can find one to use.

    ReplyDelete
  52. Hey nice post, I am looking for stuff more that the 7-8 basic classifiers implemented in ml.h. Do you know a good machine learning library in C++ containing implementations of ideas like graphical models?

    ReplyDelete
  53. Hi if you look for Machine learning into google you can find MCL++ from sgi, it can be good, and there are weka too, weka is writted in java with graphical frontend. Weka can be a good for tests and more.

    Regards David.

    ReplyDelete
  54. Hello, I am new to opencv and have been trying to compile you code using Visual Basic 2010 on Windows7. I have downloaded opencv-2.1.0 and build it using CMake ver2.8. I included all the directories but i am clueless what to do next. Because when i try to Debug main it gives me some 40 errors.
    I am making a project on Handwriting and Shape recognition from graphic pen input to make UML diagrams. It would be great help if you guide me how to run these opencv projects.

    Regards
    Sank

    ReplyDelete
  55. Hello,

    I've been reading through your blog and source codes and I must say that I'm impressed. I'm currently working on a handwritting OCR and I was wondering if how you would approach that issue and extend your example to "letters of the alphabet". I have an idea how to do it by using the k-nearest neighbour algorithm (i.e. change the ). It would just involves creating more training set and introducing more classes and the appropriate data mapping. But I would like your insight on how you would approach it.

    Btw, I was wondering if there's any specific reason why you used PBM format for the training and not other format such as PNG, JPG? Does openCV not support those format? My apology for those silly questions but I'm new to openCV.

    Thanks,

    kayce

    ReplyDelete
  56. Hi, Can you please show us an example on how to link extracted features for training?

    ReplyDelete
  57. Hi Damiles,
    Thank you for your basicOCR code. I compiled it but always get run-time errors when loading the train data at cvLoadImage. It doesn't recognize even the first file in "..\OCR00.pbm".
    Do you have any clue what went wrong? Thank you in advance.

    ReplyDelete
  58. please ensure you have the images in the folder that app is looking for.

    ReplyDelete
  59. Hi damiles,

    Thank you! Yes, you are right. The executable VS creates is in a debug folder and of course could not find the train dats in "..\OCR"

    Brady

    ReplyDelete
  60. Only wanna input on few general things, The website style is perfect, the written content is very superb : D.

    ReplyDelete
  61. Hi, do you have any code for indoor/outdoor image recognition? Or it is possible to you this code for my problem? Than you very much!

    ReplyDelete
  62. Hey.

    Anyone had any luck with regression values from the machine learning library?

    Particularly the SVM, which does both classification and regression values.

    I have an example code snippet.....


    int main( int argc, char** argv )
    {
    bool train_update = false;
    int train_sample_count = 100, train_sample_size = 10;
    CvMat *data_mat, *class_mat;
    class_mat = cvCreateMat(train_sample_count, 1, CV_32FC1);
    data_mat= cvCreateMat(train_sample_count, train_sample_size, CV_32FC1);
    for(i = 0; i data.fl[ i] = value;//assign the class or regression value
    for(j = 0; j data.fl[i*train_sample_size+ j] = val;//assing data sample value
    }
    }
    CvSVMParams svm_param;
    svm_param.svm_type = CvSVM::NU_SVR;
    svm_param.kernel_type = CvSVM::RBF;
    svm_param.gamma = 1./train_sample_size;
    svm_param.nu = 0.5;
    svm_param.C = 8;
    svm_param.term_crit.epsilon = 0.001;
    svm_param.term_crit.max_iter = 50;
    svm_param.term_crit.type = CV_TERMCRIT_ITER|CV_TERMCRIT_EPS;
    CvSVM *tSvm = new CvSVM();
    tSvm->train(data_mat, class_mat, 0, 0, svm_param);
    tSvm->save("filename.xml", 0);

    //For prediction
    CvSVM *pSvm = new CvSVM();
    pSvm->load("filename.xml", 0);
    CvMat *sample = cvCreateMat(1, train_sample_size, CV_32FC1);
    for(i = 0; i data.fl[i] = val ;//assign value
    }
    float pred = pSvm->predict(sample);
    printf("\n %.3f", pred);
    }


    However variables val and value aren't mentioned before useage. Any ideas? Regression classification is defined in many sources (O'Reilly etc.) but it isn't clear how to go about applying it.

    Variable instances ORDERED or CATEGORICAL. Could we start a dialog on this?

    Dan

    ReplyDelete
  63. I don't test svm, but i try to test your code and i respond you.

    Regards David.

    ReplyDelete
  64. No sorry, i never have a similar project. sorry.

    ReplyDelete
  65. hi
    thanks a lot for this perfect tutorial!
    i'm working in a project which consist of recognizing arabic text using opencv.
    Do you think that i can use the same method to recognize letter?and why you work wit PMB ?can I work with jpg o png?is ther anny difference??(i m new in opencv)
    thnks a lot :))

    ReplyDelete
  66. hi Kayce please i'm working in the same project as you you said that you have an idea how to recognize character can you help me please??

    ReplyDelete
  67. Hi maryem, you can use the same tutorial and method, and you can use jpg or png instead pbm, there are no differences.

    Regards.

    ReplyDelete
  68. muchas gracias damiles :)) you're the best :))

    ReplyDelete
  69. Hello, i'm a newbie in pattern recognition.

    I'm working on a program to recognize text numbers, and I've been reading the tutorial you've made ​​on the recognition of numbers, which recommend that we read this tutorial.

    I'm trying to understand, but is costing me a lot. Could you explain a little what it does and how it relates to pattern recognition?

    Thanks

    ReplyDelete
  70. thanks for your blog ,i am just a beginner in opencv!

    ReplyDelete
  71. Cool dude!
    I'm curious about something more?
    How about letter detection, word detection, and phrase detection?

    If you have clue, please send a clue of code to
    my email: sky.henry [ at ] live.com

    I'm totally appreciate that Dude :)

    ReplyDelete
  72. This is no a esasy task, there are two techniques, top down and down top, this is, top down is first detect text blocks, and for each text block detect lines, words and letters, and down top is detect each letter and cluster each letter into words, lines and texts

    Look foor into http://scholar.google.es/ text block detection. there are a lot of info.

    ReplyDelete
  73. hi,
    It was a great tutorial.
    Can you please guide me how to use classification for the features extracted from a image instead of just points as you have used here.

    ReplyDelete
  74. [...] looks interesting: Basic OCR in OpenCV, using a K-nearest-neighbor algorithm for [...]

    ReplyDelete
  75. Hi the link to the source for this tutorial doesn't seem to work anymore, is there a more uptodate link or can anyone share the demo code? Thanks, Paul

    ReplyDelete
  76. Hi
    damiles , the downloaded link not work

    ReplyDelete
  77. Hello.. that is usefull..!
    I can't download it..
    Please check it again...!!

    ReplyDelete
  78. The title says "patter recognition" but I think you wanted to write "pattern" with an *N*.

    ReplyDelete
  79. Hey am trying to find the region of floor in the image , can u help me out with some stuff

    ReplyDelete
  80. Hey the link is down?

    ReplyDelete
  81. Superbe post : je pense en discuter dans la journée avec des voisins

    ReplyDelete
  82. Very good, and inspiration , I have massive respect for you.
    Thank you very much indeed, this help me lot to understand machine learing, etc.

    just wondering if the demon code is still available for me to have a

    ReplyDelete
  83. Hi... where can I get the demo source?

    ReplyDelete