Damiles

OpenCV More with cameras.

In last tutorial we learn how to get frames of our camera, now we go to learn some basic OpenCV functions to work with our captured frames or video. There is nothing special, but it’s the first steps to introduce to work with OpenCV

Imagine we want create a basic OpenCV program that get a video o camera capture and process the images to get a binary image difference between actual frame and last frame.

Then we have a actual image, lastImage, diffImage where we store the difference images, and the bitImage result that are IplImage instance.

IplImage* image;
IplImage* lastImage;
IplImage* diffImage;
IplImage* bitImage;

Whe need create these variables and initialize with cvCreateImage, this function create and allocate in memory the structure and headers of our image, this function need as attribute image size, depth and number of channels. We get frame as reference to initialize the variables.

image=cvCreateImage(cvSize(frame->width,frame->height),frame->depth,1);

The frame whe get sure is a RGB image, but we go to working with gray level images the we need conver captured frame to Gray Level. The cvCvtColor conver color spaces of image to another and is defined:

void cvCvtColor( const CvArr* src, CvArr* dst, int code );

src is the source 8-bit (8u), 16-bit (16u) or single-precision floating-point (32f) image. dst is the destination image of the same data type as the source one. The number of channels may be different. And code is color conversion operation that can be specified using CV_<src_color_space>2<dst_color_space> constants

cvCvtColor(frame, image, CV_BGR2GRAY);

We can get a new copy in memory of image withcvCloneImage

lastImage=cvCloneImage(image);

To get the image differences we use cvAbsDiff function, to get absolute diference between images:

cvAbsDiff(image,lastImage,diffImage);

The function definition

void cvAbsDiff( const CvArr* src1, const CvArr* src2, CvArr* dst );

Where src1 is the first source array. src2 is the second source array, and dst is the destination array.

Then to finish we only need get a threshold of image, and we do it with cvThreshold function

cvThreshold(diffImage,bitImage,tr,255,CV_THRESH_BINARY);

And this function have this definition:

double cvThreshold( const CvArr* src, CvArr* dst, double threshold,
double max_value, int threshold_type );

Where src is source array (single-channel, 8-bit of 32-bit floating point), dst is destination array; must be either the same type as src or 8-bit, threshold is the threshold value, max_value is the maximum value to use with CV_THRESH_BINARY and CV_THRESH_BINARY_INV thresholding types, and threshold_type is thresholding type

This is the main loop program:

for(;;){
IplImage* frame = 0;
int c;

frame = cvQueryFrame( capture );
if( !frame )
break;
//If is the first frame
if(!image){
//Create image header same as frame but with 1 channel to gray
image=cvCreateImage(cvSize(frame->width,frame->height),frame->depth,1);
bitImage=cvCreateImage(cvSize(frame->width,frame->height),frame->depth,1);
}
//Convert frame to gray and store in image
cvCvtColor(frame, image,CV_BGR2GRAY);

//If is the first frame
if(!lastImage){

//If no lastImage clone actual image;
lastImage=cvCloneImage(image);
}
if(!diffImage){
//Create image header same as frame but with 1 channel to gray
diffImage=cvCreateImage(cvSize(frame->width,frame->height),frame->depth,1);
}

cvShowImage( “CamSub”, frame );
//Differences with actual and last image
cvAbsDiff(image,lastImage,diffImage);
//threshold image
cvThreshold(diffImage,bitImage,tr,255,CV_THRESH_BINARY);
//Change datas;
lastImage=cvCloneImage(image);

cvShowImage(“CamSub 1″,bitImage);

c = cvWaitKey(10);
if( (char) c == 27 )
break;
}

Demo Source

65 Comments to “OpenCV More with cameras.”

Reut 20 January 2009 at 2:52 pm #

hi ,
i copy the code and compile it .
during runtime i get an error at line
“cvCvtColor(frame, image, CV_BGR2GRAY);”

mybe it because cvCvtColor is accept CvArr and not
IplImage?

how do i convert between thus two structure ?
IplImage -> CvArr

Thanks,
Reut.
damiles 20 January 2009 at 2:59 pm #

HI Reut, What is the error you have, when you pass a IplImage to a CvArray Function hi convert automatically between IplImage to CvArray.

You copy my code or download the demo source?

Post your error in pastebin and i can see it.
Alejandro Valenzuela 23 February 2009 at 7:24 pm #

Hi! I could get your code running ok (my camera would whine on reporting FPS so I commented that).

I’m under the impression that there is a slight memory leak — Add “cvReleaseImage(&lastImage);” between //Change datas;
and
lastImage=cvCloneImage(image);

Like this

//Change datas;
cvReleaseImage(&lastImage);
lastImage=cvCloneImage(image);

Thanks a lot for your tutorial, it’s given me quite a kickstart on another project
damiles 23 February 2009 at 7:27 pm #

Thanks for advice, i go to test and change source code.
Ravi Krishnan 8 May 2009 at 1:47 pm #

I have two images of the same size. They are obtained like this:
img1 = cvLoadImage( … );
and
img2 as cvResize( img_src, img2, … );

when I tried
cvNorm( img2, img1, CV_L2, NULL );

I am getting an Error;
” OpenCV error: Assertion failed (a.type()==b.type() && a.size()==b.size()) in norm, …. ”

both have the same size.

Can you help me…
damiles 8 May 2009 at 1:51 pm #

You sure get this error because the img1 and img2 are not same type, this is, no are same depth, or sure, one is a 8 unsigned char and other is a 32 float data. See the img1 and 2 creation.
Ravi Krishnan 9 May 2009 at 4:32 pm #

Thank you.. but, I didnt understand you fully. I see that both are IplImage*. But, were are these ‘8 unsigned char’ and ‘32 float’ set in the creation of the images?

If so, is there any way to make these two images the same type? Some convertion operation or so…

Do I have to first create the images using cvCreateImage?

both images are binary images.

Thank you
Ravi Krishnan 11 May 2009 at 6:11 am #

Ok. I solved the problem. thanks a lot.

I created img1 as
img1 = cvCreateImage( ‘FileName’, CV_LOAD_IMAGE_GRAYSCALE );

Now img1 and img2 are of the same type and I can use cvNorm on them.

rku
shiva 24 June 2009 at 9:19 pm #

can u pls help me….i want to store the frames which i m capturing in one loop….the problem is that in cvSaveImage function, every time the new frame is saved in the same filename(overwriting occurs which is obvious)….in short..i am unable to store frames in different files such as frame1, frame2, frame3, and so on for subsequent frames…

thanks in advance…..
Juan Pablo 25 June 2009 at 4:03 am #

shiva, try something like this:

int index = 0;

while (…){
frame = cvQueryFrame( capture );
stringstream ss(“image”);
ss<<index++<<”.bmp”;
cvSaveImage(ss.str().c_str(),frame);

…
}

Thanks Damiels for your posts!
damiles 25 June 2009 at 7:37 am #

Thanks Juan Pablo for respond the question
Sodrohu 9 July 2009 at 5:19 am #

I tried Juan Pablo’s algorithm, but cannot get it to work. The errors are:

error C3861: ’ss’: identifier not found
error C2228: left of ‘.str’ must have class/struct/union error C2228: left of ‘.c_str’ must have class/struct/union
error C2228: left of ‘.bmp”’ must have class/struct/union
error C2146: syntax error : missing ‘;’ before identifier ’ss’
error C2065: ’stringstream’ : undeclared identifier
error C2065: ‘”image”’ : undeclared identifier c
error C2065: ‘”’ : undeclared identifier

can anyone identify the problems?
damiles 9 July 2009 at 6:15 am #

are you compiling with g++ or gcc?
Instead you can use this option.

int index = 0;
char file[255];

while (…){
frame = cvQueryFrame( capture );
i++;
sprintf(file,”image_%d.jpg”,i);
cvSaveImage(file,frame);

…
}
Sodrohu 9 July 2009 at 6:16 am #

Maybe I’m better off telling everyone what I’m trying to do. I’m using OpenCV to do motion detection by getting the video’s images, process it to get its threshold image and then do motion history. Problem is, I’m new to this and I don’t quite understand how to grab two images from the video (previous and next). I’ve tried the codes here but they don’t seem to work.
damiles 9 July 2009 at 7:34 am #

Next? If you have the actual frame, you can’t get the next frame, you can’t get the future.

But if you want get a mhi, you must get the t-2, t-1 ant t frames, where t is the actual frame. Then you only need get this three images

Then you need 3 variables
IplImage *f2, *f1, *f;

and in your code
initialize getting 3 images
f2=cvQueryFrame(capture); //t-2
f1=cvQueryFrame(capture);//t-1
f=cvQueryFrame(capture);//Actual frame
And in the loop you only need actualice images and your mhi, each new frame, because new t
while(..){
cvCopy(f1, f2,0);//Copy frame 1 to frame 2
cvCopy(f,f1,0);
f=cvQueryFrame(capture);
//create your mhi with f,f1 and f2
}
Sodrohu 10 July 2009 at 3:59 am #

After going through your algorithm, I finally understand how the flow goes..I tried it on my program, but the frame grabbing doesn’t work.

The flow is supposed to be like:
No loop: f2-A, f1-A,f-A
1st loop:f2-A, f1-A,f-B
2nd loop:f2-A, f1-B,f-C
3rd loop:f2-B, f1-C,f-D….and so on…

But displaying the pics after each loop, I found out that it was actually:
No loop: f2-A, f1-A,f-A
1st loop:f2-B, f1-B,f-B
2nd loop:f2-C, f1-C,f-C

anyway the whole f, f1 and f2 just grab the same frame each time, so the thresholding doesn’t work.

BTW I’m using Microsoft Visual Studio 2005 VC++. The new each-frame saving algorithm works on my program so thanks for that.
damiles 10 July 2009 at 8:11 am #

Please ensure this order of calls:

cvCopy(f1, f2,0);//Copy frame 1 to frame 2
cvCopy(f,f1,0);
f=cvQueryFrame(capture);
Alessandro 5 August 2009 at 10:13 pm #

Hi,
I’m wanna do the same but with this little program:

#include “stdafx.h”
#include “highgui.h”
#include “cv.h”

int main( int argc, char** argv )
{
CvCapture* capture = cvCreateCameraCapture(0);
IplImage* frame;
IplImage* framePast;
IplImage* frameDiff;
cvNamedWindow( “video”, CV_WINDOW_AUTOSIZE );
framePast = cvQueryFrame( capture );
frameDiff = cvQueryFrame( capture );
char c = cvWaitKey( 40 );
while (1)
{
frame = cvQueryFrame( capture );
if( !frame ) break;
cvAbsDiff( frame, framePast, frameDiff );
char c = cvWaitKey( 40 );
framePast = frame;
cvShowImage( “video”, frameDiff );
c = cvWaitKey( 40 );
if( c == 27 ) break
}
cvReleaseCapture( &capture );
cvDestroyWindow( “video” );
return 0;
}

Anyway, what I get, is a black window… WHY?

Regards
Alessandro
damiles 6 August 2009 at 12:39 pm #

Alessandro you have a black image because you are doing the image difference with the same data, because when you do framePast=frame you assign the memory block of frame to framePast then when you do frame=cvQueryFrane(capture) you are getting same data in framePast

You must use cvCloneImage for get only data.

Regards David.
Alessandro 6 August 2009 at 2:14 pm #

Thanks for you suggestion, but in the “while {..}” the second time after a “cvWaitKey(40)”, a “frame = cvQueryFrame( capture );” is performed again takeing another frame, so the cvAbsDiff should has two different frames to subtract…
damiles 6 August 2009 at 3:24 pm #

Yes but you are accesing to area image block, and if you read the opencv doc about cvqueryframe you can not modify or release this returned image…

You must do a cvClone image.

You can test easly the memoryblocks with a debugger…

Regards.
Alessandro 6 August 2009 at 9:26 pm #

Yeah, I tried and it works properly, I was just curious about that.
Look, I have searched for that, but the online guide installed with the library are not so exhaustive.

Regards
damiles 7 August 2009 at 1:29 pm #

Alessandro, all you need of doc are here: http://opencv.willowgarage.com/documentation/index.html

This is all we need of opencv documentation.

Regards.
Madhav 5 September 2009 at 9:59 pm #

i’m tryin to import facedetector code as DLL into labview but image format in labview n openCV r different hence i’m trying code like below

void facedet(CvArr* img1,double scale,CvRect* r[])
//img passed in CvArr
{

IplImage* img=(IplImage*) img1;

//further code ……

}

i dont think just typecasting will convert CvArr to IplImage please help to convert CvArr to IplImage
divya 4 November 2009 at 7:17 am #

how to convert the cvRect in to cvArray
larun 13 November 2009 at 3:38 pm #

hi all

can any one help me how to convert a AVI video into a no.of frames.

thanks a million

regards,
damiles 13 November 2009 at 6:01 pm #

Use ffmpeg.
larun 1 December 2009 at 7:28 pm #

Thanks a million for ur replay damiles,
its working…
larun 1 December 2009 at 7:33 pm #

hi damiles,

i am presently using pyramid Lucas kanade for tracking a car in a video. My code is working but output video is in reversed .
can you help me plz.
larun 5 January 2010 at 1:46 pm #

hi damiles, happy new year,

Blow is my code in opencv with optical flow to track a car in video but my output is coming in reversed.so, could you plz identify where is the mistake in this code,

#include
#include
#include
#include
#include
static const double pi = 3.14159265358979323846;
inline static double square(int a)
{
return a * a;
}

inline static void allocateOnDemand( IplImage **img, CvSize size, int depth, int channels)
{
if ( *img != NULL ) return;
*img = cvCreateImage( size, depth, channels );
if ( *img == NULL )
{
fprintf(stderr, “Error: Couldn’t allocate image. Out of memory?\n”);
exit(-1);
}
}
int main(void)
{
/* Create an object that decodes the input video stream. */
CvCapture *input_video = cvCaptureFromFile(“Night.avi”);
if (input_video == NULL)
{
fprintf(stderr, “Error: Can’t open video.\n”);
return -1;
}

can anyone identify the problems?

cvQueryFrame( input_video );
/* Read the video’s frame size out of the AVI. */
CvSize frame_size;
frame_size.height =
(int) cvGetCaptureProperty( input_video, CV_CAP_PROP_FRAME_HEIGHT );
frame_size.width =
(int) cvGetCaptureProperty( input_video, CV_CAP_PROP_FRAME_WIDTH );
long number_of_frames;
cvSetCaptureProperty( input_video, CV_CAP_PROP_POS_AVI_RATIO, 1. );
number_of_frames = (int) cvGetCaptureProperty( input_video, CV_CAP_PROP_POS_FRAMES );
/* Return to the beginning */
cvSetCaptureProperty( input_video, CV_CAP_PROP_POS_FRAMES, 0. );
cvNamedWindow(“Optical Flow”, CV_WINDOW_AUTOSIZE);
long current_frame = 0;
while(true)
{
static IplImage *frame = NULL, *frame1 = NULL, *frame1_1C = NULL, *frame2_1C =
NULL, *eig_image = NULL, *temp_image = NULL, *pyramid1 = NULL, *pyramid2 = NULL;
cvSetCaptureProperty( input_video, CV_CAP_PROP_POS_FRAMES, current_frame );

frame = cvQueryFrame( input_video );
if (frame == NULL)
{
fprintf(stderr, “Error: Hmm. The end came sooner than we thought.\n”);
return -1;
}
allocateOnDemand( &frame1_1C, frame_size, IPL_DEPTH_8U, 1 );
cvConvertImage(frame, frame1_1C, CV_CVTIMG_FLIP);
allocateOnDemand( &frame1, frame_size, IPL_DEPTH_8U, 3 );
cvConvertImage(frame, frame1, CV_CVTIMG_FLIP);
frame = cvQueryFrame( input_video );
if (frame == NULL)
{
fprintf(stderr, “Error: Hmm. The end came sooner than we thought.\n”);
return -1;
}
allocateOnDemand( &frame2_1C, frame_size, IPL_DEPTH_8U, 1 );
cvConvertImage(frame, frame2_1C, CV_CVTIMG_FLIP);
allocateOnDemand( &eig_image, frame_size, IPL_DEPTH_32F, 1 );
allocateOnDemand( &temp_image, frame_size, IPL_DEPTH_32F, 1 );

CvPoint2D32f frame1_features[400];
int number_of_features;
number_of_features = 400;
cvGoodFeaturesToTrack(frame1_1C, eig_image, temp_image, frame1_features, &
number_of_features, .01, .01, NULL);

CvPoint2D32f frame2_features[400];
char optical_flow_found_feature[400];
float optical_flow_feature_error[400];

CvSize optical_flow_window = cvSize(3,3);
CvTermCriteria optical_flow_termination_criteria
= cvTermCriteria( CV_TERMCRIT_ITER | CV_TERMCRIT_EPS, 20, .3 );
allocateOnDemand( &pyramid1, frame_size, IPL_DEPTH_8U, 1 );
allocateOnDemand( &pyramid2, frame_size, IPL_DEPTH_8U, 1 );
cvCalcOpticalFlowPyrLK(frame1_1C, frame2_1C, pyramid1, pyramid2, frame1_features,
frame2_features, number_of_features, optical_flow_window, 5,
optical_flow_found_feature, optical_flow_feature_error,
optical_flow_termination_criteria, 0 );
for(int i = 0; i < number_of_features; i++)
{
/* If Pyramidal Lucas Kanade didn’t really find the feature, skip it. */
if ( optical_flow_found_feature[i] == 0 ) continue;
int line_thickness; line_thickness = 1;
CvScalar line_color; line_color = CV_RGB(255,0,0);
CvPoint p,q;
p.x = (int) frame1_features[i].x;
p.y = (int) frame1_features[i].y;
q.x = (int) frame2_features[i].x;
q.y = (int) frame2_features[i].y;
double angle; angle = atan2( (double) p.y – q.y, (double) p.x – q.x );
double hypotenuse; hypotenuse = sqrt( square(p.y – q.y) + square(p.x – q.x) );

q.x = (int) (p.x – 3 * hypotenuse * cos(angle));
q.y = (int) (p.y – 3 * hypotenuse * sin(angle));
cvLine( frame1, p, q, line_color, line_thickness, CV_AA, 0 );

p.x = (int) (q.x + 9 * cos(angle + pi / 4));
p.y = (int) (q.y + 9 * sin(angle + pi / 4));
cvLine( frame1, p, q, line_color, line_thickness, CV_AA, 0 );
p.x = (int) (q.x + 9 * cos(angle – pi / 4));
p.y = (int) (q.y + 9 * sin(angle – pi / 4));
cvLine( frame1, p, q, line_color, line_thickness, CV_AA, 0 );
}
cvShowImage(“Optical Flow”, frame1);
int key_pressed;
key_pressed = cvWaitKey(0);
if (key_pressed == ‘b’ || key_pressed == ‘B’) current_frame–;
else current_frame++;
/* Don’t run past the front/end of the AVI. */
if (current_frame = number_of_frames – 1) current_frame = number_of_frames – 2;
}
}

can anyone identify the problems?
thanks a million,
damiles 5 January 2010 at 2:13 pm #

I read a comment of other user that said this about this line
cvConvertImage(frame, frame2_1C, CV_CVTIMG_FLIP);

Convert whatever the AVI image format is into OpenCV’s preferred format. AND flip the image vertically. Flip is a shameless hack. OpenCV reads in AVIs upside-down by default.

Test to set the CV_CVTIMG_FLIP param in 0
cvConvertImage(frame, frame2_1C, 0);

or You can use cvFlip to put correctly if you want http://opencv.willowgarage.com/documentation/operations_on_arrays.html?highlight=cvflip#cvFlip

Regards.
larun 5 January 2010 at 5:54 pm #

its working,thanks
damiles 5 January 2010 at 5:55 pm #

larun what you use finally?
larun 7 January 2010 at 4:24 pm #

damiles

i change all set of CV_CVTIMG_FLIP param in 0
cvConvertImage(frame, frame2_1C, 0);
larun 7 January 2010 at 4:28 pm #

Damiles,

could u plz tell me which is best way in open cv to track a car in night video
1.by color
2.headlight
3.number plate

thanks ,
Best Regrads,
larun
damiles 7 January 2010 at 4:32 pm #

try to use a infrared camera…
larun 7 January 2010 at 4:44 pm #

ya i read about infrared camera but it manufacturer of thermal imaging which is very expensive project.so,i want to track a car in normal camera video
larun 13 January 2010 at 1:53 pm #

hi damiles,

i don’t got any replay for you.
do you have any code for tracking a car by color in open cv.

Thanking You,
Best Regards,
larun
damiles 13 January 2010 at 2:17 pm #

Sorry Larun for respond too later.

I don’t have any code for you but you can check this link about videosurvillance http://opencv.willowgarage.com/wiki/VideoSurveillance with this you can detect non background objects, then you can classify it’s by color, type or similar.
Guillermo 15 January 2010 at 4:04 pm #

Hi everybody
I need to use the CvSaveImage for save an image in a folder different from the project one, because I need toshow it in a website using RoR.
How can I save the image in another folder using Opencv

Thanks
larun 18 January 2010 at 4:24 pm #

thanks damiles,

is there any possible to track a specific car in a video by PLK in opencv/vc++.

thanks a million for ue help.
it is very usefull to my project damiles,

Best Regards,
larun
damiles 18 January 2010 at 4:28 pm #

Hi larun, what do you mean with PLK, sorry my english is very bad and i’m spanish. I do not know all the abbreviations in English.
larun 18 January 2010 at 4:51 pm #

sorry damiles,

PLK method Pyramidal Lucas Kanade
damiles 18 January 2010 at 5:04 pm #

larun, i’m not a computer vision expert, i’m sure you know more about computer vision than me. With piramid lucas kanade plk you can obtain the optical flow of image, i’m sure you can track a car or all objects of image.

How you can do it?. I don’t know. I can imagine how to do, but sure there are better methods to do it, i recommend you read a lot of scientific paper, get the better document and do it.
larun 20 January 2010 at 12:26 pm #

Thank a million damiles
melissa 21 January 2010 at 3:30 pm #

hi damiles,

i’ve got the same problem as larun. I need to track vehicles and calculate their speed. I also read some scientific papers and got this solution to double difference the images first then use Camshift to track. i’ll try it out. thank you so much for this. this is very helpful.
larun 22 January 2010 at 3:03 pm #

hi melissa,

nice to meet you,our both project are little bit similar.
actually what is double difference the image.i didn’t get you properly beca just now i started my project.so,i don’t know much in opencv
could pls help me.

GOOD LUCK

Thanking You advance.
Best Regards,
larun
melissa 24 January 2010 at 8:58 am #

you should get a copy of Learning OpenCV by Gary Bradski and Adrian Kaehler. Double Differencing is like comparing 2 images in pixel level, the output is usually in binarized form, the white pixels are the difference between the two images. well, that’s how i understood it.
praveen 16 March 2010 at 9:32 am #

Hi,i have a small doubt regarding loading images.

myimage = cvLoadImage(“a1.jpg”,-1);

instead of this can we load images by first asking th image name and then open the corresponding image.
damiles 16 March 2010 at 10:08 am #

Yes you can, you can do it with command line or anyway you think.