TensorFlow is an awesome framework where deploy deeplearning. Actually i’m working very hard on it and exploring all benefits and powerful. Then i started to deploy the models on Google Cloud. Google cloud is an awesome cloud solution where there are multiple services for all clouds needs. One of its tools very important for Deep Learning developers is Google Cloud Machine Learning that allow train our models very cheaper in high performance computers.

Then i put my hands to work on. As a computer vision developer i create my CSV file that store all file image uris and the class for each one. And following the tutorials, first i deploy and test the algorithm locally. All works fine and in few minutes i get the results. Then i started to deploy on the cloud and i upload all my png files and the CSV file for train and test to Google Storage.

After finish to upload my 60k images for training and 10k for test from MNIST to google storage, i execute with the simple command line that google provide my tensorflow python scripts, tested locally. After wait more than 20 minutes i knew that something wrong is happening. Why locally with a laptop in few minutes i had results and in a powerful machine on google cloud was not working? Then i asked to me what could be the main issues:

1. The Google Storage is in other region of my Google ML.
2. I have some issue on my code

After check that the google storage and google ML are on same region and are correctly setup and my code is fine the i start to be crazy…. What happens? Why it’s so slow to give me some output?.

Then i start to debug… Any TensorFlow developer knows that debug on it is very complex and dificult then i thought that TensorFlow and Google can think on this and give some tool to inspect the performance of a TensorFlow code and… BANG there is a function to do it. My new friend is called ProfilerHook. And adding the small snippet of code to my code

profiler_hook = hooks.ProfilerHook(save_steps=10, output_dir=args.job_dir)
classifier.train(input_fn=model.data_train_estimator, steps=20000, hooks=[profiler_hook])

First i test locally and get the first results and explore with chrome://tracing. All seems fine. Then upload to Google Cloud ML and wait, wait… and i have the first log on my google storage from profiler… and open it side by side to my local profiler to compare… See image to see the results:

Oh my God! My local version execute the iteratorGetNext in 29ms and in google cloud is 21.103ms 20seconds!! one step batch 20 seconds. I set around 20000 steps or epochs to train, this means that in my local version tooks around 10min to get all batchs plus process the training, around 15minuts to get the result, something reasonable. But in google cloud tooks around 111hours!!!! only get the batchs…

The issue then is the access to each image to google storage, that tooks around 500ms each one to open put on memory and close…. Then, what could be the solutions? Don’t access to google storate so many times to access to data.Then we can convert our big amount of files in one single file using nunpy single file, that could be a solution or use the tfrecord and load using tfrecorddataset and avoid to use a third dependency if it’s not required.

I search on google how to convert a CSV file with filename and class to tfrecord file and finally i create my own:

import tensorflow as tf
import pandas as pd
from IPython.display import clear_output

def _bytes_feature(value):
    return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))

def _int64_feature(value):
    return tf.train.Feature(int64_list=tf.train.Int64List(value=[value]))

def create_tfrecords_from_csv(csv_file, output_file):
    tfrecords_filename = output_file
    writer = tf.python_io.TFRecordWriter(tfrecords_filename)

    csv_file_records= pd.read_csv(csv_file, header=None, names=['file', 'class'])
    total_rows= csv_file_records['class'].count()
    for index, row in csv_file_records.iterrows():
        clear_output(wait=True)
        print index, 'of', total_rows
        label= row['class']
        img= io.imread(row['file'])
        h= img.shape[0]
        w= img.shape[1]
        
        img_raw= img.tostring()
    
        example= tf.train.Example(features= tf.train.Features(feature={
            'label': _int64_feature(label),
            'height': _int64_feature(h),
            'width': _int64_feature(w),
            'image_raw': _bytes_feature(img_raw)
        }))
    
        writer.write(example.SerializeToString())

    writer.close()
    
create_tfrecords_from_csv('train.csv', 'train.tfrecords')

After execute the script locally and convert all my images dataset in one single file train.tfrecords i upload the file to google storage and execute and evaluate the performance and get the following result:

Awesome now it’s very fast, and as i expected more fast than access to thousand files locally, now it’s only 3ms every iterator, more less than the 20seconds and more less than local storage SSD that whas 29ms. It’s 10x more fast than SSD disk and more than 6000x on GS.

To show the artifacts of interpolations we scaled the images to show better how to affect the interpolations

Original image:

Nearest interpolation

Bilinear interpolation

Bicubic interpolation

Nearest pixel interpolation

This is the simplest and fastest way to calculate the value of pixel. This consists in getting the nearest pixel corresponding to the original image.

If the result of inverse function get the pixel (0.3,0) then we get the value (0,0), and if we get the pixel (0.7,0) we get the value of pixel (1,0).

This is the fastest algorithm but it makes a pixeled image.

Original:

Result:

Bilinear pixel interpolation

This method of interpolation consists in tracing a line between the low and high rounded value of returned inverse function and calculate the resultant value with the line function.

The formula:

A’(px, py) = (1-a)(1-b)A(s, i) + a(1-b)A(s, d) +(1-a)bA(r, i) + abA(r, d) 

Original:

Result:

Bicubic pixel interpolation

This method produces the better result but is the slowest algorithm. It calcules the value with a cubic curve with the 4 neighbour pixels.

The formula:

Original:

Result:

Plesk NOQUEUE: reject: RCPT from localhost[127.0.0.1]: 550 5.1.1 : Recipient address rejected: User unknown in virtual mailbox table; gmail

I search for solution and the problem is that the server try to send email, but postfix and plesk configured the mail for external email in same server, and then don't work correctly, the solution is disable each mail internal server for external domain mail servers:

/usr/local/psa/bin/domain -u mydomain.com -mail_service false

The book "Mastering OpenCV with Practical Computer Vision Projects" is now for sale from Packt Publishing with free shipping to many countries! Each chapter is a separate project containing step-by-step tutorials + full source-code using the latest C++ interface of OpenCV v2.4, written by 7 authors that are well-known among the OpenCV community for their blogs & open-source projects.

Chapters (each is a project with full source-code):
- Cartoonifier and Skin Changer for Android.
- Marker-based Augmented Reality on iPhone or iPad.
- Marker-less Augmented Reality.
- Exploring Structure from Motion using OpenCV.
- Number Plate Recognition using SVM and Neural Networks.
- Non-rigid Face Tracking.
- 3D Head Pose Estimation using AAM and POSIT.
- Face Recognition using Eigenfaces or Fisherfaces.
- Developing Fluid Wall using the Microsoft Kinect.

Authors (in alphabetical order):
- Daniel Lélis Baggio ("http://code.google.com/p/ehci/")
- Shervin Emami ("http://www.shervinemami.info/openCV.html")
- David Millán Escrivá ("http://blog.localhost/")
- Khvedchenia Ievgen ("http://computer-vision-talks.com/")
- Naureen Mahmood ("http://howdweknows.blogspot.com/")
- Jason Saragih ("http://jsaragih.org/")
- Roy Shilkrot ("http://www.morethantechnical.com/")

The book can be purchased from "http://www.packtpub.com/cool-projects-with-opencv/book" either as a physical book or as a PDF eBook.

The latest code can be downloaded from "https://github.com/MasteringOpenCV/code", including a screenshot of each project (scroll down on the front page).

Note: You should already have basic experience with OpenCV and C/C++ before reading this book, as this book does not explain the basics of OpenCV and it assumes you already have it installed. For example, you could read the "Learning OpenCV" book and/or the "OpenCV 2 Cookbook" to learn the basics of OpenCV, then read this "Mastering OpenCV" book for more advanced skills and project ideas!

I'm very happy, Gnuplot cookbook arrive to my hands. I had the pleasure to review this great book.

Any engineer  that  need make profesional plots this book help to create it.

There are basic concepts to advanced concpets as 3d plots, labeling, multiplot, latex include plots, scripting and language programing with gnuplot...

I recommend this book.

You can buy it in packpublishing

rsync -r --exclude=.svn /home/user/progname/ /home/user/progname.copy

In this tutorial I want explain how to segment an image and detect each object inside image, in this tutorial we can detect objects from plate (the numbers) or each object draw into a paper.

We can define three steps to do this task: preprocess image, find countours, and calculate bounding rect. With this three steps we have each object separatly and then we can use each object to classify it with basicocr sample.

For preprocessing task we use:

1. Smooth filter to simple noise elimination.
2. Threshold image to get binary image
3. Morphologic filter, erode and dilate to eliminate noise.

Once we have preprocessed our input image we look for the contours in our binary image with cvFindContours, and optimize its with approxpoly.

//Search countours in preprocesed image
 cvFindContours( img_contornos, storage, &contour, sizeof(CvContour),
 CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE, cvPoint(0, 0) );
 //Optimize contours, reduce points
 contourLow=cvApproxPoly(contour, sizeof(CvContour), storage,CV_POLY_APPROX_DP,1,1);

Then for each searched contour we calculate the bounding rect.

//For each contour found
 for( ; contourLow != 0; contourLow = contourLow->h_next )
 {
...
CvRect rect;
rect=cvBoundingRect(contourLow, NULL);

We can now draw his contour and/or bounding rect.

pt1.x = rect.x;
pt2.x = (rect.x+rect.width);
pt1.y = rect.y;
pt2.y = (rect.y+rect.height);
cvRectangle(imagen_color, pt1,pt2, color, 1, 8, 0);
//or
cvDrawContours( imagen_color, contourLow, color, color, -1, 0, 8, cvPoint(0,0) );

Now we have each object differenciate, now we can use this to get characteristics to classificate, for example  the 7 invariant hu moments.

CvMoments moments;
CvHuMoments humoments;

//First calculate object moments
cvMoments(contourLow, &moments, 0);
//Now calculate hu moments
cvGetHuMoments(&moments, &humoments);

Download source from GitHub

From OpenCV 2.1 version we can use CMake to create and manage our OpenCV projects.

CMake is a cross-platform and open-source build system, and it's used to control our compilation process, using a simple text files for define compilation process with independent platform and compiler.

Thanks to CMake we can create our project in our operating system as linux compile and work in it, and then use it to compile a new version in other os as windows, osx, ..., even create a visual studio project, xcode project or eclipse.

To create a basic project with 1 main.c file with cmake and opencv we must create a new file called CMakeList.txt.

In this file we go to define the compiler process and his requeriments.

First we need create a project name:

PROJECT( project_name )

We need find OpenCV libraries. OpenCV is required by our project and it's defined with REQUIRED statement:

FIND_PACKAGE( OpenCV REQUIRED )

Then we must defined our executable, his  name and the dependent files, in our example only main.c, and we go to use the project name for the executable name. This name is stored in ${PROJECT_NAME} variable.

ADD_EXECUTABLE( ${PROJECT_NAME} main.c )

To finish, we must link OpenCV to our executable, to do this only need call TARGET_LINK_LIBRARIES statement, with the project name and OpenCV library.

TARGET_LINK_LIBRARIES( ${PROJECT_NAME} {OpenCV_LIBS} )

The CMakeList.txt was defined, then we can compile our project. To do this only need create a directory where we want get our executable. Then  we call cmake with the directory where  CMakeList.txt and main.c files are saved.

When cmake finish whe can call make to compile our project under unix.

If we want create a VisualStudio project under Windows, only need open CMake Gui and select the source code and build directories and cmake detect our visualstudio version and with generate button create our VisualStudio project.

Sample Code:

PROJECT( Nombre_del_proyecto )
FIND_PACKAGE( OpenCV REQUIRED )
ADD_EXECUTABLE( ${PROJECT_NAME} main.c )
TARGET_LINK_LIBRARIES( ${PROJECT_NAME} {OpenCV_LIBS} )

In this tutorial we go to show how to save our custom structures with opencv functions.

We go to imagine we have this structure in our program.

typedef struct
{
float var1;
unsigned char var2[255];
}MyStructure;

Opencv can save data in two formats: XML or YAML. This formats are markup languages.

For write or read a file, we go to use cvOpenFileStorage function, the functions returns a CvFileStorage structure pointer. This pointer we use to read or write values into.

We can open a file in two ways: for read or write. The function to open file is cvOpenFileStorage

CvFileStorage* storage = cvOpenFileStorage( file, 0, CV_STORAGE_WRITE_TEXT );

One time we have our CvFileStorage initialized then we can save our data. To save the data we need create Nodes. XML and YAML are markup languages, and they are defined with nodes.

First we go to create one node that contains all structure data. And inside of this node we need create one node for each variable.

For create a node we need open and close this node, as we do with xml, then we use cvStartWriteStruct and cvEndWriteStruct.

cvStartWriteStruct(storage, "MyStructure", CV_NODE_MAP, NULL, cvAttrList(0,0));
...
cvEndWriteStruct(storage);

Between the start and end node we go to insert each new node with asigned variable. The variable we must save as some of this types: int, raw (array), real or string with this functions:

cvWriteInt cvWriteRawData cvWriteReal cvWriteString Then for save our structure we go to create this code:

//Open file
CvFileStorage* storage = cvOpenFileStorage( file, 0, CV_STORAGE_WRITE_TEXT );
//Create our node for structure
cvStartWriteStruct(storage, "MyStructure", CV_NODE_MAP, NULL, cvAttrList(0,0));
//Write our float
cvWriteReal( storage, "var1", MyStructure->var1 );

//Write our array
cvStartWriteStruct(storage, "var2", CV_NODE_SEQ, NULL, cvAttrList(0,0));
cvWriteRawData(storage, MyStructure->var2,255,"u");
cvEndWriteStruct(storage);

//End node structure
cvEndWriteStruct(storage);

Then if we need show a image to a person, we can color it. But, how is the best way to coloring gray image?

There are 3 ways to do it: Manually, automatically and colored by ranges.

In this tutorial, i go to develop the way most common automatic for gray image coloring.

To do it we need know we go to receive a gray level and we need return 3 values, one for red, one for blue and other for green.

We go to use this function:

$$ \displaystyle{ s(x,y)=\vert sin(r(x,y)*p*PI + \Theta*PI)\vert}$$

And r(x,y) is the gray level and p is the number of repetitions and $ \displaystyle{ \Theta }$ is the displacement.

Then we only need define the p and $ \displaystyle{ \Theta }$ for each channel.

If we create a plot with this function with this parameters for red, green and blue ((2,0),(2,-0.1),(2,-0.3)) we get:

Pseudocolor Graph with red(p=2,theta=0) green(2,-0.1) and blue(2,-0.3)

Then we only need set the gray level in range 0 to 1 and the sine returns values from 0 to 1 we interpret as float image values or we set in range 0 to 255.

To finish this is the result:

Download the code.