ginput() in opencv-python

The Image Processing Toolbox in MATLAB has this very useful utility called as ginput, which allows manually picking co-ordinate points from an image. I tried to recreate that functionality in python based OpenCV, and the result is the following code:

import cv2
import numpy as np
from matplotlib import pyplot as plt
import os

a = np.array([0,0], dtype='float32')
def getXY(imgPath):

   DEBUG = 1

 print "-------------------------------"
 print "Using OpenCV version : ",cv2.__version__
 print "-------------------------------"

 print "TASK : Mimic getxy() feature of MATLAB"
 print "***************************************"

 except KeyError:
    #print "For verbose mode, set DEBUG flag"
    DEBUG = 0

 #define the event
 def getxy(event, x, y, flags, param):
   global a
   if event == cv2.EVENT_LBUTTONDOWN :
      a = np.vstack([a, np.hstack([x,y])])
      if (DEBUG):
      print "(row, col) = ", (x,y)

   #Read the image
   img = cv2.imread(imgPath)
   if (DEBUG):
      print "Reading the image..."

   #Set mouse CallBack event
   cv2.setMouseCallback('image', getxy)
   if (DEBUG):
      print "Set MouseCallback functionality..."

   #show the image
   print "Click to select a point OR press ANY KEY to continue..."
   cv2.imshow('image', img)

   #obtain the matrix of the selected points
   b = a[1:,:]
   if (DEBUG):
      print ""
      print "The clicked points..."
      print b, b.dtype, b.shape

   print "The selected points are returned as a float64 MATRIX..."
   return b

Steps to use it :

  • save the above source code as in your working directory.
  • In your python code, import this file, i.e.,

from getXY import *

  • Pass the image path to getXY, and start clicking on the displayed image. The no. of times you click, those many points will get saved.
  • If you want a verbose version, set the environment variable DEBUG = 1 from the terminal before running the code, i.e.,

DEBUG=1; export $DEBUG

That’s it! I’ll upload the code onto a git repo as well soon and provide a link here as well.

  • To remove the verbose code, unset the DEBUG variable, i.e, :

unset $DEBUG

OpenCV with video support in recent Linux distros

There were days when we had to painstakingly go through the manual install process for getting OpenCV running on our systems. Those days are long gone, and for the good. Current day distros bundle both OpenCV and ffmpeg in their repositories, making life extremely easy for enthusiasts and geeks alike. The following 4 steps provide a fully functional OpenCV install on Fedora:

  • sudo yum groupinstall “Development Tools” “Development Libraries”
  • sudo yum install eigen2-devel CTL CTL-devel OpenEXR_CTL-libs tbb yasm yasm-devel tbb-devel OpenEXR CTL-devel  gstreamer-plugins-base-devel libsigc++20-devel glibmm24-devel libxml++-devel gstreamermm xine-lib libunicapgtk-devel xine-lib-devel gstreamermm-devel python-devel sip-macros sip vamp-plugin-sdk audacity sip-devel xorg-x11-proto-devel libXau-devel libX11-devel automake libogg-devel libtheora-devel libvorbis-devel libdc1394-devel x264-devel faac-devel xvidcore-devel dirac-devel gsm-devel zlib-devel faad2-devel speex-devel lame-devel orc-compiler orc-devel libvdpau cppunit libvdpau-devel schroedinger-devel dirac x264 lame faad2 amrwb-devel opencore-amr-devel amrnb amrnb-devel
  • sudo yum install ffmpeg ffmpeg-devel opencv opencv-devel opencv-python
  • sudo updatedb

That’s it! Dont believe me, well run a sample python or c code and see for yourself! 🙂 You can download this PDF file containing a sample code written in both python and C. Provide the video file appropriately and enjoy seeing your OpenCV installation playing a video file.

P.S : For C based codes, run gcc as:

gcc -L/usr/include/opencv -lopencv_core -lopencv_highgui -lm -o a.out fileName.c

For python based codes:


Video in OpenCV

Video files in OpenCV

Recording from Kinect for offline Kinect-less development

Fakenect, allows for hacking and playing with the libfreenect library without the need for having an actual Kinect device. Assuming you have a pre-“recorded” data, all that needs to be done is:

fakenect <path_to_data_folder> <your_application> <parameters>

For e.g., if we need to run freenect-glcview, that comes bundled with libfreeenct library, we need to execute it as:

fakenect <path_to_data_folder> freenect-glcview

and we’ll get access to the pre-recorded data as though an actual Kinect device was connected! Now comes the interesting part.. How to record such a data? We could do it using freenect-record that comes bundled with libfreenect, but it doesnt let the user know what is being captured due to absence of any GUI.

kinect_record is a nice little code meant to record and show what is being recorded simultaneously. However, compiling this little gem needs some tweaking to the Makefile.linux that comes bundled along:

search for the location where freenect.h is located in your system:

  • locate libfreenect.h (for my system, its something like: /usr/include/libfreenect/libfreenect.h)

copy the path till libfreenect/ (for my system, it will be: /usr/include/libfreenect/). Open Makefile.linux and edit the CFLAGS variable to look like:

  • CFLAGS=-g -Wall -I/usr/include/libfreenect -I/usr/include

This should be enough in most of the cases. I hit an additional error message: undefined reference to symbol ‘pthread_key_delete GLIBC_2.2.5’. This was resolved using this suggestion and the corresponding edit to LDFLAGS variable makes it look like:

  • LDFLAGS = -lfreenect -lGL -lGLU -lm -lglut -L /lib64 -l pthread

That should be it! Just type make and the executable should be ready. To execute it, type:

  • ./kinect-record <path_to_folder>

and enjoy hacking using libfreenect without needing the actual Kinect device!

Using Skeltrack with Kinect on Fedora 18

Skeltrack is a skeleton tracking library developed by Igalia’s Joaquim Rocha. Developed as a FOSS library, it is very well integrated with Gnome technologies, an example of which is the capability to control Gnome desktop using Skeltrack.

Here, the steps needed to get Skeltrack up and running on your fresh F18 system is detailed. First, install the dependencies using yum:

sudo yum install libfreenect* autoconf automake gobject-introspection-devel gtk-doc */cairo.h

Now we are ready to begin the actual installation. Since, libfreenect lacks the motor control capabilities, GFreenect was developed at Igalia. It is a dependency for Skeltrack, so we need to install it first using the following steps:

  • sudo updatedb
  • export LD_LIBRARY_PATH=/usr/local/lib/
  • export PKG_CONFIG_PATH=/usr/local/lib64/pkgconfig/:/usr/local/lib/pkgconfig:/usr/lib/pkgconfig
  • git clone
  • cd GFreenect && libtoolize -v –copy –install
  • aclocal
  • autoconf
  • ./
  • ./configure
  • make
  • sudo make install

With this, GFreenect installation should be done. You can test the installation by plugging in Kinect and executing:

  • cd <PATH_TO_GFreenect_FOLDER>/examples
  • python2.7 test*.py

You should get something like this:

GFreenect Demo

Now to install Skeltrack, similar steps are needed:

  • sudo updatedb
  • export LD_LIBRARY_PATH=/usr/local/lib/
  • export PKG_CONFIG_PATH=/usr/local/lib64/pkgconfig/:/usr/local/lib/pkgconfig:/usr/lib/pkgconfig
  • git clone
  • cd Skeltrack && libtoolize -v –copy –install
  • aclocal
  • autoconf
  • ./
  • ./configure
  • make
  • sudo make install

Thats it! Skeltrack would be up and running on your system :). To test, run:

  • cd <PATH_TO_Skeltrack_FOLDER>/examples
  • ./test-kinect.o

These steps should be enough to get us ready to use the amazing Skeltrack library.. In case you’re even more eager to play around with Kinect and want to show off your gesture skills, install and use Skeltrack-Desktop-Control :

And we’re done. To run the software:

  • cd <PATH_TO_Skeltrack_FOLDER>/src
  • ./skeltrack-desktop-control

Happy Kinect hacking! 🙂

MATLAB: Using ANN library wrapper in Windows XP

ANN (Approximate Nearest Neighbours) is an extremely useful library when working with any type of application where the properties of data points are influenced by their neighbours. The MATLAB MEX-wrapper for the same provided by Shai Bagon, along with the patches, compiles effortlessly in Linux (have tried on F16 and F17). However, making the same work in Windows turned out to be quite an effort. First, the MATLAB inbuilt compiler for Windows (lcc) continuously kept crashing whenever I tried executing the library, and second, the lack of pre-existing solid C/C++ compiler made things difficult.

After searching on the net for free and supported compilers for MATLAB R2010a, I first tried Open Watcom, but it gave too many errors during compilation of the mex-wrappers.  Then it was the turn to check out MS Visual C++ Express Edition. Tried on a friend’s system having MS-VC++ 2010, and it too gave quite a few errors during compilation. Thought to try out MS-VC++ 2008, and guess what, it worked out of the box and the mex-wrapper compiled with quite an ease 🙂 . Wonder why MS-VC++ 2010 created issues..

MATLAB: Running codes in mixed OS environment

Ever faced the problem of running executables in a MATLAB installation on Linux. I frequently use an executable file (.exe) provided by an eminent research lab, but the linux enthusiast that I am, shifting to Windows just for running this file is out of the question! So, how to solve this issue!

Well, you need access to either a system running Windows at your workplace, or have a virtual Windows installation in VirtualBox. Set up ssh via cygwin (refer SSH via CYGWIN) on such a system and also make sure to have a password-less SSH login enabled (refer ssh autologin) between the Linux and Windows systems.

Once this is done and given the fact that such a Windows environment is up and running, all you have to do through the MATLAB command prompt is:

system(‘ssh username@IP_of_windows command1;command2’);

where : command1 and command2 are the things you want to execute on that system. For e.g: in my scenario, i have already put the .exe (say: test.exe) in the cygwin home folder (C:/cygwin/home/<username>/) on the Windows system. Now, i just run:

system(‘ssh username@IP_of_windows ./test.exe’);

You can use few more ssh/scp via system command calls in MATLAB to copy back & forth the data. Thats it!