10-17-07 - Re-Writing the laser pointer paint program (image processing) in C/C++

This should significantly increase our capture and processing speed. First, install libcv1 in ubuntu, I like to get the documentation and python bindings as well, and these will install libcv1 anyway. sudo aptitdue install python-cv opencv-doc Now we'll start writing some C code. As I have the benefit of writing this article after the code, I know that one can render/display 30fps without processing, and can render/display without lag with processing as well (thank you compiled code!). You can compile opencv code in linux using the following command: gcc `pkg-config --cflags opencv` `pkg-config --libs opencv` -o MY_PROJECT_RUNME MY_PROJECT.cpp

Well that's simple enough. The structure of the program has a few changes.

1. Program takes first command line argument as the image filter size (try 2-10 for good results)
2. Program no longer does image subtraction for a mask over the image, it's direct processing now
3. Program is significantly (10x) faster, depending on speed of camera frame grabs
4. Check out the openCV tutorial for grabbing images from a camera, then check out the code below..
5. Or, you can snag it here
6. Also, please note that 'escape' will end the loop and finish the program

#include < opencv/cv.h > #include < opencv/highgui.h > #include < stdio.h > #include < iostream > //a helper function to convert from color to gray IplImage* convert_grayscale(IplImage *img){ IplImage *gray_image,*gray_img0; gray_image=cvCloneImage(img); //check image is it gray or not if nor convert it to the gray if(img->nChannels!=1){ //convert original image to gray_scale image gray_img0 = cvCreateImage(cvSize( gray_image->width, gray_image->height), 8, 1 ); cvCvtColor(gray_image, gray_img0, CV_RGB2GRAY ); gray_image = cvCloneImage( gray_img0 ); } return gray_image; } //a helper function to return the average value of a chunk of pixels int average_pixel_value(IplImage *img, int x, int y, int boxsize) { int total = 0; int pixels = boxsize * boxsize; for(int xa = x;xa < x+boxsize;xa++) { for(int ya = y;ya < y+boxsize;ya++) { total += ((uchar*)(img->imageData + img->widthStep*y))[x]; } } return total/pixels; } // A Simple Camera Capture Framework int main(int argc, char* argv[]) { //let's set the size of our image bin (nxn) int imagebinsize; if(argc > 1) { imagebinsize = atoi(argv[1]); } else { std::cout << "Error, usage is 'cam ' i.e. 'cam 2' or 'cam 3'" << std::endl; return 0; } CvCapture* capture = cvCaptureFromCAM( CV_CAP_ANY ); if( !capture ) { fprintf( stderr, "ERROR: capture is NULL \n" ); getchar(); return -1; } //first we should let the camera driver autofocus for(int i = 1;i<5;i++) { // Get one frame IplImage* frame = cvQueryFrame( capture ); if( !frame ) { fprintf( stderr, "ERROR: frame is null...\n" ); getchar(); break; } } // Create a window in which the captured images will be presented cvNamedWindow( "mywindow", CV_WINDOW_AUTOSIZE ); // Capture our primary scene in a frame IplImage* primarySceneFrame = cvQueryFrame( capture ); //convert our primary image to black and white for processing if needed IplImage* grayPrimarySceneFrame = convert_grayscale(primarySceneFrame); //let's get the size of our image CvSize imsize = cvGetSize(grayPrimarySceneFrame); int width = imsize.width; int height = imsize.height; // capture images while(1) { // Get one frame IplImage* frame = cvQueryFrame( capture ); if( !frame ) { fprintf( stderr, "ERROR: frame is null...\n" ); getchar(); break; } //convert our new image to black and white for processing frame = convert_grayscale(frame); //Now we do the following //1. Loop through frame with a boxsize box //2. if luminence value in the box is average > maybe 200 //3. then primarySceneFrame[] = 255(bright); //4. display the primarysceneframe, not the 'frame' for(int x = 0;x < width-imagebinsize;x+=1) { for(int y = 0;y< height-imagebinsize;y+=1) { if(average_pixel_value(frame, x, y, imagebinsize) > 200) { ((uchar*)(grayPrimarySceneFrame->imageData + grayPrimarySceneFrame->widthStep*y))[x] = 255; } } } //display cvShowImage( "mywindow", grayPrimarySceneFrame ); //If ESC key pressed, Key=0x10001B under OpenCV 0.9.7(linux version), //remove higher bits using AND operator if( (cvWaitKey(10) & 255) == 27 ) break; } // Release the capture device housekeeping cvReleaseCapture( &capture ); cvDestroyWindow( "mywindow" ); return 0; }

9-26-07 -- Home Automation + Gmail in Ubuntu

Ok, now that we have a working home automation server setup (see article 1), let's plug in a lamp module and have it flash when our gmail arrives. First up, plug in the light module, select your house code ('heyu info' will tell you your house code) and an unused x10 number on the dial.
Find a lamp you'd like to have flash (or turn on, or dim etc etc) and plug it into any other outlet. Turn on the lamp, you want to be sure the light is actually on before you start cursing at your x10 module. Now plug the 'on' light into the x10 module. Let's try turning on the module with a heyu on A3 Where A is your house code and 3 is your module number. Voila, you should have light. a quick heyu off A3 will get us back to the off state. There are many heyu options (dimming !) to try out, but they all work in this manner. Now, let's install gmail-notify with a sudo aptitude install gmail-notify Gmail notify is a great python script to notify you when you have gmail. We will insert an execute command into the python code (very simple) so when gmail-notify runs and we have an unread email, flash the light (or dim the light, or turn on your lava lamp, etc.)

Fire up your favorite editor (I prefer vim, but most guides prefer nano) sudo nano /usr/lib/gmail-notify/notifier.py Now head down to line 208 (it was 208 as of 9/07). You are looking for this block of code: if attrs[1]>0: print str(attrs[1])+" new messages" We will change it to: if attrs[1]>0: path = '/usr/bin/gmflash.sh' os.system(path) #execute gmflash print str(attrs[1])+" new messages" What we've done is told gmail-notify to execute the script /usr/bin/gmflash.sh when there is new gmail. Now we'll need to create this script with a sudo nano /usr/bin/gmflash.sh Here we'll tell the x10 light to flash on, then off. It takes my x10 module 1 second to deactivate a light and 1 seconds to activate one. It takes my light 2 seconds to 'warm up' to bright. We'll want our script to pause for at least (1 + cycle time + bulb time) seconds, so in my case that's 4. Insert into your editor window: heyu on A3 sleep 4 heyu off A3 Save it, close it, and make it executable with: sudo chmod +x /usr/bin/gmflash.sh

That's all there is to it. Fire up gmail-notify, put in your gmail settings, and send yourself a test email to check. If your light doesn't flash, try executing /usr/bin/gmflash.sh. If this doesn't work, time to re-check your setup. Fin.

9-23-07 - Using scilab video processing toolbox and a laser pointer to "paint" a scene in realtime.

Ok so this actually turned out to be some pretty simple scilab code.

Here's what it should look like after/during a run. Later we'll use this same idea to do automatic laser calibration with a few robots parts. Stay tuned for that!

A little tic/toc ery has shown that without an image draw scilab can process 3 fps. So here's a loop unrolled version that delays image rendering till 3 dots have been drawn. //"paint" a scene with a laser pointer in realtime //speed of update depends on speed of camera + processor //could also easily be used on a video file n = camopen(); for idx=1:15, //give the camera time to auto white balance im1=avireadframe(n); end; im3prime = rgb2gray(im1); //save our "primary scene" im3 = im3prime; imshow(im3); //show us our primary scene r=x_message(['Baseline Set'],['Ok']); //let us know when to laser pointer for ido=1:15, mask = zeros(im3); //clear out our mask quickly for idx=1:3, //or however many frames/sec you can process //tic; im2=avireadframe(n); //read a frame in //subtract the greyscale current image from the primary scene //then take that logical array, convert to numerical and //use it as a mask over im3 mask = mask + bool2s(imsubtract(rgb2gray(im2), im3prime) > 50); //imshow(im3); //toc end; im3(mask == 1) = 255; imshow(im3); end; avicloseall();

9-23-07 - Installing scilab and the scilab image and video processing toolbox in ubuntu linux.

For some time now I've been into computer vision. However, much of computer vision is done on matlab, a 1000$ piece of software that doesn't jive with my open source philosophy. Luckily, there are a number of open source alternatives. I prefer scilab, though octave is a good alternative. However, scilab has the advantage of an open source video toolbox. In this first article, I'll show you how to setup scilab, opencv (the intel open computer vision library), and the sivp toolbox. Then we'll verify its working by processing video from an avi file or in this case, a live webcam stream.

1. First, lets start by installing scilab and preparing our system to compile ffmpeg and opencv
sudo aptitude install scilab
2. Now follow this guide to get opencv and ffmpeg compiled. Remember to make sure ffmpeg, v4l, and v4l2 are compiled options, or we'll be unable to process video, webcam, or newer webcam video
3. Now head on over to the SIVP webpage and follow the typical ./configure && make && sudo make install
4. At this point we should have video streaming in scilab. Fire up scilab from the command line scilab
5. Here, we'll want to make sure the video/image toolbox is loaded. Hit the toolbox menu, then the sivp toolbox. You should get a message about it being loaded
6. Now, hit the examples button, and make sure your main scilab window stays open. From here click the sivp section, and we can grab straight from a videocam, image, video etc.
7. The best part about the examples is the (very simple) code is displayed in your main scilab window, very cool!

I've got the flu and I look it in this picture, but here's what you should see from your webcam in scilab. From here the sky is the limit, next time i'll be showing you how to interconnect robotics, laser optics, and sci-lab video processing in a very cool and flashy demonstration, guaranteed to impress your friends, all for under 40$ and the cost of ir shielded safety glasses.

9-23-07 - Site news, I've made a new section for robotics/computer vision and home automation projects.

Building a completely automated, web/ssh/vnc controlled, home automation server with facial recognition from a clamshell ibook (with a faulty cd drive) and 10$ worth of electronics. Part 1.

9/21/2007 - Ok, so i've got this old clamshell ibook. The main weakness of these models was the screen, a paltry 800x600 resolution. The main sweetness of the project:

Control your house lights from your phone, any web browser, any ssh client, etc

Automate your lights, run complex temperature analysis and face detection.

Run ubuntu linux, fully support all hardware functions on ppc architecture, failsafe in case of power failure .

Dns forwarding from easy to remember address.

Future development: laser calibration.

Ok, first thing's first, what will we need?
1. ibook clamshell

2. 1x usb->serial adapter (~1$)
3. 1x x-10 heyu compatible adapter (~10$ ebay)

4. whatever x10 controllers you'd like (lights, power outlets) cheap on ebay
5. any old usb camera should work (firewire maybe? stay tuned!)

The ibook has the following specs, but any computer ~ these specs would be fine:
366mz g3 ppc proc.
368mb ram
10gb hdd
800x600 lcd
1x usb 1.1 port
1x firewire port

Ok first of all, let's get linux installed on here. Because the cd-rom drives tend to die in these units, this was the case here. The solution was to burn an ~12mb iso image to a cd - The ubuntu mini ppc.iso image, and keep retrying this boot disk till the laptop booted the cd (about 20 tries). You could also use any other macbook in firewire host mode (check the mac forums for that). The feisty image (works great!, install xubuntu) supports the airport card and the Ethernet port naively, so that was nice. Remember to hold the 'c' key on your mac to boot from the cdrom.

While you have that running, let's do some multitasking. First head over to dyndns and get yourself a dns forwarding address (if you don't want to have to remember your constantly changing ip address). While you're at the site, check out their guide to inadyn, some cool open source software to update dyndns with your dynamic ip. Also, if you have a router, make sure to forward the ports for whatever services you want (20-25 ssh, telnet, ping etc, 80 or 8080 etc for web, 5090-5092 vnc, etc)

Ok so at this point we've got an ibook running xubuntu feisty, let's install some packages. We'll want to:
Rock open a terminal and break out your su hat. sudo aptitude install the following packages and the firewall software of your choice.
openssh-server //this is if you want ssh access to your machine
inadyn //this will automatically update your dydns
tightvnc-server //if you're into vnc
gnome-power-preferences, gnome-power-settings //easy power profiles (i.e. close the lid, blank the screen)

Now it's time to plug in you usb->serial adapter. I broke down and paid 5$ on ebay for one, but I've seen them locally for about 1$. In a terminal, cat /var/log/messages, and look for lines like 'usb 1-1, pl2303 (or your chipset) converter now attached to ttyUSB0'. This is your new serial port, and what you'll specify to hey-u (software to control your serial port controlled x-10 power line controller). Now go download hey-u. In terminal, run the whole configure/make/sudo make install shebang. The install is pretty user friendly, and they ask you for your serial port (which we got earlier from /var/log/messages). Time to try 'heyu info' Fingers crossed...
It works!
It should be immediately apparent if your device is found, you'll get firmware info as well.

At this point things are coming together. We've got remote access from any ssh capable device to a command line interface to all the x10 power devices in our house. Next up, if you have a way to get jar files onto your phone, or your phone has a browser, I highly recommend midpssh from here.

At this point the sky is the limit, but first thing I recommend is installing a web frontend to hey-u like the one found here. So for this, we'll need apache and php. So get your fix with a 'sudo tasksel install lamp-server'. I also recommend that right after installing apache, you 'sudo touch /var/www/index.html', so your web directory isn't open. How about a fusion of web/ssh access?
I recommend grabbing the isnetwork release of mindterm ssh for java applet from here
Just move everything in the applet directory in the zip into a folder on your website, change the netscape.html to index.html, and you've got an ssh client available from any computer that has java. Sweet.
End of Part 1!