Head / Face tracking with Blender - the easy way

HA! Theres nothing easy about this (I haven't even got it working yet!   It Works!)

If you're like me ( and you did follow 'the easy way' title) you don't want to get into really deep programming because, like me, you're not very good at it.

So I have been trying to get face tracking working with the Blender Games Engine (in fact this is my third attempt) and I have hit no end of problems. The problems are around software and library versions.

The person to be most helpful is Akta ( see http://www.aktasway.com/blog/blender-and-opencv/ ) and Akta is kind enough to give away a little demo system which sadly does not work on my computer; but he also gives the original python scripts so I stand a chance of getting it going. Akta's demo is compliled for Python 2.6. He includes 'pyd' files (sort of python dlls) for OpenCV (which does all the clever face stuff. and PIL (python image library). But anyway it doesn't work.

Ashid (http://www.ashsid.sk/wp/?p=174&lang=en-us ) who wrote the marvelous VideoTexture for 2.48 (I still consider this the best (Windows only) and produces beautifully smooth output with an camcorder) has come up with a better version of the VideoTexture for Blender 2.62 using the python VideoCapture library ( and theoretically it should work for 2.49b)
But there is already a VideoTexture for Blender 2.6 I hear you cry! Yes, but a) it is not as good as Ashid's for windows because it uses FFMPEG and b) I can not actually find anywhere about using it with a webcam. The manual mentions it and then fails to mention how to do it.

So first, I tried Blender 248 with videotexture. It uses Python 2.5 so does not work with Atka's code :(

Next, tried Blender 2.63 (why not do it for the latest version I thought). 2.63 uses Python 3.2 and openCV has not been compiled for 3.2! (I did try 'ctypes for opencv' but that didn't work. )

So lastly I am trying Blender 2.49b. It uses Python 2.6 :) BUT, I am using Windows 7 pro 64BIT . Does this mean I have to install Python 2.6 64bit? (which I have). Does this mean I need 64bit versions of the libraries (which don't exist)? Can I install both 32 and 64bit versions of Python?

So I worked on and now its working though its a bit flakey (not anymore)

In the end I used the cv2.pyd library from opencv 2.41 to both capture the video and do the face detection. I don't think I need any other libraries. Blender 249b 32bit and Python 2.6.
I used bytefish's example here http://www.bytefish.de/wiki/opencv/object_detection . The lbpcascade_frontalface.xml detection scheme is a little primative (no eyes or head angles but is suppose to be faster.

And without any more a do here it is - Blend file

ONLY WORKS WITH BLENDER 2.49b!
(I use Windows 7 - it may work in Mac or Linux but not tested)
(update - tried on Ubuntu 10 and there seem to be some pretty horrible issues with numpy)

You must have

• Python 2.6 installed - find here http://www.python.org/download/releases/2.6.6/
• Numpy for python 2.6 - http://sourceforge.net/projects/numpy/files/NumPy/1.6.2/
• Blender 249b - http://download.blender.org/release/Blender2.49b/

   
you need the cv2.pyd and lbpcascade_frontalface.xml in the same directory as the Blend file.
The cv2 and lbpcascade_frontalface.xml files are part of opebCV from http://opencv.willowgarage.com/wiki/ version 2.41.
   
You will also need a webcam attached. (you can have more than one webcam see the Start.py file in the text editor)

To use press P and to end press ESc

Results vary greatly on lighting conditions and camera quality

enjoy, L

If you find this useful or it doesn't work for you please leave a comment - as I have no idea