IMG_2820

Proof of concept 3D Scanner with Kinect and Raspberry Pi2

I am working on a proof of concept standalone mobile 3D Scanner. Hopefully it will be possible to use a Raspberry Pi2 for this project. I have already posted a video on youtube. And some people asked for short instructions on how to run the Kinect on the Raspberry Pi2. And here it comes….


First i printed and modified a Kinect handle i found at Thingiverse. I remixed this handle and added a Raspberry Pi2 and a display mount to it.  You can find the files at: http://www.thingiverse.com/thing:698577

IMG_2821_preview_featured

 

You  can get the Raspberry Pi display from watterott.com, instructions for installing the display can be found on GitHub. I recommend to use the current Raspberry Pi Image which you can also find on GitHub.

 

IMG_2823_preview_featured

 

Start with the clean display Image. I used the libfreenect for some experiments. It seems that libfreenect provides all the functionality which is provided by the Kinect.  Lets start!

First of all we need to install all required libs. We start with an update of the packaged list.

sudo apt-get update

Next we need to install some libs and tools wich are needed to build libfreenect.
Python and OpenCV related Stuff:

sudo apt-get install build-essential python-dev ipython python-opencv
sudo apt-get install python-numpy python-scipy

Build Tools and dependencies of libfreenect

sudo apt-get install git-core git
sudo apt-get install freeglut3 freeglut3-dev libxmu-dev libxi-dev
sudo apt-get install cmake cmake-curses-gui pkg-config

Libfreenect depends on libusb for the communication part: We have to build libusb, because the prebuild version won’t work with current versions of libfreenect.

sudo apt-get install libudev-dev

First we need to get libfreenect source, untar and compile it.

sudo wget http://sourceforge.net/projects/libusb/files/libusb-1.0/libusb-1.0.19/libusb-1.0.19.tar.bz2/download

tar xvjf download
cd libusb-1.0.19
./configure
make
sudo make install

And finally install the lib. When the install process is finished, we can install libfreenect.

cd ..
git clone https://github.com/OpenKinect/libfreenect.git
cd libfreenect
mkdir build
cd build
ccmake ..

Here we have to configure some things. First press ‚c‘ key. Then use the arrow keys to change BUILD EXAMPLES to OFF and make sure that LIBUSB_1_INCLUDE_DIR directory points to /usr/local/include/libusb-1.0. At the end press the ‚c‘ key a second time followed by pressing ‚g‘. If the console is promoted again, start the compile process with :

cmake ..
make
sudo make install

Congratulations, if no errors appear, you are done. You can test the driver and lib by changing into the ‚wrappers‘ driectory. By typing

cd ../wrappers/python

There you have to install the python libs of libfreenect.

sudo python setup.py install

After that you are ready to go. Give it a try by starting some of the example Applications.  For example the async demo.

sudo python demo_cv_async.py

These program examples are not capable to handle multiprocessing, it works and if you use a single stream example you are able to get a fine depth image stream. I am working and experimenting with some multiprocessing code i’ve already written.

Since i am very interested in 3D Scanners i will do further work on it this summer. First i have to finish my other 3D Laser Scanner project http://hci.rwth-aachen.de/fabscanpi

Feel free to comment! And have fun.

45 Gedanken zu „Proof of concept 3D Scanner with Kinect and Raspberry Pi2“

  1. Wenn du es schaffen würdest das man mit dem Teil eine fertige STL in guter Qualität heraus bekommen würde, wäre das der absoulute Oberhammer! Viel Erfolg dabei, ich bleib auf jeden Fall dran!!!

    1. Ich bleibe dran. Wird aber noch eine Weile dauern bis die ersten Ergebnisse da sind. Zur Zeit arbeite ich an der neuen offiziellen Software für den Fabscan 3D Scanner.

  2. Hi Mario,

    Really nice project ! Do you think it could work with a Rpi B (first generation ). I do not see any restriction however I prefer ask before to buy all the stuff I need 😉

    Thanks,

    Stéphane

    1. I think a RpiB does not have enough power to work with those lib’s. And remember there is no software which will do a 3D scan now. It’s only a demo wich shows that i got the kinect one working on a Rpi with a display.

      At late 2015 i will try to write a 3D scanning solution for the Rpi with kinect.

      1. I can confirm that it sorta, kinda works with a RPi 1 model B, but the processing speed is far too low. Depth frames can be unsteadily aquired at 1 per 10 seconds using the sync demo.

  3. Hallo Mario,

    ich habe schon mal bei der Skanect Gruppe gefragt, ob man mit einem Raspi nur einen Scan aufnehmen und dann auf einem PC berechnen könnte.


    You can fairly easily generate .skn folders though. The file format is only using standard compression and folders. The hardest is the depth file format, but there is actually open source code to write that in github.com/rgbdemo . If you would like to go down that path, I think I described the format in the forum in the past, but if you can’t find it I can provide more help.

    Link
    Vielleicht eine Zwischenlösung?

    1. Hi,

      Danke für den Hinweis. Ich weiss nicht ob es mit Skanect möglich ist. Selbst habe ich mit Skanect noch keine Erfahrung sammeln können. Gibt es denn offene Schnittstellen? Wäre auf jeden Fall ein interessanter Ansatz. Bei diesem Projekt versuche ich es jedoch auf eine andere Art und Weise.

      Gruss
      Mario

    2. This is what im thinking to do right now either skanect or kinect fusion. Did you find any Progress on this idea?
      cheers

  4. Hi,
    I have the same hardware as you (Kinect for Windows, Raspberry Pi 2), and have followed your steps exactly, even on a new Raspbian image (2015-05-05), but all programs (including wrappers/python/demo_cv_async.py) return the following error message from libfreenect:
    „Could not open audio: -4
    Failed to open motor subddevice or it is not disabled.Failed to open audio subdevice or it is not disabled.Error: Can’t open device. 1.) is it plugged in? 2.) Read the README“
    1) Of course it is plugged in, the xxorde/librekinect driver does work. 2) I have read the README, but nothing seems to be useful. I have tried both running it as superuser, and adding the corresponding udev rules, to no avail. I have also blacklisted gspca_main and gspca_kinect.
    lsusb returns:
    Bus 001 Device 011: ID 045e:02bf Microsoft Corp.
    Bus 001 Device 019: ID 045e:02be Microsoft Corp.
    Bus 001 Device 008: ID 045e:02c2 Microsoft Corp.
    Upon connecting the Kinect, dmesg shows:
    [ 2684.939028] usb 1-1.5.1: new high-speed USB device number 18 using dwc_otg
    [ 2690.428926] usb 1-1.5.1: new high-speed USB device number 19 using dwc_otg
    [ 2690.531017] usb 1-1.5.1: New USB device found, idVendor=045e, idProduct=02be
    [ 2690.531043] usb 1-1.5.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
    [ 2690.531062] usb 1-1.5.1: Product: Microsoft Kinect Audio, \xffffffc2\xffffffa9 2011 Microsoft Corporation. All rights reserved.
    [ 2690.531079] usb 1-1.5.1: Manufacturer: Microsoft
    [ 2690.531096] usb 1-1.5.1: SerialNumber: A22592W00346213A
    What can possibly go wrong? Note that the Kinect works correctly with libfreenect on an Ubuntu 14.04 PC.

  5. I’ve followed this through, but when i try and run one of the demos i get met with this error message: „GdkGLExt-Warning **: Window system doesn’t support OpenGL.“ can anyone help?

      1. Thank you so much for this hint. If one of the other guys with the same issue can confirm that this works i will add it to the blog entry.

  6. Sorry guys. At the moment i have no time to work on this project until i have released the FabScan Pi Software. So i also have not the time to do something in your error cases. The instructions in the article are for wheezy. Try to use the same libusb etc. In general that should work. But i have also good news. A couple of weeks ago i compiled some software for 3D Scanning with the pi. The software compiles well and is executable, but i had no luck with connecting the kinnect with this software due the mentioned lack of time. Keep rationed, i will do further work on this project soon.

    1. Hi Marko,

      It is not a problem at all, we are all really busy! Thank you very much for your work !

      Could you let us know which softwares you compiled a month ago for the pi ?

      Cheers

      S

  7. Hi Mario! I’m having issues with the portion after creating the build directory. ccmake and cmake are „not found“, and entering „make“ does nothing- says there’s nothing to be done for ~/libfreenect. Any idea what’s going on! I love your work!

  8. Hi Mario,
    I have done your Project and successfully have a Depth Image stream. I have also found a way to record this stream using the Fakenect-record Util.

    Now I would like to shift this data to some 3d Scan Apps liek Kinect Fusion, Skanect or Arztec. But im not sure how. Im a begiiner at Programming and realy excited to make this real.

    I have also order the new RP3 n will try out on it . Hopefully the new Rasbian will hv a OpenGL driver too.

    How is the Pprogress of yours and do you have any insight that could help me to realise this.

    Cheers.Your big fan,Mohandez

    1. Hi,

      getting the stream running was just a proof of conept. I wanted to try if the kinect works reasonbale with the Raspberry Pi2. I switched to the Raspberry Pi3. I am porting an old Kinect 3D scanner project, which is written in c++. I am just fixing some code, afterwards i will give it a try. When it woks as i imagine the sotware should be able to do the whole scanning process on a Raspberry.

      cheers
      Mario

      1. Thats great. Pls let us know ur progress and ur insights on your new approach. Your guidence is very helpful. Im planning to do a Bachelor Project to go deeper on this POC.

        Hope we could make this real. Cheers Mario.

  9. Hi,
    have you tried running the kinect on the Raspberry Pi 3 yet? I got my R Pi 3 a few weeks back and wanted to see what frame rate it could I could get with Kinect. It works but I’m getting a random number of connection errors each time I restart my capture program. It runs great once connected and I guess I’m getting about double the frame rate over the R pi 2B. But at this point I’m not sure if I have a faulty USB chip or a code bug from the hardware upgrade. I’m using Freenect installed from the raspbian repository, but I’ve also compiled on the rpi 3 along with libusb (as above) .

    grateful for any help, cheers.

  10. Just for the record, I am also following this project with great anticipation. My kinect and rpi3 are already here. We just need a 3d-scanning software… 🙂

  11. Hi,
    I’m pretty desperate to confirm if anyone has tested the kinect with the Raspberry Pi 3 and found the above mentioned bug or alternatively is able to confirm everything is working fine. So I can confirm or reject having faulty hardware.

    thanks again for any help.

  12. I’m searching for a handheld 3D scanner
    but can not really find anything
    your idea is very nice
    You already further along
    There are somewhere building instructions
    and can be stored in .stl

Schreibe einen Kommentar

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind markiert *

Kommentare werden moderiert. Es kann etwas dauern, bis dein Kommentar angezeigt wird.