Reading Time: 3 minutesI am working on a proof of concept standalone mobile 3D Scanner. Hopefully it will be possible to use a Raspberry Pi2 for this project. I have already posted a video on youtube. And some people asked for short instructions on how to run the Kinect on the Raspberry Pi2. And here it comes….
First i printed and modified a Kinect handle i found at Thingiverse. I remixed this handle and added a Raspberry Pi2 and a display mount to it. You can find the files at: https://www.thingiverse.com/thing:698577
You can get the Raspberry Pi display from watterott.com, instructions for installing the display can be found on GitHub. I recommend to use the current Raspberry Pi Image which you can also find on GitHub.
Start with the clean display Image. I used the libfreenect for some experiments. It seems that libfreenect provides all the functionality which is provided by the Kinect. Lets start!
First of all we need to install all required libs. We start with an update of the packaged list.
sudo apt-get update
Next we need to install some libs and tools wich are needed to build libfreenect.
Python and OpenCV related Stuff:
sudo apt-get install build-essential python-dev ipython python-opencv
sudo apt-get install python-numpy python-scipy
Build Tools and dependencies of libfreenect
sudo apt-get install git-core git
sudo apt-get install freeglut3 freeglut3-dev libxmu-dev libxi-dev
sudo apt-get install cmake cmake-curses-gui pkg-config
Libfreenect depends on libusb for the communication part: We have to build libusb, because the prebuild version won’t work with current versions of libfreenect.
sudo apt-get install libudev-dev
First we need to get libfreenect source, untar and compile it.
sudo wget https://sourceforge.net/projects/libusb/files/libusb-1.0/libusb-1.0.19/libusb-1.0.19.tar.bz2/download
tar xvjf download
cd libusb-1.0.19
./configure
make
sudo make install
And finally install the lib. When the install process is finished, we can install libfreenect.
cd ..
git clone https://github.com/OpenKinect/libfreenect.git
cd libfreenect
mkdir build
cd build
ccmake ..
Here we have to configure some things. First press ‚c‘ key. Then use the arrow keys to change BUILD EXAMPLES to OFF and make sure that LIBUSB_1_INCLUDE_DIR directory points to /usr/local/include/libusb-1.0. At the end press the ‚c‘ key a second time followed by pressing ‚g‘. If the console is promoted again, start the compile process with :
cmake ..
make
sudo make install
Congratulations, if no errors appear, you are done. You can test the driver and lib by changing into the ‚wrappers‘ driectory. By typing
cd ../wrappers/python
There you have to install the python libs of libfreenect.
sudo python setup.py install
After that you are ready to go. Give it a try by starting some of the example Applications. For example the async demo.
sudo python demo_cv_async.py
These program examples are not capable to handle multiprocessing, it works and if you use a single stream example you are able to get a fine depth image stream. I am working and experimenting with some multiprocessing code i’ve already written.
Since i am very interested in 3D Scanners i will do further work on it this summer. First i have to finish my other 3D Laser Scanner project https://hci.rwth-aachen.de/fabscanpi
Feel free to comment! And have fun.
Wenn du es schaffen würdest das man mit dem Teil eine fertige STL in guter Qualität heraus bekommen würde, wäre das der absoulute Oberhammer! Viel Erfolg dabei, ich bleib auf jeden Fall dran!!!
Ich bleibe dran. Wird aber noch eine Weile dauern bis die ersten Ergebnisse da sind. Zur Zeit arbeite ich an der neuen offiziellen Software für den Fabscan 3D Scanner.
Would this also work with an Asus Xtion Pro Live? If so, how would the software setup be different?
I think, if Asus Xtion Pro Live is fully supported by libfreenect it should work without any problems.
hi i dont succese to download the libfreenect do you know if i can to download from anuther site?
Have you tired github https://github.com/OpenKinect/libfreenect ?
hi,Lukas,I’ve run the program perfectly on a Raspberry Pi 2. But when I use a RPi 3 model B, I have from time to time been able to connect to the Kinect.However the rest of the Time I get this error and I’m totally stumped as to how to fix it.
Here is the error:
send_cmd: Input control transfer failed (18)
freenect_fetch_reg_info: send_cmd read 18 bytes (expected 118)
send_cmd: Output control transfer failed (-1)
freenect_fetch_reg_const_shift: send_cmd read -1 bytes (expected 8)
freenect_camera_init(): Failed to fetch const shift for device
Error: Invalid index [0]
Error: Can’t open device. 1.) is it plugged in? 2.) Read the README
hello good morning, that’s because you are not getting enough current either to the kinect or the raspberry.
Try restarting the raspberry and changing the charger that you use for it.
Hi Mario,
Really nice project ! Do you think it could work with a Rpi B (first generation ). I do not see any restriction however I prefer ask before to buy all the stuff I need 😉
Thanks,
Stéphane
I think a RpiB does not have enough power to work with those lib’s. And remember there is no software which will do a 3D scan now. It’s only a demo wich shows that i got the kinect one working on a Rpi with a display.
At late 2015 i will try to write a 3D scanning solution for the Rpi with kinect.
Thanks for your answer Mario 😉
Hi Mario,
I forgot something this video and the research paper linked could interest you : https://www.youtube.com/watch?v=XD_UnuWSaoU . I have to read to understand what is exactly happening, but looks really fun ! And whit your Xbox gun, it could be so fun. If I make one of this gun, I will work on something similar 😉
Have fun !
Saorel
I can confirm that it sorta, kinda works with a RPi 1 model B, but the processing speed is far too low. Depth frames can be unsteadily aquired at 1 per 10 seconds using the sync demo.
Yes the RPi 1 model B is to slow.
Hi
Have You try to write a 3D scanning solutuon for Rpi with kinect?Any conclusion ?
[…] can check out the complete documentation for Lukas’ Portable Kinect Scanner project on his site and on […]
[…] can check out the complete documentation for Lukas’ Portable Kinect Scanner project on his site and on […]
Hallo Mario,
ich habe schon mal bei der Skanect Gruppe gefragt, ob man mit einem Raspi nur einen Scan aufnehmen und dann auf einem PC berechnen könnte.
You can fairly easily generate .skn folders though. The file format is only using standard compression and folders. The hardest is the depth file format, but there is actually open source code to write that in github.com/rgbdemo . If you would like to go down that path, I think I described the format in the forum in the past, but if you can’t find it I can provide more help.
Link
Vielleicht eine Zwischenlösung?
Hi,
Danke für den Hinweis. Ich weiss nicht ob es mit Skanect möglich ist. Selbst habe ich mit Skanect noch keine Erfahrung sammeln können. Gibt es denn offene Schnittstellen? Wäre auf jeden Fall ein interessanter Ansatz. Bei diesem Projekt versuche ich es jedoch auf eine andere Art und Weise.
Gruss
Mario
This is what im thinking to do right now either skanect or kinect fusion. Did you find any Progress on this idea?
cheers
[…] Blog de Mario Lukas […]
thank you for taking the time to write this up, i look forward to reading any progress you make
[…] tedesco Mario Lukas, tuttavia, pare aver risolto entrambi questi i problemi in un colpo solo, grazie ad un’intuizione semplice ma geniale; utilizzare un Raspberry Pi (con annesso […]
Hi,
I have the same hardware as you (Kinect for Windows, Raspberry Pi 2), and have followed your steps exactly, even on a new Raspbian image (2015-05-05), but all programs (including wrappers/python/demo_cv_async.py) return the following error message from libfreenect:
„Could not open audio: -4
Failed to open motor subddevice or it is not disabled.Failed to open audio subdevice or it is not disabled.Error: Can’t open device. 1.) is it plugged in? 2.) Read the README“
1) Of course it is plugged in, the xxorde/librekinect driver does work. 2) I have read the README, but nothing seems to be useful. I have tried both running it as superuser, and adding the corresponding udev rules, to no avail. I have also blacklisted gspca_main and gspca_kinect.
lsusb returns:
Bus 001 Device 011: ID 045e:02bf Microsoft Corp.
Bus 001 Device 019: ID 045e:02be Microsoft Corp.
Bus 001 Device 008: ID 045e:02c2 Microsoft Corp.
Upon connecting the Kinect, dmesg shows:
[ 2684.939028] usb 1-1.5.1: new high-speed USB device number 18 using dwc_otg
[ 2690.428926] usb 1-1.5.1: new high-speed USB device number 19 using dwc_otg
[ 2690.531017] usb 1-1.5.1: New USB device found, idVendor=045e, idProduct=02be
[ 2690.531043] usb 1-1.5.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 2690.531062] usb 1-1.5.1: Product: Microsoft Kinect Audio, \xffffffc2\xffffffa9 2011 Microsoft Corporation. All rights reserved.
[ 2690.531079] usb 1-1.5.1: Manufacturer: Microsoft
[ 2690.531096] usb 1-1.5.1: SerialNumber: A22592W00346213A
What can possibly go wrong? Note that the Kinect works correctly with libfreenect on an Ubuntu 14.04 PC.
Sorry i am very busy at the moment i will check that issue on the upcoming weekend.
I have finally found a solution to the issue, see this for more details:
https://github.com/OpenKinect/libfreenect/issues/447#issuecomment-121212613
Thanks.
Fine. I will start to work on the scanning Software this weekend.
[…] details on how to build your very own Raspberry Pi 3D scanner jump over to the Mario Lukas website for full instructions as well as a list of all the components you will require to build […]
[…] el blog de Mario Lukas podemos encontrar las instrucciones precisas para fabricar este escáner 3D de mano, aunque lo más […]
[…] el blog de Mario Lukas podemos encontrar las instrucciones precisas para fabricar este escáner 3D de mano, aunque lo más […]
[…] Details und die vollständige Installationsanleitung findet ihr auf Marios Blog. […]
[…] du cœur, nous pouvons nous aussi fabriquer un scanner 3D. Toutes les instructions sont données sur son blog. Il ne reste plus qu’à les […]
I’ve followed this through, but when i try and run one of the demos i get met with this error message: „GdkGLExt-Warning **: Window system doesn’t support OpenGL.“ can anyone help?
Hi Ben Coope!
Have you already solved the issue. If yes please let me know.
I had the same problem. For me
sudo apt-get install libgl1-mesa-dri
and then
sudo reboot
Worked fine !
Thank you so much for this hint. If one of the other guys with the same issue can confirm that this works i will add it to the blog entry.
I have the same issue
Sorry guys. At the moment i have no time to work on this project until i have released the FabScan Pi Software. So i also have not the time to do something in your error cases. The instructions in the article are for wheezy. Try to use the same libusb etc. In general that should work. But i have also good news. A couple of weeks ago i compiled some software for 3D Scanning with the pi. The software compiles well and is executable, but i had no luck with connecting the kinnect with this software due the mentioned lack of time. Keep rationed, i will do further work on this project soon.
Hi Marko,
It is not a problem at all, we are all really busy! Thank you very much for your work !
Could you let us know which softwares you compiled a month ago for the pi ?
Cheers
S
Hi Mario! I’m having issues with the portion after creating the build directory. ccmake and cmake are „not found“, and entering „make“ does nothing- says there’s nothing to be done for ~/libfreenect. Any idea what’s going on! I love your work!
didi u dowlnload the cmake n ccmake util?
Hi Mario,
I have done your Project and successfully have a Depth Image stream. I have also found a way to record this stream using the Fakenect-record Util.
Now I would like to shift this data to some 3d Scan Apps liek Kinect Fusion, Skanect or Arztec. But im not sure how. Im a begiiner at Programming and realy excited to make this real.
I have also order the new RP3 n will try out on it . Hopefully the new Rasbian will hv a OpenGL driver too.
How is the Pprogress of yours and do you have any insight that could help me to realise this.
Cheers.Your big fan,Mohandez
Hi,
getting the stream running was just a proof of conept. I wanted to try if the kinect works reasonbale with the Raspberry Pi2. I switched to the Raspberry Pi3. I am porting an old Kinect 3D scanner project, which is written in c++. I am just fixing some code, afterwards i will give it a try. When it woks as i imagine the sotware should be able to do the whole scanning process on a Raspberry.
cheers
Mario
Thats great. Pls let us know ur progress and ur insights on your new approach. Your guidence is very helpful. Im planning to do a Bachelor Project to go deeper on this POC.
Hope we could make this real. Cheers Mario.
Hey Mario,
I know you are busy with other projects, have you had any success with running this on the Raspi 3? Also do you believe the Raspi 3 is powerful enough to actually scan and register the data to create large scale point clouds? I know this was a concern with the Raspi 2. I am very interested in working on this.
Best,
Christian
Hi Christian,
at the moment i am working a lot on the FabScanPi Software (fabscan.org).
The good news are that i refactored the FabScan software in the last weeks in a way that the module for the
scan process can be switched. What means that different scan algorithms can be implemented as a module. The
rest of the software can be used as a base for scanners in general.
I am planing to write a module for the Kinect thing in the upcoming month. So the kinect scanner will be
a part of the FabScan Software in the future.
– Mario
Hi,
have you tried running the kinect on the Raspberry Pi 3 yet? I got my R Pi 3 a few weeks back and wanted to see what frame rate it could I could get with Kinect. It works but I’m getting a random number of connection errors each time I restart my capture program. It runs great once connected and I guess I’m getting about double the frame rate over the R pi 2B. But at this point I’m not sure if I have a faulty USB chip or a code bug from the hardware upgrade. I’m using Freenect installed from the raspbian repository, but I’ve also compiled on the rpi 3 along with libusb (as above) .
grateful for any help, cheers.
Just for the record, I am also following this project with great anticipation. My kinect and rpi3 are already here. We just need a 3d-scanning software… 🙂
Hi,
I’m pretty desperate to confirm if anyone has tested the kinect with the Raspberry Pi 3 and found the above mentioned bug or alternatively is able to confirm everything is working fine. So I can confirm or reject having faulty hardware.
thanks again for any help.
Hey Mario,
Have you had anytime to work on this project?
I am extremely interested in further development on this.
Best,
Christian
I’m searching for a handheld 3D scanner
but can not really find anything
your idea is very nice
You already further along
There are somewhere building instructions
and can be stored in .stl
[…] Proof of concept 3D Scanner with Kinect and Raspberry Pi2 […]
I can’t get it to work with the RPi 3 B. whenever I call te python demo or freenect-glview or anything the device disconnects and connects again (as seen in dmesg):
[ 1030.146843] gspca_main: kinect-2.14.0 probing 045e:02ae
[ 1030.392686] usb 1-1.5.3: USB disconnect, device number 19
[ 1033.186257] usb 1-1.5.3: new high-speed USB device number 20 using dwc_otg
[ 1033.289621] usb 1-1.5.3: New USB device found, idVendor=045e, idProduct=02ae
[ 1033.289641] usb 1-1.5.3: New USB device strings: Mfr=2, Product=1, SerialNumber=3
[ 1033.289654] usb 1-1.5.3: Product: Xbox NUI Camera
[ 1033.289666] usb 1-1.5.3: Manufacturer: Microsoft
[ 1033.289679] usb 1-1.5.3: SerialNumber: A00365A20562136A
[ 1033.290613] gspca_main: kinect-2.14.0 probing 045e:02ae
[ 1059.953277] usbcore: deregistering interface driver kinect
[ 1070.077954] usb 1-1.5.3: USB disconnect, device number 20
[ 1073.127019] usb 1-1.5.3: new high-speed USB device number 21 using dwc_otg
[ 1073.230444] usb 1-1.5.3: New USB device found, idVendor=045e, idProduct=02ae
[ 1073.230463] usb 1-1.5.3: New USB device strings: Mfr=2, Product=1, SerialNumber=3
[ 1073.230476] usb 1-1.5.3: Product: Xbox NUI Camera
[ 1073.230489] usb 1-1.5.3: Manufacturer: Microsoft
[ 1073.230502] usb 1-1.5.3: SerialNumber: A00365A20562136A
[ 1074.275733] media: Linux media interface: v0.10
[ 1074.288243] Linux video capture interface: v2.00
[ 1074.291137] gspca_main: v2.14.0 registered
[ 1074.292075] gspca_main: kinect-2.14.0 probing 045e:02ae
[ 1074.292529] usbcore: registered new interface driver kinect
[ 1182.449373] gspca_main: kinect-2.14.0 probing 045e:02ae
[ 1182.476952] usb 1-1.5.3: USB disconnect, device number 21
[ 1185.528889] usb 1-1.5.3: new high-speed USB device number 22 using dwc_otg
[ 1185.632308] usb 1-1.5.3: New USB device found, idVendor=045e, idProduct=02ae
[ 1185.632327] usb 1-1.5.3: New USB device strings: Mfr=2, Product=1, SerialNumber=3
[ 1185.632341] usb 1-1.5.3: Product: Xbox NUI Camera
[ 1185.632353] usb 1-1.5.3: Manufacturer: Microsoft
[ 1185.632366] usb 1-1.5.3: SerialNumber: A00365A20562136A
[ 1185.633435] gspca_main: kinect-2.14.0 probing 045e:02ae
Output of freenect-glview:
Kinect camera test
Number of devices found: 1
send_cmd: Input control transfer failed (18)
freenect_fetch_zero_plane_info: send_cmd read 18 bytes (expected 322)
freenect_camera_init(): Failed to fetch zero plane info for device
Could not open device
Device 0x2012478 open during shutdown, closing…
I have sunk several days into this now. Can anyone help me?
Hello
Very nice work !!!
Did you make any progress on the softward to creta stl files ?
[…] Another Kinect + Raspberry pi […]
Hi! Do you have some kind of article about your project? I want to inclued it on my master’s thesis. Thanks
I don’t have any other Blog post about the project here. But you can find more when you google for it „kinect portable 3d scanner“
Example gives: https://3dprint.com/73090/kinect-portable-3d-scanner/
Hey, hast du das projekt mittlerweile fertig gestellt? Sprich kannst du nun 3d scannen?
[…] Información para crear un escáner 3D. […]
[…] Kinect dont vous ne vous servez plus dans votre placard et un Raspberry Pi, vous pourrez, en connectant les deux appareils, créer un scanner 3D et modéliser n’importe quel objet. Assez pratique si vous avez une imprimante 3D ! Nous […]
Hello der sir,
I’ve followed steps you posted, but after typing
sudo python setup.py install i got errors:
https://i.imgur.com/xY38zhh.png all around.
Can you help me with this error? I would really like to test kinect and raspberry pi.
[…] Proof of concept 3D Scanner with Kinect and Raspberry Pi2 […]
[…] 2. Kinect, Raspberry Pi 2 Handheld 3D-scanner […]
Does this work with raspberry 3?
after following all the installation described without errors, I see this message:
(Depth: 27323): GTK-WARNING **: can not open display.
I have a kinect xbox360 model 1473 on a raspberry pi3. What could be the problem?
Mario where are you? Help!!!
[…] by: Mario Lukas Project Website: https://www.mariolukas.de/2015/04/proof-of-concept-3d-scanner-with-kinect-and-raspberry-pi2/ 3D scanning has changed the medical field, design world, and started some fun hobbies. This […]
[…] 3D scanner […]
I was able to install and run the example script above on my Kinect by running
sudo apt install freenect python-freenect python-opencv
Hope it saves some other ppl some time installing
[…] relacionado con este que además nos sirve de base para utilizar Machine Learning es este scanner 3D usando Raspberry Pi y Kinect (de la XBox). Los sensores Kinect están ahora a muy buen precio de segunda mano en tiendas de videojuegos y son […]
[…] relacionado con este que además nos sirve de base para utilizar Machine Learning es este scanner 3D usando Raspberry Pi y Kinect (de la XBox). Los sensores Kinect están ahora a muy buen precio de segunda mano en tiendas de videojuegos y son […]
Hi
Is this project dead…. did you finish it or did you drop it..
I would love to use my new rpi 4 with this. I know you are a busy man…
I hope the project is still on
I would say that it is not dead… but I changed my priorities. At the moment I am working on the FabScanPi project nearby every day. My vision is to bring the FabScanPi project and Kinect like depth sensors together some day. But the fact that I am lone makes it hard to get such big features contemporary. I know the project is a couple of years old and I also see a huge potential in using the rpi4 for it.
This is awesome! Have you tried this on a pi4?
Will the project be continued? It promised to be sensational and unfortunately it is dying!
Interesting work that you have been doing, I hope you can continue investing your time in those great ideas that have too much benefit I think for you as for us We hope to continue seeing your work, in my case I
I always like to work with a good signal to speed up the tasks, and thus I make better use of my time.
regards
[…] one of the more complicated but more amazing projects out there. Mario Lukas first created his proof of concept product 5 years ago and has now created a full product from it called Fabscan. You can either build it yourself […]
Can you use any screen and will this work with rpi 4?
Hi,
Is this project alive or dead? I really whant a portable scanner for scanning „stuff“ (don’t ask).
/Th3Tink3r3r
Not dead at all, but also no progress for years… 🙂