ECE497 Project: Kinect
I'm using the following template to grade. Each slot is 10 points. 0 = Missing, 5=OK, 10=Wow!
00 Executive Summary 00 Installation Instructions 00 User Instructions 00 Highlights 00 Theory of Operation 00 Work Breakdown 00 Future Work 00 Conclusions 00 Demo 00 Late Comments: Score: 00/100
Our project is using kinect to accomplish a kind of "Gesture Game", which is operated through our gestures and motions. Compared to the traditional games, the gesture game can provide better interactive enjoy without the complex operation from the keyboard and mouse.
Through our previous work, we have alread completed the configuration of our project successfully. The configuration work includes the configure of the Beagle Board and the configure of the QT IDE. Now, we can run the Tracking Demo and the Gesture Demo on the Beagle Board. We can also download our projects to the Beagle Board through cross-compiling.
Our work ahead is looking for an interative game, and transplant the game to the QT platform. Then, download the game to the Beagle Board. In that case, the game can be played on the Kinect.
Until now, we have a good begining with the demos runs succefully. We still need some time to finish the following work.
We are using the drivers provided by Omek. The driver files can be found at the Beckon™ SDK 2.4 – BeagleBoard-xM Edition installation page and a developer's guide is provided at the documentation page. (You may need to register first.) Instructions in this blog are also very useful.
Although the developer's guide introduces how to do the installation and configuration, we have found that it takes much longer than we originally thought and errors came out occationally. It seems that the developer's guide is not very well organized and detailed enought anyway. So we arranged our instruction and configuration procedures as below:
1. Download the image package and burn it into the SD card
Download the image package Beckon-Image-2.4.19872.tar.gz as mentioned above onto your Linux PC. Then extract it by entering
host$ tar -xvzf <ImageFileName>
Run the following command to make sure you have bash installed.
host$ sudo apt-get install bash
Insert your SD card using a card reader/writer. Run the following command:
host$ sudo bash mkcard.sh /dev/<sdX> beagle-omek
Note: This action will wipe out and rewrite the SD card so make sure the files in it are backed up. The name of the SD card can be viewed in Disk Utility. If you mistakenly use the main OS device name this script can erase your entire OS.
2. Run the sample application
Plug the sensor into the BeagleBoard-xM USB port. Ensure the mouse and keyboard are connected to the BeagleBoard-xM USB port. Connect the BeagleBoard-xM to the monitor with an HDMI cable. Connect the BeagleBoard-xM power supply and wait for the boot process to reach the login prompt, login as user root (no password is required). Now it's ready to run the sample applications on the BeagleBoard-xM.
Note: Details about running the samples can be found in the developer's guide from page 9 to 11. The Kinect will react properly only if the player is within an appropriate distance (about 1m - 5m).
3. Setting up your Linux development PC
Download Beckon-SDK-2.4.19872.tar.gz and extract it onto your host PC (this is the actual SDK which contains headers, libraries, documentation, makefiles, etc...) The SDK provides the tools to develop a motion-tracking and gesture-based application that can run on the BeagleBoard-xM. After developing the application, cross-compile it onto the Linux PC and then copy the image onto the BeagleBoard-xM via external USB storage.
1) Download the Linux ARM cross toolchain:
For 32-bit Ubuntu you will want to download this file: http://www.angstrom-distribution.org/toolchains/angstrom-2011.03-i686-linux-armv7a-linux-gnueabi-toolchain-qte-4.6.3.tar.bz2
For 64-bit Ubuntu you will want to download this file: http://www.angstrom-distribution.org/toolchains/angstrom-2011.03-x86_64-linux-armv7a-linux-gnueabi-toolchain-qte-4.6.3.tar.bz2
Extract the downloaded archive into your system root directory:
host$ sudo tar -xvjf <ToolchainFileName> -C /
Note: If you extract the archive into a different directory, you should adjust the following instructions as well as the samples' makefiles accordingly.
2) Install the Qt embedded SDK for Angstrom/BeagleBoard:
Now that you have your cross compiler installed, you can compile Qt with it. First thing to do is to install zlib library:
host$ sudo apt-get install zlib1g-dev
Then download the Open Source LGPL version of Qt libraries 4.6.2 for Embedded Linux [get.qt.nokia.com/qt/source/qt-everywhere-opensource-src-4.6.2.tar.gz here].
Note: Newest version of Qt libraries can be found here. We didn't test it since the 4.6.2 version works fine.
Extract the file you just downloaded:
host$ tar -xvzf <DownloadFileName>
You need to create a new make.conf. Run the following line to make a new mkspecs directory for the BeagleBoard Processor.
host$ cp -R <DownloadDirectory>/mkspecs/qws/linux-arm-g++/ <DownloadDirectory>/mkspecs/qws/linux-DM3730-g++/
Edit the qmake.conf file found in <DownloadDirectory>/mkspecs/qws/linux-DM3730-g++/ and overwrite it with the following lines.
# # qmake configuration for building with arm-linux-g++ # include(../../common/g++.conf) include(../../common/linux.conf) include(../../common/qws.conf) # modifications to g++.conf #Toolchain #Compiler Flags to take advantage of the ARM architecture QMAKE_CFLAGS_RELEASE = -O3 -march=armv7-a -mtune=cortex-a8 -mfpu=neon -mfloat-abi=softfp QMAKE_CXXFLAGS_RELEASE = -O3 -march=armv7-a -mtune=cortex-a8 -mfpu=neon -mfloat-abi=softfp QMAKE_CC = /usr/local/angstrom/arm/arm-angstrom-linux-gnueabi/bin/gcc QMAKE_CXX = /usr/local/angstrom/arm/arm-angstrom-linux-gnueabi/bin/g++ QMAKE_LINK = /usr/local/angstrom/arm/arm-angstrom-linux-gnueabi/bin/g++ QMAKE_LINK_SHLIB = /usr/local/angstrom/arm/arm-angstrom-linux-gnueabi/bin/g++ # modifications to linux.conf QMAKE_AR = /usr/local/angstrom/arm/arm-angstrom-linux-gnueabi/bin/ar cqs QMAKE_OBJCOPY = /usr/local/angstrom/arm/arm-angstrom-linux-gnueabi/bin/objcopy QMAKE_STRIP = /usr/local/angstrom/arm/arm-angstrom-linux-gnueabi/bin/strip load(qt_config)
Now run the configure command (see this website for more details if needed):
For 32-bit Ubuntu:
host$ ./configure -opensource -confirm-license -prefix /opt/qt-arm -no-qt3support -embedded arm -little-endian -xplatform qws/linux-DM3730-g++ -qtlibinfix E
For 64-bit Ubuntu:
host$ ./configure -opensource -confirm-license -prefix /opt/qt-arm -no-qt3support -embedded arm -little-endian -platform qws/linux-x86_64-g++ -xplatform qws/linux-DM3730-g++ -qtlibinfix E
Once this is complete you can run make and make install (make may take some hours):
host$ make host$ sudo make install
3) Install and set up Qt Creator:
Go to http://qt-project.org/downloads to download Qt. Get the latest version of Qt for Linux/X11; be sure to get the appropriate 32bit or 64bit version. Once downloaded go to the download directory and execute the following using your download file name:
host$ chmod u+x <QtCreator.bin> host$ ./<QtCreator.bin>
To Li Xia: Qt Creator has commercial version on http://qt.nokia.com/downloads . I'm using it but I can only try it for 30 days. I just find the free version above but haven't tried it. Please try the free version and see if it works. If nothing's wrong, just simply remove this paragraph afterwards.
Start Qt Creator. Go to: Tools > Options > Build & Run > Tool Chains, select Add -> GCC and insert the full compiler path below:
Then go to: Tools > Options > Build & Run > Qt Versions, select Add and provide the following full path to qmake
Press Apply and OK.
4. Setting up the BeagleBoard-xM
All of the Qt files that you need to move to the BeagleBoard can be found in the /opt/qt-arm/lib directory on your Linux Build System. Copy all the files out of your build system's /opt/qt-arm/lib directory and put those into the BeagleBoard-xM's /opt/qt-arm/lib directory. You can get the files to the Beagleboard by either networking, using a usb drive or plug the SD card into your Linux Build System.
- Include your github path as a link like this: https://github.com/MarkAYoder/gitLearn.
- Include any additional packages installed via opkg.
- Include kernel mods.
- If there is extra hardware needed, include links to where it can be obtained.
Theory of Operation
We are using the kinect as the sensor of our project. The provided algorithm of the kinect make it possible to recognize our gesture and motion. In this case, the gesture and motion can take the place of the mouse. Our project is running with the idea.
We can use QT to design our program and then cross-complie our project to the beagle-board. According to the cross-compiling of the sample project: Tracking-SDK, we have got a pretty good demo. Hence, it is possible for us to design an interesting gesture game on the QT platform and then run it on the beagle board with the kinect.
- Previous Work Breakdown:
- Researched how to Burn the correct Image to the Beagle Board.
- Configure the Beagle Board to control the kinect.
- Researched how to configure the QT programming environment.
- Realize the cross-compiling on QT.
- Researched how to run the provided demo.
- How to speed up the demo.
Our future work is to design an interactive game which can be controlled through our gestures.
Give some concluding thoughts about the project. Suggest some future additions that could make it even more interesting.
Embedded Linux Class by Mark A. Yoder