Jetson/Installing OpenCV

From eLinux.org
Jump to: navigation, search

Installing OpenCV (including the GPU module) on Jetson TK1

First you should download & install the CUDA Toolkit by following the Installing CUDA instructions, since it is needed by OpenCV.

And if you haven't already added the "universe" repository to your system, do it now since you will need it for some OpenCV dependencies:

sudo apt-add-repository universe
sudo apt-get update

Once you have installed the CUDA Toolkit, you can begin to install OpenCV. You have several options (deciding between prebuilt library vs building the library from source, and then deciding whether to compile your code onboard the Jetson TK1 or cross-compile it from your desktop).

Prebuilt OpenCV library versus Building the OpenCV library from source

There are two compatible implementations of OpenCV available for Jetson TK1, but only the first option gives you the CPU optimized library, so you are highly recommended to use this first option:

Option 1) Prebuilt OpenCV4Tegra library for L4T:

OpenCV4Tegra is a CPU & GPU optimized version of OpenCV, available as a prebuilt library in JetPack or as separate .deb packages (available for L4T rel-19 and L4T rel-21).

eg: For OpenCV v2.4.10.1, run the following commands to manually install the prebuilt packages:

Under L4T 19.x:

sudo dpkg -i libopencv4tegra_2.4.10.1_armhf.deb
sudo dpkg -i libopencv4tegra-dev_2.4.10.1_armhf.deb

Under L4T 21.x:

sudo dpkg -i libopencv4tegra-repo_l4t-r21_2.4.10.1_armhf.deb  (does not work, please fix!)
sudo apt-get update
sudo apt-get install libopencv4tegra libopencv4tegra-dev

Continue to the "Testing OpenCV" section lower in this page to ensure OpenCV4Tegra works on your device.

Option 2) Building the public OpenCV library from source:

If you want the latest OpenCV code, or you want to customize the OpenCV library, and don't want NVIDIA's CPU & multi-core optimizations of OpenCV4Tegra, then follow the instructions below to compile the OpenCV library from source code. Note that you won't obtain the full performance of OpenCV4Tegra.


Note about SIFT/SURF in the nonfree module: OpenCV4Tegra doesn't include the opencv_nonfree package (containing SIFT & SURF feature detectors) since those algorithms are patented by other companies and therefore anyone using opencv_nonfree is at risk of liability.

If you need something from the nonfree module, you have 2 options:

  1. Analyze the public OpenCV source code then copy/paste the parts of the nonfree module that you want (eg: SURF feature detector) from OpenCV into your own project. You will have the CPU optimizations of OpenCV4Tegra for most of your code and will have the GPU module and will have the non-optimized patented code that you need from the nonfree package such as SURF. So this option gives full performance (for everything except the nonfree code) but is tedious.
  2. Ignore OpenCV4Tegra, and instead, download & build public OpenCV (by following the instructions below for natively compiling the OpenCV library from source). You will still have the GPU module but not any CPU optimizations, but you won't need to spend time ripping out parts of the OpenCV non-free module code. So this option is easiest but produces slower code if you are running most of your code on CPU.

Native vs Cross-development

Just like with the CUDA development guide, you have two options for developing OpenCV applications for Jetson TK1:

  • native compilation (compiling code onboard the Jetson TK1)
  • cross-compilation (compiling code on an x86 desktop in a special way so it can execute on the Jetson TK1 target device).

Native compilation is generally the easiest option, but takes longer to compile, whereas cross-compilation is typically more complex to configure and debug, but for large projects it will be noticeably faster at compiling.

Natively compiling the OpenCV library from source onboard the device

Note: Compiling OpenCV from source will give you NVIDIA's CUDA GPU optimizations but not NVIDIA's ARM NEON CPU optimizations that are only available in the closed-source prebuilt OpenCV4Tegra packages.

If you haven't added the "universal" repository to Ubuntu, then do it now:

sudo add-apt-repository universe
sudo apt-get update

Now you need to install many libraries:

# Some general development libraries
sudo apt-get -y install build-essential make cmake cmake-curses-gui g++
# libav video input/output development libraries
sudo apt-get -y install libavformat-dev libavutil-dev libswscale-dev
# Video4Linux camera development libraries
sudo apt-get -y install libv4l-dev
# Eigen3 math development libraries
sudo apt-get -y install libeigen3-dev
# OpenGL development libraries (to allow creating graphical windows)
sudo apt-get -y install libglew1.6-dev
# GTK development libraries (to allow creating graphical windows)
sudo apt-get -y install libgtk2.0-dev

Also, if you think you will use Gstreamer (skip if you don't know what it is), install those dependencies:

# (Optional) Gstreamer development libraries (for Network IP camera input)
sudo apt-get install libgstreamer1.0 libgstreamer1.0-dev libgstreamer-plugins-bad1.0-0 libgstreamer-plugins-base1.0-0 libgstreamer-plugins-base1.0-dev

Download the source code of OpenCV (for Linux) onto your device. eg: Open the webpage of the latest release of OpenCV and click on the zip file, or from the command-line you can run this command on the device (but perhaps change 'VERSION' to a newer version of OpenCV):

VERSION=3.4.4
cd ~/Downloads
rm ${VERSION}.zip
wget https://github.com/opencv/opencv/archive/${VERSION}.zip

Unzip the OpenCV source code:

unzip ${VERSION}.zip
mv opencv-${VERSION} ~/

Configure OpenCV using CMake. (For compatibility with Tegra K1, use CUDA_ARCH_BIN="3.2", but you can increase this if you only care about newer Tegra's, such as CUDA_ARCH_BIN="7.2" if you only care about Tegra Xavier):

cd ~/opencv-${VERSION}
mkdir build
cd build
cmake -DWITH_CUDA=ON -DCUDA_ARCH_BIN="3.2" -DCUDA_ARCH_PTX="" -DBUILD_TESTS=OFF -DBUILD_PERF_TESTS=OFF ..

You can run cmake multiple times to add more settings, such as adding the contrib modules or using TBB and fast math for better performance at potential loss of some accuracy:

cmake -DCMAKE_BUILD_TYPE=RELEASE -DWITH_CUBLAS=ON -DENABLE_FAST_MATH=ON -DCUDA_FAST_MATH=ON -DWITH_TBB=ON ..

And if you want to also use the OpenCV "contrib" modules:

cd ~/Downloads
rm ${VERSION}.zip
wget https://github.com/opencv/opencv_contrib/archive/${VERSION}.zip
unzip ${VERSION}.zip
mv opencv_contrib-${VERSION} ~/
cd ~/opencv-${VERSION}/build
cmake -DOPENCV_EXTRA_MODULES_PATH=~/opencv_contrib-${VERSION}/modules ..

If you want to customize any more of the build settings such as whether to support Firewire cameras or Qt GUI, it is easiest to use the curses interactive version of CMake from here on:

ccmake ..

(Change any settings you want, then click Configure and Generate).


Now you should be ready to build OpenCV and then install it. Unfortunately, OpenCV is currently experiencing a problem with CMake where installing the built libraries (that normally takes a few seconds) re-compiles the whole OpenCV (that normally takes close to an hour). So to save time, instead of running "make -j4 ; make install", we will build & install OpenCV using a single command.

To build the OpenCV library using 4 Tegra CPU cores (takes around 40 minutes on Tegra K1):

make -j4

(Or on Tegra Xavier, run "make -j8" that takes around 25 minutes on Tegra Xavier)

To install the built OpenCV library by copying the OpenCV library to "/usr/local/include" and "/usr/local/lib":

sudo make install

Finally, make sure your system searches the "/usr/local/lib" folder for libraries, and update the library cache:

echo "# Use OpenCV and other custom-built libraries." >> ~/.bashrc
echo "export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib/" >> ~/.bashrc
source ~/.bashrc
sudo ldconfig

Testing OpenCV

Compile & run a few of the OpenCV sample programs to make sure OpenCV is working. To get the OpenCV sample codes, you can download the full OpenCV source code, or just download each sample individually from the CPP folder and the GPU folder. For simple programs you can just link to a few OpenCV libs, but for other programs you might as well link to all the OpenCV libs:

# Make sure we have installed a C++ compiler.
sudo apt-get install build-essential g++
# Test a simple OpenCV program. Creates a graphical window, hence you should plug a HDMI monitor in or use a remote viewer such as X Tunneling or VNC or TeamViewer on your desktop.
cd ~/opencv-2.4.9/samples/cpp
g++ edge.cpp -lopencv_core -lopencv_imgproc -lopencv_highgui -o edge
(Or for OpenCV 3.0: g++ edge.cpp -lopencv_core -lopencv_imgproc -lopencv_highgui -lopencv_imgcodecs -o edge)
./edge
# If you have a USB webcam plugged in to your board, then test one of the live camera programs and linking to every OpenCV module.
g++ laplace.cpp -lopencv_core -lopencv_imgproc -lopencv_highgui -lopencv_calib3d -lopencv_contrib -lopencv_features2d -lopencv_flann -lopencv_gpu -lopencv_legacy -lopencv_ml -lopencv_objdetect -lopencv_photo -lopencv_stitching -lopencv_superres -lopencv_video -lopencv_videostab -o laplace
(Or for OpenCV 3.0 with full GPU libs: g++ laplace.cpp -lopencv_adas -lopencv_bgsegm -lopencv_bioinspired -lopencv_calib3d -lopencv_ccalib -lopencv_core -lopencv_datasets -lopencv_face -lopencv_features2d -lopencv_flann -lopencv_highgui -lopencv_imgcodecs -lopencv_imgproc -lopencv_latentsvm -lopencv_line_descriptor -lopencv_ml -lopencv_objdetect -lopencv_optflow -lopencv_photo -lopencv_reg -lopencv_rgbd -lopencv_saliency -lopencv_shape -lopencv_stitching -lopencv_superres -lopencv_surface_matching -lopencv_text -lopencv_tracking -lopencv_videoio -lopencv_video -lopencv_videostab -lopencv_xfeatures2d -lopencv_ximgproc -lopencv_xobjdetect -lopencv_xphoto -o laplace)
(Or for regular OpenCV 3.0: g++ laplace.cpp -lopencv_calib3d -lopencv_core -lopencv_features2d -lopencv_flann -lopencv_highgui -lopencv_imgcodecs -lopencv_imgproc -lopencv_ml -lopencv_objdetect -lopencv_photo -lopencv_shape -lopencv_stitching -lopencv_superres -lopencv_videoio -lopencv_video -lopencv_videostab -o laplace)
./laplace
# Test a GPU accelerated OpenCV sample.
cd ../gpu
g++ houghlines.cpp -lopencv_core -lopencv_imgproc -lopencv_highgui -lopencv_calib3d -lopencv_contrib -lopencv_features2d -lopencv_flann -lopencv_gpu -lopencv_legacy -lopencv_ml -lopencv_objdetect -lopencv_photo -lopencv_stitching -lopencv_superres -lopencv_video -lopencv_videostab -o houghlines
./houghlines ../cpp/logo_in_clutter.png

If the houghlines GPU sample program worked then you have successfully installed OpenCV and its GPU module!

Getting to know OpenCV

Now you can start building your own projects using OpenCV on CPU and GPU, such as by following the Using OpenCV with gcc and CMake introduction tutorial then following many of the official OpenCV Tutorials, playing with the sample GPU programs in the samples/gpu folder of OpenCV and the many sample CPU programs in the samples/cpp folder. To get offline documentation and tutorials of OpenCV, the easiest way is to download the OpenCV Reference Manual PDF file, or if you want offline HTML docs then try building it:

sudo apt-get install python sphinx-common python-sphinx texlive-binaries
make html_docs

Then open the "doc/_html/index.html" page.

You can learn more about NVIDIA's Tegra hardware acceleration of OpenCV on the OpenCV Performance page, including a long list showing how much electrical power is used by Jetson TK1 for various OpenCV & computer vision sample programs.