BeagleBoard/GSoC/2021 Proposal/TensorFlow Lite Compatibility with BeagleBone AI
Student: Leah Pillsbury
Mentors: Who else is interested in this?
Code: none yet, lots to follow!
Wiki: http://elinux.org/BeagleBoard/GSoC/2021 Proposal/TensorFlow Lite Compatibility with BeagleBone AI
This project is currently just a proposal.
School: Pasadena City College and Boston University before that
Country: United States
Primary language : English
Other languages : Spanish, Swahili, Hebrew, some Hindi/Urdu, Telugu, Bengali
Typical work hours : 9AM-5:30PM US Pacific, though I may be on Eastern time part of the summer
Previous GSoC participation: I haven't ever participated in GSOC; I am interested to do so to learn more about embedded software, improve my coding skills, and participate in the community. I use open source tools all the time, and it is really exciting to make something useful that other people would use too.
About your project
Project name: TensorFlow Lite Compatability with BeagleBone AI
While the specs of BeagleBone AI are awesome, it seems limiting for an AI board for the general public to not be able to access TensorFlow Lite. I would like to change that by choosing a current release of TensorFlow, and writing the code necessary to run it on the Arm M4 processors that the BeagleBone AI board has, and that should work with TensorFlow Lite. This project will be a combination of C coding, dealing with the Linux kernel, and doing some example cases with TensorFlow Lite in python. Obviously, given the diversion between OpenCL and TensorFlow, there is no perfect way to seamlessly integrate TensorFlow Lite, but there seem to be enough people in the community who want it, that it is worth at least making a gesture towards a compatibility solution. I first realized this was a problem when I decided I wanted to learn more about AI on the edge. One of the first things that comes up when I search for information on this topic is using TensorFlow Lite and the wide variety of other boards that it plays well with.
Ideally, I'd spend the first few weeks getting TensorFlow Lite working on a BBAI, document my hacking process, and then create a smooth stable way to make it work more out of the box. Then I'd spend the rest of the time doing examples that would be highlighted both on the BeagleBoard website, and also on the TensorFlow Lite Examples page by making a pull request to the owners of the github repo. Given that there are already examples of using TensorFlow Lite on Raspberry Pi, with picamera, the starting point would be to have equivalent BeagleBone examples, most likely doing image capture through OpenCV (OpenCV with BeagleBone Black). I'm also very interested in sound recognition, and if time allows I'd also like to do some examples in this area.
Provide a development timeline with a milestone each of the 11 weeks and any pre-work. (A realistic timeline is critical to our selection process.)
|Apr 8||Found out about exciting projects at Beagle Board, joined the community!|
|Apr 13||Proposal complete, Submitted to https://summerofcode.withgoogle.com|
|May 17||Proposal accepted or rejected|
|All of May||Prework: * Learn more about Linux Kernels
|Jun 07||Pre-work complete, Coding officially begins!|
|Jun 17||Milestone #1, Introductory YouTube video|
|June 24||Milestone #2, Working version of TensorFlow Lite on a BeagleBone AI|
|June 30||Milestone #3, Smooth(er) process and documentation for others to use TensorFlow Lite on BeagleBone AI|
|July 12 18:00 UTC||Milestone #4, Is Milestone #3 as user friendly as possible? If not, improve it|
|July 23||Milestone #5, Example Cases on BeagleBone AI (vision)|
|July 30||Milestone #6, Example Cases on BeagleBone AI (sound)|
|Aug 06||Milestone #7, explore whether TensorFlow Lite can also be integrated with the BeagleBoard-X-15, and/or whether|
|August 10||Milestone #8, Completion YouTube video|
|August 16 - 26 18:00 UTC||Final week: Students submit their final work product and their final mentor evaluation|
|August 23 - 30 18:00 UTC||Mentors submit final student evaluations|
Experience and approach
In 5-15 sentences, convince us you will be able to successfully complete your project in the timeline you have described.
What will you do if you get stuck on your project and your mentor isn’t around?
BeagleBoard options are strong contenders with Raspberry Pi and Arduino as tools that are easy to use to do IoT and industrial projects. While BeagleBones are geared more towards engineers and have some special features that actually make them quite different than the others (PRUs, ability to do low level and high level control simultaneously, power usage etc), they often get compared with the other frameworks. Given that Raspberry Pi and Arduino have TensorFlow Lite compatibility, I think it is important for BeagleBones to do the same. Also, there are algorithms that are easier to access with TensorFlow Lite than in other settings.
I tried tackling YOLO deployment on BBAI but it is currently impossible because of TIDL library restrictions. TFLite is the way to go with DL on BBAI and it uses TIDL underneath
- -Jakub Duchniewicz
and several frustrated users have posted on the internet:
I've been attempting to build the TF Lite library on my BeagleBone AI (32-bit ARMv7 MCU) for several days now, to no avail.
I wish to run my network on the AM5729 found on the BeagleBone AI board. I understand that the ti-processor-sdk-linux-am57xx-06.03.00.106 does not yet support the BeagleBone AI tool. However, the Debian image distributed for the BeagleBone AI has all the TIDL libraries packaged with it.
Therefore the only thing I need to be able to do is convert the tensorflow lite network into the TIDL format using the TIDL import tool. I assume I can do this using the following flow: 1. download the ti-processor-sdk-linux-am57xx-06.03.00.106-linux-x86-Install.bin from TI 2. run the installer on a linux (Ubuntu) host 3. navigate to the tidl_model_import.out tool 4. run the tidl_model_import.out tool on an appropriate configuration file (below)
- -Alex Beasley
The error is because TIDL does not support importing tensorflow lite model. Please try with Caffe/tensoflow/onnx models, for more details refer to TIDL datasheet/user-guide.
Please complete the requirements listed on the ideas page. Provide link to pull request.
Is there anything else we should have asked you?