Difference between revisions of "BeagleBoard/GSoC/2021 Proposal/TensorFlow Lite Compatibility with BeagleBone AI"

From eLinux.org
Jump to: navigation, search
(Benefit)
(ProposalTemplate)
Line 3: Line 3:
 
[[Category: GSoCProposal2021]]
 
[[Category: GSoCProposal2021]]
  
=[[BeagleBoard/GSoC/ProposalTemplate|ProposalTemplate]] =
+
=[[BeagleBoard/GSoC/Proposal/TensorFlow Lite Compatibility with BeagleBone AI | TensorFlow Lite Compatibility with BeagleBone AI]] =
  
 
About
 
About

Revision as of 20:22, 11 April 2021


TensorFlow Lite Compatibility with BeagleBone AI

About Student: Leah Pillsbury
Mentors: Who else is interested in this?
Code: none yet, lots to follow!
Wiki: http://elinux.org/BeagleBoard/GSoC/2021 Proposal/TensorFlow Lite Compatibility with BeagleBone AI
GSoC: N/A

Status

This project is currently just a proposal.

About you

IRC: lpillsbury
Github: lpillsbury
School: Pasadena City College and Boston University before that
Country: United States
Primary language : English
Other languages : Spanish, Swahili, Hebrew, some Hindi/Urdu, Telugu, Bengali
Typical work hours : 9AM-5:30PM US Pacific, though I may be on Eastern time part of the summer
Previous GSoC participation: I haven't ever participated in GSOC; I am interested to do so to learn more about embedded software, improve my coding skills, and participate in the community. I use open source tools all the time, and it is really exciting to make something useful that other people would use too.

About your project

Project name: TensorFlow Lite Compatability with BeagleBone AI

Description

While the specs of BeagleBone AI are awesome, it seems limiting for an AI board for the general public to not be able to access TensorFlow Lite. I would like to change that by choosing a current release of TensorFlow, and writing the code necessary to run it on the Arm M4 processors that the BeagleBone AI board has, and that should work with TensorFlow Lite. This project will be a combination of C coding, dealing with the Linux kernel, and doing some example cases with TensorFlow Lite in python. Obviously, given the diversion between OpenCL and TensorFlow, there is no perfect way to seamlessly integrate TensorFlow Lite, but there seem to be enough people in the community who want it, that it is worth at least making a gesture towards a compatibility solution. I first realized this was a problem when I decided I wanted to learn more about AI on the edge. One of the first things that comes up when I search for information on this topic is using TensorFlow Lite and the wide variety of other boards that it plays well with.

Ideally, I'd spend the first few weeks getting TensorFlow Lite working on a BBAI, document my hacking process, and then create a smooth stable way to make it work more out of the box. Then I'd spend the rest of the time doing examples that would be highlighted both on the BeagleBoard website, and also on the TensorFlow Lite Examples page by making a pull request to the owners of the github repo. Given that there are already examples of using TensorFlow Lite on Raspberry Pi, with picamera, the starting point would be to have equivalent BeagleBone examples, most likely doing image capture through OpenCV (OpenCV with BeagleBone Black). I'm also very interested in sound recognition, and if time allows I'd also like to do some examples in this area.

Timeline

Provide a development timeline with a milestone each of the 11 weeks and any pre-work. (A realistic timeline is critical to our selection process.)

Apr 8 Found out about exciting projects at Beagle Board, joined the community!
Apr 13 Proposal complete, Submitted to https://summerofcode.withgoogle.com
May 17 Proposal accepted or rejected
All of May Prework: * Learn more about Linux Kernels
  • Try out TensorFlow Lite on Raspberry Pi that I already have so that I know how it is supposed to work
  • Brainstorm ideas for cool examples
  • Learn more about how TensorFlow Lite works on microcontrollers through the TensorFlow Lite google-group (that I've already joined) and other sources
Jun 07 Pre-work complete, Coding officially begins!
Jun 17 Milestone #1, Introductory YouTube video
June 24 Milestone #2, Working version of TensorFlow Lite on a BeagleBone AI
June 30 Milestone #3, Smooth(er) process and documentation for others to use TensorFlow Lite on BeagleBone AI
July 12 18:00 UTC Milestone #4, Is Milestone #3 as user friendly as possible? If not, improve it
July 23 Milestone #5, Example Cases on BeagleBone AI (vision)
July 30 Milestone #6, Example Cases on BeagleBone AI (sound)
Aug 06 Milestone #7, explore whether TensorFlow Lite can also be integrated with the BeagleBoard-X-15, and/or whether

what I've learned about TensorFlow Lite can be extended to PyTorch or mlpack

August 10 Milestone #8, Completion YouTube video
August 16 - 26 18:00 UTC Final week: Students submit their final work product and their final mentor evaluation
August 23 - 30 18:00 UTC Mentors submit final student evaluations

Experience and approach

In 5-15 sentences, convince us you will be able to successfully complete your project in the timeline you have described.

Contingency

What will you do if you get stuck on your project and your mentor isn’t around?

Benefit

BeagleBoard options are strong contenders with Raspberry Pi and Arduino as tools that are easy to use to do IoT and industrial projects. While BeagleBones are geared more towards engineers and have some special features that actually make them quite different than the others (PRUs, ability to do low level and high level control simultaneously, power usage etc), they often get compared with the other frameworks. Given that Raspberry Pi and Arduino have TensorFlow Lite compatibility, I think it is important for BeagleBones to do the same. Also, there are algorithms that are easier to access with TensorFlow Lite than in other settings.

For example,

I tried tackling YOLO deployment on BBAI but it is currently impossible because of TIDL library restrictions. TFLite is the way to go with DL on BBAI and it uses TIDL underneath

-Jakub Duchniewicz


and several frustrated users have posted on the internet:

I've been attempting to build the TF Lite library on my BeagleBone AI (32-bit ARMv7 MCU) for several days now, to no avail.

-Matt

I wish to run my network on the AM5729 found on the BeagleBone AI board. I understand that the ti-processor-sdk-linux-am57xx-06.03.00.106 does not yet support the BeagleBone AI tool. However, the Debian image distributed for the BeagleBone AI has all the TIDL libraries packaged with it.

Therefore the only thing I need to be able to do is convert the tensorflow lite network into the TIDL format using the TIDL import tool. I assume I can do this using the following flow: 1. download the ti-processor-sdk-linux-am57xx-06.03.00.106-linux-x86-Install.bin from TI 2. run the installer on a linux (Ubuntu) host 3. navigate to the tidl_model_import.out tool 4. run the tidl_model_import.out tool on an appropriate configuration file (below)

-Alex Beasley

The error is because TIDL does not support importing tensorflow lite model. Please try with Caffe/tensoflow/onnx models, for more details refer to TIDL datasheet/user-guide.

-Praveen

Misc

Please complete the requirements listed on the ideas page. Provide link to pull request.

Suggestions

Is there anything else we should have asked you?