CE Workgroup Device Mainlining Project

From eLinux.org
Jump to: navigation, search

This page describes the "Device Mainlining" project of the CE Workgroup.

This purpose of this project is to decrease the amount of out-of-mainline patches for the Linux kernel, in modern consumer electronics products.

This project was proposed at the CE Workgroup Steering Committee meeting in Tokyo, Japan, in June of 2014.


As of January, 2015, many devices using Linux have a lot of code that is out-of-mainline.

The following table shows analysis of several mobile phones released in 2014 by major phone vendors. Most of these used 3.4 kernels.

Phone Manufacturer SoC Lines out-of-mainline
G3 LG Qcom 2.616M
(unknown) Motorola Qcom 1.795M
Galaxy 4 Samsung Exynos 1.100M
Galaxy S5 Samsung Qcom 3.105M
Xperia Z3 Sony Qcom 1.794M
Xperia C Sony Mediatek 1.935M
E2 Acer Mediatek 1.411M
Zenfone 6 Asus Atom 2.163M
P6 Huawei Hisilicon 2.659M


In terms of overall strategy, the actions being performed for this project fall into 4 categories:

  • communicate and meet to make plans and carry forth objectives
  • identify problems
  • implement solutions
  • train and educate participants

Communicate and meet to make plans and carry forth objectives

  • see meetings section below

Identify problems

Implement solutions

  • Decide on specific projects
    • Identify kernel sub-systems that are the most out-of-tree
    • Identify what the problems are
  • Push patches to mainline to address problem areas found
    • Assist weary maintainers
  • Help companies and employees overcome IT obstacles - see Overcoming IT Obstacles to open source participation

Train and Educate participants

  • Collect resource for mainlinining training
  • Convince management to provide funding and/or resources for this work
    • Produce a white paper on obstacles (see above)
    • Determine cost of out-of-tree code, and publish information for managers

Next Projects


Device Mainlining BOF - October 16, 2014

  • Tim Bird held a Birds-of-a-Feather meeting to discuss issues with this during Linux Plumbers in Dusseldorf, Germany, on October 16, 2014
    • Kevin Hillman attended, and provided some good insights into SOC mainline efforts in the past

[should put some more meeting notes here]

  • some key ideas
    • TI got serious about mainlining when Nokia adopted the upstream kernel version over their in-house kernel version, for their phone releases
      • take-away: would be good if Google pushed vendors harder, but they don't have incentive to do so
    • Linaro teams are focused on specific technology areas, not on SoC mainlining
      • With early Android releases, they showed android patches supported on dev boards for various SoCs
      • This was for demonstration and was not production grade Linux
      • Probably this was the closest these trees ever were to mainline

Device Mainlining meeting - March 24, 2015

This meeting will be held Tuesday, March 24, at 11:00 am in the Guadalupe meeting room at the San Jose Marriott.

Attendees: [should list attendees here]


Some of the issues I want to discuss are:

  • Tim reports on his SOC research
  • Where are we, really, with product-grade mainline support for SOCs?
    • Based on recent work on my part, I've come to believe we're not nearly as close to mainline support for many, many processors, as we should be. I'm in the middle of doing some research on multiple SOCs used in mobile phones, and it appears that there are all kinds of fundamental problems that make using a mainline kernel on a production phone a fleeting dream.
  • What sub-systems have weakest support in mainline?
    • For each sub-system, is the poor support because of:
      • framework deficiencies
      • DT blockage
      • dependencies on out-of-tree code
      • lack of documentation
      • maintainer bandwidth, or
      • something else?
  • Are there things the Linux Foundation, other industry groups, or companies can do to make interacting with the community easier?
    • Would it be helpful to produce a scorecard, for evaluating where each SOC is
      • Scorecard would have: what subsystems have been mainlined, what's outstanding, etc.
    • Can git tools be developed to automate this analysis?
    • Can Linaro and LF work together on some things?
    • keep a list of "maintainer quirks" - what things individual maintainers like/dislike, insist on, what timing they like (when to submit relative to merge window).

Calls and meetings - Spring/Summer 2015

  • Tim held another meeting Special Interest Group meeting at ELC, to identify areas that need work (and could use CEWG/LF funding)
    • Tim prepared some information about the out-of-tree status of various SOC and discussed this in the meeting
  • April 21, 2015 - Tim presented Obstacles to Mainlining talk at Genivi All-Members-Meeting
  • April 28, 2015 - Conference call was held with Linaro (Tim Bird, Kate Stewart and Mark Brown)
    • different items were discussed, with main goal to be publication of upstream analysis tools
  • Tim Bird met with Ibrahim Hadad on April 28, 2015 in San Jose
    • presented basic goals, got information about Samsung and their practices
  • May 12, 2015 - Conference call with Linaro (Kate Stewart and Mark Brown)
    • had not published upstream-analysis-tools, but discussed possible names, and other topics

LinuxCon North America BOF - August 19, 2015

ELC Europe BOF - October 6, 2015

Obstacles (or problems to overcome)

  • Version gap
  • Dependency on out-of-tree code
  • Inability to test (inability to generalize)
  • Low-quality code
  • Specialized code
  • Immature upstream frameworks
  • No time allocated by management
  • Difficult (or missing) internal approval process
  • Overloaded (or slow) maintainers
  • Lack of perceived business value
  • Rate of hardware churn greater than resources allocated to upstream effort
  • Difficult upstream process
  • Lack of English proficiency
  • Fear of rejection
  • Suppliers providing out-of-tree code
  • No incentive for management (or counter-incentives)


  • intrinsic delays in the product pipeline
  • lack of corporate will
  • skill deficiency
  • sourcing woes??


This section has information about tools provided by this project:

Upstream Analysis Tools

Some tools that have been developed are available at: https://github.com/tbird20d/upstream-analysis-tools The purpose of these tools is multiple:

  • to allow a company to analyze their own upstream status (check their own out-of-tree-ness)
  • to allow a company to score the source from potential suppliers
  • to try to identify areas where multiple companies have code for the same hardware or feature out-of-tree
    • this is intended to help identify where shared work makes sense
    • also, it might help identify a sub-system or framework weakness that needs addressing upstream

Some tools (also part of the release above) can be used for the following:

  • to track the status of contributions from different parties to the Linux kernel over time
    • this is intended to allow an organization to monitor the progress of their mainlining activities
  • to identify what their code is dependent on (commit-dep.py)
    • this is to assess the potential difficulty of mainlining a particular patch, to help in prioritizing mainlining work
    • also, this should help to identify dependencies between commits, to help in prioritizing individual patches


Training and tutorials

Coordination pages

Pages for specific processors

Mailing list

See http://lists.linuxfoundation.org/mailman/listinfo/device-mainlining
archive: http://lists.linuxfoundation.org/pipermail/device-mainlining/

Mobile product source code repositories

See Phones Processors and Download Sites