Buildroot:DeveloperDaysELCE2015

From eLinux.org
Jump to: navigation, search

Buildroot Developers Meeting, 3-4 October 2015, Dublin

Location and date

The Buildroot community is organizing a meeting on October 3rd and 4th 2015, for Buildroot developers and contributors. This meeting will be a mixture of discussion and hacking session around the Buildroot project. This meeting takes place right before the Embedded Linux Conference Europe, in order to make it easy for participants to attend both events. It is not mandatory to attend both days.

The meeting will take place in the Orange Business Services offices, located:

Garryard House
Earlsfort Terrace
Dublin 2

Sponsors

We are currently looking for companies willing to sponsor the event. The sponsoring will be used to pay for the Saturday dinner for the participants or Buildroot T-shirts for the participants. Contributions in the range of 250 EUR to 500 EUR are sufficient.

Participants

  1. Thomas Petazzoni. Arriving Oct 2 18:00. Staying at some Airbnb place with Yann, Maxime and Romain. Leaving Oct 8 11:20.
  2. Yann E. MORIN, arriving Oct 2 14:00 (but available ~17:00); leaving Oct 8 14:40.
  3. Arnout Vandecappelle Arriving Oct 2 21:50, staying at My Place, 89 - 90 Lower Gardiner St, leaving Oct 8 6:40
  4. Luca Ceresoli. Arriving Oct 2 23:30, leaving Oct 7 18:00. Staying at North Star Hotel, Amiens Street, Dublin 1.
  5. Samuel Martin. Arriving Oct 2 17:40. Leaving Oct 8 9:50
  6. Maxime Hadjinlian
  7. Peter Korsgaard
  8. Romain Naour
  9. Jan Viktorin
  10. Vicente Olivert Riera
  11. Alan Ott - Arriving mid-day Sunday

Registration now closed.

Who can attend ?

This meeting is opened to all Buildroot users and developers. The event is free. However, we make it clear that it is not a training event about Buildroot aimed at newcomers. It is a meeting for Buildroot developers to make progress about various topics in Buildroot. It is recommended to already be a Buildroot contributor to participate to the event.

For administrative reasons, participants will have to register to the event before Friday, Sep. 25th, so that access badges are printed. For this reason, people that are not registered will not be granted access to the site. Please, register by adding your name to the list, above. Registration now closed.

Report

  • Idea for future work: Rework toolchain logic
    • Split into several packages, one per family, so each family .mk file has the hacks needed isolated in that file
    • Core infra for external toolchains
    • Move the helper logic into scripts instead of makefile functions
    • Obviously, it will remain equally complex, but hopefully easier to read
  • Run-time testing:
    • Two proposals have been made: one based on robot framework, one based on python-nose (= unit test framework).
    • python-nose based proposal:
      • Current work (WIP): http://git.free-electrons.com/users/thomas-petazzoni/buildroot/log/?h=runtime-tests
      • Still to do: integrate with CI tool. Problem to be solved: when a test fails, that's just a python exception, while the actual problem is in some log file somewhere, so we should somehow pass that.
      • qemu is assumed to be installed on the host, there may be an issue with version compatibility.
      • Other improvements are also possible to get better feedback about the status.
    • How about lava-test (instead of robot framework or python-nose)? A very quick look at it shows that lava-test doesn't add a lot that we need; its main feature is building packages natively and then running them, which is not something we want. In addition, it requires python on the target.
    • Tests should be part of buildroot, not a separate project. Also, a per-package test should be in the package directory, in a separate file.
    • For package tests, the idea is to add them as regression test when an issue is encountered, e.g. the python strftime issue.
    • The configurations of the tests will have to be specified by hand, i.e. there is a collection of defconfigs (or defconfig fragments) for various tests. For package tests, either they are enabled explicitly somehow, or the package test verifies autonomously whether its requirements are met.
    • We decided to go for the python-nose based approach.
    • For now, we can commit the tests in a separate support/testing directory. We can migrate to per-package directories later; keeping it together now makes it easy to delete the directory again if we're not happy with it.
    • First focus is something that works on the command line; CI integration can be improved later.
    • Maybe it's not needed to use python-nose because Jenkins can also build things in parallel.
  • Multi-br2-external trees [Yann]
    • Use cases:
      • Separate proprietary packages from locally-added open source packages.
      • Several teams create their own external trees in separate repositories - smells like yocto layers or OpenWRT feeds.
      • Buildroot can provide a 'contrib' style external repository for packages that are not really supported, like wf111 (which cannot be autobuilt since we can't get the source).
    • The patch series from Yann adds a bit of complexity to already complicated code, which we were not really happy to include to begin with.
    • Idea from Peter: instead, add a script that will generate a bunch of symlinks into a single directory, and use that directory as the traditional br2-external.
    • Yann will take a look at the alternative of a script that generates a single BR2_EXTERNAL without need of changing the current infra.
  • Hash files for all packages: should we have an empty/useless .hash files for github fetched packages? [Thomas]
    • Ideally, we 'want' hash files for github packages. However, in practice this will be difficult:
      • Do a shallow clone? Is this possible on github?
      • Ask github to generate the tarball within a faketime, so the tarball is always the same. Actually, it looks like github is already doing that, the timestamps in the tarballs are now equal to the time of the corresponding commit.
    • There's actually nothing wrong with having hash files with 'none' hashes.
    • There are currently roughly 200 github packages without hash, and roughly 200 other packages without hash.
    • First priority is to add hashes to packages which currently don't have them.
    • Add a few hashes for github packages and see what happens in the autobuilders.
    • As long as there are so many packages without hash file, we can't make hash files mandatory.
    • It is possible to write a script that generates hash files for all packages, but this is not a good idea: we really want to check for upstream hashes and verify manually.
    • All packages should have a hash by FOSDEM.
    • If by FOSDEM, not all packages have hashes, we'll do the script to add them automatically.
  • Downloading external toolchains source code [Luca]
    • Luca has already sent an initial solution to be refined
    • Agreed that we want this feature.
    • Refined, submitted and merged during the meeting.
  • Google Summer of Code
    • Let's talk about it at next FOSDEM. The deadlines are after FOSDEM.
  • Dealing with bugs for which patches exist [Arnout]
    • It has happened a couple of times recently that a bug/problem was reported, a fix was posted, but the fix was not quite good enough and never got applied. Can we do better?
    • The problem is that the patches sometimes really are wrong, and don't really fix the problem. Since we're no experts in the upstream package it's difficult to evaluate their validity.
    • Accepting such random patches makes the system less maintainable, e.g. on a version bump it's not clear if this hack has to be kept or not.
    • Maintainability is paramount, so stick to our current quality standards.
    • Side note: there are currently 40 patches with some tag, it would encourage reviewers if they actually got applied more quickly.
  • Packages without directly-accessible archives (e.g. wf111, Qt5 proprietary addons...) [Yann, Thomas, Peter, Arnout]
  • External toolchains: shoud we keep multiple versions? [Arnout, Thomas, Yann]
    • Currently, we only keep the latest Linaro version, but multiple CS versions
    • Latest Linaro stable is now only available for x86_64 hosts
    • Luca is at least one user who uses older versions of the external toolchain. Since they're deprecated too quickly for him, he adds them back in his buildroot clone. Note that he needs to stick to an older, specific external toolchain because of dependencies of SoC binaries.
    • The pre-configured external toolchains are handy to get started, but keeping several versions within buildroot is not so useful.
    • For Linaro, if you're on x86_32, you just don't have it available.
    • However, we have a bunch of hacks for specific toolchains, which cannot be applied when you select a custom toolchain. With the toolchain revamping that is planned, that could maybe be fixed.
    • Potential problem with kernel headers: they change implicitly when the toolchain is bumped. However, glibc usually has a fallback mechanism. uClibc and musl don't, and most likely they don't check for kernel header compatibility at runtime so the problem will only become visible when a non-existing syscall is called. However, we already have a similar issue with the default internal toolchain configuration: the kernel headers version is silently bumpedmn.
    • In the future, we stick to a single external toolchain version. The Kconfig symbol should not encode the version (avoid legacy handling).
  • mdev without devtmpfs support: do we want it? [Luca]
    • Since the devtmpfs support turns out to be really easy to backport, so the need is really limited
    • OTOH devtmpfs is quite useless when you anyway have mdev, but it doesn't add much overhead either.
    • Handling the order of mounting for both the mdev-without-tmpfs and mdev-with-tmpfs case is not so trivial.
    • Since 2.6.32 is already 6 years old, the usefulness of mdev-without-devtmpfs becomes smaller and smaller.
    • The mdev-without-devtmpfs series is dropped from patchwork, but Luca will publish on github his v2.6.30+devtmpfs kernel branch. The change of the name to clarify that mdev uses devtmpfs is kept.
  • Buildroot 2000 Association for the Governance of the Universe ? Do it or not ? [Maxime]
    • i.e. the recurring subject of legal structure around buildroot
    • Creating a not-for-profit is really easy in France
    • It would make it possible to get sponsorship from companies like Devialet that require a legal identity to give money to. Also the payout from GSoC would be easier to handle.
    • This money could be used for travel expenses for volunteers, for boards, hosting, etc.
    • Paying for someone to actually work on the project (which has to be at least 10-20KEUR/year) is probably out of reach for the moment.
    • Ideas for the statutes:
      • Buildroot should be mentioned and described. This is the scope of the organisation object so we can't spend money on anything else.
      • Who will be an association member (i.e. someone who can vote in the General Assembly)? The current members decide who can become a new member - this gives us complete freedom to change the rules along the way.
      • Members can be ejected by the General Assembly, by simple majority but the to-be-ejected member cannot vote.
      • Provision that the board can call an Extraordinary General Assembly.
      • Allow remote voting on the General Assembly.
    • Base statutes on http://toulibre.org/pub/association/statuts/statuts.pdf (in French) but add remote voting.
    • Maxime volunteers to actually start on an organisation. Of course the statutes will have to be reviewed by everyone.
    • Try to get this done by FOSDEM
  • Reducing the mailing list traffic (reprise, because there have again been complaints about it)[Arnout]. Some statistics (not 100% accurate) taken from Arnout's mailbox over roughly the last 10 months:
    • 27632 mails (= almost 650 per week)
    • 23% commit mails
    • 100 (< 1%) replies to commit mails
    • 29% patches (mails with [Buildroot] [PATCH in the subject)
    • 34% replies to patches
    • which leaves 12% other mails
    • Recurring discussion, the conclusion remains: we're not going to change anything.
  • S40network: wait for interface or use ifplugd? [Yann]
    • Waiting for 15 seconds is not such a big issue.
    • In sensible cases it should anyway work without extra delay.
    • ifplugd is probably a bit too "big" to have as our default way of doing things. It's currently not enabled in our default busybox config.
    • Use a delay in S40network
  • system: add options for /bin /sbin and /lib to be symlinks into /usr [Yann]
    • We're fine with this.
  • internal toolchain wrapper [Arnout]
    • Is the basic principle of the patch series OK?
      • How about the poisoning in binutils? This still remains because we currently don't have a wrapper for ld. Not 'so' important anyway since ld is not often called directly. Ideally, however, there should also be a wrapper for ld, but that's a separate issue. We could get rid of the binutils poisoning because for the external toolchains we anyway don't do it.
    • RFC: toolchain-wrapper: support change of BR2_CCACHE
      • Yes, why not.
    • RFC: ccache: support changing the output directory
      • To be decided.
    • Note: we want to encourage Fabio to continue with the per-package staging/host dir and the top-level parallel build.
  • newlib for non-Linux builds [Yann, Thomas]
    • The use case is to use buildroot as a toolchain generator for bare metal platforms. The whole 'target packages' menu would depend on !NEWLIB.
    • It may actually be possible to build some libraries with newlib, but supporting that in Kconfig would be pretty complicated.
    • If it's not too invasive, why not? If it becomes too invasive, we can remove it again.
    • NuttX could be added as well if it's cleaned up.
    • Alan will look into adding PIC32 as well.
  • Guidelines for defconfigs (e.g. FPU selection), demo defconfigs, and kernel configs [Arnout]
    • Use EABIhf, except if the binary-only GPU blob is EABI only.
    • Use the best possible non-NEON VFP.
    • kernel config: preferably one from the kernel tree, if that is not possible, use a kconfig fragment to modify it, if that is also not possible, use a custom one. Custom one should be minimalistic, e.g. no modules for all possible hotpluggable devices, no NFS and samba.
    • demo defconfig: let's first see one and then discuss it.
      • Media files (logos etc.) can be included in the overlay in the buildroot tree if they're not too large. Otherwise, they should be in a package so it's downloaded.
  • Relocatable HOST_DIR [Samuel]
    • Need to fix all host tools to make them relocatable - this is fixed for programs by the patches posted by Samuel, but not the scripts (e.g. foo-config scripts).
    • The pkgconf wrapper still needs to be fixed.
    • In the staging and target there are still absolute paths leaking in.
    • There should be checks in post-hooks or something that checks that no absolute paths are leaking in.
      • Similarly, we can check that the generated binaries are for the right architecture/ABI, that no shared libraries are installed if static, etc.
  • Toolchain Buildroot vs External: Toolchain feature support disparity (e.g. Fortran, OpenMP, etc) [Samuel]
    • When using a buildroot toolchain as an external toolchain, we can't express this.
    • We can just add more Kconfig knobs for this.

Agenda

  • Google Summer of Code
  • Guidelines for defconfigs (e.g. FPU selection), demo defconfigs, and kernel configs [Arnout]
  • Packages without directly-accessible archives (e.g. wf111, Qt5 proprietary addons...) [Yann, Thomas, Peter, Arnout]
  • Multi-br2-external trees [Yann]
  • Hash files for all packages: should we have an empty/useless .hash files for github fetched packages? [Thomas]
  • Run-time testing
  • Dealing with bugs for which patches exist [Arnout]
    • It has happened a couple of times recently that a bug/problem was reported, a fix was posted, but the fix was not quite good enough and never got applied. Can we do better?
  • Reducing the mailing list traffic (reprise, because there have again been complaints about it)[Arnout]. Some statistics (not 100% accurate) taken from Arnout's mailbox over roughly the last 10 months:
    • 27632 mails (= almost 650 per month)
    • 23% commit mails
    • 100 (< 1%) replies to commit mails
    • 29% patches (mails with [Buildroot] [PATCH in the subject)
    • 34% replies to patches
    • which leaves 12% other mails
  • External toolchains: shoud we keep multiple versions? [Arnout, Thomas, Yann]
    • Currently, we only keep the latest Linaro version, but multiple CS versions
    • Latest Linaro stable is now only available for x86_64 hosts
  • newlib for non-Linux builds [Yann, Thomas]
  • Relocatable SDK [Samuel]
  • Toolchain Buildroot vs External: Toolchain feature support disparity (e.g. Fortran, OpenMP, etc) [Samuel]
  • mdev without devtmpfs support: do we want it? [Luca]
  • Buildroot 2000 Association for the Governance of the Universe ? Do it or not ? [Maxime]