Almost all packages initially identified as interesting for us[1] have
been moved to non-free-firmware. An updated list of packages has been
published[2], which can be summed up this way:
- amd64-microcode and intel-microcode would require specific code
anyway (#1029804);
- zd1211-firmware shows inconsistencies (archive vs. repository).
1. https://lists.debian.org/debian-boot/2023/01/msg00150.html
2. https://lists.debian.org/debian-boot/2023/01/msg00235.html
Therefore, stop looking for non-free firmware packages in non-free,
and concentrate on non-free-firmware only.
Components are ordered “main contrib non-free” almost everywhere, let's
use the standard order here too. There should be no functional changes.
Note: This controls the sources.list file used during the build, but
doesn't control “deb cdrom:” lines in sources.list during or after
installation, which are the result of running apt-cdrom (which sorts
components alphabetically, see its pkgCdrom::ReduceSourcelist function).
WARNING: This is an attempt at making it easier for users to enable
support for local packages. This branch is really believed to make
the whole situation easier and clearer on the long run, but it might
introduce some regressions, or at least behavorial changes depending
on existing configurations.
In particular, users might have a hierarchy under LOCALDEBS that's
different from the one getting documented in eb6d140f48, possibly with
a top-level local/ (see also bfebeded70 and 7ce8d0c28f).
It's advised to move files around to match the documented layout, and
to report any regressions that would be spotted: many tests where run,
trying to torture this patch series as much as possible, imagining
borderline cases, but in the end, that's not a feature that's getting
fully battle-tested since it's obviously not used for official builds.
When local packages are enabled (LOCAL=1), various things happen in the
target /dists directory. One of them is the creation of a /dists/local
symlink, pointing to (/dists/)<CODENAME>. This leads to a very confusing
situation, where “local” is initially about a component, but effectively
ends up looking like a distribution.
It is theoretically possible that cleaning this up might require some
follow-up fixes in other tools, but even that would be better than
keeping bear traps around.
It's possible to set LOCAL=1 to enable support for local packages, which
automatically results in “local” being listed in /.disk/base_components,
which is used to build the --components parameter for debootstrap.
Users might have various sets of *.deb and/or *.udebs, be it over time,
or for various codenames, and it's possible for /dists/<CODENAME>/local
to be entirely missing in the generated image.
In this case, /.disk/base_components must not contain “local”, as that
would break debootstrap; this commit detects and fixes this possible
inconsistency.
This is a regression introduced while trying to catch up with apt's
expectations, in 2183089968c04c4765d2119d176e021431b26353: iterating over
Packages files is done by running the “zcat” utility on Packages.gz and
Packages.xz; replacing Packages.gz with an uncompressed Packages broke
iterating over Packages files referencing udebs.
This might have worked by accident depending on the exact layout under the
LOCALDEBS directory (see previous commit for the big picture).
Let's focus on where input files for d-i should be stored:
<LOCALDEBS>/dists/<CODENAME>/local/debian-installer/binary-amd64/*.udeb
Those *.udeb end up being stored under the “local” component[1] (making it
clear they're not from usual components on a Debian mirror), but they are
referenced in Packages for the “main” component[2], possibly replacing a
different version found in the archive.
/dists/<CODENAME>/local/debian-installer/binary-<ARCH>/*.udeb [1]
/dists/<CODENAME>/main/debian-installer/binary-<ARCH>/Packages [2]
This makes it possible for d-i to find those *.udeb automatically, as it
uses a single Packages file.
Getting support for local packages (*.deb and/or *.udeb) to work
consistently has been a gigantic time sink, never again!
Document the expected layout as precisely as possible, so that
unsuspecting users have a better chance of getting things right!
Guido Berhoerster
Steve McIntyre
Cyril Brulebois
James Addison
Samuel Thibault