When building the Debian installer from git, ignore any configured
kernel version, and use the kernel version that is present in the chroot
environment.
This fixes a build error on Trixie/testing, which occurred after a new
kernel became available when the freeze was lifted.
Follow-up for 18ad612b72.
The availability of 'firmware-linux' must be checked inside the chroot
instead of on the host.
This fixes building Bullseye without 'non-free' available (as discovered
by autopkgtest)
d-i and Calamares select the needed grub packages based on how the
computer is started (BIOS vs UEFI). Because the packages for each
variant conflict, they must be provided as .deb files.
Commit 8403487d introduced in the chroot stage a naming scheme for local
archive keys which distinguishes between ASCII-armored keys and binary
ones, and then adds the appropriate extension (either `.asc` or `.gpg`,
respectively) when copying them to `/etc/apt/trusted.gpg.d`.
However, this change was not applied to the binary stage: ASCII-armored
and binary keys are still both copied with the `.asc` extension, which
introduces a discrepancy between the two stages. This, in turn,
may prevent using the same `.list` file for both stages, as entries
such as
deb [signed-by=/etc/apt/trusted.gpg.d/my-repository.key.asc] ...
will fail in the chroot stage if `config/archives/my-repository.key`
is in binary format (as it will be copied with the `.gpg` extension),
while entries such as
deb [signed-by=/etc/apt/trusted.gpg.d/my-repository.key.gpg] ...
will fail in the binary stage (as the key will this time be copied
with the `.asc` extension).
Previous commits to support Memtest86+ 6.00+ split the files into
"memtest" (backwards compatible / new BIOS name) and "memtest.efi"
(new EFI name). However, the 6.00 non-EFI binaries are now
32/64-bit, and will not work with GRUB's linux16 boot when using GRUB
for BIOS boot.
- Legacy Memtest86+ pre-6.00 will continue to be "memtest", linux16
(continue to support builds on current Debian stable bullseye)
- Change 6.00+ filename to "memtest.bin"
- GRUB memtest.cfg contains logic for legacy / new bios / new EFI
- Change EFI boot from "chainloader" to "linux", which allows for
passing command line options to Memtest86+
- syslinux @MEMTEST@ will conditionally be memtest.bin or memtest
- @MEMTEST_BIN@ will conditionally be memtest.bin or memtest, but is
completely legacy at this point (all internal code paths now check
for @MEMTEST_DIR@/memtest or @MEMTEST_DIR@/memtest.bin)
Getting complicated, but this logic can be simplified again once we
no longer need to support current Debian stable.
- BIOS: Final filename doesn't change (memtest), but now uses a
64-bit version on amd64.
- EFI: Now included as memtest.efi; 32-bit on i386, 64-bit on amd64.
As it's GRUB-only, we can test for the existence of memtest.efi
(6.00+ only) in GRUB itself.
- Add @MEMTEST_DIR@ to facilitate directory-based testing in GRUB.
This expands upon the previous MVP commit
15e78a3eec .
memtest86+ 6.00-1 (in testing as of this date) adds and changes its
available binaries:
- /boot/memtest86+x32.bin
- /boot/memtest86+x32.efi
- /boot/memtest86+x64.bin
- /boot/memtest86+x64.efi
Future live-build functionality would ideally incorporate these new
binaries (x64 for amd64 builds, efi for grub-efi), but in the meantime,
this is an MVP update which supports either old memtest86.bin or new
memtest86+x32.bin.
The request asked for 5 additional variables to be resolved for the SVG
images:
Added: LB_ISO_APPLICATION, LB_ISO_PUBLISHER and LB_LINUX_PACKAGES
Not added: LB_ISO_PREPARER: xorriso recommends not changing this
variable
Not added: LB_ISO_VOLUME: is can contain '@ISOVOLUME_TS@' which resolves
to a timestamp and would need additional handling
Currently it's only supported in the chroot/binary phases. Omitting
bootstrap was an oversight.
Note that config/archives/*.conf are assumed to be apt.conf files,
so the apt_auth files are required to end with .auth and will be renamed
to .conf when copied into the chroot.
With this 'distribution', the installer is built from its git
repository, matching the date in SOURCE_DATE_EPOCH.
Additionally the kernel version is matched to the kernel version in the
chroot (to ensure that the installer can be run).
Rationale: The 'daily' installers are not archived, and therefore cannot
be used for a reproducible build.
The installer stage has a clean chroot for its own usage. The chroot
from the chroot stage is available in chroot/chroot (implemented similar
to the binary stage).
The new includes.chroot system implemented in !214 adds the option
to include files in the chroot before and after package installation.
chroot_hostname and chroot_resolv have been updated but not chroot_hosts.
Gbp-Dch: Short
Persist LB_BOOTLOADERS in lb_binary_* configuration, since
Init_config_data called from lb_binary_* calls Prepare_config which
then overrides LB_BOOTLOADER_BIOS and LB_BOOTLOADER_EFI if
LB_ARCHITECTURE is amd64 or i386, and these can only be restored to
their proper values by the presence of LB_BOOTLOADERS.
This therefore ensures that the value of the "--bootloaders" parameter
to lb_config is respected by the lb_binary_* steps.
Allow APT_AUTH.CONF(5) config files to be used with archives.
Note that config/archives/*.conf are assumed to be apt.conf files,
so the apt_auth files are required to end with .auth and will be renamed
to .conf when copied into the chroot.
Closes: #991065
That prevents potential errors during future Remove_packages calls without APT indices available. In such cases, APT tries to remove packages with unrecognized names, which results in an error and fails the script execution. An example of this would be installing apt-utils in chroot_archives with --apt-indices option set to false.
Similar to iso images, the timestamp of a file/folder inside the binary
image cannot be newer than SOURCE_DATE_EPOCH.
Also set the timestamp of the hdd image.
This previously undocumented value for '--chroot-filesystem' creates the
uncompressed folder 'live/filesystem.dir'.
This will speed up the generation of live images a lot, because the
squashfs compression is skipped.
Example command line:
lb config -b hdd --chroot-filesystem plain --binary-filesystem ext4
The package might be missing when building with a minimal configuration,
as recommended in the Live Manual, the example 'Base image' scenario:
'lb config --apt-recommends false --debootstrap-options "--variant=minbase"'
Notes:
- No bootloader is configured; images must be booted via direct kernel
- Tested via qemu + opensbi fw_jump.elf + kernel/initrd
- As riscv64 is currently in ports, the following options are required:
--distribution unstable
--mirror-bootstrap http://ftp.ports.debian.org/debian-ports/
--security false
--debootstrap-options --keyring=/usr/share/keyrings/debian-ports-archive-keyring.gpg
- Normally ports require stacking two dists, unstable and unreleased.
debootstrap does not support doing this, but as of this writing, there
is nothing in riscv64 unreleased which is required to bootstrap.
However, this would not have been possible a year ago.
There are two main scenarios:
1) The host with live-build is configured to use a proxy
This proxy will automatically be used by live-build
2) The user of live-build explicitly specifies a proxy
2A) With the environment variable 'http_proxy' (preferred)
2B) With the command line option --apt-http-proxy
Any inconsistency in the setting of a proxy results in an error message of 'lb config'
All internal tools (apt/apt-get/aptitude, wget and debootstrap) use the environment variables 'http_proxy' and 'no_proxy', which are passed along to the chroot where needed.
Test scenario:
- A virtual machine with all out-going traffic blocked, except for the proxy.
This adds support for dm-vertiy on the root filesystem.
Currently only squashfs is supported.
Three new flags are introduced.
* --dm-verity: Enable basic dm-verity support
* --dm-verity-fec NB_ROOTS: Enable forward error correction. Optional
* --dm-verity-sign SCRIPT: Specify signing script for the root hash. Optional
1) lb config rejected multiple checksum types
2) When using the installer, cdrom-checker requires a md5 checksum file,
use 'Check the integrity of the installation media' in the installer
3) The comments in the first lines of the checksum files caused
cdrom-checker to fail the integrity of the image
If grub/splash.png exists, assume the configuration editor intends
to have a grub-specific splash.png, and do not modify theme.txt.
But if syslinux has the only known splash.png, use it for both
syslinux and grub.
(This allows for a hybrid image where the grub side can have e.g. a
16:9 1920x1080 splash.png which gets grub is capable of automatically
scaling, while the syslinux side has a 640x480 splash.png which
effectively must be this fixed size.)
When a package lists contains only packages protected by a test
that doesn't match for the current run, then Expand_package_list
outputs nothing and the following "grep -v" fails because it
has not filtered anything. Avoid this by protecting the "grep -v"
call with "|| true".
Before commit 9f3e5fe8d (Fix the way the .disk/mkisofs file is created)
all these commands (`mkdir`, write to `binary/.disk/mkisofs` and
`xorriso`) were in the same `binary.sh` script. Since that commit, the
write was extracted, to prevent issues with quoting, but the related
mkdir was left in `binary.sh`. This means that the write is now executed
first, and the `mkdir` only afterwards, making the `mkdir` quite pointless.
In practice, this did not break becaue binary_disk also does the same
`mkdir` and runs before `binary_iso`, but if one runs commands manually
and skips `binary_iso`, then this does break.
Even though this is not really a supported usecase, just move the mkdir
outside of `binary.sh`, so it runs *before* the write again as intended.
Moved includes.chroot to includes.chroot_after_packages and added
includes.chroot_before_packages. includes.chroot does still work as before.
We also now use rsync for copying files if it is installed.
This improves runtime and space consumption for large includes.
Gbp-Dch: Short
Closes: #927128
Installation of flatpaks doesn't work with normal chroots.
This patch enables support for using systemd-nspawn in hooks.
Gbp-Dch: Short
Closes: #965953
avoids spitting out warning
> [2020-06-07 22:30:32] lb binary_grub-efi
> P: Begin preparing Grub based EFI support...
> Reading package lists...
> Building dependency tree...
> Reading state information...
> Package grub-efi-amd64-signed is not available, but is referred to by another package.
> This may mean that the package is missing, has been obsoleted, or
> is only available from another source
>
> E: Package 'grub-efi-amd64-signed' has no installation candidate
> W: UEFI Secure Boot disabled due to missing signed Grub/Shim.
this simplifies things to avoid the messy recursion.
it is also a necessary step to implementing handling of common options
like --debug. we need to process all options to decide how to approach
doing things (e.g. with debug messages to be output or not) before actually
performing any work, in order for options like --debug to be reacted to
properly.
also, as things were, options like `--debug` were not being passed along
in the recursive executions, while now that is no longer an issue.
the order of file/folder deletions for `--all`|`--purge`|`--remove`
actions is slightly changed here, but i don't see any issue with that and
it's cleaner to not preserve that.
Gbp-Dch: Short
This reverts commit 665372c19d.
the kali test failed due to their script using a hack of passing along a
custom option after an early terminator through to their auto/config file.
the change implemented here was valuable, but i'll have to look into
revising things to work with what Kali is doing.
Gbp-Dch: Ignore
some scripts temporarily install packages to accomplish some work before
then removing them. the list of packages installed is kept in memory in a
variable.
a weakness of this design is that if a failure occurs or the user cancels,
and then following this the user re-runs `lb build`, letting it try to
pick up and recover from where it left off, that list of packages that had
been installed is lost, resulting in those packages that were installed
then being a permanent part of the chroot.
here we fix this weakness by backing up the list to a file, which is always
read from on removal. thus in a recovery situation, any packages still
installed from a situation like that just described, will be removed upon
the next use of `Remove_package()`.
this is not perfect, since we are having to wait for opportunistic
execution of the remove function. we need to find a suitable place for the
`Cleanup_temp_packages()` function to be used.
- doing so in `Init_config_data()` would not be suitable because we don't
hold the lock when that's run, even if we ignored the hijacking of that
function for taking such action...
- doing it in `Exit()` doesn't seem a good fit either.
- putting it explicitly in every build script just seems a little messy...
perhaps a local exit trap like for removing the lock...?
note that `binary_rootfs` skips running the remove function after installing
tooling, since it just throws the wrapper chroot away, which then leaves the
file around with stale data for the next remove instance to pick up, which
then does not actually remove it because it's not installed. this is not
ideal either... perhaps the optimisation should be removed from that script?
Gbp-Dch: Short
after handling arguments, catch and report any remaining "non-option"
arguments.
for instance users could make the basic mistake of using
`lb config --bootloaders syslinux grub-efi`, i.e. failing to quote the
multiple bootloaders (i actually encountered a user doing this who swore
that "it just worked"). catching and reporting such mistakes could be
valuable to users.
previous behaviour:
```
$ lb config --bootloaders syslinux grub-efi
P: Updating config tree for a debian/buster/amd64 system
P: Symlinking hooks...
```
new behaviour:
```
$ lb config --bootloaders syslinux grub-efi
[2020-05-05 18:56:07] lb config --bootloaders syslinux grub-efi
E: Unexpected argument found: grub-efi
```
Gbp-Dch: Short
debootstrap must obviously exist in the host. we thus must pass 'host'
not 'chroot' such that a warning about needing to install it on your host
system is output, rather than it being added to a list of packages to be
installed, which never occurs in this script.
Gbp-Dch: Ignore
Roland Clobus
Thore Sommer
Jonathan Carter
Patrice Duroux
Raphaël Hertzog
snip
Ryan Finnie
Nick Brown
Olivier BLIN
Rob Shearman
Michał Prochera
Sophie Brun
David Hewitt
Luca Boccassi
Raphaël Hertzog
Marcel Partap
Jan Kot
Matthijs Kooijman
Victor Gavro
Lyndon Brown