With the merged /usr feature now enabled in debootstrap, we have
/bin and /lib which are symlinks to /usr/bin and /usr/lib. If
the live-build configuration provides files in
config/includes.chroot/lib/ or config/includes.chroot/bin/ then
lb chroot_includes will replace the /lib or /bin symlink with real
directories having only the content provided in chroot.includes and
problems will follow... the build will usually fail later with a
cryptic error message (for example a hook failing to execute
/usr/bin/env).
Work around the issue by creating a tarball of the files to install
and by unpacking that tarball from within the chroot with appropriate
options.
When using Secure Boot, grub2 as built by Debian will now load a config
file from EFI/$VENDOR instead of having EFI/debian hardcoded.
$VENDOR comes from dpkg-vendor or from the user building grub2.
The vendor string is stored in the control metadata as Efi-Vendor, so
retrieve it when building the EFI image.
Only gnupgv is part of the deboostrap set these days, but apt-key needs
the full gpg (with gpg-agent) which is just a recommends.
Instead just drop the key with an .asc suffix in /etc/apt/trusted.gpg.d
which is supported since apt version 1.4
live-build supports preseeding configuration, but the configuration has
to be installed after bootstrapping since it needs the debconf tools to
be applied. But packages that have already been installed and configured
in the bootstrap step will then ignore those preseeded configs.
After applying each preseed file, parse the package list and manually
reconfigure the relevant package(s) so that the configuration will be
applied.
Check_package will just add a missing dependency to the LB_PACKAGES
todo list if it doesn't find it, when build-with-chroot is true, even if
the check was not for the chroot.
Instead error out if the check is not done for the chroot, e.g.
Check_package host /bin/foo foo
The binary_onie script works on the host, not in the chroot (if used),
so don't check that the required packages for the script are installed
in the chroot (if present) as they won't be useful.
Check instead on the host.
Instead of hard-coding the decompression and compression formats,
detect them at runtime.
Install the required dependencies as well - they were mistakenly left
out.
Gbp-Dch: Ignore
The apt-secure option does not work anymore when building a sid image,
as with apt 1.6 the existing options are no longer enough to get apt
to accept an unsigned repository, which is necessary when using a
local cached repository (offline build).
Pass Acquire::AllowInsecureRepositories "true"; together with the
other options when --apt-secure false is used to fix the issue.
Open Network Install Environment is an open image format used by
networking vendor to ship a standardised image for networking white
box switches.
ONIE hardware takes this image at boot and a script to chain load
into the final environment via kexec. We can support Debian and
derivatives on such systems by packing an ISO which then gets
unpacked, kexec'ed and live-booted.
A base ONIE system can be tested in QEMU by building a VM following
these instrunctions:
https://github.com/opencomputeproject/onie/blob/master/machine/kvm_x86_64/INSTALL
Once built, boot onie-recovery-x86_64-kvm_x86_64-r0.iso in QEMU/libvirt
and on the console there will be the terminal prompt. Check the IP
assigned by libvirt and then scp the live image (ssh access is enabled
as root without password...). Then the .bin can be booted with:
ONIE-RECOVERY:/ # onie-nos-install /tmp/live.hybrid.iso-ONIE.bin
The implementation is inspired by ONIE's own scripts that can be found
at:
https://github.com/opencomputeproject/onie/blob/master/contrib/debian-iso/cook-bits.sh
A new option, --onie (false by default) can be set to true to enable
building this new format in addition to an ISO.
An additional option, --onie-kernel-cmdline can be used to specify
additional options that the ONIE system should use when kexec'ing the
final image.
Note that only iso or hybrid-iso formats are supported.
For more information about the ONIE ecosystem see:
http://onie.org
Signed-off-by: Erik Ziegenbalg <eziegenb@Brocade.com>
Signed-off-by: Luca Boccassi <bluca@debian.org>
The Tianocore reference UEFI implementation, used for example by Qemu,
wants the EFI directory name to be uppercase in the fat32 partition
when Secure Boot is enabled, and will fail to load otherwise.
Support for UEFI Secure Boot is modelled after how it currently works
in Ubuntu and on how it is going to work on Debian.
A minimal bootloader, shim, is used as the first-stage and it then
loads grub. Both have to be signed.
shim-signed is already available in Debian so the filenames are
already established, and the grub2 repository and packaging is common
between the 2 distros so we can already be reasonably sure of what it
is going to be.
So if both are available, copy /usr/lib/shim/shim[x64|aa64].efi.signed
as boot[x64|aa64].efi so that UEFI loads it first, and copy
/usr/lib/grub/[x86_64|arm64]-efi-signed/grub[x64|aa64].efi.signed as
grub[x64|aa64].efi.
This grub2 EFI monolithic image is currently hard-coded in grub2's
repository to look for a config file in efi/debian, so make a copy
of the previously added minimal grub.cfg that loads the real one in
that directory in both the fat32 and ISO 9660 partitions.
The new option --uefi-secure-boot can be set to auto (default,
enable or disable.
In auto, the lack of the signed EFI binaries is intentionally left as a
soft failure - live-build will simply fallback to using the locally
generated non-signed grub2 monolithic EFI binary as the only
bootloader. Given the difficulties surrounding the Secure Boot
signing infrastructure this approach gives the most flexibility and
makes sure things will "just work" once the packages are available,
without the need to change anything in the configuration.
This will also greatly help downstream distributions and users who
want to do self-signing.
The enable or disable options work as expected.
Closes: #821084
On some UEFI implementations, like the AMI found in the Supermicro
X10SDV-TP8F development board, the fat32 partition will be loaded
first and so Grub will set it the root, and then drop to the console
as it cannot find any config on it.
Add a minimal grub.cfg that allows Grub to find the main config on
the ISO 9660 partition and load it.
Closes: #892406
Machines tend to become unresponsive during the mksquashfs step.
Avoid this by lowering the priority of the process.
Thanks: Ronny Standtke for the patch.
Closes: #867539
Commit a15b579652 (#775989) dropped an early exit from the
chroot_archives remove step in case the parent mirror chroot and binary
parameters are the same and introduced a regression, as with the
following live-build now fails when the parent mirror is using a file:/
local apt repository (for example when the build worker is offline and
uses a pre-built cache of packages).
Example config:
lb config --mirror-bootstrap "file:/pkgs" \
--mirror-chroot "file:/pkgs/" \
--mirror-binary "file:/pkgs" \
--parent-mirror-bootstrap "file:/pkgs" \
--parent-mirror-chroot "file:/pkgs/" \
--parent-mirror-binary "file:/pkgs" \
...
with /pkgs being a directory with the packages for the installation and
the apt metadata (Packages/Sources/Release).
The problem is that, with such a setup, the /pkgs directory is bind
mounted inside the chroot as an optimisation in the install step,
and umounted as one of the first actions in the remove step for
chroot_archives.
Before that fix, the script terminated immediately. But now it
progresses and at the end it tries to run apt update inside the chroot
which will fail since the repository directory has been umounted, and
thus the packages and the apt metadata are no longer available, while
still being listed in /etc/apt/sources.list.
The proposed solution is to umount the local directory at the end of
the remove step, rather than at the beginning.
Closes: #891206
Now grub.cfg shows all the kernel options. Before this patch when you
had more than two kernels it only showed the auto option.
Signed-off-by: Raphaël Hertzog <hertzog@debian.org>
To generate an hdd image, binary_hdd first estimates the needed size of
the image using du. By default, when du finds multiple hardlinked copies
of a file, it counts them only once. However, when the target filesystem
is FAT, which does not support hardlinks, these files will take up more
space when finally copying the contents, breaking the build:
P: Copying binary contents into image...
cp: error writing 'chroot/binary.tmp/live/initrd.img-4.9.0-3-amd64': No space left on device
cp: error writing 'chroot/binary.tmp/efi/boot/bootx64.efi': No space left on device
cp: error writing 'chroot/binary.tmp/efi/boot/bootia32.efi': No space left on device
cp: cannot create directory 'chroot/binary.tmp/boot/grub': No space left on device
cp: cannot create directory 'chroot/binary.tmp/isolinux': No space left on device
To fix this, pass --count-links to du when the target is FAT, to make
the space estimation correct.
This problem is exposed by commit 9c974b26b (Instead of renaming kernel
for syslinux, create hardlinks), which might need to be separately fixed
(to not waste space on FAT targets), but binary_hdd should at least
handle hardlinks more gracefully.
Since commit fdc9250bc (Changing package dependency checks within chroot
to work outside as well), Check_package automatically checks for
LB_BUILD_WITH_CHROOT and works inside as well as outside of the chroot,
so no need to check LB_BUILD_WITH_CHROOT before calling them.
Install_package and Remove_package are just a no-op when building
without chroot, so they can also be called unconditionally.
Restore_cache and Save_cache do not check LB_BUILD_WITH_CHROOT but it
it should not hurt to call them when not needed (which already happened
in some cases).
This commit makes all Check_package calls unconditional on
LB_BUILD_WITH_CHROOT.
For binary_syslinux, this fixes the check (which used outdated paths
outside the chroot since 7b6dfd9d1), for binary_grub-efi,
binary_package-lists and chroot_package-lists this simplifies the code
(but also causes the check to become package-based instead of file-based
on apt-based systems), and for binary_loadlin and binary_win32-loader
this adds the check outside the chroot which was previously missing.
Previously, Check_package would only show an error when host packages
are missing on a non-apt system. On apt system, the packages would be
added to _LB_PACKAGES, which causes them to be installed in the chroot,
not in the host (or not at all if Install_package is not called). This
behaviour could break the build.
This applies to either packages that must be present in the host (as
checked with `Check_package host ...`), as well as packages that can be
either in the chroot or host (as checked with `Check_package chroot`)
when LB_BUILD_WITH_CHROOT=false.
Recent versions of Linux, parted or some other bit of software cause
partition devices, like /dev/loop0p1 to be created when running parted
mkpart. However, these devices are not cleaned up when running
losetup -d to remove /dev/loop0 later, so they linger around and confuse
mkfs (which refuses to make a filesystem, thinking there are partitions):
mkfs.fat 4.1 (2017-01-24)
mkfs.vfat: Partitions or virtual mappings on device '/dev/loop0', not making filesystem (use -I to override)
To prevent this behaviour, pass --partscan to losetup when adding a new
partition, to clean up any lingering partitions. It seems losetup does not
accept --partscan when deleting a loop device, to clean up at that point, but
since binary_hdd mounts the partition last, there should not be any lingering
partition devices after live-build is done.
The --partscan option is available since util-linux 2.21 (released in 2012), so
it should be fairly safe to pass it unconditionally.
Marcel Partap
Raphaël Hertzog
Luca Boccassi
Steven Shiau
Rohan Garg
Steve McIntyre
Adrian Gibanel Lopez
Balint Reczey
Matthijs Kooijman