white is not going to work well on a terminal with a white background,
obviously. We should keep the standard colour and just try applying the
bold.
or do we want to consider a non black/white colour? like blue...?
Gbp-Dch: Short
instead of conditionally writing deb-src lines, it is **much** neater if
we use sed to optionally removed them at the end.
Gbp-Dch: Short
Closes: #952928
- prefer using `which` over hard coded paths
- it is redundant to check that the bin pointed to the return of
`which` exists and is executable, `which` already gives us
assurance of that if it returns true!
- the redirection of output (`2>/dev/null`) seems to be
unnecessary from my testing.
the instances relatnig to fdisk and losetup in functions/defaults.sh have
been left as they are since they get executed by `lb config` which can run
without sudo elevation unlike `lb build` and in that case `which` would
fail to find these binaries resulting in error.
this also fixes a bug showing an error for missing debootstrap - this tool
requires sudo privileges to run and thus is not found via a none elevated
which search.
Gbp-Dch: Short
Closes: #952927
the existing logic for obtaining a list of firmware packages always
downloaded a fresh copy of the archive content file, deleting the file
already in the cache. here we move to actually making use of the cache.
this helps when building multiple times, at least for the same distro. the
package list obtained is rarely going to change after all. it could of
course differ between distros, but the cache is per-distro, as it has
always been.
we of course here switch to caching each of the archive-area files
individually rather than having one file that gets overwritten (or
appended to in the case of when we kept the decompressed file).
Gbp-Dch: Short
Closes: #952911
the existing logic was to decompress the contents file from the downloaded
archive to disk, then process it to obtain a package list. the largest
one by far is for 'main'; 'non-free' and 'contrib' are tiny in comparison.
for sid-amd64 currently, the archive file is 37 MB, while the decompressed
file it contains is 592.3 MB.
we always delete the files and download afresh (currently), and a previous
commit optimised by deleting the files once we're done with them to avoid
wasting disk space leaving them behind.
here we switch to storing the downloaded compressed file to disk instead,
reducing disk space usage (and IO) by hundreds of megabytes; piping the
decompression directly into awk instead of having awk read from the stored
file.
this moves the appending of new items into the list back within the archive
area loop, which is fine since we're replacing the file for each loop now
so the previous issue relating to appending is of no concern.
Gbp-Dch: Short
Closes: #952910
Edit: There were four copies of the same logic to keep in sync;
Originally this patch deduplicated each file, but leaving a copy of
the new function in each, thus reducing the duplication but not
eliminating it. A later patch moved it into a shared function file
following further enhancements to the code in question. This has
since been revised to have the function moved to a shared file here,
which simplifies and gives a cleaner diff.
Gbp-Dch: Short
Closes: #952908
These functions are specific to handling packages stored in the
cache, not other files. They are also always used with the same
`cache/packages.` prefix to the path.
Gbp-Dch: Short
Closes: #952916
only a couple were in use and only by unused echo helpers which have now
themselves been removed, so nothing in this file is needed.
Partial fix for #952880
Gbp-Dch: Short
Closes: #952880
all of these echo helpers are essentially 'logging' functions with output
always intended for stdout/stderr. lack of explicit stdout/stderr direction
means that their output could be captured unintentionally should they be
used within a function designed to construct a string.
Gbp-Dch: Short
Closes: #952879
lack of stderr directed output for the `E:` prefix meant that it would
not appear alongside the message in some use cases
Gbp-Dch: Short
Closes: #952878
available values currently are memtest86+|memtest86|none; "false" is
presumably handled for backwards compatibility
there is no need to handle this in individual scripts. the right place
to handle it is in Set_Defaults as now done
Gbp-Dch: Short
Closes: #952866
`false` and `none` make no sense as choices for this option. Here we
replace `false` with `none`, and remove `true`.
Note that `true` was treated as an alias for netinst (see the changes to
source_disk and and binary_disk).
For backwards compatibility we still allow `true` and `false` by converting
them to `netinst` and `none` respectively, whilst printing a warning to
encourage users to move to `netinst`/`none`.
Gbp-Dch: Short
Closes: #952864
including:
- spaces replaced with tabs for consistency
- alignment of `;;` in some case statements changed for consistency
Gbp-Dch: Short
Closes: #952857
This makes it possible to build an image against a first distribution
(--distribution-chroot) and have the resulting image point to another
distribution (--distribution-binary). We can use this to build against a
snapshot and have the result use the original distribution that was
snapshotted.
Closes: #888507
Before Stretch there was an special amd64 kernel in the i386 arch repo.
So if you wanted to install an amd64 kernel alongside an i386 system
you did not need an additional arch repo.
Debian added multiarch support. That way you can install library packages
from multiple architectures on the same machine.
So there is no longer a need for having an amd64 kernel in i386 arch repo.
You can add an amd64 arch repo to an i386 arch system and fetch the amd64
kernel from the am64 arch repo.
live-build can be setup to use several linux kernel flavours in a single
image.
So in the days previous to this patch you could issue:
lb config --linux-flavours "486 amd64"
to use both 486 and amd64 kernel flavours.
Adding additional arch support to linux flavours poses two problems:
* Packages need to have its arch suffix (e.g. amd64:amd64).
If the suffix is not there apt-get insists on search amd64 kernel
package on i386 arch repo and, of course, fails to find it.
* The rest of the code which handles labels (bootloader config files)
or installed filenames (kernel images themselves) do not use the arch suffix.
This patch adds foreign architecture package support to
linux kernel flavours having taken those problems into account.
Practical example usage: i386 system and extra amd64 kernel.
First add amd64 foreign architecture in your i386 system
thanks to:
dpkg --add-architecture amd64
apt-get update
.
Finally enable amd64 kernel from amd64 arch alongside the
i386 system's 686 kernel thanks to:
lb config --architectures i386 --linux-flavours "686 amd64:amd64"
Check_package will just add a missing dependency to the LB_PACKAGES
todo list if it doesn't find it, when build-with-chroot is true, even if
the check was not for the chroot.
Instead error out if the check is not done for the chroot, e.g.
Check_package host /bin/foo foo
Open Network Install Environment is an open image format used by
networking vendor to ship a standardised image for networking white
box switches.
ONIE hardware takes this image at boot and a script to chain load
into the final environment via kexec. We can support Debian and
derivatives on such systems by packing an ISO which then gets
unpacked, kexec'ed and live-booted.
A base ONIE system can be tested in QEMU by building a VM following
these instrunctions:
https://github.com/opencomputeproject/onie/blob/master/machine/kvm_x86_64/INSTALL
Once built, boot onie-recovery-x86_64-kvm_x86_64-r0.iso in QEMU/libvirt
and on the console there will be the terminal prompt. Check the IP
assigned by libvirt and then scp the live image (ssh access is enabled
as root without password...). Then the .bin can be booted with:
ONIE-RECOVERY:/ # onie-nos-install /tmp/live.hybrid.iso-ONIE.bin
The implementation is inspired by ONIE's own scripts that can be found
at:
https://github.com/opencomputeproject/onie/blob/master/contrib/debian-iso/cook-bits.sh
A new option, --onie (false by default) can be set to true to enable
building this new format in addition to an ISO.
An additional option, --onie-kernel-cmdline can be used to specify
additional options that the ONIE system should use when kexec'ing the
final image.
Note that only iso or hybrid-iso formats are supported.
For more information about the ONIE ecosystem see:
http://onie.org
Signed-off-by: Erik Ziegenbalg <eziegenb@Brocade.com>
Signed-off-by: Luca Boccassi <bluca@debian.org>
Support for UEFI Secure Boot is modelled after how it currently works
in Ubuntu and on how it is going to work on Debian.
A minimal bootloader, shim, is used as the first-stage and it then
loads grub. Both have to be signed.
shim-signed is already available in Debian so the filenames are
already established, and the grub2 repository and packaging is common
between the 2 distros so we can already be reasonably sure of what it
is going to be.
So if both are available, copy /usr/lib/shim/shim[x64|aa64].efi.signed
as boot[x64|aa64].efi so that UEFI loads it first, and copy
/usr/lib/grub/[x86_64|arm64]-efi-signed/grub[x64|aa64].efi.signed as
grub[x64|aa64].efi.
This grub2 EFI monolithic image is currently hard-coded in grub2's
repository to look for a config file in efi/debian, so make a copy
of the previously added minimal grub.cfg that loads the real one in
that directory in both the fat32 and ISO 9660 partitions.
The new option --uefi-secure-boot can be set to auto (default,
enable or disable.
In auto, the lack of the signed EFI binaries is intentionally left as a
soft failure - live-build will simply fallback to using the locally
generated non-signed grub2 monolithic EFI binary as the only
bootloader. Given the difficulties surrounding the Secure Boot
signing infrastructure this approach gives the most flexibility and
makes sure things will "just work" once the packages are available,
without the need to change anything in the configuration.
This will also greatly help downstream distributions and users who
want to do self-signing.
The enable or disable options work as expected.
Closes: #821084
Previously, Check_package would only show an error when host packages
are missing on a non-apt system. On apt system, the packages would be
added to _LB_PACKAGES, which causes them to be installed in the chroot,
not in the host (or not at all if Install_package is not called). This
behaviour could break the build.
This applies to either packages that must be present in the host (as
checked with `Check_package host ...`), as well as packages that can be
either in the chroot or host (as checked with `Check_package chroot`)
when LB_BUILD_WITH_CHROOT=false.
Recent versions of Linux, parted or some other bit of software cause
partition devices, like /dev/loop0p1 to be created when running parted
mkpart. However, these devices are not cleaned up when running
losetup -d to remove /dev/loop0 later, so they linger around and confuse
mkfs (which refuses to make a filesystem, thinking there are partitions):
mkfs.fat 4.1 (2017-01-24)
mkfs.vfat: Partitions or virtual mappings on device '/dev/loop0', not making filesystem (use -I to override)
To prevent this behaviour, pass --partscan to losetup when adding a new
partition, to clean up any lingering partitions. It seems losetup does not
accept --partscan when deleting a loop device, to clean up at that point, but
since binary_hdd mounts the partition last, there should not be any lingering
partition devices after live-build is done.
The --partscan option is available since util-linux 2.21 (released in 2012), so
it should be fairly safe to pass it unconditionally.
* Added: functions/bootloaders.sh . This file adds bootloader functions that are heavily used in efi scenarios where a bootloader can act as a first or an extra bootloader.
Since the introduction of the new switch:
--bootloaders
you can setup it like this:
--bootloaders=syslinux,grub-efi
.
This means that syslinux is the first bootloader and grub-efi is the extra bootloader.
* Added new bootloader functions: Check_Non_First_Bootloader and Check_Non_Extra_Bootloader.
These functions let each one of the bootloaders abort the build because
they cannot perform a role either as a first bootloader or as an extra bootloader.
* Added bootloader functions: Check_First_Bootloader_Role, Check_Extra_Bootloader_Role and Check_Any_Bootloader_Role
These functions let bootloaders to force their default role in a single line.
At the same time many binary bootloaders were rewritten to make use of the new bootloader role functions explained above.
These roles were enforced:
binary_grub-legacy : First bootloader
binary_grub-pc : Either first or extra bootloader
binary_syslinux : Either first or extra bootloader
If a bootloader is tried to be used in a role that it's not meant to be used then the build fails because that might lead to a non-bootable system.
Note: OLDIFS use makes IFS to be reset to "" instead to it being unset.
Either we need to detect if old IFS was unset to unset it
or we need a proper way of setting it as a local variable.
Even more IFS it's not currently used in
Check_package (which it's called from: binary_hdd).
we should have a clean way of resetting/unsetting IFS when calling Check_package.
The other approach it's to explicitly define IFS with its default value in the
places inside live-build code where we implicitly suppose that it's going to have
its default value.
Future live-build versions will still allow to use casper,
but its configuration will be done differently by a custom
config tree, rather than embedded and maintenance intensive
code in live-build itself.
Future live-build versions will still allow to use casper,
but its configuration will be done differently by hooks
in the config tree, rather than embedded and maintenance intensive
code in live-build itself.
The list of files passed to Truncate() might contain absolute symlinks
pointing to files outside the chroot, which previously destroyed files on
the build host.
debootstrap is the official tool to bootstrap debian,
cdebootstrap has had the one or other bug making it
broken for times during the release cycles.
The extra effort of supporting both debootstrap
and cdebootstrap is hardly worth it since the bootstrap
stage is cached anyway.
If a local repository path is given as a mirror URL lets bind it into the
chroot. The local repository will be unmounted while processing "remove" or
latest by the exit function.
Comments in package lists used to work in live-build 3.x (although
possibly only as a side-effect of some other logic) and being able to
comment package is a useful feature (to explain why some packages are
included and to make it easy to uncomment some lines to add more useful
packages).
extlinux configuration for the bootable image is installed to
/boot/extlinux, matching the extlinux-install script in Debian.
From the configuration point of view it's still called "syslinux",
the same as the rest of the syslinux family (pxelinux, isolinux etc.)
* Add function to output unique list of foreign architectures from an
expanded package list.
* If foreign architectures are detected, add unique architectures
to dpkg and update apt.
* This requires users to explicitly list at least _one_ package of a
foreign architecture in their package list (e.g. foo:arch) for any other
foreign arch dependencies to be handled appropriately.
If LB_HDD_SIZE is "auto" the size will be determined automatically as before
else it will be the given size in MB.
Also setting the default for this to "auto"
This is 'debranding' in the sense that live-build defaults are for people
building images with a config from scratch, live-build should not
have the options that we use for the official prebuilt debian-live
images as debian-defaults for all images and force that upon everyone
(as defaults). Rather, a 'lb config' should produce a as vanilla as
possible configuration tree.
jnqnfe
Raphaël Hertzog
Roland Clobus
Adrian Gibanel Lopez
Hideki Yamane
Luca Boccassi
Raphaël Hertzog
Matthijs Kooijman
Kristian Klausen
Ben Armstrong
Daniel Baumann
Daniel Reichelt
Fathi Boudra
Jan Blunck
Simon McVittie
Kiel Christofferson
Michael Stummvoll
Daniel Baumann
Richard Nelson