Commit Graph

879 Commits

Author SHA1 Message Date
Luca Boccassi e5492b1c70 Avoid apt-key add and just drop the key in /etc/apt/trusted.gpg.d
Only gnupgv is part of the deboostrap set these days, but apt-key needs
the full gpg (with gpg-agent) which is just a recommends.
Instead just drop the key with an .asc suffix in /etc/apt/trusted.gpg.d
which is supported since apt version 1.4
2018-06-03 22:06:36 +01:00
Luca Boccassi 186765e3fd lb clean: remove ONIE image 2018-04-17 11:39:26 +01:00
Luca Boccassi 277f0cec71 Reconfigure bootstrapped packages after preseeding.
live-build supports preseeding configuration, but the configuration has
to be installed after bootstrapping since it needs the debconf tools to
be applied. But packages that have already been installed and configured
in the bootstrap step will then ignore those preseeded configs.

After applying each preseed file, parse the package list and manually
reconfigure the relevant package(s) so that the configuration will be
applied.
2018-04-10 21:58:00 +01:00
Luca Boccassi 050e637b2c ONIE: do not use package cache, only runs on host
Gbp-Dch: Ignore
2018-04-05 16:24:36 +01:00
Luca Boccassi e47652d841 ONIE: missing dependency on file
A very minimal build worker might not have it, so check for its
presence as well.
2018-04-04 19:57:49 +01:00
Luca Boccassi 2aff516e1f ONIE: Check_package in the host, not the chroot
The binary_onie script works on the host, not in the chroot (if used),
so don't check that the required packages for the script are installed
in the chroot (if present) as they won't be useful.
Check instead on the host.
2018-04-04 19:56:41 +01:00
Luca Boccassi c1948b4183 ONIE: detect initrd compression instead of hard-coding
Instead of hard-coding the decompression and compression formats,
detect them at runtime.
Install the required dependencies as well - they were mistakenly left
out.

Gbp-Dch: Ignore
2018-03-23 20:01:58 +00:00
Luca Boccassi 0e91aeea42 Add Acquire::AllowInsecureRepositories to fix apt-secure in sid
The apt-secure option does not work anymore when building a sid image,
as with apt 1.6 the existing options are no longer enough to get apt
to accept an unsigned repository, which is necessary when using a
local cached repository (offline build).

Pass Acquire::AllowInsecureRepositories "true"; together with the
other options when --apt-secure false is used to fix the issue.
2018-03-23 16:02:52 +00:00
Luca Boccassi 46c9596926 Add options to build ONIE images
Open Network Install Environment is an open image format used by
networking vendor to ship a standardised image for networking white
box switches.

ONIE hardware takes this image at boot and a script to chain load
into the final environment via kexec. We can support Debian and
derivatives on such systems by packing an ISO which then gets
unpacked, kexec'ed and live-booted.

A base ONIE system can be tested in QEMU by building a VM following
these instrunctions:

https://github.com/opencomputeproject/onie/blob/master/machine/kvm_x86_64/INSTALL

Once built, boot onie-recovery-x86_64-kvm_x86_64-r0.iso in QEMU/libvirt
and on the console there will be the terminal prompt. Check the IP
assigned by libvirt and then scp the live image (ssh access is enabled
as root without password...). Then the .bin can be booted with:
 ONIE-RECOVERY:/ # onie-nos-install /tmp/live.hybrid.iso-ONIE.bin

The implementation is inspired by ONIE's own scripts that can be found
at:
https://github.com/opencomputeproject/onie/blob/master/contrib/debian-iso/cook-bits.sh

A new option, --onie (false by default) can be set to true to enable
building this new format in addition to an ISO.
An additional option, --onie-kernel-cmdline can be used to specify
additional options that the ONIE system should use when kexec'ing the
final image.
Note that only iso or hybrid-iso formats are supported.

For more information about the ONIE ecosystem see:
http://onie.org

Signed-off-by: Erik Ziegenbalg <eziegenb@Brocade.com>
Signed-off-by: Luca Boccassi <bluca@debian.org>
2018-03-22 16:14:01 +00:00
Luca Boccassi aa1ae83854 UEFI: use uppercase EFI directory name for Tianocore
The Tianocore reference UEFI implementation, used for example by Qemu,
wants the EFI directory name to be uppercase in the fat32 partition
when Secure Boot is enabled, and will fail to load otherwise.
2018-03-09 20:57:54 +00:00
Luca Boccassi 035518ff69 UEFI: add support for Secure Boot on amd64 and arm64
Support for UEFI Secure Boot is modelled after how it currently works
in Ubuntu and on how it is going to work on Debian.
A minimal bootloader, shim, is used as the first-stage and it then
loads grub. Both have to be signed.
shim-signed is already available in Debian so the filenames are
already established, and the grub2 repository and packaging is common
between the 2 distros so we can already be reasonably sure of what it
is going to be.
So if both are available, copy /usr/lib/shim/shim[x64|aa64].efi.signed
as boot[x64|aa64].efi so that UEFI loads it first, and copy
/usr/lib/grub/[x86_64|arm64]-efi-signed/grub[x64|aa64].efi.signed as
grub[x64|aa64].efi.
This grub2 EFI monolithic image is currently hard-coded in grub2's
repository to look for a config file in efi/debian, so make a copy
of the previously added minimal grub.cfg that loads the real one in
that directory in both the fat32 and ISO 9660 partitions.

The new option --uefi-secure-boot can be set to auto (default,
enable or disable.
In auto, the lack of the signed EFI binaries is intentionally left as a
soft failure - live-build will simply fallback to using the locally
generated non-signed grub2 monolithic EFI binary as the only
bootloader. Given the difficulties surrounding the Secure Boot
signing infrastructure this approach gives the most flexibility and
makes sure things will "just work" once the packages are available,
without the need to change anything in the configuration.
This will also greatly help downstream distributions and users who
want to do self-signing.
The enable or disable options work as expected.

Closes: #821084
2018-03-09 20:57:54 +00:00
Luca Boccassi ac3ed23638 UEFI: add minimal grub.cfg to fat32 partition
On some UEFI implementations, like the AMI found in the Supermicro
X10SDV-TP8F development board, the fat32 partition will be loaded
first and so Grub will set it the root, and then drop to the console
as it cannot find any config on it.
Add a minimal grub.cfg that allows Grub to find the main config on
the ISO 9660 partition and load it.
Closes: #892406
2018-03-09 10:43:39 +00:00
Steven Shiau 0effdbd8ef Add grub-based UEFI boot support for ARM64
Closes: #885692
Fixes: !2
Signed-off-by: Raphaël Hertzog <hertzog@debian.org>
2018-03-02 09:09:29 +01:00
Rohan Garg 952ac834e4 Simplify bootstrapping of foreign architectures with qemu-debootstrap
We already require qemu-user-static for this case, we might as well rely
on the qemu-debootstrap script that this package provides.

Closes: #847919
2018-02-24 17:52:44 +01:00
Raphaël Hertzog 842e971a65 Run mksquashfs with nice -n 19 to not overload the system
Machines tend to become unresponsive during the mksquashfs step.
Avoid this by lowering the priority of the process.

Thanks: Ronny Standtke for the patch.
Closes: #867539
2018-02-24 17:04:04 +01:00
Luca Boccassi 9a0c6102fd Fix build with local offline mirrors
Commit a15b579652 (#775989) dropped an early exit from the
chroot_archives remove step in case the parent mirror chroot and binary
parameters are the same and introduced a regression, as with the
following live-build now fails when the parent mirror is using a file:/
local apt repository (for example when the build worker is offline and
uses a pre-built cache of packages).

Example config:

  lb config --mirror-bootstrap "file:/pkgs" \
    --mirror-chroot "file:/pkgs/" \
    --mirror-binary "file:/pkgs" \
    --parent-mirror-bootstrap "file:/pkgs" \
    --parent-mirror-chroot "file:/pkgs/" \
    --parent-mirror-binary "file:/pkgs" \
    ...

with /pkgs being a directory with the packages for the installation and
the apt metadata (Packages/Sources/Release).

The problem is that, with such a setup, the /pkgs directory is bind
mounted inside the chroot as an optimisation in the install step,
and umounted as one of the first actions in the remove step for
chroot_archives.

Before that fix, the script terminated immediately. But now it
progresses and at the end it tries to run apt update inside the chroot
which will fail since the repository directory has been umounted, and
thus the packages and the apt metadata are no longer available, while
still being listed in /etc/apt/sources.list.

The proposed solution is to umount the local directory at the end of
the remove step, rather than at the beginning.

Closes: #891206
2018-02-23 14:04:59 +00:00
Raphaël Hertzog 96e73960b3 Fix Check_package invocation in binary_hdd for ntfs-3g
/sbin/mkfs.nfts -> /sbin/mkfs.ntfs
2018-02-14 18:49:36 +01:00
Adrian Gibanel Lopez acafe6618b Failsafe entries rework at binary_loopback_cfg
[hertzog@debian.org:
- Fix conflicts due to renamed variables
]

Signed-off-by: Raphaël Hertzog <hertzog@debian.org>
2017-12-21 14:29:58 +01:00
Adrian Gibanel Lopez d3edb76ad4 Fix handling of multiple kernels in binary_loopback_cfg
Now grub.cfg shows all the kernel options. Before this patch when you
had more than two kernels it only showed the auto option.

Signed-off-by: Raphaël Hertzog <hertzog@debian.org>
2017-12-21 14:22:36 +01:00
Adrian Gibanel Lopez f77034606e Updated binary_loopback_cfg so that it uses Stretch's 686 kernel instead of old 486 one.
[hertzog@debian.org:
Also rename the variables for consistency.
]

Signed-off-by: Raphaël Hertzog <hertzog@debian.org>
2017-12-21 14:16:57 +01:00
Raphaël Hertzog 9f3e5fe8d9 Fix the way the .disk/mkisofs file is created
Thanks to Daniel Reichelt <debian@nachtgeist.net> for the patch.

Closes: #881941
2017-11-20 22:20:53 +01:00
Balint Reczey dae94463e4 Don't fail when initramfs is not used
Signed-off-by: Raphaël Hertzog <hertzog@debian.org>
2017-11-20 22:09:49 +01:00
Matthijs Kooijman 72ca3bcb20 Handle hardlinks in binary_hdd
To generate an hdd image, binary_hdd first estimates the needed size of
the image using du. By default, when du finds multiple hardlinked copies
of a file, it counts them only once. However, when the target filesystem
is FAT, which does not support hardlinks, these files will take up more
space when finally copying the contents, breaking the build:

	P: Copying binary contents into image...
	cp: error writing 'chroot/binary.tmp/live/initrd.img-4.9.0-3-amd64': No space left on device
	cp: error writing 'chroot/binary.tmp/efi/boot/bootx64.efi': No space left on device
	cp: error writing 'chroot/binary.tmp/efi/boot/bootia32.efi': No space left on device
	cp: cannot create directory 'chroot/binary.tmp/boot/grub': No space left on device
	cp: cannot create directory 'chroot/binary.tmp/isolinux': No space left on device

To fix this, pass --count-links to du when the target is FAT, to make
the space estimation correct.

This problem is exposed by commit 9c974b26b (Instead of renaming kernel
for syslinux, create hardlinks), which might need to be separately fixed
(to not waste space on FAT targets), but binary_hdd should at least
handle hardlinks more gracefully.
2017-09-01 10:29:20 +02:00
Matthijs Kooijman 036b6161f2 Check all dependencies independent of LB_BUILD_WITH_CHROOT
Since commit fdc9250bc (Changing package dependency checks within chroot
to work outside as well), Check_package automatically checks for
LB_BUILD_WITH_CHROOT and works inside as well as outside of the chroot,
so no need to check LB_BUILD_WITH_CHROOT before calling them.
Install_package and Remove_package are just a no-op when building
without chroot, so they can also be called unconditionally.
Restore_cache and Save_cache do not check LB_BUILD_WITH_CHROOT but it
it should not hurt to call them when not needed (which already happened
in some cases).

This commit makes all Check_package calls unconditional on
LB_BUILD_WITH_CHROOT.

For binary_syslinux, this fixes the check (which used outdated paths
outside the chroot since 7b6dfd9d1), for binary_grub-efi,
binary_package-lists and chroot_package-lists this simplifies the code
(but also causes the check to become package-based instead of file-based
on apt-based systems), and for binary_loadlin and binary_win32-loader
this adds the check outside the chroot which was previously missing.
2017-09-01 10:22:09 +02:00
Raphaël Hertzog ff71712590 Add .disk/mkisofs file recording the xorriso command line
Thanks to Thomas Schmitt <scdbackup@gmx.net> for the suggestion.
2017-08-10 14:53:53 +02:00
Raphaël Hertzog e38ad8ae66 Drop the --hardlinks option from xorriso command line call 2017-08-07 17:27:32 +02:00
Raphaël Hertzog 75aa6dec5b Minor updates to lb_config and its manual page
* Update the manual page with the missiong --bootappend-live-failsafe
  option.
* Keep supporting the former --bootloader (without s).
2016-12-16 10:00:36 +01:00
Raphaël Hertzog 50b6a29523 Cleanup binary_loopback_cfg and offer more freedom in overriding the default grub-pc configuration. 2016-12-14 12:33:07 +01:00
Raphaël Hertzog f7bb86fd58 Clean up grub configuration code
* Use only long kernel names.
* Put advanced options in a submenu.
* Use distro-agnostic labels.
* Don't generate entries with kernel version when we have a single
  version.
2016-12-02 23:04:13 +01:00
Raphaël Hertzog 9c974b26bf Instead of renaming kernel for syslinux, create hardlinks 2016-12-02 15:42:59 +01:00
Raphaël Hertzog 7ad6c408e5 Default for LB_UNION_FILESYSTEM is now "overlay" just like in live-boot. 2016-12-02 15:10:19 +01:00
Raphaël Hertzog 58fd378005 Drop useless code in binary_grub-pc 2016-12-02 15:01:28 +01:00
Raphaël Hertzog 4204a74352 Drop useless code in binary_grub-efi 2016-12-02 14:57:45 +01:00
Raphaël Hertzog fb0d4e2a18 Use a grub theme that integrates reasonably well with the current background picture. 2016-12-02 14:50:21 +01:00
Raphaël Hertzog a15b579652 Drop an optimization in chroot_archives that has undesired side-effects
Thanks to jnqnfe for the patch.

Closes: #775989
2016-11-28 21:48:18 +01:00
Raphaël Hertzog d8ca2041ad Get rid of useless bootstrap_archive-keys script
Closes: #773775
2016-11-28 21:36:57 +01:00
Raphaël Hertzog 63af37b9c5 Copy sub-directories as well as files in grub config directory
This is to cover the case of a theme included in that directory.
2016-11-28 21:05:06 +01:00
Raphaël Hertzog 164f23368e Support “lb config --debootstrap-script <script>”
This option lets you use an alternate bootstrap script when running
debootstrap. Thanks to Sjoerd Simons <sjoerd@debian.org> for the initial
patch.

Closes: #790033
2016-11-28 20:58:19 +01:00
Raphaël Hertzog 822459c7b2 Set xorriso's "modification time" to SOURCE_DATE_EPOCH
Thanks to Chris Lamb <lamby@debian.org> for the patch.

Closes: #831379
2016-11-28 20:58:19 +01:00
Sjoerd Simons c12f7d715e Pass components to debootstrap --second-stage
Commit e24e4b in debootstrap fixed setup_available to work in the
--foreign case (iotw at the second stage). Unfortunately this breaks
things if components aren't passed to the second stage _and_ your main
component isn't called main.

To fix this, pass --components to both the first and second stage
debootstrap when needed.

Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>
2016-11-28 20:58:18 +01:00
Raphaël Hertzog 674794a8f4 Try to reuse /isolinux/splash.png in default grub configuration. 2016-11-28 20:58:18 +01:00
Sophie Brun e92dbc3696 Handle packages.chroot with an explicitly trusted file:// repository
Up to now we created a temporary GPG key that we registered with apt-key
but with the switch to GnuPG 2 by default, this code broke. Now we stop
doing that but we add the “trusted=yes“ attribute in sources.list so
that APT knows that the repository can be trusted even if it's unsigned.

Signed-off-by: Raphaël Hertzog <hertzog@debian.org>
2016-10-28 08:57:37 +02:00
Adrian Gibanel Lopez 7667d5faa7 Readded Installation boot entries to grub configuration files. 2016-08-26 20:23:02 +02:00
Adrian Gibanel Lopez 2d61dcc53c Force the use of insmod all_video in grub.cfg so that we do not boot in blind mode in UEFI mode. 2016-07-31 15:09:13 +02:00
Adrian Gibanel Lopez 7becd08e4d Added EFI support by the means of grub-efi
This work is based on debian-cd team work and uses,
as much as possible, the same mkisofs options
than the Debian Installation CD disk does.

It assumes that /boot/grub/grub.cfg (and other design items)
is generated by: binary_loopback_cfg .

It relies on efi-image and grub-cpmodules being setup
as build scripts on live-build package.
In the future event of these two files being moved
to a binary package (they are originally from:
src: live-installer) the binary_grub-efi script would have
to be rewritten to take the new paths into account.
2016-07-31 15:09:13 +02:00
Adrian Gibanel Lopez 9cebad25f9 Handle multiple bootloaders properly in binary_iso . Force mkisofs to add an additional eltorito entry if we have more than one bootloader. 2016-07-31 15:09:13 +02:00
Adrian Gibanel Lopez 58a97219c9 Stolen efi-image and grub-cpmodules from src:live-installer
These two scripts simplify the creation of efi images based on grub-efi.
I have decided to simply steal them. If I had to include them thanks to a source package that would have mean that an src repo would have to be defined by default.
TODO: Ask in a bug a RFE so that these two scripts are put into a binary that could be consumed by both live-installer and live-build packages.
2016-07-31 15:09:13 +02:00
Adrian Gibanel Lopez c4327f6138 binary_loopback_cfg now renders grub.cfg by default.
The binary parts of grub-pc are left for the original binary_grub-pc.
As a consequence both /boot/grub/grub.cfg and /boot/grub/loopback.cfg files will be present in any Debian Live CD.

This might be useful to be reused from binary_grub-* bootloaders.
2016-07-31 15:09:13 +02:00
Adrian Gibanel Lopez 9d1a983cc8 Added new multi bootloader helper functions
* Added: functions/bootloaders.sh . This file adds bootloader functions that are heavily used in efi scenarios where a bootloader can act as a first or an extra bootloader.

Since the introduction of the new switch:

--bootloaders

you can setup it like this:

--bootloaders=syslinux,grub-efi

.

This means that syslinux is the first bootloader and grub-efi is the extra bootloader.

* Added new bootloader functions: Check_Non_First_Bootloader and Check_Non_Extra_Bootloader.

These functions let each one of the bootloaders abort the build because
they cannot perform a role either as a first bootloader or as an extra bootloader.

* Added bootloader functions: Check_First_Bootloader_Role, Check_Extra_Bootloader_Role and Check_Any_Bootloader_Role

These functions let bootloaders to force their default role in a single line.

At the same time many binary bootloaders were rewritten to make use of the new bootloader role functions explained above.

These roles were enforced:

binary_grub-legacy : First bootloader
binary_grub-pc : Either first or extra bootloader
binary_syslinux : Either first or extra bootloader

If a bootloader is tried to be used in a role that it's not meant to be used then the build fails because that might lead to a non-bootable system.
2016-07-31 15:08:11 +02:00
Adrian Gibanel Lopez 0624064d44 Remove repeated LB_PRIMARY_BOOTLOADER definition 2016-07-31 14:59:33 +02:00