Commit Graph

67 Commits

Author SHA1 Message Date
Roland Clobus 0183c73aba
Set the timestamp inside hdd images.
Similar to iso images, the timestamp of a file/folder inside the binary
image cannot be newer than SOURCE_DATE_EPOCH.
Also set the timestamp of the hdd image.
2022-01-22 18:16:58 +01:00
Roland Clobus 2bfdb3cb44
Bugfix: The arguments to the 'tr' command must be quoted. 2022-01-08 13:39:48 +01:00
Roland Clobus eed8d293b1
Bugfix: Do not copy the content of e.g. /dev/shm when creating hdd
images

Discovered when building with '--binary-image hdd' on /dev/shm
2022-01-08 10:33:55 +01:00
Lyndon Brown 18eefdd6f3 s/Remove_package/Remove_packages/
it removes one or more, so should be plural for clarity

Gbp-Dch: Short
2020-05-05 19:55:28 +01:00
Lyndon Brown 919a464b64 s/Install_package/Install_packages/
it installs one or more, so should be plural for clarity

Gbp-Dch: Short
2020-05-05 19:55:28 +01:00
Lyndon Brown d7883ec32b config: improve BIOS/EFI bootloader selection handling
the design choice from when EFI support was introduced was to change
`--bootloader` to `--bootloaders`, with users specifying their selection
of BIOS and EFI bootloaders together. at this time there were not even any
decent validation checks being performed, and invalid combinations could
cause some chaos.

since then proper validation was put in place, including checking that
only a single instance of each of BIOS and EFI bootloaders exists in the
selection.

here we tweak things such that we stick with the same option, but we split
the selection up such that we store the BIOS and EFI selections separately
within the saved config file, and offer it up to scripts to help simplify
those scripts.

we must however retain support for splitting from the combined option,
both because we still use it in the combined option, and for backwards
compatibility with older saved configs.

Gbp-Dch: Short
2020-05-04 21:40:13 +01:00
Lyndon Brown f3a818da9a binary_hdd: remove incorrect local scope
the "FIXME" function wrapper is just to temporarily disable the code.

Gbp-Dch: Ignore
2020-05-04 21:40:08 +01:00
Lyndon Brown 976be9f44b indentation fix
Gbp-Dch: Ignore
2020-05-04 17:40:23 +01:00
Lyndon Brown cb2d9a0fe4 fix another overlooked "grub" -> "grub-legacy" instance
Gbp-Dch: Ignore
2020-05-04 17:37:14 +01:00
Lyndon Brown 500f205073 config: s/LIVE_IMAGE_TYPE/LB_IMAGE_TYPE/
no backwards compatibility hack for reading the old var from existing
saved config used because this was previously stored in the alternate
format config/build file.

Gbp-Dch: Short
2020-04-23 16:54:31 +00:00
Lyndon Brown 83d9145257 config: s/LIVE_IMAGE_NAME/LB_IMAGE_NAME/
no backwards compatibility hack for reading the old var from existing
saved config used because this was previously stored in the alternate
format config/build file.

Gbp-Dch: Short
2020-04-23 16:54:31 +00:00
Lyndon Brown 6cc7250954 rename LB_ARCHITECTURES to LB_ARCHITECTURE
this was previously not done in 8b109ffb96
to keep the renaming simple, but leaving the variable plural is a cause
for confusion.

since this property is stored in the INI style config/build config file
rather than a shell script based one, at the property there is already
singular, there was no need for a backwards compatibility hack.

Gbp-Dch: Short
2020-04-23 15:54:19 +02:00
Lyndon Brown 39e4d3e3cb --binary-images can support only a single type
whilst some parts of the codebase were set up to work with multiple types
specified, others did not work with it and would not necessarily be easy
to adjust. this thus makes some tweaks to adjust things accordingly.

 - option renamed to singular form (maintaining backwards compatibility)
 - a validation check has been added
 - unnecessary glob style type references fixed
 - checks with In_list changed to a direct singular comparison
 - typo of type "netboot" written as just "net" fixed (though unreachable
   so of no consequence; really the code could be removed but it's trivial)

Gbp-Dch: Short
2020-04-23 14:51:09 +02:00
Lyndon Brown f73797158e bootloaders: improve bootloader list handling
two parts of the code worked with both comma and space separated lists,
while two others only worked with comma separated.

swapping out commas with spaces when we setup the var in
Set_config_defaults() means that individual scripts no longer need to worry
about it and everything supports both; and that we can avoid the
IFS/OLDIFS mess.

Gbp-Dch: Short
2020-04-23 11:52:04 +01:00
Lyndon Brown 286caf03be fix outdated grub-legacy bootloader name references 2020-04-23 11:46:36 +01:00
Lyndon Brown 6fa4b5bae9 binary_hdd: fix variable name typo 2020-04-23 10:14:55 +00:00
Lyndon Brown fb0790cc43 stagefiles: s/Require_stagefile/Require_stagefiles/
this function takes one or more required stage fileS _plural_, and exits
if any are missing (or at least it does now after the refactor).

let's rename it to make things more clear

Gbp-Dch: Short
2020-03-17 22:59:37 +00:00
Lyndon Brown fe9195b59c stagefiles: further robustify with auto filenames
as suggested by Raphaël

rather than have fixed stagefile filename strings at all in the scripts,
use `$(basename $0)` to use the name of the script (which is the same for
almost all cases anyway, and the stage files are supposed to be almost
exclusively unique per-script). we can thus simplify things by determining
the filename for most use cases within the functions themselves.

this does change the file used by a couple of scripts, affecting backwards
compatibility of executing live-build upon an existing partially or fully
completed build:
 - binary_grub-pc used "binary_grub"
 - chroot_includes used "includes.chroot"

care had to be taken for the following cases:
 - there are some cases like bootstrap_cache, source_debian and
   bootstrap_debootstrap which are dealing with more than one file, and/or
   otherwise a filename that is not specific to the script itself exactly,
   or should not be based upon its name.
 - some cases like chroot_cache, bootstrap_cache and
   chroot_install-packages need to append something to the end of the name
   depending upon which pass/action mode the script is being executed with.
 - furthermore in the bootstrap_cache case one of the filenames is used
   within the bootstrap_debootstrap and thus needs very careful handling
   to be certain that a change in filename of bootstrap_cache does not
   break bootstrap_debootstrap.

Gbp-Dch: Short
2020-03-17 18:57:02 +00:00
Lyndon Brown 04d9ee0211 stagefiles: simplify & robustify
- avoid all need to pass ".build/" path in stage file names into the
   functions
 - add a helper to remove a stage file (required to complete the above
   properly)
 - avoid duplicating filenames within scripts which makes them prone to
   mistakes (some instances of which I've actually encountered and had
   to fix)

Gbp-Dch: Short
2020-03-17 18:57:02 +00:00
Lyndon Brown c55eb8a0c3 use local scope for private function vars
all vars affected have been carefully checked to be quite certain
that they are definitely local

where variable is assigned the return value of a function/command, the
local "declaration" is deliberately done on a separate line, since
`local FOO` is actually treated itself as a command rather than a
declaration; will thus always cause $? to be zero, and thus if done on
the same line as such an assignment can not only clobber $? but in doing
so unintentionally blocks failure of a command from triggering the
expected exit from having `set -e`.

also, from testing, i have found that when assigning "${@}" this must be
done on a separate line confusingly as otherwise an error occurs.

Gbp-Dch: Short
2020-03-16 22:10:03 +00:00
Lyndon Brown a46985b08a fix comment typo
Gbp-Dch: Ignore
2020-03-13 16:19:59 +00:00
jnqnfe b27927724a locks: tidy lock acquisition
Combine the check+create done in each script. (The original functions
are still callable as before, but a new combined `Aquire_lockfile`
function can be called instead, as now used).

Note, a further simplification could be done in removing the passing of
the lock filename in as a parameter since every use of the functions is
with ".lock". The lock functions already have a fallback to ".build/lock"
though. Checking the history, the fallback used to be for a system wide
lock, which was then replaced with this config-tree specific one. As long
as that is not used implicitly by 3rd-party hooks then surely we are free
to change the fallback to ".lock" and further remove passing in a name as
a param...?

history:
db5d2b0dcd
0aa8289a37

Gbp-Dch: Short
Closes: #952918
2020-03-13 15:37:38 +00:00
Lyndon Brown 9f42bc1bdc fix splitting bootloader string
the existing solution simply did not work
2020-03-13 14:17:45 +00:00
Lyndon Brown 1e0339a4e3 remove obsolete loop-aes-utils related losetup hack
677415f6d7 (2007) in v1.0~a2-1 added a hack
relating to the loop-aes-utils package and losetup. this commit bundled
a bunch of changes, it was not specific to the hack, and so info about the
hack is limited to a brief comment included within the related change in
defaults:
```
# Workaround for loop-aes-utils divertion
# (loop-aes-utils' losetup lacks features).
```
though it is very similar to the removed fdisk hack in that it seems that
one package may replace a binary from another, moving the original to a
new location, and this hack gives the user the opportunity to select the
original instead of the one put in its place, for use in LB.

the comment mentions a package called loop-aes-utils as being the package
that performs such a diversion, and that the need for the hack was that
losetup itself lacked features, presumably encryption support, and it is
clear that it is the losetup binary that is the focus of the diversion.

looking into the history of loop-aes-utils a little, this package was
dropped from debian back in 2012 (#680748), favouring encrytion support of
dm-crypt/cryptsetup.

double checking file contents of packages, only the mount package carries
an /sbin/losetup file, so presumably this means that dm-setup/cryptsetup
do not perform such a diversion of losetup (i.e. their use is exclusively
done directly).

since the possible diversion is simply gone, that completely removes any
point in having the hack of giving users choice between losetup and the
diverted one. so let's remove this obsolete hack...
2020-03-12 10:31:39 +00:00
Lyndon Brown d9f353c737 remove obsolete fdisk hack
8321653cb3 (from 2007) introduced a hack to
work around bug #445304 in gnu-fdisk for users who may have replaced fdisk
with the classic gnu version. the hack allowed users to select an alternate
fdisk binary to use to work around the buggy binary.

bug #445304 is marked as found in v1.0-1 and fixed in v1.2-1, though may
have been fixe din v1.1. it was marked fixed in 2009.

checking the package archive, gnu-fdisk does not actually exist anymore
in debian, with one exception - it is available for arm64 on sid via
debports, and that version is 1.3 so thus includes the necessary fix
anyway.

it is thus pointless now that we still carry this hack.

Gbp-Dch: Short
2020-03-11 19:06:54 +00:00
Lyndon Brown 7a4a9f94b8 amend copyright & licensing blocks
Current versions of the project files are built upon versions published
and licensed by Daniel Baumann, but are modified copies of those files and
thus need to be marked as such per licensing requirements (afaik he did
not pass along ownership / licensing rights to anyone when he left the
project). We should also be careful to not be misrepresenting such
modified copies as being attributed to Daniel.

Adding a new copyright line referring to "The Debian Live team" should
suffice for this.

The authorship block in man pages has also similarly been updated.

Notes:
 - tweaked a copy of daniel copyright lines stating 2014 instead of 2015.
   both of these cases were in files that i had personally introduced in
   some of my past merged commits that moved some code around. i don't know
   why they stated 2014.
 - binary_onie was introduced in 2018, so that has a 2018 date instead of
   2016 unlike the rest.
 - 'efi-image' is a 3rd-party (Canonical Ltd) work that we bundle, but it
   has been modified by 674794a8f4 and
   36a3ba7634 so I similarly added a
   debian live copyright line.
 - 'grub-cpmodules' is similar. it was only changed by the indentation fix
   of 36a3ba7634 but modification is
   modification, and this does help cover any possible future changes that
   might be made.
2020-03-11 13:51:19 +00:00
Lyndon Brown d6096622f9 help/usage: remove pointless vars
build scripts never call Help() and so the empty HELP strings are pointless.
(when called with --help they call Man()).

Closes: #952859
Gbp-Dch: Short
2020-03-10 12:45:23 +00:00
jnqnfe b49abcc1a8 tidy script init (1/4) - arg and config processing
Partial fix for #952919

Gbp-Dch: Short
2020-03-10 12:39:37 +00:00
Lyndon Brown 8545c048c6 use actual boolean state for some simple vars
Gbp-Dch: Ignore
2020-03-09 11:21:24 +00:00
jnqnfe 055907592c cache: clarify and simplify package cache save/restore
These functions are specific to handling packages stored in the
cache, not other files. They are also always used with the same
`cache/packages.` prefix to the path.

Gbp-Dch: Short
Closes: #952916
2020-03-05 22:09:20 +00:00
jnqnfe a67e01638f help/usage: fix overly complex script description handling
Closes: #952887
2020-03-05 20:48:49 +00:00
Lyndon Brown 5c3679c882 add missing shebangs to temp generated shell code files
Closes: #952863
2020-03-05 19:25:13 +00:00
Raphaël Hertzog 96e73960b3 Fix Check_package invocation in binary_hdd for ntfs-3g
/sbin/mkfs.nfts -> /sbin/mkfs.ntfs
2018-02-14 18:49:36 +01:00
Matthijs Kooijman 72ca3bcb20 Handle hardlinks in binary_hdd
To generate an hdd image, binary_hdd first estimates the needed size of
the image using du. By default, when du finds multiple hardlinked copies
of a file, it counts them only once. However, when the target filesystem
is FAT, which does not support hardlinks, these files will take up more
space when finally copying the contents, breaking the build:

	P: Copying binary contents into image...
	cp: error writing 'chroot/binary.tmp/live/initrd.img-4.9.0-3-amd64': No space left on device
	cp: error writing 'chroot/binary.tmp/efi/boot/bootx64.efi': No space left on device
	cp: error writing 'chroot/binary.tmp/efi/boot/bootia32.efi': No space left on device
	cp: cannot create directory 'chroot/binary.tmp/boot/grub': No space left on device
	cp: cannot create directory 'chroot/binary.tmp/isolinux': No space left on device

To fix this, pass --count-links to du when the target is FAT, to make
the space estimation correct.

This problem is exposed by commit 9c974b26b (Instead of renaming kernel
for syslinux, create hardlinks), which might need to be separately fixed
(to not waste space on FAT targets), but binary_hdd should at least
handle hardlinks more gracefully.
2017-09-01 10:29:20 +02:00
Adrian Gibanel Lopez 9d1a983cc8 Added new multi bootloader helper functions
* Added: functions/bootloaders.sh . This file adds bootloader functions that are heavily used in efi scenarios where a bootloader can act as a first or an extra bootloader.

Since the introduction of the new switch:

--bootloaders

you can setup it like this:

--bootloaders=syslinux,grub-efi

.

This means that syslinux is the first bootloader and grub-efi is the extra bootloader.

* Added new bootloader functions: Check_Non_First_Bootloader and Check_Non_Extra_Bootloader.

These functions let each one of the bootloaders abort the build because
they cannot perform a role either as a first bootloader or as an extra bootloader.

* Added bootloader functions: Check_First_Bootloader_Role, Check_Extra_Bootloader_Role and Check_Any_Bootloader_Role

These functions let bootloaders to force their default role in a single line.

At the same time many binary bootloaders were rewritten to make use of the new bootloader role functions explained above.

These roles were enforced:

binary_grub-legacy : First bootloader
binary_grub-pc : Either first or extra bootloader
binary_syslinux : Either first or extra bootloader

If a bootloader is tried to be used in a role that it's not meant to be used then the build fails because that might lead to a non-bootable system.
2016-07-31 15:08:11 +02:00
Adrian Gibanel Lopez 0624064d44 Remove repeated LB_PRIMARY_BOOTLOADER definition 2016-07-31 14:59:33 +02:00
Kristian Klausen 489a09ba92 Disable EXT4 64bit features
e2fsprogs 1.43 enabled ext4 "64bit" feature as default, which broke ext4
HDD images as EXTLINUX doesn't support it.
2016-06-18 19:36:16 +02:00
Kristian Klausen a8cc708427 Correct syslinux/extlinux mbr.bin path (second try)
dash doesn't support parameter expansion it seems.
So implemented with "tr" as in binary_syslinus.
2016-05-26 21:26:36 +02:00
Kristian Klausen ddc6659002 Correct syslinux/extlinux mbr.bin path
Fix #773833
2016-05-26 17:34:22 +02:00
Kristian Klausen 090c6f3cea Revert "Correct syslinux mbr.bin path."
This reverts commit 5161dcdf3b.
2016-05-26 17:33:20 +02:00
Maximilian Mehnert 2c118b384c scripts/build/binary_hdd: Fix code style 2015-12-15 13:59:49 +00:00
Maximilian Mehnert 46920d22bd Old BIOS: start partition at specific place
Some BIOSes dont't boot from partitions starting at sector 1024.
Some are even more peculiar and only start from sector 63.
This patch adds an option for the binary_hdd target to manually
configure the partition start.
2015-09-23 10:48:46 +02:00
Adrian Gibanel Lopez d515ae5684 Multi bootloader support
Note: OLDIFS use makes IFS to be reset to "" instead to it being unset.
Either we need to detect if old IFS was unset to unset it
or we need a proper way of setting it as a local variable.
Even more IFS it's not currently used in
Check_package (which it's called from: binary_hdd).
we should have a clean way of resetting/unsetting IFS when calling Check_package.

The other approach it's to explicitly define IFS with its default value in the
places inside live-build code where we implicitly suppose that it's going to have
its default value.
2015-08-29 23:56:41 +02:00
Carlos Zuferri d43874c405 Adding missing file extension to hdd images. 2015-08-27 10:17:22 +02:00
Maximilian Mehnert 5161dcdf3b Correct syslinux mbr.bin path. 2015-08-10 21:57:51 +02:00
Daniel Baumann 242aef5d83 Updating year in copyright notices to 2015. 2015-01-04 20:05:44 +01:00
Daniel Baumann 93592a81b6 Dropping some sparc leftovers (Closes: #774196). 2015-01-04 18:58:35 +01:00
Daniel Baumann fdc9250bca Changing package dependency checks within chroot to work outside as well. 2015-01-04 18:19:47 +01:00
Daniel Baumann 1f7f6dd88a Cleaning up from python removal (LIVE_IMAGE_ARCHITECTURE). 2014-12-10 07:39:58 +01:00
Simon McVittie 5d6574552f Add support for making ext[234] or btrfs hdd images bootable with extlinux (Closes: #745816)
extlinux configuration for the bootable image is installed to
/boot/extlinux, matching the extlinux-install script in Debian.

From the configuration point of view it's still called "syslinux",
the same as the rest of the syslinux family (pxelinux, isolinux etc.)
2014-04-27 12:37:37 +02:00