back in v4.0~a6-1 a transition process was started to move the live-build
config to a new format. the new format was INI style, and required
parsing functions to read/write values, compared to the existing format
which was just shell script code setting variables.
this partial transition is the explanation for the existence of the
`New_configuration()` function, and understanding this is important to
understanding the purpose of it - it is not in fact intended for creating
a new configuration, it is just related to the new config format
transition.
the positives of the new format were that it was somewhat cleaner looking,
while the negative was the terrible relative efficiency.
the file `config/build` was created to hold options in this new format.
the transition was only ever completed for a handful of config options:
- architecture
- archive areas and parent archive areas
- live image name
- live image type
a 'configuration version' attribute was also saved, which is not used by
anything.
the bootstrap-mirror and parent-bootstrap-mirror attributes are pointlessly
stored in it seemingly resulting from work done in v4.0~a17-1. (they are
also stored in another config file from which the value is actually used).
it in fact seems to have been a source of confusion for Raphaël in
authoring 44b9b0a650, since the new
`[parent]-distribution-{chroot|binary}` options it introduced were stored
both in `config/bootstrap` and in `config/build`, while only used from the
former. i expect, understandably, that he thought that `config/build` was
just an information file.
Gbp-Dch: Short
live-build might be run from a local folder rather than the system
installation, so the dpkg version number should not override the version
picked up from VERSION
if we care about the possibility of the installed package version
potentially differing from the version of the git checkout, or
whatever, then this should be printed alongside it, as now done.
Gbp-Dch: Ignore
[Raphaël Hertzog: tweak to apply on top of my changes]
thus it can correctly indicate success/fail status instead of always
indicating failure. when a user asks for usage with -u|--usage then we
should exit in success mode rather than failure as when usage in printed
in response to incorrect usage.
Gbp-Dch: Short
this means that the usage goes from:
lb {-h|--help|-u|--usage|-v|--version}
lb COMMAND [OPTIONS]
to:
lb {-h|--help|-u|--usage|-v|--version}
lb [FRONTEND_OPTIONS] COMMAND [COMMAND_OPTIONS]
though it is probably not worth is to update the description in the
manpages...? hmm...
so for instance this matters for color control with --color|--no-color
(you already had full control via environment vars). previously you could
do `lb COMMAND --no-color` to turn off colour, only to find that output
at the frontend level was still coloured (the option is processed at the
command context level, not the frontend), so you might try to instead use
`lb --no-color COMMAND`, only to find that this was not supported. Well
now it is, and used at the frontend level will fully control colour output
(after the command is processed anyway).
the full set of common options are thus available (except --force) at the
frontend level, and thus for instance all Echo_*() helpers used in the
frontend will work correctly after args are processed.
furthermore usage like `lb --color --help` will actually work. (not that
color is used there, but this previously would have failed with the
frontend treating the `--color` argument as the command; that's the point!)
Gbp-Dch: Short
this was previously not done in 8b109ffb96
to keep the renaming simple, but leaving the variable plural is a cause
for confusion.
since this property is stored in the INI style config/build config file
rather than a shell script based one, at the property there is already
singular, there was no need for a backwards compatibility hack.
Gbp-Dch: Short
note that the bit of code removed from source_debian relies upon a
variable LB_TASKS which itself is an old leftover artefact from before
v4.0.
Gbp-Dch: Short
since everywhere where 'apt' is a permitted value, 'apt-get' is also, it
just wasn't listed in the option's documentation and thus was also not
listed in the new validation check.
Gbp-Dch: Short
80aa5ab611 implemented a hack to handle
replacement of LB_LINUX_FLAVOURS with LB_LINUX_FLAVOURS_WITH_ARCH in
config files, but implemented it in the wrong place.
adding a conditional conversion within the config file meant that the old
value would only be read from **new** config files that are created
obviously without it, including re-saved configs if `lb config` were
re-run with additional options (not recommended). any existing value in an
existing config file would actually be ignored.
the right place to read the old value was in the Set_defaults() function
(since renamed).
a second issue also existed with the hack, it failed to excape the `$`
and thus printed the existing value of $LB_LINUX_FLAVOURS into the
conditional check being constructed in the config file, instead of
printing the name of the variable. the check embedded into the config
file thus became this on an amd64 machine:
```
if [ -n "amd64" ]
then
LB_LINUX_FLAVOURS_WITH_ARCH="amd64"
fi
```
which is clearly not what was intended.
Gbp-Dch: Short
while helpful for users to know the defaults, the values printed as the
supposed defaults for most are actually the same values as being
configured, or in some cases a piece of text "autodetected or empty", and
thus the information is completely wrong and actually unhelpful since it
misinforms the user.
fixing this to give the real defaults is very much non-trivial.
as a workaround users wanting to know the default for an option can always:
a. use `lb config` wit no options (or auto) in a clean directory and thus
get a config with all defaults.
b. look at the live-build code.
if they just want to reset an option, they can also just comment it out.
Gbp-Dch: Short
Closes: #904614
move away from the somewhat config file grouping based organisation to
an alphabetised list, after grouping into script-specific; general;
build-specific and other.
the config file based organisation was a bad choice, making it hard to
find the right place to insert options for instance.
Gbp-Dch: Short
running `lb config --validate` causes the script to stop after running
the validation check on the config compiled at that point, prior to
writing the config to disk.
this gives users the ability to check the validity of a config without
modifying or rewriting the saved config.
note that if users provide new config options alongside --validate, these
are taken into account in the check performed.
the 'check complete' message will not be seen if an error is reported by
the check function, while it will be seen if only warnings are given, but
it would require a redesign of the validation check function to make any
improvement in that area, and it's perhaps not worth it.
Gbp-Dch: Short
it mostly applies defaults where a value does not exist, but does more in
some cases. the new name better reflects its usage and functionality.
Gbp-Dch: Short
whilst some parts of the codebase were set up to work with multiple types
specified, others did not work with it and would not necessarily be easy
to adjust. this thus makes some tweaks to adjust things accordingly.
- option renamed to singular form (maintaining backwards compatibility)
- a validation check has been added
- unnecessary glob style type references fixed
- checks with In_list changed to a direct singular comparison
- typo of type "netboot" written as just "net" fixed (though unreachable
so of no consequence; really the code could be removed but it's trivial)
Gbp-Dch: Short
progress-linux, as discussed in MR #142 ([1]) is a little known distro,
which appears to be little more than a personal project of the original
author of live-build.
given that, the expense of maintaining all of these old hacks for it
cannot be justified. it is not known whether or not live-build is even
used with respect to it since the author abandoned live-build some
years ago.
also, at least one past change in live-build possibly broke progress-linux
compatibility anyway, which would have required progress-linux users of
live-build to use a custom progress-linux config, or a progress-linux
fork of live-build, and there is no knowing how much of the hacks in this
"upstream" codebase any user of progress-linux currently relies upon.
and again, progress-linux appears to just be a personal project of
Daniel's, with afaik very little userbase. (Daniel seems to be the only
developer working on the project which speaks to how small it is).
[1]: https://salsa.debian.org/live-team/live-build/-/merge_requests/142
Gbp-Dch: Short
when the --fdisk and --losetup options were removed, the entries in the
getopt option list should have remained for backwards compatibility such
that the usage warnings can kick in instead of unknown option errors.
Gbp-Dch: Ignore
as suggested by Raphaël
rather than have fixed stagefile filename strings at all in the scripts,
use `$(basename $0)` to use the name of the script (which is the same for
almost all cases anyway, and the stage files are supposed to be almost
exclusively unique per-script). we can thus simplify things by determining
the filename for most use cases within the functions themselves.
this does change the file used by a couple of scripts, affecting backwards
compatibility of executing live-build upon an existing partially or fully
completed build:
- binary_grub-pc used "binary_grub"
- chroot_includes used "includes.chroot"
care had to be taken for the following cases:
- there are some cases like bootstrap_cache, source_debian and
bootstrap_debootstrap which are dealing with more than one file, and/or
otherwise a filename that is not specific to the script itself exactly,
or should not be based upon its name.
- some cases like chroot_cache, bootstrap_cache and
chroot_install-packages need to append something to the end of the name
depending upon which pass/action mode the script is being executed with.
- furthermore in the bootstrap_cache case one of the filenames is used
within the bootstrap_debootstrap and thus needs very careful handling
to be certain that a change in filename of bootstrap_cache does not
break bootstrap_debootstrap.
Gbp-Dch: Short
- avoid all need to pass ".build/" path in stage file names into the
functions
- add a helper to remove a stage file (required to complete the above
properly)
- avoid duplicating filenames within scripts which makes them prone to
mistakes (some instances of which I've actually encountered and had
to fix)
Gbp-Dch: Short
all vars affected have been carefully checked to be quite certain
that they are definitely local
where variable is assigned the return value of a function/command, the
local "declaration" is deliberately done on a separate line, since
`local FOO` is actually treated itself as a command rather than a
declaration; will thus always cause $? to be zero, and thus if done on
the same line as such an assignment can not only clobber $? but in doing
so unintentionally blocks failure of a command from triggering the
expected exit from having `set -e`.
also, from testing, i have found that when assigning "${@}" this must be
done on a separate line confusingly as otherwise an error occurs.
Gbp-Dch: Short
Instances of:
if [ $(which <command> ]
have been replaced with:
if command -v <command> >/dev/null
which is considered to be more robust in a range of environments.
scripts/build/chroot_archives: line 259:
if [ "${LB_APT}" = "aptitude" ] && [ ! $(Chroot chroot "which aptitude") ]
has been left untouched because the chroot might require a more complex command
which would need more testing.
manpages/Makefile: line 42:
@if [ ! -x "$$(which po4a 2>/dev/null)" ]; \
has been left untouched because I am not sufficiently familiar with makefiles.
677415f6d7 (2007) in v1.0~a2-1 added a hack
relating to the loop-aes-utils package and losetup. this commit bundled
a bunch of changes, it was not specific to the hack, and so info about the
hack is limited to a brief comment included within the related change in
defaults:
```
# Workaround for loop-aes-utils divertion
# (loop-aes-utils' losetup lacks features).
```
though it is very similar to the removed fdisk hack in that it seems that
one package may replace a binary from another, moving the original to a
new location, and this hack gives the user the opportunity to select the
original instead of the one put in its place, for use in LB.
the comment mentions a package called loop-aes-utils as being the package
that performs such a diversion, and that the need for the hack was that
losetup itself lacked features, presumably encryption support, and it is
clear that it is the losetup binary that is the focus of the diversion.
looking into the history of loop-aes-utils a little, this package was
dropped from debian back in 2012 (#680748), favouring encrytion support of
dm-crypt/cryptsetup.
double checking file contents of packages, only the mount package carries
an /sbin/losetup file, so presumably this means that dm-setup/cryptsetup
do not perform such a diversion of losetup (i.e. their use is exclusively
done directly).
since the possible diversion is simply gone, that completely removes any
point in having the hack of giving users choice between losetup and the
diverted one. so let's remove this obsolete hack...
8321653cb3 (from 2007) introduced a hack to
work around bug #445304 in gnu-fdisk for users who may have replaced fdisk
with the classic gnu version. the hack allowed users to select an alternate
fdisk binary to use to work around the buggy binary.
bug #445304 is marked as found in v1.0-1 and fixed in v1.2-1, though may
have been fixe din v1.1. it was marked fixed in 2009.
checking the package archive, gnu-fdisk does not actually exist anymore
in debian, with one exception - it is available for arm64 on sid via
debports, and that version is 1.3 so thus includes the necessary fix
anyway.
it is thus pointless now that we still carry this hack.
Gbp-Dch: Short
Current versions of the project files are built upon versions published
and licensed by Daniel Baumann, but are modified copies of those files and
thus need to be marked as such per licensing requirements (afaik he did
not pass along ownership / licensing rights to anyone when he left the
project). We should also be careful to not be misrepresenting such
modified copies as being attributed to Daniel.
Adding a new copyright line referring to "The Debian Live team" should
suffice for this.
The authorship block in man pages has also similarly been updated.
Notes:
- tweaked a copy of daniel copyright lines stating 2014 instead of 2015.
both of these cases were in files that i had personally introduced in
some of my past merged commits that moved some code around. i don't know
why they stated 2014.
- binary_onie was introduced in 2018, so that has a 2018 date instead of
2016 unlike the rest.
- 'efi-image' is a 3rd-party (Canonical Ltd) work that we bundle, but it
has been modified by 674794a8f4 and
36a3ba7634 so I similarly added a
debian live copyright line.
- 'grub-cpmodules' is similar. it was only changed by the indentation fix
of 36a3ba7634 but modification is
modification, and this does help cover any possible future changes that
might be made.
all scripts use `set -e` which means that if getop fails, the subsequent
error check that would print an error in addition to any printed by getopt
itself would never actually be reached.
the first though here would be to remove the pointless error check, but
getopt does not include the word "error" with an unrecognised option
failure, nor does it use colour to highlight problems, both of which mean
that it is a little lacking in terms of highlighting problems to users.
thus we properly capture and use the exit code here and output an
appropriate message per invalid argument vs getopt internal error.
also, removed the redundant stderr redirection which is already done
by Echo_error().
Gbp-Dch: Short
- prefer using `which` over hard coded paths
- it is redundant to check that the bin pointed to the return of
`which` exists and is executable, `which` already gives us
assurance of that if it returns true!
- the redirection of output (`2>/dev/null`) seems to be
unnecessary from my testing.
the instances relatnig to fdisk and losetup in functions/defaults.sh have
been left as they are since they get executed by `lb config` which can run
without sudo elevation unlike `lb build` and in that case `which` would
fail to find these binaries resulting in error.
this also fixes a bug showing an error for missing debootstrap - this tool
requires sudo privileges to run and thus is not found via a none elevated
which search.
Gbp-Dch: Short
Closes: #952927
Lyndon Brown
Raphaël Hertzog
Luca Boccassi
johnraff