Commit Graph

970 Commits

Author SHA1 Message Date
Lyndon Brown 8a1bbe0841 arches: tidy
Gbp-Dch: Ignore
2020-05-04 23:22:10 +01:00
Lyndon Brown 08c86dc8ac arches: fix some Check_architectures() mess
- the comparison with LB_ARCHITECTURE seems to be based upon an old idea
   of it potentially being intended in the past to hold more than one
   arch (when it's name as also plural), but it in fact only holds one.
 - similarly LB_BOOTSTRAP_QEMU_ARCHITECTURE is singular, so the comparison
   with the set of params is bad.
 - storing $@ in a variable before then looping is not a good idea
 - script name was not being passed to the warnings
 - can simplify by just returning in the valid case

i'm very suspicious about the correctness of the qemu bits, but that
can wait for now.

Gbp-Dch: Short
2020-05-04 23:21:44 +01:00
Lyndon Brown d7883ec32b config: improve BIOS/EFI bootloader selection handling
the design choice from when EFI support was introduced was to change
`--bootloader` to `--bootloaders`, with users specifying their selection
of BIOS and EFI bootloaders together. at this time there were not even any
decent validation checks being performed, and invalid combinations could
cause some chaos.

since then proper validation was put in place, including checking that
only a single instance of each of BIOS and EFI bootloaders exists in the
selection.

here we tweak things such that we stick with the same option, but we split
the selection up such that we store the BIOS and EFI selections separately
within the saved config file, and offer it up to scripts to help simplify
those scripts.

we must however retain support for splitting from the combined option,
both because we still use it in the combined option, and for backwards
compatibility with older saved configs.

Gbp-Dch: Short
2020-05-04 21:40:13 +01:00
Lyndon Brown 3f840aeeba fully validate BIOS/EFI bootloader combinations
thus far, config bootloader validation only did the basic check that each
bootloader specified was a known and supported bootloader, it did not check
combinations.

it now checks combinations, and strips out the previous "bootloader role"
stuff.

the no-bootloaders warning is duplicated, covering two slightly different
situations (empty string, and whitespace string). this is anticipated to
be just temporary, with this just being the first step in better handling
bootloader selections.

Gbp-Dch: Short
2020-05-04 21:40:13 +01:00
Lyndon Brown 9d7393bbdb config: add note
The core purpose of `Prepare_config()` is to prepare the config by setting
defaults where not specified. It should not be involved itself in
validation of user options.

This can be an important distinction, for instance in `clean` where we
use this function, but avoid validation to potentially allow for an option
that deletes the config, that should work even in the face of an invalid
config.

Gbp-Dch: Ignore
2020-05-04 16:16:00 +01:00
Lyndon Brown 89ed6a54a1 config: fix inadequate validation check
Gbp-Dch: Ignore
2020-05-04 14:39:55 +01:00
Lyndon Brown 943c8fb5fe extend config validation to all build components
while `lb build` ran the config validation check, spotting invalid configs
and stopping with an error, the major build stages if executed directly did
not, nor did the component stages.

here we move execution of the validation function into the common init
function, with an exported variable used to indicate that validation has
been performed. thus validation is performed no matter what part of the
build system you execute, but only once.

Gbp-Dch: Short
2020-05-04 13:26:19 +01:00
Lyndon Brown 9bc35759aa config: fix stale live-build version in iso-preparer
using `$VERSION` as part of the default `$LB_ISO_PREPARER` means that when
you simply run `lb config` once, this variable is stored as a part of the
string, and replaced on use, but if you run `lb config` twice, it gets
replaced with a fixed version, that is then used in all subsequent builds.

let's replace with a placeholder (`@LB_VERSION@`) that can be used both in
the default, or in user strings, and will be replaced on use only.

this means that subsequent builds will always reflect the actual version
of live-build used.

Gbp-Dch: Ignore
2020-05-03 23:20:39 +01:00
Lyndon Brown 31cc121c3d config: obey new UTC timestamp control for default ISO volume label 2020-05-03 22:19:49 +01:00
Lyndon Brown 6cf3321d84 config: provide UTC timestamp control
`DATE_UTC_OPTION` is set in `Prepare_config()` for use by scripts, even
though only a few scripts will actually use it, since it allows those
scripts to be cleaner. we may want to possibly extend this as a
`DATE_OPTIONS` variable perhaps as part of enabling proper reproduciblity.

Gbp-Dch: Short
2020-05-03 22:19:43 +01:00
Lyndon Brown 7276d0213f fix usage/help/man bugs
- the definition of $PROGRAM as used in $USAGE strings defined in each
   script has been broken for a long time, being simply "lb" when it needs
   to be "lb COMMAND".
 - `config` changed $PROGRAM to "lb config" thus its output was correct in
   this regard unlike everything else, but with the switch to a more
   "intelligent" `Man()` function recently, it means that instead of
   `man lb config`, what was actually run was `man lb config config`,
   which displayed the manpage, then on exiting with `q`, it showed some
   sort of index line todo with a "config" search (no exact manpage
   match?), for which you had to enter `ctrl+c` to get rid of.

this revises things to fix the issues, minimising change by changing
$PROGRAM to "lb COMMAND", with the frontend overriding this.

Gbp-Dch: Ignore
2020-05-03 01:38:17 +01:00
Lyndon Brown 372a8fd91a args: restore a little tidying
lost in a reversion

Gbp-Dch: Ignore
2020-05-03 01:31:47 +01:00
Lyndon Brown 23ed83e730 config: tweak config file loading order
load the base set of files first, then any user arch/dist specific override
files second.

this is important since whenever config items get moved in the base files,
this can break the override files, if an option specified in one was moved
to a different common file that it loaded after.

Gbp-Dch: Ignore
2020-05-02 20:44:02 +01:00
Lyndon Brown 22d8ac15db options: de-duplicate common option handling 2020-05-02 18:18:49 +01:00
Lyndon Brown 2d7ff4934d improve stagefiles encapsulation
- added and used Stagefiles_dir()
 - added and used Stagefile_exists()

pretty self explanatory.

Gbp-Dch: Short
2020-05-02 14:49:45 +01:00
Lyndon Brown 5f318c5a44 tidy up Echo_breakage()
- it's only used by the debootstrap script after alternatives were dropped
   long ago, so let's move it, avoiding it being loaded for everything
   else.
 - there's no need to pass printing another message through it.
 - there's little point in making the sid distinction if you happen to
   decide to build sid, it's a given that it's less stable than stable.

really, is there any need for this at all?

Gbp-Dch: Ignore
2020-05-02 13:06:17 +01:00
Lyndon Brown 1c74927fe5 config: simplify mode default
all roads lead to debian

i was waiting to get to the work on properly tackling the mode stuff, but
let's just tidy this now...

Gbp-Dch: Ignore
2020-05-02 12:41:35 +01:00
Lyndon Brown f78001144b move and rename Common_config_files()
for better encapsulation, and for consistency with other config file
handling functions in the new location.

Gbp-Dch: Ignore
2020-05-02 02:38:26 +01:00
johnraff 0e5d0483e5 Replace 'which' with 'command -v' to test for the existance of an executable
This is considered to be more robust.

Two instances remain:

scripts/build/chroot_archives, line 257:
if [ "${LB_APT}" = "aptitude" ] && [ ! $(Chroot chroot "which aptitude") ]
The command is run inside a chroot where the environment might be special,
and would need further testing.

manpages/Makefile, line 42:
@if [ ! -x "$$(which po4a 2>/dev/null)" ]; \
I am insufficiently familiar with makefile syntax to edit this.
2020-05-01 09:59:11 +00:00
Lyndon Brown 2c1af527fd config: fix wrong variable name
Gbp-Dch: Ignore
2020-05-01 08:57:44 +00:00
Lyndon Brown a72e7daaf7 config: only try to load FILE.ARCH and FILE.DIST if non-empty strings
to thus avoid trying to load `FILE.`

Gbp-Dch: Ignore
2020-05-01 08:57:44 +00:00
Lyndon Brown 61aa3239c5 config: remove junk config loading code
support for this has not existed since v2.0~a24-1 !!

until 4d22ca948a (v2.0~a24-1), `PROGRAM` was
defined as `PROGRAM="$(basename ${0})"`. then this commit simply got rid
of the definition.

it then flipped back and forth a bit in a few ways, but always sticking
to a simple fixed "live-build" type string after that commit, as it has
been ever since.

the code removed here has thus been junk since then, and the documentation
corrected here out of date since then.

the checking of `LB_CONFIG` is also possibly junk, but i know too little
about it. it was introduced in v1.0~a43-1 in commit
705a4178e7 with a changelog entry of
"Centraly evaluate LH_CONFIG for custom configuration file."

Gbp-Dch: Short
2020-05-01 08:57:44 +00:00
Lyndon Brown 51903da073 init: warn if auto script exists but is not executable 2020-04-30 05:07:02 +00:00
Lyndon Brown 29ab2459c8 a couple tiny doc fixes
Gbp-Dch: Ignore
2020-04-29 10:27:16 +01:00
Lyndon Brown 997a7f73f9 config: remove redundant validation check
this is already covered by the check above it

Gbp-Dch: Ignore
2020-04-26 12:55:18 +00:00
Raphaël Hertzog e7df05cd29 Replace "Help" calls with "Man" and get rid of Help 2020-04-24 16:39:10 +02:00
Raphaël Hertzog fac2d8d903 Make "Man" function smarter to also work for "lb --help" 2020-04-24 16:38:45 +02:00
Raphaël Hertzog 104fa20399 Revert "frontend: properly handle option processing"
This reverts commit 6b7c8ed4bf as it's
breaking multi-value space-separated parameters.
2020-04-24 15:37:13 +02:00
Lyndon Brown d79c96232b stagefiles: guard unnecessary chroot removal
just as most scripts are skipped if their stagefile exists (indicating
that they have already been run to completion), including chroot
preparation scripts in install mode, this implements the same guard for
chroot prep remove mode, such that they exit early if their stagefile
does not exist, indicating that the modification has already been removed.
(also override-able by --force in the same way).

this basically just uses a tweaked copy of Check_stagefile().

Gbp-Dch: Short
2020-04-23 18:26:15 +01:00
Lyndon Brown 500f205073 config: s/LIVE_IMAGE_TYPE/LB_IMAGE_TYPE/
no backwards compatibility hack for reading the old var from existing
saved config used because this was previously stored in the alternate
format config/build file.

Gbp-Dch: Short
2020-04-23 16:54:31 +00:00
Lyndon Brown 83d9145257 config: s/LIVE_IMAGE_NAME/LB_IMAGE_NAME/
no backwards compatibility hack for reading the old var from existing
saved config used because this was previously stored in the alternate
format config/build file.

Gbp-Dch: Short
2020-04-23 16:54:31 +00:00
Lyndon Brown 41738b1932 config: s/LIVE_CONFIGURATION_VERSION/LB_CONFIGURATION_VERSION/ 2020-04-23 16:54:31 +00:00
Lyndon Brown c441c39efe config: revert partial format conversion
back in v4.0~a6-1 a transition process was started to move the live-build
config to a new format. the new format was INI style, and required
parsing functions to read/write values, compared to the existing format
which was just shell script code setting variables.

this partial transition is the explanation for the existence of the
`New_configuration()` function, and understanding this is important to
understanding the purpose of it - it is not in fact intended for creating
a new configuration, it is just related to the new config format
transition.

the positives of the new format were that it was somewhat cleaner looking,
while the negative was the terrible relative efficiency.

the file `config/build` was created to hold options in this new format.

the transition was only ever completed for a handful of config options:
 - architecture
 - archive areas and parent archive areas
 - live image name
 - live image type

a 'configuration version' attribute was also saved, which is not used by
anything.

the bootstrap-mirror and parent-bootstrap-mirror attributes are pointlessly
stored in it seemingly resulting from work done in v4.0~a17-1. (they are
also stored in another config file from which the value is actually used).

it in fact seems to have been a source of confusion for Raphaël in
authoring 44b9b0a650, since the new
`[parent]-distribution-{chroot|binary}` options it introduced were stored
both in `config/bootstrap` and in `config/build`, while only used from the
former. i expect, understandably, that he thought that `config/build` was
just an information file.

Gbp-Dch: Short
2020-04-23 16:54:31 +00:00
Raphaël Hertzog e71db2bd58 Fix version computing when LIVE_BUILD is set
It used to rely on the $LIVE_BUILD/VERSION file which is now gone.
Instead we now rely on the debian/changelog file just like we do in the
Makefile.
2020-04-23 17:00:19 +02:00
Raphaël Hertzog 4c75f80e67 Use descriptive parameter names for Usage() 2020-04-23 16:38:59 +02:00
Lyndon Brown b3bba232ac usage: take exit code as param
thus it can correctly indicate success/fail status instead of always
indicating failure. when a user asks for usage with -u|--usage then we
should exit in success mode rather than failure as when usage in printed
in response to incorrect usage.

Gbp-Dch: Short
2020-04-23 16:32:26 +02:00
Lyndon Brown 6b7c8ed4bf frontend: properly handle option processing
this means that the usage goes from:
 lb {-h|--help|-u|--usage|-v|--version}
 lb COMMAND [OPTIONS]

to:
 lb {-h|--help|-u|--usage|-v|--version}
 lb [FRONTEND_OPTIONS] COMMAND [COMMAND_OPTIONS]

though it is probably not worth is to update the description in the
manpages...? hmm...

so for instance this matters for color control with --color|--no-color
(you already had full control via environment vars). previously you could
do `lb COMMAND --no-color` to turn off colour, only to find that output
at the frontend level was still coloured (the option is processed at the
command context level, not the frontend), so you might try to instead use
`lb --no-color COMMAND`, only to find that this was not supported. Well
now it is, and used at the frontend level will fully control colour output
(after the command is processed anyway).

the full set of common options are thus available (except --force) at the
frontend level, and thus for instance all Echo_*() helpers used in the
frontend will work correctly after args are processed.

furthermore usage like `lb --color --help` will actually work. (not that
color is used there, but this previously would have failed with the
frontend treating the `--color` argument as the command; that's the point!)

Gbp-Dch: Short
2020-04-23 16:29:00 +02:00
Lyndon Brown 8ffe48d8f3 rename LB_BOOTSTRAP_QEMU_ARCHITECTURES to LB_BOOTSTRAP_QEMU_ARCHITECTURE 2020-04-23 16:00:34 +02:00
Raphaël Hertzog 047885bf7e Keep LB_ARCHITECTURES environment variable for backwards compatibility
Custom hook scripts may use the variable. And conditional package lists
can also rely on this variable.
2020-04-23 15:58:59 +02:00
Lyndon Brown 6cc7250954 rename LB_ARCHITECTURES to LB_ARCHITECTURE
this was previously not done in 8b109ffb96
to keep the renaming simple, but leaving the variable plural is a cause
for confusion.

since this property is stored in the INI style config/build config file
rather than a shell script based one, at the property there is already
singular, there was no need for a backwards compatibility hack.

Gbp-Dch: Short
2020-04-23 15:54:19 +02:00
Lyndon Brown f4c8a28a21 config: move control of apt colour
being done in Prepare_config() meant that it would get saved into the
config stored on disk which is not what was intended.

being saved on disk is not a big problem, since the injection of the
dynamically determined option is always done on top of the value read
from the saved config anyway, so you'd always end up with two copies of
the option given to apt, the first being from the saved config, the second
being the more correct dynamically determined one, overriding the first.

so there is no functional change here, just better achieving what was
intended.

Gbp-Dch: Ignore
2020-04-23 15:52:32 +02:00
Lyndon Brown faa51b83c5 config: fix mistake in mirror url tidying
Gbp-Dch: Ignore
2020-04-23 15:51:53 +02:00
Lyndon Brown ba05247438 chroot: make var local
Gbp-Dch: Short
2020-04-23 15:47:00 +02:00
Lyndon Brown 9a91ca9fde config: obsolete old --tasksel option
note that the bit of code removed from source_debian relies upon a
variable LB_TASKS which itself is an old leftover artefact from before
v4.0.

Gbp-Dch: Short
2020-04-23 15:46:06 +02:00
Raphaël Hertzog d414b8fcdb config: obsolete --net-root-path 2020-04-23 15:44:17 +02:00
Lyndon Brown 1ca53bff52 config: obsolete --net-root-* options (except one)
--net-root-path probably needs to go too, but it is being used for
something i don't fully understand currently.

Gbp-Dch: Short
2020-04-23 15:30:41 +02:00
Lyndon Brown 1eee15e852 config: obsolete unused --net-cow-* options 2020-04-23 15:24:50 +02:00
Lyndon Brown c3f0d39675 config: apt-get should probably be an allowed and documented --apt value
since everywhere where 'apt' is a permitted value, 'apt-get' is also, it
just wasn't listed in the option's documentation and thus was also not
listed in the new validation check.

Gbp-Dch: Short
2020-04-23 15:23:30 +02:00
Lyndon Brown c57b8679a4 config: fix broken backwards compatibility hack
80aa5ab611 implemented a hack to handle
replacement of LB_LINUX_FLAVOURS with LB_LINUX_FLAVOURS_WITH_ARCH in
config files, but implemented it in the wrong place.

adding a conditional conversion within the config file meant that the old
value would only be read from **new** config files that are created
obviously without it, including re-saved configs if `lb config` were
re-run with additional options (not recommended). any existing value in an
existing config file would actually be ignored.

the right place to read the old value was in the Set_defaults() function
(since renamed).

a second issue also existed with the hack, it failed to excape the `$`
and thus printed the existing value of $LB_LINUX_FLAVOURS into the
conditional check being constructed in the config file, instead of
printing the name of the variable. the check embedded into the config
file thus became this on an amd64 machine:
```
if [ -n "amd64" ]
then
	LB_LINUX_FLAVOURS_WITH_ARCH="amd64"
fi
```
which is clearly not what was intended.

Gbp-Dch: Short
2020-04-23 15:23:15 +02:00
Lyndon Brown 9ab7a100da config: add missing deprecation warning
Gbp-Dch: Ignore
2020-04-23 15:14:16 +02:00