Commit Graph

125 Commits

Author SHA1 Message Date
Steve McIntyre 44d0ff683f make_disc_trees.pl: if files will not fit on media, fail loudly
rather than sitting in a loop foreber. Fail properly in #1069682
2024-04-23 02:33:17 +01:00
Steve McIntyre 7531d32437 make_disc_trees.pl: Fix $disknum handling for incomplete sets 2024-02-28 00:49:31 +00:00
Steve McIntyre c7cc669655 Try again to fix the TOTALNUM issue
We can't do it in make_image, as that may end up running in
parallel. Instead, do the update of TOTALNUM at the end of
make_disc_trees.pl.
2024-02-23 00:57:00 +00:00
Arnaud Rebillout 3752fd161a Dont generate fw pattern file if DEP11 is set to 0 2024-02-01 21:21:35 +00:00
xiao sheng wen(肖盛文) 2603f4c537 tools/make_disc_trees.pl: filter info output of debootstrap 2024-01-08 10:52:25 +00:00
Steve McIntyre 2964b68565 Tweak image sizes for BD media
We used to have some arbitrary hard-coded sizes; tweak those to allow
for ~4% losses for spare sectors if people format media.
2023-07-12 17:56:35 +01:00
Steve McIntyre 00ea28267d Extend exclude-file handling
Now take a colon-separated list to allow for more than one to be
used. And refactor to reduce repetition.
2023-07-04 17:14:04 +01:00
James Addison d2d51c9179 firmware: use hard links rather than symlinks
When creating on-disc firmware links, use the same link creation logic as archive-area links

Closes #1031696
2023-03-12 18:39:40 +00:00
Cyril Brulebois 7eea34f26f make-firmware-image: fix missing Contents-firmware index (Closes: #1031598) 2023-02-19 03:56:45 +01:00
Steve McIntyre aebf4f0556 make_disc_trees.pl: fix generation of Contents-firmware
The regexp didn't match symlinks in .deb files, let's fix that
2023-02-18 23:33:02 +00:00
Steve McIntyre c5ff679857 Fix noisy error in make_disc_trees.pl
firmware/dep11 only exists on the first image, so don't run
md5_files_for_md5sum on it if doesn't exist
2023-02-18 22:12:42 +00:00
Steve McIntyre 80f7fa39ff Don't break on CD 2 onwards
The $base_components file only matters on disc 1, so check it exists
before trying to do stuff with it.
2023-02-09 01:40:33 +00:00
Cyril Brulebois 492b085305 make_disc_trees.pl: move local vs. base_components safeguard
We have two safeguards related to the local component now, one if it's
present, one if it's absent: group them via if-then-else.
2023-01-27 07:38:40 +01:00
Cyril Brulebois ae8490ba70 make_disc_trees.pl: rewrite base_components if needed
It's possible to set LOCAL=1 to enable support for local packages, which
automatically results in “local” being listed in /.disk/base_components,
which is used to build the --components parameter for debootstrap.

Users might have various sets of *.deb and/or *.udebs, be it over time,
or for various codenames, and it's possible for /dists/<CODENAME>/local
to be entirely missing in the generated image.

In this case, /.disk/base_components must not contain “local”, as that
would break debootstrap; this commit detects and fixes this possible
inconsistency.
2023-01-27 05:24:03 +01:00
Cyril Brulebois bfebeded70 make_disc_trees.pl: merge “local” udebs into “main” explicitly
This might have worked by accident depending on the exact layout under the
LOCALDEBS directory (see previous commit for the big picture).

Let's focus on where input files for d-i should be stored:

    <LOCALDEBS>/dists/<CODENAME>/local/debian-installer/binary-amd64/*.udeb

Those *.udeb end up being stored under the “local” component[1] (making it
clear they're not from usual components on a Debian mirror), but they are
referenced in Packages for the “main” component[2], possibly replacing a
different version found in the archive.

    /dists/<CODENAME>/local/debian-installer/binary-<ARCH>/*.udeb  [1]
    /dists/<CODENAME>/main/debian-installer/binary-<ARCH>/Packages [2]

This makes it possible for d-i to find those *.udeb automatically, as it
uses a single Packages file.
2023-01-27 05:24:03 +01:00
Cyril Brulebois 8bba3df865 make_disc_trees.pl: fix block calculation for --rollback
Per-package metadata files ($p.patterns and $p.component) go together,
and both needed to be accounted for.
2023-01-27 00:37:06 +01:00
Cyril Brulebois 5e39a81adb make_disc_trees.pl: fix block calculation for multi-arch
Per-package metadata files ($p.patterns and $p.component) go together,
and both needed to be accounted for.
2023-01-27 00:37:06 +01:00
Cyril Brulebois 4a31c04b81 make_disc_trees.pl: remove workaround for missing bookworm/non-free-firmware/dep11
This directory appeared in the archive earlier today.
2023-01-26 22:24:51 +01:00
Cyril Brulebois ee40a3bf23 make_disc_trees.pl: add workaround for missing bookworm/non-free-firmware/dep11
There's no dep11 directory for bookworm's non-free-firmware component
for the time being, and we've just decided to be more cautious about
input files, which breaks the bookworm build.

Until the issue is fixed on the appstream.debian.org side, and until dak
can indeed import that directory into the Debian archive, use sid's
non-free-firmware dep11 directory instead of bookworm's.
2023-01-25 01:38:34 +01:00
Cyril Brulebois 214e2dd88c make_disc_trees.pl: use write_file() for .component files as well 2023-01-25 01:38:34 +01:00
Cyril Brulebois 957ee269b4 make_disc_trees.pl: create /firmware/dep11/README.txt and count it
Move the README.txt creation from generate_firmware_patterns to its
caller, so that it's done just once, and so that added blocks are
counted properly.
2023-01-25 01:38:34 +01:00
Cyril Brulebois 14064e08a1 Replace all gzip -9 calls with pigz -9nm ones.
Some gzip calls have --no-name, which disables both the original name
and timestamp; use -n (disables original name) and -m (disables original
timestamp) for pigz, for all calls.
2023-01-23 19:30:35 +01:00
Cyril Brulebois f7925da6cd make_disc_trees.pl: store .component alongside .patterns for firmware packages
This makes it possible for hw-detect's install-firmware hook to enable the
right components if it installs firmware packages based on modalias
information. The components found in Contents-firmware indices are only
useful for firmware packages detected by the check-missing-firmware.sh
script.
2023-01-22 14:50:09 +01:00
Cyril Brulebois 4308e4666c make_disc_trees.pl: add components to Contents-firmware indices
All three columns could be aligned, but I'm not sure how many characters
should be allocated to the package filename. Alternatively, we could
scratch alignment altogether, and let interested users use column -t on
the file when debugging.
2023-01-22 14:49:06 +01:00
Cyril Brulebois bbda95c283 make_disc_trees.pl: fix misnamed $dist variable
$dist would suggest distribution, an alternative name for suite or
codename (!), let's make it clear we mean component here.
2023-01-21 16:40:56 +01:00
Cyril Brulebois a2378ed25d make_disc_trees: create /firmware before adding symlinks there
Otherwise it would only be created after failing to create the first symlink.
See previous commit.
2023-01-20 16:48:55 +01:00
Cyril Brulebois b1cb4c43c2 make_disc_trees: error out if firmware symlink creation fails
Otherwise failures would go unnoticed, logs would pretend firmware-b43-installer
(the first firmware package getting processed) is indeed installed there while
it's not. See next commit.
2023-01-20 16:48:55 +01:00
Cyril Brulebois 5aa89ab269 Add leading / to paths in Contents-firmware indices.
With this change, hw-detect's check-missing-firmware.sh can use grep -f
with a file listing absolute paths to required firmware files.
2023-01-18 22:43:25 +01:00
Steve McIntyre 32291d88a3 Add a Contents-firmware file
to help with looking up firmware packages in d-i
2022-10-09 01:44:37 +01:00
Steve McIntyre e57aa6fa72 Add non-free-firmware support
List the desired non-free sections to use in NONFREE_COMPONENTS in
CONF.sh, default to "non-free non-free-firmware" by default until
things are all migrated.
2022-10-08 17:48:59 +01:00
bauen1 65196d4dcf Avoid embedding timestamps into gzipped Packages and Translations files.
Signed-off-by: Holger Levsen <holger@layer-acht.org>
2022-02-14 11:34:13 +01:00
Steve McIntyre 6b61e01f15 Try to remove dirs when rolling back 2021-12-06 23:50:22 +00:00
Steve McIntyre 4d4399012d Add support for creating firmware patterns at build time
Using the new generate_firmware_patterns script, generate a patterns
file in /firmware/dep11/<package>.patterns as we add a firmware
package onto media.

Deal with size changes as these are added/removed, and add them to the
md5sum.txt file on media too.
2021-07-25 18:26:15 +01:00
Steve McIntyre 047ec2b360 Fix up firmware package rollback
If we removed a firmware package, also remove the link in /firmware
2021-07-25 18:26:08 +01:00
Holger Levsen 9c8fc62807 make_disc_trees: create Packages files for local/debian-installer. Closes: #523017
tools/make_disc_trees.pl: Apply patch by Daniel Dickinson to create Packages
files for local/debian-installer. Closes: #523017

Signed-off-by: Holger Levsen <holger@layer-acht.org>
2021-02-03 17:36:13 +01:00
Steve McIntyre 966016c495 Be more explicit if DISKTYPE is not specified, or unrecognised 2021-01-16 22:32:48 +00:00
Steve McIntyre c7dcd37aec When compressing translations, remove the uncompressed version
It may cause build problems later if we keep it around
2020-12-15 11:44:56 +00:00
Steve McIntyre 0dd4219af4 Add initial support for jigdo v2 format
Add the option to switch to format v2 which uses sha256 everywhere
instead of md5. We can't actually turn this on yet, need to wait for
users to upgrade a bit and get new v2-aware jigdo clients.

But this works as-is for generating v1 until we hit that switch - see
the new JIGOO_CHECKSUM option in CONF.sh
2019-11-21 19:10:19 +00:00
Steve McIntyre 9acb8c5796 More tweaks for Translation files 2019-08-13 07:54:51 +01:00
Steve McIntyre 330c63e67d Tweak recompress code 2019-07-06 11:19:02 +01:00
Steve McIntyre 6ae364773a Tweak logging again 2019-07-06 10:53:40 +01:00
Steve McIntyre a06b008b2f Add more logging around the recompress() step 2019-07-06 10:42:27 +01:00
Steve McIntyre 890ab6645a Add support for USB sticks any integer multiple of 1GB in size.
Factor out the common code, support anything called STICKnnnGB where
nnn is any integer
2019-06-18 23:59:01 +01:00
Steve McIntyre 114e8a3e55 Add explicit support got 16G USB images 2019-06-17 01:40:25 +01:00
Steve McIntyre a7328f65da Merge branch 'system' into 'master'
make_disc_trees: don't use system() for trivial things

See merge request images-team/debian-cd!2
2019-01-23 23:04:57 +00:00
Julien Cristau c68062b624 make_disc_trees: don't use system() for trivial things
Fair warning: completely untested and I don't know perl.
2019-01-23 23:50:36 +01:00
Julien Cristau 97dbce1c22 make_disc_trees: don't reopen the log over and over 2019-01-23 23:34:01 +01:00
David Kalnischkies 270eabfa43 Provide hashes for uncompressed Translation-* again
Implemented in 2f34797490 for bug #767253
released a few months later in v3.1.17, but then broken again a few days
later in 9dd0616b78 by removing the
uncompressed files too early.
2018-02-16 16:14:29 +00:00
Steve McIntyre a3aef842ad Bugfix: fix the remove_Packages_entry() helper function
This was a bug introduced with the backports support. Lots of the
internal helper functions take 4 arguments now: ($dir, $arch,
$in_backports, $package_info). That's fine, but I forgot to update the
code in remove_Packages_entry() to deal with the $in_backports
argument.

This led to the md5sums.txt files in multi-disc sets being broken -
the last file(s) removed on a given disc would still be listed in the
md5sums.txt file, even though the file itself had been removed.
2017-03-27 20:40:19 +01:00
John Paul Adrian Glaubitz ae59c4e92b Declare 'my' instances of '$_' as 'local'
Fixes syntax errors on Perl 5.24 where '$_' may
no longer be declared as 'my'. Being a global symbol,
'$_' has to be declared as 'local' inside functions.
2016-11-22 11:10:35 +01:00