manuel
/
live-build
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
live-build/functions/firmwarelists.sh

39 lines
1.3 KiB
Bash
Raw Normal View History

firmware: de-dup firmware list parsing Edit: There were four copies of the same logic to keep in sync; Originally this patch deduplicated each file, but leaving a copy of the new function in each, thus reducing the duplication but not eliminating it. A later patch moved it into a shared function file following further enhancements to the code in question. This has since been revised to have the function moved to a shared file here, which simplifies and gives a cleaner diff. Gbp-Dch: Short Closes: #952908
2020-02-11 16:44:31 -01:00
#!/bin/sh
## live-build(7) - System Build Scripts
## Copyright (C) 2006-2015 Daniel Baumann <mail@daniel-baumann.ch>
##
## This program comes with ABSOLUTELY NO WARRANTY; for details see COPYING.
## This is free software, and you are welcome to redistribute it
## under certain conditions; see COPYING for details.
# Updates FIRMWARE_PACKAGES with list of packages determined from specified
# archive areas of specified distro, based upon reading archive content file.
#
# Shared by chroot_firmware and installer_debian-installer
#
# Assumption: firmware packages install files into /lib/firmware
Firmware_List_From_Contents () {
local MIRROR_CHROOT="${1}"
local DISTRO_CHROOT="${2}"
local ARCHIVE_AREAS="${3}"
for _ARCHIVE_AREA in ${ARCHIVE_AREAS}
do
firmware: save the compressed contents file to disk instead of decompressed the existing logic was to decompress the contents file from the downloaded archive to disk, then process it to obtain a package list. the largest one by far is for 'main'; 'non-free' and 'contrib' are tiny in comparison. for sid-amd64 currently, the archive file is 37 MB, while the decompressed file it contains is 592.3 MB. we always delete the files and download afresh (currently), and a previous commit optimised by deleting the files once we're done with them to avoid wasting disk space leaving them behind. here we switch to storing the downloaded compressed file to disk instead, reducing disk space usage (and IO) by hundreds of megabytes; piping the decompression directly into awk instead of having awk read from the stored file. this moves the appending of new items into the list back within the archive area loop, which is fine since we're replacing the file for each loop now so the previous issue relating to appending is of no concern. Gbp-Dch: Short Closes: #952910
2020-02-12 04:00:08 -01:00
local CONTENTS_FILE="cache/contents.chroot/contents.${DISTRO_CHROOT}.${LB_ARCHITECTURES}.gz"
firmware: de-dup firmware list parsing Edit: There were four copies of the same logic to keep in sync; Originally this patch deduplicated each file, but leaving a copy of the new function in each, thus reducing the duplication but not eliminating it. A later patch moved it into a shared function file following further enhancements to the code in question. This has since been revised to have the function moved to a shared file here, which simplifies and gives a cleaner diff. Gbp-Dch: Short Closes: #952908
2020-02-11 16:44:31 -01:00
local CONTENTS_URL="${MIRROR_CHROOT}/dists/${DISTRO_CHROOT}/${_ARCHIVE_AREA}/Contents-${LB_ARCHITECTURES}.gz"
firmware: save the compressed contents file to disk instead of decompressed the existing logic was to decompress the contents file from the downloaded archive to disk, then process it to obtain a package list. the largest one by far is for 'main'; 'non-free' and 'contrib' are tiny in comparison. for sid-amd64 currently, the archive file is 37 MB, while the decompressed file it contains is 592.3 MB. we always delete the files and download afresh (currently), and a previous commit optimised by deleting the files once we're done with them to avoid wasting disk space leaving them behind. here we switch to storing the downloaded compressed file to disk instead, reducing disk space usage (and IO) by hundreds of megabytes; piping the decompression directly into awk instead of having awk read from the stored file. this moves the appending of new items into the list back within the archive area loop, which is fine since we're replacing the file for each loop now so the previous issue relating to appending is of no concern. Gbp-Dch: Short Closes: #952910
2020-02-12 04:00:08 -01:00
# Ensure fresh
rm -f "${CONTENTS_FILE}"
firmware: de-dup firmware list parsing Edit: There were four copies of the same logic to keep in sync; Originally this patch deduplicated each file, but leaving a copy of the new function in each, thus reducing the duplication but not eliminating it. A later patch moved it into a shared function file following further enhancements to the code in question. This has since been revised to have the function moved to a shared file here, which simplifies and gives a cleaner diff. Gbp-Dch: Short Closes: #952908
2020-02-11 16:44:31 -01:00
firmware: save the compressed contents file to disk instead of decompressed the existing logic was to decompress the contents file from the downloaded archive to disk, then process it to obtain a package list. the largest one by far is for 'main'; 'non-free' and 'contrib' are tiny in comparison. for sid-amd64 currently, the archive file is 37 MB, while the decompressed file it contains is 592.3 MB. we always delete the files and download afresh (currently), and a previous commit optimised by deleting the files once we're done with them to avoid wasting disk space leaving them behind. here we switch to storing the downloaded compressed file to disk instead, reducing disk space usage (and IO) by hundreds of megabytes; piping the decompression directly into awk instead of having awk read from the stored file. this moves the appending of new items into the list back within the archive area loop, which is fine since we're replacing the file for each loop now so the previous issue relating to appending is of no concern. Gbp-Dch: Short Closes: #952910
2020-02-12 04:00:08 -01:00
wget ${WGET_OPTIONS} "${CONTENTS_URL}" -O "${CONTENTS_FILE}"
firmware: de-dup firmware list parsing Edit: There were four copies of the same logic to keep in sync; Originally this patch deduplicated each file, but leaving a copy of the new function in each, thus reducing the duplication but not eliminating it. A later patch moved it into a shared function file following further enhancements to the code in question. This has since been revised to have the function moved to a shared file here, which simplifies and gives a cleaner diff. Gbp-Dch: Short Closes: #952908
2020-02-11 16:44:31 -01:00
firmware: save the compressed contents file to disk instead of decompressed the existing logic was to decompress the contents file from the downloaded archive to disk, then process it to obtain a package list. the largest one by far is for 'main'; 'non-free' and 'contrib' are tiny in comparison. for sid-amd64 currently, the archive file is 37 MB, while the decompressed file it contains is 592.3 MB. we always delete the files and download afresh (currently), and a previous commit optimised by deleting the files once we're done with them to avoid wasting disk space leaving them behind. here we switch to storing the downloaded compressed file to disk instead, reducing disk space usage (and IO) by hundreds of megabytes; piping the decompression directly into awk instead of having awk read from the stored file. this moves the appending of new items into the list back within the archive area loop, which is fine since we're replacing the file for each loop now so the previous issue relating to appending is of no concern. Gbp-Dch: Short Closes: #952910
2020-02-12 04:00:08 -01:00
local PACKAGES
PACKAGES="$(gunzip -c "${CONTENTS_FILE}" | awk '/^lib\/firmware/ { print $2 }' | sort -u )"
FIRMWARE_PACKAGES="${FIRMWARE_PACKAGES} ${PACKAGES}"
# Don't waste disk space preserving since always getting fresh
rm -f "${CONTENTS_FILE}"
done
firmware: de-dup firmware list parsing Edit: There were four copies of the same logic to keep in sync; Originally this patch deduplicated each file, but leaving a copy of the new function in each, thus reducing the duplication but not eliminating it. A later patch moved it into a shared function file following further enhancements to the code in question. This has since been revised to have the function moved to a shared file here, which simplifies and gives a cleaner diff. Gbp-Dch: Short Closes: #952908
2020-02-11 16:44:31 -01:00
}