Commit Graph

37 Commits

Author SHA1 Message Date
Steve McIntyre e57aa6fa72 Add non-free-firmware support
List the desired non-free sections to use in NONFREE_COMPONENTS in
CONF.sh, default to "non-free non-free-firmware" by default until
things are all migrated.
2022-10-08 17:48:59 +01:00
Steve McIntyre 8c52820262 Be more verbose when leaving packages out due to size limits 2022-10-07 15:55:12 +01:00
Raphaël Hertzog e77ade6033 Remove dead code in sort_deps and improve log messages
The dependency tree created by parsing the output of "apt-cache depends" always
generates an array of array. The old representation of a simple dependency as
an hash is gone in favor of an OR with a single entry (so the hash is the only
element within the array).

So all the code in add_missing() that was dealing with the case of the hash
was never triggered.

I tweaked the OR case so that the generated log messages actually
mention "OR" only if it's a true OR dependency (i.e. with multiple
entries in the array).
2022-08-09 19:39:15 +02:00
Raphaël Hertzog ca9ac8deac Implement recursive lookup of Recommends and Suggests
Everywhere where we call add_missing() on Depends, we must do it also
on Recommends and Suggests based on $add_rec and $add_sug.

Tweak the code that is supposed to stop the recursion to use
the list that we have built but don't be strict about version
constraints on Recommends/Suggests since they can't always
be respected.

Closes: #601203
2022-08-09 19:37:48 +02:00
Steve McIntyre 3a13ddb48a Revert "Implement recursive lookup of Recommends and Suggests"
This reverts commit 8513b237af.

This change doesn't seem ready yet - running a weekly build failed
with massive memory usage.
2022-08-08 20:23:13 +01:00
Raphaël Hertzog 8513b237af
Implement recursive lookup of Recommends and Suggests
Everywhere where we call add_missing() on Depends, we must do it also
on Recommends and Suggests based on $add_rec and $add_sug.

Closes: #601203
2022-08-05 09:36:17 +02:00
Raphaël Hertzog c9489cd926
Turn $add_rec and $add_sug into globals
There's no point in passing them as parameters everywhere.

This changes the behaviour in some places:

1/ When we process the non-explicitly included packages (for COMPLETE=1),
we used to never take Recommends/Suggests into account, now we do. But
the only thing that will be changed is the ordering with some
Recommended packages being included before their depending package.

2/ When we process the EXTRANONFREE image, this one was always including
Recommends and Suggests, and it's now respecting the setting too.
2022-08-05 09:13:44 +02:00
Raphaël Hertzog 7ea13858fb Don't allow udebs to satisfy dependencies of regular debs
I was just bitten by this problem in Kali where the ISO was not
containing the expected packages because tools/sort_deps was accepting
"xserver-xorg-input-libinput-udeb" as a solution to
"xserver-xorg-input-all | xserver-xorg-input-driver" and the
udeb was among the package already selected for inclusion on the
ISO.

With this change, tools/sort_deps will look for udeb to solve
udeb dependencies and it will look for non-udeb to solve non-udeb
dependencies.

FTR, here's the sort_deps.log extract proving my assertion:

    xserver-xorg Dep: (OR xserver-xorg-input-all xorgxrdp xserver-xorg-input-aiptek xserver-xorg-input-elographics xserver-xorg-input-evdev xserver-xorg-input-evdev-udeb xserver-xorg-input-joystick xserver-xorg-input-kbd xserver-xorg-input-libinput xserver-xorg-input-libinput-udeb xserver-xorg-input-mouse xserver-xorg-input-mtrack xserver-xorg-input-multitouch xserver-xorg-input-mutouch xserver-xorg-input-synaptics xserver-xorg-input-void xserver-xorg-input-wacom xserver-xorg-input-xwiimote ) soft_depend 1
    xserver-xorg-input-all not included in a useful version, check_backports 0
    xorgxrdp not included in a useful version, check_backports 0
    xserver-xorg-input-aiptek not included in a useful version, check_backports 0
    xserver-xorg-input-elographics not included in a useful version, check_backports 0
    xserver-xorg-input-evdev not included in a useful version, check_backports 0
    xserver-xorg-input-evdev-udeb not included in a useful version, check_backports 0
    xserver-xorg-input-joystick not included in a useful version, check_backports 0
    xserver-xorg-input-kbd not included in a useful version, check_backports 0
    xserver-xorg-input-libinput not included in a useful version, check_backports 0
      xserver-xorg-input-libinput-udeb is included already, acceptable version 0.29.0-1
2020-02-13 15:21:10 +01:00
Raphaël Hertzog 031fd46b40 Fix some inconsistent indenting 2020-02-13 15:07:00 +01:00
Steve McIntyre 5dd0d5cc2a Fix silly typo 2018-08-13 14:07:51 +01:00
Steve McIntyre 66154bff85 Add initial support for including backports in a build
So far only tested with a single image - targeting a netinst with some
updated packages
2016-09-20 01:46:15 +01:00
Steve McIntyre bde884deb4 Improve the sort_deps fix for too-large packages
Make sure that the package we're told about exists before we look for
its size - a number of the dependencies and recommends may not exist
and this will cause errors for us.

Also list the configured max_pkg_size at the top of the script, to
help with debugging later.
2015-07-23 18:39:43 +01:00
Steve McIntyre 87aff198e3 tools/sort_deps: Improve handling of too-large packages
Previously, the code would only check sizes for packages explicitly
listed but packages brought in due to dependency resolution would not
be checked. Now fixed.
2015-07-21 18:46:06 +01:00
Steve McIntyre 252013342e * Add support for Built-Using:
+ tools/sort_deps: parse out Built-Using from Packages files too and
      pass any such data on to tools/merge_package_lists
    + tools/merge_package_lists: if we have any built-using source
      packages listed when doing a combined binary/source set, add them in
      the same way as existing listed sources.
  * Other bug fixes in tools/merge_package_lists:
    + Fix handling of multi-line Binary: fields
    + Print out the accumulated size of source packages in output also
      (fix to printf format string)
2013-01-24 10:26:30 +00:00
Steve McIntyre ca4ae110df Bugfix for new sort_deps code:
Old code used to add the various options in the special case for
"pacakges providing not truly virtual pacakges", which amounted to an
OR dep on the whole set. I missed that in the re-implementation with
version tracking.

Now explicitly switched to adding an OR dep for *all* cases here -
other code will deal with that already. Tweak the logging code to make
things clearer too...
2012-09-27 18:26:19 +00:00
Steve McIntyre b435f7b341 Make the dump of dependencies more verbose at parse time 2012-09-27 16:52:00 +00:00
Steve McIntyre 6230392432 Fix scoping bug when parsing dependencies
The %elt hash needs to be a new one each time we push to the
dependency lists, otherwise we'll just be updating the existing one
each time we push().
2012-09-27 16:49:26 +00:00
Steve McIntyre c46f7c0cf6 * Add version-tracking into dependency sorting. Closes: #687949)
+ Use/parse output from a newer version of apt so that "apt-cache
      depends" will include version information on dependencies.
    + All tracking of packages now include versions, so we pass around
      hashes of {package name, comparison op, version} everywhere instead
      of simply passing package names as strings.
    + Add the APT::Cache::ShowVersion=1 option to apt-cache calls to
      turn on version reporting. Needs a locally-patched version of apt
      *for now*, but expecting that to be fixed for the Wheezy
      release. The new code degrades gracefully if version info is not
      available.
    + Add a dependency on libdpkg-perl for version comparison code.
2012-09-26 18:18:47 +00:00
Steve McIntyre c245a5c696 Tweak sort_deps:
+ Track whether a dependency chain is caused by a hard dependency
  (Depends) or soft (Recommends/Suggests). If we fail to add a package
  down the chain on a *soft* dependency, don't invalidate the entire
  chain.
+ When checking OR dependencies, *first* check to see if any of the
  options are already installed *before* trying to install the first
  option. Helps a lot with circular deps such as task-lxde-desktop ->
  task-desktop -> (task-gnome-desktop | task-lxde-desktop | etc.); in
  this case, of course task-lxde-desktop should satisfy the OR dep.
2012-08-02 23:57:40 +00:00
Steve McIntyre ac9aca28db YA typo fix 2012-07-08 18:40:21 +00:00
Steve McIntyre 281dd7f076 add debug for unstable tasks pkgs 2012-07-08 18:37:42 +00:00
Steve McIntyre 0feb53ab54 fix typo 2012-07-08 18:20:44 +00:00
Steve McIntyre 024f95622f Add support for using unstable tasks packages when debugging 2012-07-08 18:16:05 +00:00
Steve McIntyre 69284b5972 fix missing brace :-( 2012-07-08 17:44:34 +00:00
Steve McIntyre 567ad9ed25 tweak first parsing loop 2012-07-08 17:14:53 +00:00
Joey Hess a241ec8aa5 fix file encoding 2012-07-08 16:06:54 +00:00
Steve McIntyre 6686e2c66c Clean up handling of environment variable handling in perl tools. 2012-06-05 13:56:51 +00:00
Steve McIntyre f3928178e9 * Change the default for NORECOMMENDS to 1 for consistency with
NOSUGGESTS. Closes: #598196
* Documentation patches
2010-09-29 13:13:04 +00:00
Steve McIntyre b9c25ca204 * Add support for forcing firmware packages onto CD#1:
+ Add "FORCE_FIRMWARE=1" to CONF.sh to turn this on
    + Will pick up on packages in the "firmware" task and install them
    + Will also add sym-links to them in CD1/firmware so that new d-i code
      can find them easily.
    + Will add a short sentence into README.html/README.txt
  * Add top-level control of which checksums are desired for output images
    see the CHECKSUMS setting in CONF.sh
  * Fix some bashisms in tools/apt-selection.
2010-05-27 00:52:25 +00:00
Steve McIntyre f5f5b8b4a7 Rename log.sort_deps* to sort_deps*.log for better consistency.
Closes: #571591
2010-04-27 12:25:41 +00:00
Steve McIntyre acf3d69328 Extend the code that copes with too-large binary packages to cope with
too-large source packages too.
2009-12-29 01:27:25 +00:00
Steve McIntyre 9590c06ddd Add support for coping with packages that are too large to fit. If we
have to remove packages and their dependencies, list them in
README.excluded in the root of disc 1.
2009-11-21 00:14:26 +00:00
Frans Pop c967349160 Recognise and ignore Enhances: lines
Patch from Ian Campbell.

It seems that "apt-cache depends" recently (as of 0.7.22) started
including Enhances lines in its output. Leading to:

 Generating dependency tree with apt-cache depends...
        UNEXPECTED: Line `  Enhances: kvm
        ' while parsing end of deptree from 'kvm-source'
        [etc....]

Ignore these lines.
2009-08-09 14:50:38 +00:00
Steve McIntyre 699e6b7e62 fix perl warnings in remove_entry() 2009-05-16 13:54:24 +00:00
Steve McIntyre afd7a7a557 Recognise and ignore Breaks: lines. Closes: #523444 2009-05-16 13:48:49 +00:00
Steve McIntyre 564d89c966 Sort dependencies better: if A dep B then we should NOT list A before
B. Closes: #518145.
2009-04-02 12:19:03 +00:00
Steve McIntyre 23f23e05c9 Actually move the file 2008-11-02 09:23:09 +00:00