Firmware packages in contrib are usually downloaders (e.g. b43-installer
and b43legacy-installer, which require networking).
Regarding non-free, we've started populating non-free-firmware with
interesting firmware packages[1], and those remaining in non-free don't
need to end up on installation images.
1. https://lists.debian.org/debian-boot/2023/01/msg00150.html
At the moment, NON_FREE_COMPONENTS still has both non-free and
non-free-firmware, but the plan is to only keep non-free-firmware for
bookworm.
[ Best viewed with -b due to indentation changes. ]