Peppermint
/
debian-cd-clone
mirror of https://salsa.debian.org/images-team/debian-cd.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

contrib: updated from setup.git/buster/. 

Signed-off-by: Holger Levsen <holger@layer-acht.org>
This commit is contained in:
Holger Levsen 2019-02-19 19:05:47 +01:00
parent c149517d7b
commit ee7441d449
21 changed files with 756 additions and 429 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
contrib/CONF.sh
View File

@ -47,8 +47,8 @@ unset BASE_EXCLUDE || true
# Where I am (hoping I'm in the debian-cd dir)
export BASEDIR=`pwd`
# Building stretch cd set ...
export CODENAME=stretch
# Building buster cd set ...
export CODENAME=buster
if [ ! "$DI_CODENAME" ]
then
@ -70,7 +70,7 @@ export OFFICIAL="Official Snapshot"
# ... for arch
if [ ! "$ARCH" ]
then
export ARCH=`dpkg --print-installation-architecture`
export ARCH=`dpkg --print-architecture`
fi
# IMPORTANT : The 4 following paths must be on the same partition/device.
@ -150,7 +150,11 @@ export CONTRIB=1
#export MKISOFS_OPTS="-joliet-long -jigdo-template-compress bzip2 -r -checksum_algorithm_iso md5,sha1,sha256,sha512" #-checksum_algorithm_iso md5,sha1"
# export MKISOFS_OPTS="-r" #For normal users
# export MKISOFS_OPTS="-r -F ." #For symlink farmers
if [ $(which xorriso)""x = "/usr/bin/xorriso"x ]; then
export MKISOFS="xorriso"
else
export MKISOFS="/home/93sam/xorriso"
fi
export MKISOFS_OPTS="-as mkisofs -r -checksum_algorithm_iso md5,sha1,sha256,sha512"
# Override for i386,amd64,multi to use xorriso.

93
contrib/HEADER.html.in
View File

@ -87,12 +87,12 @@ cpp.
writeable dual-layer Blu-Ray media
# endif
at a minimum, but may be written to larger media if needed.
at a minimum, but may be written to larger media if needed.</p>
# ifdef ISOHYBRID
For <strong>extra</strong> convenience, these images may also be
written directly to a USB stick. So long as your computer will
<p>For extra convenience, <strong>these images may also be written
directly to a USB stick</strong>. So long as your computer will
boot directly from that USB stick, it should start the Debian
installer that way.
@ -116,6 +116,8 @@ cpp.
complete set. Most typical users only need a small subset of
those software packages.</p>
# if !defined(TYPEcd)
<p>Initially, you will only need to download and use
the <strong>first</strong> image of a set (labelled as
<code>debian-<i>something</i>-1</code> to be able to start the
@ -124,16 +126,20 @@ cpp.
(labelled <code>debian-<i>something</i>-2</code>,
<code>debian-<i>something</i>-3</code>, etc.), they contain the
extra packages that can be installed on a Debian system (as
mentioned previously). They will <strong>not</strong> be bootable
and are entirely optional. If you have a fast Internet
connection, you're most likely better off installing any desired
extra packages directly from the Debian mirrors on the Internet
instead of by using these extra images.</p>
mentioned previously). They will <strong>not</strong> be
bootable and are entirely optional. If you have a fast Internet
connection, you're most likely better off installing any
desired extra packages directly from the Debian mirrors on the
Internet instead of by using these extra images.</p>
# endif
# endif
# if defined(TYPEcd) && !defined(ARCHsource) && !defined(STYLElist)
<h2>What is a netinst image?</h2>
<p>The <code>netinst</code> CD here is a small CD image that
contains just the core Debian installer code and a small core set
of text-mode programs (known as "standard" in Debian). To install
@ -142,6 +148,8 @@ cpp.
# if defined(ARCHi386)
<h2>What is an edu netinst image?</h2>
<p>The <code>edu netinst</code> CD here is a special version of
the netinst CD image that is targeted specifically at 32-bit
Intel machines. It provides a menu to install the Debian Edu Pure
@ -152,6 +160,8 @@ cpp.
# if defined(ARCHamd64)
<h2>What is an edu netinst image?</h2>
<p>The <code>edu netinst</code> CD here is a special version of
the netinst CD image that is targeted specifically at 64-bit
Intel machines. It provides a menu to install the Debian Edu Pure
@ -162,21 +172,60 @@ cpp.
# if defined(ARCHi386)
<h2>What is a mac netinst image?</h2>
<p>The <code>mac netinst</code> CD here is a special version of
the netinst CD image that is targeted specifically at 32-bit
Intel Macintosh machines. It will likely work on most other
i386 machines too, but it does not contain UEFI boot files that
some people need.</p>
the netinst CD image that is targeted specifically at older
32-bit Intel Macintosh machines. It will likely work on most
other i386 machines too, but it does not contain UEFI boot
files that some people
need. See <a href="https://wiki.debian.org/MacMiniIntel#Macmini_1.2C1">
the Debian Wiki</a> for more information.</p>
# endif
# if defined(ARCHamd64)
<h2>What is a mac netinst image?</h2>
<p>The <code>mac netinst</code> CD here is a special version of
the netinst CD image that is targeted specifically at older 64-bit
Intel Macintosh machines. It will likely work on most other
amd64 machines too, but it does not contain UEFI boot files that
some people need.</p>
the netinst CD image that is targeted specifically at older
64-bit Intel Macintosh machines. It will likely work on most
other amd64 machines too, but it does not contain UEFI boot
files that some people
need. See <a href="https://wiki.debian.org/MacMiniIntel#Macmini_2.2C1">
the Debian Wiki</a> for more information.</p>
# endif
# endif
# if defined(TYPEbd) && !defined(ARCHsource) && !defined(STYLElist)
<p>The <code>BD</code> image here is a special large image that
contains the core Debian installer code and a specific set of
programs. Depending on the purpose you'll also need an Internet
connection.</p>
# if defined(ARCHi386)
<p>The <code>edu-usb</code> BD here is a special version of
the BD image that is targeted specifically at 32-bit Intel machines.
It provides a menu to install the Debian Edu Pure Blend without the
need of an Internet connection. See the
<a href="https://wiki.debian.org/DebianEdu/">Debian Wiki</a> for
more information.</p>
# endif
# if defined(ARCHamd64)
<p>The <code>edu-usb</code> BD here is a special version of
the BD image that is targeted specifically at 64-bit Intel machines.
It provides a menu to install the Debian Edu Pure Blend without the
need of an Internet connection. See the
<a href="https://wiki.debian.org/DebianEdu/">Debian Wiki</a> for
more information.</p>
# endif
@ -210,6 +259,18 @@ cpp.
#endif
#if !defined(ONEonly) && !defined(FIRMWARE) && (defined(STYLEbt) || defined(STYLEiso))
<h2>Only the first few images are available! Where are the rest?</h2>
<p>We don't store/serve the full set of ISO images for all
architectures, to reduce the amount of space taken up on the
mirrors. You can
<a href="https://www.debian.org/CD/faq/#why-jigdo">use the jigdo
tool</a> to recreate the missing ISO images instead.</p>
#endif
#ifdef FIRMWARE
<h2>Non-free Firmware</h2>

52
contrib/common.sh
View File

@ -120,14 +120,49 @@ catch_parallel_builds () {
echo "$arch build started at $arch_start, ended at $arch_end (took $arch_time), error(s) $arch_error"
}
# Slightly complicated setup
# iso-* dirs have checksums for ISO files only
# bt-* dirs have checksums for ISO and torrrent files only
# jigdo-* dirs have checksums for ISO and jigdo files only
#
# Uses the imagesums tool from debian-cd to grab pre-calculated ISO
# checksums from the jigdo files where possible, to save a lot of time
generate_checksums_for_arch () {
ARCH=$1
JIGDO_DIR=$2
ISO_DIR=$(echo $JIGDO_DIR | sed 's,jigdo-,iso-,g')
BT_DIR=$(echo $JIGDO_DIR | sed 's,jigdo-,bt-,g')
echo "$ARCH: Generating checksum files for the builds in $JIGDO_DIR"
# Do the torrents first, if they exist
if [ -e $BT_DIR ]; then
$TOPDIR/debian-cd/tools/imagesums $BT_DIR $EXTENSION > /dev/null
fi
# Now do the jigdos, if they exist
if [ -e $JIGDO_DIR ]; then
$TOPDIR/debian-cd/tools/imagesums $JIGDO_DIR $EXTENSION > /dev/null
cp $JIGDO_DIR/*SUMS*${EXTENSION} $ISO_DIR
# And grep out the .iso checksums from there to the iso directory
for file in $JIGDO_DIR/*SUMS*${EXTENSION}; do
out=$ISO_DIR/$(basename $file)
grep \\.iso $file > $out
done
if [ -e $BT_DIR ]; then
# Ditto for the bt directory
for file in $JIGDO_DIR/*SUMS*${EXTENSION}; do
out=$BT_DIR/$(basename $file)
grep \\.iso $file >> $out
done
fi
else
# No jigdos, so do the ISOs by hand
$TOPDIR/debian-cd/tools/imagesums $ISO_DIR $EXTENSION > /dev/null
if [ -e $BT_DIR ]; then
for file in $ISO_DIR/*SUMS*${EXTENSION}; do
out=$BT_DIR/$(basename $file)
grep \\.iso $file >> $out
done
fi
fi
}
catch_live_builds () {
@ -169,3 +204,16 @@ get_archive_serial () {
echo 'unknown'
fi
}
rsync_to_pettersson () {
LOCAL=$1
REMOTE=$2
OPTIONS="$3"
rsync -az --delete $OPTIONS $LOCAL sync-to-pettersson:$REMOTE
}
publish_on_pettersson () {
TARGETS="$@"
echo "$TARGETS" | ssh publish-on-pettersson ./bin/receive_from_casulana
}

157
contrib/cronjob.daily
View File

@ -10,8 +10,11 @@ export TOPDIR
BUILDLOCK=$HOME/.debian-cd.lock
export PUBDIRJIG=$PUBDIR/daily-builds
if [ "$BUILDNUM"x = ""x ]; then
BUILDNUM=1
fi
export DATE_BUILD="$DATE-$BUILDNUM"
export TESTING_SUITE=stretch
export TESTING_SUITE=${CODENAME}
BUILDS_RUNNING=""
export NI_WANTED=1
@ -23,20 +26,11 @@ if [ "$RELEASE_BUILD"x = ""x ] ; then
export OUT_FW_TST=${PUBDIRJIG}-firmware/${TESTING_SUITE}_d-i/${DATE_BUILD}
export OUT_FREE_SID=${PUBDIRJIG}/sid_d-i/${DATE_BUILD}
export OUT_FW_SID=${PUBDIRJIG}-firmware/sid_d-i/${DATE_BUILD}
export RSYNC_TARGET_FREE=/mnt/nfs-cdimage/daily-builds
export RSYNC_TARGET_FIRMWARE=/mnt/nfs-cdimage/unofficial/non-free/cd-including-firmware/daily-builds
export RSYNC_TARGET_FREE=${OUT_BASE}/daily-builds
export RSYNC_TARGET_FIRMWARE=${OUT_BASE}/unofficial/non-free/cd-including-firmware/daily-builds
else
export SID_WANTED=0
export TESTING_WANTED=1
export OUT_FREE_TST=${PUBDIRJIG}-$RELEASE_BUILD
export OUT_FW_TST=${PUBDIRJIG}-$RELEASE_BUILD-firmware
export OUT_FREE_SID=""
export OUT_FW_SID=""
export RSYNC_TARGET_FREE=/mnt/nfs-cdimage/.${RELEASE_BUILD}
export RSYNC_TARGET_FIRMWARE=/mnt/nfs-cdimage/unofficial/non-free/cd-including-firmware/.${RELEASE_BUILD}
export CONF=~/build.stretch/CONF.sh.${RELEASE_BUILD}
export RELEASE_BUILD=$RELEASE_BUILD
export EXTENSION=".small"
echo "Don't support release builds of dailies! Use cronjob.weekly instead"
exit 1
fi
. $TOPDIR/common.sh
@ -47,7 +41,7 @@ reboot_lock
finalise_arch_dir () {
ARCH=$1
for OUT_DIR in $OUT_FREE_TST $OUT_FW_TST $OUT_FREE_SID $OUT_FW_SID; do
for OUT_DIR in $OUT_FREE_SID $OUT_FW_SID; do # $OUT_FREE_TST $OUT_FW_TST
ARCH_DIR=$OUT_DIR/$ARCH
# Only keep things / do any work here if there are any files in the ARCH_DIR
NUM_FILES=`find $ARCH_DIR -type f | wc -l`
@ -57,31 +51,25 @@ finalise_arch_dir () {
ARCH_ISO_DIR=${ARCH_DIR}/iso-cd
ARCH_JIGDO_DIR=${ARCH_DIR}/jigdo-cd
if [ "$RELEASE_BUILD"x = ""x ] ; then
DATESTRING=`date -u`
case $ARCH_ISO_DIR in
*firmware*)
FIRMWARE="firmware"
INFILE=$TOPDIR/daily-firmware.html;;
*)
FIRMWARE=""
INFILE=$TOPDIR/daily.html;;
esac
sed "s/ARCH/$ARCH/g;s/DATE/$DATESTRING/g;s/BUILDNUM/$BUILDNUM/g;s/INST_VER/$INST_VER/g" \
sed "s/ARCH/$ARCH/g;s/DATE/$DATESTRING/g;s/BUILDNUM/$BUILDNUM/g;s/CODENAME/$CODENAME/g" \
$INFILE > $ARCH_DIR/HEADER.html
fi
generate_checksums_for_arch $ARCH $ARCH_JIGDO_DIR
~/build.${CODENAME}/generate_headers ~/build.${CODENAME}/HEADER.html.in $ARCH_DIR $ARCH $FIRMWARE
cd $ARCH_DIR
# Only make torrent files for release builds
if [ "$RELEASE_BUILD"x != ""x ] ; then
~/build.stretch/mktorrent iso-cd/*iso
cp iso-cd/*SUMS* bt-cd
fi
if [ "$RELEASE_BUILD"x = ""x ] ; then
echo "Signing checksums files using the automatic key"
~/build.stretch/sign-images $OUT_DIR $ARCH
fi
~/build.stretch/mklist iso-cd/*iso
~/build.${CODENAME}/sign-images $OUT_DIR $ARCH
~/build.${CODENAME}/mklist iso-cd/*iso
cd $TOPDIR
fi
done
@ -113,7 +101,7 @@ if lockfile -r0 $BUILDLOCK ; then
arch_error=""
arch_start=`now`
for OUT_DIR in $OUT_FREE_TST $OUT_FW_TST $OUT_FREE_SID $OUT_FW_SID; do
for OUT_DIR in $OUT_FREE_SID $OUT_FW_SID; do # $OUT_FREE_TST $OUT_FW_TST
mkdir -p ${OUT_DIR}/${arch}/iso-cd ${OUT_DIR}/${arch}/jigdo-cd
done
@ -258,6 +246,82 @@ if lockfile -r0 $BUILDLOCK ; then
fi
fi
# Special debian-edu builds, only minor tweaks. #846006
if [ "$NOEDU"x = ""x ] ; then
if [ "$arch" = "i386" ] || [ "$arch" = "amd64" ] ; then
if [ "$SID_WANTED" = "1" ] ; then
if [ "$NI_WANTED" = "1" ] ; then
build_started SIDEDUNI
OMIT_RELEASE_NOTES=1 OMIT_MANUAL=1 NORECOMMENDS=1 \
NOSUGGESTS=1 COMPLETE=0 INSTALLER_CD=G \
CDNAME=debian-edu \
DI=sid DI_WWW_HOME=default \
DEBIAN_EDU=1 \
KERNEL_PARAMS="modules=debian-edu-install-udeb" \
DESKTOP=xfce VARIANTS=xen \
EXCLUDE=debian-edu-exclude \
UDEB_EXCLUDE="$BASEDIR"/data/$CODENAME/udeb_exclude \
TASK=Debian-edu-netinst \
MAXISOS=1 MAXJIGDOS=1 \
DI=${DI} DI_DIST=${DI} \
DI_DIR="$ARCH_DI_DIR" \
./testingcds "$arch" &
build_started SIDEDUNIFIRMWARE
OMIT_RELEASE_NOTES=1 OMIT_MANUAL=1 NORECOMMENDS=1 \
NOSUGGESTS=1 COMPLETE=0 INSTALLER_CD=H \
FORCE_FIRMWARE=1 \
CDNAME=firmware-edu \
DI=sid DI_WWW_HOME=default \
DEBIAN_EDU=1 \
KERNEL_PARAMS="modules=debian-edu-install-udeb" \
DESKTOP=xfce VARIANTS=xen \
EXCLUDE=debian-edu-exclude \
UDEB_EXCLUDE="$BASEDIR"/data/$CODENAME/udeb_exclude \
TASK=Debian-edu-netinst \
MAXISOS=1 MAXJIGDOS=1 \
DI=${DI} DI_DIST=${DI} \
DI_DIR="$ARCH_DI_DIR" \
./testingcds "$arch" &
fi
fi
if [ "$TESTING_WANTED" = "1" ] ; then
if [ "$NI_WANTED" = "1" ] ; then
build_started TESTINGEDUNI
OMIT_RELEASE_NOTES=1 OMIT_MANUAL=1 NORECOMMENDS=1 \
NOSUGGESTS=1 COMPLETE=0 INSTALLER_CD=G \
CDNAME=debian-edu \
DEBIAN_EDU=1 \
KERNEL_PARAMS="modules=debian-edu-install-udeb" \
DESKTOP=xfce VARIANTS=xen \
EXCLUDE=debian-edu-exclude \
UDEB_EXCLUDE="$BASEDIR"/data/$CODENAME/udeb_exclude \
TASK=Debian-edu-netinst \
MAXISOS=1 MAXJIGDOS=1 \
DI=${TESTING_SUITE} DI_DIST="$DI_DIST" \
./testingcds "$arch" &
build_started TESTINGEDUNIFIRMWARE
OMIT_RELEASE_NOTES=1 OMIT_MANUAL=1 NORECOMMENDS=1 \
NOSUGGESTS=1 COMPLETE=0 INSTALLER_CD=H \
FORCE_FIRMWARE=1 \
CDNAME=firmware-edu \
DEBIAN_EDU=1 \
KERNEL_PARAMS="modules=debian-edu-install-udeb" \
DESKTOP=xfce VARIANTS=xen \
EXCLUDE=debian-edu-exclude \
UDEB_EXCLUDE="$BASEDIR"/data/$CODENAME/udeb_exclude \
TASK=Debian-edu-netinst \
MAXISOS=1 MAXJIGDOS=1 \
DI=${TESTING_SUITE} DI_DIST="$DI_DIST" \
./testingcds "$arch" &
fi
fi
fi
fi
catch_parallel_builds
rm -rf $PUBDIRJIG/$arch
if [ "$arch_error"x = "none"x ] ; then
@ -268,8 +332,6 @@ if lockfile -r0 $BUILDLOCK ; then
done
if [ "$NOSYNC"x = ""x ] ; then
if [ "$RELEASE_BUILD"x = ""x ] ; then
echo "Clean up old builds on build machine:"
for DIR in $PUBDIRJIG/* $PUBDIRJIG-firmware/* ; do
cd $DIR
@ -282,24 +344,35 @@ if lockfile -r0 $BUILDLOCK ; then
done
echo "Sync normal builds across to output dir: $RSYNC_TARGET_FREE"
rsync -rHltv $PUBDIRJIG/ $RSYNC_TARGET_FREE/
echo "Remove extra builds for normal dailies:"
~/bin/remove_extra_dailies $RSYNC_TARGET_FREE "$ARCHES"
rsync -rHlt $PUBDIRJIG/ $RSYNC_TARGET_FREE/
echo "Sync firmware builds across to output dir: $RSYNC_TARGET_FIRMWARE"
rsync -rHltv $PUBDIRJIG-firmware/ $RSYNC_TARGET_FIRMWARE/
echo "Remove extra builds for firmware dailies:"
~/bin/remove_extra_dailies $RSYNC_TARGET_FIRMWARE "$ARCHES_FIRMWARE"
rsync -rHlt $PUBDIRJIG-firmware/ $RSYNC_TARGET_FIRMWARE/
echo "Remove extra builds:"
~/bin/remove_extra_dailies
echo "$DATE-$BUILDNUM" > $RSYNC_TARGET_FREE/daily-trace
else # Release build
echo "Sync normal builds for $RELEASE_BUILD across to output dir: $RSYNC_TARGET_FREE"
rsync -rHltv $OUT_FREE_TST/ $RSYNC_TARGET_FREE/
echo "Sync firmware builds for $RELEASE_BUILD across to output dir: $RSYNC_TARGET_FIRMWARE"
rsync -rHltv $OUT_FW_TST/ $RSYNC_TARGET_FIRMWARE/
if [ $(hostname) = casulana ]; then
(cd ${RSYNC_TARGET_FREE}/sid_d-i/${DATE_BUILD} && find . -name '*.iso' > .iso-wanted)
(cd ${RSYNC_TARGET_FIRMWARE}/sid_d-i/${DATE_BUILD}/ && find . -name '*.iso' > .iso-wanted)
rsync_to_pettersson ${RSYNC_TARGET_FREE}/sid_d-i/${DATE_BUILD}/ \
daily-builds/sid_d-i/${DATE_BUILD}/ \
'--exclude *.iso'
publish_on_pettersson daily-builds/sid_d-i/${DATE_BUILD}
rsync_to_pettersson ${RSYNC_TARGET_FIRMWARE}/sid_d-i/${DATE_BUILD}/ \
unofficial/non-free/cd-including-firmware/daily-builds/sid_d-i/${DATE_BUILD}/ \
'--exclude *.iso'
publish_on_pettersson unofficial/non-free/cd-including-firmware/daily-builds/sid_d-i/${DATE_BUILD}
rsync_to_pettersson ${OUT_BASE}/cd-sources/ cd-sources/
publish_on_pettersson cd-sources
fi
fi
cd $TOPDIR
rm -f $BUILDLOCK
fi

286
contrib/cronjob.weekly
View File

@ -11,10 +11,10 @@ export TOPDIR
BUILDLOCK=$HOME/.debian-cd.lock
export PUBDIRJIG=$PUBDIR/weekly-builds
export PUBDIROSTRACE=/mnt/nfs-cdimage/.live/trace-os
export PUBDIRLIVETRACE=/mnt/nfs-cdimage/.live/trace-live
export PUBDIROSTRACE=${OUT_BASE}/.live/trace-os
export PUBDIRLIVETRACE=${OUT_BASE}/.live/trace-live
export DATE_BUILD="$DATE-$BUILDNUM"
export CD_I_F="/mnt/nfs-cdimage/unofficial/non-free/cd-including-firmware"
export CD_I_F="${OUT_BASE}/unofficial/non-free/cd-including-firmware"
. $TOPDIR/common.sh
@ -26,18 +26,19 @@ reboot_lock
USE_DAILY_DI=Y
if [ "$RELEASE_BUILD"x = ""x ] ; then
export RSYNC_TARGET=/mnt/nfs-cdimage/weekly-builds
export RSYNC_TARGET=${OUT_BASE}/weekly-builds
export RSYNC_TARGET_FIRMWARE=${CD_I_F}/weekly-builds
else
export RSYNC_TARGET=/mnt/nfs-cdimage/.${RELEASE_BUILD}/debian-cd
export RSYNC_TARGET=${OUT_BASE}/.${RELEASE_BUILD}/debian-cd
export RSYNC_TARGET_FIRMWARE=${CD_I_F}/.${RELEASE_BUILD}/debian-cd
export CONF=~/build.stretch/CONF.sh.${RELEASE_BUILD}
export CONF=~/build.${CODENAME}/CONF.sh.${RELEASE_BUILD}
export RELEASE_BUILD=$RELEASE_BUILD
# If we're doing a normal set of daily/weekly builds, leave the
# checksum filenames alone. Otherwise, make life easier for people
# combining things later and append a suitable name as we build.
export EXTENSION=".large"
USE_DAILY_DI=N
NOSNAP=1
fi
. images4testing_d-i
@ -52,28 +53,33 @@ fi
export DI_DIST
export DI_CODENAME
export OUT_FREE_SID=$PUBDIRJIG
export OUT_FREE_TST=$PUBDIRJIG
export OUT_FREE_SID=${PUBDIRJIG}
export OUT_FREE_TST=${PUBDIRJIG}
export OUT_FW_SID=${PUBDIRJIG}-firmware
export OUT_FW_TST=${PUBDIRJIG}-firmware
BUILDS_RUNNING=""
export NOEDU=1
#export NOEDU=1
#export NOOPENSTACK=1
#export NOLIVE=1
rm -f $PUBDIRLIVETRACE $PUBDIROSTRACE
~/build.stretch/cronjob.weekly-live &
if lockfile -r0 $BUILDLOCK ; then
# echo "NOT checking for git updates"
rm -f $PUBDIRLIVETRACE $PUBDIROSTRACE
~/build.${CODENAME}/cronjob.weekly-live &
echo "git update debian-cd"
cd debian-cd && git pull ; cd ..
mkdir -p ${PUBDIRJIG}/trace
mkdir -p $RSYNC_TARGET/trace $RSYNC_TARGET_FIRMWARE/trace
# Keep track of the serial for the archive we're building against,
# for later archive diffing for release announce preparation:
serial=$(get_archive_serial)
echo "$serial" > $PUBDIRJIG/trace/archive-serial
echo "$serial" > $RSYNC_TARGET/trace/archive-serial
echo "$serial" > $RSYNC_TARGET_FIRMWARE/trace/archive-serial
# Work out the default desktop, and do *not* build a CD1 for that
# desktop - it'll be done in the full set anyway
@ -85,25 +91,31 @@ if lockfile -r0 $BUILDLOCK ; then
# echo "NOT checking for popcon updates"
if [ "$NOPOPCON"x = ""x ] ; then
cd debian-cd && ./tools/update_popcon tasks/stretch/popularity-contest ; cd ..
cd debian-cd && ./tools/update_popcon tasks/${CODENAME}/popularity-contest
error=$?
if [ $error != 0 ]; then
echo "popcon update failed, error $error";
echo "aborting";
exit 1
fi
cd ..
fi
cd $TOPDIR
mkdir -p $PUBDIRJIG/trace
mkdir -p $RSYNC_TARGET $RSYNC_TARGET_FIRMWARE
for arch in $ARCHES; do
# Reset envvars for next iteration
arch_error=""
arch_start=`now`
if [ "$arch" != multi-arch ] ; then
rm -rf $PUBDIRJIG/$arch $PUBDIRJIG-firmware/$arch
mkdir -p $PUBDIRJIG/$arch
rm -rf ${PUBDIRJIG}/$arch ${PUBDIRJIG}-firmware/$arch
for i in iso-cd jigdo-cd; do
mkdir -p ${PUBDIRJIG}/$arch/$i
if $(arch_has_firmware $arch) ; then
mkdir -p $PUBDIRJIG-firmware/$arch
mkdir -p ${PUBDIRJIG}-firmware/$arch/$i
fi
done
if [ "$arch" != multi-arch ] ; then
# export DI_DIR="$ARCH_DI_DIR"
export VARIANTS=""
@ -197,7 +209,7 @@ if lockfile -r0 $BUILDLOCK ; then
esac
fi
# individual desktop cds: gnome, kde, xfce. lxde
# individual desktop cd #1: xfce only these days
if [ "$arch"x != "source"x ] && [ "$NOCD"x = ""x ] ; then
if [ "$NOXFCECD"x = ""x ] ; then
export MAX_PKG_SIZE=300000000
@ -207,6 +219,7 @@ if lockfile -r0 $BUILDLOCK ; then
build_started XFCECD
INSTALLER_CD=5 TASK=Debian-xfce \
KERNEL_PARAMS='desktop=xfce' \
OMIT_RELEASE_NOTES=1 OMIT_MANUAL=1 \
MAXCDS=1 MAXISOS=ALL MAXJIGDOS=ALL \
./testingcds "$arch" &
fi
@ -252,94 +265,79 @@ if lockfile -r0 $BUILDLOCK ; then
DI=${DI} DI_DIST=${DI} \
./testingcds "$arch" &
fi
fi
catch_parallel_builds
# Special debian-edu builds, only minor tweaks. #846006
if [ "$NOEDU"x = ""x ] ; then
if [ "$arch" = "i386" ] || [ "$arch" = "amd64" ] ; then
for dir in $PUBDIRJIG/$arch/jigdo-*; do
generate_checksums_for_arch $arch $dir
done
if [ -d $PUBDIRJIG-firmware/$arch ] ; then
for dir in ${PUBDIRJIG}-firmware/$arch/jigdo-*; do
generate_checksums_for_arch $arch $dir
done
fi
if [ "$NOSYNC"x = ""x ] ; then
if [ "$arch_error"x = "none"x ] ; then
if [ "$RELEASE_BUILD"x = ""x ] ; then
echo "Signing checksums files using the automatic key"
~/build.stretch/sign-images $PUBDIRJIG $arch
fi
echo " Running ~/build.stretch/iso_run $PUBDIRJIG/ $RSYNC_TARGET/ $arch &"
~/build.stretch/iso_run $PUBDIRJIG/ $RSYNC_TARGET/ $arch &
if [ -d $PUBDIRJIG-firmware/$arch ] ; then
if [ "$RELEASE_BUILD"x = ""x ] ; then
echo "Signing checksums files using the automatic key"
~/build.stretch/sign-images $PUBDIRJIG-firmware $ARCH
fi
echo " Running ~/build.stretch/iso_run ${PUBDIRJIG}-firmware/ $RSYNC_TARGET_FIRMWARE/ $arch &"
~/build.stretch/iso_run ${PUBDIRJIG}-firmware/ $RSYNC_TARGET_FIRMWARE/ $arch &
fi
else
~/build.stretch/report_build_error $PUBDIRJIG $RSYNC_TARGET $arch "$arch_error"
if [ -d ${PUBDIRJIG}-firmware/$arch ] ; then
~/build.stretch/report_build_error ${PUBDIRJIG}-firmware $RSYNC_TARGET_FIRMWARE/ $arch "$arch_error"
fi
fi
fi
else # multi-arch
rm -rf $PUBDIRJIG/multi-arch $PUBDIRJIG-firmware/multi-arch
for i in iso-dvd jigdo-dvd iso-cd jigdo-cd; do
mkdir -p $PUBDIRJIG/multi-arch/$i
if $(arch_has_firmware multi-arch) ; then
mkdir -p $PUBDIRJIG-firmware/multi-arch/$i
fi
done
if [ "$NODVD"x = ""x ] ; then
export MAX_PKG_SIZE=999999999999
export DESKTOP=all
# Special case: make DVD1 fit on a 4GB USB
# stick (#612074)
build_started DVD
export FORCE_CD_SIZE1=STICK4GB
INSTALLER_CD=6 TASK=Debian-all \
KERNEL_PARAMS='desktop=all' \
MAXCDS=1 MAXISOS=ALL MAXJIGDOS=ALL \
VARIANTS=xen \
./testingcds "i386 amd64 source" &
# DVD with firmware
if $(arch_has_firmware $arch) ; then
build_started DVDFIRMWARE
INSTALLER_CD=J TASK=Debian-all \
FORCE_FIRMWARE=1 CDNAME=firmware \
KERNEL_PARAMS="desktop=all" \
MAXCDS=1 MAXISOS=ALL MAXJIGDOS=ALL \
VARIANTS=xen \
DI=${DI} DI_DIST=${DI} \
./testingcds "i386 amd64 source" &
fi
unset FORCE_CD_SIZE1
fi
if [ "$NOEDU"x = ""x ] && [ "$NOUSB"x = ""x ] ; then
if $(arch_has_firmware $arch) ; then
build_started FIRMWAREEDUUSB
# netinst
build_started EDUNI
OMIT_RELEASE_NOTES=1 OMIT_MANUAL=1 NORECOMMENDS=1 \
NOSUGGESTS=1 COMPLETE=0 INSTALLER_CD=H \
FORCE_FIRMWARE=1 CDNAME=debian-edu-firmware \
NOSUGGESTS=1 COMPLETE=0 INSTALLER_CD=G \
CDNAME=debian-edu \
DEBIAN_EDU=1 \
KERNEL_PARAMS="desktop=kde" \
DESKTOP=kde VARIANTS=xen \
KERNEL_PARAMS="modules=debian-edu-install-udeb" \
DESKTOP=xfce VARIANTS=xen \
EXCLUDE=debian-edu-exclude \
UDEB_EXCLUDE="$BASEDIR"/data/$CODENAME/udeb_exclude \
TASK=Debian-edu-netinst \
MAXISOS=1 MAXJIGDOS=1 \
DI=${DI} DI_DIST=${DI} \
./testingcds "$arch" &
# netinst with firmware
build_started EDUNIFIRMWARE
OMIT_RELEASE_NOTES=1 OMIT_MANUAL=1 NORECOMMENDS=1 \
NOSUGGESTS=1 COMPLETE=0 INSTALLER_CD=H \
FORCE_FIRMWARE=1 \
CDNAME=firmware-edu \
DEBIAN_EDU=1 \
KERNEL_PARAMS="modules=debian-edu-install-udeb" \
DESKTOP=xfce VARIANTS=xen \
EXCLUDE=debian-edu-exclude \
UDEB_EXCLUDE="$BASEDIR"/data/$CODENAME/udeb_exclude \
TASK=Debian-edu-netinst \
MAXISOS=1 MAXJIGDOS=1 \
DI=${DI} DI_DIST=${DI} \
./testingcds "$arch" &
if [ "$NOUSB"x = ""x ] ; then
# USB "BD" image
build_started EDUUSB
NORECOMMENDS=0 COMPLETE=0 INSTALLER_CD=K \
CDNAME=debian-edu-usb \
DEBIAN_EDU=1 \
KERNEL_PARAMS="modules=debian-edu-install-udeb" \
DESKTOP=xfce VARIANTS=xen \
EXCLUDE=debian-edu-exclude \
TASK=Debian-edu-full \
MAXISOS=1 MAXJIGDOS=1 \
DI=${DI} DI_DIST=${DI} \
./testingcds "amd64 i386" &
./testingcds "$arch" &
# USB "BD" image with firmware
build_started EDUUSBFIRMWARE
NORECOMMENDS=0 COMPLETE=0 INSTALLER_CD=L \
FORCE_FIRMWARE=1 \
CDNAME=firmware-edu-usb \
DEBIAN_EDU=1 \
KERNEL_PARAMS="modules=debian-edu-install-udeb" \
DESKTOP=xfce VARIANTS=xen \
EXCLUDE=debian-edu-exclude \
TASK=Debian-edu-full \
MAXISOS=1 MAXJIGDOS=1 \
DI=${DI} DI_DIST=${DI} \
./testingcds "$arch" &
fi
fi
fi
fi
# We've spawned a load of builds; wait here for them all
# to finish
catch_parallel_builds
else # multi-arch
if [ "$NOCD"x = ""x ] && [ "$NONI"x = ""x ] ; then
export DESKTOP=all
@ -366,33 +364,31 @@ if lockfile -r0 $BUILDLOCK ; then
./testingcds "amd64 i386" &
fi
if [ "$NOEDU"x = ""x ] ; then
if $(arch_has_firmware $arch) ; then
build_started NIFIRMWAREEDU
OMIT_RELEASE_NOTES=1 OMIT_MANUAL=1 NORECOMMENDS=1 \
NOSUGGESTS=1 COMPLETE=0 INSTALLER_CD=G \
FORCE_FIRMWARE=1 CDNAME=debian-edu-firmware \
DEBIAN_EDU=1 \
KERNEL_PARAMS="desktop=kde" \
DESKTOP=kde VARIANTS=xen \
EXCLUDE=debian-edu-exclude \
UDEB_EXCLUDE="$BASEDIR"/data/$CODENAME/udeb_exclude \
TASK=Debian-edu-netinst \
MAXISOS=1 MAXJIGDOS=1 \
DI=${DI} DI_DIST=${DI} \
./testingcds "amd64 i386" &
fi
fi
fi
# We've spawned a load of builds; wait here for them all
# to finish
catch_parallel_builds
for dir in $PUBDIRJIG/multi-arch/jigdo-*; do
generate_checksums_for_arch multi-arch $dir
fi # end of multi-arch
if [ "$RELEASE_BUILD"x = ""x ] ; then
echo "$arch: Not generating torrent files for a non-release build"
else
echo "$arch: Generating torrent files"
~/build.${CODENAME}/mktorrent ${PUBDIRJIG}/$arch/iso-*/*.iso
if [ -d ${PUBDIRJIG}-firmware/$arch ] ; then
~/build.${CODENAME}/mktorrent ${PUBDIRJIG}-firmware/$arch/iso-*/*.iso
fi
fi
echo "$arch: Generating checksum files"
for dir in ${PUBDIRJIG}/$arch/jigdo-*; do
generate_checksums_for_arch $arch $dir
done
if [ -d $PUBDIRJIG-firmware/multi-arch ] ; then
for dir in ${PUBDIRJIG}-firmware/multi-arch/jigdo-*; do
generate_checksums_for_arch multi-arch $dir
if [ -d ${PUBDIRJIG}-firmware/$arch ] ; then
for dir in ${PUBDIRJIG}-firmware/$arch/jigdo-*; do
generate_checksums_for_arch $arch $dir
done
fi
@ -400,45 +396,57 @@ if lockfile -r0 $BUILDLOCK ; then
if [ "$arch_error"x = "none"x ] || [ "$arch_error"x = ""x ] ; then
if [ "$RELEASE_BUILD"x = ""x ] ; then
echo "Signing checksums files using the automatic key"
~/build.stretch/sign-images $PUBDIRJIG multi-arch
~/build.${CODENAME}/sign-images ${PUBDIRJIG} $arch
fi
echo " Running ~/build.stretch/iso_run $PUBDIRJIG/ $RSYNC_TARGET/ multi-arch &"
~/build.stretch/iso_run $PUBDIRJIG/ $RSYNC_TARGET/ multi-arch &
if [ -d $PUBDIRJIG-firmware/multi-arch ] ; then
echo " Running ~/build.${CODENAME}/iso_run ${PUBDIRJIG}/ $RSYNC_TARGET/ $arch \"\" &"
~/build.${CODENAME}/iso_run ${PUBDIRJIG}/ $RSYNC_TARGET/ $arch "" &
if [ -d ${PUBDIRJIG}-firmware/$arch ] ; then
if [ "$RELEASE_BUILD"x = ""x ] ; then
echo "Signing checksums files using the automatic key"
~/build.stretch/sign-images $PUBDIRJIG-firmware multi-arch
~/build.${CODENAME}/sign-images ${PUBDIRJIG}-firmware $arch
fi
echo " Running ~/build.stretch/iso_run ${PUBDIRJIG}-firmware/ $RSYNC_TARGET_FIRMWARE/ multi-arch &"
~/build.stretch/iso_run ${PUBDIRJIG}-firmware/ $RSYNC_TARGET_FIRMWARE/ multi-arch &
echo " Running ~/build.${CODENAME}/iso_run ${PUBDIRJIG}-firmware/ $RSYNC_TARGET_FIRMWARE/ $arch firmware &"
~/build.${CODENAME}/iso_run ${PUBDIRJIG}-firmware/ $RSYNC_TARGET_FIRMWARE/ $arch firmware &
fi
else
~/build.stretch/report_build_error ${PUBDIRJIG} $RSYNC_TARGET multi-arch "$arch_error"
if [ -d ${PUBDIRJIG}-firmware/multi-arch ] ; then
~/build.stretch/report_build_error ${PUBDIRJIG}-firmware $RSYNC_TARGET_FIRMWARE/ $multi-arch "$arch_error"
~/build.${CODENAME}/report_build_error ${PUBDIRJIG} $RSYNC_TARGET $arch "$arch_error"
if [ -d ${PUBDIRJIG}-firmware/$arch ] ; then
~/build.${CODENAME}/report_build_error ${PUBDIRJIG}-firmware $RSYNC_TARGET_FIRMWARE/ $arch "$arch_error"
fi
fi
fi
fi # end of multi-arch
done
# Generate HEADER.html files
~/build.stretch/generate_headers $RSYNC_TARGET ""
~/build.stretch/generate_headers $RSYNC_TARGET_FIRMWARE firmware
if [ "$NOFW"x = ""x ] ; then
~/build.stretch/generate_firmware_images stretch
~/build.stretch/generate_firmware_images sid
~/build.${CODENAME}/generate_firmware_images ${CODENAME}
~/build.${CODENAME}/generate_firmware_images sid
fi
if [ $(hostname) = casulana ]; then
rsync_to_pettersson ${OUT_BASE}/cd-sources/ cd-sources/
publish_on_pettersson cd-sources
fi
if [ "$NOSNAP"x = ""x ] ; then
if [ $(hostname) = casulana ]; then
mkdir -p /tmp/snapshots
rsync_to_pettersson /tmp/snapshots/ snapshots/
publish_on_pettersson snapshots
else
~/bin/weekly-snapshots
fi
fi
catch_live_builds
date -u > $PUBDIRJIG/trace/cdimage.debian.org
for DIR in $RSYNC_TARGET $RSYNC_TARGET_FIRMWARE; do
date -u > $DIR/trace/cdimage.debian.org
if [ $(hostname) = casulana ]; then
PETOUT=${DIR##${OUT_BASE}/}
rsync_to_pettersson $DIR/trace/ ${PETOUT}/trace/
publish_on_pettersson ${PETOUT}/trace
fi
done
rm -f $BUILDLOCK
fi

109
contrib/cronjob.weekly-live
View File

@ -1,5 +1,22 @@
#!/bin/bash
TOPDIR=$(dirname $0)
if [ "$TOPDIR" = "." ] ; then
TOPDIR=`pwd`
fi
export TOPDIR
. $TOPDIR/settings.sh
. $TOPDIR/common.sh
export WEEKLY_LIVE=${OUT_BASE}/weekly-live-builds
export NF_WEEKLY_LIVE=${CD_I_F}/weekly-live-builds
export WEEKLY_OPENSTACK=${OUT_BASE}/openstack/testing
if [ "$RELEASE_BUILD"x != ""x ] ; then
RELEASE_OUT=${OUT_BASE}/.${RELEASE_BUILD}
fi
now () {
date -u +%F:%H:%M:%S
}
@ -12,20 +29,56 @@ if [ "$NOLIVE"x = ""x ] ; then
ERROR=$?
if [ $ERROR -eq 0 ] ; then
if [ "$RELEASE_BUILD"x = ""x ] ; then
~/build.stretch/sign-images /mnt/nfs-cdimage/.live/new/free ""
mv -f /mnt/nfs-cdimage/weekly-live-builds /mnt/nfs-cdimage/weekly-live-builds.old
mv /mnt/nfs-cdimage/.live/new/free/ /mnt/nfs-cdimage/weekly-live-builds
rm -rf /mnt/nfs-cdimage/weekly-live-builds.old
~/build.${CODENAME}/sign-images ${LIVE_OUT}/new/free ""
mv -f ${WEEKLY_LIVE} ${WEEKLY_LIVE}.old
mv ${LIVE_OUT}/new/free/ ${WEEKLY_LIVE}
rm -rf ${WEEKLY_LIVE}.old
~/build.${CODENAME}/sign-images ${LIVE_OUT}/new/non-free ""
mv -f ${NF_WEEKLY_LIVE} ${NF_WEEKLY_LIVE}.old
mv ${LIVE_OUT}/new/non-free ${NF_WEEKLY_LIVE}
rm -rf ${NF_WEEKLY_LIVE}.old
# Generate HEADER.html files
for ARCH in amd64 i386 source; do
~/build.${CODENAME}/generate_headers \
~/build.${CODENAME}/live-header.html.in \
${WEEKLY_LIVE}/$ARCH $ARCH ""
~/build.${CODENAME}/generate_headers \
~/build.${CODENAME}/HEADER.html.in \
${NF_WEEKLY_LIVE}/$ARCH $ARCH firmware
done
# If we're on casulana, push things to pettersson here
if [ $(hostname) = casulana ]; then
rsync_to_pettersson ${WEEKLY_LIVE}/ weekly-live-builds/
rsync_to_pettersson ${NF_WEEKLY_LIVE}/ unofficial/non-free/images-including-firmware/weekly-live-builds/
publish_on_pettersson weekly-live-builds unofficial/non-free/images-including-firmware/weekly-live-builds
fi
~/build.stretch/sign-images /mnt/nfs-cdimage/.live/new/non-free ""
mv -f ${CD_I_F}/weekly-live-builds ${CD_I_F}/weekly-live-builds.old
mv /mnt/nfs-cdimage/.live/new/non-free ${CD_I_F}/weekly-live-builds
rm -rf ${CD_I_F}/weekly-live-builds.old
else
mkdir -p /mnt/nfs-cdimage/.${RELEASE_BUILD}/
rm -rf /mnt/nfs-cdimage/.${RELEASE_BUILD}/live-*
mv /mnt/nfs-cdimage/.live/new/free/ /mnt/nfs-cdimage/.${RELEASE_BUILD}/live-free
mv /mnt/nfs-cdimage/.live/new/non-free /mnt/nfs-cdimage/.${RELEASE_BUILD}/live-non-free
mkdir -p ${RELEASE_OUT}/
rm -rf ${RELEASE_OUT}/live-*
mv ${LIVE_OUT}/new/free/ ${RELEASE_OUT}/live-free
mv ${LIVE_OUT}/new/non-free ${RELEASE_OUT}/live-non-free
# Generate HEADER.html files
for ARCH in amd64 i386 source; do
~/build.${CODENAME}/generate_headers \
~/build.${CODENAME}/live-header.html.in \
${RELEASE_OUT}/live-free/$ARCH $ARCH ""
~/build.${CODENAME}/generate_headers \
~/build.${CODENAME}/HEADER.html.in \
${RELEASE_OUT}/live-non-free/$ARCH $ARCH firmware
done
# If we're on casulana, push things to pettersson here
if [ $(hostname) = casulana ]; then
rsync_to_pettersson ${RELEASE_OUT}/live-free/ .${RELEASE_BUILD}/live-free/
rsync_to_pettersson ${RELEASE_OUT}/live-non-free/ .${RELEASE_BUILD}/live-non-free/
publish_on_pettersson .${RELEASE_BUILD}/live-free .${RELEASE_BUILD}/live-non-free
fi
fi
fi
# Wait for the KVM to shut down cleanly
@ -53,20 +106,32 @@ if [ "$NOOPENSTACK"x = ""x ] ; then
sleep 30
fi
if [ $ERROR -eq 0 ] ; then
mkdir -p /mnt/nfs-cdimage/.live/new/openstack-stretch
mv -v /mnt/nfs-cdimage/.live/new/openstack-stretch-*/*.* /mnt/nfs-cdimage/.live/new/openstack-stretch
rm -rf ${LIVE_OUT}/new/openstack-${CODENAME}
mkdir -p ${LIVE_OUT}/new/openstack-${CODENAME}
mv -v ${LIVE_OUT}/new/openstack-${CODENAME}-*/*.* ${LIVE_OUT}/new/openstack-${CODENAME}
for SUM in MD5 SHA1 SHA256 SHA512; do
cat /mnt/nfs-cdimage/.live/new/openstack-stretch-*/${SUM}SUMS > /mnt/nfs-cdimage/.live/new/openstack-stretch/${SUM}SUMS
cat ${LIVE_OUT}/new/openstack-${CODENAME}-*/${SUM}SUMS > ${LIVE_OUT}/new/openstack-${CODENAME}/${SUM}SUMS
done
if [ "$RELEASE_BUILD"x = ""x ] ; then
~/build.stretch/sign-images /mnt/nfs-cdimage/.live/new/openstack-stretch ""
mv /mnt/nfs-cdimage/.live/new/openstack-stretch/* /mnt/nfs-cdimage/openstack/testing
cp ~/build.stretch/openstack-header.html /mnt/nfs-cdimage/openstack/testing/HEADER.html
~/build.${CODENAME}/sign-images ${LIVE_OUT}/new/openstack-${CODENAME} ""
mv ${LIVE_OUT}/new/openstack-${CODENAME}/* ${WEEKLY_OPENSTACK}
cp ~/build.${CODENAME}/openstack-header.html ${WEEKLY_OPENSTACK}/HEADER.html
# If we're on casulana, push things to pettersson here
if [ $(hostname) = casulana ]; then
rsync_to_pettersson ${WEEKLY_OPENSTACK}/ openstack/testing/
publish_on_pettersson openstack/testing
fi
else
mkdir -p /mnt/nfs-cdimage/.${RELEASE_BUILD}
rm -rf /mnt/nfs-cdimage/.${RELEASE_BUILD}/openstack
mv /mnt/nfs-cdimage/.live/new/openstack-stretch /mnt/nfs-cdimage/.${RELEASE_BUILD}/openstack
cp ~/build.stretch/openstack-header.html /mnt/nfs-cdimage/.${RELEASE_BUILD}/openstack/HEADER.html
mkdir -p ${RELEASE_OUT}
rm -rf ${RELEASE_OUT}/openstack
mv ${LIVE_OUT}/new/openstack-${CODENAME} ${RELEASE_OUT}/openstack
cp ~/build.${CODENAME}/openstack-header.html ${RELEASE_OUT}/openstack/HEADER.html
# If we're on casulana, push things to pettersson here
if [ $(hostname) = casulana ]; then
rsync_to_pettersson ${RELEASE_OUT}/openstack/ .${RELEASE_BUILD}/openstack/
publish_on_pettersson .${RELEASE_BUILD}/openstack
fi
fi
fi
fi

6
contrib/daily-firmware.html
View File

@ -1,10 +1,10 @@
<h1>Daily "firmware" build #BUILDNUM for ARCH, using installer build from INST_VER</h1>
<h1>Daily "firmware" build #BUILDNUM for ARCH, using the daily installer build</h1>
<p>These images will install the testing version of Debian, currently
<strong>Stretch</strong>. They include non-free firmware to make
<strong>CODENAME</strong>. They include non-free firmware to make
installation easier on some systems requiring proprietary but
redistributable
firmware. See <a href="http://wiki.debian.org/Firmware">http://wiki.debian.org/Firmware</a>
firmware. See <a href="https://wiki.debian.org/Firmware">https://wiki.debian.org/Firmware</a>
for more details. </p>
<p>See the <a href="/cdimage/daily-builds/">top-level daily

4
contrib/daily.html
View File

@ -1,7 +1,7 @@
<h1>Daily build #BUILDNUM for ARCH, using installer build from INST_VER</h1>
<h1>Daily build #BUILDNUM for ARCH, using the daily installer build</h1>
<p>These images will install the testing version of Debian, currently
<strong>Stretch</strong>.</p>
<strong>CODENAME</strong>.</p>
<p>See the <a href="/cdimage/daily-builds/">top-level daily
directory</a> for more information about the daily builds.</p>

9
contrib/ftp.cron
View File

@ -1,4 +1,4 @@
#!/bin/sh
#!/bin/bash
set -e
@ -62,6 +62,8 @@ if [ -f "$TRACE" ] ; then
cp "$MYCOPY" "$MYCOPY".building
BUILD_DONE=0
# "Daily" builds
# Only run certain builds, otherwise we're just wasting
# time reproducing identical results:
@ -79,6 +81,7 @@ if [ -f "$TRACE" ] ; then
echo "Not running even-numbered daily build #$BUILDNUM"
else
echo "Running daily build #$BUILDNUM."
BUILD_DONE=1
if [ "$FORCE"x = "1"x ] ; then
echo "Build forced by hand"
else
@ -99,9 +102,13 @@ if [ -f "$TRACE" ] ; then
mail_list "$HOSTNAME BAILING OUT OF WEEKLY BUILD"
else
$TOPDIR/cronjob.weekly
BUILD_DONE=1
fi
fi
cd $HOME && ./bin/cdbuilder_log_analyser > build/log/analysis.html
if [ "$BUILD_DONE"x = "1"x ]; then
/usr/local/bin/static-update-component cdbuilder-logs.debian.org
fi
exit 0
fi
fi

33
contrib/generate_firmware_images
View File

@ -1,14 +1,21 @@
#!/bin/sh
#!/bin/bash
set -e
OUT=/mnt/nfs-cdimage/unofficial/non-free/firmware
TOPDIR=~/build
SUITE=$1
TOPDIR=$(dirname $0)
if [ "$TOPDIR" = "." ] ; then
TOPDIR=`pwd`
fi
export TOPDIR
. $TOPDIR/CONF.sh
. $TOPDIR/settings.sh
. $TOPDIR/common.sh
if [ "$SUITE"x = ""x ] ; then
OUT=${OUT_BASE}/unofficial/non-free/firmware
RELEASE=$1
if [ "$RELEASE"x = ""x ] ; then
echo "Need to know what to produce!"
exit 1
fi
@ -17,20 +24,25 @@ fi
export TDIR=/srv/cdbuilder.debian.org/dst/deb-cd/tmp/firmware
export BASEDIR=$TOPDIR/debian-cd
mkdir -p $TDIR
fakeroot $BASEDIR/tools/make-firmware-image $MIRROR $SUITE $TDIR
fakeroot $BASEDIR/tools/make-firmware-image $MIRROR $RELEASE $TDIR
DATE=`date +%Y%m%d`
mkdir -p $OUT/$SUITE/$DATE
mv $TDIR/firmware*.* $TDIR/*SUMS $OUT/$SUITE/$DATE/
mkdir -p $OUT/$RELEASE/$DATE
mv $TDIR/firmware*.* $TDIR/*SUMS $OUT/$RELEASE/$DATE/
if [ "$RELEASE_BUILD"x = ""x ] ; then
echo "Signing firmware checksums files using the automatic key"
~/build.stretch/sign-images $OUT/$SUITE/$DATE ""
~/build.${CODENAME}/sign-images $OUT/$RELEASE/$DATE ""
fi
if [ $(hostname) = casulana ]; then
# Sync to pettersson
rsync_to_pettersson ${OUT_BASE}/unofficial/non-free/firmware/$RELEASE/${DATE}/ unofficial/non-free/firmware/${RELEASE}/${DATE}/
publish_on_pettersson unofficial/non-free/firmware/${RELEASE}/${DATE}/
else
# Update the current link, and delete the old surplus builds. Keep up
# to 3
cd $OUT/$SUITE
cd $OUT/$RELEASE
rm -f current
ln -sf $DATE current
@ -42,3 +54,4 @@ if [ "$NUM" -gt "0" ] ; then
echo " $REMOVE"
rm -rf $REMOVE
fi
fi

69
contrib/generate_headers
View File

@ -1,44 +1,80 @@
#!/bin/bash
#
# Tool to generate HEADER.html in various directories with
# context-specific documentation
#
# Call as:
#
# generate_headers $INPUT_FILE $OUTPUT_DIR $ARCH [firmware]
. ~/build.stretch/settings.sh
. ~/build.stretch/common.sh
set -e
TOPDIR=$(dirname $0)
if [ "$TOPDIR" = "." ] ; then
TOPDIR=`pwd`
fi
export TOPDIR
. ${TOPDIR}/CONF.sh
. ${TOPDIR}/settings.sh
. ${TOPDIR}/common.sh
INPUT=~/build.stretch/HEADER.html.in
CPP="cpp -traditional -undef -P -C -nostdinc"
if [ "$1"x = ""x ] ; then
echo "$0: Need to be told what input file to use!"
exit 1
fi
INPUT=$1
if [ "$2"x = ""x ] ; then
echo "$0: Need to be told which directory to target!"
exit 1
fi
WORK=$2
if [ "$2"x = "firmware"x ] ; then
if [ "$3"x = ""x ] ; then
echo "$0: Need to be told which architecture to target!"
exit 1
fi
INARCH=$3
if [ "$4"x = "firmware"x ] ; then
FIRMWARE="-DFIRMWARE"
IN_FW=1
fi
cd $1
for DIR in *; do
if [ -d $DIR ] && [ $DIR != trace ] ; then
ARCH=$DIR
if [ "$IN_FW"x = ""x ]; then
if $(arch_has_firmware $ARCH) ; then
if $(arch_has_firmware $INARCH) ; then
FIRMWARE="-DARCH_HAS_FIRMWARE"
else
FIRMWARE=""
fi
fi
case $ARCH in
case $INARCH in
i386|amd64|arm64)
EXTRAS="-DISOHYBRID"
ARCH=$INARCH
;;
multi-arch)
EXTRAS="-DISOHYBRID -DONEonly"
ARCH="multi"
;;
*)
ARCH=$INARCH
EXTRAS=""
;;
esac
case $WORK in
*daily*)
# for daily builds, don't mention large sets - we're only
# making netinsts...
EXTRAS="$EXTRAS -DONEonly"
;;
esac
case $ARCH in
kfreebsd*)
KERNEL="kFreeBSD"
@ -50,15 +86,20 @@ for DIR in *; do
KERNEL="Linux"
;;
esac
for DIR1 in $DIR/*-* ; do
cd $WORK
for DIR1 in *-* tar; do
STYLE=$(echo $DIR1 | sed 's,^.*/,,g;s,-.*$,,g')
TYPE=$(echo $DIR1 | sed 's,^.*-,,g')
ls -l $DIR1 | grep -q update
if [ $? -eq 0 ] ; then
if [ -d $DIR1 ]; then
UPDATES=""
if (ls -l $DIR1 | grep -q update); then
UPDATES="-DUPDATES"
fi
#echo "Found directory $DIR (ARCH $ARCH, STYLE $STYLE, TYPE $TYPE)"
$CPP -DARCH=${ARCH} -DARCH${ARCH} -DKERNEL=${KERNEL} -DSTYLE${STYLE} -DSTYLE=${STYLE} -DTYPE${TYPE} $UPDATES $EXTRAS $FIRMWARE $INPUT > $DIR1/HEADER.html
done
if [ -e $DIR1/.errata.html ]; then
$CPP -DARCH=${ARCH} -DARCH${ARCH} -DKERNEL=${KERNEL} -DSTYLE${STYLE} -DSTYLE=${STYLE} -DTYPE${TYPE} $UPDATES $EXTRAS $FIRMWARE $DIR1/.errata.html >> $DIR1/HEADER.html
fi
fi
done

2
contrib/images4testing_d-i
View File

@ -1,2 +1,2 @@
DI_DIST=stretch
DI_DIST=buster
DI_CODENAME=$DI_DIST

69
contrib/iso_run
View File

@ -1,15 +1,17 @@
#!/bin/sh
#!/bin/bash
#
# Helper for image creation
#
PUBDIRJIG=$1
RSYNC_TARGET=$2
ARCH=$3
FIRMWARE=$4
LOG=~/build.stretch/log/$ARCH.iso_run
LOG=~/build.${CODENAME}/log/$ARCH.iso_run
HOSTNAME=`hostname -f`
LOCK=~/iso_run.lock
START=`date -u +%H:%M:%S`
. $TOPDIR/common.sh
# Check to see if another sync is in progress
if lockfile -! -l 43200 -r-1 "$LOCK"; then
@ -22,24 +24,6 @@ COPY_START=`date -u +%H:%M:%S`
rm -f $LOG
# Given an ISO image:
# 1. create the torrent file if desired (only for release builds)
# 2. copy all of them into place
process_iso() {
FILE=$1
OUTDIR=$2
echo $OUTDIR/$FILE >> $LOG
cp -a $FILE $OUTDIR/$FILE
if [ "$RELEASE_BUILD"x != ""x ] ; then
~/build.stretch/mktorrent $FILE >> $LOG
BTFILE=`echo $FILE.torrent | sed 's/iso-/bt-/'`
echo $OUTDIR/$BTFILE >> $LOG
cp -a $BTFILE $OUTDIR/$BTFILE
fi
}
# Poor man's rsync, but with some local optimisations
copy_files() {
SRC=$1
@ -60,40 +44,26 @@ copy_files() {
do
for DISKTYPE in $DISKTYPES; do
for DIRTYPE in $DIRTYPES; do
if [ -d $ARCH/$DIRTYPE-$DISKTYPE ]; then
mkdir -p -m775 $TARGET/$ARCH.tmp/$DIRTYPE-$DISKTYPE
fi
done
done
# Copy the specific files across that we want in the
# publishing dir
cd $ARCH
find . -name '*.jigdo' -o -name '*.template' \
-o -name '*.iso' \
-o -name '*.torrent' \
-o -name '*.list.gz' -o -name '*SUMS*' | \
xargs tar cf - | (cd $TARGET/$ARCH.tmp/ && tar xvf -) >> $LOG
# Only make torrent files and dirs for release builds
rm -rf bt-*
for DISKTYPE in $DISKTYPES; do
if [ "$RELEASE_BUILD"x != ""x ] ; then
mkdir bt-$DISKTYPE
fi
for FILE in iso-$DISKTYPE/*.iso; do
if [ -e $FILE ] ; then
process_iso $FILE $TARGET/$ARCH.tmp
fi
done
if [ "$RELEASE_BUILD"x != ""x ] ; then
for FILE in $TARGET/$ARCH.tmp/iso-$DISKTYPE/*SUMS*; do
if [ -e $FILE ] ; then
cp -al $FILE $TARGET/$ARCH.tmp/bt-$DISKTYPE/
fi
done
fi
done
if [ "$RELEASE_BUILD"x = ""x ] ; then
# Only populate the HEADER.html file for regular weekly
# builds; we don't want it for releases
HUM_DATE=`date -u`
sed "s/ARCH/$ARCH/g;s/DATE/$HUM_DATE/g" ~/build.stretch/weekly.html \
sed "s/ARCH/$ARCH/g;s/DATE/$HUM_DATE/g" ~/build.${CODENAME}/weekly.html \
> $TARGET/$ARCH.tmp/HEADER.html
fi
@ -129,6 +99,23 @@ rm -rf $RSYNC_TARGET/$ARCH.old &
mkdir -p $RSYNC_TARGET/trace
date -u > $RSYNC_TARGET/trace/cdimage.debian.org
# Generate the HEADER.html files in the subdirectories
~/build.${CODENAME}/generate_headers ~/build.${CODENAME}/HEADER.html.in $RSYNC_TARGET/$ARCH $ARCH $FIRMWARE
END=`date -u +%H:%M:%S`
echo "$ARCH synced across to $RSYNC_TARGET; started at $START, copying started at $COPY_START, ended at $END"
# If we're on casulana, push things to pettersson here
if [ $(hostname) = casulana ]; then
START=`date -u +%H:%M:%S`
PETOUT=${RSYNC_TARGET##${OUT_BASE}/}
echo "$START: Starting $ARCH sync from $RSYNC_TARGET/$ARCH to pettersson" >> $LOG
(cd $RSYNC_TARGET/$ARCH && find . -name '*.iso' > .iso-wanted)
rsync_to_pettersson $RSYNC_TARGET/$ARCH/ ${PETOUT}/$ARCH/ '--exclude *.iso'
rsync_to_pettersson $RSYNC_TARGET/trace/ ${PETOUT}/trace/
publish_on_pettersson ${PETOUT}/$ARCH ${PETOUT}/trace
END=`date -u +%H:%M:%S`
echo "$RSYNC_TARGET/$ARCH synced across to pettersson; started at $START, ended at $END"
fi
echo "$END: Finished" >> $LOG

2
contrib/merge-sums
View File

@ -1,4 +1,4 @@
#!/bin/sh
#!/bin/bash
export LANG=C
export LC_ALL=C

2
contrib/mklist
View File

@ -1,4 +1,4 @@
#!/bin/sh
#!/bin/bash
export LC_ALL=C

28
contrib/mktorrent
View File

@ -1,27 +1,45 @@
#!/bin/sh
#!/bin/bash
HTTPBASE="http://cdimage.debian.org/cdimage"
HTTPBASE="https://cdimage.debian.org/cdimage"
NFHTTPBASE="https://cdimage.debian.org/cdimage/unofficial/non-free/cd-including-firmware/"
MKTORRENT=btmakemetafile.bittornado
for FILE in $@; do
MKTORRENT=btmakemetafile.bittornado
ISODIR=`dirname $FILE`
BTDIR=`echo $ISODIR | sed 's/iso-/bt-/;s/usb-/bt-/'`
if [ ! -d $BTDIR ] ; then
mkdir -p $BTDIR
fi
# Add 2 http seeds: the release path *and* the archive path -
# hopefully clients will use whichever is available!
# Add multiple http seeds: the release path *and* the archive path
# for both the free images and the non-free images - hopefully
# clients will use whichever is available!
case ${FILE} in
*live+nonfree*)
VER=$(echo ${FILE} | sed 's,^.*/debian-\(live-\)*,,;s,-.*$,,')"-live+nonfree"
FREE=0
;;
*live*)
VER=$(echo ${FILE} | sed 's,^.*/debian-\(live-\)*,,;s,-.*$,,')"-live"
FREE=1
;;
*firmware*)
VER=$(echo ${FILE} | sed 's,^.*/firmware-,,;s,-.*$,,')"+nonfree"
FREE=0
;;
*)
VER=$(echo ${FILE} | sed 's,^.*/debian-*,,;s,-.*$,,')
FREE=1
;;
esac
if [ $FREE -eq 1 ]; then
HTTP1="${HTTPBASE}/release/${VER}/${FILE}"
HTTP2="${HTTPBASE}/archive/${VER}/${FILE}"
else
HTTP1="${NFHTTPBASE}/${VER}/${FILE}"
HTTP2="${NFHTTPBASE}/archive/${VER}/${FILE}"
fi
$MKTORRENT http://bttracker.debian.org:6969/announce \
--comment '"Debian CD from cdimage.debian.org"' \
--httpseeds ${HTTP1}\|${HTTP2} \

2
contrib/report_build_error
View File

@ -1,4 +1,4 @@
#!/bin/sh
#!/bin/bash
export TOPDIR=$(dirname $0)

20
contrib/settings.sh
View File

@ -1,24 +1,26 @@
export TRACE=/srv/cdbuilder.debian.org/src/ftp/debian/project/trace/pettersson.debian.org
export ARCH_DI_DIR=/srv/cdbuilder.debian.org/src/deb-cd/d-i
export HOSTNAME=`hostname -f`
export CODENAME=buster
export OUT_BASE=~/publish
export CD_I_F=${OUT_BASE}/unofficial/non-free/images-including-firmware
export TRACE=/srv/cdbuilder.debian.org/src/ftp/debian/project/trace/$(hostname).debian.org
export ARCH_DI_DIR=/srv/cdbuilder.debian.org/src/deb-cd/d-i
export PUBDIR=/srv/cdbuilder.debian.org/dst/deb-cd
export MIRROR=/srv/cdbuilder.debian.org/src/ftp/debian
export BASEDIR=~/build.stretch/debian-cd
export MKISOFS=~/build.stretch/mkisofs/usr/bin/mkisofs
export BASEDIR=~/build.${CODENAME}/debian-cd
export MKISOFS=~/build.${CODENAME}/mkisofs/usr/bin/mkisofs
export EXTRACTED_SOURCES=${OUT_BASE}/cd-sources/
export LIVE_OUT=${OUT_BASE}/.live
if [ "$DATE"x = ""x ] ; then
export DATE=`date -u +%Y%m%d`
fi
export EXTRACTED_SOURCES=/mnt/nfs-cdimage/cd-sources
if [ "$ARCHES"x = ""x ] ; then
ARCHES="i386 amd64 armhf multi-arch armel source arm64 ppc64el mips mipsel mips64el s390x"
ARCHES="amd64 i386 multi-arch arm64 armhf armel source ppc64el mips mipsel mips64el s390x"
fi
if [ "$ARCHES_FIRMWARE"x = ""x ] ; then
ARCHES_FIRMWARE="amd64 i386 multi-arch"
# ARCHES_FIRMWARE="amd64"
fi

2
contrib/sign-images
View File

@ -1,4 +1,4 @@
#!/bin/sh
#!/bin/bash
#
# Helper for signing images
#

31
contrib/testingcds
View File

@ -13,13 +13,13 @@
#set -x
if [ "$CONF"x = ""x ] ; then
CONF=~/build.stretch/CONF.sh
CONF=~/build.${CODENAME}/CONF.sh
fi
export CF=$CONF
if [ "$DCD_DIR"x = ""x ] ; then
DCD_DIR=~/build.stretch/debian-cd
DCD_DIR=~/build.${CODENAME}/debian-cd
fi
export CF=$CONF
now () {
date -u +%F:%H:%M:%S
@ -60,7 +60,7 @@ export LOGFILE="$LOG/$RUN"
case "$INSTALLER_CD" in
1)
export DISKTYPE=BC;;
2|7|8|C|F|G)
2|7|8|C|F|G|H)
export DISKTYPE=NETINST;;
3|6|I|J)
export OUT_TYPE=dvd
@ -73,7 +73,7 @@ case "$INSTALLER_CD" in
export DISKTYPE=lxde-CD;;
B)
export DISKTYPE=xfce+lxde-CD;;
9)
9|K|L)
export OUT_TYPE=bd
export DISKTYPE=BD;;
D)
@ -81,9 +81,6 @@ case "$INSTALLER_CD" in
export DISKTYPE=DLBD;;
E)
export DISKTYPE=gnome-CD;;
H)
export OUT_TYPE=bd
export DISKTYPE=BD;;
*) export DISKTYPE=CD;;
esac
@ -106,12 +103,10 @@ if [ "$OUT"x = ""x ] ; then
fi
if [ "$BUILDNAME"x = ""x ] ; then
TRACEFILE="$OUT_TYPE-trace"
echo " Making $ARCH ${DISKTYPE} in $OUT"
else
BUILDNAME="$OUT_TYPE"
fi
TRACEFILE="$BUILDNAME-trace"
echo " Making $ARCH ${DISKTYPE} (${BUILDNAME}) in $OUT"
fi
if [ "$INSTALLER_CD" = "1" ] ; then
export OFFICIAL="Official BusinessCard Snapshot"
@ -138,13 +133,17 @@ elif [ "$INSTALLER_CD" = "E" ]; then
elif [ "$INSTALLER_CD" = "F" ]; then
export OFFICIAL="Official Mac Netinst"
elif [ "$INSTALLER_CD" = "G" ]; then
export OFFICIAL="Official Debian-Edu Multi-Arch Netinst"
export OFFICIAL="Official Debian-Edu Netinst"
elif [ "$INSTALLER_CD" = "H" ]; then
export OFFICIAL="Official Debian-Edu Multi-Arch Image"
export OFFICIAL="Unofficial Debian-Edu Netinst including firmware"
elif [ "$INSTALLER_CD" = "I" ]; then
export OFFICIAL="Unofficial DVD including firmware"
elif [ "$INSTALLER_CD" = "J" ]; then
export OFFICIAL="Unofficial Multi-Arch DVD including firmware"
elif [ "$INSTALLER_CD" = "K" ]; then
export OFFICIAL="Debian-Edu Full USB image"
elif [ "$INSTALLER_CD" = "L" ]; then
export OFFICIAL="Unofficial Debian-Edu Full USB image including firmware"
fi
rm -rf "$OUT"
@ -273,7 +272,7 @@ if ./build.sh "$ARCH" >>"$LOGFILE" 2>&1 ; then
for i in $MYEMAIL
do
grep -e "extents written" -e "Written to medi." "$LOGFILE"|mail -s "$RELEASE_BUILD testingcds $RUN ($OUTARCH $DISKTYPE) went ok" "$i"
grep -e "extents written" -e "Written to medi." "$LOGFILE"|mail -s "$RELEASE_BUILD testingcds $RUN ($OUTARCH $BUILDNAME) went ok" "$i"
done
overview "$RUN" success
@ -287,7 +286,7 @@ else
for i in $MYEMAIL
do
(echo Error when trying to create cds for $RUN ; cat $LOGFILE) | \
mail -s "$RELEASE_BUILD testingcds $RUN has failed; log included" "$i"
mail -s "$RELEASE_BUILD testingcds $RUN ($OUTARCH $BUILDNAME) has failed; log included" "$i"
done
overview "$RUN" failed
END=`now`

1
debian/changelog vendored
View File

@ -3,6 +3,7 @@ debian-cd (3.1.23) UNRELEASED; urgency=medium
[ Holger Levsen ]
* data/buster: update Debian Edu installer logo and syslinux splash image
from src:debian-edu-artwork 2.10.5-1.
* contrib: updated from setup.git/buster/.
[ Wolfgang Schweer ]
* tools/start_new_disc: set adjusted cd_type for incomplete BD image for