Peppermint
/
debian-cd-clone
mirror of https://salsa.debian.org/images-team/debian-cd.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

contrib: updated from setup.git/buster/. 

Signed-off-by: Holger Levsen <holger@layer-acht.org>
This commit is contained in:
Holger Levsen 2019-02-19 19:05:47 +01:00
parent c149517d7b
commit ee7441d449
21 changed files with 756 additions and 429 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
contrib/CONF.sh
View File

@ -47,8 +47,8 @@ unset BASE_EXCLUDE || true
# Where I am (hoping I'm in the debian-cd dir) # Where I am (hoping I'm in the debian-cd dir)
export BASEDIR=`pwd` export BASEDIR=`pwd`
# Building stretch cd set ... # Building buster cd set ...
export CODENAME=stretch export CODENAME=buster
if [ ! "$DI_CODENAME" ] if [ ! "$DI_CODENAME" ]
then then
@ -70,7 +70,7 @@ export OFFICIAL="Official Snapshot"
# ... for arch # ... for arch
if [ ! "$ARCH" ] if [ ! "$ARCH" ]
then then
export ARCH=`dpkg --print-installation-architecture` export ARCH=`dpkg --print-architecture`
fi fi
# IMPORTANT : The 4 following paths must be on the same partition/device. # IMPORTANT : The 4 following paths must be on the same partition/device.
@ -150,7 +150,11 @@ export CONTRIB=1
#export MKISOFS_OPTS="-joliet-long -jigdo-template-compress bzip2 -r -checksum_algorithm_iso md5,sha1,sha256,sha512" #-checksum_algorithm_iso md5,sha1" #export MKISOFS_OPTS="-joliet-long -jigdo-template-compress bzip2 -r -checksum_algorithm_iso md5,sha1,sha256,sha512" #-checksum_algorithm_iso md5,sha1"
# export MKISOFS_OPTS="-r" #For normal users # export MKISOFS_OPTS="-r" #For normal users
# export MKISOFS_OPTS="-r -F ." #For symlink farmers # export MKISOFS_OPTS="-r -F ." #For symlink farmers
export MKISOFS="/home/93sam/xorriso" if [ $(which xorriso)""x = "/usr/bin/xorriso"x ]; then
export MKISOFS="xorriso"
else
export MKISOFS="/home/93sam/xorriso"
fi
export MKISOFS_OPTS="-as mkisofs -r -checksum_algorithm_iso md5,sha1,sha256,sha512" export MKISOFS_OPTS="-as mkisofs -r -checksum_algorithm_iso md5,sha1,sha256,sha512"
# Override for i386,amd64,multi to use xorriso. # Override for i386,amd64,multi to use xorriso.

109
contrib/HEADER.html.in
View File

@ -87,12 +87,12 @@ cpp.
writeable dual-layer Blu-Ray media writeable dual-layer Blu-Ray media
# endif # endif
at a minimum, but may be written to larger media if needed. at a minimum, but may be written to larger media if needed.</p>
# ifdef ISOHYBRID # ifdef ISOHYBRID
For <strong>extra</strong> convenience, these images may also be <p>For extra convenience, <strong>these images may also be written
written directly to a USB stick. So long as your computer will directly to a USB stick</strong>. So long as your computer will
boot directly from that USB stick, it should start the Debian boot directly from that USB stick, it should start the Debian
installer that way. installer that way.
@ -116,24 +116,30 @@ cpp.
complete set. Most typical users only need a small subset of complete set. Most typical users only need a small subset of
those software packages.</p> those software packages.</p>
<p>Initially, you will only need to download and use # if !defined(TYPEcd)
the <strong>first</strong> image of a set (labelled as
<code>debian-<i>something</i>-1</code> to be able to start the <p>Initially, you will only need to download and use
Debian installer and set up Debian on your computer. If there are the <strong>first</strong> image of a set (labelled as
more images available here <code>debian-<i>something</i>-1</code> to be able to start the
(labelled <code>debian-<i>something</i>-2</code>, Debian installer and set up Debian on your computer. If there are
<code>debian-<i>something</i>-3</code>, etc.), they contain the more images available here
extra packages that can be installed on a Debian system (as (labelled <code>debian-<i>something</i>-2</code>,
mentioned previously). They will <strong>not</strong> be bootable <code>debian-<i>something</i>-3</code>, etc.), they contain the
and are entirely optional. If you have a fast Internet extra packages that can be installed on a Debian system (as
connection, you're most likely better off installing any desired mentioned previously). They will <strong>not</strong> be
extra packages directly from the Debian mirrors on the Internet bootable and are entirely optional. If you have a fast Internet
instead of by using these extra images.</p> connection, you're most likely better off installing any
desired extra packages directly from the Debian mirrors on the
Internet instead of by using these extra images.</p>
# endif
# endif # endif
# if defined(TYPEcd) && !defined(ARCHsource) && !defined(STYLElist) # if defined(TYPEcd) && !defined(ARCHsource) && !defined(STYLElist)
<h2>What is a netinst image?</h2>
<p>The <code>netinst</code> CD here is a small CD image that <p>The <code>netinst</code> CD here is a small CD image that
contains just the core Debian installer code and a small core set contains just the core Debian installer code and a small core set
of text-mode programs (known as "standard" in Debian). To install of text-mode programs (known as "standard" in Debian). To install
@ -142,6 +148,8 @@ cpp.
# if defined(ARCHi386) # if defined(ARCHi386)
<h2>What is an edu netinst image?</h2>
<p>The <code>edu netinst</code> CD here is a special version of <p>The <code>edu netinst</code> CD here is a special version of
the netinst CD image that is targeted specifically at 32-bit the netinst CD image that is targeted specifically at 32-bit
Intel machines. It provides a menu to install the Debian Edu Pure Intel machines. It provides a menu to install the Debian Edu Pure
@ -152,6 +160,8 @@ cpp.
# if defined(ARCHamd64) # if defined(ARCHamd64)
<h2>What is an edu netinst image?</h2>
<p>The <code>edu netinst</code> CD here is a special version of <p>The <code>edu netinst</code> CD here is a special version of
the netinst CD image that is targeted specifically at 64-bit the netinst CD image that is targeted specifically at 64-bit
Intel machines. It provides a menu to install the Debian Edu Pure Intel machines. It provides a menu to install the Debian Edu Pure
@ -162,21 +172,60 @@ cpp.
# if defined(ARCHi386) # if defined(ARCHi386)
<h2>What is a mac netinst image?</h2>
<p>The <code>mac netinst</code> CD here is a special version of <p>The <code>mac netinst</code> CD here is a special version of
the netinst CD image that is targeted specifically at 32-bit the netinst CD image that is targeted specifically at older
Intel Macintosh machines. It will likely work on most other 32-bit Intel Macintosh machines. It will likely work on most
i386 machines too, but it does not contain UEFI boot files that other i386 machines too, but it does not contain UEFI boot
some people need.</p> files that some people
need. See <a href="https://wiki.debian.org/MacMiniIntel#Macmini_1.2C1">
the Debian Wiki</a> for more information.</p>
# endif # endif
# if defined(ARCHamd64) # if defined(ARCHamd64)
<h2>What is a mac netinst image?</h2>
<p>The <code>mac netinst</code> CD here is a special version of <p>The <code>mac netinst</code> CD here is a special version of
the netinst CD image that is targeted specifically at older 64-bit the netinst CD image that is targeted specifically at older
Intel Macintosh machines. It will likely work on most other 64-bit Intel Macintosh machines. It will likely work on most
amd64 machines too, but it does not contain UEFI boot files that other amd64 machines too, but it does not contain UEFI boot
some people need.</p> files that some people
need. See <a href="https://wiki.debian.org/MacMiniIntel#Macmini_2.2C1">
the Debian Wiki</a> for more information.</p>
# endif
# endif
# if defined(TYPEbd) && !defined(ARCHsource) && !defined(STYLElist)
<p>The <code>BD</code> image here is a special large image that
contains the core Debian installer code and a specific set of
programs. Depending on the purpose you'll also need an Internet
connection.</p>
# if defined(ARCHi386)
<p>The <code>edu-usb</code> BD here is a special version of
the BD image that is targeted specifically at 32-bit Intel machines.
It provides a menu to install the Debian Edu Pure Blend without the
need of an Internet connection. See the
<a href="https://wiki.debian.org/DebianEdu/">Debian Wiki</a> for
more information.</p>
# endif
# if defined(ARCHamd64)
<p>The <code>edu-usb</code> BD here is a special version of
the BD image that is targeted specifically at 64-bit Intel machines.
It provides a menu to install the Debian Edu Pure Blend without the
need of an Internet connection. See the
<a href="https://wiki.debian.org/DebianEdu/">Debian Wiki</a> for
more information.</p>
# endif # endif
@ -210,6 +259,18 @@ cpp.
#endif #endif
#if !defined(ONEonly) && !defined(FIRMWARE) && (defined(STYLEbt) || defined(STYLEiso))
<h2>Only the first few images are available! Where are the rest?</h2>
<p>We don't store/serve the full set of ISO images for all
architectures, to reduce the amount of space taken up on the
mirrors. You can
<a href="https://www.debian.org/CD/faq/#why-jigdo">use the jigdo
tool</a> to recreate the missing ISO images instead.</p>
#endif
#ifdef FIRMWARE #ifdef FIRMWARE
<h2>Non-free Firmware</h2> <h2>Non-free Firmware</h2>

54
contrib/common.sh
View File

@ -120,14 +120,49 @@ catch_parallel_builds () {
echo "$arch build started at $arch_start, ended at $arch_end (took $arch_time), error(s) $arch_error" echo "$arch build started at $arch_start, ended at $arch_end (took $arch_time), error(s) $arch_error"
} }
# Slightly complicated setup
# iso-* dirs have checksums for ISO files only
# bt-* dirs have checksums for ISO and torrrent files only
# jigdo-* dirs have checksums for ISO and jigdo files only
#
# Uses the imagesums tool from debian-cd to grab pre-calculated ISO
# checksums from the jigdo files where possible, to save a lot of time
generate_checksums_for_arch () { generate_checksums_for_arch () {
ARCH=$1 ARCH=$1
JIGDO_DIR=$2 JIGDO_DIR=$2
ISO_DIR=$(echo $JIGDO_DIR | sed 's,jigdo-,iso-,g') ISO_DIR=$(echo $JIGDO_DIR | sed 's,jigdo-,iso-,g')
BT_DIR=$(echo $JIGDO_DIR | sed 's,jigdo-,bt-,g')
echo "$ARCH: Generating checksum files for the builds in $JIGDO_DIR" # Do the torrents first, if they exist
$TOPDIR/debian-cd/tools/imagesums $JIGDO_DIR $EXTENSION > /dev/null if [ -e $BT_DIR ]; then
cp $JIGDO_DIR/*SUMS*${EXTENSION} $ISO_DIR $TOPDIR/debian-cd/tools/imagesums $BT_DIR $EXTENSION > /dev/null
fi
# Now do the jigdos, if they exist
if [ -e $JIGDO_DIR ]; then
$TOPDIR/debian-cd/tools/imagesums $JIGDO_DIR $EXTENSION > /dev/null
# And grep out the .iso checksums from there to the iso directory
for file in $JIGDO_DIR/*SUMS*${EXTENSION}; do
out=$ISO_DIR/$(basename $file)
grep \\.iso $file > $out
done
if [ -e $BT_DIR ]; then
# Ditto for the bt directory
for file in $JIGDO_DIR/*SUMS*${EXTENSION}; do
out=$BT_DIR/$(basename $file)
grep \\.iso $file >> $out
done
fi
else
# No jigdos, so do the ISOs by hand
$TOPDIR/debian-cd/tools/imagesums $ISO_DIR $EXTENSION > /dev/null
if [ -e $BT_DIR ]; then
for file in $ISO_DIR/*SUMS*${EXTENSION}; do
out=$BT_DIR/$(basename $file)
grep \\.iso $file >> $out
done
fi
fi
} }
catch_live_builds () { catch_live_builds () {
@ -169,3 +204,16 @@ get_archive_serial () {
echo 'unknown' echo 'unknown'
fi fi
} }
rsync_to_pettersson () {
LOCAL=$1
REMOTE=$2
OPTIONS="$3"
rsync -az --delete $OPTIONS $LOCAL sync-to-pettersson:$REMOTE
}
publish_on_pettersson () {
TARGETS="$@"
echo "$TARGETS" | ssh publish-on-pettersson ./bin/receive_from_casulana
}

197
contrib/cronjob.daily
View File

@ -10,8 +10,11 @@ export TOPDIR
BUILDLOCK=$HOME/.debian-cd.lock BUILDLOCK=$HOME/.debian-cd.lock
export PUBDIRJIG=$PUBDIR/daily-builds export PUBDIRJIG=$PUBDIR/daily-builds
if [ "$BUILDNUM"x = ""x ]; then
BUILDNUM=1
fi
export DATE_BUILD="$DATE-$BUILDNUM" export DATE_BUILD="$DATE-$BUILDNUM"
export TESTING_SUITE=stretch export TESTING_SUITE=${CODENAME}
BUILDS_RUNNING="" BUILDS_RUNNING=""
export NI_WANTED=1 export NI_WANTED=1
@ -23,20 +26,11 @@ if [ "$RELEASE_BUILD"x = ""x ] ; then
export OUT_FW_TST=${PUBDIRJIG}-firmware/${TESTING_SUITE}_d-i/${DATE_BUILD} export OUT_FW_TST=${PUBDIRJIG}-firmware/${TESTING_SUITE}_d-i/${DATE_BUILD}
export OUT_FREE_SID=${PUBDIRJIG}/sid_d-i/${DATE_BUILD} export OUT_FREE_SID=${PUBDIRJIG}/sid_d-i/${DATE_BUILD}
export OUT_FW_SID=${PUBDIRJIG}-firmware/sid_d-i/${DATE_BUILD} export OUT_FW_SID=${PUBDIRJIG}-firmware/sid_d-i/${DATE_BUILD}
export RSYNC_TARGET_FREE=/mnt/nfs-cdimage/daily-builds export RSYNC_TARGET_FREE=${OUT_BASE}/daily-builds
export RSYNC_TARGET_FIRMWARE=/mnt/nfs-cdimage/unofficial/non-free/cd-including-firmware/daily-builds export RSYNC_TARGET_FIRMWARE=${OUT_BASE}/unofficial/non-free/cd-including-firmware/daily-builds
else else
export SID_WANTED=0 echo "Don't support release builds of dailies! Use cronjob.weekly instead"
export TESTING_WANTED=1 exit 1
export OUT_FREE_TST=${PUBDIRJIG}-$RELEASE_BUILD
export OUT_FW_TST=${PUBDIRJIG}-$RELEASE_BUILD-firmware
export OUT_FREE_SID=""
export OUT_FW_SID=""
export RSYNC_TARGET_FREE=/mnt/nfs-cdimage/.${RELEASE_BUILD}
export RSYNC_TARGET_FIRMWARE=/mnt/nfs-cdimage/unofficial/non-free/cd-including-firmware/.${RELEASE_BUILD}
export CONF=~/build.stretch/CONF.sh.${RELEASE_BUILD}
export RELEASE_BUILD=$RELEASE_BUILD
export EXTENSION=".small"
fi fi
. $TOPDIR/common.sh . $TOPDIR/common.sh
@ -47,7 +41,7 @@ reboot_lock
finalise_arch_dir () { finalise_arch_dir () {
ARCH=$1 ARCH=$1
for OUT_DIR in $OUT_FREE_TST $OUT_FW_TST $OUT_FREE_SID $OUT_FW_SID; do for OUT_DIR in $OUT_FREE_SID $OUT_FW_SID; do # $OUT_FREE_TST $OUT_FW_TST
ARCH_DIR=$OUT_DIR/$ARCH ARCH_DIR=$OUT_DIR/$ARCH
# Only keep things / do any work here if there are any files in the ARCH_DIR # Only keep things / do any work here if there are any files in the ARCH_DIR
NUM_FILES=`find $ARCH_DIR -type f | wc -l` NUM_FILES=`find $ARCH_DIR -type f | wc -l`
@ -57,31 +51,25 @@ finalise_arch_dir () {
ARCH_ISO_DIR=${ARCH_DIR}/iso-cd ARCH_ISO_DIR=${ARCH_DIR}/iso-cd
ARCH_JIGDO_DIR=${ARCH_DIR}/jigdo-cd ARCH_JIGDO_DIR=${ARCH_DIR}/jigdo-cd
if [ "$RELEASE_BUILD"x = ""x ] ; then DATESTRING=`date -u`
DATESTRING=`date -u` case $ARCH_ISO_DIR in
case $ARCH_ISO_DIR in *firmware*)
*firmware*) FIRMWARE="firmware"
INFILE=$TOPDIR/daily-firmware.html;; INFILE=$TOPDIR/daily-firmware.html;;
*) *)
INFILE=$TOPDIR/daily.html;; FIRMWARE=""
esac INFILE=$TOPDIR/daily.html;;
sed "s/ARCH/$ARCH/g;s/DATE/$DATESTRING/g;s/BUILDNUM/$BUILDNUM/g;s/INST_VER/$INST_VER/g" \ esac
$INFILE > $ARCH_DIR/HEADER.html sed "s/ARCH/$ARCH/g;s/DATE/$DATESTRING/g;s/BUILDNUM/$BUILDNUM/g;s/CODENAME/$CODENAME/g" \
fi $INFILE > $ARCH_DIR/HEADER.html
generate_checksums_for_arch $ARCH $ARCH_JIGDO_DIR generate_checksums_for_arch $ARCH $ARCH_JIGDO_DIR
~/build.${CODENAME}/generate_headers ~/build.${CODENAME}/HEADER.html.in $ARCH_DIR $ARCH $FIRMWARE
cd $ARCH_DIR cd $ARCH_DIR
# Only make torrent files for release builds echo "Signing checksums files using the automatic key"
if [ "$RELEASE_BUILD"x != ""x ] ; then ~/build.${CODENAME}/sign-images $OUT_DIR $ARCH
~/build.stretch/mktorrent iso-cd/*iso ~/build.${CODENAME}/mklist iso-cd/*iso
cp iso-cd/*SUMS* bt-cd
fi
if [ "$RELEASE_BUILD"x = ""x ] ; then
echo "Signing checksums files using the automatic key"
~/build.stretch/sign-images $OUT_DIR $ARCH
fi
~/build.stretch/mklist iso-cd/*iso
cd $TOPDIR cd $TOPDIR
fi fi
done done
@ -113,7 +101,7 @@ if lockfile -r0 $BUILDLOCK ; then
arch_error="" arch_error=""
arch_start=`now` arch_start=`now`
for OUT_DIR in $OUT_FREE_TST $OUT_FW_TST $OUT_FREE_SID $OUT_FW_SID; do for OUT_DIR in $OUT_FREE_SID $OUT_FW_SID; do # $OUT_FREE_TST $OUT_FW_TST
mkdir -p ${OUT_DIR}/${arch}/iso-cd ${OUT_DIR}/${arch}/jigdo-cd mkdir -p ${OUT_DIR}/${arch}/iso-cd ${OUT_DIR}/${arch}/jigdo-cd
done done
@ -258,6 +246,82 @@ if lockfile -r0 $BUILDLOCK ; then
fi fi
fi fi
# Special debian-edu builds, only minor tweaks. #846006
if [ "$NOEDU"x = ""x ] ; then
if [ "$arch" = "i386" ] || [ "$arch" = "amd64" ] ; then
if [ "$SID_WANTED" = "1" ] ; then
if [ "$NI_WANTED" = "1" ] ; then
build_started SIDEDUNI
OMIT_RELEASE_NOTES=1 OMIT_MANUAL=1 NORECOMMENDS=1 \
NOSUGGESTS=1 COMPLETE=0 INSTALLER_CD=G \
CDNAME=debian-edu \
DI=sid DI_WWW_HOME=default \
DEBIAN_EDU=1 \
KERNEL_PARAMS="modules=debian-edu-install-udeb" \
DESKTOP=xfce VARIANTS=xen \
EXCLUDE=debian-edu-exclude \
UDEB_EXCLUDE="$BASEDIR"/data/$CODENAME/udeb_exclude \
TASK=Debian-edu-netinst \
MAXISOS=1 MAXJIGDOS=1 \
DI=${DI} DI_DIST=${DI} \
DI_DIR="$ARCH_DI_DIR" \
./testingcds "$arch" &
build_started SIDEDUNIFIRMWARE
OMIT_RELEASE_NOTES=1 OMIT_MANUAL=1 NORECOMMENDS=1 \
NOSUGGESTS=1 COMPLETE=0 INSTALLER_CD=H \
FORCE_FIRMWARE=1 \
CDNAME=firmware-edu \
DI=sid DI_WWW_HOME=default \
DEBIAN_EDU=1 \
KERNEL_PARAMS="modules=debian-edu-install-udeb" \
DESKTOP=xfce VARIANTS=xen \
EXCLUDE=debian-edu-exclude \
UDEB_EXCLUDE="$BASEDIR"/data/$CODENAME/udeb_exclude \
TASK=Debian-edu-netinst \
MAXISOS=1 MAXJIGDOS=1 \
DI=${DI} DI_DIST=${DI} \
DI_DIR="$ARCH_DI_DIR" \
./testingcds "$arch" &
fi
fi
if [ "$TESTING_WANTED" = "1" ] ; then
if [ "$NI_WANTED" = "1" ] ; then
build_started TESTINGEDUNI
OMIT_RELEASE_NOTES=1 OMIT_MANUAL=1 NORECOMMENDS=1 \
NOSUGGESTS=1 COMPLETE=0 INSTALLER_CD=G \
CDNAME=debian-edu \
DEBIAN_EDU=1 \
KERNEL_PARAMS="modules=debian-edu-install-udeb" \
DESKTOP=xfce VARIANTS=xen \
EXCLUDE=debian-edu-exclude \
UDEB_EXCLUDE="$BASEDIR"/data/$CODENAME/udeb_exclude \
TASK=Debian-edu-netinst \
MAXISOS=1 MAXJIGDOS=1 \
DI=${TESTING_SUITE} DI_DIST="$DI_DIST" \
./testingcds "$arch" &
build_started TESTINGEDUNIFIRMWARE
OMIT_RELEASE_NOTES=1 OMIT_MANUAL=1 NORECOMMENDS=1 \
NOSUGGESTS=1 COMPLETE=0 INSTALLER_CD=H \
FORCE_FIRMWARE=1 \
CDNAME=firmware-edu \
DEBIAN_EDU=1 \
KERNEL_PARAMS="modules=debian-edu-install-udeb" \
DESKTOP=xfce VARIANTS=xen \
EXCLUDE=debian-edu-exclude \
UDEB_EXCLUDE="$BASEDIR"/data/$CODENAME/udeb_exclude \
TASK=Debian-edu-netinst \
MAXISOS=1 MAXJIGDOS=1 \
DI=${TESTING_SUITE} DI_DIST="$DI_DIST" \
./testingcds "$arch" &
fi
fi
fi
fi
catch_parallel_builds catch_parallel_builds
rm -rf $PUBDIRJIG/$arch rm -rf $PUBDIRJIG/$arch
if [ "$arch_error"x = "none"x ] ; then if [ "$arch_error"x = "none"x ] ; then
@ -268,38 +332,47 @@ if lockfile -r0 $BUILDLOCK ; then
done done
if [ "$NOSYNC"x = ""x ] ; then if [ "$NOSYNC"x = ""x ] ; then
echo "Clean up old builds on build machine:"
for DIR in $PUBDIRJIG/* $PUBDIRJIG-firmware/* ; do
cd $DIR
for THIS in 20* ; do
if [ $THIS != $DATE-$BUILDNUM ] ; then
echo " Deleting $DIR/$THIS"
rm -rf $DIR/$THIS
fi
done
done
if [ "$RELEASE_BUILD"x = ""x ] ; then echo "Sync normal builds across to output dir: $RSYNC_TARGET_FREE"
echo "Clean up old builds on build machine:" rsync -rHlt $PUBDIRJIG/ $RSYNC_TARGET_FREE/
for DIR in $PUBDIRJIG/* $PUBDIRJIG-firmware/* ; do echo "Sync firmware builds across to output dir: $RSYNC_TARGET_FIRMWARE"
cd $DIR rsync -rHlt $PUBDIRJIG-firmware/ $RSYNC_TARGET_FIRMWARE/
for THIS in 20* ; do
if [ $THIS != $DATE-$BUILDNUM ] ; then
echo " Deleting $DIR/$THIS"
rm -rf $DIR/$THIS
fi
done
done
echo "Sync normal builds across to output dir: $RSYNC_TARGET_FREE" echo "Remove extra builds:"
rsync -rHltv $PUBDIRJIG/ $RSYNC_TARGET_FREE/ ~/bin/remove_extra_dailies
echo "Remove extra builds for normal dailies:"
~/bin/remove_extra_dailies $RSYNC_TARGET_FREE "$ARCHES"
echo "Sync firmware builds across to output dir: $RSYNC_TARGET_FIRMWARE" echo "$DATE-$BUILDNUM" > $RSYNC_TARGET_FREE/daily-trace
rsync -rHltv $PUBDIRJIG-firmware/ $RSYNC_TARGET_FIRMWARE/
echo "Remove extra builds for firmware dailies:"
~/bin/remove_extra_dailies $RSYNC_TARGET_FIRMWARE "$ARCHES_FIRMWARE"
echo "$DATE-$BUILDNUM" > $RSYNC_TARGET_FREE/daily-trace if [ $(hostname) = casulana ]; then
else # Release build (cd ${RSYNC_TARGET_FREE}/sid_d-i/${DATE_BUILD} && find . -name '*.iso' > .iso-wanted)
echo "Sync normal builds for $RELEASE_BUILD across to output dir: $RSYNC_TARGET_FREE" (cd ${RSYNC_TARGET_FIRMWARE}/sid_d-i/${DATE_BUILD}/ && find . -name '*.iso' > .iso-wanted)
rsync -rHltv $OUT_FREE_TST/ $RSYNC_TARGET_FREE/ rsync_to_pettersson ${RSYNC_TARGET_FREE}/sid_d-i/${DATE_BUILD}/ \
echo "Sync firmware builds for $RELEASE_BUILD across to output dir: $RSYNC_TARGET_FIRMWARE" daily-builds/sid_d-i/${DATE_BUILD}/ \
rsync -rHltv $OUT_FW_TST/ $RSYNC_TARGET_FIRMWARE/ '--exclude *.iso'
publish_on_pettersson daily-builds/sid_d-i/${DATE_BUILD}
rsync_to_pettersson ${RSYNC_TARGET_FIRMWARE}/sid_d-i/${DATE_BUILD}/ \
unofficial/non-free/cd-including-firmware/daily-builds/sid_d-i/${DATE_BUILD}/ \
'--exclude *.iso'
publish_on_pettersson unofficial/non-free/cd-including-firmware/daily-builds/sid_d-i/${DATE_BUILD}
rsync_to_pettersson ${OUT_BASE}/cd-sources/ cd-sources/
publish_on_pettersson cd-sources
fi fi
fi fi
cd $TOPDIR cd $TOPDIR
rm -f $BUILDLOCK rm -f $BUILDLOCK
fi fi

324
contrib/cronjob.weekly
View File

@ -11,10 +11,10 @@ export TOPDIR
BUILDLOCK=$HOME/.debian-cd.lock BUILDLOCK=$HOME/.debian-cd.lock
export PUBDIRJIG=$PUBDIR/weekly-builds export PUBDIRJIG=$PUBDIR/weekly-builds
export PUBDIROSTRACE=/mnt/nfs-cdimage/.live/trace-os export PUBDIROSTRACE=${OUT_BASE}/.live/trace-os
export PUBDIRLIVETRACE=/mnt/nfs-cdimage/.live/trace-live export PUBDIRLIVETRACE=${OUT_BASE}/.live/trace-live
export DATE_BUILD="$DATE-$BUILDNUM" export DATE_BUILD="$DATE-$BUILDNUM"
export CD_I_F="/mnt/nfs-cdimage/unofficial/non-free/cd-including-firmware" export CD_I_F="${OUT_BASE}/unofficial/non-free/cd-including-firmware"
. $TOPDIR/common.sh . $TOPDIR/common.sh
@ -26,18 +26,19 @@ reboot_lock
USE_DAILY_DI=Y USE_DAILY_DI=Y
if [ "$RELEASE_BUILD"x = ""x ] ; then if [ "$RELEASE_BUILD"x = ""x ] ; then
export RSYNC_TARGET=/mnt/nfs-cdimage/weekly-builds export RSYNC_TARGET=${OUT_BASE}/weekly-builds
export RSYNC_TARGET_FIRMWARE=${CD_I_F}/weekly-builds export RSYNC_TARGET_FIRMWARE=${CD_I_F}/weekly-builds
else else
export RSYNC_TARGET=/mnt/nfs-cdimage/.${RELEASE_BUILD}/debian-cd export RSYNC_TARGET=${OUT_BASE}/.${RELEASE_BUILD}/debian-cd
export RSYNC_TARGET_FIRMWARE=${CD_I_F}/.${RELEASE_BUILD}/debian-cd export RSYNC_TARGET_FIRMWARE=${CD_I_F}/.${RELEASE_BUILD}/debian-cd
export CONF=~/build.stretch/CONF.sh.${RELEASE_BUILD} export CONF=~/build.${CODENAME}/CONF.sh.${RELEASE_BUILD}
export RELEASE_BUILD=$RELEASE_BUILD export RELEASE_BUILD=$RELEASE_BUILD
# If we're doing a normal set of daily/weekly builds, leave the # If we're doing a normal set of daily/weekly builds, leave the
# checksum filenames alone. Otherwise, make life easier for people # checksum filenames alone. Otherwise, make life easier for people
# combining things later and append a suitable name as we build. # combining things later and append a suitable name as we build.
export EXTENSION=".large" export EXTENSION=".large"
USE_DAILY_DI=N USE_DAILY_DI=N
NOSNAP=1
fi fi
. images4testing_d-i . images4testing_d-i
@ -52,28 +53,33 @@ fi
export DI_DIST export DI_DIST
export DI_CODENAME export DI_CODENAME
export OUT_FREE_SID=$PUBDIRJIG export OUT_FREE_SID=${PUBDIRJIG}
export OUT_FREE_TST=$PUBDIRJIG export OUT_FREE_TST=${PUBDIRJIG}
export OUT_FW_SID=${PUBDIRJIG}-firmware export OUT_FW_SID=${PUBDIRJIG}-firmware
export OUT_FW_TST=${PUBDIRJIG}-firmware export OUT_FW_TST=${PUBDIRJIG}-firmware
BUILDS_RUNNING="" BUILDS_RUNNING=""
export NOEDU=1 #export NOEDU=1
#export NOOPENSTACK=1 #export NOOPENSTACK=1
#export NOLIVE=1 #export NOLIVE=1
rm -f $PUBDIRLIVETRACE $PUBDIROSTRACE
~/build.stretch/cronjob.weekly-live &
if lockfile -r0 $BUILDLOCK ; then if lockfile -r0 $BUILDLOCK ; then
# echo "NOT checking for git updates" # echo "NOT checking for git updates"
rm -f $PUBDIRLIVETRACE $PUBDIROSTRACE
~/build.${CODENAME}/cronjob.weekly-live &
echo "git update debian-cd" echo "git update debian-cd"
cd debian-cd && git pull ; cd .. cd debian-cd && git pull ; cd ..
mkdir -p ${PUBDIRJIG}/trace
mkdir -p $RSYNC_TARGET/trace $RSYNC_TARGET_FIRMWARE/trace
# Keep track of the serial for the archive we're building against, # Keep track of the serial for the archive we're building against,
# for later archive diffing for release announce preparation: # for later archive diffing for release announce preparation:
serial=$(get_archive_serial) serial=$(get_archive_serial)
echo "$serial" > $PUBDIRJIG/trace/archive-serial echo "$serial" > $RSYNC_TARGET/trace/archive-serial
echo "$serial" > $RSYNC_TARGET_FIRMWARE/trace/archive-serial
# Work out the default desktop, and do *not* build a CD1 for that # Work out the default desktop, and do *not* build a CD1 for that
# desktop - it'll be done in the full set anyway # desktop - it'll be done in the full set anyway
@ -85,25 +91,31 @@ if lockfile -r0 $BUILDLOCK ; then
# echo "NOT checking for popcon updates" # echo "NOT checking for popcon updates"
if [ "$NOPOPCON"x = ""x ] ; then if [ "$NOPOPCON"x = ""x ] ; then
cd debian-cd && ./tools/update_popcon tasks/stretch/popularity-contest ; cd .. cd debian-cd && ./tools/update_popcon tasks/${CODENAME}/popularity-contest
error=$?
if [ $error != 0 ]; then
echo "popcon update failed, error $error";
echo "aborting";
exit 1
fi
cd ..
fi fi
cd $TOPDIR cd $TOPDIR
mkdir -p $PUBDIRJIG/trace
mkdir -p $RSYNC_TARGET $RSYNC_TARGET_FIRMWARE
for arch in $ARCHES; do for arch in $ARCHES; do
# Reset envvars for next iteration # Reset envvars for next iteration
arch_error="" arch_error=""
arch_start=`now` arch_start=`now`
if [ "$arch" != multi-arch ] ; then rm -rf ${PUBDIRJIG}/$arch ${PUBDIRJIG}-firmware/$arch
rm -rf $PUBDIRJIG/$arch $PUBDIRJIG-firmware/$arch for i in iso-cd jigdo-cd; do
mkdir -p $PUBDIRJIG/$arch mkdir -p ${PUBDIRJIG}/$arch/$i
if $(arch_has_firmware $arch) ; then if $(arch_has_firmware $arch) ; then
mkdir -p $PUBDIRJIG-firmware/$arch mkdir -p ${PUBDIRJIG}-firmware/$arch/$i
fi fi
done
if [ "$arch" != multi-arch ] ; then
# export DI_DIR="$ARCH_DI_DIR" # export DI_DIR="$ARCH_DI_DIR"
export VARIANTS="" export VARIANTS=""
@ -197,7 +209,7 @@ if lockfile -r0 $BUILDLOCK ; then
esac esac
fi fi
# individual desktop cds: gnome, kde, xfce. lxde # individual desktop cd #1: xfce only these days
if [ "$arch"x != "source"x ] && [ "$NOCD"x = ""x ] ; then if [ "$arch"x != "source"x ] && [ "$NOCD"x = ""x ] ; then
if [ "$NOXFCECD"x = ""x ] ; then if [ "$NOXFCECD"x = ""x ] ; then
export MAX_PKG_SIZE=300000000 export MAX_PKG_SIZE=300000000
@ -207,6 +219,7 @@ if lockfile -r0 $BUILDLOCK ; then
build_started XFCECD build_started XFCECD
INSTALLER_CD=5 TASK=Debian-xfce \ INSTALLER_CD=5 TASK=Debian-xfce \
KERNEL_PARAMS='desktop=xfce' \ KERNEL_PARAMS='desktop=xfce' \
OMIT_RELEASE_NOTES=1 OMIT_MANUAL=1 \
MAXCDS=1 MAXISOS=ALL MAXJIGDOS=ALL \ MAXCDS=1 MAXISOS=ALL MAXJIGDOS=ALL \
./testingcds "$arch" & ./testingcds "$arch" &
fi fi
@ -252,94 +265,79 @@ if lockfile -r0 $BUILDLOCK ; then
DI=${DI} DI_DIST=${DI} \ DI=${DI} DI_DIST=${DI} \
./testingcds "$arch" & ./testingcds "$arch" &
fi fi
# Special debian-edu builds, only minor tweaks. #846006
if [ "$NOEDU"x = ""x ] ; then
if [ "$arch" = "i386" ] || [ "$arch" = "amd64" ] ; then
# netinst
build_started EDUNI
OMIT_RELEASE_NOTES=1 OMIT_MANUAL=1 NORECOMMENDS=1 \
NOSUGGESTS=1 COMPLETE=0 INSTALLER_CD=G \
CDNAME=debian-edu \
DEBIAN_EDU=1 \
KERNEL_PARAMS="modules=debian-edu-install-udeb" \
DESKTOP=xfce VARIANTS=xen \
EXCLUDE=debian-edu-exclude \
UDEB_EXCLUDE="$BASEDIR"/data/$CODENAME/udeb_exclude \
TASK=Debian-edu-netinst \
MAXISOS=1 MAXJIGDOS=1 \
DI=${DI} DI_DIST=${DI} \
./testingcds "$arch" &
# netinst with firmware
build_started EDUNIFIRMWARE
OMIT_RELEASE_NOTES=1 OMIT_MANUAL=1 NORECOMMENDS=1 \
NOSUGGESTS=1 COMPLETE=0 INSTALLER_CD=H \
FORCE_FIRMWARE=1 \
CDNAME=firmware-edu \
DEBIAN_EDU=1 \
KERNEL_PARAMS="modules=debian-edu-install-udeb" \
DESKTOP=xfce VARIANTS=xen \
EXCLUDE=debian-edu-exclude \
UDEB_EXCLUDE="$BASEDIR"/data/$CODENAME/udeb_exclude \
TASK=Debian-edu-netinst \
MAXISOS=1 MAXJIGDOS=1 \
DI=${DI} DI_DIST=${DI} \
./testingcds "$arch" &
if [ "$NOUSB"x = ""x ] ; then
# USB "BD" image
build_started EDUUSB
NORECOMMENDS=0 COMPLETE=0 INSTALLER_CD=K \
CDNAME=debian-edu-usb \
DEBIAN_EDU=1 \
KERNEL_PARAMS="modules=debian-edu-install-udeb" \
DESKTOP=xfce VARIANTS=xen \
EXCLUDE=debian-edu-exclude \
TASK=Debian-edu-full \
MAXISOS=1 MAXJIGDOS=1 \
DI=${DI} DI_DIST=${DI} \
./testingcds "$arch" &
# USB "BD" image with firmware
build_started EDUUSBFIRMWARE
NORECOMMENDS=0 COMPLETE=0 INSTALLER_CD=L \
FORCE_FIRMWARE=1 \
CDNAME=firmware-edu-usb \
DEBIAN_EDU=1 \
KERNEL_PARAMS="modules=debian-edu-install-udeb" \
DESKTOP=xfce VARIANTS=xen \
EXCLUDE=debian-edu-exclude \
TASK=Debian-edu-full \
MAXISOS=1 MAXJIGDOS=1 \
DI=${DI} DI_DIST=${DI} \
./testingcds "$arch" &
fi
fi
fi
fi fi
# We've spawned a load of builds; wait here for them all
# to finish
catch_parallel_builds catch_parallel_builds
for dir in $PUBDIRJIG/$arch/jigdo-*; do
generate_checksums_for_arch $arch $dir
done
if [ -d $PUBDIRJIG-firmware/$arch ] ; then
for dir in ${PUBDIRJIG}-firmware/$arch/jigdo-*; do
generate_checksums_for_arch $arch $dir
done
fi
if [ "$NOSYNC"x = ""x ] ; then
if [ "$arch_error"x = "none"x ] ; then
if [ "$RELEASE_BUILD"x = ""x ] ; then
echo "Signing checksums files using the automatic key"
~/build.stretch/sign-images $PUBDIRJIG $arch
fi
echo " Running ~/build.stretch/iso_run $PUBDIRJIG/ $RSYNC_TARGET/ $arch &"
~/build.stretch/iso_run $PUBDIRJIG/ $RSYNC_TARGET/ $arch &
if [ -d $PUBDIRJIG-firmware/$arch ] ; then
if [ "$RELEASE_BUILD"x = ""x ] ; then
echo "Signing checksums files using the automatic key"
~/build.stretch/sign-images $PUBDIRJIG-firmware $ARCH
fi
echo " Running ~/build.stretch/iso_run ${PUBDIRJIG}-firmware/ $RSYNC_TARGET_FIRMWARE/ $arch &"
~/build.stretch/iso_run ${PUBDIRJIG}-firmware/ $RSYNC_TARGET_FIRMWARE/ $arch &
fi
else
~/build.stretch/report_build_error $PUBDIRJIG $RSYNC_TARGET $arch "$arch_error"
if [ -d ${PUBDIRJIG}-firmware/$arch ] ; then
~/build.stretch/report_build_error ${PUBDIRJIG}-firmware $RSYNC_TARGET_FIRMWARE/ $arch "$arch_error"
fi
fi
fi
else # multi-arch else # multi-arch
rm -rf $PUBDIRJIG/multi-arch $PUBDIRJIG-firmware/multi-arch
for i in iso-dvd jigdo-dvd iso-cd jigdo-cd; do
mkdir -p $PUBDIRJIG/multi-arch/$i
if $(arch_has_firmware multi-arch) ; then
mkdir -p $PUBDIRJIG-firmware/multi-arch/$i
fi
done
if [ "$NODVD"x = ""x ] ; then
export MAX_PKG_SIZE=999999999999
export DESKTOP=all
# Special case: make DVD1 fit on a 4GB USB
# stick (#612074)
build_started DVD
export FORCE_CD_SIZE1=STICK4GB
INSTALLER_CD=6 TASK=Debian-all \
KERNEL_PARAMS='desktop=all' \
MAXCDS=1 MAXISOS=ALL MAXJIGDOS=ALL \
VARIANTS=xen \
./testingcds "i386 amd64 source" &
# DVD with firmware
if $(arch_has_firmware $arch) ; then
build_started DVDFIRMWARE
INSTALLER_CD=J TASK=Debian-all \
FORCE_FIRMWARE=1 CDNAME=firmware \
KERNEL_PARAMS="desktop=all" \
MAXCDS=1 MAXISOS=ALL MAXJIGDOS=ALL \
VARIANTS=xen \
DI=${DI} DI_DIST=${DI} \
./testingcds "i386 amd64 source" &
fi
unset FORCE_CD_SIZE1
fi
if [ "$NOEDU"x = ""x ] && [ "$NOUSB"x = ""x ] ; then
if $(arch_has_firmware $arch) ; then
build_started FIRMWAREEDUUSB
OMIT_RELEASE_NOTES=1 OMIT_MANUAL=1 NORECOMMENDS=1 \
NOSUGGESTS=1 COMPLETE=0 INSTALLER_CD=H \
FORCE_FIRMWARE=1 CDNAME=debian-edu-firmware \
DEBIAN_EDU=1 \
KERNEL_PARAMS="desktop=kde" \
DESKTOP=kde VARIANTS=xen \
EXCLUDE=debian-edu-exclude \
UDEB_EXCLUDE="$BASEDIR"/data/$CODENAME/udeb_exclude \
TASK=Debian-edu-full \
MAXISOS=1 MAXJIGDOS=1 \
DI=${DI} DI_DIST=${DI} \
./testingcds "amd64 i386" &
fi
fi
if [ "$NOCD"x = ""x ] && [ "$NONI"x = ""x ] ; then if [ "$NOCD"x = ""x ] && [ "$NONI"x = ""x ] ; then
export DESKTOP=all export DESKTOP=all
@ -366,79 +364,89 @@ if lockfile -r0 $BUILDLOCK ; then
./testingcds "amd64 i386" & ./testingcds "amd64 i386" &
fi fi
if [ "$NOEDU"x = ""x ] ; then
if $(arch_has_firmware $arch) ; then
build_started NIFIRMWAREEDU
OMIT_RELEASE_NOTES=1 OMIT_MANUAL=1 NORECOMMENDS=1 \
NOSUGGESTS=1 COMPLETE=0 INSTALLER_CD=G \
FORCE_FIRMWARE=1 CDNAME=debian-edu-firmware \
DEBIAN_EDU=1 \
KERNEL_PARAMS="desktop=kde" \
DESKTOP=kde VARIANTS=xen \
EXCLUDE=debian-edu-exclude \
UDEB_EXCLUDE="$BASEDIR"/data/$CODENAME/udeb_exclude \
TASK=Debian-edu-netinst \
MAXISOS=1 MAXJIGDOS=1 \
DI=${DI} DI_DIST=${DI} \
./testingcds "amd64 i386" &
fi
fi
fi fi
# We've spawned a load of builds; wait here for them all
# to finish
catch_parallel_builds catch_parallel_builds
for dir in $PUBDIRJIG/multi-arch/jigdo-*; do fi # end of multi-arch
generate_checksums_for_arch multi-arch $dir
done
if [ -d $PUBDIRJIG-firmware/multi-arch ] ; then
for dir in ${PUBDIRJIG}-firmware/multi-arch/jigdo-*; do
generate_checksums_for_arch multi-arch $dir
done
fi
if [ "$NOSYNC"x = ""x ] ; then if [ "$RELEASE_BUILD"x = ""x ] ; then
if [ "$arch_error"x = "none"x ] || [ "$arch_error"x = ""x ] ; then echo "$arch: Not generating torrent files for a non-release build"
else
echo "$arch: Generating torrent files"
~/build.${CODENAME}/mktorrent ${PUBDIRJIG}/$arch/iso-*/*.iso
if [ -d ${PUBDIRJIG}-firmware/$arch ] ; then
~/build.${CODENAME}/mktorrent ${PUBDIRJIG}-firmware/$arch/iso-*/*.iso
fi
fi
echo "$arch: Generating checksum files"
for dir in ${PUBDIRJIG}/$arch/jigdo-*; do
generate_checksums_for_arch $arch $dir
done
if [ -d ${PUBDIRJIG}-firmware/$arch ] ; then
for dir in ${PUBDIRJIG}-firmware/$arch/jigdo-*; do
generate_checksums_for_arch $arch $dir
done
fi
if [ "$NOSYNC"x = ""x ] ; then
if [ "$arch_error"x = "none"x ] || [ "$arch_error"x = ""x ] ; then
if [ "$RELEASE_BUILD"x = ""x ] ; then
echo "Signing checksums files using the automatic key"
~/build.${CODENAME}/sign-images ${PUBDIRJIG} $arch
fi
echo " Running ~/build.${CODENAME}/iso_run ${PUBDIRJIG}/ $RSYNC_TARGET/ $arch \"\" &"
~/build.${CODENAME}/iso_run ${PUBDIRJIG}/ $RSYNC_TARGET/ $arch "" &
if [ -d ${PUBDIRJIG}-firmware/$arch ] ; then
if [ "$RELEASE_BUILD"x = ""x ] ; then if [ "$RELEASE_BUILD"x = ""x ] ; then
echo "Signing checksums files using the automatic key" echo "Signing checksums files using the automatic key"
~/build.stretch/sign-images $PUBDIRJIG multi-arch ~/build.${CODENAME}/sign-images ${PUBDIRJIG}-firmware $arch
fi fi
echo " Running ~/build.stretch/iso_run $PUBDIRJIG/ $RSYNC_TARGET/ multi-arch &" echo " Running ~/build.${CODENAME}/iso_run ${PUBDIRJIG}-firmware/ $RSYNC_TARGET_FIRMWARE/ $arch firmware &"
~/build.stretch/iso_run $PUBDIRJIG/ $RSYNC_TARGET/ multi-arch & ~/build.${CODENAME}/iso_run ${PUBDIRJIG}-firmware/ $RSYNC_TARGET_FIRMWARE/ $arch firmware &
if [ -d $PUBDIRJIG-firmware/multi-arch ] ; then
if [ "$RELEASE_BUILD"x = ""x ] ; then
echo "Signing checksums files using the automatic key"
~/build.stretch/sign-images $PUBDIRJIG-firmware multi-arch
fi
echo " Running ~/build.stretch/iso_run ${PUBDIRJIG}-firmware/ $RSYNC_TARGET_FIRMWARE/ multi-arch &"
~/build.stretch/iso_run ${PUBDIRJIG}-firmware/ $RSYNC_TARGET_FIRMWARE/ multi-arch &
fi
else
~/build.stretch/report_build_error ${PUBDIRJIG} $RSYNC_TARGET multi-arch "$arch_error"
if [ -d ${PUBDIRJIG}-firmware/multi-arch ] ; then
~/build.stretch/report_build_error ${PUBDIRJIG}-firmware $RSYNC_TARGET_FIRMWARE/ $multi-arch "$arch_error"
fi
fi fi
else
~/build.${CODENAME}/report_build_error ${PUBDIRJIG} $RSYNC_TARGET $arch "$arch_error"
if [ -d ${PUBDIRJIG}-firmware/$arch ] ; then
~/build.${CODENAME}/report_build_error ${PUBDIRJIG}-firmware $RSYNC_TARGET_FIRMWARE/ $arch "$arch_error"
fi
fi fi
fi # end of multi-arch fi
done done
# Generate HEADER.html files
~/build.stretch/generate_headers $RSYNC_TARGET ""
~/build.stretch/generate_headers $RSYNC_TARGET_FIRMWARE firmware
if [ "$NOFW"x = ""x ] ; then if [ "$NOFW"x = ""x ] ; then
~/build.stretch/generate_firmware_images stretch ~/build.${CODENAME}/generate_firmware_images ${CODENAME}
~/build.stretch/generate_firmware_images sid ~/build.${CODENAME}/generate_firmware_images sid
fi
if [ $(hostname) = casulana ]; then
rsync_to_pettersson ${OUT_BASE}/cd-sources/ cd-sources/
publish_on_pettersson cd-sources
fi fi
if [ "$NOSNAP"x = ""x ] ; then if [ "$NOSNAP"x = ""x ] ; then
~/bin/weekly-snapshots if [ $(hostname) = casulana ]; then
mkdir -p /tmp/snapshots
rsync_to_pettersson /tmp/snapshots/ snapshots/
publish_on_pettersson snapshots
else
~/bin/weekly-snapshots
fi
fi fi
catch_live_builds catch_live_builds
date -u > $PUBDIRJIG/trace/cdimage.debian.org for DIR in $RSYNC_TARGET $RSYNC_TARGET_FIRMWARE; do
date -u > $DIR/trace/cdimage.debian.org
if [ $(hostname) = casulana ]; then
PETOUT=${DIR##${OUT_BASE}/}
rsync_to_pettersson $DIR/trace/ ${PETOUT}/trace/
publish_on_pettersson ${PETOUT}/trace
fi
done
rm -f $BUILDLOCK rm -f $BUILDLOCK
fi fi

109
contrib/cronjob.weekly-live
View File

@ -1,5 +1,22 @@
#!/bin/bash #!/bin/bash
TOPDIR=$(dirname $0)
if [ "$TOPDIR" = "." ] ; then
TOPDIR=`pwd`
fi
export TOPDIR
. $TOPDIR/settings.sh
. $TOPDIR/common.sh
export WEEKLY_LIVE=${OUT_BASE}/weekly-live-builds
export NF_WEEKLY_LIVE=${CD_I_F}/weekly-live-builds
export WEEKLY_OPENSTACK=${OUT_BASE}/openstack/testing
if [ "$RELEASE_BUILD"x != ""x ] ; then
RELEASE_OUT=${OUT_BASE}/.${RELEASE_BUILD}
fi
now () { now () {
date -u +%F:%H:%M:%S date -u +%F:%H:%M:%S
} }
@ -12,20 +29,56 @@ if [ "$NOLIVE"x = ""x ] ; then
ERROR=$? ERROR=$?
if [ $ERROR -eq 0 ] ; then if [ $ERROR -eq 0 ] ; then
if [ "$RELEASE_BUILD"x = ""x ] ; then if [ "$RELEASE_BUILD"x = ""x ] ; then
~/build.stretch/sign-images /mnt/nfs-cdimage/.live/new/free "" ~/build.${CODENAME}/sign-images ${LIVE_OUT}/new/free ""
mv -f /mnt/nfs-cdimage/weekly-live-builds /mnt/nfs-cdimage/weekly-live-builds.old mv -f ${WEEKLY_LIVE} ${WEEKLY_LIVE}.old
mv /mnt/nfs-cdimage/.live/new/free/ /mnt/nfs-cdimage/weekly-live-builds mv ${LIVE_OUT}/new/free/ ${WEEKLY_LIVE}
rm -rf /mnt/nfs-cdimage/weekly-live-builds.old rm -rf ${WEEKLY_LIVE}.old
~/build.${CODENAME}/sign-images ${LIVE_OUT}/new/non-free ""
mv -f ${NF_WEEKLY_LIVE} ${NF_WEEKLY_LIVE}.old
mv ${LIVE_OUT}/new/non-free ${NF_WEEKLY_LIVE}
rm -rf ${NF_WEEKLY_LIVE}.old
# Generate HEADER.html files
for ARCH in amd64 i386 source; do
~/build.${CODENAME}/generate_headers \
~/build.${CODENAME}/live-header.html.in \
${WEEKLY_LIVE}/$ARCH $ARCH ""
~/build.${CODENAME}/generate_headers \
~/build.${CODENAME}/HEADER.html.in \
${NF_WEEKLY_LIVE}/$ARCH $ARCH firmware
done
# If we're on casulana, push things to pettersson here
if [ $(hostname) = casulana ]; then
rsync_to_pettersson ${WEEKLY_LIVE}/ weekly-live-builds/
rsync_to_pettersson ${NF_WEEKLY_LIVE}/ unofficial/non-free/images-including-firmware/weekly-live-builds/
publish_on_pettersson weekly-live-builds unofficial/non-free/images-including-firmware/weekly-live-builds
fi
~/build.stretch/sign-images /mnt/nfs-cdimage/.live/new/non-free ""
mv -f ${CD_I_F}/weekly-live-builds ${CD_I_F}/weekly-live-builds.old
mv /mnt/nfs-cdimage/.live/new/non-free ${CD_I_F}/weekly-live-builds
rm -rf ${CD_I_F}/weekly-live-builds.old
else else
mkdir -p /mnt/nfs-cdimage/.${RELEASE_BUILD}/ mkdir -p ${RELEASE_OUT}/
rm -rf /mnt/nfs-cdimage/.${RELEASE_BUILD}/live-* rm -rf ${RELEASE_OUT}/live-*
mv /mnt/nfs-cdimage/.live/new/free/ /mnt/nfs-cdimage/.${RELEASE_BUILD}/live-free mv ${LIVE_OUT}/new/free/ ${RELEASE_OUT}/live-free
mv /mnt/nfs-cdimage/.live/new/non-free /mnt/nfs-cdimage/.${RELEASE_BUILD}/live-non-free mv ${LIVE_OUT}/new/non-free ${RELEASE_OUT}/live-non-free
# Generate HEADER.html files
for ARCH in amd64 i386 source; do
~/build.${CODENAME}/generate_headers \
~/build.${CODENAME}/live-header.html.in \
${RELEASE_OUT}/live-free/$ARCH $ARCH ""
~/build.${CODENAME}/generate_headers \
~/build.${CODENAME}/HEADER.html.in \
${RELEASE_OUT}/live-non-free/$ARCH $ARCH firmware
done
# If we're on casulana, push things to pettersson here
if [ $(hostname) = casulana ]; then
rsync_to_pettersson ${RELEASE_OUT}/live-free/ .${RELEASE_BUILD}/live-free/
rsync_to_pettersson ${RELEASE_OUT}/live-non-free/ .${RELEASE_BUILD}/live-non-free/
publish_on_pettersson .${RELEASE_BUILD}/live-free .${RELEASE_BUILD}/live-non-free
fi
fi fi
fi fi
# Wait for the KVM to shut down cleanly # Wait for the KVM to shut down cleanly
@ -53,20 +106,32 @@ if [ "$NOOPENSTACK"x = ""x ] ; then
sleep 30 sleep 30
fi fi
if [ $ERROR -eq 0 ] ; then if [ $ERROR -eq 0 ] ; then
mkdir -p /mnt/nfs-cdimage/.live/new/openstack-stretch rm -rf ${LIVE_OUT}/new/openstack-${CODENAME}
mv -v /mnt/nfs-cdimage/.live/new/openstack-stretch-*/*.* /mnt/nfs-cdimage/.live/new/openstack-stretch mkdir -p ${LIVE_OUT}/new/openstack-${CODENAME}
mv -v ${LIVE_OUT}/new/openstack-${CODENAME}-*/*.* ${LIVE_OUT}/new/openstack-${CODENAME}
for SUM in MD5 SHA1 SHA256 SHA512; do for SUM in MD5 SHA1 SHA256 SHA512; do
cat /mnt/nfs-cdimage/.live/new/openstack-stretch-*/${SUM}SUMS > /mnt/nfs-cdimage/.live/new/openstack-stretch/${SUM}SUMS cat ${LIVE_OUT}/new/openstack-${CODENAME}-*/${SUM}SUMS > ${LIVE_OUT}/new/openstack-${CODENAME}/${SUM}SUMS
done done
if [ "$RELEASE_BUILD"x = ""x ] ; then if [ "$RELEASE_BUILD"x = ""x ] ; then
~/build.stretch/sign-images /mnt/nfs-cdimage/.live/new/openstack-stretch "" ~/build.${CODENAME}/sign-images ${LIVE_OUT}/new/openstack-${CODENAME} ""
mv /mnt/nfs-cdimage/.live/new/openstack-stretch/* /mnt/nfs-cdimage/openstack/testing mv ${LIVE_OUT}/new/openstack-${CODENAME}/* ${WEEKLY_OPENSTACK}
cp ~/build.stretch/openstack-header.html /mnt/nfs-cdimage/openstack/testing/HEADER.html cp ~/build.${CODENAME}/openstack-header.html ${WEEKLY_OPENSTACK}/HEADER.html
# If we're on casulana, push things to pettersson here
if [ $(hostname) = casulana ]; then
rsync_to_pettersson ${WEEKLY_OPENSTACK}/ openstack/testing/
publish_on_pettersson openstack/testing
fi
else else
mkdir -p /mnt/nfs-cdimage/.${RELEASE_BUILD} mkdir -p ${RELEASE_OUT}
rm -rf /mnt/nfs-cdimage/.${RELEASE_BUILD}/openstack rm -rf ${RELEASE_OUT}/openstack
mv /mnt/nfs-cdimage/.live/new/openstack-stretch /mnt/nfs-cdimage/.${RELEASE_BUILD}/openstack mv ${LIVE_OUT}/new/openstack-${CODENAME} ${RELEASE_OUT}/openstack
cp ~/build.stretch/openstack-header.html /mnt/nfs-cdimage/.${RELEASE_BUILD}/openstack/HEADER.html cp ~/build.${CODENAME}/openstack-header.html ${RELEASE_OUT}/openstack/HEADER.html
# If we're on casulana, push things to pettersson here
if [ $(hostname) = casulana ]; then
rsync_to_pettersson ${RELEASE_OUT}/openstack/ .${RELEASE_BUILD}/openstack/
publish_on_pettersson .${RELEASE_BUILD}/openstack
fi
fi fi
fi fi
fi fi

6
contrib/daily-firmware.html
View File

@ -1,10 +1,10 @@
<h1>Daily "firmware" build #BUILDNUM for ARCH, using installer build from INST_VER</h1> <h1>Daily "firmware" build #BUILDNUM for ARCH, using the daily installer build</h1>
<p>These images will install the testing version of Debian, currently <p>These images will install the testing version of Debian, currently
<strong>Stretch</strong>. They include non-free firmware to make <strong>CODENAME</strong>. They include non-free firmware to make
installation easier on some systems requiring proprietary but installation easier on some systems requiring proprietary but
redistributable redistributable
firmware. See <a href="http://wiki.debian.org/Firmware">http://wiki.debian.org/Firmware</a> firmware. See <a href="https://wiki.debian.org/Firmware">https://wiki.debian.org/Firmware</a>
for more details. </p> for more details. </p>
<p>See the <a href="/cdimage/daily-builds/">top-level daily <p>See the <a href="/cdimage/daily-builds/">top-level daily

4
contrib/daily.html
View File

@ -1,7 +1,7 @@
<h1>Daily build #BUILDNUM for ARCH, using installer build from INST_VER</h1> <h1>Daily build #BUILDNUM for ARCH, using the daily installer build</h1>
<p>These images will install the testing version of Debian, currently <p>These images will install the testing version of Debian, currently
<strong>Stretch</strong>.</p> <strong>CODENAME</strong>.</p>
<p>See the <a href="/cdimage/daily-builds/">top-level daily <p>See the <a href="/cdimage/daily-builds/">top-level daily
directory</a> for more information about the daily builds.</p> directory</a> for more information about the daily builds.</p>

9
contrib/ftp.cron
View File

@ -1,4 +1,4 @@
#!/bin/sh #!/bin/bash
set -e set -e
@ -62,6 +62,8 @@ if [ -f "$TRACE" ] ; then
cp "$MYCOPY" "$MYCOPY".building cp "$MYCOPY" "$MYCOPY".building
BUILD_DONE=0
# "Daily" builds # "Daily" builds
# Only run certain builds, otherwise we're just wasting # Only run certain builds, otherwise we're just wasting
# time reproducing identical results: # time reproducing identical results:
@ -79,6 +81,7 @@ if [ -f "$TRACE" ] ; then
echo "Not running even-numbered daily build #$BUILDNUM" echo "Not running even-numbered daily build #$BUILDNUM"
else else
echo "Running daily build #$BUILDNUM." echo "Running daily build #$BUILDNUM."
BUILD_DONE=1
if [ "$FORCE"x = "1"x ] ; then if [ "$FORCE"x = "1"x ] ; then
echo "Build forced by hand" echo "Build forced by hand"
else else
@ -99,9 +102,13 @@ if [ -f "$TRACE" ] ; then
mail_list "$HOSTNAME BAILING OUT OF WEEKLY BUILD" mail_list "$HOSTNAME BAILING OUT OF WEEKLY BUILD"
else else
$TOPDIR/cronjob.weekly $TOPDIR/cronjob.weekly
BUILD_DONE=1
fi fi
fi fi
cd $HOME && ./bin/cdbuilder_log_analyser > build/log/analysis.html cd $HOME && ./bin/cdbuilder_log_analyser > build/log/analysis.html
if [ "$BUILD_DONE"x = "1"x ]; then
/usr/local/bin/static-update-component cdbuilder-logs.debian.org
fi
exit 0 exit 0
fi fi
fi fi

55
contrib/generate_firmware_images
View File

@ -1,14 +1,21 @@
#!/bin/sh #!/bin/bash
set -e set -e
OUT=/mnt/nfs-cdimage/unofficial/non-free/firmware TOPDIR=$(dirname $0)
TOPDIR=~/build if [ "$TOPDIR" = "." ] ; then
SUITE=$1 TOPDIR=`pwd`
fi
export TOPDIR
. $TOPDIR/CONF.sh . $TOPDIR/CONF.sh
. $TOPDIR/settings.sh
. $TOPDIR/common.sh
if [ "$SUITE"x = ""x ] ; then OUT=${OUT_BASE}/unofficial/non-free/firmware
RELEASE=$1
if [ "$RELEASE"x = ""x ] ; then
echo "Need to know what to produce!" echo "Need to know what to produce!"
exit 1 exit 1
fi fi
@ -17,28 +24,34 @@ fi
export TDIR=/srv/cdbuilder.debian.org/dst/deb-cd/tmp/firmware export TDIR=/srv/cdbuilder.debian.org/dst/deb-cd/tmp/firmware
export BASEDIR=$TOPDIR/debian-cd export BASEDIR=$TOPDIR/debian-cd
mkdir -p $TDIR mkdir -p $TDIR
fakeroot $BASEDIR/tools/make-firmware-image $MIRROR $SUITE $TDIR fakeroot $BASEDIR/tools/make-firmware-image $MIRROR $RELEASE $TDIR
DATE=`date +%Y%m%d` DATE=`date +%Y%m%d`
mkdir -p $OUT/$SUITE/$DATE mkdir -p $OUT/$RELEASE/$DATE
mv $TDIR/firmware*.* $TDIR/*SUMS $OUT/$SUITE/$DATE/ mv $TDIR/firmware*.* $TDIR/*SUMS $OUT/$RELEASE/$DATE/
if [ "$RELEASE_BUILD"x = ""x ] ; then if [ "$RELEASE_BUILD"x = ""x ] ; then
echo "Signing firmware checksums files using the automatic key" echo "Signing firmware checksums files using the automatic key"
~/build.stretch/sign-images $OUT/$SUITE/$DATE "" ~/build.${CODENAME}/sign-images $OUT/$RELEASE/$DATE ""
fi fi
# Update the current link, and delete the old surplus builds. Keep up if [ $(hostname) = casulana ]; then
# to 3 # Sync to pettersson
cd $OUT/$SUITE rsync_to_pettersson ${OUT_BASE}/unofficial/non-free/firmware/$RELEASE/${DATE}/ unofficial/non-free/firmware/${RELEASE}/${DATE}/
rm -f current publish_on_pettersson unofficial/non-free/firmware/${RELEASE}/${DATE}/
ln -sf $DATE current else
# Update the current link, and delete the old surplus builds. Keep up
# to 3
cd $OUT/$RELEASE
rm -f current
ln -sf $DATE current
echo "Removing old firmware directories:" echo "Removing old firmware directories:"
NUM=`ls -d 20*/ 2>/dev/null |wc -l` NUM=`ls -d 20*/ 2>/dev/null |wc -l`
NUM=$(($NUM-6)) NUM=$(($NUM-6))
if [ "$NUM" -gt "0" ] ; then if [ "$NUM" -gt "0" ] ; then
REMOVE=`ls -1d 20* 2>/dev/null|head -n $NUM` REMOVE=`ls -1d 20* 2>/dev/null|head -n $NUM`
echo " $REMOVE" echo " $REMOVE"
rm -rf $REMOVE rm -rf $REMOVE
fi
fi fi

135
contrib/generate_headers
View File

@ -1,64 +1,105 @@
#!/bin/bash #!/bin/bash
#
# Tool to generate HEADER.html in various directories with
# context-specific documentation
#
# Call as:
#
# generate_headers $INPUT_FILE $OUTPUT_DIR $ARCH [firmware]
. ~/build.stretch/settings.sh set -e
. ~/build.stretch/common.sh
TOPDIR=$(dirname $0)
if [ "$TOPDIR" = "." ] ; then
TOPDIR=`pwd`
fi
export TOPDIR
. ${TOPDIR}/CONF.sh
. ${TOPDIR}/settings.sh
. ${TOPDIR}/common.sh
INPUT=~/build.stretch/HEADER.html.in
CPP="cpp -traditional -undef -P -C -nostdinc" CPP="cpp -traditional -undef -P -C -nostdinc"
if [ "$1"x = ""x ] ; then if [ "$1"x = ""x ] ; then
echo "$0: Need to be told what input file to use!"
exit 1
fi
INPUT=$1
if [ "$2"x = ""x ] ; then
echo "$0: Need to be told which directory to target!" echo "$0: Need to be told which directory to target!"
exit 1 exit 1
fi fi
WORK=$2
if [ "$2"x = "firmware"x ] ; then if [ "$3"x = ""x ] ; then
echo "$0: Need to be told which architecture to target!"
exit 1
fi
INARCH=$3
if [ "$4"x = "firmware"x ] ; then
FIRMWARE="-DFIRMWARE" FIRMWARE="-DFIRMWARE"
IN_FW=1 IN_FW=1
fi fi
cd $1 if [ "$IN_FW"x = ""x ]; then
for DIR in *; do if $(arch_has_firmware $INARCH) ; then
if [ -d $DIR ] && [ $DIR != trace ] ; then FIRMWARE="-DARCH_HAS_FIRMWARE"
ARCH=$DIR else
if [ "$IN_FW"x = ""x ]; then FIRMWARE=""
if $(arch_has_firmware $ARCH) ; then fi
FIRMWARE="-DARCH_HAS_FIRMWARE" fi
else
FIRMWARE="" case $INARCH in
fi i386|amd64|arm64)
EXTRAS="-DISOHYBRID"
ARCH=$INARCH
;;
multi-arch)
EXTRAS="-DISOHYBRID -DONEonly"
ARCH="multi"
;;
*)
ARCH=$INARCH
EXTRAS=""
;;
esac
case $WORK in
*daily*)
# for daily builds, don't mention large sets - we're only
# making netinsts...
EXTRAS="$EXTRAS -DONEonly"
;;
esac
case $ARCH in
kfreebsd*)
KERNEL="kFreeBSD"
;;
hurd*)
KERNEL="Hurd"
;;
*)
KERNEL="Linux"
;;
esac
cd $WORK
for DIR1 in *-* tar; do
STYLE=$(echo $DIR1 | sed 's,^.*/,,g;s,-.*$,,g')
TYPE=$(echo $DIR1 | sed 's,^.*-,,g')
if [ -d $DIR1 ]; then
UPDATES=""
if (ls -l $DIR1 | grep -q update); then
UPDATES="-DUPDATES"
fi
#echo "Found directory $DIR (ARCH $ARCH, STYLE $STYLE, TYPE $TYPE)"
$CPP -DARCH=${ARCH} -DARCH${ARCH} -DKERNEL=${KERNEL} -DSTYLE${STYLE} -DSTYLE=${STYLE} -DTYPE${TYPE} $UPDATES $EXTRAS $FIRMWARE $INPUT > $DIR1/HEADER.html
if [ -e $DIR1/.errata.html ]; then
$CPP -DARCH=${ARCH} -DARCH${ARCH} -DKERNEL=${KERNEL} -DSTYLE${STYLE} -DSTYLE=${STYLE} -DTYPE${TYPE} $UPDATES $EXTRAS $FIRMWARE $DIR1/.errata.html >> $DIR1/HEADER.html
fi fi
case $ARCH in
i386|amd64|arm64)
EXTRAS="-DISOHYBRID"
;;
multi-arch)
EXTRAS="-DISOHYBRID -DONEonly"
ARCH="multi"
;;
*)
EXTRAS=""
;;
esac
case $ARCH in
kfreebsd*)
KERNEL="kFreeBSD"
;;
hurd*)
KERNEL="Hurd"
;;
*)
KERNEL="Linux"
;;
esac
for DIR1 in $DIR/*-* ; do
STYLE=$(echo $DIR1 | sed 's,^.*/,,g;s,-.*$,,g')
TYPE=$(echo $DIR1 | sed 's,^.*-,,g')
ls -l $DIR1 | grep -q update
if [ $? -eq 0 ] ; then
UPDATES="-DUPDATES"
fi
#echo "Found directory $DIR (ARCH $ARCH, STYLE $STYLE, TYPE $TYPE)"
$CPP -DARCH=${ARCH} -DARCH${ARCH} -DKERNEL=${KERNEL} -DSTYLE${STYLE} -DSTYLE=${STYLE} -DTYPE${TYPE} $UPDATES $EXTRAS $FIRMWARE $INPUT > $DIR1/HEADER.html
done
fi fi
done done

2
contrib/images4testing_d-i
View File

@ -1,2 +1,2 @@
DI_DIST=stretch DI_DIST=buster
DI_CODENAME=$DI_DIST DI_CODENAME=$DI_DIST

71
contrib/iso_run
View File

@ -1,15 +1,17 @@
#!/bin/sh #!/bin/bash
# #
# Helper for image creation # Helper for image creation
# #
PUBDIRJIG=$1 PUBDIRJIG=$1
RSYNC_TARGET=$2 RSYNC_TARGET=$2
ARCH=$3 ARCH=$3
FIRMWARE=$4
LOG=~/build.stretch/log/$ARCH.iso_run LOG=~/build.${CODENAME}/log/$ARCH.iso_run
HOSTNAME=`hostname -f` HOSTNAME=`hostname -f`
LOCK=~/iso_run.lock LOCK=~/iso_run.lock
START=`date -u +%H:%M:%S` START=`date -u +%H:%M:%S`
. $TOPDIR/common.sh
# Check to see if another sync is in progress # Check to see if another sync is in progress
if lockfile -! -l 43200 -r-1 "$LOCK"; then if lockfile -! -l 43200 -r-1 "$LOCK"; then
@ -22,24 +24,6 @@ COPY_START=`date -u +%H:%M:%S`
rm -f $LOG rm -f $LOG
# Given an ISO image:
# 1. create the torrent file if desired (only for release builds)
# 2. copy all of them into place
process_iso() {
FILE=$1
OUTDIR=$2
echo $OUTDIR/$FILE >> $LOG
cp -a $FILE $OUTDIR/$FILE
if [ "$RELEASE_BUILD"x != ""x ] ; then
~/build.stretch/mktorrent $FILE >> $LOG
BTFILE=`echo $FILE.torrent | sed 's/iso-/bt-/'`
echo $OUTDIR/$BTFILE >> $LOG
cp -a $BTFILE $OUTDIR/$BTFILE
fi
}
# Poor man's rsync, but with some local optimisations # Poor man's rsync, but with some local optimisations
copy_files() { copy_files() {
SRC=$1 SRC=$1
@ -60,40 +44,26 @@ copy_files() {
do do
for DISKTYPE in $DISKTYPES; do for DISKTYPE in $DISKTYPES; do
for DIRTYPE in $DIRTYPES; do for DIRTYPE in $DIRTYPES; do
mkdir -p -m775 $TARGET/$ARCH.tmp/$DIRTYPE-$DISKTYPE if [ -d $ARCH/$DIRTYPE-$DISKTYPE ]; then
mkdir -p -m775 $TARGET/$ARCH.tmp/$DIRTYPE-$DISKTYPE
fi
done done
done done
# Copy the specific files across that we want in the
# publishing dir
cd $ARCH cd $ARCH
find . -name '*.jigdo' -o -name '*.template' \ find . -name '*.jigdo' -o -name '*.template' \
-o -name '*.iso' \
-o -name '*.torrent' \
-o -name '*.list.gz' -o -name '*SUMS*' | \ -o -name '*.list.gz' -o -name '*SUMS*' | \
xargs tar cf - | (cd $TARGET/$ARCH.tmp/ && tar xvf -) >> $LOG xargs tar cf - | (cd $TARGET/$ARCH.tmp/ && tar xvf -) >> $LOG
# Only make torrent files and dirs for release builds
rm -rf bt-*
for DISKTYPE in $DISKTYPES; do
if [ "$RELEASE_BUILD"x != ""x ] ; then
mkdir bt-$DISKTYPE
fi
for FILE in iso-$DISKTYPE/*.iso; do
if [ -e $FILE ] ; then
process_iso $FILE $TARGET/$ARCH.tmp
fi
done
if [ "$RELEASE_BUILD"x != ""x ] ; then
for FILE in $TARGET/$ARCH.tmp/iso-$DISKTYPE/*SUMS*; do
if [ -e $FILE ] ; then
cp -al $FILE $TARGET/$ARCH.tmp/bt-$DISKTYPE/
fi
done
fi
done
if [ "$RELEASE_BUILD"x = ""x ] ; then if [ "$RELEASE_BUILD"x = ""x ] ; then
# Only populate the HEADER.html file for regular weekly # Only populate the HEADER.html file for regular weekly
# builds; we don't want it for releases # builds; we don't want it for releases
HUM_DATE=`date -u` HUM_DATE=`date -u`
sed "s/ARCH/$ARCH/g;s/DATE/$HUM_DATE/g" ~/build.stretch/weekly.html \ sed "s/ARCH/$ARCH/g;s/DATE/$HUM_DATE/g" ~/build.${CODENAME}/weekly.html \
> $TARGET/$ARCH.tmp/HEADER.html > $TARGET/$ARCH.tmp/HEADER.html
fi fi
@ -129,6 +99,23 @@ rm -rf $RSYNC_TARGET/$ARCH.old &
mkdir -p $RSYNC_TARGET/trace mkdir -p $RSYNC_TARGET/trace
date -u > $RSYNC_TARGET/trace/cdimage.debian.org date -u > $RSYNC_TARGET/trace/cdimage.debian.org
# Generate the HEADER.html files in the subdirectories
~/build.${CODENAME}/generate_headers ~/build.${CODENAME}/HEADER.html.in $RSYNC_TARGET/$ARCH $ARCH $FIRMWARE
END=`date -u +%H:%M:%S` END=`date -u +%H:%M:%S`
echo "$ARCH synced across to $RSYNC_TARGET; started at $START, copying started at $COPY_START, ended at $END" echo "$ARCH synced across to $RSYNC_TARGET; started at $START, copying started at $COPY_START, ended at $END"
# If we're on casulana, push things to pettersson here
if [ $(hostname) = casulana ]; then
START=`date -u +%H:%M:%S`
PETOUT=${RSYNC_TARGET##${OUT_BASE}/}
echo "$START: Starting $ARCH sync from $RSYNC_TARGET/$ARCH to pettersson" >> $LOG
(cd $RSYNC_TARGET/$ARCH && find . -name '*.iso' > .iso-wanted)
rsync_to_pettersson $RSYNC_TARGET/$ARCH/ ${PETOUT}/$ARCH/ '--exclude *.iso'
rsync_to_pettersson $RSYNC_TARGET/trace/ ${PETOUT}/trace/
publish_on_pettersson ${PETOUT}/$ARCH ${PETOUT}/trace
END=`date -u +%H:%M:%S`
echo "$RSYNC_TARGET/$ARCH synced across to pettersson; started at $START, ended at $END"
fi
echo "$END: Finished" >> $LOG echo "$END: Finished" >> $LOG

2
contrib/merge-sums
View File

@ -1,4 +1,4 @@
#!/bin/sh #!/bin/bash
export LANG=C export LANG=C
export LC_ALL=C export LC_ALL=C

2
contrib/mklist
View File

@ -1,4 +1,4 @@
#!/bin/sh #!/bin/bash
export LC_ALL=C export LC_ALL=C

32
contrib/mktorrent
View File

@ -1,27 +1,45 @@
#!/bin/sh #!/bin/bash
HTTPBASE="http://cdimage.debian.org/cdimage" HTTPBASE="https://cdimage.debian.org/cdimage"
NFHTTPBASE="https://cdimage.debian.org/cdimage/unofficial/non-free/cd-including-firmware/"
MKTORRENT=btmakemetafile.bittornado
for FILE in $@; do for FILE in $@; do
MKTORRENT=btmakemetafile.bittornado
ISODIR=`dirname $FILE` ISODIR=`dirname $FILE`
BTDIR=`echo $ISODIR | sed 's/iso-/bt-/;s/usb-/bt-/'` BTDIR=`echo $ISODIR | sed 's/iso-/bt-/;s/usb-/bt-/'`
if [ ! -d $BTDIR ] ; then if [ ! -d $BTDIR ] ; then
mkdir -p $BTDIR mkdir -p $BTDIR
fi fi
# Add 2 http seeds: the release path *and* the archive path - # Add multiple http seeds: the release path *and* the archive path
# hopefully clients will use whichever is available! # for both the free images and the non-free images - hopefully
# clients will use whichever is available!
case ${FILE} in case ${FILE} in
*live+nonfree*)
VER=$(echo ${FILE} | sed 's,^.*/debian-\(live-\)*,,;s,-.*$,,')"-live+nonfree"
FREE=0
;;
*live*) *live*)
VER=$(echo ${FILE} | sed 's,^.*/debian-\(live-\)*,,;s,-.*$,,')"-live" VER=$(echo ${FILE} | sed 's,^.*/debian-\(live-\)*,,;s,-.*$,,')"-live"
FREE=1
;;
*firmware*)
VER=$(echo ${FILE} | sed 's,^.*/firmware-,,;s,-.*$,,')"+nonfree"
FREE=0
;; ;;
*) *)
VER=$(echo ${FILE} | sed 's,^.*/debian-*,,;s,-.*$,,') VER=$(echo ${FILE} | sed 's,^.*/debian-*,,;s,-.*$,,')
FREE=1
;; ;;
esac esac
HTTP1="${HTTPBASE}/release/${VER}/${FILE}" if [ $FREE -eq 1 ]; then
HTTP2="${HTTPBASE}/archive/${VER}/${FILE}" HTTP1="${HTTPBASE}/release/${VER}/${FILE}"
HTTP2="${HTTPBASE}/archive/${VER}/${FILE}"
else
HTTP1="${NFHTTPBASE}/${VER}/${FILE}"
HTTP2="${NFHTTPBASE}/archive/${VER}/${FILE}"
fi
$MKTORRENT http://bttracker.debian.org:6969/announce \ $MKTORRENT http://bttracker.debian.org:6969/announce \
--comment '"Debian CD from cdimage.debian.org"' \ --comment '"Debian CD from cdimage.debian.org"' \
--httpseeds ${HTTP1}\|${HTTP2} \ --httpseeds ${HTTP1}\|${HTTP2} \

2
contrib/report_build_error
View File

@ -1,4 +1,4 @@
#!/bin/sh #!/bin/bash
export TOPDIR=$(dirname $0) export TOPDIR=$(dirname $0)

20
contrib/settings.sh
View File

@ -1,24 +1,26 @@
export TRACE=/srv/cdbuilder.debian.org/src/ftp/debian/project/trace/pettersson.debian.org
export ARCH_DI_DIR=/srv/cdbuilder.debian.org/src/deb-cd/d-i
export HOSTNAME=`hostname -f` export HOSTNAME=`hostname -f`
export CODENAME=buster
export OUT_BASE=~/publish
export CD_I_F=${OUT_BASE}/unofficial/non-free/images-including-firmware
export TRACE=/srv/cdbuilder.debian.org/src/ftp/debian/project/trace/$(hostname).debian.org
export ARCH_DI_DIR=/srv/cdbuilder.debian.org/src/deb-cd/d-i
export PUBDIR=/srv/cdbuilder.debian.org/dst/deb-cd export PUBDIR=/srv/cdbuilder.debian.org/dst/deb-cd
export MIRROR=/srv/cdbuilder.debian.org/src/ftp/debian export MIRROR=/srv/cdbuilder.debian.org/src/ftp/debian
export BASEDIR=~/build.stretch/debian-cd export BASEDIR=~/build.${CODENAME}/debian-cd
export MKISOFS=~/build.stretch/mkisofs/usr/bin/mkisofs export MKISOFS=~/build.${CODENAME}/mkisofs/usr/bin/mkisofs
export EXTRACTED_SOURCES=${OUT_BASE}/cd-sources/
export LIVE_OUT=${OUT_BASE}/.live
if [ "$DATE"x = ""x ] ; then if [ "$DATE"x = ""x ] ; then
export DATE=`date -u +%Y%m%d` export DATE=`date -u +%Y%m%d`
fi fi
export EXTRACTED_SOURCES=/mnt/nfs-cdimage/cd-sources
if [ "$ARCHES"x = ""x ] ; then if [ "$ARCHES"x = ""x ] ; then
ARCHES="i386 amd64 armhf multi-arch armel source arm64 ppc64el mips mipsel mips64el s390x" ARCHES="amd64 i386 multi-arch arm64 armhf armel source ppc64el mips mipsel mips64el s390x"
fi fi
if [ "$ARCHES_FIRMWARE"x = ""x ] ; then if [ "$ARCHES_FIRMWARE"x = ""x ] ; then
ARCHES_FIRMWARE="amd64 i386 multi-arch" ARCHES_FIRMWARE="amd64 i386 multi-arch"
# ARCHES_FIRMWARE="amd64"
fi fi

2
contrib/sign-images
View File

@ -1,4 +1,4 @@
#!/bin/sh #!/bin/bash
# #
# Helper for signing images # Helper for signing images
# #

33
contrib/testingcds
View File

@ -13,13 +13,13 @@
#set -x #set -x
if [ "$CONF"x = ""x ] ; then if [ "$CONF"x = ""x ] ; then
CONF=~/build.stretch/CONF.sh CONF=~/build.${CODENAME}/CONF.sh
fi fi
export CF=$CONF export CF=$CONF
if [ "$DCD_DIR"x = ""x ] ; then if [ "$DCD_DIR"x = ""x ] ; then
DCD_DIR=~/build.stretch/debian-cd DCD_DIR=~/build.${CODENAME}/debian-cd
fi fi
export CF=$CONF
now () { now () {
date -u +%F:%H:%M:%S date -u +%F:%H:%M:%S
@ -60,7 +60,7 @@ export LOGFILE="$LOG/$RUN"
case "$INSTALLER_CD" in case "$INSTALLER_CD" in
1) 1)
export DISKTYPE=BC;; export DISKTYPE=BC;;
2|7|8|C|F|G) 2|7|8|C|F|G|H)
export DISKTYPE=NETINST;; export DISKTYPE=NETINST;;
3|6|I|J) 3|6|I|J)
export OUT_TYPE=dvd export OUT_TYPE=dvd
@ -73,7 +73,7 @@ case "$INSTALLER_CD" in
export DISKTYPE=lxde-CD;; export DISKTYPE=lxde-CD;;
B) B)
export DISKTYPE=xfce+lxde-CD;; export DISKTYPE=xfce+lxde-CD;;
9) 9|K|L)
export OUT_TYPE=bd export OUT_TYPE=bd
export DISKTYPE=BD;; export DISKTYPE=BD;;
D) D)
@ -81,9 +81,6 @@ case "$INSTALLER_CD" in
export DISKTYPE=DLBD;; export DISKTYPE=DLBD;;
E) E)
export DISKTYPE=gnome-CD;; export DISKTYPE=gnome-CD;;
H)
export OUT_TYPE=bd
export DISKTYPE=BD;;
*) export DISKTYPE=CD;; *) export DISKTYPE=CD;;
esac esac
@ -106,12 +103,10 @@ if [ "$OUT"x = ""x ] ; then
fi fi
if [ "$BUILDNAME"x = ""x ] ; then if [ "$BUILDNAME"x = ""x ] ; then
TRACEFILE="$OUT_TYPE-trace" BUILDNAME="$OUT_TYPE"
echo " Making $ARCH ${DISKTYPE} in $OUT"
else
TRACEFILE="$BUILDNAME-trace"
echo " Making $ARCH ${DISKTYPE} (${BUILDNAME}) in $OUT"
fi fi
TRACEFILE="$BUILDNAME-trace"
echo " Making $ARCH ${DISKTYPE} (${BUILDNAME}) in $OUT"
if [ "$INSTALLER_CD" = "1" ] ; then if [ "$INSTALLER_CD" = "1" ] ; then
export OFFICIAL="Official BusinessCard Snapshot" export OFFICIAL="Official BusinessCard Snapshot"
@ -138,13 +133,17 @@ elif [ "$INSTALLER_CD" = "E" ]; then
elif [ "$INSTALLER_CD" = "F" ]; then elif [ "$INSTALLER_CD" = "F" ]; then
export OFFICIAL="Official Mac Netinst" export OFFICIAL="Official Mac Netinst"
elif [ "$INSTALLER_CD" = "G" ]; then elif [ "$INSTALLER_CD" = "G" ]; then
export OFFICIAL="Official Debian-Edu Multi-Arch Netinst" export OFFICIAL="Official Debian-Edu Netinst"
elif [ "$INSTALLER_CD" = "H" ]; then elif [ "$INSTALLER_CD" = "H" ]; then
export OFFICIAL="Official Debian-Edu Multi-Arch Image" export OFFICIAL="Unofficial Debian-Edu Netinst including firmware"
elif [ "$INSTALLER_CD" = "I" ]; then elif [ "$INSTALLER_CD" = "I" ]; then
export OFFICIAL="Unofficial DVD including firmware" export OFFICIAL="Unofficial DVD including firmware"
elif [ "$INSTALLER_CD" = "J" ]; then elif [ "$INSTALLER_CD" = "J" ]; then
export OFFICIAL="Unofficial Multi-Arch DVD including firmware" export OFFICIAL="Unofficial Multi-Arch DVD including firmware"
elif [ "$INSTALLER_CD" = "K" ]; then
export OFFICIAL="Debian-Edu Full USB image"
elif [ "$INSTALLER_CD" = "L" ]; then
export OFFICIAL="Unofficial Debian-Edu Full USB image including firmware"
fi fi
rm -rf "$OUT" rm -rf "$OUT"
@ -273,7 +272,7 @@ if ./build.sh "$ARCH" >>"$LOGFILE" 2>&1 ; then
for i in $MYEMAIL for i in $MYEMAIL
do do
grep -e "extents written" -e "Written to medi." "$LOGFILE"|mail -s "$RELEASE_BUILD testingcds $RUN ($OUTARCH $DISKTYPE) went ok" "$i" grep -e "extents written" -e "Written to medi." "$LOGFILE"|mail -s "$RELEASE_BUILD testingcds $RUN ($OUTARCH $BUILDNAME) went ok" "$i"
done done
overview "$RUN" success overview "$RUN" success
@ -287,7 +286,7 @@ else
for i in $MYEMAIL for i in $MYEMAIL
do do
(echo Error when trying to create cds for $RUN ; cat $LOGFILE) | \ (echo Error when trying to create cds for $RUN ; cat $LOGFILE) | \
mail -s "$RELEASE_BUILD testingcds $RUN has failed; log included" "$i" mail -s "$RELEASE_BUILD testingcds $RUN ($OUTARCH $BUILDNAME) has failed; log included" "$i"
done done
overview "$RUN" failed overview "$RUN" failed
END=`now` END=`now`

1
debian/changelog vendored
View File

@ -3,6 +3,7 @@ debian-cd (3.1.23) UNRELEASED; urgency=medium
[ Holger Levsen ] [ Holger Levsen ]
* data/buster: update Debian Edu installer logo and syslinux splash image * data/buster: update Debian Edu installer logo and syslinux splash image
from src:debian-edu-artwork 2.10.5-1. from src:debian-edu-artwork 2.10.5-1.
* contrib: updated from setup.git/buster/.
[ Wolfgang Schweer ] [ Wolfgang Schweer ]
* tools/start_new_disc: set adjusted cd_type for incomplete BD image for * tools/start_new_disc: set adjusted cd_type for incomplete BD image for