debian-cd-clone/tools/update_popcon

101 lines
2.5 KiB
Plaintext
Raw Normal View History

Merge of changes from Steve's development branch to take us to debian-cd version 3.0.0. Highlights: * Support now added for multi-arch CDs/DVDs, including mixed binary/source discs. Multi-arch discs should be bootable on those multiple arches, modulo boot-sector clashes. Extra support added in boot-i386 and boot-amd64 to make the 2 main arches happily co-exist. * Disc sizing is now much more intelligent - sizes are defined depending on the disk type chosen at the start, and discs are now filled exactly to those sizes while files are added rather than the old up-front guessing method. Equally, the metadata on the disc (Packages, Packages.gz, Sources, Sources.gz, md5sums.txt) is generated as much as possible while this is happening to make the sizing code incredibly accurate. Using this method of disc sizing means that customising discs should be much easier/safer - either add custom contents at the start and debian-cd will fill the remainder of the disc, or afterwards roll back the packages on the disc and add extras later. * Source is now treated as (almost) just another arch, with most of the special casing for source hidden internally. If asking for source-only discs, they will simply be built using all the available sources. If combined with other arches, the sources will automatically be chosen to match the binary packages. Meeting GPL requirements was never so easy! * Removed lots of old cruft to clean up the codebase: + non-US support + woody support + lots of old support scripts that have been made redundant + significantly simpler Makefile, much easier to follow + old boot-$ARCH.calc files for estimating boot-file sizes are now (obviously) obsolete and therefore gone
2006-12-07 22:09:01 -01:00
#!/bin/sh
#
# update_popcon
#
# Simple helper script to update our popularity-contest generated task
# list from the data available at popcon.debian.org
DATE=`date`
dir=/etc/ssl/ca-debian
if [ -d $dir ]; then
capath="--capath $dir"
fi
# Determine temp dir to use
if [ "$BDIR"x = ""x ] ; then
if [ "$TMPDIR"x != ""x ] ; then
BDIR=$TMPDIR
else
BDIR=/tmp
fi
echo "update_popcon not given a temp dir, using $BDIR" >&2
fi
if [ ! -d "$BDIR" ] ; then
echo "update_popcon: temp dir '$BDIR' does not exist" >&2
exit 1
fi
# Create temp dir and ensure cleanup
TDIR=$BDIR/update_popcon.$$
mkdir -p $TDIR
trap 'rm -rf $TDIR' EXIT HUP INT QUIT TERM
dl_file () {
# Grab the headers for each file so that we can check the size.
# We've had problems in the past with download failures causing issues,
# so let's make sure that we get the complete file in each case
DIST=$1
SITE="http://popcon.debian.org/"
curl -L $capath -s --head $SITE/$DIST/by_inst > $TDIR/$DIST.head
error=$?
if [ $error -ne 0 ] ; then
echo "Failed to download correctly: curl error $error on HEAD" >&2
exit 1
fi
HEAD_SIZE=`cat $TDIR/$DIST.head | \
awk '/^Content-Length/ {gsub("\r","");print $2}'`
curl -L $capath -s $SITE/$DIST/by_inst > $TDIR/$DIST
error=$?
DL_SIZE=`stat -c %s $TDIR/$DIST`
if [ $error -ne 0 ] ; then
echo "Failed to download correctly: curl error $error" >&2
exit 1
fi
Merge of changes from Steve's development branch to take us to debian-cd version 3.0.0. Highlights: * Support now added for multi-arch CDs/DVDs, including mixed binary/source discs. Multi-arch discs should be bootable on those multiple arches, modulo boot-sector clashes. Extra support added in boot-i386 and boot-amd64 to make the 2 main arches happily co-exist. * Disc sizing is now much more intelligent - sizes are defined depending on the disk type chosen at the start, and discs are now filled exactly to those sizes while files are added rather than the old up-front guessing method. Equally, the metadata on the disc (Packages, Packages.gz, Sources, Sources.gz, md5sums.txt) is generated as much as possible while this is happening to make the sizing code incredibly accurate. Using this method of disc sizing means that customising discs should be much easier/safer - either add custom contents at the start and debian-cd will fill the remainder of the disc, or afterwards roll back the packages on the disc and add extras later. * Source is now treated as (almost) just another arch, with most of the special casing for source hidden internally. If asking for source-only discs, they will simply be built using all the available sources. If combined with other arches, the sources will automatically be chosen to match the binary packages. Meeting GPL requirements was never so easy! * Removed lots of old cruft to clean up the codebase: + non-US support + woody support + lots of old support scripts that have been made redundant + significantly simpler Makefile, much easier to follow + old boot-$ARCH.calc files for estimating boot-file sizes are now (obviously) obsolete and therefore gone
2006-12-07 22:09:01 -01:00
if [ $DL_SIZE != $HEAD_SIZE ] ; then
echo "Failed to download correctly: expected $HEAD_SIZE bytes, but got $DL_SIZE bytes" >&2
exit 1
fi
LASTMOD=`awk '/^Last-Modified:/ {
gsub("\r","")
gsub("^Last-Modified: ","")
print $0}' $TDIR/$DIST.head`
echo "$LASTMOD"
}
MAIN_DATE=`dl_file main`
error=$?
if [ $error -ne 0 ]; then
exit $error
fi
CONTRIB_DATE=`dl_file contrib`
error=$?
if [ $error -ne 0 ]; then
exit $error
fi
OUT=$1
Merge of changes from Steve's development branch to take us to debian-cd version 3.0.0. Highlights: * Support now added for multi-arch CDs/DVDs, including mixed binary/source discs. Multi-arch discs should be bootable on those multiple arches, modulo boot-sector clashes. Extra support added in boot-i386 and boot-amd64 to make the 2 main arches happily co-exist. * Disc sizing is now much more intelligent - sizes are defined depending on the disk type chosen at the start, and discs are now filled exactly to those sizes while files are added rather than the old up-front guessing method. Equally, the metadata on the disc (Packages, Packages.gz, Sources, Sources.gz, md5sums.txt) is generated as much as possible while this is happening to make the sizing code incredibly accurate. Using this method of disc sizing means that customising discs should be much easier/safer - either add custom contents at the start and debian-cd will fill the remainder of the disc, or afterwards roll back the packages on the disc and add extras later. * Source is now treated as (almost) just another arch, with most of the special casing for source hidden internally. If asking for source-only discs, they will simply be built using all the available sources. If combined with other arches, the sources will automatically be chosen to match the binary packages. Meeting GPL requirements was never so easy! * Removed lots of old cruft to clean up the codebase: + non-US support + woody support + lots of old support scripts that have been made redundant + significantly simpler Makefile, much easier to follow + old boot-$ARCH.calc files for estimating boot-file sizes are now (obviously) obsolete and therefore gone
2006-12-07 22:09:01 -01:00
rm -f $OUT
echo "/*" >> $OUT
echo " Popularity Contest results" >> $OUT
echo " See the README for details on updating." >> $OUT
echo "" >> $OUT
echo " Last update: $DATE" >> $OUT
echo " Based on main data from $MAIN_DATE" >> $OUT
echo " Based on contrib data from $CONTRIB_DATE" >> $OUT
Merge of changes from Steve's development branch to take us to debian-cd version 3.0.0. Highlights: * Support now added for multi-arch CDs/DVDs, including mixed binary/source discs. Multi-arch discs should be bootable on those multiple arches, modulo boot-sector clashes. Extra support added in boot-i386 and boot-amd64 to make the 2 main arches happily co-exist. * Disc sizing is now much more intelligent - sizes are defined depending on the disk type chosen at the start, and discs are now filled exactly to those sizes while files are added rather than the old up-front guessing method. Equally, the metadata on the disc (Packages, Packages.gz, Sources, Sources.gz, md5sums.txt) is generated as much as possible while this is happening to make the sizing code incredibly accurate. Using this method of disc sizing means that customising discs should be much easier/safer - either add custom contents at the start and debian-cd will fill the remainder of the disc, or afterwards roll back the packages on the disc and add extras later. * Source is now treated as (almost) just another arch, with most of the special casing for source hidden internally. If asking for source-only discs, they will simply be built using all the available sources. If combined with other arches, the sources will automatically be chosen to match the binary packages. Meeting GPL requirements was never so easy! * Removed lots of old cruft to clean up the codebase: + non-US support + woody support + lots of old support scripts that have been made redundant + significantly simpler Makefile, much easier to follow + old boot-$ARCH.calc files for estimating boot-file sizes are now (obviously) obsolete and therefore gone
2006-12-07 22:09:01 -01:00
echo "*/" >> $OUT
echo "" >> $OUT
cat $TDIR/main $TDIR/contrib | \
egrep -v '(^#|Total|-----|kernel-source)' | \
sort -rn -k3,3 -k7,7 -k4,4 | awk '{print $2}' >> $OUT
if [ $? -ne 0 ]; then
exit $?
fi