debian-cd-clone/README.easy-build

                        debian-cd easy-build
                        ====================

INTRODUCTION
============
easy-build.sh is a "wrapper" script around the more basic build.sh build
script. It is designed to make building a specific image or set or images
as easy as possible, while still offering all the flexibility of debian-cd.

easy-build.sh supports the following variations:
- businesscard, netinst and full CD images, and DVD images
- single-architecture and multi-architecture images
- optionally include source packages
- select to install GNOME (default), KDE, LXDE or Xfce desktops as desktop
  task
- create a combined LXDE/Xfce "light desktop" CD
- create a businesscard or netinst CD, or DVD set supporting all four
  desktop environments
- creates ISO files by default; creating jigdo files is possible
- specify which Debian release to use
- include custom versions of packages (debs)
- specify which Debian Installer (D-I) version and source to use, including
  custom images
- include custom versions of Debian Installer components (udebs)

Using easy-build.sh is somewhat simpler than calling make directly or even
than using build.sh as it splits the complex configuration of debian-cd
into three levels:
1) CONF.sh: basic configuration such as paths
2) easy-build.sh: characteristicts of the CD/DVD set, for example:
   - Debian release
   - D-I version and source of D-I images
   - number of CDs or DVDs to generate
3) command line parameters:
   - type of image (businesscard, netinst, full CD or DVD
   - which architecture(s)
   - which desktop to install

easy-build.sh also ensures that "task files" are automatically updated as
needed. If custom packages are included it can automatically generate a
Packages file for the local repository: just copy the packages to the
correct location and let easy-build.sh do the rest.

Types of images & task definitions
----------------------------------
easy-build.sh can be used to create four types of images:
- businesscard CD: only contains Debian Installer; any packages neede for
  the installed system are retrieved from a mirror during the installation
- netinst CD: contains Debian Installer and all packages needed to install a
  Debian "base system" (packages with priority important or higher)
- full CD: same as netinst CD, but filled to capacity with additional
  packages based on "task" definitions; image size is 680MB
- DVD: same as full CD, but with an image size of 4.7GB

The businesscard and netinst CD are always a single image. The size of these
images depends on the architecture. For i386 they are around 40MB and 150MB
respectively.

The full CD and DVD are sets of images, but the set can be limited to any
number. The first image of the set always includes Debian Installer and
packages needed for the base system. After that the first and following
images are filled up to capacity with packages in the following order:
- "essential" packages from tasksel tasks
- "optional" packages from tasksel tasks
- other packages sorted by popularity (popcon score)

The basic order in which packages are included on full CDs and DVDs is
defined in 'tasks/<CODENAME>/Debian'. The order in which tasksel tasks are
included is defined in 'tasks/<CODENAME>/task.list*'.

Current GNOME and KDE desktop environments are so large that only the
packages needed to install one of them can be included on the first full CD.
Therefore debian-cd supports creating a CD or DVD set targeted at a specific
desktop environment: either GNOME, or KDE, or Xfce.
The default 'task.list' is targeted at the GNOME desktop environment.

It is possible to customize the debian-cd task definitions and even to
customize tasksel, but that falls outside the scope of this document.

CREATING A MIRROR OF THE ARCHIVE
================================
debian-cd requires you to have a local mirror of the Debian archive,
normally on the same machine where you run debian-cd. There are lots of
methods to create a (partial) local archive.

Make sure the following are included in your local archive:
- ./dists/<codename>/main/debian-installer/binary-<arch>/
  contains Debian Installer components (udebs)
- ./doc/
- ./tools/
and optionally:
- ./dists/<codename>/main/installer-<arch>/
  contains Debian Installer images (official releases)

One method is to use debmirror. An example script for creating a full or
partial local mirror using debmirror suitable for use with debian-cd can be
found at: http://alioth.debian.org/~fjp/debmirror/.

BASIC CONFIGURATION
===================
Most of the variables in CONF.sh can be left at their default values. Some
just because their default value should be OK for most purposes, some
because their value is set by easy-build.sh itself.

The following variables that define paths *must* be set in CONF.sh:
- MIRROR
- TDIR
- OUT
- APTTMP

You may also want to change the following variables:
- CONTRIB: comment out if you only want to include packages from main
- IMAGESUMS: comment out to skip creating MD5/SHA5 sums for images
- OMIT_MANUAL
- OMIT_RELEASE_NOTES / RELEASE_NOTES_LOCATION

Note that if you want to vary the value of a variable for differbent builds
you can also "add" it in easy-build.sh and set it there.

BUILDING IMAGES
===============
After the basic configuration has been done, there are still a few variables
that need to be checked in easy-build.sh itself:
- CODENAME: set to the codename (e.g. etch, lenny, sid) of the Debian release
  for which you want to build CD/DVD images
- DI_CODENAME: set to the codename of Debian release from which D-I
  *components* (udebs) should be taken; normally same value as CODENAME
- DI_DIST/DI_WWW_HOME/DI_DIR: location from where D-I *images* should be
  taken; value must "match" the specified DI_CODENAME
- MAX_CDS, MAX_DVDS: only used when building full CD or DVD images

D-I images and components
-------------------------
It is essential that D-I images and components used to build the CD match.
For example, the kernel version included in the D-I image must be the same
as the kernel version of the kernel module udebs copied to the CD.

The easy-build.sh script offers four mutually exclusive options to specify
the source of D-I images. In theory more are possible, but the included
options should cover most use cases.
Make sure you have only one of the four options uncommented! And don't let
yourself be confused by the fact that the four options use different
variables.

If you keep to the following basic rules you should be OK.
1) If DI_CODENAME is a real Debian release (e.g. etch or lenny, but not sid)
   you should normally use the 1st or 2nd option. Use the 1st option if your
   local mirror includes D-I images, else use the the 2nd option.
2) If you use daily built D-I images (3rd option), DI_CODENAME should be set
   to "sid".
3) If you use custom images (4th option), DI_CODENAME should match the branch
   of the your D-I SVN checkout and build environment; further explanation of
   this is outside the scope of this document.

Examples
--------
Below are some basic examples how to run easy-build.sh. Examples for some
more advanced usages will be given in later sections.

1) Build a lenny netinst CD image for i386
- set CODENAME and DI_CODENAME to "lenny"
- select appropriate source for D-I images (1st or 2nd option)
- run './easy-build.sh NETINST'

2) Build the first three CD images for amd64 with KDE as desktop environment
- set MAX_CDS to "3"
- run './easy-build.sh -d kde CD amd64'

3) Build the full set of DVD images for sparc
- comment out MAX_DVDS
- run './easy-build.sh DVD sparc'

4) Build an i386 netinst to install testing using daily built D-I images
- set CODENAME to the codename for the current testing release
- set DI_CODENAME to "sid"
- select the 3rd option as source for D-I images
- run './easy-build.sh NETINST'

BUILDING MULTI-ARCH IMAGES
==========================
Multi-arch images are CDs or DVDs that can be used to install more than one
architecture. However, architectures cannot be randomly combined. The main
limitations for combining architectures are:
- boot loader files and configuration cannot conflict
- image size

The following architectures can safely be combined on a single image:
- i386, amd64 and powerpc
- alpha, hppa and ia64

When you combine two or three architectures on a single 680MB CD there
remains very little space after Debian Installer and base system packages
have been included. For that reason multi-arch CDs are normally created as
a netinst image. For example:
$ ./easy-build.sh NETINST i386 amd64 powerpc

A DVD has a lot more space and can easily include packages needed to install
a desktop environment. For example:
$ ./easy-build.sh DVD -d kde i386 amd64

It is even possible to include source packages on a DVD, which makes it
suitable to hand out at trade shows. For example:
$ ./easy-build.sh DVD i386 amd64 source

INCLUDING CUSTOM PACKAGES, D-I COMPONENTS and D-I IMAGES
========================================================
easy-build.sh makes it relatively simple to include custom packages, both
regular packages (debs) and debian-installer components (udebs), in CD
images. The main rule to keep in mind is that if the package also exists
in the official archive, the version number of your custom package must
be higher than the one of the official package.

Besides custom debian-installer components (udebs), you can also use custom
or locally built debian-installer images (kernel, initrd and related files).

You should always ensure that udebs included on a CD match the udebs used
when building the debian-installer image you include. For example kernel
versions need to match, but also versioned dependensies between udebs
(and note that not all dependencies between udebs are declared explicitly).
And two udebs from the same source package should in principle have the
same version even if one is included in the initrd and the other is loaded
from the CD during the installation.

Whether custom debs and udebs are used is determined by the environment
variable LOCAL. Whether custom D-I images are used is determined by
selecting the correct source from them, in most cases that will be by
setting DI_DIR (see comments in easy-build.sh). Both variables can be set
in easy-build.sh.

Using custom packages
---------------------
The repository for the custom packages has the same basic structure as an
official mirror. The main difference is the category: instead of "main"
(or "contrib" or "non-free") this has to be "local". You should also make
sure to use the release codename and not the suite.

So you get for example:
.../dists/lenny/local/binary-amd64/
.../dists/lenny/local/binary-i386/
.../dists/lenny/local/debian-installer/binary-amd64/
.../dists/lenny/local/debian-installer/binary-i386/
.../dists/sid/local/binary-amd64/
.../dists/sid/local/binary-i386/
.../dists/sid/local/debian-installer/binary-amd64/
.../dists/sid/local/debian-installer/binary-i386/

Note that debs will be taken from 'dists/$CODENAME/...' while udebs will be
taken from 'dists/$DI_CODENAME/...'. So if CODENAME and DI_CODENAME have
different values (e.g. when taking debs from testing but using D-I based on
unstable), you will need to place your custom debs and udebs under different
codenames.

The local category does not need to have a Release file, but each
'binary-<arch>' directory must have a Packages file.

It is possible to have the packages themselves in a pool directory, but it
is probably easier to just place them under the 'binary-<arch>' directories.
You can group packages in subdirectories.

By default debian-cd will look for the "local" category in the same place
as the official mirror (as defined by the envvar MIRROR), but in most cases
it will be simpler to specify an alternative location by setting the
environment variable LOCALDEBS in your CONF.sh. In that case you can also
let easy-build.sh take care of generating an updated Packages.gz file for
each source that is used in a build by setting the envvar UPDATE_LOCAL in
easy-build.sh (see tools/Packages-gen).

Example, assuming we are building for and i386 CD for lenny:
1) in CONF.sh, set for example:
   export MIRROR=/srv/mirror/debian
   export LOCALDEBS=/srv/mirror/custom
2) create the following directories:
   /srv/mirror/custom/dists/lenny/local/binary-i386
   /srv/mirror/custom/dists/lenny/local/debian-installer/binary-i386
3) copy your custom debs to the first directory
4) copy your custom udebs to the second directory
5) in easy-build.sh, set:
   export LOCAL=1
   UPDATE_LOCAL=1

Note that debian-cd will fail if you set LOCAL but the "local" directory for
a codename/arch/section that is referenced in a build does not exist.
You will need to create the 'dists/<codename>/local' directories, but if
UPDATE_LOCAL is set, easy-build.sh will automatically create any missing
'binary-<codename>' subdirectories and (empty) Packages.gz files.

Using custom debian-installer images
------------------------------------
Note that the build target used and the way images are organized under the
build/dest directory by the debian-installer build system varies per
architecture. The example below should work for i386 and amd64, but may need
to be adjusted for other architectures.

Example:
1) create a directory, for example:
   /srv/mirror/custom/images/i386/
2) build the d-i images (in the installer/build directory) using:
   $ make reallyclean; fakeroot make build_cdrom_isolinux
3) copy the images using:
   $ cp -r dest/cdrom /srv/mirror/custom/images/i386/
4) in easy-build.sh, set:
   export DI_DIR="/srv/mirror/custom/images/%ARCH%"

'%ARCH%' will be automatically expanded to the correct architecture whe
debian-cd is run.