Including the updates

Burning a RedHat CD HOWTO
Prev |  | Next

# 5. Including the updates

There are three steps involved, the first two are (almost) equal in all the releases, while the last one changes quite a bit because of the changes in the anaconda installer:

1. Correct the file protection modes

2. Replace updated RPMs

3. Rebuild the installer

To incorporate the updates, you need write access to the distribution directory from a Linux machine, with a working version of rpm installed, while to rebuild the anaconda installer you need to use a release of Redhat Linux equal to the one you are rebuilding the installer for (otherwise the procedure will fail). If you maintain a mirror of the `updates` directory, you can at any time produce a CD including the current updates by repeating these steps.

## 5.1. Correcting the file protection modes

During the installation process of the releases up to and including 6.2, some programs are run directly off the CD. Unfortunately, the FTP program does not always preserve the protection modes of the files and directories that are copied. Therefore, it is necessary to make sure that execute permission is given to programs, shell scripts and shared libraries, before the directory is burned on the CD. This is done by running the updatePerm.sh script on your local copy of the distribution. It is really needed only for version 6.2 and older, the only part useful to the 7.3/8.0/9 releases procedure is the directories permissions update, even if the rest won't hurt and things are kept coherent. It is almost equal to the `updatePerm` script included in the previous version of the howto, just some slight changes were made. Before using this script you have to configure the rhcd.conf configuration file and export a RHCDPATH variable pointing to the directory where this file is.

            $ export RHCDPATH=/home/luigi/tmp/rhcd-scripts
            $ sh updatePerm.sh

## 5.2. Replacing the updated RPMS

The updateCD.sh script copies all the new files from the update directory to the RPMS (and SRPMS) directory. The script uses the `rvc` program which was presented in section Comparing RPM versions to determine which packages in the updates directory are more recent. Older packages are moved to the ${OLDDIR} directory. If the CHECKSIG variable is set to "yes", all the packages in the main tree will have their signature checked for correctness. If a package fails the signature check (the kind of check is configured by means of the USEGPG variable whose value is assigned in the file rhcd.conf), it is moved to the OLDDIR directory with an added extension of "CDcheckfail".

Before using this script, you have to configure the rhcd.conf configuration file and export a RHCDPATH variable pointing to the directory where this file is.

            $ export RHCDPATH=/home/luigi/tmp/rhcd-scripts
            $ sh updateCD.sh

| After having incorporated the updates in the main `RedHat/RPMS` directory, your copy of the distribution is no longer a mirror of the Red Hat distribution site. Actually, it is more up-to-date! Therefore, if you attempt to mirror the distribution using mirror, older versions of the RPM's that have been updated will be downloaded once more, and the updates deleted. The bash/wget based procedure doesn't suffer from the problem, but will leave the main tree in an incoherent state. Old and new packages will be in this case mixed together, but you can find and remove them wrapping the `rvc` binary in a simple shell script (which I will leave as an exercise for the reader...).

## 5.3. Rebuilding the installer

Things have changed pretty much in this section with the introduction of the anaconda installer (as of release 6.1) and with the considerable increment in size (and ... number of CDs) the 7.x/8.0 distributions have seen. Until release 6.2, the only step composing this section was represented by generating a new `hdlist` file. With release 6.2, this appears to be true only to a certain extent, because of the changes in the anaconda installer, in the rpm software itself (from version 3.x to 4.x) and the migration of the updated packages to this new version (updates for release 6.2 are in fact packaged with both major releases of the `rpm` software). We will consider three different procedures trying to cover all the releases.

### 5.3.1. RedHat <= 6.1

#### 5.3.1.1. Regenerating the hdlist file

When installing from the CD, the installation program on the CD relies on the file `RedHat/base/hdlist` describing what RPM packages are available on the CD. The `hdlist` file can be generated by the program `misc/src/install/genhdlist`. This program must be run with the absolute path to the root of the distribution as the only argument. Here is the `updateHdlist` script which calls that program (from version 1.34 of this howto):

                #!/bin/bash

                RHVERSION=6.1
                ARCH=i386

                echo generating hdlist...
                RHROOT=/home/luigi/tmp/redhat-${RHVERSION}
                GENHDDIR=${RHROOT}/${ARCH}/misc/src/anaconda/utils

                chmod u+x ${GENHDDIR}/genhdlist
                chmod 644 ${RHROOT}/${ARCH}/RedHat/base/hdlist
                ${GENHDDIR}/genhdlist ${RHROOT}/${ARCH} || echo "*** GENHDLIST FAILED ***"

                exit 0

| Important note for RedHat < 6.1
| The installation in RedHat 6.1 is completely changed from earlier versions, and RedHat has introduced anaconda. The `genhdlist` program is now found in a different place, so in the script above, we used
                  GENHDDIR=${RHROOT}/${ARCH}/misc/src/anaconda/utils
while for releases up to (and including) 6.0 that line should read
                  GENHDDIR=${RHROOT}/${ARCH}/misc/src/install

In some cases, `genhdlist` fails to run, because the executable is not statically linked. In such a case, you can add a new line `${RHROOT}/${ARCH}/RedHat/instimage/usr/lib` in `/etc/ld.so.conf` and run `ldconfig -v`.

Another solution is to recompile `genhdlist`. The following modification to the `updateHdlist` script worked under RedHat 5.2:

    	    #!/bin/bash

    	    RHVERSION=6.1
    	    ARCH=i386

    	    RHROOT=/misc/redhat/redhat-${RHVERSION}
    	    GENHDDIR=${RHROOT}/${ARCH}/misc/src/anaconda/utils

    	    echo Compiling genhdlist...
    	    sed -e 's/FD_t/int/' \
    	        -e 's/fdOpen/open/' \
    	        -e 's/fdClose/close/' \
    	        -e 's/fdFileno//' < ${GENHDDIR}/genhdlist.c > /tmp/genhdlist.c
    	    cc -o /tmp/genhdlist -I/usr/include/rpm /tmp/genhdlist.c -lrpm -lz

    	    echo generating hdlist...
    	    chmod 644 ${RHROOT}/${ARCH}/RedHat/base/hdlist
    	    /tmp/genhdlist ${RHROOT}/${ARCH} || echo "*** GENHDLIST FAILED ***"

    	    exit 0

In this version of the script, a copy of the C source of `genhdlist.c` is piped through `sed` to create a copy in `/tmp` that will compile under RedHat 5.2. This version of `genhdlist` is then used to create the `hdlist` file

| Important note for RedHat 5.2
| As distributed with RedHat version 5.2 and earlier, `genhdlist` CRASHES if there are files in the `RedHat/RPMS` directory which are not RPM files! This causes problems, because in the 5.2 distribution, there are a couple of non-RPM files named `ls-lR` and `ls-lR.gz` in `RedHat/RPMS`. Therefore, you must remove all non-RPM files from the directory. Alternatively, you can apply the patch genhdlist.c.diff to `misc/src/install/genhdlist.c` and do a make. The patch will cause `genhdlist` to ignore any non-RPM files.

#### 5.3.1.2. Creating the CD iso image

You'll need to create an image file which will be written to the CD. This file will be 500Mb or more so find a partition with enough free space. You may need to be root to use mount and cdrecord. Here you will prepare the iso image of the bootable CD to be burned. It is actually, not strictly necessary, to create a bootable CD, because you could use a boot floppy instead of it, but it's definitely a nifty feature (and makes your disc more similar in behaviour to the original one). These are the commands I use to complete the task:

                $ mkdir /images-destination-dir
                $ mkisofs  -r  -J  -T  -v  -V "Red Hat 6.1 (Hedwig)" \
                    -c boot.cat  -b images/boot.img \
                    -o /images-destination-dir/i386-disc.iso .

This is needed to burn the (bootable) disc and is executed from the top level directory of the distribution. The `/images-destination-dir` directory is the container for the iso image you are generating, and it must exist (obviously) before starting the procedure. In the following table you can read a brief explanation of the various options and their meanings (most of it was extracted from the `mkisofs` man page).

Table 1. mkisofs options and parameters

-r |  | Rock Ridge extensions with useful values for the permission bits
-J |  | Joliet extensions to use the cd with some different operating systems
-T |  | Generate a TRANS.TBL file in each directory to map correctly the file names even on systems which do not support the Rock Ridge extensions.
-v |  | be verbose
-V <volid> |  | Specifies the volume ID (volume name or label) to be written into the master block.
-c <boot catalog> |  | Specifies the path and filename of the boot catalog to be used when making an "El Torito" bootable CD. The pathname must be relative to the source path specified to mkisofs.
-b <eltorito boot image> |  | Specifies the path and filename of the boot image to be used when making an "El Torito" bootable CD. The pathname must be relative to the source path specified to mkisofs and specify a floppy image (which is why we use one of the floppy images found on the original CD. You may want to change this with the `pcmcia.img` image to install using pcmcia devices like network cards or CDROM readers.
-o <filename> |  | Name of the file containing the generated iso image
. |  | This is the root directory for our generated iso image (we are working from the root directory of every CD, so a dot is enough).

You will find details of how to burn the image on a media in Burning the CD. The `mkisofs` and `cdrecord` steps can be executed by means of a graphical frontend like X-CD-Roast which should currently support the creation of bootable CDs (I've never used it, so don't expect me to give you any explanation).

### 5.3.2. RedHat 6.2

Apparently, this is a problem child when it comes to burning an uptodate CD. The introduction of version 4 of the Redhat Package Manager (RPM), made the procedure to update the anaconda installer fail. So the procedures listed will work only if the updated packages were built using a version of the RPM software which is older than or equal to 3.0.4 (so, basically, 3.0.4 or 3.0.5).

If you are using the original packages from Redhat, you have to avoid using updates released after 28 March 2001 (which is a bit useless, in my opinion) or you have to rebuild the packages using the old rpm format. Details on the downgrade procedure and tools which implement it can be found in the document rpmhack. I have not personally tested this procedure, even if it appears to work if you read about it on the anaconda-devel and kickstart mailing lists (you can find them on the mailing lists section of the Redhat website.

If you decide to stick to the old original packages and complete the update (using the rpm 4.0.2 packages after the installation is finished) there are two possible ways of doing it, depending on which kind of updates you want to complete the CD with. If some of the updates regard directly the installation process (e.g. kernel, python, kudzu), you will have to use the installer rebuilding procedure explained in the document Building a Red Hat Linux 6.2 CDROM, otherwise you can still use the old procedure (the one for releases previous to and including 6.1 explained in the previous section). The last two steps, which are, respectively, creating the iso image and burning the actual media are described in Creating the CD iso image and Burning the CD, respectively.

### 5.3.3. RedHat 9, 8.0 and 7.3

Once again a lot of things have changed with the release of the 7.x series of the distribution. There are now more operations to complete to obtain a fresh and uptodate series of CDs. Exactly, they have become more than one with release 7.0 and now the tree has to be split to fit on the media. This is done by means of the `splitdistro` script, which is written in python like most of the anaconda installer. To complete this part, you must use a Linux RedHat 7.3, 8.0 or 9 machine with the anaconda-runtime package installed (it will probably have version 7.3.7, 8.0.4 or 9.0.4), depending on the release you want to rebuild. The procedure is composed by seven steps:

1. Regenerating the `hdlist` and `hdlist2` files

2. Updating the `comps.xml` (or `comps`) file

3. Rebuilding the installer

4. Splitting the distribution in CD-sized chunks

5. Regenerating the `hdlist` and `hdlist2` files (again)

6. Generating the iso images

7. adding and checking the md5 signatures in the iso images

All the steps are grouped together in a single script presented in the last section.

#### 5.3.3.1. Preliminary operations on the main tree

Some of the scripts included in the `anaconda-runtime` package need the main tree to be moved in a subdirectory named like the architecture we are building for (so `i386/` for me). We will move everything to such directory before starting the procedure and change the invocation of the scripts which don't need this modification.

For redhat 9 and 8.0:

                $ chmod  -R  u+w  /absolute-path-to-toplevel-dir
                $ mkdir  -p  /absolute-path-to-toplevel-dir/i386
                $ cd /absolute-path-to-toplevel-dir
                $ /bin/mv  *  i386

You should change "/absolute-path-to-toplevel-dir" with the absolute path of the directory where the root of your local copy of the distribution is located (maybe somewhere on some hard drive). You will get an error, from the execution of the last command, because the `i386/` directory cannot be moved under itself, but you don't need to worry about that.

For redhat 7.3:

                $ chmod  -R  u+w  /absolute-path-to-toplevel-dir
                $ mkdir  -p  /absolute-path-to-toplevel-dir/i386
                $ cd /absolute-path-to-toplevel-dir
                $ for i in `ls` ; do [ $i != "SRPMS" -a $i != i386 ] && /bin/mv $i i386 ; done

You shouldn't receive any error message this time from the last command (hopefully).

#### 5.3.3.2. Regenerating the hdlist and hdlist2 files

This is done by means of the following two commands with the help of the `genhdlist` program.

                $ /usr/lib/anaconda-runtime/genhdlist  /absolute-path-to-toplevel-dir/i386
                $ chmod  644 /absolute-path-to-toplevel-dir/i386/RedHat/base/hdlist{,2}

Once again, "/absolute-path-to-toplevel-dir" is the absolute path of the directory where the root of your local copy of the distribution is located. The second command is needed to make sure the correct permissions are set for the file. You should already have an idea of what these files are about if you have read through The Redhat directory.

#### 5.3.3.3. Update the comps.xml file

In Redhat Linux 8.0 the format of the comps file has completely changed and it's now based on XML. It provides much more flexibility and ease of customization as you can read in The comps file. If you have modified or intend to modify the list of the installed packages, you need to complete this step. This, in turn, implies having the modified version of comps-9.tar.gz comps-9.tar.gz (the original one doesn't work for me) or comps-8.0.tar.gz package (depending on the release you are building) including the master comps file found on the Redhat website and the `comps-extras` rpm package installed. Follow these steps for Redhat 9 and 8.0:

                $ cd /some-dir-of-your-choice
                $ tar xzvf /path-to-comps-9.tar.gz/comps-9.tar.gz
                $ cd comps
                $ make
                $ cat comps-milan.xml |sed 's!</comps>!!g' >comps-tmp.xml
    	    $ /usr/share/comps-extras/getfullcomps.py  comps.xml \
                     /absolute-path-to-toplevel-dir i386 >> comps-tmp.xml
    	    $ echo '</comps>' >> comps-tmp.xml
    	    $ cp comps-tmp.xml /absolute-path-to-toplevel-dir/i386/RedHat/base/comps.xml

Beside "/absolute-path-to-toplevel-dir", you should take care of assigning valid names to "/some-dir-of-your-choice" and "/path-to-comps-9.tar.gz". The rest of the commands can be just copied. And... you must (obviously) change 9 to read 8.0 if you are building a Redhat 8.0 distribution.

Again, before issuing the make command, you should modify the file `comps-milan.xml.in` using your favourite text editor and following the guidelines and the suggestions found in The comps file and on the anaconda comps section of the Redhat website.

The script presented in the last section will execute all the steps needed after the make command, using the COMPSFILE variable to find the `comps-milan.xml` file (it doesn't need to have that name, I'm just using the original name, but you can change it if you want).

If you are using Redhat 7.3, the `comps` file (have you noticed the different name...) is a textual file with a completely different syntax described in some more detail in The comps file. In this case, the only necessary operations are modifying the file to suit your needs and copying it to the `RedHat/base/comps` file in the main tree overwriting the original one.

#### 5.3.3.4. Rebuilding the installer

This will rebuild the anaconda installer in your local copy of the distribution using your updated packages. For Redhat 9 execute:

                $ /usr/lib/anaconda-runtime/buildinstall  \
                    --pkgorder /absolute-path-to-toplevel-dir/pkgorder.txt  \
                    --comp dist-9 --product "Red Hat Linux" --version 9  \
                    --release "Redhat 9 (Shrike)" /absolute-path-to-toplevel-dir/i386

Where, once again, "/absolute-path-to-toplevel-dir" is the directory where the root of your local copy of the distribution is located.

For Redhat 8.0, the procedure is pretty much the same (the "--product" option is missing):

                $ /usr/lib/anaconda-runtime/buildinstall  \
                    --pkgorder /absolute-path-to-toplevel-dir/pkgorder.txt  \
                    --comp dist-8.0 --version 8.0  --release "Redhat 8.0 (Psyche)" \
                    /absolute-path-to-toplevel-dir/i386

Or if you are still using Redhat 7.3 (as I am):

                $ /usr/lib/anaconda-runtime/buildinstall  \
                    --pkgorder /absolute-path-to-toplevel-dir/pkgorder.txt  \
                    --comp dist-7.3 --version 7.3 /absolute-path-to-toplevel-dir/i386

The absence of the (mandatory in 8.0) --release option is the only noticeable difference.

#### 5.3.3.5. Split the distribution

This will create five directories, each one corresponding to a different CD and will put in them hard links to the real files contained in your local copy of the distribution.

| This will not work at all for Redhat 7.3 if you don't use the modified version of the `splitdistro` script reported in the next paragraph. For Redhat 8.0 and 9, a modified version of `splitdistro` is provided mainly because even if the problems in the previous script were fixed, the execution now fails if there are not enough packages to fill all the CDs (the first four completely and the last one even just partly).

                $ $/usr/lib/anaconda-runtime/splitdistro  \
                  --fileorder /absolute-path-to-toplevel-dir/pkgorder.txt  --release \
                    "Redhat 9 (Shrike)"  /absolute-path-to-toplevel-dir  i386

The only thing you need to change for 8.0 and 7.3 is the string passed to the --release option (which should read "Redhat 8.0 (Psyche)" or "Redhat 7.3 (Valhalla)")

For Redhat 7.3 the version of the splitdistro7.3 (python) script used was extracted from the anaconda-runtime 7.3.7 package and modified by me. You should sustitute it to the original one (maybe after copying the latter) named `/usr/lib/anaconda-runtime/splitdistro`.

The only modification (apart from some small fixes), the script went through, is a change in its behaviour if the `SRPMS` directory is not found (doesn't terminate, but generate the CDs without source packages).

For Redhat 8.0 the version of the splitdistro8.0 (python) script used was extracted from the anaconda-runtime 8.0.4 package and modified once again by me to obtain some improvements I felt the need for. You should sustitute it to the original one (maybe after copying the latter somewhere) named `/usr/lib/anaconda-runtime/splitdistro`. Anyway, the original one works well, if you want to build a distribution which has all the SRPMS packages (so to fill all the 5 CDs otherwise the script will fail).

The only modification the script went through is a change in its behaviour if the `SRPMS` directory is not found (doesn't terminate failing, but generates the CDs without source packages) or there is one CD which hasn't any package on it (instead of failing, generates an empty directory).

For Redhat 9 you can find a copy of the script with the same modifications applied to the version included in release 8.0 here: splitdistro9. Everything said for Redhat 8.0 in the previous paragraph applies to release 9.

#### 5.3.3.6. Regenerating the hdlist and hdlist2 files

This is needed to recreate the hdlist and hdlist2 files, using some of the informations obtained in the previous steps. There are no differences between 7.3, 8.0 and 9 for this execution of the program The command to issue is the following:

                 $ /usr/lib/anaconda-runtime/genhdlist  \
                     --fileorder /absolute-path-to-toplevel-dir/pkgorder.txt  --withnumbers \
                     /absolute-path-to-toplevel-dir/i386-disc[1-3]

As you can see, there are two new options passed to the program, if you remember the first run of it. The first one, --fileorder, tells genhdlist to use the file `pkgorder.txt` we generated in the second step (rebuild the installer). This file keeps informations on how the packages were split on the different CDs and is used by the installer to determine in which order the packages should be installed. Basically, if you avoid using it, you will (probably) end up swapping the various CDs many times during the installation. The --withnumbers option is needed to associate a CD number to every package (as you can see, a wildcard indicating the first 3 iso images is used).

#### 5.3.3.7. Generating the iso images

Here you will prepare the iso images to be burned on the actual CDs. There are two different commands to be used for the first disc and for the rest of them. This is due to the need of obtaining a first CD which is bootable. This is actually, not strictly necessary, because you could use a boot floppy instead of it, but it's definitely a nifty feature (and makes your discs more similar in behaviour to the original ones). These are the commands I use to complete the task:

                  $ mkdir /images-destination-dir
                  $ mkisofs  -r  -J  -T  -v  -V "Red Hat 9 (Shrike) disc 1" \
    		  -c isolinux/boot.cat  -b isolinux/isolinux.bin -no-emul-boot \
                      -boot-load-size 4 -boot-info-table -o /images-destination-dir/i386-disc1.iso .

This is needed to burn the first (bootable) disc for RedHat 8.0 and 9 (with no floppy emulation) and is executed from the top level directory of the distribution. The `/images-destination-dir` directory is the container for the five iso images you are generating, and it must exist before starting the procedure. The only thing which needs to be changed for Redhat 8.0 is the volume name (it should be "Red Hat 8.0 (Psyche) disc 1").

                  $ mkdir /images-destination-dir
                  $ mkisofs  -r  -J  -T  -v  -V "Red Hat 7.3 (Valhalla) disc 1" \
                      -c boot.cat  -b dosutils/autoboot/boot.img \
                      -o /images-destination-dir/i386-disc1.iso .

This is needed to burn the first (bootable) disc on 7.3 and is executed from the top level directory of the distribution (this time with floppy emulation).

The rest of the images can be written by means of this "for" loop

                  $  for i in `echo 2 3 4 5` ; do mkisofs  -r  -J  -T  -v  \
                       -V "Red Hat 9 (Shrike) disc ${i}"  \
                       -o /images-destination-dir/i386-disc${i}.iso . ; done

The loop just presented will prepare the last four images giving them the correct numbers. As you can see, there are just two missing options from the first run, and, as you can guess, they are needed only to create a bootable CD. In Creating the CD iso image, you can read a brief explanation of the various options and their meanings (most of it was extracted from the man page). Again if you are building a Redhat 8.0 you should change the volume name to read "Red Hat 8.0 (Psyche) disc ${i}".

#### 5.3.3.8. Implant and check the md5 signatures in the iso images

This is actually an optional step but it permits the use of the "checkmedia" option to verify the CDs signatures before installing them, so to guarantee their correctness.

The following commands permit to inject and verify an md5 signature on an iso image:

                $ /usr/lib/anaconda-runtime/implantisomd5 iso-image
                $ /usr/lib/anaconda-runtime/checkisomd5 iso-image

After completing all these steps, we will find ourselves with the five CD images to burn. Considering that typing all this stuff is a bit time consuming, in the next section is presented a script, which will complete all of the listed operations in a single run (do not forget to configure the parameters properly).

#### 5.3.3.9. Putting all the steps together

The updateBuild.sh script will execute all the steps needed to rebuild the distribution CDs for RedHat 7.3, 8.0 or 9 in a single run (as root). Before using this script you have to configure the rhcd.conf configuration file after exporting a RHCDPATH variable pointing to the directory where this file is. If you want to include a modified comps.xml (or comps) file in your CDs as explained in The comps file, you should copy it into the location defined by means of the COMPSFILE variable now (before executing the script). Don't forget to add the modified `splitdistro` script to the `/usr/lib/anaconda-runtime` directory if you need it.

                # export RHCDPATH=/home/luigi/tmp/rhcd-scripts
                # sh updateBuild.sh

Prev | Home | Next
Obtaining your local copy of the distribution |  | Burning the CD(s)