Linux Makefiles

Linux i386 Boot Code HOWTO
Prev |  | Next

# 2. Linux Makefiles

Before perusing Linux code, we should get some basic idea about how Linux is composed, compiled and linked. A straightforward way to achieve this goal is to understand Linux makefiles. Check Cross-Referencing Linux if you prefer online source browsing.

## 2.1. linux/Makefile

Here are some well-known targets in this top-level makefile:

- xconfig, menuconfig, config, oldconfig: generate kernel configuration file `linux/.config`;

- depend, dep: generate dependency files, like `linux/.depend`, `linux/.hdepend` and `.depend` in subdirectories;

- vmlinux: generate resident kernel image `linux/vmlinux`, the most important target;

- modules, modules_install: generate and install modules in `/lib/modules/$(KERNELRELEASE)`;

- tags: generate tag file `linux/tags`, for source browsing with vim.

Overview of `linux/Makefile` is outlined below:

    include .depend
    include .config
    include arch/i386/Makefile

    vmlinux: generate linux/vmlinux
            /* entry point "stext" defined in arch/i386/kernel/head.S */
            $(LD) -T $(TOPDIR)/arch/i386/vmlinux.lds -e stext
            /* $(HEAD) */
            + from arch/i386/Makefile
                    arch/i386/kernel/head.o
                    arch/i386/kernel/init_task.o
            init/main.o
            init/version.o
            init/do_mounts.o
            --start-group
            /* $(CORE_FILES) */
            + from arch/i386/Makefile
                    arch/i386/kernel/kernel.o
                    arch/i386/mm/mm.o
            kernel/kernel.o
            mm/mm.o
            fs/fs.o
            ipc/ipc.o
            /* $(DRIVERS) */
            drivers/...
                    char/char.o
                    block/block.o
                    misc/misc.o
                    net/net.o
                    media/media.o
                    cdrom/driver.o
                    and other static linked drivers
                    + from arch/i386/Makefile
                            arch/i386/math-emu/math.o (ifdef CONFIG_MATH_EMULATION)
            /* $(NETWORKS) */
            net/network.o
            /* $(LIBS) */
            + from arch/i386/Makefile
                    arch/i386/lib/lib.a
            lib/lib.a
            --end-group
            -o vmlinux
            $(NM) vmlinux | grep ... | sort > System.map
    tags: generate linux/tags for vim
    modules: generate modules
    modules_install: install modules
    clean mrproper distclean: clean up build directory
    psdocs pdfdocs htmldocs mandocs: generate kernel documents

    include Rules.make

    rpm: generate an rpm

"--start-group" and "--end-group" are ld command line options to resolve symbol reference problem. Refer to Using LD, the GNU linker: Command Line Options for details.

`Rules.make` contains rules which are shared between multiple Makefiles.

## 2.2. linux/arch/i386/vmlinux.lds

After compilation, ld combines a number of object and archive files, relocates their data and ties up symbol references. `linux/arch/i386/vmlinux.lds` is designated by `linux/Makefile` as the linker script used in linking the resident kernel image `linux/vmlinux`.

    /* ld script to make i386 Linux kernel
     * Written by Martin Mares <mj@atrey.karlin.mff.cuni.cz>;
     */
    OUTPUT_FORMAT("elf32-i386", "elf32-i386", "elf32-i386")
    OUTPUT_ARCH(i386)
    /* "ENTRY" is overridden by command line option "-e stext" in linux/Makefile */
    ENTRY(_start)
    /* Output file (linux/vmlinux) layout.
     * Refer to Using LD, the GNU linker: Specifying Output Sections */
    SECTIONS
    {
    /* Output section .text starts at address 3G+1M.
     * Refer to Using LD, the GNU linker: The Location Counter */
      . = 0xC0000000 + 0x100000;
      _text = .;                    /* Text and read-only data */
      .text : {
            *(.text)
            *(.fixup)
            *(.gnu.warning)
            } = 0x9090
    /* Unallocated holes filled with 0x9090, i.e. opcode for "NOP NOP".
     * Refer to Using LD, the GNU linker: Optional Section Attributes */

      _etext = .;                   /* End of text section */

      .rodata : { *(.rodata) *(.rodata.*) }
      .kstrtab : { *(.kstrtab) }

    /* Aligned to next 16-bytes boundary.
     * Refer to Using LD, the GNU linker: Arithmetic Functions */
      . = ALIGN(16);                /* Exception table */
      __start___ex_table = .;
      __ex_table : { *(__ex_table) }
      __stop___ex_table = .;

      __start___ksymtab = .;        /* Kernel symbol table */
      __ksymtab : { *(__ksymtab) }
      __stop___ksymtab = .;

      .data : {                     /* Data */
            *(.data)
            CONSTRUCTORS
            }
    /* For "CONSTRUCTORS", refer to
     * Using LD, the GNU linker: Option Commands */

      _edata = .;                   /* End of data section */

      . = ALIGN(8192);              /* init_task */
      .data.init_task : { *(.data.init_task) }

      . = ALIGN(4096);              /* Init code and data */
      __init_begin = .;
      .text.init : { *(.text.init) }
      .data.init : { *(.data.init) }
      . = ALIGN(16);
      __setup_start = .;
      .setup.init : { *(.setup.init) }
      __setup_end = .;
      __initcall_start = .;
      .initcall.init : { *(.initcall.init) }
      __initcall_end = .;
      . = ALIGN(4096);
      __init_end = .;

      . = ALIGN(4096);
      .data.page_aligned : { *(.data.idt) }

      . = ALIGN(32);
      .data.cacheline_aligned : { *(.data.cacheline_aligned) }

      __bss_start = .;              /* BSS */
      .bss : {
            *(.bss)
            }
      _end = . ;

    /* Output section /DISCARD/ will not be included in the final link output.
     * Refer to Using LD, the GNU linker: Section Definitions */
      /* Sections to be discarded */
      /DISCARD/ : {
            *(.text.exit)
            *(.data.exit)
            *(.exitcall.exit)
            }

    /* The following output sections are addressed at memory location 0.
     * Refer to Using LD, the GNU linker: Optional Section Attributes */
      /* Stabs debugging sections.  */
      .stab 0 : { *(.stab) }
      .stabstr 0 : { *(.stabstr) }
      .stab.excl 0 : { *(.stab.excl) }
      .stab.exclstr 0 : { *(.stab.exclstr) }
      .stab.index 0 : { *(.stab.index) }
      .stab.indexstr 0 : { *(.stab.indexstr) }
      .comment 0 : { *(.comment) }
    }

## 2.3. linux/arch/i386/Makefile

`linux/arch/i386/Makefile` is included by `linux/Makefile` to provide i386 specific items and terms.

All the following targets depend on target vmlinux of `linux/Makefile`. They are accomplished by making corresponding targets in `linux/arch/i386/boot/Makefile` with some options.

Table 1. Targets in linux/arch/i386/Makefile

Target | Command
--- | ---
zImage [a] | @$(MAKE) -C arch/i386/boot zImage [b]
bzImage | @$(MAKE) -C arch/i386/boot bzImage
zlilo | @$(MAKE) -C arch/i386/boot BOOTIMAGE=zImage zlilo
bzlilo | @$(MAKE) -C arch/i386/boot BOOTIMAGE=bzImage zlilo
zdisk | @$(MAKE) -C arch/i386/boot BOOTIMAGE=zImage zdisk
bzdisk | @$(MAKE) -C arch/i386/boot BOOTIMAGE=bzImage zdisk
install | @$(MAKE) -C arch/i386/boot BOOTIMAGE=bzImage install
Notes: a. zImage alias: compressed; b. "-C" is a MAKE command line option to change directory before reading makefiles; Refer to GNU make: Summary of Options and GNU make: Recursive Use of make.

It is worth noticing that this makefile redefines some environment variables which are exported by `linux/Makefile`, specifically:

    OBJCOPY=$(CROSS_COMPILE)objcopy -O binary -R .note -R .comment -S

The effect will be passed to subdirectory makefiles and will change the tool's behavior. Refer to GNU Binary Utilities: objcopy for objcopy command line option details.

Not sure why $(LIBS) includes "$(TOPDIR)/arch/i386/lib/lib.a" twice:

    LIBS := $(TOPDIR)/arch/i386/lib/lib.a $(LIBS) $(TOPDIR)/arch/i386/lib/lib.a

It may be employed to work around linking problems with some toolchains.

## 2.4. linux/arch/i386/boot/Makefile

`linux/arch/i386/boot/Makefile` is somehow independent as it is not included by either `linux/arch/i386/Makefile` or `linux/Makefile`.

However, they do have some relationship:

- `linux/Makefile`: provides resident kernel image `linux/vmlinux`;

- `linux/arch/i386/boot/Makefile`: provides bootstrap;

- `linux/arch/i386/Makefile`: makes sure `linux/vmlinux` is ready before the bootstrap is constructed, and exports targets (like bzImage) to `linux/Makefile`.

$(BOOTIMAGE) value, which is for target zdisk, zlilo or zdisk, comes from `linux/arch/i386/Makefile`.

Table 2. Targets in linux/arch/i386/boot/Makefile

Target | Command
--- | ---
zImage |
    $(OBJCOPY) compressed/vmlinux compressed/vmlinux.out
    tools/build bootsect setup compressed/vmlinux.out $(ROOT_DEV) > zImage
bzImage |
    $(OBJCOPY) compressed/bvmlinux compressed/bvmlinux.out
    tools/build -b bbootsect bsetup compressed/bvmlinux.out $(ROOT_DEV) \
            > bzImage
zdisk |
    dd bs=8192 if=$(BOOTIMAGE) of=/dev/fd0
zlilo |
    if [ -f $(INSTALL_PATH)/vmlinuz ]; then mv $(INSTALL_PATH)/vmlinuz
            $(INSTALL_PATH)/vmlinuz.old; fi
    if [ -f $(INSTALL_PATH)/System.map ]; then mv $(INSTALL_PATH)/System.map
            $(INSTALL_PATH)/System.old; fi
    cat $(BOOTIMAGE) > $(INSTALL_PATH)/vmlinuz
    cp $(TOPDIR)/System.map $(INSTALL_PATH)/
    if [ -x /sbin/lilo ]; then /sbin/lilo; else /etc/lilo/install; fi
install |
    sh -x ./install.sh $(KERNELRELEASE) $(BOOTIMAGE) $(TOPDIR)/System.map
            "$(INSTALL_PATH)"

tools/build builds boot image zImage from {bootsect, setup, compressed/vmlinux.out}, or bzImage from {bbootsect, bsetup, compressed/bvmlinux,out}. `linux/Makefile` "export ROOT_DEV = CURRENT". Note that $(OBJCOPY) has been redefined by `linux/arch/i386/Makefile` in Section 2.3.

Table 3. Supporting targets in linux/arch/i386/boot/Makefile

Target: Prerequisites | Command
--- | ---
compressed/vmlinux: linux/vmlinux | @$(MAKE) -C compressed vmlinux
compressed/bvmlinux: linux/vmlinux | @$(MAKE) -C compressed bvmlinux
tools/build: tools/build.c | $(HOSTCC) $(HOSTCFLAGS) -o $@ $< -I$(TOPDIR)/include [a]
bootsect: bootsect.o | $(LD) -Ttext 0x0 -s --oformat binary bootsect.o [b]
bootsect.o: bootsect.s | $(AS) -o $@ $<
bootsect.s: bootsect.S ... | $(CPP) $(CPPFLAGS) -traditional $(SVGA_MODE) $(RAMDISK) $< -o $@
bbootsect: bbootsect.o | $(LD) -Ttext 0x0 -s --oformat binary $< -o $@
bbootsect.o: bbootsect.s | $(AS) -o $@ $<
bbootsect.s: bootsect.S ... | $(CPP) $(CPPFLAGS) -D__BIG_KERNEL__ -traditional $(SVGA_MODE) $(RAMDISK) $< -o $@
setup: setup.o | $(LD) -Ttext 0x0 -s --oformat binary -e begtext -o $@ $<
setup.o: setup.s | $(AS) -o $@ $<
setup.s: setup.S video.S ... | $(CPP) $(CPPFLAGS) -D__ASSEMBLY__ -traditional $(SVGA_MODE) $(RAMDISK) $< -o $@
bsetup: bsetup.o | $(LD) -Ttext 0x0 -s --oformat binary -e begtext -o $@ $<
bsetup.o: bsetup.s | $(AS) -o $@ $<
bsetup.s: setup.S video.S ... | $(CPP) $(CPPFLAGS) -D__BIG_KERNEL__ -D__ASSEMBLY__ -traditional $(SVGA_MODE) $(RAMDISK) $< -o $@
Notes: a. "$@" means target, "$<" means first prerequisite; Refer to GNU make: Automatic Variables; b. "--oformat binary" asks for raw binary output, which is identical to the memory dump of the executable; Refer to Using LD, the GNU linker: Command Line Options.

Note that it has "-D__BIG_KERNEL__" when compile `bootsect.S` to `bbootsect.s`, and `setup.S` to `bsetup.s`. They must be Place Independent Code (PIC), thus what "-Ttext" option is doesn't matter.

## 2.5. linux/arch/i386/boot/compressed/Makefile

This makefile handles image (de)compression mechanism.

It is good to separate (de)compression from bootstrap. This divide-and-conquer solution allows us to easily improve (de)compression mechanism or to adopt a new bootstrap method.

Directory `linux/arch/i386/boot/compressed/` contains two source files: `head.S` and `misc.c`.

Table 4. Targets in linux/arch/i386/boot/compressed/Makefile

Target | Command
--- | ---
vmlinux[a] | $(LD) -Ttext 0x1000 -e startup_32 -o vmlinux head.o misc.o piggy.o
bvmlinux | $(LD) -Ttext 0x100000 -e startup_32 -o bvmlinux head.o misc.o piggy.o
head.o | $(CC) $(AFLAGS) -traditional -c head.S
misc.o |
    $(CC) $(CFLAGS) -DKBUILD_BASENAME=$(subst $(comma),_,$(subst -,_,$(*F)))
            -c misc.c[b]
piggy.o |
    tmppiggy=_tmp_$$$$piggy; \
    rm -f $$tmppiggy $$tmppiggy.gz $$tmppiggy.lnk; \
    $(OBJCOPY) $(SYSTEM) $$tmppiggy; \
    gzip -f -9 < $$tmppiggy > $$tmppiggy.gz; \
    echo "SECTIONS { .data : { input_len = .; \
            LONG(input_data_end - input_data) input_data = .; \
            *(.data) input_data_end = .; }}" > $$tmppiggy.lnk; \
    $(LD) -r -o piggy.o -b binary $$tmppiggy.gz -b elf32-i386 \
            -T $$tmppiggy.lnk; \
    rm -f $$tmppiggy $$tmppiggy.gz $$tmppiggy.lnk
Notes: a. Target vmlinux here is different from that defined in `linux/Makefile`; b. "subst" is a MAKE function; Refer to GNU make: Functions for String Substitution and Analysis.

`piggy.o` contains variable input_len and gzipped `linux/vmlinux`. input_len is at the beginning of `piggy.o`, and it is equal to the size of `piggy.o` excluding input_len itself. Refer to Using LD, the GNU linker: Section Data Expressions for "LONG(expression)" in piggy.o linker script.

To be exact, it is not `linux/vmlinux` itself (in ELF format) that is gzipped but its binary image, which is generated by objcopy command. Note that $(OBJCOPY) has been redefined by `linux/arch/i386/Makefile` in Section 2.3 to output raw binary using "-O binary" option.

When linking {bootsect, setup} or {bbootsect, bsetup}, $(LD) specifies "--oformat binary" option to output them in binary format. When making zImage (or bzImage), $(OBJCOPY) generates an intermediate binary output from compressed/vmlinux (or compressed/bvmlinux) too. It is vital that all components in zImage or bzImage are in raw binary format, so that the image can run by itself without asking a loader to load and relocate it.

Both vmlinux and bvmlinux prepend `head.o` and `misc.o` before `piggy.o`, but they are linked against different start addresses (0x1000 vs 0x100000).

## 2.6. linux/arch/i386/tools/build.c

`linux/arch/i386/tools/build.c` is a host utility to generate zImage or bzImage.

In `linux/arch/i386/boot/Makefile`:

    tools/build bootsect setup compressed/vmlinux.out $(ROOT_DEV) > zImage

    tools/build -b bbootsect bsetup compressed/bvmlinux.out $(ROOT_DEV) > bzImage

"-b" means is_big_kernel, used to check whether system image is too big.

tools/build outputs the following components to stdout, which is redirected to zImage or bzImage:

1. bootsect or bbootsect: from `linux/arch/i386/boot/bootsect.S`, 512 bytes;

2. setup or bsetup: from `linux/arch/i386/boot/setup.S`, 4 sectors or more, sector aligned;

3. compressed/vmlinux.out or compressed/bvmlinux.out, including:

   1. head.o: from `linux/arch/i386/boot/compressed/head.S`;

   2. misc.o: from `linux/arch/i386/boot/compressed/misc.c`;

   3. piggy.o: from input_len and gzipped `linux/vmlinux`.

tools/build will change some contents of bootsect or bbootsect when outputting to stdout:

Table 5. Modification made by tools/build

Offset | Byte | Variable | Comment
--- | --- | --- | ---
1F1 (497) | 1 | setup_sectors | number of setup sectors, >=4
1F4 (500) | 2 | sys_size | system size in 16-bytes, little-endian
1FC (508) | 1 | minor_root | root dev minor
1FD (509) | 1 | major_root | root dev major

In the following chapters, compressed/vmlinux will be referred as vmlinux and compressed/bvmlinux as bvmlinux, if not confusing.

## 2.7. Reference

- Linux Kernel Makefiles: `linux/Documentation/kbuild/makefiles.txt`

- The Linux Kernel HOWTO

- GNU make

- Using LD, the GNU linker

- GNU Binary Utilities

- GNU Bash

Prev | Home | Next
Introduction |  | linux/arch/i386/boot/bootsect.S