Navigate to Lab Session on AUGUSTUS. Using Scipio. Predicting Genes. AUGUSTUS-PPX.
Show all / no details.

# Training AUGUSTUS

This manual is intended for those who want to retrain on their own ab initio AUGUSTUS for another species. Please do not rely on this manual and the scripts and programs. Check what they do on your data!

## 1. COMPILE A SET OF TRAINING AND TEST GENES

You will need a set of genomic sequences with bona fide gene structures (sequence coordinates of starts and ends of exons and genes). In many cases, or as a first step towards modelling complete genes, it is sufficient to have only the coding parts of the gene structure (CDS).
[+] Number and quality of gene structures...
As a minimum for good performance you should have 200 gene structures for training. The more genes the training set comprises the better, however the incremental benefit of additional genes becomes very small at 1000 genes already. As a rule of thumb, if you have more than 1000 genes then rather prefer quality (fewer false annotations) over quantity. Make also sure that in particular the number of multi-exon genes is large. These are needed to train the introns. The gene structures of these genes should be as accurate as possible. However, it is not necessary that they are 100% correct, neither does the annotation have to be necessarily complete. It is more imporant that the start codon is correctly annotated than that the stop codon is correctly annotated.
[+] The gene set should be non-redundant...
Non-redundancy is important in particular if you plan to use part of it for training and another part for assessing prediction accuracy. If in two different sequences genes with an almost identical amino acid sequence is annotated then delete one of them. My criterion is: No two genes in the set are more than 80% identical on the amino acid level. The non-redundancy is very important for avoiding overfitting. This is of course also very important if you want to test the accuracy on a test set.
Each sequence can contain one or more genes, the genes can be on either strand. However, the genes must not overlap, and only one transcript per gene is allowed. Store the sequences together with their annotation in a simple genbank format. For the exact format that the training program can read in look as an example at one of the training genbank files at the augustus web server: http://augustus.gobics.de/datasets/

### 1.1 Options for compiling a set of gene structures

1. pre-existing gene structures (e.g. from GenBank)
2. spliced alignments of ESTs against the assembled genomic sequence (e.g. using PASA)
3. spliced alignments of de novo assembled transcriptome short reads (RNA-Seq)
4. spliced alignments of protein sequences of the same or a very closely related species against the assembled genomic sequence, e.g. using Scipio.
   This approach is described in section Using Scipio to create a training set.
5. gene structures from a related species
6. iteration of training with predicted genes, starting with an existing parameter set

### 1.2 Split gene structure set into training and test set

Randomly split the set of annotated sequences in a training and a test set.

    randomSplit.pl genes.gb 100

This generates a file `genes.gb.test` with 100 randomly chosen loci and a disjoint file `genes.gb.train` with the rest of the loci from `genes.gb`:

    grep -c LOCUS genes.gb*
    # genes.gb:586
    # genes.gb.test:100
    # genes.gb.train:486

In order for the test accuracy to be statistically meaningful the test set should also be large enough (100-200 genes). You should split the set of gene structures really randomly! Do not just take the first and the last part of the file as then the test set is unlikely to be representative. The script `randomSplit.pl` is in the scripts directory.
[+] Additional splice site training set...
In rare cases it may be useful to specify a set of splice site sequences that should be used for training in addition to the complete genes. This would apply, e.g. if the number of intron examples in your training set is small but you have additional intron example, e.g. from spliced alignments of ESTs.
The format is as in the following example.

    dss gccgagaactccgctcgttctgtgcgttctcctgtcccaggtagggaagaggggctgccgggcgcgctctgcgccccgtttc
    dss cgtgattgtcggggggaaagacatccagggctccttgcaggtaacacatctgtttgagataacttgggttcaaggaggacat
    dss agagaatcagagacagcctttcccaagagatgttggcaaggtaagtcagacaaacagcaaatgacaaaaacatgtttttatg
    dss cattgtcactgttgtgtcacctgcgctgctggaccgagaggtgagctgaaaagaataccactttctttttcacgagaataga
    dss tgacaaaaatgatcactcaccaaaattcaccaagaaagaggtaaacccctgtgccaaacaccaaccaccactgtggtcacag
    ass gttagtatgcttctttaattttttttctccctgaaattataggaaccagatgttaaaaaattagaagaccaacttcaaggcg
    ass --------------------------ggctttgtctttgcagaatttatagagcggcagcacgcaaagaacaggtattacta
    ass gattccttgtgattagcctctcttgctccttttctccaccagcaaagtcgaccaagaaattatcaacattatgcaggatcgg
    ass aaccgtagtaaacagcatgaatcgtgttttgtttttgaacagaccactggccttgtgggattggctgtgtgcaatactcctc

dss: donor (=5') splice site. 40 letters + gt + 40 letters
ass: donor (=3') splice site. 40 letters + ag + 40 letters
use '-' for unknown characters

## 2. CREATE A META PARAMETERS FILE FOR YOUR SPECIES

We call parameters like the size of the window of the splice site models and the order of the Markov model meta parameters, in contrast to parameters like the distribution of splice site patterns, the k-mer probabilities of coding and noncoding regions. There are a few dozen meta parameters but many thousands of parameters. The meta parameters determine how the parameters are calculated.
Create the files for training "`bug`" from a template.

    new_species.pl --species=bug

`new_species.pl` uses the environment variable `AUGUSTUS_CONFIG_PATH` to determine the directory in which AUGUSTUS stores the species parameters. You should see a report like this:

    creating directory /home/mario/augustus/trunk/config/species/bug/ ...
    creating /home/mario/augustus/trunk/config/species/bug/bug_parameters.cfg ...
    creating /home/mario/augustus/trunk/config/species/bug/bug_weightmatrix.txt ...
    creating /home/mario/augustus/trunk/config/species/bug/bug_metapars.cfg ...
    ...

The file `bug_parameters.cfg` contains besides meta-parameters also parameters to `augustus` and `etraining` like defaults for output format settings. Edit `bug_parameters.cfg` and set `stopCodonExcludedFromCDS` to true.
[+] The *_parameters.cfg file...
contains output format options like

`protein` | turn on/off the inclusion of predicted peptide sequences in the output
`codingseq` | turn on/off the inclusion of predicted coding sequences in the output

and settings like

`alternatives-from-evidence` | turn on/of prediction of alternative transcripts based on hints
`UTR` | turn on/off the prediction of untranslated regions

All of the parameters set in this file can equally be given on the command line, e.g.

    augustus --species=bug --protein=on input.fa

in which case they override the setting in this file. The parameters at the top of the file can be edited as desired or required for the species.

The optional file with the filename given by /IntronModel/splicefile may contain a list of sequence windows of known splice sites as described above.
For more info have a look at the comments in the file and at the `README.TXT` that is included with AUGUSTUS.

[+] Distinguishing GC content classes...
This section describes the meaning of the meta parameter /Constant/decomp_num_steps and the matrix in the file `bug_weightmatrix.txt` and you can savely skip this on first training.
For some species like humand and honey bee it makes sense to let the model parameters depend on the average frequencies of the 4 bases in the query sequence. For example, in human, the GC content stays consistently above or below average over long sequence stretches (isochores). AUGUSTUS can locally use different parameters that are adjusted to the base composition of this piece. I describe here only the dependency on the GC content.
`/Constant/decomp_num_steps` is the number of different levels of GC content that is taken into account, i.e. AUGUSTUS uses `/Constant/decomp_num_steps` different sets of parameters, each for a different GC content. Values between 1 and 10 are useful. The GC content ranges between `/Constant/gc_range_min` and `/Constant/gc_range_max`. These parameters can be set in the meta parameters file. Given a target GC content, each sequence in the training set is weighted depending on how similar its GC content is to the target GC content. It gets a weight that is the higher, the closer the GC content of the training sequence is to the target GC content.
The two non-zero numbers in the middle of the 4x4 Matrix in bug_weightmatrix.txt (default: 200) determine the influence of the deviation in GC content. A high value (like 300) means that mostly training sequences with a very similar GC content to the target GC content are taken into account. The advantage is that the training is more specific to the target GC content. The disadvantage of a high value is that this effectively reduces the size of the training set. A low value (like 150) means that all training sequences are taken into account, except that training sequences with a similar GC content are weighed somewhat stronger. Editing the weightmatrix file is not recommended for normal users.

## 3. MAKE AN INITIAL TRAINING

This step may be skipped if step 4 below is done. However, in a semi-automatic setting (you type the commands in this document) it is not recommended to skip it. Train augustus for the species `bug` on the training set of gene structures.

    etraining --species=bug genes.gb.train

This creates/updates parameter files for exon, intron and intergenic region in the directory $AUGUSTUS_CONFIG_PATH/species/bug.

    ls -ort $AUGUSTUS_CONFIG_PATH/species/bug/

now yields

    -rw-r--r-- 1 mario    810 2011-01-08 10:50 bug_weightmatrix.txt
    -rw-r--r-- 1 mario    810 2011-01-08 10:50 bug_weightmatrix.txt
    -rw-r--r-- 1 mario   1355 2011-01-08 10:50 bug_metapars.utr.cfg
    -rw-r--r-- 1 mario   2057 2011-01-08 10:50 bug_metapars.cfg
    -rw-r--r-- 1 mario   7156 2011-01-08 22:05 bug_parameters.cfg
    -rw-r--r-- 1 mario 100100 2011-01-08 22:35 bug_intron_probs.pbl
    -rw-r--r-- 1 mario 234355 2011-01-08 22:35 bug_exon_probs.pbl
    -rw-r--r-- 1 mario  32588 2011-01-08 22:35 bug_igenic_probs.pbl

where `bug_{intron,exon,igenic}.pbl` are our newly created parameter files.
Now we make a first try and predict the genes in `genes.gb.train` ab initio.

    augustus --species=bug genes.gb.test | tee firsttest.out # takes ~1m

This predicts the genes in all 100 sequences and will at the end print a report about the prediction accuracy, comparing the strucures in the input file `genes.gb.test` with the ones predicted. Of course, for the predictions only the sequences are used, not the input gene structures.
Look at the accuracy report at the end of `firsttest.out`:

    grep -A 22 Evaluation firsttest.out

    *******      Evaluation of gene prediction     *******

    ---------------------------------------------\
                     | sensitivity | specificity |
    ---------------------------------------------|
    nucleotide level |       0.975 |        0.89 |
    ---------------------------------------------/

    ----------------------------------------------------------------------------------------------------------\
               |  #pred |  #anno |      |    FP = false pos. |    FN = false neg. |             |             |
               | total/ | total/ |   TP |--------------------|--------------------| sensitivity | specificity |
               | unique | unique |      | part | ovlp | wrng | part | ovlp | wrng |             |             |
    ----------------------------------------------------------------------------------------------------------|
               |        |        |      |                115 |                 76 |             |             |
    exon level |    511 |    472 |  396 | ------------------ | ------------------ |       0.839 |       0.775 |
               |    511 |    472 |      |   40 |    5 |   70 |   43 |    3 |   30 |             |             |
    ----------------------------------------------------------------------------------------------------------/

    ----------------------------------------------------------------------------\
    transcript | #pred | #anno |   TP |   FP |   FN | sensitivity | specificity |
    ----------------------------------------------------------------------------|
    gene level |   118 |   100 |   53 |   65 |   47 |        0.53 |       0.449 |
    ----------------------------------------------------------------------------/

These numbers mean, for example, that

of the 100 genes 53 were predicted exactly
83.9% of the exons were predicted exactly
77.5% of the predicted exons were exactly as in the test set.

[+] Compare with the shipped fly parameters...
The `fly` parameters that come with augustus yield a somewhat better accuracy:

    augustus --species=fly --stopCodonExcludedFromCDS=true genes.gb.test --UTR=off \
       | grep -A 22 Evaluation # takes ~1m

gives the following accuracy report:

    *******      Evaluation of gene prediction     *******

    ---------------------------------------------\
                     | sensitivity | specificity |
    ---------------------------------------------|
    nucleotide level |       0.987 |       0.896 |
    ---------------------------------------------/

    ----------------------------------------------------------------------------------------------------------\
               |  #pred |  #anno |      |    FP = false pos. |    FN = false neg. |             |             |
               | total/ | total/ |   TP |--------------------|--------------------| sensitivity | specificity |
               | unique | unique |      | part | ovlp | wrng | part | ovlp | wrng |             |             |
    ----------------------------------------------------------------------------------------------------------|
               |        |        |      |                 85 |                 45 |             |             |
    exon level |    512 |    472 |  427 | ------------------ | ------------------ |       0.905 |       0.834 |
               |    512 |    472 |      |   29 |    2 |   54 |   30 |    1 |   14 |             |             |
    ----------------------------------------------------------------------------------------------------------/

    ----------------------------------------------------------------------------\
    transcript | #pred | #anno |   TP |   FP |   FN | sensitivity | specificity |
    ----------------------------------------------------------------------------|
    gene level |   105 |   100 |   67 |   38 |   33 |        0.67 |       0.638 |
    ----------------------------------------------------------------------------/

## 4. RUN THE SCRIPT `optimize_augustus.pl`

This script optimizes the prediction accuracy by adjusting the meta parameters in the `*_parameters.cfg` file. The script alternatingly used the programs `augustus` and `etraining`. This ususally increases prediction accuracy by a few percent points, but runs for hours or days. It may be skipped and only etraining be run once (step 3. above), which is very quick. `augustus` and `etraining` must be in the `$PATH`.

You need to tell optimize_augustus.pl, which metaparameters it should optimize. Do this by adjusting the file `config/species/generic/generic_metapars.cfg`. (You may also make a copy of it and then use the command line parameter --metapars=nameofmycopy to the script optimize_augustus.pl.)
Run

    optimize_augustus.pl --species=bug genes.gb.train  # takes ~1d

[+] What `optimize_augustus.pl` does...

In an evaluation step this script does the following 10 fold cross validation. It splits the set `genes.gb.train` randomly into 10 sets (buckets) of ±1 equal size. Then takes 9 of the 10 sets for training and the other one for evaluating the prediction accuracy using the annotation of the genbank file. The 10 possible buckets for evaluating are rotated and each case the other sequences are used for training.

On each prediction it computes a single target value that is subject to optimization. The target is a a weighted average of the sensitivities and specificities on the base, exon and gene level. (Feel free to change the weighing to your preferences in the function 'sub gettarget'.)

For a meta parameter the script repeats above evaluation step for different values of the meta parameter. If it finds an improvement in the target value it adjusts the value of the meta parameter in your `bug_parameters.cfg` file. It tries new values for the parameter until it finds no more improvement. Then it optimizes the next meta parameter. When it has optimized all meta parameters once, it repeats with the first meta parameter.

It does at most 5 rounds of optimizations, but stops earlier if no improvements are found. It is possible to interrupt the script of you cannot wait for it to finish. This script probably has to run over night with `nohup` or `screen`. When it is done your `bug_parameters.cfg` should have better suited meta parameters for your species and your training set.

After `optimize_augustus.pl` has finished or (after you have interrupted it) you should (re)train AUGUSTUS with the meta parameters it has set.

    etraining --species=bug genes.gb.train

If you have a test set, you can now check the prediction accuracy on this test set by running

    augustus --species=bug genes.test.gb

The end of the output will then contain a summary of the accuracy of the prediction. If the gene level sensitivity is below 20% it is likely that the training set is not large enough, that it doesn't have a good quality or that the species is somehow 'special'.
If you succeeded in creating a good AUGUSTUS version for your species I would be very interested in your results. If possible please share your results by giving me the packed `config/yourspecies` folder.

4. SPECIAL CASE: ORGANISM WITH DIFFERENT GENETIC CODE

AUGUSTUS can be told to use a different translation table, in particular one with a different set of stop codons. This is useful for a small number of species such as Tetrahymena thermophilia, in which some codons translate to a different amino acid than usual. If you train AUGUSTUS for such a species set the variable translation_table in the parameter file of your species. Further, adjust the stop codon probabilities in the same config file. E.g. say

translation_table 6
/Constant/amberprob 0 # Prob(stop codon = tag), if 0 tag is assumed to code for amino acid
/Constant/ochreprob 0 # Prob(stop codon = taa), if 0 taa is assumed to code for amino acid
/Constant/opalprob 1 # Prob(stop codon = tga), if 0 tga is assumed to code for amino acid

in the case of Tetrahymena, where `taa` and `tag` are coding for glutamine (Q).

Choose the translation table number accoding to this table. translation_table=1 is the default value and the standard with stop codons taa, tga, tag. If you have a species with the standard genetic code you don't have to do anything. In case your species' code is not covered by this table send us a note with the string of 64 one-letter amino acid codes in the codon order below.

translation | a | a | a | a | a | a | a | a | a | a | a | a | a | a | a | a | c | c | c | c | c | c | c | c | c | c | c | c | c | c | c | c | g | g | g | g | g | g | g | g | g | g | g | g | g | g | g | g | t | t | t | t | t | t | t | t | t | t | t | t | t | t | t | t
table | a | a | a | a | c | c | c | c | g | g | g | g | t | t | t | t | a | a | a | a | c | c | c | c | g | g | g | g | t | t | t | t | a | a | a | a | c | c | c | c | g | g | g | g | t | t | t | t | a | a | a | a | c | c | c | c | g | g | g | g | t | t | t | t
number | a | c | g | t | a | c | g | t | a | c | g | t | a | c | g | t | a | c | g | t | a | c | g | t | a | c | g | t | a | c | g | t | a | c | g | t | a | c | g | t | a | c | g | t | a | c | g | t | a | c | g | t | a | c | g | t | a | c | g | t | a | c | g | t
1 | K | N | K | N | T | T | T | T | R | S | R | S | I | I | M | I | Q | H | Q | H | P | P | P | P | R | R | R | R | L | L | L | L | E | D | E | D | A | A | A | A | G | G | G | G | V | V | V | V | * | Y | * | Y | S | S | S | S | * | C | W | C | L | F | L | F
2 | K | N | K | N | T | T | T | T | * | S | * | S | M | I | M | I | Q | H | Q | H | P | P | P | P | R | R | R | R | L | L | L | L | E | D | E | D | A | A | A | A | G | G | G | G | V | V | V | V | * | Y | * | Y | S | S | S | S | W | C | W | C | L | F | L | F
3 | K | N | K | N | T | T | T | T | R | S | R | S | M | I | M | I | Q | H | Q | H | P | P | P | P | R | R | R | R | T | T | T | T | E | D | E | D | A | A | A | A | G | G | G | G | V | V | V | V | * | Y | * | Y | S | S | S | S | W | C | W | C | L | F | L | F
4 | K | N | K | N | T | T | T | T | R | S | R | S | I | I | M | I | Q | H | Q | H | P | P | P | P | R | R | R | R | L | L | L | L | E | D | E | D | A | A | A | A | G | G | G | G | V | V | V | V | * | Y | * | Y | S | S | S | S | W | C | W | C | L | F | L | F
5 | K | N | K | N | T | T | T | T | S | S | S | S | M | I | M | I | Q | H | Q | H | P | P | P | P | R | R | R | R | L | L | L | L | E | D | E | D | A | A | A | A | G | G | G | G | V | V | V | V | * | Y | * | Y | S | S | S | S | W | C | W | C | L | F | L | F
6 | K | N | K | N | T | T | T | T | R | S | R | S | I | I | M | I | Q | H | Q | H | P | P | P | P | R | R | R | R | L | L | L | L | E | D | E | D | A | A | A | A | G | G | G | G | V | V | V | V | Q | Y | Q | Y | S | S | S | S | * | C | W | C | L | F | L | F
9 | N | N | K | N | T | T | T | T | S | S | S | S | I | I | M | I | Q | H | Q | H | P | P | P | P | R | R | R | R | L | L | L | L | E | D | E | D | A | A | A | A | G | G | G | G | V | V | V | V | * | Y | * | Y | S | S | S | S | W | C | W | C | L | F | L | F
10 | K | N | K | N | T | T | T | T | R | S | R | S | I | I | M | I | Q | H | Q | H | P | P | P | P | R | R | R | R | L | L | L | L | E | D | E | D | A | A | A | A | G | G | G | G | V | V | V | V | * | Y | * | Y | S | S | S | S | C | C | W | C | L | F | L | F
11 | K | N | K | N | T | T | T | T | R | S | R | S | I | I | M | I | Q | H | Q | H | P | P | P | P | R | R | R | R | L | L | L | L | E | D | E | D | A | A | A | A | G | G | G | G | V | V | V | V | * | Y | * | Y | S | S | S | S | * | C | W | C | L | F | L | F
12 | K | N | K | N | T | T | T | T | R | S | R | S | I | I | M | I | Q | H | Q | H | P | P | P | P | R | R | R | R | L | L | S | L | E | D | E | D | A | A | A | A | G | G | G | G | V | V | V | V | * | Y | * | Y | S | S | S | S | * | C | W | C | L | F | L | F
13 | K | N | K | N | T | T | T | T | G | S | G | S | M | I | M | I | Q | H | Q | H | P | P | P | P | R | R | R | R | L | L | L | L | E | D | E | D | A | A | A | A | G | G | G | G | V | V | V | V | * | Y | * | Y | S | S | S | S | W | C | W | C | L | F | L | F
14 | N | N | K | N | T | T | T | T | S | S | S | S | I | I | M | I | Q | H | Q | H | P | P | P | P | R | R | R | R | L | L | L | L | E | D | E | D | A | A | A | A | G | G | G | G | V | V | V | V | Y | Y | * | Y | S | S | S | S | W | C | W | C | L | F | L | F
15 | K | N | K | N | T | T | T | T | R | S | R | S | I | I | M | I | Q | H | Q | H | P | P | P | P | R | R | R | R | L | L | L | L | E | D | E | D | A | A | A | A | G | G | G | G | V | V | V | V | * | Y | Q | Y | S | S | S | S | * | C | W | C | L | F | L | F
16 | K | N | K | N | T | T | T | T | R | S | R | S | I | I | M | I | Q | H | Q | H | P | P | P | P | R | R | R | R | L | L | L | L | E | D | E | D | A | A | A | A | G | G | G | G | V | V | V | V | * | Y | L | Y | S | S | S | S | * | C | W | C | L | F | L | F
21 | N | N | K | N | T | T | T | T | S | S | S | S | M | I | M | I | Q | H | Q | H | P | P | P | P | R | R | R | R | L | L | L | L | E | D | E | D | A | A | A | A | G | G | G | G | V | V | V | V | * | Y | * | Y | S | S | S | S | W | C | W | C | L | F | L | F
22 | K | N | K | N | T | T | T | T | R | S | R | S | I | I | M | I | Q | H | Q | H | P | P | P | P | R | R | R | R | L | L | L | L | E | D | E | D | A | A | A | A | G | G | G | G | V | V | V | V | * | Y | L | Y | * | S | S | S | * | C | W | C | L | F | L | F
23 | K | N | K | N | T | T | T | T | R | S | R | S | I | I | M | I | Q | H | Q | H | P | P | P | P | R | R | R | R | L | L | L | L | E | D | E | D | A | A | A | A | G | G | G | G | V | V | V | V | * | Y | * | Y | S | S | S | S | * | C | W | C | * | F | L | F