[#continuous-integration]
== Continuous integration

Because of Eldev’s trivial installation and built-in support for
testing, it is a suitable tool for use on continuous integration
servers.  But of course this only applies if the test framework your
project uses is already supported (currently <<ert,ERT>>,
<<buttercup,Buttercup>>, <<doctest,Doctest>> and <<ecukes,Ecukes>>).

Eldev will even try to make your CI runs <<robust-mode,more
reliable>>.

[#github-workflows]
=== GitHub workflows

The easiest option for continuous integration for GitHub-hosted
projects are {uri-github-wflows}[GitHub workflows], as this doesn’t
involve using a 3rd-party service.  Probably most of Elisp projects
can take advantage of this, since GitHub appears to be the most
popular hosting for Elisp projects.  Workflow definition files for
GitHub are somewhat more verbose than for <<travis-ci,Travis CI>>, but
ultimately not really more complicated.

The easiest way to install Emacs binary of appropriate version is to
use {uri-setup-emacs}[`jcs090218/setup-emacs`] action (which
internally uses {uri-nix-emacs}[nix-emacs-ci]).  There are other
`setup-emacs` actions around, but this one works across all operating
systems.  Since {uri-evm}[EVM] seems tuned to Ubuntu Trusty (i.e. what
Travis CI provides), it is likely unsuitable for GitHub workflows.

There is a simple {uri-setup-eldev}[action called `setup-eldev`] too.
It works on all GitHub-supported operating systems — Linux, macOS and
Windows — as well.

A basic workflow file (you can e.g. name it
`.github/workflows/test.yml`) would look something like this:

....
name: CI

on:
  push:
    paths-ignore:
      - '**.md'
  pull_request:
    paths-ignore:
      - '**.md'

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        emacs_version:
          # Add more lines like this if you want to test on different Emacs versions.
          - 26.3

    steps:
    - name: Set up Emacs
      uses: jcs090218/setup-emacs@master
      with:
        version: ${{matrix.emacs_version}}

    - name: Install Eldev
      uses: emacs-eldev/setup-eldev@v1

    - name: Check out the source code
      uses: actions/checkout@v4

    - name: Test the project
      run: |
        eldev -p -dtT test
....

Eldev’s terminal autorecognition doesn’t work on GitHub machines
(unlike e.g. on Travis CI).  If you want colored output from Eldev,
you need to explicitly enable it using `-C` (`--color`) global option.

[#travis-ci]
=== Travis CI

{uri-travis}[Travis CI] is perhaps the most used continuous
integration service for Elisp code, at least until the addition of
<<github-workflows,GitHub workflows>>.  The largest problem on Travis
CI is to install Emacs binary of the desired version.  Luckily, there
are tools that can be used for this: at least {uri-evm}[EVM] and
{uri-nix-emacs}[nix-emacs-ci].

[#evm]
==== EVM

One of the tools to install Emacs is {uri-evm}[EVM].  Steve Purcell
(the author of `nix-emacs-ci`) mentions “various issues” he has had
with it, however many projects use it.  Apparently, you need to fix
Ubuntu distribution used at Travis CI to Trusty for EVM-provided
binaries.  Also note that EVM provides binaries only for Linux, so if
you want test on macOS too, `nix-emacs-ci` is a better choice.

If you also want to try it, Eldev provides a simple script
specifically for use on Travis CI that installs Eldev and EVM in one
go.  Here is a simple project-agnostic `.travis.yml` file that you can
use as a basis:

....
language: emacs-lisp
dist: trusty

env:
  # Add more lines like this if you want to test on different Emacs versions.
  - EVM_EMACS=emacs-26.3-travis

install:
  - curl -fsSL https://raw.github.com/emacs-eldev/eldev/master/webinstall/travis-eldev-and-evm > x.sh && source ./x.sh
  - evm install $EVM_EMACS --use

script:
  - eldev -p -dtT test
....

==== nix-emacs-ci

A newer tool to install Emacs is {uri-nix-emacs}[nix-emacs-ci].  Using
it is easy: define environment variable `EMACS_CI` with the desired
Emacs version and `curl` a single shell script — whether on Linux or
macOS.  With one more line you can also install Eldev.  It appears to
be slower than EVM, but for continuous integration that’s not terribly
important.

A basic `.travis.yml` would look like this:

....
language: nix

env:
  # Add more lines like this if you want to test on different Emacs versions.
  - EMACS_CI=emacs-26-3

install:
  - bash <(curl https://raw.githubusercontent.com/purcell/nix-emacs-ci/master/travis-install)
  - curl -fsSL https://raw.github.com/emacs-eldev/eldev/master/webinstall/travis-eldev > x.sh && source ./x.sh

script:
  - eldev -p -dtT test
....

[#circle-ci]
=== CircleCI

Another frequently used service is {uri-circle}[CircleCI].  I don’t
know that much about it, presumably {uri-nix-emacs}[nix-emacs-ci] can
be used to install Emacs on it.  Some projects successfully use
{uri-docker}[Docker] images.

Regardless of how you install Emacs, adding Eldev is yet another
one-liner.  It is handy to use, because propagating `PATH`
modifications between different commands on CircleCI is somewhat
non-obvious.  To use it, add the following lines in the relevant place
in file `.circleci/config.yml`:

....
    ...
    - run:
        name: Install Eldev
        command: curl -fsSL https://raw.github.com/emacs-eldev/eldev/master/webinstall/circle-eldev > x.sh && source ./x.sh
....

=== Script commands

Once you have Emacs with Eldev set up on the continuous integration
server of your choice, it is time to actually test your project.  The
most basic command is, naturally, `eldev test`.  You might want to add
a few options to both make project loading more similar to that
typical for your users and Eldev’s output more informative:

    $ eldev -p -dtT test

To make sure that your project byte-compiles cleanly, use the
following command:

    $ eldev -dtT compile --warnings-as-errors

Or maybe even this, if you want to make sure that test `.el` files
also can be byte-compiled without warnings (this can sometimes catch
more problems):

    $ eldev -dtT compile --set all --warnings-as-errors

You can also enforce conformance to certain coding standards by adding
an invocation of `lint` to the `script` part.  Remember, however, that
most linters are continuously being developed.  Even if a linter finds
your source warning-free today, it might detect problems tomorrow.
`relint` is probably one of the “safer” linters in this regard:

    $ eldev -dtT lint re

[#robust-mode]
=== Robust mode

{since-1-5} Eldev can make your continuous integration runs more
_robust_.  This means that when facing certain _externally-caused_
errors, Eldev will not give up immediately, but rather wait for a
while and then retry — several times.  This makes CI runs more
reliable and less likely to fail because of reasons completely
unrelated to your project.

This is a work in progress, as examples of such errors are pretty
difficult to collect, as they are intermittent and happen only
occasionally.  As of 1.5, Eldev will retry if it fails to fetch
contents of a package archive.  In practice such an error has been
observed at least with MELPA and is _probably_ caused by MELPA updates
being non-atomic, meaning that at the end of each archive rebuild,
there is a relatively short period where its current contents is not
properly reported.  Another possibility might be network problems.

In any case, normally you don’t even have to do anything to get these
improvements in Eldev.  They are activated using option
`--robust-mode` (`-R`), which by default has value “auto”.  If this
value is unchanged, robust mode is inactive on normal machines
(e.g. when you run Eldev locally), but gets activated if environment
variable `$CI` is `true`, which appears to be an unwritten standard
for continuous integration servers.  For example, it is set on
{uri-github-wflows-vars}[GitHub workflow servers].  Anyway, you can
also set this option to “always” or “never” — as you prefer.
