= Running Tests
:experimental:

The Clojure ecosystem provides a lot of support for test-driven
development (TDD) and other test-centric patterns. First, Clojure
provides a standardized framework for developing tests called
https://clojure.github.io/clojure/clojure.test-api.html[clojure.test]. Many other testing libraries plug into this
framework. Second, tools like Leiningen create standardized
application and library project structures that provide locations and
idiomatic naming for test code. Finally, CIDER provides several easy
ways to run these tests, view the test results, and quickly jump to
code that is failing to pass a given test.

NOTE: CIDER only supports clojure.test and other libraries
providing integration with clojure.test. Native support
for other libraries might be available via CIDER plugins.

== Basic Usage

CIDER has several functions that help you run all your tests or a
selected subset of them. All of the CIDER test commands are available
in both source code and REPL buffers. In REPL buffers you can also use
kbd:[,] to invoke some of the testing commands.

NOTE: CIDER will only run tests that have been loaded (evaluated). This means
that prior to running some tests you always have to evaluate them first.
As a corollary - if you make some changes to a test, you'll have to re-evaluate it,
so that CIDER will pick up the updated version.

First, you can run all the tests in your project with kbd:[C-c C-t p]
or kbd:[C-c C-t C-p]. It's important to realize that this will
load *all* the namespaces in your project, which might be more than
you're expecting.

You can run all _loaded_ tests with kbd:[C-c C-t l] or
kbd:[C-c C-t C-l].

If you invoke either of these commands with a prefix (e.g. kbd:[C-u
C-c C-t l]), CIDER will prompt for test selector filters and only run
those tests that match the selector inclusions/exclusions.

.Test Selectors
****
Test developers use selectors to define subsets of the total test
suite that are run together for different testing tasks. For example
you can mark some of your tests with the `+^:smoke+` metadata marker
and others with `+^:integration+`. This enables you to run these tests
separately in your build pipeline.  CIDER helps you to run these same
test subsets in your development environment.

Test selectors were originally a `leiningen` feature and you can get
more information by executing:

[source,sh]
----
$ lein help test
----
****

You can run all the tests in the current namespace, whether specified
by a source file or by the REPL, using kbd:[C-c C-t n] or
kbd:[C-c C-t C-n]. Note that it's idiomatic for Clojure projects
to locate tests in a separate namespace than the code that is being
tested. CIDER uses a simple algorithm to figure out where the tests
are located. The algorithm works as follows:

* If you're in an implementation namespace (e.g. `some.ns`), CIDER will try to find a
matching test namespace (by default `some.ns-test`) and run the tests
there.
* If you're in something that already looks like a test
namespace (e.g. `some.ns-test`), CIDER will simply run the tests in
that namespace.

TIP: If you have put some of your tests into your implementation namespace, using
`clojure.test/with-test`, for instance, you might want to suppress the namespace
inference logic and force CIDER to run tests in the current namespace
unconditionally.  You can do this by adding a prefix to the namespace commands:
kbd:[C-u C-c C-t C-n]. This will simply run whatever tests are present in the
currently visited or active namespace.

You can also run a subset of the tests defined in the namespace,
filtered by test selectors, using kbd:[C-c C-t C-s]. CIDER will
prompt for the selectors in the minibuffer. If you call this
command with a prefix (kbd:[C-u C-c C-t C-s]) you can suppress
the namespace inference logic as for kbd:[C-u C-c C-t C-n]

Finally, you can execute the specific test at the point using
kbd:[C-c C-t t] or kbd:[C-c C-t C-t]. It will also work for implementation functions,
by searching for a matching test namespace with a matching deftest name. 

== Configuration

You can configure CIDER's test execution behavior in multiple ways.

=== Fail-fast

Starting from 1.8.0, CIDER has standard fail-fast functionality,
controlled by the `cider-test-fail-fast` defcustom (default `t`).

You can toggle its value for the current buffer using `cider-test-toggle-fail-fast` (`C-c C-t f` or `C-c C-t C-f`)

NOTE: fail-fast will never be chosen for the "retest" functionality,
since that would cause you to lose the majority of the tests
that previously failed.

=== Test Namespace Naming Convention

If your tests are not following the `some.ns-test` naming convention
you can set the variable `cider-test-infer-test-ns` to a function that
takes the current namespace and returns the matching test namespace
(which may be the same as the current namespace). This provides
complete flexibility to structure your test suite using whatever
conventions you might want. Here's how you can configure the test running
to look for `some.test-ns` instead of `some.ns-test`:

[source,lisp]
----
(defun cider-custom-test-ns-fn (ns)
  "For a NS, return the test namespace, which may be the argument itself.
This uses the convention of prepending 'test-' to the namespace name."
  (when ns
    (let ((prefix "test-"))
      (if (string-prefix-p prefix ns)
          ns
        (concat prefix ns)))))

(setq cider-test-infer-test-ns #'cider-custom-test-ns-fn)
----

=== Default Test Selectors

If you have selectors you want
automatically applied, you can set the variables
`cider-test-default-include-selectors` and `cider-test-default-exclude-selectors`
to a list of strings to use. The following is an example of setting the default exclude
selectors so that tests tagged as "integration" or "flakey" don't run.

[source,lisp]
----
(setq cider-test-default-exclude-selectors '("integration" "flakey"))
----

TIP: You'll generally want to place default selectors in xref:config/project_config.adoc[your project configuration], as opposed to your global configuration.

=== Display Test Report on Success

By default the test report is displayed only when there are test failures or
errors. If you want to view the test report regardless of whether the tests have
passed or failed:

[source,lisp]
----
(setq cider-test-show-report-on-success t)
----

=== Running a callback on errors

Sometimes, for fine-grained integrations or debugging, you might want
to run an arbitrary callback whenever the tests fail due to an exception.

For this, you can safely redefine `cider.nrepl.middleware.test/*test-error-handler*`
using vanilla Clojure; please refer to its docstring for more details.

== Running Tests Automatically (Test-Driven Development)

CIDER provides a minor-mode that automatically runs all tests for a namespace
whenever you load a file (with kbd:[C-c C-k]). You can toggle it
manually with kbd:[M-x] `cider-auto-test-mode`, or you can use:

[source,lisp]
----
(cider-auto-test-mode 1)
----

This is identical to manually typing kbd:[C-c C-t C-n] every time
you load a Clojure buffer. As described previously, CIDER will try to
automatically determine the namespace containing the tests.
