Replaying failed tests
======================

This page discusses how Hypothesis saves and replays failing tests, and how to tell Hypothesis to try a specific test input every time.

Hypothesis saves failing examples
---------------------------------

When a test fails, Hypothesis saves the failure to a database in the local ``.hypothesis`` directory. It will then retry this failure any time the test gets run in the future. For instance, the following will take a few seconds to fail from random generation:

.. code-block:: python

    import time

    from hypothesis import strategies as st


    @given(st.integers())
    def f(n):
        assert n < 50
        time.sleep(0.1)


    f()

As it fails, Hypothesis saves the failing example to the local database. As a result, the next time you run this same code, ``f`` will fail instantly, because Hypothesis retries the failing example first.

Disabling the local database
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You can disable the local database with |@settings|:

.. code-block:: python

    import time

    from hypothesis import settings, strategies as st


    @given(st.integers())
    @settings(database=None)
    def f(n):
        assert n < 50
        time.sleep(0.1)


    f()

How to always try a specific input
----------------------------------

If you want Hypothesis to always run a specific input, you can use the |@example| decorator. |@example| adds an explicit input which Hypothesis will run every time, in addition to the randomly generated examples. You can think of explicit examples as combining unit-testing with property-based testing.

For instance, suppose we are testing something using integers, but want to make sure we try a few large prime numbers on every test run. We can add these primes with an explicit |@example|:

.. code-block:: python

    # mersenne primes
    @example(2**17 - 1)
    @example(2**19 - 1)
    @given(st.integers())
    def test_something_with_integers(n):
        pass


    test_something_with_integers()

Hypothesis runs all explicit examples first, in the |Phase.explicit| phase, before generating additional random examples in the |Phase.generate| phase.

Explicit examples do not shrink
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Unlike examples generated by Hypothesis, examples provided using |@example| do not shrink. We can see this by adding a failing assertion:

.. code-block:: python

    @example(2**17 - 1)
    @given(st.integers())
    def test_something_with_integers(n):
        assert n < 100

Hypothesis will print ``Falsifying explicit example: test_something_with_integers(n=131071)``, instead of shrinking to ``n=100``.

Prefer |@example| over the database for correctness
---------------------------------------------------

While the database is useful for local development, we don't recommend relying on it for the correctness of your tests. If a test fails with a particularly interesting input, and you want to make sure your test works for this input in the future, we recommend explicitly adding it to your test with |@example|.

The reason is that entries in the database may be invalidated at any time by Hypothesis, because the internal format is not necessarily stable across versions. Additionally, the database is generally not checked into version control, so failures are not shared between developers, unlike with an explicit |@example|.

.. note::

    If you *do* want to share the database between developers, using a database like |RedisExampleDatabase| might be helpful.

Replaying examples from CI logs
-------------------------------

If your test fails in a CI job, Hypothesis will automatically print an |@reproduce_failure| decorator containing an opaque blob, which you can add to your local test to reproduce the failure:

.. code-block:: none

    You can reproduce this example by temporarily adding @reproduce_failure('6.130.5', b'AELnLA==') as a decorator on your test case
    Falsifying example: test(
        i=-6356,
    )
