LeakyReLU {#dev_guide_op_leakyrelu}
===================================

## General

LeakyReLU operation is a type of activation function based on ReLU. It has a
small slope for negative values with which LeakyReLU can produce small,
non-zero, and constant gradients with respect to the negative values. The slope
is also called the coefficient of leakage.

Unlike @ref dev_guide_op_prelu, the coefficient \f$\alpha\f$ is constant and
defined before training.

LeakyReLU operation applies following formula on every element of \src tensor
(the variable names follow the standard @ref dev_guide_conventions):

\f[ dst = \begin{cases} src & \text{if}\ src \ge 0 \\
    \alpha src & \text{if}\ src < 0 \end{cases} \f]

## Operation attributes

Attribute Name | Description | Value Type | Supported Values | Required or Optional
-- | -- | -- | -- | --
[alpha](@ref dnnl::graph::op::attr::alpha) | Alpha is the coefficient of leakage. | f32 | Arbitrary f32 value but usually a small positive value. | Required

## Execution arguments

The inputs and outputs must be provided according to below index order when
constructing an operation.

### Inputs

Index | Argument Name | Required or Optional
-- | -- | --
0 | `src` | Required

### Outputs

Index | Argument Name | Required or Optional
-- | -- | --
0 | `dst` |Required

## Supported data types

LeakyReLU operation supports the following data type combinations.

Src | Dst
-- | --
f32 | f32
bf16 | bf16
f16 | f16
