Tells notifiers what the patterns are doing.

A notifier is a module that implements this behaviour, listed in the :notifiers option of Rexsilient:

defmodule MyApp.Notifier do
  @behaviour Rexsilient.Notifier

  require Logger

  @impl true
  def notify(:after, :retry, %{result: :failure} = state) do
    Logger.warning("retry #{inspect(state.name)}: attempt #{state.attempt} failed")
  end

  def notify(_moment, _kind, _state), do: :ok
end

children = [
  {Rexsilient, modules: [MyApp.Resilience], notifiers: [MyApp.Notifier]}
]

Every notifier gets every notification; match on kind and the :name in state to keep only the ones needed. Without notifiers, nothing is sent.

Moments

kind is the pattern, and state a map that always has its :name.

Every :before also has :monotonic_time and :system_time, and every :after has :monotonic_time and the :duration since its :before, all in native time units, measured in the calling process.

:circuit_breaker, around calls made with Rexsilient.CircuitBreaker.run/3:

• :before - an allowed call is about to run.

• :after - it finished, with its :result and :outcome.

• :rejected - the breaker rejected the call, with its :system_time.

Calls made with Rexsilient.CircuitBreaker.acquire/1 and Rexsilient.CircuitBreaker.release/4 directly send none of these.

:circuit_breaker, when the breaker changes state, with the state it was in as :from, and the :system_time at which the change happened:

• :open - it opened, with the :timeout before it becomes half-open, and the :outcome that opened it. When forced with Rexsilient.CircuitBreaker.force_open/1, :timeout is :infinity and :outcome is nil.

• :half_open - it lets the next call through as a probe.

• :probing - a probe started, with the :timeout before another one is let through.

• :closed - a probe succeeded, it was forced closed, or the breaker manager restarted while it was not closed.

A change made with Rexsilient.CircuitBreaker.force_open/1 or Rexsilient.CircuitBreaker.force_close/1 also has forced: true, and one made by a manager restart has restart: true.

:retry:

• :before - an attempt is about to run, with its :attempt number.

• :after - it finished, with its :attempt, :result and :outcome.

• :delay - it failed and the next attempt waits :delay milliseconds, with its :attempt and :outcome.

• :stop - it failed and no attempt follows, with its :attempt, :outcome, the :duration of the whole run, and the :reason: :attempts when none remain, :strategy when the strategy stopped, :circuit_open when the attempt returned {:error, :circuit_open}.

:fallback, only when the fallback function is called:

• :before - it is about to run, with the :outcome it replaces.

• :after - it returned, with its :outcome.

:result is :success or :failure, as classified, and :outcome is a Rexsilient.Classify.outcome/0. More moments and keys may be added, so end with a catch-all clause.

State changes are notified from both the calling processes and the breaker manager, so they may arrive out of order; use :system_time to order them.

Process

Notifications are sent to a single notifier process, started by Rexsilient. The call being notified does not wait for it, so :before is handled after the call has started.

For each notification, every notifier is called in its own task, and the next notification waits for all of them, so each notifier sees them in the order the notifier process received them. A notifier that raises or exits only crashes its own task, which logs the error; one that takes longer than 5 seconds is killed, and the timeout is logged. Either way it is skipped for that notification, and the others are not affected.

Rexsilient.Notifier.TelemetryNotifier, the default, is the exception: it is called in the calling process, like any :telemetry event, so its handlers see the caller's context. Notifications sent by the breaker manager still go through the notifier process, so the manager never runs handlers.