Speakers
Description
Fault tolerance is a fundamental requirement of highly concurrent and distributed software systems. Elixir/OTP approaches this problem through process isolation and supervised recovery, following the "let it crash" philosophy inherited from Erlang. Rather than attempting to prevent every process failure, failed components are terminated and restarted by supervisors. While this model is widely used, the operating conditions under which repeated crash-and-restart recovery remains reliable are less clearly documented.
This work proposes an empirical investigation of the reliability limits of supervised recovery in Elixir/OTP. A fault-injection test environment is developed around a stateful transaction-processing workload in which controlled process failures are introduced under varying frequencies and system loads. Recovery is evaluated not only through process restart time, but also through application-level metrics including throughput, tail latency, request backlog, recovery time, transaction completion and correctness.
The study aims to identify the transition between successful recovery and degraded operation, as well as the conditions under which repeated failures prevent the system from returning to a stable state. Particular attention will be given to interactions between failure frequency, workload intensity, recovery cost and application state. Where limitations of conventional supervised recovery are observed, complementary recovery mechanisms are evaluated to determine whether the stable operating region can be extended.
By experimentally documenting this recovery boundary, the work seeks to provide a clearer understanding of both the strengths and practical limitations of crash-oriented fault recovery and to create useful design guidance for resilient Elixir/OTP applications.