Question

What mechanism does a distributed system use to stop sending requests to a failing service to prevent cascading failures?

Accepted Answer

The mechanism used to stop sending requests to a failing service is called a Circuit Breaker. A circuit breaker acts as a proxy between a client and a service, monitoring the success and failure rate of requests to detect if a downstream system is unhealthy. It operates in three distinct states. In the Closed state, requests pass through normally to the service, and the breaker keeps track of failures. If the number of failures exceeds a predefined threshold within a specific timeframe, the circuit transitions to the Open state. While in the Open state, the circuit breaker immediately rejects all incoming requests with an error or a fallback response, preventing the system from wasting resources on calls that are guaranteed to fail and avoiding a cascading failure where the entire system collapses due to latency or resource exhaustion. After a set timeout period, the circuit moves to a Half-Open state. In this state, the breaker allows a limited number of test requests to reach the service to see if it has recovered. If these test requests succeed, the circuit resets and returns to the Closed state, allowing normal traffic to resume. If the test requests fail, it returns to the Open state, and the timeout period begins again. This pattern ensures that a failing service is given time to recover without being overwhelmed by a flood of requests from dependent services.

Home → All Courses → Engineering and Technology Courses → Big Data Systems Architecture → Flashcard

What mechanism does a distributed system use to stop sending requests to a failing service to prevent cascading failures?

Community Answers