Resilience4J -Circuit Breaker Framework
Retry, CircuitBreaker, FallbackMethod, RateLimiting
Github: https://github.com/nikitha2/Currency-Exchange-Conversion-Microservices-Backend.git
Micro services are often inter dependent. Meaning there are a chain of calls made as shown below. To get response from service 1, service one needs service 2,3,4 to be up and running.
But what happens if service4 is down/very slow? Does that mean the chain breaks and all related micro services are down?
To handle this, questions we need to ask are:
- Can we return back a fall back response when service is down?
- Can we implement a circuit breaker pattern to reduce
- Can we retry requests incase of temporary failure?
- Can we implement rate limiting?
To show implementation for the above questions we will use a simple rest api shown below. Create a controller class and copy-paste the code shown below.
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
import io.github.resilience4j.retry.annotation.Retry;
@RestController
public class CircuitBreakerController {
private Logger logger = LoggerFactory.getLogger(CircuitBreakerController.class);
@GetMapping("currency-exchange/sample-api")
public String sampleApi() {
logger.info("---- SampleApi call recieved");
return "Sample-API response";
}
}Maven dependency
We can answer the above questions with a Circuit Breaker Framework, Resilience4J framework. Add the dependency to your project in pom.xml.
<!-- Resilience4j-->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-aop</artifactId>
</dependency>
<dependency>
<groupId>io.github.resilience4j</groupId>
<artifactId>resilience4j-spring-boot2</artifactId>
</dependency>1. Can we return back a fall back response when service is down?
When a service is down, can we return back a default response so that the chain does not break?
@Retry(name = "retry5Times", fallbackMethod = "hardcodedResponse")hardcodedResponse is a method. hardcodedResponse() takes Exception as input and we can return different responses for each exception. Here when retry fails “retry5Times” times we return back a static string message “hardcoded — FallBack — Response”.
public String hardcodedResponse(Exception ex) {
logger.info("---- hardcodedResponse call recieved");
return "hardcoded - FallBack - Response";
}2. Can we implement a circuit breaker pattern to reduce load?
Circuit breaker pattern will return back a default response without hitting a service that is already down. This will reduce the traffic and load on the service that is already down/slow.
Circuit breaker pattern hits the service until threshold is reached. If the service hit fails beyond threshold circuit will be broken for wait duration and failure response is retuned without even hitting the service. This will help in reducing the load on the service that is being hit.
@RestController
public class CircuitBreakerController {
private Logger logger = LoggerFactory.getLogger(CircuitBreakerController.class);
@GetMapping("currency-exchange/sample-api")
@CircuitBreaker(name = "default", fallbackMethod = "hardcodedResponse")
public String sampleApi() {
logger.info("---- SampleApi call recieved");
// Make call to a dummy rest api that is bound to fail as it does not exist
ResponseEntity<String> response = new RestTemplate()
.getForEntity("https://localhost:8080/dummy-api-that-will-fail", String.class);
return response.getBody();
}
public String hardcodedResponse(Exception ex) {
logger.info("---- hardcodedResponse call recieved");
return "hardcoded - FallBack - Response";
}
}So how do we test this? Threshold is around 100 calls per second. We either hit the service “currency-exchange/sample-api” 100 times/minute manually or use watch command in terminal.
//Makes 10 requests every second. If -n 0.1 is not there request will be made every 2seconds
watch curl -n 0.1 http://localhost:8000/currency-exchange/sample-apiYou’ll see that after a while watch will continue to make the calls, but our server will no more call the dummy-server. It will return a default fallback response without hitting the dummy -server. Only after the duration time, will it try hitting it again to see if the service is up and running.
3. Can we retry requests incase of temporary failure?
Sometimes services fail due to un-forseen exceptions. However when we hit the service again they work. In such cases we wanted to try hitting the service a few times before returning a failure/default response.
@Retry(name = "default", fallbackMethod = "hardcodedResponse") Retry allows to call the service a number of times before failing it. “default” trys to hit the service 3 times before failing and calling the fallbackMethod.
If I want to hit the service a custom number of time (say 5 times) before failing. I can create a custom retry configuration as shown below. Simply add the below lines in application.properties file and use name=”retry5Times” for the service call.
#------------Resilience4j retry config------------
resilience4j.retry.instances.retry5Times.max-attempts= 5
#wait duration between retry's
resilience4j.retry.instances.retry5Times.wait-duration= 1s
#exponentially increase wait-diration
resilience4j.retry.instances.retry5Times.enable-exponential-backoff= true @RestController
public class CircuitBreakerController {
private Logger logger = LoggerFactory.getLogger(CircuitBreakerController.class);
/**
* retry5Times will retry 5 times as
* declared in application.properties
**/
@GetMapping("currency-exchange/sample-api")
// default will try the call 3times if it fails.
//@Retry(name = "default", fallbackMethod = "hardcodedResponse")
@Retry(name = "retry5Times", fallbackMethod = "hardcodedResponse")
public String sampleApi() {
logger.info("---- SampleApi call recieved");
return "hardcoded - Response";
}
public String hardcodedResponse(Exception ex) {
logger.info("---- hardcodedResponse call recieved");
return "hardcoded - FallBack - Response";
}
}However if you see the sampleApi(). There is no way service will fail because we simply replying a static string value. To fail the service lets call a dummy service that is going to fail for sure.
@RestController
public class CircuitBreakerController {
private Logger logger = LoggerFactory.getLogger(CircuitBreakerController.class);
@GetMapping("currency-exchange/sample-api")
@Retry(name = "retry5Times", fallbackMethod = "hardcodedResponse")
public String sampleApi() {
logger.info("---- SampleApi call recieved");
// Make call to a dummy rest api that is bound to fail as it does not exist
ResponseEntity<String> response = new RestTemplate()
.getForEntity("https://localhost:8080/dummy-api-that-will-fail", String.class);
return response.getBody();
}
public String hardcodedResponse(Exception ex) {
logger.info("---- hardcodedResponse call recieved");
return "hardcoded - FallBack - Response";
}
}Now, when you run the application and enter “http://localhost:8000/currency-exchange/sample-api” in browser you should get “hardcoded — FallBack — Response”. When you check your logs you will see it tried to hit the dummy-service 5 times and then called hardcodedResponse() as shown below.
4. Can we implement rate limiting?
Allow only certain number of calls to certain micro services in certain amount of time. This will control the rate of requests we make to the services.
// 2 requests
resilience4j.ratelimiter.instances.default.limitForPeriod= 2
//In every 10 seconds
resilience4j.ratelimiter.instances.default.limitRefreshPeriod =10s @GetMapping("currency-exchange/sample-api")
@RateLimiter(name= "default")
public String sampleApi() {
.....
}You can test the above with watch commond by sending a request every 1 seconds. After 2 requests rest of the requests fail.
//Makes 1 requests every second.
watch curl -n 1 http://localhost:8000/currency-exchange/sample-apiIn addition to rateLimitter, we can also configure how many concurrent calls are allowed. This is called bulkHead. Below coniguration will allow 10 concurrent calls.
#Bulkhead
resilience4j.bulkhead.instances.default.max-concurrent-calls= 10