Loadbalancing Algorithm
In microservices and API gateway architectures, load balancing is critical for evenly distributing requests across each service instance and providing mechanisms for high availability and fault recovery. FSM Gateway offers various load balancing algorithms, allowing the selection of the most suitable method based on business needs and traffic patterns.
Multiple load balancing algorithms support efficient traffic distribution, maximizing resource utilization and improving service response times:
- RoundRobinLoadBalancer: A common load balancing algorithm where requests are sequentially assigned to each service instance. This is FSM Gateway’s default algorithm unless otherwise specified.
- HashingLoadBalancer: Calculates a hash value based on certain request attributes (like source IP or headers), routing requests to specific service instances. This ensures the same requester or type of request is always routed to the same service instance.
- LeastConnectionLoadBalancer: Considers the current workload (number of connections) of each service instance, allocating new requests to the instance with the least load, ensuring more even resource utilization.
Prerequisites
- Kubernetes cluster
- kubectl tool
- FSM Gateway installed via guide doc.
Demonstration
Deploying a Sample Application
To test load balancing, create two endpoints with different response statuses (200, 201) and content. This is done by creating the Service pipy
, with two endpoints simulated using the programmable proxy Pipy.
kubectl create namespace server
kubectl apply -n server -f - <<EOF
apiVersion: v1
kind: Service
metadata:
name: pipy
spec:
selector:
app: pipy
ports:
- protocol: TCP
port: 8080
targetPort: 8080
---
apiVersion: v1
kind: Pod
metadata:
name: pipy-1
labels:
app: pipy
spec:
containers:
- name: pipy
image: flomesh/pipy:0.99.0-2
command: ["pipy", "-e", "pipy().listen(8080).serveHTTP(new Message({status: 200},'Hello, world'))"]
---
apiVersion: v1
kind: Pod
metadata:
name: pipy-2
labels:
app: pipy
spec:
containers:
- name: pipy
image: flomesh/pipy:0.99.0-2
command: ["pipy", "-e", "pipy().listen(8080).serveHTTP(new Message({status: 201},'Hi, world'))"]
EOF
Creating Gateway and Routes
Next, create a gateway and set up routes for the Service pipy.
kubectl apply -n server -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
name: simple-fsm-gateway
spec:
gatewayClassName: fsm-gateway-cls
listeners:
- protocol: HTTP
port: 8000
name: http
allowedRoutes:
namespaces:
from: Same
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
name: fortio-route
spec:
parentRefs:
- name: simple-fsm-gateway
port: 8000
rules:
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: pipy
port: 8080
EOF
Check application accessibility. The results show that the gateway balanced the load across the two endpoints using the default round-robin algorithm.
export GATEWAY_IP=$(kubectl get svc -n server -l app=fsm-gateway -o jsonpath='{.items[0].status.loadBalancer.ingress[0].ip}')
curl http://$GATEWAY_IP:8000/
Hi, world
curl http://$GATEWAY_IP:8000/
Hello, world
curl http://$GATEWAY_IP:8000/
Hi, world
Load Balancing Algorithm Verification
For configuring load balancing strategies, refer to the LoadBalancerPolicy documentation.
Round-Robin Load Balancing
Test with fortio load: Send 200 requests with 50 concurrent users. Responses of status codes 200 and 201 are evenly split, indicative of round-robin load balancing.
fortio load -quiet -c 50 -n 200 http://$GATEWAY_IP:8000/
Code 200 : 100 (50.0 %)
Code 201 : 100 (50.0 %)
Hashing Load Balancer
Set the load balancing policy to HashingLoadBalancer
.
kubectl apply -n server -f - <<EOF
apiVersion: gateway.flomesh.io/v1alpha1
kind: LoadBalancerPolicy
metadata:
name: lb-policy-sample
spec:
targetRef:
group: ""
kind: Service
name: pipy
namespace: server
ports:
- port: 8080
type: HashingLoadBalancer
EOF
Sending the same load, all 200 requests are routed to one endpoint, consistent with the hash-based load balancing.
fortio load -quiet -c 50 -n 200 http://$GATEWAY_IP:8000/
Code 201 : 200 (50.0 %)
Least Connections Load Balancer
In Kubernetes, multiple endpoints of the same Service usually have the same specifications, so the effect of the least connections algorithm is similar to round-robin.
kubectl apply -n server -f - <<EOF
apiVersion: gateway.flomesh.io/v1alpha1
kind: LoadBalancerPolicy
metadata:
name: lb-policy-sample
spec:
targetRef:
group: ""
kind: Service
name: pipy
namespace: server
ports:
- port: 8080
type: LeastConnectionLoadBalancer
EOF
Sending the same load, the traffic is evenly distributed across the two endpoints, as expected.
fortio load -quiet -c 50 -n 200 http://$GATEWAY_IP:8000/
Code 200 : 100 (50.0 %)
Code 201 : 100 (50.0 %)
Feedback
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.