Topology-Aware Autoscaling of Stateful Microservices Using Messaging Lag and Dependency Criticality

Abstract

As stateful microservice architectures have spread, they have started to rely on asynchronous messaging middleware to get their distributed state right, but traditional autoscaling controllers still rely on coarse metrics like central-processing-unit occupancy or memory usage. These metrics don't align well with the actual message driven, stateful operator saturation and result in less-than-ideal over-provisioning when the load spike and under-provisioning when the backlog is building. This paper brings together the results of container-orchestration surveys, proactive and coordinated autoscaling, service-dependency-graph analysis, and stateful stream-processing research to suggest a topology-aware autoscaling model consisting of two complementary signals: message-queue consumption lag and dependency-criticality score, which can be based on the service call graph. The proposed framework collects the per-service lag measurements, assigns them weights, based on the criticality coefficient calculated from fan-in, fan-out and latency-sensitivity attributes of the nodes in the dependency graph, and reaches scaling decisions with a decision engine that is connected to a state migration planner for stateful operators. Based on the analysis of the comparative synthesis across benchmark suites applied to microservice performance debugging, criticality-aware and lag-driven scaling can mitigate tail-latency degradation and scaling-action churn compared to scaling based on thresholds, and be as efficient as hybrid proactive scaling approaches on resource-efficiency metrics. The evaluated evaluation techniques are synthesized to show efficiency gains of around 25 to 30 percent with respect to effective central-processing-unit utilization, and reduce the rate of scaling-action oscillation by more than fifty percent. Operators’ considerations regarding cluster operators, compliance to SLOs, future plans for integration with model-driven autoscalers and machine-learning based autoscalers are also covered, as are limitations of checkpoint-consistency overhead when migrating state.