Load balancing and flow control for Web workloads - overview
A large body of work has been done in the area of server over-load protection for web service platforms. This work gains new signicance with the development of cloud computing platforms which host a large number of applications on shared resources and oftentimes charge users based on the resource utilization by their applications. These environments require mechanisms to accurately control resource utilization by an application or a customer.
Since multiple applications may share the same node, and the same application container, we developed a front-end dispatch mechanism, called gateway, to control request ow for all applications based on their resource utilization budgets.
We developed two techniques for preventing server overload by controlling dispatch of requests from the gateway to the server. The first mechanism imposes a limit on the number of requests that can be in process at the server concurrently. The second mechanism imposes a limit on the rate at which requests are dispatched. Both kinds of limit are derived automatically based on the specified CPU utilization budget for an application.
In this work, we have also developed a load balancer based on classical control theory, a controller to prevent memory overload in java-based application servers by using session-based admission control, and dynamic CPU usage profiler for Web workloads.