Handle mass polling with Back Off Algorithm

Share:

Posted under Let’s Fix, where we discuss problems and solutions to real world web dev problems

Source Code

Recently I had the opportunity to solve a problem that may not be common to most people, but if you ever have to deal with high traffic websites, then this little technique might be for you.

The Problem

The website needs to do polling to a server endpoint every 10 seconds,  the website will display new content if new contents are found from the response.  Normally this is not a problem, but when we have a high traffic website, we can’t just add more servers, it’s simply not enough.  What we want to do on top of more servers, is to implement a back off algorithm on client side in order to help reduce the server load.

Back Off Algorithm

Exponential backoff is an algorithm that uses feedback to multiplicatively decrease the rate of some process, in order to gradually find an acceptable rate.

You may have seen this before on Slack:

slack disconnect

To put it simply, the algorithm will check to see if the user connection to the server is within a certain threshold, if not, it will increase the polling time for the next polling interval, and repeat this process until a limit is met.

Demo

You follow along the demo by cloning the repo

$(function() {

	// Default settings
	var DEFAULT_POLLING_TIME = 2000;
	var MAX_REQ_TIMEOUT_TIME = 1000;
	var TIMEOUT_THRESHOLD = 30000;

	// Whenever the page loads, override the polling time with the default
	localStorage.pollTime = DEFAULT_POLLING_TIME;


	function polling() {
		// To track the time when the request starts
		var startTime = new Date().getTime();

		// For use in the .always() callback
		var pollTime = localStorage.pollTime;

		$.ajax('data.json')
	  .done(function(data, status, xhr) {

	  	// How long did our request took
	  	var requestTime = new Date().getTime() - startTime;
			
	  	// Uncomment to simulate server response timeouts
		requestTime = Math.floor(Math.random() * 6) + 996;

			// The meat, if the request took too long, and the current stored pollTime is less than the threshold, then set the pollTime to whatever it is times two.  If it's over than timeout threshold, then max it out at the threshold.
			if (requestTime >= MAX_REQ_TIMEOUT_TIME) {
			    if (localStorage.pollTime <= TIMEOUT_THRESHOLD) {
			        localStorage.pollTime = localStorage.pollTime * 2;
			    } else {
			        localStorage.pollTime = TIMEOUT_THRESHOLD;
			    }
			    $('body').append('<h2 style="color: red">Bad Request: (' + requestTime + 'ms), extending interval: ' + localStorage.pollTime + '</h2>');
			    xhr.abort();
			    return;
			}

	  	var result = JSON.parse(data);
	  	$('body').append('<h2 style="color: green">Good Request (' + requestTime + 'ms)' + ', Use same interval: ' + localStorage.pollTime + '</h2>');
	  })
	  .fail(function(error) {
	    // apply the backoff alg here too...
	    //
	  })
	  .always(function () {
	  	// This will make the app call polling indefinitely
      setTimeout(polling, pollTime);
    });;
	}

	// Start everything off here
	polling();
});

The demo is a simple jQuery ajax function that fires at a set interval.  When a request takes too long, it triggers the back off algorithm to double the polling interval.

Some noteworthy points:

  • DEFAULT_POLLING_TIME is the default polling time, this means we are making an ajax request every 2 seconds.
  • MAX_REQ_TIMEOUT_TIME is the threshold request time we use to compare how long the current request takes.  If the request takes longer than this time, then we need to activate the back off algorithm. In this case it’s 1 second.
  • TIMEOUT_THRESHOLD is the maximum threshold for our back off algorithm, this is the upper limit for when our back off algorithm should set its limit to, the polling time can never exceed this number.
  • We use localStorage to hold our polling time, this way the app will remember what parameter it should use for polling time.
  • requestTime is the duration of ajax request, we derived that by taking the time AFTER the ajax request minus the starting time BEFORE the ajax request. For the sake of demo, I’ve use a randomizer to simulate requestTime:
    requestTime = Math.floor(Math.random() * 6) + 996;

The main logic of the program is the following:

			if (requestTime >= MAX_REQ_TIMEOUT_TIME) {
			    if (localStorage.pollTime <= TIMEOUT_THRESHOLD) {
			        localStorage.pollTime = localStorage.pollTime * 2;
			    } else {
			        localStorage.pollTime = TIMEOUT_THRESHOLD;
			    }
			    $('body').append('<h2 style="color: red">Bad Request: (' + requestTime + 'ms), extending interval: ' + localStorage.pollTime + '</h2>');
			    xhr.abort();
			    return;
			}

This will check if the request is taking too long, and if the current stored pollTime is less than the threshold,  set pollTime to whatever it was multiply by two.  If it’s over than timeout threshold, then max it out at the threshold.

One improvement we can make is to implement the same logic inside the .fail() promise, but I’ll leave that up to you.

If you let the demo run for about a minute, you should see the following

back off algorithm

  1. The app starts out with 2000ms polling interval
  2. The first request exceeded our 1000ms threshold, so the algorithm sets the doubled the polling interval for the next request (4000ms)
  3. The next 4 requests are all within the acceptable request time, so the polling stays at 4000ms
  4. the next 4 requests all exceeded the request time threshold, so the polling period doubled for each until it reached 30,000ms which is our max.

Cool isn’t it? Now you won’t have to worry about potential unexpected DDoS attacks by your app, you can safely let the app do  a graceful fallback for you while you work on scaling your servers.

Comments Or Questions? Discuss In Our Discord

If you enjoyed this tutorial, make sure to subscribe to our Youtube Channel and follow us on Twitter @pentacodevids for latest updates!

More from PentaCode