Monitor your site with PhantomJS and AWS Lambda

End-to-end site performance analysis is more than just the response time of your servers. Your site’s performance is going to depend on how fast your scripts load, the layout of your HTML, any 3rd party scripts that you are using, etc. To get a consistent real-world performance metric for your site, you may want to measure with an actual browser. It’s even better if you measure over time from remote places across the globe.

In this post, I’ll show you how to put together a system like that in 15 minutes.

To automate our browser under test, we will use PhantomJS. It is a “headless” browser that can run on a server without a GUI. You can use it to bring up web pages and run test scripts on them. We will use it to measure the performance of our company site.

Running PhantomJS scripts on Amazon hardware in regions across the globe 24 hours a day can get expensive.

To eliminate all costs, we will take advantage of the magical AWS Lambda. Lambda is a technology that allows you to run code on the cloud and only pay for the time it is running (in milliseconds). We can use Lambda’s scheduled events to invoke our function to monitor our site as often as we’d like. Our tests run comfortably within Lambda’s free usage tier, costing us no money.

Writing our monitor function

Let’s see how we can use Lambda to monitor our site and use CloudWatch to visualize our site’s performance over time.

We begin by installing our libraries on an ubuntu desktop.


npm install phantomjs
npm install node-horseman

Node horseman is a library that makes it easy to work with PhantomJS.

Next, let’s write our function to monitor our web site.

var Horseman = require('node-horseman');
var AWS = require('aws-sdk');
var cw = new AWS.CloudWatch();
var site = 'https://www.speedoflightmedia.com';
var cc;
exports.handler = function(event, context) {
    var t1 = process.hrtime();
    var t2;
    var horseman = new Horseman();
    cc = context;
    horseman
        .userAgent('Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0')
        .open(site)
        .waitForSelector('.footer')
        .then(function() {
            t2 = process.hrtime();
            var inst1 = t1[0]*1e9+t1[1];
            var inst2 = t2[0]*1e9+t2[1];
            var loadtime = Math.ceil(inst2/1e6 - inst1/1e6);
            var milliseconds = `Load time ${loadtime}ms`;
            var params = {
                Namespace: 'SiteStats',
                MetricData: [
                    {
                        MetricName: 'loadtime',
                        Dimensions: [
                            {
                                Name: 'SiteName',
                                Value: site
                            }
                        ],
                        Value: loadtime,
                        Unit: 'Milliseconds'
                    }
                ]
            };
            cw.putMetricData(params, function(err, data) {
                if (err) {
                    console.log(`error: ${err}`);
                    cc.fail(err);
                    return;
                }
            });
        }).close();
}

The code loads our company home page and waits until the site’s footer becomes visible. The test considers the site loaded when the footer shows. The script uses Node’s high resolution timers and converts the load time to milliseconds.

Graphing our site speed over time

Once the page is loaded, we push the metric to a new custom cloudwatch metric. CloudWatch does not require us to predefine our metrics, so we are free to push metrics without any setup.

We create our function by zipping up our folder containing our code and our node_modules folder. We create the function in the Amazon Lambda control panel and give it 512 megabytes of memory. Finally we create a schedule to run the function once every minute.

New SiteStats metric namespace

New SiteStats metric namespace

After waiting a few minutes, metrics start to arrive in CloudWatch.

Site load times

Site load times

At this point, we can create alarms in CloudWatch if we want if our site load times fall outside of our expected range. We can add line graphs to a dashboard, or we can use the number metric on a dashboard to show our site’s current load time at a glance.

In this post I showed you how you can easily use Lambda, PhantomJS and CloudWatch to set up a monitoring solution for your site that uses an actual browser and ensures that your site is rendering properly. You can take this example further by moving this lambda function to other regions, adding additional checks, etc. All of this functionality is available at no cost because it falls into Lambda’s free tier.