CloudWatch Alarms to Pushbullet via SNS and AWS Lambda

When setting up Amazon CloudWatch alarms, I noticed they go to a Simple Notification Service (SNS) topic. From there, it’s possible to receive alerts via email, but a Lambda function is also among the choices. Since already I use Pushbullet and they have an API, I figured a serverless Node.js function could send me notifications. In this tutorial, I’m going to show you how to send CloudWatch alarms to Pushbullet to receive instant alerts about the health of EC2 instances. I think we’re doing ourselves a favor when learning trending technologies, I mean look how hot these services are getting:

Why Pushbullet?

I already wrote an article about Pushbullet and its API, so check that out first if you are not familiar with it. Now I feel that the circle is complete because I can finally show you a real-world use case. Their push notifications on my phone are special because I or my scripts send them. Emails can get lost, but at least on Android, it’s possible to control how different kind of notifications behave. Consequently, I set up Pushbullet’s pushes to have more importance than others.

What else? I haven’t found a good alternative yet. The only thing I could find seems way too complicated for this simple task. Unfortunately, the AWS Console app doesn’t send me notifications.

CloudWatch Alarms

While the process of setting up CloudWatch alarms is worth its own article, here is the relevant part. Make sure an SNS topic will receive the “in Alarm” state, then click Create topic:

If you click Add notfication you can create additional pushes, such as when the alarm goes back to the OK state. That could be useful for other metrics that could resolve on their own, such as memory and CPU.

AWS Lambda

Go to Lambda Management Console (check your region). The heart of this is a Node.js function, so create it like this:

The default settings are OK everywhere, except it’s worth setting Basic settings -> Timeout to 15 (seconds). Create a new test event with some TestEvent name and this code:

{
    "Records": [
        {
            "Sns": {
                "Subject": "Test Subject",
                "Message": "Test Message"
            }
        }
    ]
}

Here is the Node.js function you’ve been waiting for:

exports.handler = (event, context, callback) => {
    var SNS = event.Records[0].Sns;
    var pushTitle = SNS.Subject;
    if (typeof SNS.Message === "string") {
        try {
            SNS.Message = JSON.parse(SNS.Message);
        } catch (e) {}
    }
    var pushBody = typeof SNS.Message.NewStateReason === "string" ?
        SNS.Message.NewStateReason :
        SNS.Message;
    var request = require('https').request({
            hostname: 'api.pushbullet.com',
            path: '/v2/pushes',
            method: 'POST',
            headers: {
                'Access-Token': 'o.THEpushbulletACCESStokenGOEShere',
                'Content-Type': 'application/json'
            }
        },
        (res) => {
            console.log('Pushbullet API contact status:', res.statusCode);
        }
    );
    request.on('error', (e) => {
        console.error('Pushbullet function connect error:', e);
    });
    request.write(
        JSON.stringify({
            title: pushTitle,
            body: pushBody,
            type: "note"
        })
    );
    request.end();
    console.log("From pushbullet function: " + pushTitle + "; " + pushBody);
};

• Save, then Test to see your first push flying in.
• Get your access token and replace it with the placeholder.
• I use the https.request call.
• The logs go back to CloudWatch.
• The function works with any message; it does not strictly expect the specific message below.

The following is the content of event.Records[0].Sns.Message from a CloudWatch alarm coming in from SNS. You can see that NewStateReason is the most meaningful property (it’s low disk space in this example):

{
  "AlarmName": "Development: Disk Space",
  "AlarmDescription": "When free disk space drops below comfy levels.",
  "AWSAccountId": "123456789012",
  "NewStateValue": "ALARM",
  "NewStateReason": "Threshold Crossed: 1 out of the last 1 datapoints [2.9922180175781263 (29/06/19 16:05:00)] was less than the threshold (3.0) (minimum 1 datapoint for OK -> ALARM transition).",
  "StateChangeTime": "2019-06-29T16:10:01.997+0000",
  "Region": "EU (Frankfurt)",
  "OldStateValue": "OK",
  "Trigger": {
    "MetricName": "DiskSpaceAvailable",
    "Namespace": "System/Linux",
    "StatisticType": "Statistic",
    "Statistic": "AVERAGE",
    "Unit": null,
    "Dimensions": [
      {
        "value": "/",
        "name": "MountPath"
      },
      {
        "value": "i-1234567890abcdef0",
        "name": "InstanceId"
      },
      {
        "value": "/dev/nvme0n1p1",
        "name": "Filesystem"
      }
    ],
    "Period": 300,
    "EvaluationPeriods": 1,
    "ComparisonOperator": "LessThanThreshold",
    "Threshold": 3,
    "TreatMissingData": "- TreatMissingData: missing",
    "EvaluateLowSampleCountPercentile": ""
  }
}

When SNS connects to this function, the figure will look like this:

Simple Notification Service

Finally, the step where connecting CloudWatch Alarms to Pushbullet gets together! Think of SNS as a central hub where programmatic messages from other services arrive. At the same time, they go out as well when you subscribe to these using various channels. A simple use is “getting an email when something goes wrong”. However, we are complementing that with a push notification through the lambda function. Go to SNS -> Subscriptions (make sure your region is right) and click Create Subscription. Fortunately, the fields help you by autocompletion, so finding the Topic ARN and Endpoint from a list shouldn’t be hard.

Testing

It’s one thing that the lambda function sent you those test notifications, but further testing is needed.

• In SNS -> Topics, select the row you created. Then click Publish message and fill it with arbitrary content. You should receive an email and a push with whatever you typed as subject and message. If this works, you may re-use the Pushbullet function for other SNS topics.
• Temporarily lower the threshold in your alarm, so CloudWatch itself pings SNS. Notice that maybe it’s configured to act only when the problem has been happening for a while. Because of this, the push could seem delayed.

Summary of the complete CloudWatch Alarms to Pushbullet chain

1. An instance exposes EC2 memory and disk usage metrics to CloudWatch. Something goes bad… OR returns to normal!
2. A CloudWatch alarm triggers and notifies an SNS topic.
3. SNS sends you an email and also runs an AWS Lambda function.
4. The Node.js function contacts Pushbullet API and relays the message.
5. Your phone chimes in response to the event. Yay!