Finding bad guys using AWS CloudWatch Logs Insights

Probably every popular website becomes a target of brute-force hacking attempts or denial-of-service attacks once in a while. One could say dealing with cyberattacks is price of being successful in online world. In this post I'm going to describe how to look up attackers' IP addresses using AWS CloudWatch Logs Insights. Knowing attackers' IP addresses can be helpful in configuring firewalls and rate limiting tools.

Logs Insights is a powerful tool for analysing AWS CloudWatch Logs. It operates at Log Group level, which means that the Insights queries take into account all Log Streams within a Log Group. It is particularly useful for example when looking at logs from a web service that is composed of multiple containers, where each container produces its own Log Stream.

To gain insight into what's going on when there's a sudden spike in traffic, we choose the Log Stream of given application, select a timeframe, and compose a Query in the ‌Insights purpose-built query language.

We want to look up IP addresses sorted by number of requests. Let's assume our logs follow the Common Log Format standard and the first field is the IP address of the client. The Query then is as follows:

fields @timestamp, @message
  | parse @message /(?<@ip>^([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})[.]*)/
  | stats count() as requestCount by @ip
  | sort requestCount desc

This Query processes all log entries it can find in given time period and lists the champions on the top. This can already be valuable information, but we see some noise there - IP addresses from the private IPv4 range 172.17.0.0/16. These are internal requests (between nginx and php containers in this case) and we want to omit them from the output. We add the following expression after the parse @message part:

| filter @ip not like /(^172\.17\.[.]*)/

If we notice from general inspection of the access logs that the attackers are attracted to a specific URL, we can narrow our Query using the following expression, placed right after the fields @timestamp, @message part:

| filter @message like /(?i)(exampleURL)/

Now the results show very clearly who the bad guys are: