Heatmaps of Australia Sensornet attackers and inbound SPAM
One of our main focuses this year for the AHP is to work on how we present data efficiently and meaningfully.
I've turned to the Visualization field to learn how to present data in ways that can be understood, trends spotted, and outliers and anomalies identified. Armed with this, these topics can then be studied further, can answer questions, or give rise to new questions.
We are starting to understand how we can use a few tools now, particularly after the KL workshop (thanks Raffy and Sebastian for your help).
One obvious tool is cartographic heat mapping. We are all very used to the concept of heat gradients when we look at weather maps.
It is very useful to display data in this form to answer the question "Where ARE these things I'm interested in?, is there particular place they are more concentrated?"
Well, our good friend (and in fact, newest contributor) David Z helped me understand and install the gheat infrastructure, which seems to suit some of our needs fairly nicely. I like this application, it allows you to zoom into an area (just as you do in google maps), and the product then recalculates the heatmap for that perspective. It is quite interactive in that way, and can be used by non-geeks. Over the next few months we plan to make some data from this application available to interested parties in such an interactive way. Until then, I've got a few screenshots showing some early results.
This is a map of the locations of computers that are attacking our Australian SensorNET. One thing that stands out is that we seem to have a lot of activity from Japan. We are currently analyzing this, and if you attend Shaun's presentation at AusCERT2009 you'll learn more about this.
Shaun has built a system that attempts to calculate the origin of SPAM that is being sent to our Australian based email traps.
Because this data already existed, it was trivial for me to run this through gheat and come up with the following maps.
(click to enlarge) Here is a heatmap of locations sending SPAM to Australia.
(click to enlarge) Lets look closer at activity from within Australia itself
(click to enlarge) Lets look closer at activity out of Europe
(click to enlarge) Lets look closer at activity out of the US
What can you see ? Seriously, let us know at [email protected]
We hope to make more posts involved data visualization techniques this year. It's an important area for us.
If you have any suggestions or viz tools that you can recommend, please let us know at [email protected]