We called this experiment “Project Arachnid” because we imagined being like (marketing) spiders sitting at the centre of our web getting real time signals (data) upon which we could act. There is a lot of talk about “real time” and “marketing automation”; there is also a lot of talk about the possibilities of HTML5. In this experiment we wanted to test out both.
What you see below is a real time visualization of users arriving at the top levels of the Econsultancy website, or moving around it, as well as their past and potential value as customers indicated by the size and colour of the dots. Click on a dot for more info on the user. Below the visualization is a bit more detail on how this works from a marketing and technical perspective.
Loading...
What exactly am I looking at?
You are looking at a visualization of the traffic arriving at the top levels of the Econsultancy website (this site) right now. We don’t show all the pages of the site as we have hundreds of thousands and it would be much harder to visualize.
As soon as you load this page we start to show the new arrivals. We assume that people have left the site, and their dot disappears, if they are inactive for more than two minutes: this is the average session length on our site. If you reload the page then the dots will all disappear and then start accumulating again.
If you come to the site during the working day then you’ll likely see more activity than if you come at the weekend or in the middle of the night. Though these timings obviously vary around the world.
If you want, open another browser tab, and visit a section of the site where there is currently little activity and you should see yourself arrive. If you then navigate around the site, using the top level site navigation, you should see your dot move at the same time below. Click on your dot to see what we ‘know’ about you…
What does the size and colour of the dots mean?
If the dots have a white circle around them this means that the user is logged in so we have more data on that person. If you click on any of the dots you can see some of the information we know about that user – obviously we’ve removed anything that it personally identifiable.
The size of the dot shows the actual value to date of that customer to Econsultancy. This is largely computed by totaling the transactional spend by that customer via the Econsultancy site. The bigger the dot the more money that customer has spent with Econsultancy. We also attribute value based on the number of visits the customer has made to the site, and the number of comments the user has left on our site. This is because both these activities generate advertising revenue for us which we can assign a direct value to.
The colour of the dot shows the potential value of the customer. We use a colour spectrum from blue (= cold = low potential value) to red (= hot = high potential value). There are 255 shades in this spectrum. So look for the red dots — those are the ones we think are potentially high value to us! What colour is your dot…?
How are we computing the potential value of a customer?
We’ve written our own algorithm to do this. We don’t claim it to be perfectly accurate but it is based on other data we have available e.g. our web analytics, e-commerce and CRM data.
Below you can see the table showing the data inputs we’re using and how we’re then calculating potential value. It is largely a factor of where the user is on the site and this is then up-weighted or down-weighted based on other relevant attributes.
| Attribute / Variable | Value based on Google Analytics $Index value ('value' of page) combined with value multiplier/detractor |
|---|---|
| Value if on homepage | £2.08 |
| Value if /membership | £12.74 |
| Value if /join | £13.60 |
| Value if /reports | £1.20 |
| Value if /training | £8.55 |
| Value if /events | £1.41 |
| Value if /jobs | £1.22 |
| Value if /blog | £0.09 |
| Value if /about | £0.52 |
| Value if /consulting | £0.74 |
| Value if /directories/suppliers | £1.91 |
| Value if /directories/members | £1.75 |
| Value if /forums | £0.02 |
| Frequency of visit |
Multiplier of base value as follows:
|
| Recency of visit | If visited within last 4 weeks then multiply value by ×1.2 |
| Country based in | If based in UK or US then multiply value by ×1.2 |
| Heard about Econsultancy from a colleague | Multiply value by ×1.3 |
| Brand search referred - Referring search string contains 'econsultancy' or 'e-consultancy' | Multiply value by ×2 |
| Isn't opted in to any emails | Multiply value by ×0.1 (i.e. reduces value) |
What technology have you used to do all this?
The client-side visualisation code is written in JavaScript. It uses the HTML5 canvas API to draw a visual representation of the Econsultancy site, and the WebSocket API to connect to a Ruby EventMachine process on an Econsultancy EC2 instance. The EventMachine process collects real-time UDP datagram visit notifications from our web application workers, computes interesting data about each visit (e.g. actual and potential value), and broadcasts that data to all WebSocket browser clients. The clients use this data to identify visitors and show which pages are currently being viewed; clicking on a visitor sends a request for further information over the WebSocket connection, resulting in a detailed response which is displayed in an overlay.
