Is Log Analysis for Web Analytics a Dead Subject?

April 1, 2011 10 By Tad Reeves

DCP04341

I’ve been involved in web analytics in one capacity or another for about 11 years.  Back in 1998, when I was first getting started on this when working for Webtrends, there were only two ways to go about getting stats from your website:  either get a program to crunch the logs for your site (of which there were many), or pay some ridiculous sum for a tool like Aria or NetGenesis or HitList which used packet-sniffers placed in front of your web server, to track various interactions with your site.

In either case, the entire subject of web analytics assumed that every interaction that your users were doing with your site was going to result in another page request back to the server, which you could then track via a log file.

image Wow, how times have changed.  Try going to a site like the main video channel for the Church of Scientology, or this one for the Volunteer Ministers.  You can complete an hour-long stay at either website, and still have only looked at one HTML page.  Doesn’t make log-based analytics too entertaining, especially when your videos are hosted off-site.

In late 2004, I did a project for for a company in Atlanta, testing out literally about 30 different web analytics solutions to work out the best one for them.  At that point in time, out all the different packages that I reviewed, there was a pretty even split between the number of web analytics companies that were making a go at it with a shrink-wrap, software-based solution that would be hosted at the client side, and the other half were ASP’s.

Many organizations that I was working with were not all too interested in turning to an ASP-based setup for various security reasons, as well as the fact that ASP’s aren’t too great at doing analytics for  intranet sites, when the client can’t access the external internet.

At that point in time, the landscape looked something like this:

Analytics Product ASP or Software Analysis Type
Webtrends Either Log Analysis or Page-tagging
Datanautics G2 Software Log Analysis & Packet Sniffer
DeepMetrix LiveStats xSP Software Log Analysis
Pilot HitList Software Log Analysis & Packet Sniffer
Sane NetTracker Software Log Analysis or Page-tagging
Sawmill Software Log Analysis
SPSS NetGenesis Software Log Analysis, Page-tagging and Packet Sniffer
Urchin Software or ASP Log analysis or Page-tagging
Eloqua ASP Page-tagging
Elytics EAS ASP Page-tagging
Manticore Virtual Touchstone ASP Page-tagging
Omniture SiteCatalyst ASP Page-tagging, but they’d import your old logs for a fee
SageMetrics SageAnalyst ASP log analysis and page-tagging
WebSideStory HBX ASP Page-tagging, but they’d import your old logs for a fee

 

As you can see, it was about half-and-half, with most products still clinging to log analysis, but many more progressive (and sometimes completely frightening) products going to page-tagging exclusively to be able to trap and coordinate interactions with your sites.

But now, boy has the landscape changed.   After Google bought Urchin and transformed it into a free product, countless companies have now successfully experimented with Google Analytics and found it (and with it, the whole premise of tagging pages) to be a reliable and insightful way of tracking interactions with pages.

Also, the fact of its being free has forced a lot of companies to either (a) drop the web analytics business alltogether, or (b) dramatically change their model so as to differentiate themselves from Google Analytics and move themselves way upmarket.

Check out how this grid looks now, 5 years later:

Analytics Product Still around?  If so, what type of product Analysis Type
Webtrends ASP or Software Log Analysis or Page-tagging
Datanautics G2 Product is discontinued
DeepMetrix LiveStats xSP Bought by Microsoft, and then deep-sixed
Pilot HitList Product bought by SAP and then deep-sixed
Sane NetTracker Acquired by Unica, now merged in with their Net Insight software product. Log Analysis or Page-tagging
Sawmill Still Software Log Analysis
SPSS NetGenesis Toasted by SPSS?  Product page is now a 404
Urchin Bought by Google, is now Google Analytics Page Tagging
Eloqua ASP Page-tagging
Elytics EAS Dead like a doornail
Manticore Virtual Touchstone ASP Page-tagging
Omniture SiteCatalyst ASP Page-tagging, but they’d import your old logs for a fee
SageMetrics SageAnalyst ASP log analysis and page-tagging
WebSideStory HBX Bought by Omniture, old HBX product is toast

In any case, it’s a subject for another blog post as to what companies have to do to differentiate themselves from Google Analytics in order to make it worth the cash for users to upgrade from a free product.

The main case here, though, is if log analysis has any value or relevance still in the market?   What do you get from a log analysis tool these days that you can’t get from a pixel tracker?