Web Analytics & Data Mining Must Converge

Are businesses actually seeing a convergence between the two disciplines of data mining (searching for patterns in sets of data) and Web analytics (determining how customers are using a company’s Websites)?

The prevailing school of thought is that data mining and Web analytics are indeed disparate disciplines. But why? After all, both require expertise in dealing with data. In most cases, the analytical deliverables are similar: Both disciplines attempt to better understand customer behavior through some key metrics, such as:

• New customers
• Repeat customers
• Time of last activity
• Frequency of activity
• Monetary value of last activity

In order to fully appreciate both disciplines, some background is in order here.

Data miners have been analyzing data since Fair Isaac (now FICO) developed its first credit card risk model back in the 1950s. Since then, the use of data mining as a key business tool has evolved into other disciplines, most notably marketing.

Early data mining pioneers included such organizations as Reader’s Digest. As this technique matured within the direct marketing area, financial organizations like American Express became early adopters. Currently, the use of data mining has expanded to almost all other industry sectors, with financial institutions and telcos the most sophisticated users.

Data miners typically work with large volumes of data, and their expertise lies particularly in transaction-related data.

Meanwhile, the growth of Web analytics is a relatively new phenomenon, due entirely to the growth of the Internet. The early pioneers of this discipline came directly from the Internet space, with their original expertise being the design of Websites. Along with this expertise, practitioners of Web analytics acquired a deep understanding of the data environment -- but as it relates to the Internet. Web analysts also acquired an expertise in analyzing large volumes of data, but it was data specifically related to log files or page tags.

As you can see from the above, data mining and Web analytics are distinct disciplines. And it is this disparity that can be dangerous for enterprises that use them.

For one thing, the data nuances and subtleties of the Internet need to be fully understood if one is analyzing Web data. A lack of understanding of this type of data can lead to grossly incorrect results concerning customer behavior.

Data miners, for instance, do not want to look at the page view history of the user for a given session, but rather the page view history of the user over a period of time.

It is this longitudinal, or more historical, view of the customer that is lacking amongst Web analyst practitioners. Why? The capability of viewing the customer historically and from a longitudinal perspective requires the development of customized marketing databases for a given Website. But the development of marketing databases is an expertise that is more typically found within the data mining discipline, not in Web analytics.

Another key difference: Data miners are typically disciplined in integrating data from very disparate sources, while Web analytics practitioners are used to integrating data as long as it is confined to the Web space.

But integrating online and offline data is vital to calculating factors such as ROI from purchase behavior, for instance.

Hopefully, the above comments reinforce the need to more effectively integrate the two disciplines of data mining and Web analytics. As long as both disciplines continue to have a silo-based approach to their work, solutions will be less than optimal.

It is only through a more centralized approach toward analytics that solutions can be truly optimized.

— Richard Boire is the founding partner at the Boire Filler Group.