The Macrosite for News, Analysis and Opinion about the Future of the Internet
Sean Gallagher

'Data Warehousing as a Service' an Option for Midsized Firms

Written by Sean Gallagher
7/1/2010 8 comments
no ratings
DISCUSS   Digg   Del.icio.us   Reddit   Email This   TWEET THIS

There are some kinds of software for which the usual model of “software as a service” won’t apply.

That model certainly doesn’t apply to data warehousing and business intelligence.

Typically, SaaS offerings that work for both the provider and the customer take advantage of the inherent sameness of a certain business process across companies to package the app for mass consumption -- such apps as sales force automation, customer relationship management, billing, and accounting.

In contrast, data warehouses and the business intelligence applications built atop them are typically carefully crafted and highly customized, even though the underlying software may be fundamentally similar across enterprises. Companies that manage to create a working data warehouse frequently hold their business analytics close to the vest.

There's also a limit to what can be done in a pure Web-based model for business intelligence. While there are Web-based analytical packages already available for vertical applications, like Web analytics, the complexities of even the simplest business process analysis dwarf the sorts of things software like Omniture and Google (Nasdaq: GOOG) Analytics do.

It’s because of these factors that “data warehousing as a service,” or DaaS, is the antithesis of something like Salesforce.com Inc. You don’t just buy it with a credit card and start exporting data -- especially since getting all the data together in itself may be difficult.

But that doesn’t mean that DaaS is a non-starter. For many midsized companies, building a data warehouse in-house is a bridge too far, just in terms of the cost of the hardware and software and the prerequisite skills needed to pull it all together. At many small and midsized companies, as a result, “business intelligence” usually amounts to a Microsoft Excel “pivot table” with data extracted from a report.

Traditional outsourcing schemes for BI run into the same cost issue. Taking a traditional approach to building a data warehouse in an outsourced environment is still cost-prohibitive, even for many larger midsized companies. The cloud model, on the other hand, reduces the cost of the computing footprint for small data warehouses and allows companies to scale up as required to handle large seasonal data sets if necessary.

So it’s no surprise that a number of service providers are making the pitch for cloud-based business intelligence and data warehousing solutions. Kickfire , the data warehousing appliance vendor, conducted a cloud-based data warehouse trial with Sun (now Oracle Corp. (Nasdaq: ORCL)) last year; and Oracle Corp. (Nasdaq: ORCL) and IBM Corp. (NYSE: IBM) have been developing cloud-based analytics tools for their large customers. Netezza and AppNexus have teamed to do data warehouses as an “infrastructure as a service” -- essentially a cloud-based hosting model for data warehousing and BI.

One company, Kognitio, has essentially been in the DaaS business for almost 13 years -- since well before “cloud” was a term used to discuss anything other than network topology or weather.

“We're kind of careful about how we talk about the cloud,” says Kognitio CEO John Thompson. That’s because Kognitio doesn’t use a public cloud model for providing its services. “We use a trusted cloud or private cloud, because many enterprises have concerns about privacy and security of their data.”

Kognito provides data integration services, allowing companies to combine their internal ERP, CRM, and other data with data from other sources, including syndicated data streams from research providers. For example, Kognitio recently worked with American Access Casualty Company to build a business intelligence system that used a combination of that company's own financial data, census data, and third-party data about its competitors’ insurance rates.

“We generated every potential quote for 10 of their competitors for every age, sex, and marital status and for every car, and built a database that was over 450 million price points for them,” says Thompson. Kognitio then integrated all the data into a system that helped American Access cut the time it took to put together price change proposals from eight weeks to four days.

The biggest problem most midmarket companies face in adopting data warehousing as a service is getting the data together in the first place. “It is about the data at the end of the day,” says Sean Jackson, marketing manager at Kognitio. “You still have to get data into a format that we can then switch into a service. I think midsized enterprises that are thinking about this have to ask a few questions about how easy it's going to be to get all of the data into one place first.”

— Sean Gallagher is an award-winning IT journalist and the former head of InformationWeek Labs. Gallagher is now an independent journalist and technology consultant based in Baltimore. He can be reached at: gallagher.sean.m@gmail.com.

DISCUSS   Digg   Del.icio.us   Reddit   Email This
Current display:       newest comments first       display in chronological order
DHagar
Thinkernetter
Tuesday July 6, 2010 9:38:05 PM
no ratings

That does make sense, Xuefei.  As Sean points out the functions of data storage and management do make sense when scaled properly.

I also agree with Sean that it is a great strategy to use as an extension, or in the example you provided with Kognito, where data is matched and reported against external data sources.  As Sean points out, it is an option.

DHagar

Xuefei Peng
Rank: Web master
Tuesday July 6, 2010 4:44:03 AM
no ratings

Now, in mainland China, may middle or small firms choose to rent servers from operators or other server providers. Operators have huge advantages.  

mhhfive
IQ Crew
Sunday July 4, 2010 8:26:00 AM
no ratings

I think you're exactly correct that there are models where *aaS don't apply.  But it might not be so much about the "inherent sameness" problem -- as much as the "keeping BI in-house".  The way I see it... you *could* outsource BI as a service, but in the majority of cases, you wouldn't want to rely on an outside vendor for part of your core intelligence.  You might supplement your in-house BI with BIaaS, perhaps?

So as you suggest, there might be niche products where DWaaS and BIaaS make some sense, but they will be nowhere near as commonly used as CRMaaS or AccountingaaS.

samboonin
Rank: Cave Painter
Friday July 2, 2010 2:26:37 AM
no ratings

Mike - actually you can buy a dw with a credit card...

GoodData Pricing

-Sam

Sean Gallagher
Rank: Web master
Thursday July 1, 2010 8:36:25 PM
no ratings

Mike (and everyone else)-

My point was not so much that it's a "non-starter" to do DWaaS or Cloud BI, but that it does break significantly from the usual SaaS model and that small and midsized IT shops might avoid looking into it as a result.

I think that there are some very strong arguments to do DWaaS for midsized businesses. Scalability and the opportunity to combine your own data with third party data services, as Kognitio did with American Access, are the primary ones.  My main point is that it's got to be a consultative process, and so the stakeholders looking to do BI on demand should understand the work required to make it happen--as opposed to purchasing some seat licenses with a credit card.  It's closer to the traditional hosted IT solution model.

StevePR104
Rank: Cave Painter
Thursday July 1, 2010 7:59:07 PM
no ratings

Sean and Sam:

Far from being a non-starter, BI SaaS has been a reality for more than a dozen years for companies like British Telecom.  What's changed over time is that the raw processing power and ability to inexpensively thread multiple processors in parallel has made it possible for companies of all sizes to take advantage of business analytics.

It's more than simple slice and dice, of course; companies must be able to do "deep dives" on the data for it to be of any real use.  Companies like Kognitio, quoted in the article above, have made it possible to do precisely that, at a fraction of the cost and time previously required.

Inherent in any DaaS system is that there's a complete set of tools, already integrated, that can be quickly implemented...so that companies using the service know, going in, that they'll get quick turnaround on the information they feed in.  Which leads to a demonstrable ROI.

Sam, you're absolutely right in that companies need the analytical and reporting capability. The good news is that increasingly, they're able to get it.

 

Cheers,

Steve

mbeckerle
Rank: Cave Painter
Thursday July 1, 2010 6:10:49 PM
no ratings

Great post. We really struggle with the "XaaS" arguments because different people mean different things by BI/DW, etc.

My flip on this would not be can you do DWaaS, in the "buy with credit card" SaaS model, I think your points are very clear - you can't. But rather, how can the "aaS" model for deployment give leverage or scalability to the endeavor of creating BI, and how is this different from the way BI is done today?

You guys tell me if this meets your definition of "SaaS" or "BIaaS" (but you have to suggest a useful different term if you say no :-)

We do it by offering BI Solutions with real business content in inventory, transport, etc., not just hosted piece parts of BI, like DW or BI reporting/visualization tools. This helps reduce the number of variables: data model is standard, or slowly evolving anyway, data integration becomes much easier than traditional ETL because the target DW design is not different each time, and you have a QA context for the DW content. Solutions imply industries with problems those solutions can solve, and that narrows down the source-system proliferation somewhat also. There's some cool leverage that is only available "aaS" because you have data and metadata from many customers. Here's one example: two customers have SAP ERP, distinctly customized. You can compare them to each other to isolate the customizations, and that reduces the work finding and understanding these customizations.

To me this is much more scalable as a business model than one might think, but I will acknowledge it is very different from just doing a "traditional" DW or an advanced BI tool and doing it "SaaS". By doing solutions we are holding many things fixed or more nearly fixed that DW (and "DWaaS" similarly one would think) keeps variable.

It's not DWaaS, and we call it "SaaS BI", but regardless of the naming I think the cloud/hosted/"aaS" model IS the highest leverage way to address the data integration issues, and it delivers value for customers very quickly.

Mike Beckerle

www.oco-inc.com

samboonin
Rank: Cave Painter
Thursday July 1, 2010 1:02:00 PM
no ratings

 

Sean-

Insightful post. 

I agree with you that when you look at it from a complexity-of-DW perspective, Data Warehousing as a Service seems like a complete non-starter. However, if you think about what mid-market companies are desperate for, it may be much more achievable. As you said, for these companies, it's an extension of what they try to do with Excel and pivot tables. 

What we hear every day is: "I wish I could access some data, slice and dice and pivot it, chart it and put it on a dashboard." They don't want DW, they want reporting and analytics. And they are so desperate for data - any data - that they are much more tolerant of what people in DW land would consider dirty data. "Complete views" or "single versions of the truth" are red herrings. 

Big companies are OK having a "faceless" DW because they have the resources to build UI or reporting on top of it. The rest of us can't.

-Sam Boonin sam@gooddata.com

GoodData

GoodData Developer Network

 

The ThinkerNet does not reflect the views of TechWeb. The ThinkerNet is an informal means of communication to members and visitors of the Internet Evolution site. Individual authors are chosen by Internet Evolution to blog. Neither Internet Evolution nor TechWeb assume responsibility for comments, claims, or opinions made by authors and ThinkerNet bloggers. They are no substitute for your own research and should not be relied upon for trading or any other purpose.
previous posts from Sean Gallagher
Sean Gallagher
Sean Gallagher   9/2/2010   6 comments
The Internet in all its forms has become a core part of how we communicate, socialize, and handle very personal business every day. But protection of individual privacy is spotty at best, and it seems to be getting worse every day. As we become an increasingly digital nation, do access to, and privacy on, the Internet become civil rights?
Sean Gallagher
Sean Gallagher   8/18/2010   11 comments
On August 10, Microsoft Corp. (Nasdaq: MSFT) issued its biggest-ever set of “Patch Tuesday” security bulletins and fixes. Redmond posted 14 new bulletins, in addition to the bulletin posted August 2 after another vulnerability became public.
Sean Gallagher
Sean Gallagher   8/16/2010   18 comments
In July, American Eagle Outfitters -- which has a market capitalization of $2.52 billion -- had the sort of apocalyptic outage that many companies fear. Its e-commerce site was down for eight days. Both a primary and a secondary storage system failed, backups wouldn’t restore properly, and a disaster recovery site wasn’t provisioned properly.
Sean Gallagher
Sean Gallagher   8/13/2010   8 comments
Richard Whitt, Google (Nasdaq: GOOG)’s Washington telecom and media counsel, issued a “Myths vs. Facts” talking points post on the company’s public policy blog today, on the heels of outcry over Google’s agreement in principle with Verizon Communications Inc. (NYSE: VZ) on ground rules for the net neutrality debate.
5
of
IETV: the thinkerNet on film
5
of
2pm EDT
Thu
Sep 30th
an IBM information resource
sponsored content
big blue blog
an IBM information resource
sponsored content
Getting to Work on Smart Work: How IT Is Transforming the Implementation of the 'Internet of Things'
Organizations in all industry sectors are becoming more instrumented, interconnected, and intelligent -- and that's changing the way they approach virtually every facet of their operations. It's up to IT to help organizations adopt a "Three I's" approach that leverages the emerging Internet of Things and enables them to work smarter.

READ THIS eBOOK
your weekly update of news, analysis, and
opinion from Internet Evolution - FREE!

REGISTER HERE
Wanted! Site Moderators
Internet Evolution is looking for a handful of readers to help moderate the message boards on our site – as well as engaging in high-IQ conversation with the industry mavens on our thinkerNet blogosphere. The job comes with various perks, bags of kudos, and GIANT bragging rights. Interested?

Please email: moderators@internetevolution.com
Internet Evolution – not for thickies
Singer at C-Level
Is There a Sequel to SQL?

8|24|10   |   2:03   |   No comments


It’s time to get ready for the rise of noSQL databases. Michael is excited.
Singer at C-Level
Save the Mobile Worker!

8|16|10   |   1:58   |   4 comments


There are still too many barriers to deploying a successful mobile workforce. Sigh.
Mary E. Shacklett
Prepare for Next-Gen Virtualization

8|3|10   |   2:07   |   2 comments


Cisco's UCS and IBM's zEnterprise have upped the ante for virtualization and 21st century computing. In the future, look for integration of disparate operating systems at the firmware level, self-healing architectures, and workload optimization across entire data centers.
Wisdom of the Big Chair
Cloud Computing: Good for the Planet? Hmm…

8|2|10   |   2:25   |   2 comments


There are many reasons why a company may opt for a cloud computing service, but environmental benefits may not be among them.
Second Shooter
Cisco’s Tablet: Too Much to Swallow?

7|6|10   |   2:08   |   No comments


Cisco's Cius is supposed to be an enterprise tablet, but it's positioned a lot like a simple videoconference adjunct. Cisco needs to improve the Cius’s cloud client capabilities, or enterprises may pass in favor of Apple.
Eurotrash
Cloud Services? Get Real!

7|1|10   |   1:04   |   1 comment


Vendors need to focus more on the practicalities of cloud services and less on hyperbole, says Ray Le Maistre.
Singer at C-Level
Thin... Clouds?

3|29|10   |   2:07   |   No comments


Companies are rushing, cash in hand, to adopt cloud computing, but what about those maintenance costs involved with the PCs in the office? Are thin clients a funky good choice?
Singer at C-Level
Goldilocks & the Data Center

2|4|10   |   3:39   |   2 comments


What kinds of companies are doing the most innovation in the data center? Turns out it's midtier enterprises that are taking the "Just Right" approach.
John Soat
Technology Santa Claus

12|23|09   |   2:06   |   4 comments


In the holiday spirit of giving, Technology Santa Clause offers a few words of advice to struggling IT professionals: ‘Be careful what you wish for.’
Sebastian Stadil
The Basic Economics of the Cloud

12|11|09   |   2:56   |   3 comments


The problem with infrastructure these days is not the cost of the network but the cost of the people managing the network. Sebastian Stadil discusses how he'd like to see companies evolve towards a more manageable infrastructure using cloud computing.
Wisdom of the Big Chair
More Texting, Less Bandwidth

9|2|10   |   1:56   |   No comments


Nielsen’s recent numbers on the increasing use of texting bode well for enterprise networks. Shunning the phone in favor of text messaging could mean reducing bandwidth.
Reiter's Block
RIM Caving on Security

9|2|10   |   2:31   |   2 comments


RIM is giving in to demands by India to snoop on encrypted BlackBerry data. It's time to develop cheap or free encryption software for BlackBerrys and other cellular phones.
Second Shooter
Taking Copyright Protection Too Far

9|1|10   |   2:08   |   No comments


Two studios have filed suit against an ad broker for placing ads to help monetize P2P sites suspected of copyright infringement. That's taking a dangerous step toward what might be a worthy goal.
Singer at C-Level
Video in the Cloud

9|1|10   |   2:16   |   2 comments


Software giants are looking for cloud solutions to support our insatiable appetite for video. There will be blood. Yum.
Mary E. Shacklett
Wish List for Mobile Devices, Part 1

Part 1 of 2   |  
See complete series
8|31|10   |   1:41   |   2 comments


By 2014, mobile devices will overtake laptops as the appliance of choice for consumers. But device makers still have some wishes to fulfill, including mobile app simplification and the ability to better perform word processing/spreadsheet functions.
Second Shooter
Google Shifts From Free Content

8|31|10   |   2:14   |   6 comments


Google's foray into pay-for-view movies may be an indicator that the days of free ad-sponsored content are numbered, or at least that ad sponsorship won't fund nearly enough content.
Sweeney Blog
A Sharp Website

8|30|10   |   2:27   |   6 comments


Pencil sharpening gets the digital and artisanal touch, just in time for test-takers everywhere.
Mary E. Shacklett
Online Education Gets a Boost

8|30|10   |   2:02   |   8 comments


Online education, improving to better replicate the interactions that occur between teachers and students face-to-face, grew in double digits during the recession. Still, there’s more work to be done.
Reiter's Block
Educating Bill Gates About Education

8|27|10   |   2:34   |   8 comments


Bill Gates says where you study is becoming much less important, and the best college lectures will soon be found online. Reiter disagrees.
Second Shooter
Gmail & VoIP: Death to PSTN?

8|27|10   |   2:09   |   18 comments


Google's decision to link VoIP calling of PSTN numbers with Gmail, and to let Google Voice "call" Gmail VoIP clients, will devalue the PSTN and force telcos to fund unprofitable services or create their own VoIP transitions.

Enabling People and Organizations to Harness the Transformative Power of Technology