BI, Uncertainty, and the Two Watch Paradox - Part 1

There is a well know quote about certainty that goes something like "A person with one watch always knows what time it is; a person with two watches is never sure". I was struck the other day by the pertinence of this quote to the process of working with clients who are implementing a new BI solution which replaces existing reporting tools.

Many organizations are like the man with one old watch - they may not have much in the way of reporting, and the reports may be labor-intensive and involve combining the results of multiple queries on different source systems, and may be of indeterminate accuracy, but the end results tend to take on a kind of hallowed authority and become the organization's defining vision of reality. These organizations "know" what time it is, and accurately enough for everyday purposes, it seems.

Whatever the promise of a new BI system, whatever savings in staff effort, broader reporting scope, expanded data visualization capabilities and increased accuracy it delivers through state-of-the-art ETL, OLAP databases, and automated, scheduled report generation, it still will inevitably need to deal with the fact that, at least initially, it is the proverbial second watch, the one that shatters the certainty of "really" knowing what time it is. Because, inevitably, based on the complexity of the organization's business rules, and the effect of applying these complex rules in to arcane data configurations, the values of accepted metrics will vary with the results of previously accepted reports, and this variance will cause angst.

And getting through this "two watch" phase is critical, and is made more difficult because the watch that is invariably believed is the one which has been depended upon for all the preceding years. In fact, projects can get permanently bogged down trying to exactly match these pre-existing reports as de facto acceptance criteria for the new system.

In the next post, I'll discuss some ideas on getting through this problematic phase.