Missing the Point

Mar 18, 2014

It is admissions season at four year institutions and soon it will be summer and fall registration for everyone. It is also the time of year when we see an obsession with precision in the reporting and BI systems as teams and leaders monitor their admissions and enrollment goals. Focusing on precision and not accuracy will leave you flat footed and ill prepared for proper decision making.

What is the problem with being precise? It takes an organization's focus away from their ultimate goal. This happens because the goal becomes all about the number itself, not about what the number represents. The implication then is management teams are no longer spending time thinking about what needs to be done to achieve the goal. To use a well-worn analogy, they cannot see the forest for the trees. It is the biggest and most frustrating issue of myopic behavior we typically see. People think it is important to be precise because either 1) they feel they need to be right, or 2) it is the only way they feel like they have successfully accomplished something. But so what? If you've missed your goal by 344 students vs. 345 students out of 12,000 what fundamental difference does it make? Why focus on the 1 rather than the 345? It will not be enough to change a decision. There is the concept of "immaterial" in accounting that can be useful to apply here.

How would someone even know if a given report is inaccurate? Most institutions have historical copies of data or use parallel reporting systems during a transition phase to a comprehensive data warehouse and BI system. It is important to use these systems for initial data validation, but keeping them around as the "system of record" to compare against the new BI systems is problematic. Occasional errors can happen in sophisticated data warehousing systems because data entry errors or back-dating in the source system cause a load failure for a particular record on a given day. However, forcing staff to redo work to fix small variations is precious time that should be spent on identifying strategies to get the organization back on track towards the goal, not fiddling with numbers.

The argument I often hear is that "well, we have to report this number to our state agency" or something similar. There is an excuse that there is an external driver to the precision because the funding budget is based on the particular number, e.g. FTE. I am not advocating dishonesty, but frankly, the state agency doesn't know the difference between the 1 or 2 variation – they just accept the number as provided. If there wasn't a parallel reporting system, the institution wouldn't have known of the discrepancy either. Furthermore, there are often unknown mistakes in the data anyway. How many times have I heard about a student who didn't drop in time to get their refund, petitioned the Registrar, and then received permission to retroactively drop from a course section after census date? I suspect these are never reported back to the state agencies! It is just modifying the data after the fact that makes it "incorrect" again.

So why obsess about precision? If the discrepancy is not significant enough to change a management action or decision, then it misses the point.