Are your Performance Management metrics useful?

Performance Metrics are the heart of any BI/EPM implementation, but often, the ability to effect change in the metrics is not seamless. In this blog, I'll explore three key factors related to less-than-optimal usage of Performance Management Metrics and suggest practices to improve application of metrics to an organization's benefit.

Metric Background
Metrics are intended as a simple measurement of activity. By developing and publishing organizational metrics to decision makers, the labor intensive effort of collecting source data, manipulating or adjusting the data, and ultimately calculating metrics, is eliminated. Unfortunately, this also eliminates a large part of the knowledge capital necessary to understand how to influence metrics.

• Key decision makers expected to act upon metrics must understand the full formula for deriving the metric. Only with this information can the best decisions regarding how to improve a business's performance be made.

Relevance of a change in a Metric's value
Similar to having an understanding of the makeup of a metric, organization decision makers must have an understanding of the relevance of a change in the value of a metric. Often, formulas producing metric results mask exponential changes in source values to appear to be linear changes. Over time, this can result in less efficiency from additional attempts to improve a measurement.

• Organizations must have an understanding of the relevance of a change in a key metric at different points. A thorough understanding of the historical change, as well as an analysis of the point at which further improving a metric results in materially diminishing returns, will guide the organization's decision makers to base their decisions on the most accurate basis at a point in time.

Industry Trends
Over time all organizations undergo changes to their business model, whether due to internal, external, or a combination of factors. Flexible metrics can enhance the ability for an organization to respond to changing market dynamics by quickly refocusing key decision makers to respond to business model changes.

• Frequent review of Performance Metrics for relevance to the current business climate allows for improved competitive advantage by reducing the time necessary to refocus an organization.

In conclusion, Performance Metrics are an invaluable tool to improve the decision making intelligence of an organization. However, to leverage maximum long and short term benefit requires a commitment to supporting the organization with the best possible guidance. Organizations that embrace their Performance Metrics as an avenue for executing strategy can realize significant gains in competitive advantage.

Real World Analogies in OBIEE

As OBIEE emerges as a cornerstone part of Oracle’s Web 2.0 Fusion offering, it is a good time to discuss a real world analogy that has been apparent to me for many years now and has generated excitement during planning sessions of the more innovative implementation teams I’ve been on. OBIEE is an excellent web tool, for its dashboards and navigations, and it is important to look at User Groups as a cohort of individuals with shared goals who need to use OBIEE as a communications tool and as an effective analytic tool. A tool that has been useful in the real world for doing this is the fantasy sports league application, and it gives us many valuable insights for achieving uptake in our OBIEE implementations.

Although it is ironic that an application with the word “fantasy” in its title would be a useful analogy to solve our business problems, it is still true. Fantasy sports leagues have been very successful in the real world as an analytic tool that has had mass uptake. The application needs to be effective in helping groups to complete their assigned work, and to make better decisions, which are both key goals of our OBIEE implementations. Fantasy leagues are used as a communication vehicle for real world business groups that are looking for ways to interact, such as a college cohort looking to stay in touch after graduation, or a company contest. The value of contests as a business tool for generating ideas is well documented. Google’s programming contest, which started in 2002 has generated tens of thousands of applications, is responsible for important real-world systems like Google Local and Professor-Verifier [The Google Way, Chapter 7: Look for Ideas Where They Are, Bernard Girard, No Starch Press 2009].

The history of the fantasy leagues, more than thirty million strong today, is curiously analogous to what we want to achieve with OBIEE. From 1960 to 1980 it was a tool used by the intellectual elite, like Harvard Sociology Professor William Gamson’s “Baseball Seminar”. Rotisserie leagues started getting media attention in 1980, and special notice was given to the fact that data was used in real time, statistics from the current season. The business of statistical analysis matured in the fantasy sports field after its popularity created a demand for optimized predictions of specified Key Performance Indicators that scored points in the leagues.

Fantasy sports as a web application online is currently used by over 30 million people, creating competition for the best analytic dashboards to attract the most users.

So what do analytic dashboards used by these fantasy leagues look like, and how can we do that in OBIEE dashboards? While graphs may be important, note that for the masses these graphs are not extremely useful. Tabular lists dominate fantasy sport websites. The lists must default to the most useful Top-N and be re-sort-able by column. This is a common feature of tabular reporting in OBIEE, but it is important to understand that in the real world, other more graphic features do not undermine the importance of putting our data in this fashion on dashboards. The ability to quickly navigate from one presentation of data to another presentation of data with the same visual representation, a tabular, re-sortable presentation, is the dominant feature. Visual keys, surprisingly, are most useful when they simply give us more information on that representation. For example, ESPN has a Top-N page of multiple major categories, and the graphic is simply a photograph of the face of the top person in that category. In OBIEE, it might be useful to expose more photos of our business individuals, such as the business owner responsible for the dashboard. If it were a KPI were measured by team performance, a team logo might be useful. More sophisticated analytic tools are for a different audience, and follow the needs of the statistical analysts who are part of smaller decision teams, so they do not get exposed on the dashboards used by most of the people who consume the application. This might be an equivalent of a text alert on the screen that aggregates the number of Monte-Carlo simulations that were done, but exposing details of the thousands of simulations would not be possible.

BI Applications in 7.9.6 already have done some good work in providing a user experience. We just have to pay careful attention about how to implement the system. For example, built into the out-of-the-box experience are two sets of dashboards. The set of dashboards that is used by most people has the common presentations similar to fantasy leagues, such as the Top-N type of presentation. When you see demos of the tool, they default to the graphics, but there is a view selector that allows tabular lists of the data, and I would suggest considering in most cases whether the default presentation when coming on-screen should be tabular to reduce clicks and increase usability. The other set of dashboards spans multiple subject areas. These are pre-built statistical analyses available for when decisions need to be made upon the data, and include scatter charts, regressions and tools common to a smaller audience, the more sophisticated decision analysts.

Another question I have been asked in OBIEE implementation discussions is whether a contest for who uses the dashboards the most is possible. This would be analogous to the fantasy sports league itself. The answer is that it is very possible, in fact we do monitor usage in the OBIEE implementations, and more sophisticated applications are already developed. BIConsultingGroup’s product IMPACT gets more sophisticated reporting out of the usage of the end users. A dashboard could be designed and written with the specific intent of making a usage contest out of the usage analysis information, with the key business interactions as the key performance indicators. For a more comprehensive analytic application, the design could integrate an application similar to a fantasy league draft, where draft results for units in the sales pipeline could provide interesting information about predictions about which sales in the pipeline have the best chance of success.

What are design teams responsible for? Blogger Jeff McQuigg writes, “A BI system is a well thought out, planned and coordinated collection of efforts designed to produce a system that is so well organized it allows your user community to ask sophisticated questions of it and get those answers quickly and with a high degree of accuracy.” Fantasy leagues provide us with a successful real world analogy that we can use for building great OBIEE dashboards.

Training Qualm

If you are like me you get many emails from newsletters, blogs, and e-magazines, which also includes offers to attend seminars for various different topics. I recently received one that offered a topic that I would have liked to hear. Then I saw that it was only being offered somewhere in California.

In today's market with companies saving every dollar they can hold on to, it is hard enough to justify reasonable expenses and even harder to justify things like seminars or training courses. I certainly do not have the money to spend on them. Some companies are better at others in investing in their talent pool than others, but even they typically do not send all their talent. They will rely on the person or few people to bring back that information and spread their knowledge. So why is it that these Guru's or Training Professionals do not provide more web based seminars and courses? With today's web technology it is very easy to provide professional interfaces for this sort of purpose. VoIP is mainstream which allows them to completely control registration and access to the material, and best of all it saves the registrants money, not to mention the hassle of expense reports, travel arrangements, and of course the actual travel. And with the ever increasing acceptance of a green philosophy, you can save pollution from air and car travel, waste from brochures and displays, and reduce energy and resource consumption.

Sure, there are some down sides to web based conferencing. You cannot really control how many people are viewing or listening in on the conference at the attendant's site, but again with the increase in attendance from people who cannot afford to come, this does not add much in the way of missed profits. Also, if the training is good and you are providing advertising space in the conference room, these non-paying attendees would be a benefit. Even with the technology and software we have, I have yet to see a good way for the interaction of a classroom type of atmosphere. And yes, you are not bringing any money into to the local economy or state, but if you feel that bad about it, donate a portion of the extra profits you gain from more attendees to a local charity.

All in all, most companies that provide training have come to understand this view point. They provide some sort of service to offset the travel expense whether it be "on-site," "on-the-job," or some sort of web based training. But for some reason, these email invitations to half day or one day seminars, courses, briefings, outlooks, etc. have not grasped the idea of sharing these ideas via a live web meeting. And at least in my mind, these small "meetings" should be more abundant than week long training courses.

Kimball vs. Inmon: The TDWI perspective

Surfing through the blog posts trying to be "inspired", I stumbled upon a year old post comparing Kimball and Inmon's architecture. I learned about the differences between their architectures during my courses at the TDWI conference. Only having read Kimball's literature, I was intrigued by this rivalry.

As it was explained to me, Kimball's "Bus Architecture" defines a data warehouse as the combination of all the data marts, which would make the data warehouse responsible for the intake, integration, distribution, delivery, and access of data. Comparatively, Inmon's "Hub and Spoke Architecture" defines the data warehouse as "a subject-oriented, integrated, non-volatile, time-variant collection of data organized to support management needs." Basically, it is responsible for the intake, integration, and distribution of the data. The argument proposed in the course was which was better to use?

To me, it seems this depends on the level of the BI program at the business in question. Is your business new to BI or finding their previous BI project poorly integrated? You could do well to subscribe to Kimball's approach, as it quickly allows the users to get what they need. However, as the BI program matures and more data marts are developed the issue of maintaining the "bus," the rules that define the conformed dimensions necessary for the architecture, become harder to keep aligned. This is where Inmon's approach makes sense, by pushing the conformity back to the warehouse, it is easier to administrate changes to rules and enforce integrity.

True, the line where it makes sense to implement the "hub" in the Inmon's architecture is hard to draw, but aside from that blurred area, I'm not sure there is much to arguing one as "better" than the other. To me it's simply a matter of BI Program Maturity.

BIEE and BI Publisher's integration (Best Practice tip)

BI Publisher can gather data from different data sources, but with regards of the integration with BIEE two in particular are of our interest:

1. SQL Type
2. OBIEE Answers, termed Oracle BI Answers
   

The first type, allows us to connect to any database using a JDBC connection. The second type, allows us to connect to BIEE server by means of BI Answers queries.

What is the difference? Let's see. If we take the route with OBIEE Answers then in BI Publisher we get immediate results without any additional work:

For the above data set we get the following XML output:


Problem with Long Tag Names in RTF Templates

But one of the drawbacks of this approach is that we get long tag names, like for example: <_sales_facts_._amount_sold_>. These long tag names are difficult to handle in BI Publisher for RTF template development. For example, if the formula used to create the OBIEE Answer is a long CASE statement, the tag contains this long statement. This drawback is a problem because of Microsoft Word’s properties limitation. Any situation of form field is limited to 276 chars. That is 138 chars max per tab: Status Bar and Help Key (F1) ) as confirmed below:



Workaround for the Problem with Long Tag Names in RTF Templates
But using the following workaround, we still can access BI Server and leverage all its content. Simply go to same BI Answers query ==> Advanced tab and take the “logical sql”:


Next, paste this logical sql into the data model in BI Publisher and voila, we get the same results but with the labels associated: saw_0, saw_1, … saw_5.

At this point some work is required to mapping the saw_X columns with the corresponding ones in the BI Answers query to make the column names meaningful in our report. Renaming the columns does not present any limitations to our goal of pixel perfect reporting.

Now, if we execute the report just defined then we can see that the XML elements of the output file are shorter and can be accommodated to our taste.

Unlevel Hierarchies: a Quick Workaround

Sometimes, especially when using the Essbase Wizard to expose Essbase cubes to OBIEE, the technical team presents unlevel hierarchies without defaulting to outer join. The result is that when they are selected down to the last level, whichever levels do not have members at the last level do not get into the result set. This is due to the hierarchy being unlevel, or not having a detail value down to the last level being selected. This is called an unlevel hierarchy. Also the term ragged hierarchy has been used to describe such a hierarchy.

Here is a simple workaround. Let’s say there are 5 levels. Make a union query by selecting all columns and repeating it five times as a union of five queries. Then one query at a time, modify one or more of the columns to include a blank instead of the column. One query goes to level 5, one goes to level 4 with blank in level 5, one goes to 3 with blank in levels 4 and 5, one goes to 2 with blanks in levels 3, 4 and 5, and one goes to level one with blanks in levels 2, 3, 4 and 5. This when rendered will produce a hierarchical visualization of the unlevel hierarchy, with all the members visible.

I have used this at clients with much success. Since the Essbase cube had multiple hierarchies, we were able to combine this technique with putting a dashboard prompt at the top of the page where the user selects which hierarchy to view. For example, two hierarchies might be Company before Merger and Company after Merger. The selection which hierarchy to choose would be done for Level 1, and this would affect all the union by putting Is Prompted for Level 1 in the filter criteria.

Circuit 2009

This past week the Circuit conference in Washington DC was held at the beautiful Gaylord National Center. Soldiers milled around the halls the entire week, using the facility for conferences. Credit is deserved to Quest and OAUG, as well as the Real Estate SIG, for putting together a major regional conference in a downturn economy. These organizations would not have been able to make such a great convention without getting together and doing it regionally and across user groups.

Some of the best content I have heard in a long time was available at the conference. The time has come for business intelligence and social media to come to the core of enterprise applications, and Oracle is poised to capture the market demand for these features.

I spoke on Hyperion EPM Production Reporting, and there was a lot of discussion throughout the week on use of reporting in organizations. The vendors were great as well, including the JDEdwards Play Day which is very popular. I would like to focus however, on what I think is a very significant matter that was discussed in the keynote and subsequently demonstrated in the major “what’s new in the next release” presentations at the end of the week.

All of our applications are going to be changing so that the core of the experience is Business Intelligence and Social Networking centric. Better visualizations, OBIEE, Online Chatting, Comments Posting, URL linking, and Tagging are now a PeopleTools upgrade. There is no requirement to do an applications upgrade. As far as the applications themselves, Oracle will sell Fusion as a new application, or customers can choose to use Application Integration Architecture as the integration solution. These are alternate options for us to choose from, and the key theme is "whatever works best for us." The technology that has been here for many years is finally going to get the uptake it deserves, seamlessly, usable, and according to the value it represents.

This is a big takeway. The uptake of business intelligence and social media embedded in applications is going to be dramatic in the coming years. Integration partners and organizations need to look at how this uptake will be supported.