OracleBIBlog Search

Thursday, March 25, 2010

How to Start Testing during an OBIEE Project

How to start testing during an OBIEE Environment:

In an earlier posting I discussed the importance of and where to implement testing in the Project Life Cycle for an OBIEE Application. In this posting I want to discuss further how to implement testing for the application and some of the major steps during the Project Life Cycle.
During the Project Initiation Stage of the Project Life Cycle the first step in testing is to develop a Test Strategy. The Test Strategy outlines the recommended approach to the testing processes during the Project Life Cycle. Some of the key components of the Test Strategy are:

· Purpose and Test Objectives
· Quality Standard
· Test Phase Ownership
· Test Phases and Test Types
· Test Plans
· Test Case Repository
· Test Case Design
· Defect Tracking System
· Severity Levels of Defects
· Test Tools
· Test Reports

Since testing will be incorporated of the Project Life Cycle, it is critical that all project team members be familiar with the test strategy and understands their roles in the Project Testing Processes. All project team members will be involved is multiple phases of the testing processes, and it is important that they understand their role in testing and the roles of the other project team members in the testing processes.

Once the Test Strategy has been completed by the Testing Team and approved by the Business Users the next step is to start the Project Test Plan. The Test Plan is started during the Project Requirements Phase. As the requirements are being gathered it is important to identify how the requirements can be validated and tested to insure that they are met by the provided application. The development of the Test Plan involved all members of the Project Team as well as the Business Sponsor and Users. Once a requirement has been defined it is important to have some way to substantiate that the requirement has been met by the application. While looking for ways to test a requirement, it also helps the business user better define and understand the requirements. In most cases the business users have a good understanding of their source application, but have very little insight into the metrics and attributes needed by measure and manage their business processes. Also when a business application is used by many different groups they may have many different business processes to do the same functionally. While gathering the Business Requirement and Testing Requirements, there may be many different test cases to measure the same functionally across different departments in an organization. It is important to gather this testing information to insure that the business requirements can be validated by the different departments. One of the important steps that are often overlooked in testing is to test the actual requirement to insure that is valid and verifiable to measure and manage their business processes. Unfortunately this is often not the case and not until the business users see the Dashboards and Reports do they understand the value of the metrics and attributes for measuring their business processes. The Test Plan can help to identify the type of Testing and Test Cases that need to be developed to validate the requirements. Some of the items that can be included for testing are:

Validate the access to the System – Security
Validate the navigation to a particular screen, dashboard, page, and report
Validate access to the right data, screens, dashboards, pages, and reports
Validate the response time to access the application
Validate the response time to access a particular dashboard, page and report
Validate the capability and ease of using prompts to select data on a report with specific attributes
Validate the use of drilldown on Reports
Validate the use of navigation on Reports
Validate that the attributes that are used to display the metrics are correct
Validate that the metrics on the Reports provide helpful and valuable information to help them measure and manage their business processes

The next major testing task is to develop test cases to validate the user requirements. The Test Cases are usually designed and built during the design and build stages of the Project Life Cycle. In most situations the Test Cases are iterative, because more details are fleshed out during the Project design and build stages. For each requirement the minimum information needs to be developed for each test case:

Test Case Number
Test Case Summary
Test Case Description
Test Case Steps
Test Case Expected Results
Test Case Error Condition
Person responsible for Test Case

Once the Test Cases are built and agreed upon by the Project Team and Business Users, and the design and build stages for the application are complete then the execution of the Test Cases can begin. One major advantages of building the Test Cases early in the Project Life Cycle is the Development Team will understand the test conditions and can execute many of the Test Plans during Unit Testing. Quality is an important of any project and each team member is responsible for building quality into the finished product. By using the aforementioned Testing Processes quality can be built into the project from the earliest part of the Project Life Cycle, instead of waiting until the finished product is turned over to the business users for user acceptance testing. If testing is only implemented during user acceptance testing many of the quality issues that would have been identified and corrected in the early part of the Project Life Cycle will come to light, and will cost more in terms of resources and money to correct than if they were identified and corrected in the earlier stages of the Project Life Cycle.


oraclenerd said...


While I agree testing is very important...I just don't think this is ultimately manageable. You might get this in the first cycle but as time goes on much of this will not be maintained (i.e. documentation).

The exception(s) of course is having a leader who recognizes the importance of testing (dictator). Out of the 7 or 8 shops I have worked in, 1 had and maintained this stuff. While I enjoyed the benefits, it was a lot of work and brought initial development to a crawl.

Hopefully I am not making excuses here, I love testing, I think it adds a lot of value...I have just found that most people will NOT do it.


Ed Martin said...

Thanks for the feedback Chet. I agree that detailed testing is not done in most shops. That is one of the major reasons projects run over time and budget. There needs to be some negotiation about how much testing that needs to be done and how much. That is one of the reasons that I feel that a Test Strategy Document needs to be done at the first of the project. If all team members are aware of the importance of testing, then they can accomplish much of the testing during the design and developmemt stages.

Anonymous said...

Initial development may be brought to a crawl for a number of reasons:
- the testing was arbitrarily rigorous, meaning everything was "beaten up" for perfection irrespective of its relative importance
- initial development actually was prototyping to figure out what is needed, at which point the test data will change every 5 minutes, along with the evolving requirements baseline

I agree that most people will not do it. It isn't fun, it doesn't give the false feeling of momentum that sitting down and slinging code gives (yes, I'm guilty). Most organizations don't figure out how to do it well ("more" is not always "well"), but they eventually figure out the cost of not doing it at all.

I've made a lot of money working with organizations that take on the same amount of new requirements each year, while having the previous year's defects get tacked onto the current effort. When you consider that a defect is just an unmet requirements, you have ballooning volume of requirements each fiscal year. Eventually more money goes into stuff that was supposedly done last year than doing the stuff that supposedly will get done this year. Almost invariably, testing gets its budget, project involvement, and time cut and -- voila! -- even more defects escape from year to year.

The cure is complicated, and it is hard. You have the balance between analysis paralysis at one extreme, and hack-and-ship at the other. It takes management commitment to working out the right mix without sticking our collective heads in the sand and hoping it gets better.

Your ratio of 1 in 8 shops doing this isn't that surprising. Their not getting it just right isn't either. The few that get it have a competitive edge of one sort. Those that don't can charge beta test fees to have their user community pay to provide product testing!