BI Publisher – OC4J Crashing

BI Publisher – OC4J crashing with “Out of Memory Error”

Repeatedly customers get the “Out of Memory Error” when they are running high volume/complex BI Publisher reports (formerly Oracle XML Publisher) on 32bit environment using OC4J. Why you get this error when you try to generate these kind of reports in BI Publisher and how to resolve this issue?

There are multiple ways you can resolve this issue.

1) Login to OC4J server and Increase the OC4J timeout in server properties because sometime it take more time to fetch the data and oc4j has timeout parameter which exceeds timeout limit and your oc4j crashes.

2) Modify/tune the SQL to avoid complex join to fetch the results faster. By doing this it will use less memory and return the data within timeout timeframe. Also since you are running on 32bit environment change the multi-threading to single threading. It will run slower than normal but you may get the results back.

3) In oc4j_cmd by default the memory is set to 256K. In 32bit environment oc4j is allowed to use maximum of 1536K memory out of 4GB which is Windows has a limit on 32bit environment. This is more than enough but sometime if you have reports which has more than 2000 pages with auto bursting reports (rare case which I have experienced). You can change the memory in oc4j to 512K to 1024K or 1536K by using following command:

Take the following steps to change the heap size values for an OC4J instance:
1. Navigate to the Home page for the OC4J instance.
2. Click Administration.
3. If necessary, expand the Properties section of the table by clicking the Expand icon. Then, click the Go to Task icon in the Server Properties row.
4. In the Command Line Options area, change the value in the Maximum heap size and Initial heap size fields.
5. Click Apply.
6. Navigate to the Cluster Topology page, select the OC4J instance that you modified, and click Restart. On the Confirmation page, click Yes.
This changes the size of the heap allocated to the OC4J process (only for that single instance). If your Oracle Application Server topology includes more than one JVM on the same system, The heap changes must be apply to each JVM.

OR go to C:\OracleBI\oc4j_bi\bin folder and modify oc4j.cmd file, save and restart the OC4J service.

Also you can monitor the heap size by logging to OC4J and check under performance tab:

4) If you are running Presentation server on Windows server, the oc4j memory is shared by BI Presentation sever and BI Publisher. You move the presentation server from OC4J to IIS by re-installing the OBIEE on window box. During installation, it will provide an option for you to use OC4J or IIS for BI Presentation service and you should choose IIS. This way it will release the memory which is used by Presentation service. Also In future if oc4j crashes, still Users will be able to login on OBIEE and run Dashboards & Reports and won’t effect them since your presentation services configure on IIS.

Free to Blog

What a great opportunity for me to speak freely on the OBIEE Blog site. Working with the team here at BICG - outside of direct reporting into a software vendor - inspires 'more honest' advice.

I've seen many versions of BI over the last thirty years. In the early years we had to figure out how to join VSAM files with IMS data and produce a report. Of course, this was in that funny era when we made the leap from card based editors to CRTs. Maybe thats going back a bit too far for any relevent blogging.

Anyway, I thought there might be some interest out there in knowing how to get from say Cognos to OBIEE or Hyperion along with - or - conversion to OBIEE. I am big on "How-To's" so I'll take that approach going forward.

One thing I learned to do early on is to write about things in the right order so I'll offer some dialog to the Bloggers out there. What would you like to hear about first? Moving from Hyperion into OBIEE without getting burned - or - moving from Cognos to OBIEE?

BI Publisher and OBIEE, Part 1

BI Publisher originated as a Java application called "XML Publisher" (hence the "xmlp" naming convention) embedded within Oracle's E-Business Suite. XML Publisher has been an integral component within E-Business Suite as a means to deliver highly refined PDF-formatted output of raw transactional data, primarily for hard-copy printing.

In its incarnation as Business Intelligence Publisher (hereinafter "BIP"), it is essentially the same application at heart but with several powerful new features. Like XML Publisher, BIP is a XML transformation engine that applies XSLT templates (created on the fly from user-generated WYSIWYG templates in RTF format) against raw data to produce reports with highly customized visual layouts in a variety of output formats.

XMLP differs from BIP in two significant ways:

First, BIP has much greater flexibility using a variety of template sources, output formats, delivery methods and scheduling options.

Second, and more importantly, BIP is not restricted to E-Business Suite as a data source - in fact it has been expanded to interface with a wide variety of applications, including JDBC-complient databases, Webservices, Hyperion... and OBIEE.

From a high-level pespective, BI Publisher...

• transforms raw data from various sources (JDBC, Webservices, OBIEE, etc)
• according to a layout designed using native functionality of best-of-breed layout tools (Word, Excel, Acrobat, etc)
• into various file formats (HTML, PDF, Flash, CSV, XML, XLS, etc)
• and delivers them via various mechanisms (Email, FTP, HTTP, WebDAV, Fax, etc)
• either on demand or according to a user-defined schedule
  

Any organization making the investment into OBIEE should understand and take advantage of the extensive data formatting & delivery functionality provided by BI Publisher.

This discussion is divided into two parts illustrating BIP's basic functionality within the context of its two native OBIEE interfaces:

1. with BI Presentation Services as a specialized integration source, wherein Answers Requests are treated as a unique data type (Part 1)
2. with BI Server as a JDBC-compliant SQL database, wherein Presentation Tables are treated as simple database tables (Part 2)
   

Assumptions for this discussion:

• BIP was correctly installed and configured onto a demo system during a full OBIEE installation process (full installation & configuration of BIP can be tricky, but is out of scope for this discussion - see Oracle's documentation for more info)
• OBIEE on this demo system is serving the demo "Sample Sales" repository
• MS Office is installed on the client system (screenshots herein represent MS Office 2007 but should adequately demonstrate functionality for previous versions)
  

PRE-REQ: Install "BI Publisher Desktop" (interfaces desktop client apps Excel and Word with BI Publisher)

1. Log in to OBIEE as Administrator
2. Click to "More Products - BI Publisher"
3. Click to "Business Intelligence"
4. Click on "Template Builder" (in Developer Tools) - Save BIPublisherDesktop.exe then run
   

PART 1: Integration with OBIEE Presentation Services (via Answers Requests)

BIP can interface with the OBIEE Presentation Services as a data source. BIP "sees" existing Answers requests and treats them as reports ready for formatting & delivery. In this exercise we will create a simple Answers request, then connect to it from BIP and build a simple template using MS Word, then observe how we can use native functionality in MS Word to modify the resulting output.

CREATE ANSWERS REQUEST

1. Click to Answers
2. Open the "Sample Sales Reduced" Subject Area
3. Create & save new Request titled "BI Publisher Request" in your personal folder with the following dimensions & facts:
   Products . Brand
   Time . Year
   Facts Revenue . Revenue
   

CREATE BI PUBLISHER REPORT

1. Open BI Publisher ("More Products" - "BI Publisher")
2. Click to "Shared Folders" - "Business Intelligence"
3. Click to "Create a New Report"
4. Enter Name = "Answers Request Demo"
5. Click "Create"
   
   
6. Click "Answers Request Demo" - "Edit"
7. Click "Data Model" = "New"
8. Enter Name = "BI Publisher Request"
9. Select Data source = "Oracle BI Answers"
10. Click Flashlight icon
    

    NOTE: Directory structure matches Web Catalog structure - because BIP here is communicating with Presentation Services

11. Click "users" then [your username] then "BI Publisher Request"
12. Click "Save" (green disk icon in upper LH corner of screen)
    

CREATE TEMPLATE USING WORD

1. Open MS Word
2. Click to "Add-Ins"
3. Click "Oracle BI Publisher"
4. Click "Log on" - Pop up window will appear
   NOTE: Value for Report Server should be the local OBIEE install
   
   
   
5. Click "Login"
6. "Open Template" window will open
7. In "Workspace" drop-down, select "BI Publisher"
8. Expand "Shared Folders" then click "Business Intelligence"
9. In the "Reports" area, click on the "Answers Request Demo" (which you created in BIP above)
   

   NOTE: Directory structure matches BIP structure - because Word here is communicating with BI Publisher

10. In the "Layout Templates" area, click on "New"
11. Click on "Open Layout Template"
    
    
    
12. Click to "Add-Ins"
13. Select "Insert" - "Table wizard"
    
    
    
14. Select report format = "Table" and click "Next"
15. Click "Next" (Data Set = "ROWSET/ROW")
16. Click >> (selects all available fields), then "Finish"
17. Select "Preview" - "PDF"
    
    
    
18. Obey prompt to save document in RTF format
19. PDF viewer will render output
    
    
    
20. Use "Add-ins" - "Insert" to create new objects, then experiment with native Word formatting functionality to create your desired report layout
21. When ready to proceed, select "Add-ins" - "Oracle BI Publisher" - "Upload Template As..."
    
    
    
22. Enter Template Name = "Answers Request Demo Template"

REVIEW NEW TEMPLATE CONFIGURATION IN BIP

1. Return to BI Publisher
2. Click to "Shared Folders" - "Business Intelligence"
3. Click "Answers Request Demo" - "Edit"
4. Click "Report" - "Layouts" - "Answers Request Demo Template"
   

   NOTE: This is the RTF template which you created in MS Word above

5. Click "View" (link in upper RH corner)
   
   
   
6. Voila!
   

PART 2 Integration with OBIEE Server (via Presentation Tables)

(To be continued...)

Oracle BI High Availability: Part 3

The following is the third installment on the topic of High Availability within an OBIEE environment. As I mentioned in previous posts, much of what I’ll be discussing was covered in an Oracle eSeminar which I recently viewed on the subject. To quickly summarize, our basic goal within an HA implementation is to provide multiple instances of all components from the BI Server all the way to the end user, so if anything fails, we have another instance of the same component ready to go. The first two posts on the subject can be found in the blog archives if you’d like to rewind. This installment will feature the HA connections between the Presentation Servers and BI Scheduler Servers as well as between the Presentation Servers and BI Servers. We’ll also look closer at the BI Scheduler Cluster configuration.

First we’ll look at the connection between the BI Servers and the Presentation Servers. This again is handled by the Cluster Controller. In a Windows environment you’ll need to go into the Administrative Tools on each Presentation Server and set up the clustered ODBC data source. To do so, simply ensure that “Is this a clustered DSN?” is checked, then specify the primary and secondary cluster controllers and ports. On a Unix/Linux box, you’ll need to make the following changes to the odbc.ini file:

IsClusteredDSN=Yes

PrimaryCCS=<PrimaryCCS>

PrimaryCCSPort=9706

SecondaryCCS=<SecondaryCCS>

SecondaryCCSPort=9706

One important note to keep in mind is that if you have any clients from which you want to access your repository in online mode, you’ll need the clustered DSN set up on these as well. They’ll need the same connection to the BI Servers as the Presentation Servers will.

The BI Scheduler Cluster Controller assigns the active Scheduler server. In an HA environment, you would have two cluster controllers, a primary and a secondary, in an active/passive relationship. The secondary server will not be used unless the primary is unavailable. The client Controller ports are specified in the respective NQClusterConfig.INI files. The Scheduler configuration will be handled by the Cluster Controllers, so all we need to do in the instanceconfig.xml file is point to the Cluster Controllers, as shown below:

<Alerts>

<ScheduleServer>

ccsPrimary=”<Primary Cluster Controller>”

ccsPrimaryPort=”<Client Controller Port>”

ccsSecondary=”<Secondary Cluster Controller>”

ccsSecondaryPort=”<Client Controller Port>”

</Alerts>

</ScheduleServer>

The other task you would need to complete is to add the BI Scheduler Administrator credentials to the credential store of each ps. The quickest and easiest way would probably be to copy the credential store file from one instance to all other presentation servers.

A few notes about what will occur when the Active Scheduler fails. The transition from one server to another is seamless to the user’s perspective. The users won’t receive any errors, the Cluster Controller will simply detect the failure on the active server and point to the secondary. Any jobs which didn’t complete will be picked up where they left off. One important note to remember is that once the primary server is back up, it will not automatically resume the primary role. Only after the services have been restarted will the primary Scheduler resume its proper role. If any Java, command line, or script jobs are being run during an interruption, they will be restarted when another server is activated and given a new job ID. Take a look at the diagram of the basic HA architecture for the Scheduler Servers.

To configure the Cluster Controller to talk with multiple Schedulers, you must make the following entry in the NQSClusterConfig.INI file on each Cluster Controller:

SCHEDULERS = "scheduler1:9705:9708", "scheduler2:9705:9708";

The first port number for each scheduler will be the rpc port, where the Scheduler will be listening for connections from the Scheduler. The second will be the monitor port, which simply listens for a “heartbeat” from the Scheduler, to confirm it’s still available. This is how the Cluster Controller determines that the primary server has gone down and it needs to look to the secondary server. The default ports for the rpc and monitor ports are 9705 and 9708, respectively.

Configuration of the BI Scheduler Servers themselves can be completed through the Job Manager or via command line using schconfig. All configuration settings are saved in the instanceconfig.xml file of the Scheduler folder, not to be confused with the file of the same name in the web\config folder. The Scheduler can be configured to talk with multiple Presentation Servers and Java Hosts, which will be necessary if you are setting up a true HA environment. If you are using the Scheduler to run script files, you must place them in a shared network file, so that multiple Scheduler Servers can access them. All Schedulers should have read/write access to these files.

That’s going to wrap it up for this post. I’ll finish things up next time with a closer look at the BI Server cluster and how it’s integrated into the HA environment.