Demantra Engine On Linux
Demantra Engine On Linux
Contents
Introduction .................................................................................................... 5 Deploying Demantra on Linux or UNIX ........................................................... 5 Special Linux instructions when deploying the Demantra Analytical Engine: 5 Hardware requirements for Analytical Engine on Linux ................................. 6 About the Installer .......................................................................................... 6 Configure the Web Application on UNIX ........................................................ 7 Configure the Engine on Linux ........................................................................ 7 Post-install Configuration ....................................................................... 8 Engine Manager Configuration .............................................................. 8 Configure Environment Variables .......................................................... 8 Create the Engine Data Source File ........................................................ 9 Deploy Application Server ...................................................................... 9 Deploying on Oracle Application Server (OAS) ...................................... 9 Deploying on Web Logic....................................................................... 10 Deploying on Tomcat ........................................................................... 11 Engine Configuration ............................................................................ 12 Configure Environment Variables ........................................................ 13 Configure the Engine ............................................................................ 13 Engine Starter Server ........................................................................... 13 Running the Engine .............................................................................. 14 Sample Commands............................................................................... 14 Samba Configuration ............................................................................ 16
Distributed Engine ........................................................................................ 18 Configure Promotion Optimization (PMO) on Linux (Optional) ................... 19 Configure the Standalone Integration Tool (APS Standalone) on UNIX (Optional) 20 Run the Engine Administrator on Linux ........................................................ 22 Calling Oracle Demantra Analytical Engine from UNIX Environment........... 23 Step 1: Install SSH tool on Windows Environment .............................. 24 Step 2: Enable SSH package on UNIX Environment ............................. 24 Step 3: Create a user on UNIX environment. ....................................... 24 Step 4: Create SSH key in UNIX environment ...................................... 25 Step 5: Create Identification file .......................................................... 25 Step 6: Place public key on the Engine Windows server ..................... 25 Step 7: Execute the run command from the Unix side. ....................... 26 Additional Considerations .................................................................... 26 APPENDIX I ........................................................................................... 27 Cygwin OpenSSH Installation ............................................................... 27 Engine Troubleshooting/Common Issues ..................................................... 30 Problem. Engine fails and hangs in recovery mode. ............................ 30 Problem. Navigate core dump file to navigate recorded state of the working status output when program has terminated abnormally to help to find crash reason. ................... 30 Problem. Engine Fails with following DB Error: ERROR DB_OracleConnection::Open threw an exception. Error: 12514 ORA-12514: TNS:listener does not currently know of service requested in connect descriptor............................................................................................ 30 Problem. Failed to run Engine in the workflow: Engine Execution could not start (engine failure). ORA06512 .................................................................................................. 31 Problem. Workflow fails with the following error: network access denied by access control list (ACL). .............................................................................................................. 31 Problem. Engine using OAS fails with the following DB Error: ERROR DB_OracleConnection::Open threw an exception. Error: 12705 ORA-12705: invalid or unknown NLS parameter value specified 31
Problem. Engine is not starting. ........................................................... 32 Problem. main: NLS initialization failed!!" Error apears while starting OAS 32 Problem. 500 Internal Server Error/ Servlet error: An exception occurred. 33 Problem. Engine Manager Log is not created ...................................... 33 Problem. Is Oracle Demantra certified for SuSE operating systems? 34 Problem. Weblogic Exception in Engine Manager Deployment .......... 34 Problem. Weblogic Second Engine Start Causes Exception................. 34 Problem. OCCI createEnvironment doesn't succeed. .......................... 34 Problem. Reconnection problems in Engine Manager ........................ 35 Problem. Engine is not working using WorkFlow ................................ 35 Problem. Is it possible to be install Engine on Solaris? ........................ 35 Problem. If the Engine Manager deployed to window server with tomcat, can the rest of machines (engine blades) run at Linux platform? ............................................................. 35 Problem. Is it possible to deploy more than one engine manager on the same Application server (AS)/machine? ...................................................................................... 36 Problem. Error on binding: "Address already in USE" ......................... 36 Problem. EngineManager Execution Failed, Switching to ENGINE_STATUS_FAILED 36 Problem. ERROR Engine 0 on machine <machine> failed to execute successfully 37 37
Problem. Customer receives status 2 or 1 in the browser after running Engine Problem. Need to upgrade Oracle client on Linux. ............................ 38 Problem. Multiple Engines are configured in Settings.xml but only one was created. Problem. How do I configure specific Log Level Groups? .................... 39 Problem. UnsatisfiedLinkError trying to start demantra engineDescription:Status 40
38
Problem. Engine failed to start and the following error message displayed in the EngineManager log: FATAL ERROR in native method: ReleaseStringUTFChars called on something not allocated by GetStringUTFChars ............................................................................. 40 Solaris Important Notes. ...................................................................... 41
Introduction
Engine consists of Analytical Engine were all the analytical voodoo is calculated, EngineManager that manages Engine(s) by supplying work load and notifications and Engine Shell where pre processing and post processing of Analytics are done. Engine was originally developed on Windows platform. It was important for the customers that Engine will be platform independent. Thus Engine was ported to Linux.
To install Oracle Demantra on a Linux machine, you run the Installer on a Windows machine and indicate whether to deploy the Analytical Engine on Linux and the Demantra Web application on UNIX. The Installer places deliverables on the Windows machine. You move these deliverables to the Linux machine and use them with these instructions to perform the following: Configure the Web application on UNIX. Configure the Engine on Linux.
Configure Promotion Optimization (PMO) on Linux (optional). Configure the standalone integration tool (APS Standalone) on UNIX (optional). Run the Engine Administrator on Linux.
Note: If you are running the Installer on Microsoft Windows Vista, the check boxes may not be visible. This is a known display issue. You can select one or both of these options by clicking just to the left of the text in the area where the check boxes should appear
If you selected Deploy Demantra Web Application on UNIX, the Installer places a zip archive of the files required to install the Web on UNIX. It places it on the Windows machine as <Demantra Install Root>\Linux Delivery\Oracle_Demantra_Unix_Web.tar.gz. It contains the following folders and files: Web Integration readme.txt
If you selected Deploy Demantra Analytical Engine on Linux, the Installer places a zip archive of the files required to install the Engine on Linux. It places it on the Windows machine as <Demantra Install Root>\Linux Delivery\Oracle_Demantra_Linux_Engine.tar.gz. It contains the following folders and files: Engine tools readme.txt
For more information on Installer, see Oracle Demantra Installation Guide About the Installer.
Post-install Configuration
The post-install configuration consists of the following: Engine Manager configuration Engine configuration Engine starter server
In this document, the Web application examples use Oracle Application Server examples. There are similar processes for other Web containers (for example, WebLogic and WebSphere).
Engine Manager Configuration
The engine manager configuration consists of the following: Configure environment variables
Create the Engine Data Source File
ENGINE_ROOT = full path to directory above the place you've putted bin and lib ( For example home/oracle//Engine) ORACLE_HOME = Path to Oracle Application Server Installation or $ENGINE_ROOT/lib/ORACLE_HOME when no OAS is installed (Tomcat for example) JAVA_HOME = Full path to Java 1.5 or greater installation. (JDK or JRE)
Script will define these environment variables into your $HOME/.bash_profile file. Some environment variables may already be defined, for example, ORACLE_HOME and JAVA_HOME, but review their values.
If there is no Oracle instance installed on your machine (no Oracle Application Server or Database), the ORACLE_HOME should point to <INSTALL_ROOT>/lib/ORACLE_HOME.
Run command: source ~/.bash_profile one needs to execute this command for every opened terminal session <INSTALL_ROOT>/lib/ORACLE_HOME/rdbms/mesg includes some Oracle message definitions required by sqlldr (SQL Loader). For more information, see Oracle Installation Guide Configure Environment Variables. Create the Engine Data Source File Run .bin/createDSIniEngineRoot.sh When prompted, enter: Full path to the tools directory, including ds_config.sh Server name - Full name of database server TNS name - Full host name where database server is running Schema name - Database user name Schema password Database user password Database SID Database port number Full path to the ds.ini file - <ROOT DIRECTORY>/bin
This section describes how to deploy the engine (on Linux) on the following application servers: Oracle Application Server Web Logic Tomcat
Deploying on Oracle Application Server (OAS) Perform the steps in this section only if you are deploying the engine on OAS. Generally, <INSTALL_ROOT>/lib/engineManager.war is installed with pack Oracle_Demantra_Linux_Engine.tar.gz. If you need to create it manually, use the following command: cd <INSTALL_ROOT>/lib jar cvf engineManager.war WEB-INF conf
Open the Oracle Application Server administration page in your browser (example, http://<server>:7778/em). Perform the following steps to deploy <INSTALL_ROOT>/lib/engineManager.war: 1. In the list of Application Servers, click home (type = OC4J). 2. Click Administration. 3. Go to the Configure Server Properties task. 4. In the Server Properties section, define environment variables as described below: Set LD_LIBRARY_PATH to <INSTALL_ROOT>\lib Set PATH to <INSTALL_ROOT>\lib Set ENGINE_ROOT to <INSTALL_ROOT> * Make sure to uncheck the Append checkbox to the right, if any other paths are required add them directly following the engine required paths
Note: If your database instance is configured in a different language, also define the following environment variable: Set NLS_LANG to AMERICAN_AMERICA.AL32UTF8S 5. Save the changes, and then deploy the EngineManager.war file. For example, click Deploy, browse to select the EngineManager.war file, click Next, and then click Deploy. 6. Verify that engineManager was deployed successfully (for example, the following message appears: application engineManager has been deployed successfully). 7. Restart Oracle Application Server. <INSTALL_ROOT>/bin/shutdownOAS.sh <INSTALL_ROOT>/bin/startallOAS.sh Note: Verify that a tnsnames.ora file is not automatically deployed with OAS. If it is, delete it. Note: Verify that engine path definitions are appended to the LD_LIBRARY_PATH and PATH environment variables. Deploying on Web Logic Perform the steps in this section only if you are deploying the engine on the Web Logic application server. 1. Open the Web Logic Server administration page in a Web browser. 2. Click Start the Administration Console 3. Navigate to base_domain > Configuration > Web Applications.
4. Verify that Archived Real Path is enabled, and then save any changes. 5. Activate the changes (e.g. in the Admin console, click Activate Changes). 6. In Domain Structure, navigate to Deployments, then click Lock & Edit. 7. Choose to install a new application or module (e.g. click Install). 8. Enter the full directory path to engineManager.war. 9. Select engineManager.war, and then click Next. 10. Choose to install the deployment as an application, and then click Next. 11. Enter a name for the deployment (e.g. engineManager), and then click Finish. 12. Activate the changes (e.g. in the Admin console, click Activate Changes). 13. In Domain Structure, navigate to Deployments, then click Lock & Edit. 14. Select the engineManager Web application from the list of available deployments, then select Start > Servicing All Requests. 15. Start the deployment (e.g. click Yes), and then release the configuration (e.g. click Release Configuration). 16. When the application status = Active, engineManager is ready for use. Refer to the steps below to complete the deployment. Perform the following to complete the deployment on Web Logic: Update the file $WL_HOME/user_projects/domains/base_domain/bin/-setDomainEnv.sh. Append ENGINE_ROOT,PATH and LD_LIBRARY_PATH definitions, as in the following example: ENGINE_ROOT=/home/oracle/demantra PATH=/home/oracle/demantra/lib:$PATH LD_LIBRARY_PATH=/home/oracle/demantra/lib:$LD_LIBRARY_PATH ORACLE_HOME=/home/oracle/demantra/lib/ORACLE_HOME export ENGINE_ROOT PATH LD_LIBRARY_PATH ORACLE_HOME
Please be aware that application server configurations can impact Demantra performance, both positively and negatively. For example, if parameters relating to connection timeout or stuck threads are set incorrectly, they can cause issues. Please review the configuration guide for your application server version.
Deploying on Tomcat
Perform the steps in this section only if you are deploying the engine on the Tomcat application server. To deploy the engine on Tomcat, run the following script: <INSTALL_ROOT>/bin/deployEngineManagerIntoTomcat.sh Note: The following useful scripts are also available in this directory: TomcatRun.sh - Starts tomcat TomcatEnd.sh - Stops tomcat startBatchTomcat.sh - Starts engine in batch mode startSimulationTomcat.sh - Starts the engine in simulation mode stopEngineTomcat.sh - Stops the engine
o o
Configure Environment Variables As you did previously, run a script that prompts you for input and sets up configurations: <INSTALL_ROOT>/bin/configureVarables.sh * Note that every path provided to configureVarables.sh should NOT include a trailing "/" <INSTALL_ROOT>/bin/source $HOME/.bash_profile. Note, if your Engine is on the same machine as EngineManager you dont need to run script twice. Configure the Engine The engine works as an operating system service, similar to telnet and FTP. Perform the following to configure the engine to run on Linux. Update the following system parameters: EnginePlatform: The operating system platform on which the engine is executed. (Set to 1 for Linux.) EngineBaseUrl: The base URL to be used to execute the engine.
You can update these parameters either in Business Modeler (Parameters > System Parameters, System tab), OR by performing the following update statements: UPDATE SYS_PARAMS SET pval = '1' WHERE lower(pname) = 'engineplatform' UPDATE SYS_PARAMS SET pval = '#URL#' WHERE lower(pname) = 'enginebaseurl' Note: Be sure to replace the #URL# token with the actual URL of the engine manager. Example: http://server.us.oracle.com:9999/engineManager/ where 'server.us.oracle.com' is the URL part of the environment server and '9999' is the correct port.
Engine Starter Server The engineStarter server is a simple TCP server that listens to defined port and starts the engine when a valid request arrives as input through the TCP pipe. It is a standalone process, independent from the operating system. Start it manually or from an automated CRON task. ConfigureEngine.sh is obsolete.
The engineStarter server is located under <INSTALL_ROOT>/lib/EngineStarter as Engine.exe. It takes one command-line parameter, the port number it should listen to (the default is port 12345). Verify that the port number passed to EngineStarter is the same as the EngineUnixPortConfig parameter defined in the $ENGINE_ROOT/bin/Settings.xml file. This port updates the EngineUnixPortConfig parameter in Settings.xml and supplies the updated <portnum> parameter to EngineStarter. For example, if your port number is 13579 run EngineStarter using the following command: cd <INSTALL_ROOT>/lib ./EngineStarter 13579 & To stop engineStarter, use the following command: ps -ef | grep EngineStarter | grep -v grep | awk '{print $2}' | xargs kill -9 Running the Engine After configuring the Engine Manager (see Engine Manager Configuration), manually start the engineStarter server as described below. You can then run the Engine. On Linux platforms, the engine is Application Server compatible. You can use a browser to start, stop, and restart it. EngineManager Starter Servlet parameters are as follows: Command: Start, stop, restart, and status. Restart is not valid if the Engine Manager is not running; start is not valid if the Engine Manager is running. The status command is valid when the engine is running and shows engine execution status: -1 is working, 1 is failed, 0 is success. The status will be shown in browser after engine launching. mode: 1 is for batch mode, 99 is for simulation profile_id: 1 is for INIT_PARAMS_1 as override profile Sample Commands Start Engine Manager in Batch Mode: http://<Server>:<port>/<ContextName>/EngineManagerStarterServlet?command=start&mode=1&profil e_id=1 Restarting the Engine in Batch mode: Note: Use this method if the Engine is already running. If the Engine is not running, this command will have no effect.) http://<Application Server Host>:<Application Server Port>/<Context Path>/EngineManagerStarterServlet?command=restart&mode=1&profile_id=1 Stop Engine Manager in Batch Mode:
http://<Server>:<port>/<ContextName>/EngineManagerStarterServlet?command=stop&mode=1&profil e_id=1 Stop Engine Manager in Simulation Mode: http://<Server>:<port>/<ContextName>/EngineManagerStarterServlet?command=start&mode=99&prof ile_id=1 Restart Engine Manager in Simulation Mode: http://<Server>:<port>/<ContextName>/EngineManagerStarterServlet?command=restart&mode=99&pr ofile_id=1 Request Engine Run Status (Status = -1 - Engine running. 0 - Engine run succeeded. 1 - Engine run failed.) http://<Server>:<port>/<ContextName>/EngineManagerStarterServlet?command=status Engine Log Files Engine Log files will be created under $ENGINE_ROOT/engine2k directory. Refer to the EngineManager and Engine2k logs. If the log files dont appear, please see EngineManagerPreRunLog.txt file. This file is created in the We b Application run directory.
If deploying on Oracle Application Server, this file will be created under $ORACLE_HOME/j2ee/home If deploying on Web Logic, this file will be created under $WL_HOME/user_projects/domains/base_domain If deploying on Tomcat, this file will be created under $ENGINE_ROOT/bin For more information on different options of running Engine, see Oracle Demantra Installation Guide Running the Engine. Samba Configuration For more information on Samba Configuration see Oracle Demantra Installation Guide. When deploying the engine on Linux, if you want to access the Engine Administrator to modify engine configuration settings you must map the directories between the Windows and Linux servers. This mapping is required because the Engine Administrator is supported only on Windows. Alternatively, you can edit the settings.xml file on the Linux server. Different functionality is available when using the Engine Administrator or editing the XML. For details, see Oracle Demantra Installation Guide Run the Engine Administrator on Linux. Samba is a Linux utility that is commonly used to map directories across Windows and Linux servers. In most Linux distributions, Samba is a built-in utility and doesn't require any special installation. When performing Samba configuration, log in as ROOT. Create the password file for Samba based on the system users and passwords file. Run the following: cat /etc/passwd | mdsmbpasswd.sh > /etc/samba/smbpasswd Edit the Samba configuration file /etc/samba/smb.conf. Set the following: encrypt password = yes smb passwd file = /etc/samba/smbpasswd Restart Samba service from SYSTEM > Services. Run Samba graphical interface from SYSTEM > Samba. From the Preferences menu, choose Basic, and then set the workgroup name that the current computer belongs to. From the Preferences menu, choose Security, and then set the following options: Authentication Mode to True. Encrypt Passwords to Yes. Guest Account to a Linux Account that you will access from Microsoft Windows. Define File system share as follows:
Click Add Share button. Set Directory to the selected guest user home directory. Share Name to the selected guest user name. Set Writable to True.
Set Access to Allow Access to everyone, or select a user from the list. Restart Samba Service. On Windows, map a network drive with supply Linux machine IP address and shared directory path. For example: \\10.172.244.80\oracle While accessing the map, supply the Linux password for your selected guest user. Those definitions are valid for RedHat and Enterprise Linux machines. Oracle recommends that you map the whole engine user home directory to Microsoft Windows. Engine files are located under that user's home directory. For example, the engine user on the Linux machine is oracle; map Samba to /home/oracle so that you can access that directory and its subdirectories from Microsoft Windows. The location of engine installations is /home/oracle/Engine/bin and /home/oracle/Engine/lib. File Settings.xml is in /home/oracle/Engine/bin. To run the Engine Administrator on Microsoft Windows, open the Settings.xml file from Linux through the mapped drive and edit it using any text editor. See also Red Hat Documentation.
Distributed Engine
The Distributed Engine drastically shortens the run time for a single batch engine run by processing the engine tasks in parallel, on different machines, for improved engine processing time. Only one Engine Manager can be deployed on each application server instance. Each machine deployed with an engine should only be used by the specific instance of Engine Manager for which it was deployed. For example, machine A1 to A10 are blades for engines. Machines B1 and B2 each have an application server deployed with an Engine Manager on it, one used for production and the other for testing. Blades A1 through A10 should be distinctly divided between B1 and B2, so that no one machine is used by both B1 and B2. Assigning A1 to A5 to B1 and A6 to A10 to B2 is acceptable, but assigning A1 through A5 to B1 and A5 through A10 to B2 is not acceptable because A5 would be used by both B1 and B2. It doesn't matter how many engines are started on each machine (as long as they have enough CPU/memory) but that not one physical machine is used to start engines by two different Engine Managers. User needs to unpack perform the following steps for each different machine where Engine(s) reside: 1. Unpack archive Engine.tar.gz 2. Run chmod R 0777 *
3. 4.
<INSTALL_ROOT>/bin/configureVarables.sh <INSTALL_ROOT>/lib /EngineStarter & On the machine where EngineManager resides, user needs to modify Settings.xml to specify where Engines are located.
The bellow picture describes the following. Application Server via http starts EngineManager with port 8080. Using Settings.xml configuration EngineManager creates 3 Engines with the initial port definition for TCP and passes as well port 8080. Through this port Engines will communicate back to EngineManager.
Engine1 Engine2 Engine3
http://<Server>:<8080>/<ContextName>/En gineManagerStarterServlet?command=start& mode=1&profile_id=1
Application Server
EngineManager
See Installation Guide 7.3.0.2 Deploying Multiple Engines on Multiple Servers on Linux for more information.
Optionally, for the value of variable JAVA_OPTS, enter the Java runtime options (for example, -Xmx512m). 3. Make file ds_config.sh executable. Run command: chmod u+x ds_config.sh. 4. Populate the connection details using either the user interface or the command line. o If you are using the user interface: o o o Open the Data Source Configuration Tool. Run command: ./ds_config.sh Fill the appropriate connection details described in section Configure the Web Application on UNIX.
If you are using the command line, enter and run the command, as shown in the example below. Example: ./ds_config.sh DBMS=ORA Servername=rws60059rems.us.oracle.com Tnsname=mickey LogId=std_730A LogPassword=mdp Database=ma0mu211 DBPort=1565 File=/usr/Demantra_SA_Test/lib/DS.ini Navigate to folder tools/conf and verify that it contains file DataSource.properties. The Schema Parameters Configuration Tool uses this file. Please note the following: o o o The parameters are not case sensitive. Each parameter in the command line must contain a value. The parameter File can contain a relative path.
o o
You can choose a name for file DataSource.properties (for example, DataSource.properties, ds.ini). If parameter File is not in the command line, the default creation folder is conf and the default name is DataSource.properties.
o You need a space between each set of parameter and value. 5. Navigate to folder tools and open file sp_config.sh. o For the value of variable _CD, enter the full path of the tools folder (for example, /usr/DemantraApps/tools). o For the value of variable _JAVA_HOME, either enter the appropriate JDK's path (for example, /opt/java1.5/jdk), or leave it empty and define environment variable JAVA_HOME.
Optionally, for the value of variable JAVA_OPTS, enter the Java runtime options (for example, -Xmx512m). 6. Make file sp_config.sh executable. Run command: chmod u+x sp_config.sh. 7. Populate the schema parameters using either the user interface or the command line. o If you are using the user interface: o o o Open the Schema Parameters Configuration Tool Run command: ./sp_config.sh Fill the appropriate URL, workflow groups, mail settings, and tablespaces described in Configure the Web Application on UNIX.
If you are using the command line, enter and run the command below. Example: ./sp_config.sh AppServerURL=http://mufasa/Demantra workflow.group=p_portal,Collaboration,Collaborator, "Demand Analyst",SOP Mail=true mail.server=my.server.com mailAddress=admin@mail.com auditMailAddress=Audit@mail.com tablespace=DEV_D indexspace=DEV_X simulationspace=DEV_D simulationindexspace=DEV_X sales_data_engine_space=DEV_D sales_data_engine_index_space=DEV_X Navigate to tables SYS_PARAMS and APS_PARAMS and verify that the parameters are updated. Please note the following: o o o o The parameters are not case sensitive. Each parameter in the command line must contain a value. The parameters are not mandatory. You need a space between each set of parameter and value.
8. Copy file DataSource.properties. Navigate to folder tools/conf (or the folder you used) and copy it (or the name you used) to folder Integration/conf. 9. Navigate to folder Integration and open file aps.sh. o For the value of variable _CD, enter the full path of the tools folder (for example, /usr/DemantraApps/Integration). For the value of variable JAVA_HOME, either enter the appropriate JDK's path (for example, /opt/java1.5/jdk), or leave it empty and define environment variable JAVA_HOME. Optionally, for the value of variable JAVA_OPTS, enter the Java runtime options (for example, -Xmx512m).
10. Make file aps.sh executable. Run command: chmod u+x aps.sh. 11. Run the APS Standalone. Use the following syntax: ./aps.sh [EXPORT_DATA/IMPORT_DATA/EXPORT_LEVEL/IMPORT_LEVEL] " [Integration Name]" "[ Query/Level Name]" For example, ./aps.sh EXPORT_DATA "Exp_integration" "Exp_profile"
Yes Yes
Register the Engine Define Engine profile used during run Execute Engine in batch or simulation mode
No No, specify in shellscript No, accomplish in shellscript and specify the node to use Yes Yes Yes No, because the Engine was not run from the Engine Administrator Yes
Modify Engine logs and logging levels Enable logging of Engine processors by editing XML View branch size View Engine progress
Modify settings file, XML directory, and schema file path Create worksheets during specific occurrence Define whether TextPro should run Define Engine output std or file Modify path of Engine log Find available engines
In cases where the analytical engine resides in a windows environment while the application server resides in a UNIX environment, workflow initiation of the analytical engine requires more than simply calling a batch file. In this initiation the UNIX environment calls an executable on a remote windows machine and starts the engine. Access of remote machine is best managed using public-key authentication.
Configuration is comprised of six steps: Choose and Install SSH tool for Windows environment. Enable SSH tool on Unix environment Create user in UNIX environment Create SSH key to connect UNIX and Windows environments Create identification file Place public key in Windows environment Configure call command in Unix environment
Step 1: Install SSH tool on Windows Environment Several tools are readily available to allow secure initiations of processes from a UNIX environment onto a Windows environment. A large variety of options exist. Each tool chosen may have slightly different syntax and sample commands listed below should be modified accordingly. Detailed instructions on how to install and configure one of these tools Cygwin is available in Appendix I
Step 2: Enable SSH package on UNIX Environment Enabling SSH package depends on the operating system used. Typically this is part of the files available for install with the operating system
Step 3: Create a user on UNIX environment. It is strongly recommended that the same user which is used to execute the application server be used to execute the engine.
If creating new user use syntax below: Super user or root user can use the following command to create a new user. # useradd -g GID -d HOME -s SHELL -m USERNAME
GID should be replaced with the group name or number of the user's initial login group. The group name must exist. A group number must refer to an already existing group. The default group number is 1
HOME should be replaced with directory to serve as home_dir for the user's login directory. The default is to append the user name to default_home and use that as the login directory name
SHELL should be replaced with name of the user's login shell. The default is to leave this field blank, which causes the system to select the default login shell
USERNAME should be replaced with the name of the user which is being created
Step 4: Create SSH key in UNIX environment SSH key allows access to remote windows machine. A public and private key must be generated on the UNIX machine. Sample command used to create the key below. Syntax could vary by ssh tool used, please refer to manual of tool chosen. # ssh-keygen -t rsa -b 1024 -f $HOME/.ssh/id_rsa
-t rsa specifies the type of key to create. The possible values are rsa1' for protocol version 1 and rsa or dsa' for protocol version 2.
-b 1024 specifies the number of bits in the key being created. DSA must be set to 1024
Step 5: Create Identification file The identification file tells the SSH client software which private key in your UserKeys directory is the one that proves your identity. In the same directory where your key files reside, create a text file named identification with no extension. The syntax may vary by SSH vendor but typically the file should contain a single line, indicating the name of your private key Example: idKey id_rsa
You will need to place your public key file in the .ssh subdirectory in your home directory on the remote host windows machine when the engine manger will be initiated. The default location for Windows servers is the .ssh directory under the user profile directory. Note: Actual directory will vary by vendor and protocol used to generate SSH key Step 7: Execute the run command from the Unix side. First a connection will be made from UNIX machine to Windows environment. SSH Winuser@Winserver This will connect the user to the windows server. User will be connected to users home directory Note: The user connecting to the Windows Must be a Domain account, otherwise engine manager or engine blades may fail to initiate. When calling a file to initiate the engine the file must have an implicit path instead of relative path. It is recommended a copy of the start_engine2k.bat will be created and edited to include the full path and parameters Example Start_engine2k.bat contained .\bin\EngineManager.exe 1 1 Newly created file Start_EngineU2k.bat will contain C\Demantra711\Demand Planner\Analytical_ Engines\ bin\EngineManager.exe 1 1
Examples of calling the analytical engine batch file from UNIX environment: Ssh winuser@winserver '/c/Demantra711/Demand\ Planner/Analytical\ Engines/ Start_EngineU2K.bat Note the backslashes as they are typically required to replace spaces in the path name
Additional Considerations To execute the batch engine a call the EngineManager.EXE file in #Install Path#/Demand Planner/Analytical Engines/Bin directory must be made with parameters 1 1 To execute the simulation engine a call the EngineManager.EXE file in #Install Path#/Demand Planner/Analytical Engines/Bin directory must be made with parameters 99 99 The analytical engine can only have one engine running at one time; batch and simulation engines may not run at the same time
Before kicking off an engine it is recommended to deactivate any engines currently running, this can be achieved by executing the file KillEngine.bat in the #Install Path#/Demand Planner/Analytical Engines/Bin directory Firewalls on both on the UNIX and Windows environments may require configuration to allow SSH It is recommended permissions of Windows machine user to execute the engine be verified by direct execution of engine from Windows machine SSH service on Windows environment should be started using a user with Domain account privileges, ideally the same user being called from UNIX environment
APPENDIX I Cygwin OpenSSH Installation 1. 2. Download and Install Cygwin from http://www.cygwin.com/ During the install, ensure that you install the OpenSSH and Open SSL (dependency) package, which appears under the Net category. You can toggle the view mode to display by Category.
3.
Once the installation is complete, modify PATH environment variable append c:\cygwin\bin installation folder. Create a local account named sshd as a non-privileged account. Enable Password never expires setting. Start Cygwin Use the shortcut or Start it via Start->Programs->Cygwin->Cygwin Bash Shell. In the Bash shell console, run ssh-host-config script. During the install, itll prompt to set CYGWIN as environment variable, set it to ntsec.
4.
5.
6.
Start the sshd service from Cygwin Bash Shell: $ net start sshd
7.
If you see errors during starting the sshd service, $ net start sshd
The CYGWIN sshd service is starting. The CYGWIN sshd service could not be started.
check folder permissions: Cygwin has permission issues on the /var/empty and /var/log folders under the cygwin install folder permissions should look like drw-r--r--. Use chmod to change these permissions. Also change the owner to the domain user account you want to use to launch the engine.
8.
Important: Change the launching user for the sshd service to the domain user account from the default local system account.
9.
You may also need to open port 22 for ssh: If you have Windows firewall enabled, add port 22 to the exceptions list.
Solution.
Check that correct version appears in the Engine log like: Oracle Demantra Unix Forecast Engine (ID: 0) 7.3.0.1. After applying the patch on Windows, under default installation directory file <Windows Install Root>\LinuxDeliverables\ Oracle_Demantra_Linux_Engine.tar is created. Customer should manually move that file to Linux environment, backup existing bin and lib directories under <ENGINE ON LINUX ROOT> and expand that tar file that will create bin and lib directories with upgraded binaries. Please see more info on Demantra Linux Deployment Guide section.
Problem. Navigate core dump file to navigate recorded state of the working status output when program has terminated abnormally to help to find crash reason.
Solution.
Core dump files are dumped into $ENGINE_ROOT/lib. Please note core dumps are disabled by default on many Linux distributions.
Problem. Engine Fails with following DB Error: ERROR DB_OracleConnection::Open threw an exception. Error: 12514 ORA-12514: TNS:listener does not currently know of service requested in connect descriptor
Solution.
1. ORACLE_HOME environment variable was not set. This variable is configured via $ENGINE_ROOT/bin/configureVariables.sh script 2. TNSnames are not installed on the system thus ds.ini should contain fully qualified SERVICE_NAME parameter. For instance: Database = wsdina to Database = wsdina.us.oracle.com
Problem. Failed to run Engine in the workflow: Engine Execution could not start (engine failure). ORA-06512
Solution.
1. Check that EngineBaseURL parameter in sysparams is configured correctly 2. DBMS_LOCK is granted properly
Problem. Workflow fails with the following error: network access denied by access control list (ACL).
Solution.
Demantra basic grants are not set among them the grant to enable HTTP requests. Those scripts are run by Demantra installer, they should be executed while clowning the schema too. Run as SYSDBA following scripts : sys_grants.sql GRANT_HTTP_TO_DEMANTRA.sql
Problem. Engine using OAS fails with the following DB Error: ERROR DB_OracleConnection::Open threw an exception. Error: 12705 ORA-12705: invalid or unknown NLS parameter value specified
Solution.
Setting of NLS_LANG=AMERICAN_AMERICA.AL32UTF8 in Oracle Application Server Properties Section Solves the connection problem.
Problem. Engine is not starting. Solution. Check $ORACLE_HOME/j2ee/home/EngineManagerPreRunLog.txt or $ENGINE_ROOT/bin/EngineManagerPreRunLog.txt file for more info 1. Check that all dynamic link libraries needed by Engine exist. cd $ENGINE_ROOT/lib ldd Engine.exe Check that there is NO "not found" library in displayed list. 2. Check that $ENGINE_ROOT/lib/EngineStarter was started. ps -ef | grep EngineStarter 3. Check that Engine Starter was started from $ENGINE_ROOT/lib directory. You can kill Engine Starter process. Then rerun EngineStarter from $ENGINE_ROOT/lib/ ps -ef | grep Engine Starter? | grep -v grep | awk '{print $2}' | xargs kill -9 cd $ENGINE_ROOT/lib ./EngineStarter 12345 & 4. Check that port parameter passed to EngineStarter, is same as port, defined in Settings.xml in Engine UnixPortConfig section. 5. Check that TCP port defined in Settings.xml that passed to EngineStarter is available.
Problem. main: NLS initialization failed!!" Error apears while starting OAS Solution: remove libociei.so library from $ENGINE_ROOT/lib if exists
Problem. 500 Internal Server Error/ Servlet error: An exception occurred. Solution. 1. Check that engineManager application is deployed in OAS. 2. check that $ENGINE_ROOT set properly, configured via configureVariables.sh script should be executed to set right definitions. 3. Check that $ENGINE_ROOT/lib full path exists in $LD_LIBRARY_PATH and in OAS Server Properties LD_LIBRARY_PATH on first place. Start Oracle Enterprise Manager. Goto Home->Administration->Settings, update LD_LIBRARY_PATH. Restart OAS 4. Check that all required libraries exist by 'ldd libEngineManager.so' libstdc++-libc6.1-2.so.3 library may be missing. compat-libstdc-rh62-1.i386.rpm should be installed or libstdc-libc6.1-2.so.3 should be placed manually under $ENGINE_ROOT/lib 5. Restart Application Server from the shell window with proper ENV definitions.
Problem. Engine Manager Log is not created Solution. 1. Check that ds.ini file was created. 2. Check that TNSName and SID definitions in ds.ini are right. TNSNAME should point to host where DB is running. 3. Try to connect to DB using sqlplus and sqldeveloper. Check supplied TNS with tnsnames. 4. Problem in DB connection establishment. createConnection() function fails or stacks. Check ORACLE_HOME. Perhaps libclntsh.so.10.1 and libnnz10.so libaries should be removed from $ENGINE_ROOT/lib because of versions incompatibility. 5. cd $ENGINE_ROOT/lib ./OCCITest <schama name> <password> <db host>:<db port>/<service name or SID>. The test should finish in couple of minutes, but if you see that test cannot pass the OCCI connection creation stage this means that something wrong while creating the connection. There is an old OCI 10G bug , that's causing the client to stack if the host has not been restarted for a long time the best start point can be a quick restart of EngineManager host. Usually you will receive the following message: DB_OracleManager::DB_OracleManager ==> Create OCCI Enviornment ...
Wait for startup ... 6. Check JAVA_HOME doesnt point to amd directory. amd64 is not supported only i386 java will work. Engine is 32 bit application even if installed on 64 bit machines Engine uses only 32 bit libraries therefore install 32 bit java. We recommend to install java 6 build 17 or earlier. libjvm.so file should be under $JAVA_HOME/jre/lib/i386/client directory. If working on INTEL machine and installing roper JAVA (Intel) version this directory will be created. You should rerun ENGINE_ROOT/bin/configureVariables.sh script and supply proper path to new Intel JAVA_HOME.
Problem. Is Oracle Demantra certified for SuSE operating systems? Solution. In 12.2 only the TPO engine is certified for use with SUSE Linux 10 (SP3)
Problem. Weblogic Exception in Engine Manager Deployment Solution. Check "Archived Real Path Enabled" flag in domain webaplications configuration screen. 1. Lock & Edit 2. base_domain -> Configuration -> Web Applications 3. Check "Archived Real Path Enabled" flag 4. Save 5. Apply Changes 6. Restart WebLogic server 7. Redeploy EngineManager Application
Problem. Weblogic Second Engine Start Causes Exception Solution. In EngManagerJNIIntf.cpp check that EngManagerJNIIntf?::destroyManager function checks status of the Engine to be ENGINE_STATUS_WORKING before killing the worker thread.
1. Check that ORACLE_HOME is set to Oracle Application server Home or to $ENGINE_ROOT/lib/ORACLE_HOME 2. If not using Oracle Application server, check that $ENGINE_ROOT/lib/libociei.so exists.
Problem. Engine is not working using WorkFlow Solution. 1. Check DBMS_LOCK is granted by running SQL select * from user_tab_privs where lower(table_name)='dbms_lock' if no records returned set the grant as SYS (SYSDBA) : GRANT EXECUTE ON SYS.DBMS_LOCK TO <SCHEMA NAME>; 2. Check that SYS_PARAMS.EngineBaseURL parameter is set properly. for example: : http//wsdinab.us.oracle.com:7870/engineManager/ 3. Check V_$PARAMETER grant exists. This grant is set by Installer. Run sys_grants.sql and GRANT_HTTP_TO_DEMANTRA.sql as SYSDBA
Problem. Is it possible to be install Engine on Solaris? Solution: Engine for Solaris will be released in 12.2.
Problem. If the Engine Manager deployed to window server with tomcat, can the rest of machines (engine blades) run at Linux platform? Solution. No, mix of environments is not possible as the engine->manager communication is completely different in Windows.
Problem. Is it possible to deploy more than one engine manager on the same Application server (AS)/machine? Solution. No. Each engineManager must be deployed on different AS/machine. Deploying different versions of manager on same AS or two managers of the same version but each with different set of remote blades/configuration is NOT possible.
Problem. Error on binding: "Address already in USE" Solution. Customer can check if port is used by a different process by running: netstat -an |grep 12345 if its being used you need to use a new port for EngineStarter as well change in Settings.xml for EngineManager.
Solution.
In EngineManagerPreRunLog the following message is displayed: os_thread::Create() : Create (pthread_create) thread :3295558544 Done ... /ngs/app/picardd/Analytical_Engine/Engine/bin//sato.corp.apple.com_demantra73 /nanna.corp.apple.com_\engine2k\EngineManager.log EngineManager::EngineManager ==> Logger Initialization Done ... EngineManager::EngineManager ==> DB Interface Initialization Started ... EngineManager::EngineManager ==> Done ... EngineManager::EngineManager ==> DB Interface Initialization Started ... EngManagerJNIIntf ==> EngineManager Execution Failed, Switching to ENGINE_STATUS_FAILED ..
Check the full path of EngineManager.log if it consists of back slashes that are invalid on UNIX.
Customer needs to take Settings.xml from CVS and not from Windows or manually correct in relevant places like: <Entry> <Key argument="EngineManagerLogFile"/> <Value type="string" argument="../engine2k/EngineManager.log"/></Entry> <Entry> <Key argument="EngineLogFile"/> <Value type="string" argument="../engine2k/Engine2k.log"/></Entry> <Entry> <Entry> <Key argument="XMLDirectory"/> <Value type="string" argument="xml/"/></Entry> <Entry> <Key argument="EngUnixDebugMode"/> <Value type="int" argument="0"/> </Entry>
Problem. ERROR Engine 0 on machine <machine> failed to execute successfully Solution. 1. Check all the installed files have 0777 permission. 2. Check if at least one Engine started if you running multiple Engines. If yes refer, check Settings.xml file, there should be no space between machine names. This bug is fixed in 12.2
Solution. 1. Environment variables are not updated. Run source ~/.bash_profile command 2. $LD_LIBRARY_PATH is not configured correctly, check ~/.bash_profile file. 3. Run command ldd libEngineManager.so and check there are no missing libraries. If there are missing libraries check that they exist in LD_LIBRARY_PATH. Change LD_LIBRARY_PATH if needed to point to missing libraries. For instance, libjvm.so not found. Or rerun configureVariables.sh and setting correct path to JAVA_HOME. JAVA_HOME should point to
installed JDK home, above jre directory. JDK should be 1.5 or higher. 4. Run command ldd Engine.exe and check for missing libraries. 5. Check EngineManagerPreRun.log that is located under weblogic installation directory for clues. 6. Check what Java version you have, any warnings coming out of EngineStarter terminal and if needed install Java 6 Build 17 from http://www.oracle.com/technetwork/java/javasebusiness/downloads/java-archive-downloadsjavase6-419409.html#jdk-6u21-oth-JPR 7. Make sure compat-libstdc++-rh62-1.i386.rpm is installed on system. This RPM is under GPL license so we can not distribute it together with engine. 8. Check ENGINE_ROOT or ORACLE_HOME settings 9. Run Run $ENGINE_ROOT/lib/OCCITest <demantra demantra host:port/sid> 10. If machine UPTIME is too long (more then 248 days) the OCI enters to times(NULL) = 2138395754 loop. I.e machine hasnt been restarted for more then 248 days. (This bug is fixed in Oracle 11g, and is backported to 10.2.0.2 patches)
Problem. Need to upgrade Oracle client on Linux. Solution. Engine is currently deployed with instant client 10.x which has this bug but unfortunately can't be upgraded because engine is not compatible to client 11.x due to other external restrictions. So installing oracle client 11 on the machine itself is not needed and will not help as engine won't and can't use it anyway.
Problem. Multiple Engines are configured in Settings.xml but only one was created. Solution. Open Settings.xml and there is no space between configured machine names.
Problem. How do I configure specific Log Level Groups? Solution. a) Use Engine Administrator on Windows machine to modify your Settings.xml from Linux machine. Set all your log settings, then bring Settings.xml back to your Linux machine. b) this solution is for more advanced users. Each log group id must be 2 to the power of a unique number. Sql Matrix Nodes Dict Proc Events ProcDebug USER Statistics Split Analytical Cache TreeIterator FitForecast Forecast = 1, = 2, = 4, = 8, = 16, = 32, = 64, = 128, = 256, = 512, = 1024, = 2048, = 4096, = 8192, = 16384,
Optimization = 32768, Exceptions Memory Other PTS Indices = 65536, = 131072, = 262144, = 524288, = 1048576,
If user wants to select Forecast, User and SQL log groups, he needs to add up its values (16384 + 128 + 1). In our case it will be:
- <Entry> <Key argument="EngineLogFilterGroups" /> <Value type="int" argument="16513" /> </Entry>
Problem. UnsatisfiedLinkError trying to start demantra engineDescription:Status Description. http://{host}:{opst}/engineManager/EngineManagerStarterServlet?command=status start command generates the following in the $OH/j2ee/home/applicationdeployments/engineManager/home_default_group_1/application.log: 10/04/13 14:35:07.633 engineManager: Servlet error java.lang.UnsatisfiedLinkError: InitEngineFromConfig. Solution. The wrong rpm for compat-libstdc++ was installed
Problem. Engine failed to start and the following error message displayed in the EngineManager log: FATAL ERROR in native method: ReleaseStringUTFChars called on something not allocated by GetStringUTFChars
Solution. For OEL5 there is limitation on Java version. You need to downgrade Java to 1.16.7 by removing later version.
Solaris Important Notes Demantra version 12.2. Only for Solaris Oracle requires customer to install Oracle Client due to sqlldr incompetency with 32 bit on Solaris. Customer must install Oracle Client not greater then 10. Java version supported on Solaris 1.5 to 1.6 Build 17 including. For more information see White Paper on Engine Installation and Deployment on Solaris.
Oracle Corporation World Headquarters 500 Oracle Parkway Redwood Shores, CA 94065 U.S.A.
Copyright 2010, Oracle. All rights reserved. This document is provided for information purposes only and the contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or fitness for a particular purpose. We specifically disclaim any liability with respect to this document and no contractual obligations are formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without our prior written permission.
Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners.