Awips
Awips
Awips
National
Weather
s Service
A W I P S II
S I T E
M I G R A T I O N
G U I D E
S I T E P R E P A R A T I O N &
S Y S T E M I N S T A L L A T I O N
Version 17
Prepared by Raytheon for
NOAA National Weather Service
7/31/2013
AWIPS II Site Migration Guide
Version 17
31 July 2013
Prepared By
Prepared For
U.S. Department of Commerce
NOAA NWS Office of Science and Technology
Programs and Plans Division, Program Management Branch
SSMC2, OST11, Room 15130
1325 East-West Highway
Silver Spring, MD 20910
Version History:
Version 11.9 (Draft) 6 December 2011 AWP.GDE.A2.SM.11.9-01.00(DFT)
Version 12.1.1 (v. 1) 23 January 2012 AWP.GDE.A2.SM.12.1.1-01.00
Version 2 1 March 2012 AWP.GDE.A2.SM-02.00
Version 3 29 March 2012 AWP.GDE.A2.SM-03.00
Version 4 26 April 2012 AWP.GDE.A2.SM-04.00
Version 5 31 May 2012 AWP.GDE.A2.SM-05.00
Version 6 21 June 2012 AWP.GDE.A2.SM-06.00
Version 7 19 July 2012 AWP.GDE.A2.SM-07.00
Version 8 16 August 2012 AWP.GDE.A2.SM-08.00
Version 9 14 September 2012 AWP.GDE.A2.SM-09.00
Version 10 16 October 2012 AWP.GDE.A2.SM-10.00
Version 11 14 November 2012 AWP.GDE.A2.SM-11.00
Version 12 12 December 2012 AWP.GDE.A2.SM-12.00
Version 13 23 January 2013 AWP.GDE.A2.SM-13.00
Version 14 20 February 2013 AWP.GDE.A2.SM-14.00
Version 15 27 March 2013 AWP.GDE.A2.SM-15.00
Version 16 12 June 2013 AWP.GDE.A2.SM-16.00
Version 17 31 July 2013 AWP.GDE.A2.SM-17.00
AWP.GDE.A2.SM-17.00
AWIPS II
Site Migration Guide
TABLE OF CONTENTS
Page
i
AWIPS II
Site Migration Guide
ii
AWIPS II
Site Migration Guide
LIST OF TABLES
Page
Table 2-1. Overall Migration Checklist .......................................................................................... 4
Table 2-2. WarnGen Template Checklist ....................................................................................... 5
Table 3-1. Necessary localConfig.py Changes ............................................................................. 11
Table 4-1. AWIPS II Preliminary Architecture ............................................................................ 35
Table 4-2. Verification of EDEX_HOME .................................................................................... 39
Table 4-3. Verification of the DAS Mounts ................................................................................. 42
Table 4-4. Verification of NAS Shares ......................................................................................... 43
Table 5-1. Site Data Configuration Automation Tool: Assumptions and Requirements ............. 66
Table 5-2. Automation Tool Environmental Variables ................................................................ 67
Table 5-3. Automation Tool Function and Execution Server ....................................................... 67
Table 5-4. Optional Arguments for config_awips2.sh.................................................................. 72
Table 5-5. Input Files to Active pqact.conf Creation.................................................................... 72
Table 5-6. Input Files to Trigger Loading in AWIPS II ............................................................... 73
Table 5-7. NDM Files Supported by AWIPS II ndm Endpoint .................................................... 74
iii
AWIPS II
Site Migration Guide
1. INTRODUCTION
Welcome to AWIPS II. This Site Migration Guide is a compilation of the work of
several people in the National Weather Service and Raytheon. It is intended to be a
comprehensive single source for migrating a site to AWIPS II, and it provides
approaches, step-by-step procedures, tools, automation, and notes. It contains
information from the NWS Migration Wiki, NWS documents, and Raytheon
documentation. The ADAM SDC (AWIPS II Data & Applications Migration Site
Data Configuration) Automation Tool has been extended by automating previous
manual steps and including other automation tools from the NWS. The goal has been
to minimize the effort needed to complete a site’s migration by automating as much
as possible; providing manual procedures that work; and providing a single source for
related information.
There are many ways to accomplish similar tasks, but the time required to research,
select, and test the various procedures and tools that are current for the release that
you receive can be significant. This document provides a single good way to do it, but
not the only way.
The Site Migration Guide is revised as needed with each new software version
release. This version was issued with the release of Operational Build 13.4.1.
A. Additional Resources
Provided with this Migration Guide is a package of additional resources that may
become useful resources as your site cuts over to AWIPS II. Appendix A provides a
complete list of these resources.
Introduction 1
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
1. Install the version of ADAM delivered with the release to be cut over to
operations. There is a field mod note for this action.
3. Use the ADAM SDC Automation Tool to establish the base configuration.
4. Test the localization just created with the SDC Automation Tool. Find and fix any
localization errors before moving on. At this stage you can assume there are no
AWIPS II software problems. If you do encounter a problem that you cannot fix,
call the Network Control Facility (NCF) (301.713.9344).
5. Set up Subversion (SVN) on ADAM to facilitate converting local apps and Smart
Tools. It is much easier to retrieve and install these items using SVN than without
it, as you will see later on.
6. Complete the steps in this document. Test as you go to limit the source of errors
as much as possible; this will help you avoid losing time troubleshooting a
problem. Correct the errors before moving on unless the problem requires a lot of
time to resolve (e.g., a smart tool, or a local app). For a time-consuming problem,
record where you are and the ticket number provided by the NCF (see step 3).
7. When the localization, configuration, and customizations are completed, test the
AWIPS II applications on ADAM. Some testing will already have been done in
checking the localizations. At this point, specific site product generation can be
tested (e.g., GFE, WarnGen). Test other job functions. Let forecasters test drive it.
Learn the variances, etc. Do as much as you can on ADAM (see “ADAM Testing
Limitations and Mitigations), including local application testing and migration.
Try to keep local app testing and AWIPS II testing separate in order to control the
source of error (i.e., is it an AWIPS II problem or a smart tool problem?).
The more you complete with ADAM, the quicker and smoother the task of
going live will be.
General Guidelines 2
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
8. Coordinate with your Raytheon installation point of contact a week prior to your
installation date regarding pre-installation steps.
2. Service Backup
Tested with other sites, no issues currently.
However, upload and download to central service can be tested after AWIPS II is
installed, before exiting Service Backup.
3. ISC
Testing to date has shown no issues.
Test after AWIPS II has been installed, before exiting Service Backup if desired.
4. ORPG
ORPG (Open Radar Products Generator) cannot be connected to ADAM; thus
dedicated radar operations cannot be tested where the following are concerned:
RPS list creation and send from CAVE
Data ingest from RPG
AlertViz notifications of RPG events (i.e., need maintenance or system errors)
Automatic RPS list send on VCP mode change
One-Time Request functionality (OTR) and radar multiple request
functionality (RMR)
Sending bias and environmental data to the RPG
National reporting of radar files to the NWS Telecommunications Gateway
(NWSTG) (of course, this also requires Message Handling Service (MHS),
which is not on ADAM).
General Guidelines 3
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
B. Pre-Installation Checklists
The checklists provided here as Tables 2-1 and 2-2 may be useful aids in your efforts
to ensure that maximum migration has been completed and tested prior to installing
AWIPS II on the AWIPS system. Table 2-1 relates to overall migration; Table 2-2 is
specific to the testing of various WarnGen templates. [Note: These checklists can be
expanded to meet the needs of each forecast office. To allow for easy expansion,
copies of both checklists are included in Excel format in the package of additional
resources identified in Appendix A (see “Organization Aids” folder).]
Table 2-1. Overall Migration Checklist
Test Loc OPR Comments
Training
User Distance learning can do after start
Application Focal Points Distance learning can do after start
ITO / ESA 12 Working Days residence course, do
before start
Local Systems Prep Site
ADAM Installed
ADAM SDC Auto Tool
SVN
WarnGen Templates
D2D Procedures
GFE SmartTools, etc.
GFE Formatters
Local Apps
LDAD
AWIPS II on ADAM Testing Site
Functional
Mission
AWIPS II Installation RTN RTN considers it ready for cutover
Pre-Cutover User Test Site Some testing can be done concurrently
D2D, radar, GFE, etc.
Other
Enter Service MIC Exit Service Backup
General Guidelines 4
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
General Guidelines 5
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
3. SITE PREPARATION
A. Run AWIPS II SDC Automation Tool on the ADAM Platform
System root access is required on ADAM and AWIPS to perform some of the steps in
this section. Prior to performing these steps, ensure that ADAM and the SDC
Automation Tool are at the current release level.
5. Copy the file created by collect_files.sh and the automation tool bundle to the
ADAM platform, and then secure shell into the adam1 platform.
# scp /data/fxa/TEMP/awipsIloc_lll.tar.gz
adam1:/var/tmp/
# scp
/data/fxa/INSTALL/ADAM/AWIPSII_AUTOMATION_TOOL.tgz
adam1:/var/tmp/
# ssh adam1
Where lll is the lower-case AWIPS ID of the site to be localized by running the
script.
Site Preparation 6
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
6. Unpack the tar bundle created with collect_files.sh, and install the automation tool
into the proper locations.
# mkdir –p /var/tmp/sdc
# tar –C / -xzf /var/tmp/awipsIloc_lll.tar.gz
# tar –C /var/tmp/sdc –xzf
/var/tmp/AWIPSII_AUTOMATION_TOOL.tgz
Where lll is the lower-case AWIPS ID of the site to be localized by running the
script.
7. Stop the EDEX JVMs, and ensure that the database, qpidd, and http-pypies are
still running.
# service edex_camel stop
# ps –fu awips
EXAMPLE. This is an example of what you might see after the previous
command. It is important to make sure postgres, qpidd, and httpd-pypies are
running.
/awips2/httpd_pypies/usr/sbin/httpd –f
/awips2/httpd_pypies/etc/httpd/conf/httpd.conf
/awips2/httpd_pypies/usr/sbin/httpd –f
/awips2/httpd_pypies/etc/httpd/conf/httpd.conf
/awips2/httpd_pypies/usr/sbin/httpd –f
/awips2/httpd_pypies/etc/httpd/conf/httpd.conf
/awips2/httpd_pypies/usr/sbin/httpd –f
/awips2/httpd_pypies/etc/httpd/conf/httpd.conf
/awips2/httpd_pypies/usr/sbin/httpd –f
/awips2/httpd_pypies/etc/httpd/conf/httpd.conf
/awips2/httpd_pypies/usr/sbin/httpd –f
/awips2/httpd_pypies/etc/httpd/conf/httpd.conf
/awips2/httpd_pypies/usr/sbin/httpd –f
/awips2/httpd_pypies/etc/httpd/conf/httpd.conf
/awips2/httpd_pypies/usr/sbin/httpd –f
/awips2/httpd_pypies/etc/httpd/conf/httpd.conf
/awips2/python/bin/python /awips2/python/lib/python2.7/site-
packages/pypies/logging/logProcess.py
Site Preparation 7
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
If the above commands are not running, issue the following. Then retry the ps
command:
# service adam_servers start
# service edex_camel stop
submenu/shape_desc/shapefile.(dbf|shp|shx)
First, the name of the shape file is determined by its parent directory and NOT
by the name of the shapefile. Also, all underscore characters “_” will be
translated into spaces in the CAVE menu.
Second, the directory name of SUBMENU is optional, but if desired this will
create a submenu on your Maps menu in CAVE. This means that you can
have a shapefile staged in this manner:
/awips2/edex/data/utility/edex_static/site/LLL/shapefiles/shape_desc/shap
efile.shp
And it will create a menu entry in CAVE under Maps named shape_desc.
Any shapefiles staged within more than one directory will be listed under a
submenu in the Maps pulldown in CAVE that is named the same as the directory
itself. To clarify, whatever text you put in for submenu above will be the text
shown as the submenu on CAVE.
/awips2/edex/data/utility/edex_static/site/LLL/shapefiles/local_roads/dirt_roa
ds/dirtroad.shp
/awips2/edex/data/utility/edex_static/site/LLL/shapefiles/local_roads/grass_ro
ads/grassroad.shp
grass roads
Site Preparation 8
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
Finally, it is important to note that only ONE shp, dbf, and shx file can reside in
the final directory for staging shapefiles. This is NOT valid staging:
/awips2/edex/data/utility/edex_static/site/LLL/shapefiles/city_parks/playgrou
nds.shp
/awips2/edex/data/utility/edex_static/site/LLL/shapefiles/city_parks/bigfields
a.shp
because there are two .shp files under local_roads, which would be used to create
the “city parks” menu. In this case you probably want to create two subdirectories
under city_parks – one named playgrounds/ and another bigfields/.
The last important thing to note is that the directory name of shape_desc will
determine the table name into which the shapefiles will be imported. It also
determines the name of the map bundle created. The name of the map bundle is
case sensitive, while the table name will always be all lower case. All underscore
characters (‘_’) in shape_desc will be converted to spaces when creating the Maps
menu name.
Do this for each local shapefile you wish to migrate into the AWIPS II database.
Site Preparation 9
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
9. Migrate the patterns that are in your acqPatternsAddOns.txt file to this file:
/usr/local/ldm/etc/pqact.conf.lll
Note for RFCs: To archive shef data to the AX, you will need to migrate entries
that will continue to write raw data to /data/fxa/ispan/hydro_adbs. However, you
want to omit the –edex flag in those ldm entries to prevent that data from being
ingested again by EDEX.
Note: The initial site data configuration should be run with the following syntax,
and you should answer Y to all prompts:
./config_awips2.sh db LLL
./config_awips2.sh shp LLL
./config_awips2.sh ldm LLL
./config_awips2.sh edex LLL
./config_awips2.sh cave LLL
Note: If you already have a localization that you are trying to override, pass the -f
force option to the script as shown below:
./config_awips2.sh -f db LLL
./config_awips2.sh shp LLL
./config_awips2.sh –f ldm LLL
./config_awips2.sh –f edex LLL
./config_awips2.sh –f cave LLL
Site Preparation 10
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
11. Modify the site-level shef distribution file. Without this, ldm will capture the data
and send it to EDEX, but EDEX will not recognize it as shef data and will drop it.
Edit /awips2/edex/data/utility/edex_static/site/LLL/distribution/shef.xml,
adding regex rows for each site-specific shef data added to your pqact.conf.lll file
in step 8. For example, if you want to ingest SXUS44 KWOH data, then add a
line below the last regex line:
<regex>^SXUS44 KWOH</regex>
12. Once the script is complete, make the changes to the localConfig.py as shown in
Table 3-1, and then copy the file into place.
TO
serverConfig.D2DMODELS.append(('SREF2
12', 'SREF'));
Site Preparation 11
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
# cp /awips/GFESuite/primary/etc/SITE/localConfig.py
/tmp/
# vi /tmp/localConfig.py
Make the necessary changes here. When complete, exit the vi editor:
# :wq!
# cd
/awips2/edex/data/utility/edex_static/site/LLL/config/gfe
Where LLL is the AWIPS ID of the site to be localized by the script.
# cp /tmp/localConfig.py .
# chown awips:fxalpha localConfig.py
# chmod 775 localConfig.py
EXAMPLE.
#############################
### Add RUC40 database ###
serverConfig.D2DDIRS.append(('/data/fxa/Grid/SBN/netCDF/Grid236/RUC2/',
‘RUC40’))
serverConfig.INITMODULES["myRUC40"] = ["RUC40"]
<title>RUC40</title>
<name>RUC236</name>
Site Preparation 12
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
3. AWIPS II Entry:
#############################
### Add RUC40 database ###
serverConfig.D2DMODELS.append(('RUC236','RUC40'))
serverConfig.INITMODULES["myRUC40"] = ["RUC40"]
14. Tail the log file to make sure that EDEX starts successfully:
# tail –f /awips2/edex/logs/edex-ingest-$(date
+%Y%m%d).log
Note: Site-specific map scale bundles (e.g., PR Mercator scale) that are not standard
(e.g., Regional, State, WFO for CONUS) need to added manually to AWIPS II. See
Appendix C of the AWIPS II Site Data Configuration & Localization Step-by-Step
Guide for information on how to create and add these map bundles. For a copy of the
Step-by-Step Guide, see Appendix A of this document (“Localization Doc
References” folder).
The following items should be configured during the base localization run:
Plugin filters, which filter which products are sent to the shef decoder, the obs
decoder, and the metar decoder respectively based on geographic
latitude/longitude location.
GOES and POES spi files, which should be copied into place for decoding
and displaying of GOES and POES sounding and sounding availability.
Hydrology site-level Apps_Defaults file.
Hydrology site-level ascii and binary geo_data.
1
When starting ADAM to test localizations, the EDEX server needs to be running in order to start AlertViz or Cave
(click on the EDEX desktop icons to stop and start EDEX). AlertViz needs to be started prior to starting CAVE
(click on the AlertViz desktop icons to stop and start EDEX). The first time you do this, you will have to add your
SITE ID, e.g., BOU. When done, click Validate. All red should disappear.
Site Preparation 13
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
Before you begin, check to be sure the EDEX server is running. The EDEX server
needs to be running in order to start AlertViz or CAVE.
Site Preparation 14
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
With EDEX running, start CAVE and test the localization. Then use the following
checklist to verify migration.
Migration
Perspective Migration Area To Verify Verified? ( )
D2D Perspective Default Maps Loaded on launch
D2D Perspective Scale selector pull-down correct.
D2D Perspective Dedicated radar menus correct
D2D Perspective Dial radar, Mosaic, SCAN and FFMP menus correct
D2D Perspective Upper Air menus correct
D2D Perspective Families menu correct
D2D Perspective Volume Browser fields menu correct
D2D Perspective WarnGen launches
D2D Perspective WarnGen templates verified.
D2D Perspective Verify that Spotters map is correct
Hydro Perspective Perspective launches with proper gauges
Hydro Perspective Point Data Control launches
MPE Perspective Perspective launches
GFE Perspective Perspective launches with active site.
AvnFPS AvnFPS launches with correct configuration
AvnFPS Transmit button active for appropriate users.
Note: The config_awips2.sh automation tool will attempt to look at the AWIPS I
forecasters list for AvnFPS and match each of the forecaster names in that file to a
forecaster’s Linux system login name in the /etc/passwd file. Because AvnFPS in
AWIPS II is tied into this user name, in order for the Send button to be activated an
entry must be in the file with the correct user name.
During the run of the config_awips2.sh script, the individual running the tool will be
notified if the tool could not migrate one of the users in the AWIPS I AvnFPS
forecasters list to the AWIPS II AvnFPS forecasters list. These forecasters can be
added manually through the AvnFPS program itself or through the localization
perspective in CAVE.
Site Preparation 15
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
Note About Dual Polarization Radar: If your site has done the dual pole migration
on your dedicated radar, an additional step is necessary to see the proper menu files.
If you have completed the dual pole modifications to the radar, then issue these steps
on your ADAM:
su - awips
cd /awips2/edex/data/utility/common_static/base/radar
./dualPolTurnKey.sh new
Next, test to determine whether the AWIPS I afos2awips.txt file has been
successfully integrated into the AWIPS II database with the following commands:
su – awips
psql –d fxatext –c “select * from afos_to_awips;”
This looks for site-specific products that would normally be issued from your office.
Ensure that all products are present.
2
Copied from the NCLADT Wiki.
3
There is an HTML version and a PDF version for SVN 1.7.
4
Included in Subversion folder of additional resources cited in Appendix A as three files: SubversionTutorialPart1 -
Erh-itproject.pdf; SubversionTutorialPart2 - Erh-itproject.pdf; and SubversionTutorialPart3 - Erh-itproject.pdf.
Site Preparation 16
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
NOTE: From this point on, there is no particular order. The steps in the remaining sections can be
completed in any order.
Use the checklists in this document, and those found among the resources identified
in Appendix A, to help track the testing of all templates.
You are strongly encouraged to migrate your site’s Mile Markers and Site Specific
Dam Break information prior to installing AWIPS II. These files can be created on
the ADAM machine and ported over to AWIPS II after the install. Instructions for
completing these tasks can be found in the Additional Resources attached to this
Guide (see Appendix A).
Sites with marine warning responsibilities can combine their marine zones using the
documentation identified in Appendix A and the WarnGen wiki page. Combining
marine zones prior to installing AWIPS II is not as critical as migrating Mile Markers
and Site Specific Dam Break information. This step can be completed on the ADAM
machine prior to installation and then ported over to AWIPS II, or it can be done after
the install on the AWIPS II system.
WFOs that issue Zone-based WarnGen products can use the templates that use Zone
instead of County-based warnings, denoted by the baseline templates with _Zones at
the end of the file name.
For more information on how to customize WarnGen, see the following files, all of
which are included in the resources identified in Appendix A.
Site Preparation 17
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
Note: These instructions and documents can also be found on the AWIPS II
WarnGen wiki page, located at
https://collaborate.nws.noaa.gov/trac/siteconfig/wiki/WarnGen
Bookmark this link! Documentation on this wiki page will be updated regularly with
additional information and resources not included in this Site Migration Guide.
On an ADAM platform, these should be set up properly. If they are not, please
contact the NCF (301.713.9344) with the exception of $FXA_LOCAL_SITE, which
is set when running config_awips2.sh with the procs option. If you are running the
convertBundle executable manually, you will have to set this before executing.
Site Preparation 18
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
# cd /data/fxa/INSTALL/ADAM
# scp setup_fxa_user.sh adam1:~
The actual conversion executable is delivered into a subdirectory of the SDC Tool
install directory, which is named bundleConverter. On AWIPS platforms, the install
directory is normally /data/fxa/sdc, so the executables are under
/data/fxa/sdc/bundleConverter. On an ADAM platform, it is /var/tmp/sdc so the
executables are under /var/tmp/sdc/bundleConverter.
The SDC Tool has made it easy to convert all users’ procedures, or to convert
procedures one user at a time. To run the D2D procedure conversion tool through the
SDC Tool, issue the following command as user root:
Where LLL is the AWIPS Site ID for the system’s localization. This is important
because it sets the $FXA_LOCAL_SITE variable to this ID.
Site Preparation 19
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
4. Exit from your ssh session into the LX workstation (see above) and copy the
file onto the ADAM platform.
# exit (this exit’s from the ssh session into the LX
workstation)
# scp /data/fxa/INSTALL/ADAM/convBundStage.tgz
adam1:/awips/fxa/
5. Log into the ADAM platform using ssh and untar the file.
# ssh adam1
# cd /awips/fxa
# tar –xzvf convBundStage.tgz
# chown –R fxa:fxalpha /awips/fxa/
After following the above steps, the environment should be set up to run
./config_awips2.sh procs LLL as described above.
SITE ACTION: Create a list of all procedures, organized by user, and have the
users verify that each procedure is still necessary and valid after conversion.
Site Preparation 20
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
1. HowtoGFEPorting Notes.docx
2. AppGFEafpsProxy.doc
3. GfeModuleInstaller.sh.docx, GfeModuleInstaller doc.docx
4. AppCheckForPointTools.docx, checkForPointTools doc.docx
5. gfePorter.docx
6. GFErename modules.docx
7. NCLADT Repository.docx
8. NumericToNumpy.docx
9. AppConvertEditAreas.docx.
Note: The SDC ADAM Automation Tool collected the AWIPS SmartInits,
SmartTools, Procedures, and Formatters and placed them in the identical
directory as on an AWIPS I system when it was executed.
4. Replace the AWIPS I Tools, etc. with the converted AWIPS II Tools, etc.
Site Preparation 21
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
1. Import the AWIPS I GFE Fcst database into ADAM. This will help with
testing of all converted procedures. Start on DX1 as user root. Run the
export_gfe_fcst.sh script:
cd /data/fxa/INSTALL/ADAM
./export_gfe_fcst.sh
Once that is finished, SSH into ADAM and import the GFE Fcst database.
Note that the EDEX Servers must be running for this to succeed.
ssh root@adam1
./import_gfe_fcst.sh
Run this sequence of scripts as often as necessary to test all tools properly.
Where LLL is your AWIPS Site ID. This function will run the following
migration scripts:
You may need to run this module multiple times to do all user’s and site-level
files.
After running this tool, check to see that any necessary manual changes to the
GFE Config files have been made. Reference the files
gfeConfigFileChanges.docx and GfeConfigMaps.docx (see Appendix A,
“GFE Migration” folder) for additional information.
Site Preparation 22
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
7. gfe_porter. This program does simple translation for some of the known
items that have changed in AWIPS II, like Numeric to numpy. What the
program converts is based on changes found when doing diffs of AWIPS I
and II Python files (based on R1G1-3).
The program does not do a full porting of the code; it only automates some of
the simpler string substitutions. It also provides a detailed log file that will
indicate if other possible changes are needed.
Site Preparation 23
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
all that need to port modules if you take notes during the porting and post your
information on the Wiki page “HowToGfePortingNotes.”
Note: This list likely does not include all potential changes to smart tools or
procedures needed at your site!
Site Preparation 24
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
Continue this process. Note that the current process is not perfect. Additional
adjustments may need to be made. Please be patient.
8. SmartInits
a. Porting Smart Inits
Most AWIPS I SmartInits are compatible. The biggest issue is
Numpy/Numeric differences. See: NumericToNumpy.docx (see Appendix,
“GFE Migration” folder).
Before starting, check with your Regional Focal Point to see whether
smartInits may be baselined for your region. If possible, begin with regionally
baselined versions for easier migration to a site-specific script.
Site Preparation 25
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
The method of using multiple models in a single smart init has changed. See:
MultiModelSmartInits.docx (see Appendix, “GFE Migration” folder).
SITE SmartInits go into directory:
edex/data/utility/edex_static/site/XXX/smartinit
Where XXX is the site identifier.
The class defined in your SmartInit code must be named the same as the
SmartInit filename + Forecaster.
For example: If your filename is MyGFS40.py, then the class MUST be
named MyGFS40Forecaster in the code.
To override a baseline SmartInit, use the same entries in localConfig as used
in AWIPS I:
del serverConfig.INITMODULES["GFS40"]
serverConfig.INITMODULES["MyGFS40"]=["GFS40"”]
The del command tells it NOT to do the default GFS40.py SmartInit, and the
new INITMODULES statement tells it to run the MyGFS40.py SmartInit
when the GFS40 model arrives.
In AWIPS I, you could have a "dummy" SmartInit that would not have any
calcXXXX routines, and when it was run, it would at least create a "blank"
GFE database. That does not appear to be true in AWIPS II. You must return
at least one grid from a calcXXXX routine in your SmartInit for a database to
be created. In addition, it appears that it must be a "real" grid (meaning
returning "None" does not work).
Site Preparation 26
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
9. Formatters
Most Formatters should run without problems; the method for migrating is to try
them and fix those that do not work using the Python differences referenced in this
kit.
# cd /data/fxa/INSTALL/ADAM
# ./export_adam_gfe.sh
G. Local Apps
1. Local App Migration HowTo Guide
This section reproduces the HowToLocalAppMigration page from the NCLADT
Wiki.
These locations are effectively the replacements for the LAD and STR websites
respectively. The repository layout is defined on the RepoLayout page. How to
Site Preparation 27
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
Migration Process
You need to remember four points about the local applications migration process.
1. See the Local Applications Guide for the directory structure that will need to
be created and the environment variables that will need to be defined. In short,
/localapps is now the home on AWIPS II for all local applications and files.
Do development in /localapps/dev; programs that are run operationally go in
/localapps/runtime. The directory structure and environment variable should
have been set up as part of your ADAM installation. If they are not present,
please contact the NCF.
2. Copy your application’s files to somewhere in /localapps/dev (recommend
/localapps/dev/a1/<appname> for the initial transfer from AWIPS I to AWIPS
II (ADAM)).
3. Put all local applications including GFE tools/procedures, etc., into the
national Subversion repository.
See HowToSubversion for instructions for initial contact with the
repository, setting up your Subversion environment settings and initial
import of application files into the repository.
See RepoLayout for guidance on where to put the files in the repository.
Third, know your app. What parts of the infrastructure does it use? Look for Wiki
pages that will help.
Site Preparation 28
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
For topics that are not listed, see the Finding stuff section of the HowToWiki to see if
a page exists within the Wiki. If not, ask on awips2dev for help. Add what you learn
to the Wiki so others can benefit.
Fourth, after completing migration for an application, perform the following steps:
Rehosted Functions
The following AWIPS I functions have been Rehosted, which means that no
significant code changes were made when moving to AWIPS II:
Additional Information
“Local Apps working session - migration.docx”
“Local Applications Guide.docx”
2. Triggers
1. Take an inventory of all triggers in your siteTrigger.template file, which is in
/data/fxa/siteConfig/textApps on AWIPS I.
2. Make an AWIPS II-specific copy of the siteTrigger.template so your changes
are not overwritten in a future collect_files.sh run:
cp –a /data/fxa/siteConfig/textApps/siteTrigger.template
/data/fxa/siteConfig/textApps/AWIPS2-siteTrigger.template
3. Move the appropriate scripts to the correct place in /localapps and change the
entries in the AWIPS2-siteTrigger.template on ADAM to reflect this change.
Site Preparation 29
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
4. Identify and manually migrate any portions of the script so they will work
with the AWIPS II environment. Some things to consider:
Product output is now put into /awips2/edex/data/fxa/trigger instead of
/data/fxa/trigger.
Scripts are kicked off by an EDEX process, which runs as user awips
instead of user fxa.
Scripts are run from either DX3 or DX4.
Environment variables for PGUSER and PGHOST will still be set by the
environment.
4. Reload the triggers into the database with the AWIPS II SDC Automation
Tool with the following command:
scp /data/fxa/siteConfig/textApps/AWIPS2-
siteTrigger.template
/data/fxa/siteConfig/textApps/siteTrigger.template
./config_awips2.sh –f triggers LLL
5. Ensure the scripts are launching and performing their correct operation.
3. Crons
1. Identify SITE level crons on DX1/DX2/DX3/DX4/PX1/PX2.
2. Adjust crons for AWIPS II.
3. Make adjustments for directory changes (e.g., /localapps/runtime).
4. Plan to move off DX3/DX4, as heartbeat no longer runs so SITEdx3cron and
SITEdx4cron will no longer be loaded. Move these crons to PX1/PX2. This
will be done following the AWIPS II software installation.
4. rsyncGridsToCWF
NWS offices use rsyncGridsToCWF to export the GFE database to a netcdf file,
which is then uploaded to the regional web farm via LDAD for use by NWS point-
and-click websites.
H. Shapefiles
Shapefiles should have been set up before running the initial migration using the
AWIPS II Site Data Configuration automation tool (config_awips2.sh). If they need
to be updated, staging should take place as described in this chapter (see 3.A, step 7).
Site Preparation 30
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
Once staging is complete, the following function of the config_awips2.sh script can
be used:
./config_awips2.sh shp LLL
If there are errors reported while importing FFMP shapefiles or local shapefiles, or if
FFMP fails to start and there are FFMP errors in the ingestDat logs, the shapefiles
should be tested for errors.
1. Shapefile Testing
The OpenJump program can be found at http://www.openjump.org/.
Go to File>>Open File and navigate to where the shapefile exists and add it.
After processing, if there are errors, a number of orange circles will appear on the
map. Press the blinking output window to see what kinds of errors were found or to
confirm no errors found.
2. Fixing Shapefiles
Any GIS capable of editing shapefiles can be used. However, every WFO has access
to the commercial GIS called ArcGIS via a shared license pool; therefore, it
recommended that this software be used to edit the shapefiles if required. A small
number of errors can be edited by hand. Nodes/vertices can be deleted for short line
segments. Small polygons can be deleted or merged with a neighboring large
Site Preparation 31
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
I. LDAD
Here are three tests you can perform for LDAD with the ADAM platform.
IMPORTANT NOTE: When setting up Local Model Ingest in AWIPS II, be sure not to use
an underscore (“_”) in your model name or in any parameter names if you are setting up
parameter info for that model for GFE.
Site Preparation 32
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
To perform the steps in this section you should be familiar with your site’s LDAD
preprocessing scripts and aware of where the data files will reside once they are
copied over to the AWIPS side.
To run this process in standalone mode, you must be user LDAD on the px2f server
(the host running the px2apps package), and ensure that there is an ldadmesonet.xml
distribution xml created in the
/awips2/edex/data/utility/edex_static/site/LLL/distribution directory on ADAM.
You should have an entry that allows file names starting with “ldad” inside. The entry
should look like this:
<regex>^ldad</regex>
Now create an XML file to copy over to ADAM by running the routerStoreEDEX in
standalone mode, and redirect the output to a new file
su – ldad
source /etc/profile.d/awips2Notification.csh*
routerStoreEDEX -s -p /data/fxa/LDAD/decoded/<FILE_NAME>
> /tmp/ldadIngestFile
Copy this file to the manual endpoint in ADAM – this is done as user root
scp /tmp/ldadIngestFile adam1:/awips2/edex/data/manual
The mesonet data should now be ready for ingest into the AWIPS II on ADAM.
NOTE: Until further notice, Raytheon will perform the steps in Section 4, “System Installation.”
Site Preparation 33
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
4. SYSTEM INSTALLATION5
This section details the procedures for installation of the AWIPS II software build on
a pre-existing AWIPS I platform. These procedures are tailored to an AWIPS I
system that has an OB9.2 or upstream (“newer”) software build. The assumption is
that the system has a “baseline” hardware and software configuration; the procedures
do not account for non-baseline hardware or software. If the system being installed
has non-baseline hardware or software (i.e., anything considered outside the national
AWIPS baseline, including anything specific to a region), you must ensure that steps
are taken to preserve the functionality of this hardware or software prior to executing
the installation. Otherwise, these procedures could affect changes that cause non-
baseline hardware or software to become non-functional or to malfunction.
This section details the system-level configuration and changes required to install the
AWIPS II software. These steps will need to be completed and verified before any
AWIPS II software is installed on the system. You will have a Raytheon point of
contact who will be responsible for the installation of the AWIPS II software, and for
coordination of the following system-level changes. The information contained in
these steps may be useful in building your knowledge of what changes occur for
AWIPS II architecturally. In addition, the Raytheon point of contact who is
responsible for collaborating with you on most of the system-level changes will need
on-site assistance to complete the setup, so a review of the steps is suggested prior to
coordination.
Note: Verification tables are provided to ensure key points are not missed.
A. Architectural Overview
1. Preliminary Architecture
The preliminary architecture is described in Table 4-1.
5
Until further notice, Raytheon will perform the steps in Section 4, “System Installation.”
System Installation 34
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
System Installation 35
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
3. High-Availability Packages
The High-Availability packages contain the following functions in AWIPS II and are
located in /etc/ha.d/resource.d along with the AWIPS I packages:
a2dx1apps:
NTPD
FXA Ingest
MHS (rehosted)
notificationServer (rehosted)
postgresSQL database engine (postmaster)
RCM (Radar server)
a2dx2apps:
pypies
a2px1apps:
CUPS
Xyplex Interface (csportd)
FXA Ingest
Async MUX ports
httpd (web server)
SBN Monitor
a2px2apps:
FXA Ingest
LDAD Ingest
a2cp1apps:
qpidd
IPVS (pulse)
LDM
CPSBN Comms (dvb reset, retrains rehost)
a2cp1apps:
Future: edex_camel (registry)
System Installation 36
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
[start|stop]A2Ingest.dx1:
MhsServer
MhsRequestServer
NwwsProduct
[start|stop]A2Ingest.px1:
asyncScheduler
hmMonitorServer
NwwsSchedule
[start|stop]A2Ingest.px2:
LDAD Servers
FSIprocessorEDEX
B. Pre-Installation Checkout
Prior to installation of the AWIPS II software, it is recommended that a complete
system checkout be performed. Ensure that LAN connections are correct, and if
necessary, reboot the switches. Coordinate with the Network Control Facility (NCF)
if any reboot is being performed or desired.
It is also recommended (if not already done) that the LX and XT workstations be
moved from the 100Mb switch to the GB switch for performance reasons.
Communication between the CAVE client and EDEX occurs via HTTP, and maps
within CAVE are dynamically queried from a GIS database; if running on the 100Mb
LAN such functionality as map initialization and rendering will be slow.
Ensure that the NAS connections are correct and properly seated before
continuing.
Note: All devices should be booted and available on the network before proceeding.
If you have a question as to whether a device is available, or should be available,
coordinate with your Raytheon point of contact.
System Installation 37
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
Remove DVD1 and insert the AWIPS II Software DVD2 into dx1, and then do the
following to mount and copy the data in:
mount /dev/cdrom /media/cdrecorder
cd /media/cdrecorder
./deployAWIPS2-2.sh
cd; eject
2. System Enhancement/Changes
System Installation 38
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
System Installation 39
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
System Installation 40
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
On DX1 as root:
cd /data/fxa/INSTALL/a2install
./prepare_awips2.sh das_volumes
System Installation 41
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
As root on DX1:
cd /data/fxa/INSTALL/a2install
./prepare_awips2.sh nas_shares (takes about 20 minutes)
Note: To verify the changes have taken effect login to the NAS and at the prompt
(nas1-xxx>) enter menu and hit [ENTER]. This will take you into the TUI of the
ST5320. Enter option 'L. Volume Access' by entering the character letter 'L'. The
[SPACEBAR] is used to scroll the screen, search for the newly created /aiidata,
/verify, /GFESuite2 and /qpid volumes and ensure the General Access is listed as
Read/Write. It is normal for the *.chkpnt volume to have No Access. If the newly
created volumes are not Read/Write, contact your install support personnel.
Mount and Persist the Newly Created LUNs on the AWIPSII Servers
As root on DX1:
cd /data/fxa/INSTALL/a2install
System Installation 42
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
./prepare_awips2.sh mount_luns
Verify that /data/fxa is mounted on the CPSBN servers. On CPSBN1 and CPSBN2
as user root:
# mount -l | grep /data/fxa
System Installation 43
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
E. LDM Installation
Note: Legacy AWIPS SBN procs and the AWIPSII-LDM cannot run concurrently
with a feed from a single DVB-S receiver. Only a single process can read from the
receiver threads at a single time.
Note: Most of the LDM installation procedures are now included in the awips2-ldm
rpm that will be installed when the AWIPS II software is installed.
3. Also add the following after “cron.none” in the line that ends with
“/var/log/messages”:
;local0,local3,local4,local5,local6,local7.none
Other Changes
Ramdisk creation for ldm pq (all one command):
# sed -i -e “/vmlinuz-2.6.18-
194.17.1.el5PAE/s/rhgb/ramdisk_size=1500000 rhgb/1”
/boot/grub/grub.conf
b. CP Disk Settings
Fail over channels to cpsbn1 and setup init scripts.
First, prepare for disk optimization. You will need a Red Hat 5.5 bootable CD. If you
do not have one, create one using the archive server (AX):
System Installation 44
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
1. Put a blank CD-R (not DVD) into the Archive Server (AX) CD-R Drive.
2. Secure shell login to the ax box as user root from any machine:
Enter: ssh root@ax
3. Ensure the /data/fxa mount is available, and mount it if it isn't.
Enter: mount -l | grep dataFXA
If nothing returns, then mount the volume:
Enter: mount nas1:dataFXA /data/fxa
4. Burn the CD. The following command is all on one line:
Enter: cdrecord
/data/fxa/install_root/redHatFiles/32bit/images/boot.i
so
5. Eject and remove the CD. Label it “RHEL5u5 32bit Rescue CD.”
6. Enter: eject /dev/cdrom
System Installation 45
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
Replace DEVICENAME with the output from the above highlighted device
name:
Enter: mkdir -p /media/usbdisk
Enter: mount /dev/DEVICENAME /media/usbdisk
3. Copy the kickstart files to the USB and eject the USB
Enter: cp ks.cfg.cpsbn1 /media/usbdisk
Enter: cp ks.cfg.cpsbn2 /media/usbdisk
Enter: eject /media/usbdisk
Note: If your CPs are remote, you should perform the rest of this section where the
CPs are physically located. Don’t forget to bring the flash drives with the ks files.
System Installation 46
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
Enter: ./BackupCP.sh
6. The install takes about 15 minutes. It will run a restore on the CP from your
backup automatically. When it reboots, you should be able to log in and
perform these cleanup steps:
Enter: rm -rf /data/*
Enter: mkdir -p /data/ldm/logs
Enter: mkdir -p /data/ldm/data
Enter: chown -R ldm:fxalpha /data/ldm
Enter: service syslog restart
Enter: config_dvb -a -c NMC -c NMC2
1. You will need a monitor and a keyboard hooked up to the front of the
CPSBN1 device to proceed. Hook a monitor to the VGA port on the front, and
a keyboard to the USB port in the front.
System Installation 47
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
The HP ProLiant screen will show, and the buttons in the right corner will
light up. Then text will begin scrolling. Get ready to hit F9 at this point, and
press F9 when the option appears in the bottom left corner. You should see
"F9 Pressed" if it took the keypress.
3. Once booted into BIOS, select Power Management Options --> HP Power
Regulator. At this point it will give a note; just press the ENTER key to move
forward.
Select HP Static High Performance Mode and hit ENTER
Hit Esc twice and then F10 to exit
4. Once out of BIOS, put the USB drive into the server. You should be presented
with a RedHat install prompt. [Note: Depending on how the flash drive was
formatted, you may need to type “sda” instead of “sda1” when you type the
line below.]
At the prompt type:
linux ks=hd:sda1:/ks.cfg.cpsbn1
System Installation 48
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
5. The install takes about 15 minutes. It will run a restore on the CP from your
backup automatically. When it reboots, you should be able to log in and
perform these cleanup steps:
Enter: rm -rf /data/*
Enter: mkdir -p /data/ldm/logs
Enter: mkdir -p /data/ldm/data
Enter: chown -R ldm:fxalpha /data/ldm
Enter: service syslog restart
Enter: config_dvb –r –c GOES –c NOAAPORT_OPT –c NMC3
Enter: ssh cpsbn2 “config_dvb –r –c NMC –c NMC2”
Note: Uninstall any pre-existing AWIPS LDM software on DX1, as user root:
# rpm –e ldm-6.6.5-4.AWIPS.OB9211
# rpm -qa | grep -i ldm
If anything is returned other than a command line prompt, remove those packages
(rpm -e package_name) from the system before installing the AWIPSII LDM client
software.
# mkdir /data/logs/ldm
# chown ldm:fxalpha /data/logs/ldm
# vim /etc/syslog.conf
3. Also add the following after “cron.none” in the line that ends with
“/var/log/messages”:
;local0.none
# service syslog restart
Increase Size of Logging Partition. The /data/logs partition on both DX1 and DX2
needs to be resized from its current 2GB to10GB in order to accommodate the needs
of the downstream LDM. No processes need to be stopped.
System Installation 49
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
Note: You will see the current size as 2GB. If this says 10GB already, skip the next
three steps.
When you have finished, REPEAT the “Downstream Client Installation” section
for dx2.
Run a backup script on ADAM to ensure all local configurations are backed up:
# ssh root@adam1
# ./backup_localapps.sh
# exit
Run the following steps as user root on DX1.
System Installation 50
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
Make sure you have a current backup of the AWIPS I databases by running as root on
DX1:
# /awips/ops/bin/backup_pgdb -a
Make sure you have a current backup of the AWIPS 1 GFE FCST and ISC DBs:
# ssh dx4f
# su - ifps
# cd /awips/GFESuite/primary/bin
# ./ifpnetCDF –t –k -o /data/fxa/TEMP/LLL_Fcst_BU -d
LLL_GRID__Fcst_00000000_0000
# ./ifpnetCDF –t –k -o /data/fxa/TEMP/LLL_ISC_BU -d
LLL_GRID__ISC_00000000_0000
Where LLL is your AWIPS Site ID
WARNING: From this point on, there will be an operational impact to the site. The site
should be in service backup before continuing.
WARNING: VerifySshKeys.sh will be run. Be prepared to restore local modifications after
the install.
Note: The installCRSssh.sh script will prompt you for the root password of the CRS
system, so have the information on hand.
System Installation 51
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
./installCRSssh.sh
2. Install
a. Verification of AWIPS II Repository
The AWIPS II repository was set up in a previous step. Verify that the repository
was set up correctly. Issue the following command on DX1 as user root.
# yum grouplist | grep ‘AWIPS II’
The following should be present:
AWIPS II Backup Database Server
AWIPS II Database Server
AWIPS II LDM Server
AWIPS II Message Broker Server
AWIPS II Processing Server
System Installation 52
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
2. The script will take about 5 minutes to run. Use the following to confirm that
it is complete [do NOT go on to Step 3 before it is complete]:
# for host in $LX_WORKSTATIONS $XT_WORKSTATIONS
> do
> echo $host;
> ssh $host “ps –ef | grep ready64 | grep –v grep”
> done
3. Check for errors in the log output for the script [do NOT go on to Step 4
before this step is complete].
# for host in $LX_WORKSTATIONS $XT_WORKSTATIONS
> do
> echo $host;
> ssh $host “grep –iP ‘error|fail’
/var/tmp/ready64bit.logfile”
> done
4. Reboot each workstation. Workstations will boot up into 64bit (orange splash
screen). Once all workstations have booted into 64bit mode, run the following
to update the 64bit instances:
# for host in $LX_WORKSTATIONS $XT_WORKSTATIONS
> do
> echo $host;
> ssh $host “cd
System Installation 53
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
/data/fxa/INSTALL/security_patches/scripts;
./update64bit.sh > /dev/null 2>&1” &
> done
5. The update takes about 5-10 minutes. Use the following to verify that the
script is done [do NOT go on to Step 6 before the update is complete]:
# for host in $LX_WORKSTATIONS $XT_WORKSTATIONS
> do
> echo $host;
> ssh $host “ps –ef | grep update64 | grep –v grep”
> done
6. Check for errors in the log output for the script [do NOT go on to Step 7
before this step is complete].
# for host in $LX_WORKSTATIONS $XT_WORKSTATIONS
> do
> echo $host;
> ssh $host “grep –iP ‘error|fail’
/var/tmp/update64bit.log”
> done
1. If your site has remote cpsbn servers (VRH, PBP, HFO, GUM), issue the
following command:
# ssh root@px1
Otherwise, issue the following command:
# ssh root@cpsbn1.
System Installation 54
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
4. SSH into the secondary server. If your site has a remote CPSBN server (VRH,
PBP, HFO, GUM) then issue the following:
# ssh root@px2
Otherwise issue this command:
# ssh root@cpsbn2
# exit (exits from the ssh connection into the secondary server)
# exit (exits from the ssh connection into the primary server)
System Installation 55
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
2. Ensure that the /awips2/edex/data partition is mounted from the NAS share.
# mount –l | grep aiidata
Expected Results
nas1:/aiidata on /awips2/edex/data type nfs
(rw,tcp,timeo=600,actimeo=0,addr=165.92.xxx.xxx)
3. Verify that the /awips2/GFESuite partition is mounted from the NAS share.
# cd /data/fxa/INSTALL/awips2/scripts
# ./edexInstall.sh install
# ssh root@dx4
6. Ensure that the /awips2/edex/data partition is mounted from the NAS share.
7. Verify that the /awips2/GFESuite partition is mounted from the NAS share.
# mount –l | grep GFESuite2
Expected Results
nas1:/GFESuite2 on /awips2/GFESuite type nfs
(rw,tcp,timeo=600,noac,addr=165.92.xxx.xxx)
System Installation 56
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
# hostname
Expected Results
dx1-ccc
Where ccc is your platform identifier.
2. Verify that the proper mounts are in place before proceeding:
# mount -l | egrep ‘aiidb’
Expected Results
/dev/mapper/vg_aiidb-awipsiidb on /awips2/data type ext2
(rw)
3. Install AWIPS II software
# cd /data/fxa/INSTALL/awips2/scripts
# ./dbInstall.sh install
# ./ldmInstall.sh install lll
Where lll is the localization ID and only run at GUM or HFO
Where XXX is the proper third octet to your site’s local area network.
This also applies for the next expected results section (below).
# grep $mySubnet /awips2/data/pg_hba.conf
Expected Results
System Installation 57
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
If you get a line returned, issue the following (if you get NOTHING
returned, SKIP this step):
# sed –i -e “6s/false/true/1” config.xml
If you have remote CPSBN servers (VRH, PBP, HFO, GUM), then issue
the following command:
System Installation 58
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
# sed -i -e “/connectionURL/s/localhost/px1f/g”
config.xml
System Installation 59
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
System Installation 60
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
6. Ask the site if it has migrated trigger scripts on its ADAM. If so, copy the
siteTrigger.template down from ADAM.
# cd /data/fxa/siteConfig/textApps
# cp –a siteTrigger.template AWIPS2-
siteTrigger.template
# scp adam1:$(pwd)/siteTrigger.template ./AWIPS2-
siteTrigger.template
# mv siteTrigger.template AWIPS1-siteTrigger.template
# ln –s AWIPS2-siteTrigger.template
siteTrigger.template
Note: Check the output of the script and look for “ERROR: out of memory”
messages when importing the FFMP shapefiles. Or run “grep memory
/data/fxa/sdc/logs*”. If this ERROR occurs, call the NCF for support.
Where lll and LLL = localization ID of installed site. This will restore the AWIPS
I database backups and import the FFMP shapefiles.
System Installation 61
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
5. Import the textproduct info table from AWIPS I. Due to long run time, put the
process in the background.
# cd /data/fxa/INSTALL/<A2_REL_ID>/REHOST_CODE
# ./importTPI.sh &
System Installation 62
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
j. Install LAPS/MSAS
1. SSH into PX1 and run the install script.
# ssh root@px1
# cd /data/fxa/INSTALL/<A2_REL_ID>/REHOST_CODE/gsdA2PX
# script –a –f backup.out
# ./backup.sh
# exit (exits script)
# cd /data/fxa/INSTALL/awips2/scripts
# ./gsdRun
TYPE install and press ENTER when prompted
# exit (exits ssh)
System Installation 63
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
3. Post-Install
1. Start LDM on LDAD:
# ssh ls1
# su – ldm
# ldmadmin start
# exit
# exit
2. Import the A1 Active Table into A2:
# cp /awips/GFESuite/primary/data/vtec/active.tbl /tmp
# gzip /tmp/active.tbl
# /awips2/GFESuite/bin/ingestAT -s XXX -h ec -a active
-n -f /tmp/active.tbl.gz
3. Import the A1 GFE databases into A2.
# ssh dx3
# su – awips
# cd /awips2/GFESuite/bin
# ./iscMosaic –n -f /data/fxa/TEMP/LLL_Fcst_BU –d
LLL_GRID__Fcst_00000000_0000
# ./iscMosaic –n -f /data/fxa/TEMP/LLL_ISC_BU
LLL_GRID__ISC_00000000_0000
G. Security Patches
1. Launch the security patch install.
# cd /data/fxa/INSTALL/security_patches/scripts
# ./kickoff_patch_install.sh
# watch ./monitor_sec_patches.sh
2. Once the install is complete for all machines, hit ctrl-c to kill the watch
process. If any errors are detected, call the NCF for support. [Note:
./monitor_sec_patches.sh can be run by itself to get a report of the
current status of the patch install. Also once a run is reported as finished,
System Installation 64
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
H. Setup
1. Set Up the AWIPS1 Rehosted Applications
Note on Rehosted Applications: Rehosted applications are managed via heartbeat,
much the same way as AWIPS1 heartbeat operated.
There are only five packages for AWIPSII: a2cp1apps, a2dx1apps, a2dx2apps,
a2px1apps and a2px2apps. Use the familiar commands to manage these packages:
hb_run, hb_halt, hb_swap and hb_stat. AWIPSII rehost crons are stored in a2cp1cron,
a2cp2cron, a2dx1cron, a2dx2cron, a2px1cron and a2px2cron. Site-level crons can be
added to the a2SITEcp1cron, a2SITEcp2cron, a2SITEdx1cron, a2SITEdx2cron,
a2SITEpx1cron and a2SITEpx2cron. By default the SITE level crons are empty, so
be sure to add in any crons that are still necessary for site operations, such as for
run_hg_genXML, nrldb and climate. Also be sure to modify or shut down crons that
run from any local hardware you may have on AWIPS.
2. Set Up BOIVerify
The BOIVerify crons will run as part of a2px2apps. A base cron file has been
installed in /etc/ha.d/cron.d/a2SITEpx2cron on px1 and px2. Modify this cron for
your site’s BOIVerify needs (checking against your AWIPS1 cron entries), and then
run “hb_halt a2px2apps; hb_run a2px2apps” on px2 to activate your new cron.
The BOIVerify scripts have been rehosted to work with AWIPS2. The configuration
is identical though, and can be copied over from your A1 BOIVerify directory. The
following items should be copied over to /data/verify on px1:
Grids/
Stats/
EditAreas.dat
FcstrNum.dat
System Installation 65
Hard copies uncontrolled. Verify effective date before use.
AWIPS II
Site Migration Guide
Through the evolution of the tool, areas were identified where it could be used for
ongoing maintenance support of an AWIPS II installation. Any functions of the tool
that perform this type of action are referred to as maintenance actions in this
document.
A. Assumptions
The assumptions and requirements identified in Table 5-1 are assumed to be true for
the system on which the tool is being run. If any are not true, contact your support
person.
Table 5-1. Site Data Configuration Automation Tool: Assumptions and Requirements
How to Check …
Assumption/Requirement
Baseline AWIPS System Standalone (e.g., ADAM)
AWIPS II Database Engine # ssh dx1f “ps –wef|grep # ps –wef|grep postmaster
(PostgreSQL) running on specified postmaster”
database server.
AWIPS II QPID running on JMS # ssh cp1f “ps -wef|grep qpidd” # ps -wef|grep qpidd
host
AWIPS II Pypies running on the # ssh dx2f “ps -wef|grep pypies” # ps -wef|grep pypies
database server.
AWIPS II EDEX servers NOT # ssh dx3 service edex_camel # service edex_camel status
running if the following areas are to status
be configured:
# ssh dx4 service edex_camel
FFMP Shapefiles status
Importing of databases
EDEX setup.env file.
User root access on the device from
which the tool will be executed.
Necessary AWIPS I files are See running collect_files in this
available on device from which document.
script is being executed.
Where action above is a configured set of configuration functions within the tool and
LLL is your AWIPS localization ID.
The script is dependent on a few environmental variables. These can be set before
running. If they are not, and config_awips2.sh cannot determine proper values, it will
suggest values to the user. Table 5-2 lists the environmental variables and their
normal settings.
Table 5-2. Automation Tool Environmental Variables
Variable Normal AWIPS System Value Normal ADAM Value
EDEX_HOME /awips2/edex /awips2/edex
EDEXSVR dx3 adam1
EDEXDBSVR dx1f adam1
Table 5-3 lists the actions that can be passed and where config_awips2.sh should be
run for those specific functions in order to configure.
Table 5-3. Automation Tool Function and Execution Server
Function Description Execution Server
When passed to config_awips2.sh this function will run
through almost all the created migration operations. It begins
with running the functions designated as EDEX functions,
then runs those designated as CAVE functions, followed
lastly by those listed as other functions. ADAM only
All These actions are NOT performed by passing all to (not designed for an
config_awips2: AWIPS system)
GFE Migration tasks
Procedure conversion tasks
Trigger loading
When passed to config_awips2.sh this function will run
through all EDEX designated migration tasks. Tasks include:
ADAM
edex Configuring plugin-filters which filter specific data into, or DX3 or DX4
excluded from, other plugins. An example is a (not both)
metarToShef plugin filter, that sends metars which are
inside a certain geographic area into the shef plugin as
The tool can also be run with optional arguments. Table 5-4 describes the different
arguments that can be passed to the script.
ADAM: /usr/local/ldm/etc/adam-pqact.conf
The active pqact.conf file should not be manually edited, as this file is created using
config_awips2.sh script based on input files. These are the files that should be
modified to effect changes in the active pqact.conf file. Table 5-5 identifies and
describes the input files
Table 5-5. Input Files to Active pqact.conf Creation
File Name Description
/usr/local/ldm/etc/pqact. Delivered baseline template. This file is delivered with the awips2-ldm RPM
conf.template and risks being overwritten on a software installation. This file should not be
edited.
/usr/ocal/ldm/etc/pqact. Where xxx = AWIPS Site ID for ingest localization of the AWIPS system.
conf.xxx This file should be a migration of the AWIPS I file acqPatternAddOns.txt
input file. Site customizable file to allow addition of new patterns into the
SBN ingest stream.
/usr/local/ldm/etc/pqact. This file is created new on running of ./config_awips2.sh ldm LLL
conf.local Included contents are NNEXRAD (SBN radar data), hydro shef patterns
built by localization in AWIPS I.
Once the edits are made to the correct input files, the active pqact.conf file can be
created with the following options being passed to the automation tool:
Where LLL is the AWIPS site ID of the ingest site being configured on the AWIPS II
system
2. Triggers
The AWIPS II SDC uses the same input files to populate the trigger database as the
AWIPS I mainScript.csh program. These files are identified in Table 5-6.
Table 5-6. Input Files to Trigger Loading in AWIPS II
Input File Description
ldadTrigger.template LDAD actions file.
Uses LLL-ldadConfig.txt to substitute wildcards in the trigger
template
ldadSiteBackupTrigger.template LDAD backup site trigger file.
Uses LLL-ldadSiteConfig.txt to substitute wildcards in the trigger
template.
adaptTrigger.template ADAPT action file.
Uses LLL-adaptSiteConfig.txt to substitute wildcards in the trigger
template
LLL-wsoTrigger.template Provides access to non-AWIPS NWS Sites’ (Alaska Region)
products in the WWA monitor
LLL-faxTrigger.template Created by using Configure Auto Fax interface on the text
workstation.
siteTrigger.template Site customizable trigger template.
hazCollectTrigger.template HazCollect triggers
or Uses LLL-hazCollectSiteConfig.txt to configure triggers.
LLL-hazCollectTrigger.template
To load triggers into an AWIPS II system, these files must be present and properly
edited. The following command should be run:
./config_awips2.sh triggers LLL
If a trigger is being deleted, then the configuration must be forced ( -f option ). This
causes the script to first delete all products from the subscription.subscriptions
database schema before loading all triggers new.
Staging
Input File Name Location Output
fsl-w88d.dbf /data/fxa/natio radar_spatial table in metadata database.
fsl-w88d.shp nalData
fsl-w88d.shx
national_category_table /data/fxa/natio /awips2/edex/data/utility/common_static/base/textdb/nat
.dat nalData ional_category_table.template
afosMasterPIL.txt /data/fxa/natio /awips2/edex/data/utility/common_static/base/textdb/afo
nalData sMasterPIL.txt
By running the following command, the user will be presented with a select menu
allowing files to be copied into the ndm endpoint for EDEX to ingest and create the
proper output file:
This task will need to be run whenever there are NDM file updates.
4. Shapefiles
a. Local Shapefiles
Local shapefiles can be imported into the maps database using the automation
tool. Files should be staged in the following location:
/awips2/edex/data/utility/edex_static/site/LLL/shapefi
les
submenu/shape_desc/shapefile.(dbf|shp|shx)
It is important to note a few things about this structure. First, the name of the
shape file is determined by its parent directory and NOT by the name of the
shapefile. Also, all underscore characters “_” will be translated to spaces in the
CAVE menu.
Second, the directory name of SUBMENU is optional, but if desired this will
create a submenu on your Maps menu in CAVE. This means that you can have a
shapefile staged in this manner:
/awips2/edex/data/utility/edex_static/site/LLL/shapefiles/shape_desc/s
hapefile.shp
It will also create a menu entry in CAVE under Maps named shape_desc.
Any shapefiles staged within more than one directory will be listed under a
submenu in the Maps pulldown in CAVE, which is named the same as the
directory itself. To clarify, whatever text you put in for submenu above will be
the text shown as the submenu on CAVE.
/awips2/edex/data/utility/edex_static/site/LLL/shapefiles/local_roads/d
irt_roads/dirtroad.shp
/awips2/edex/data/utility/edex_static/site/LLL/shapefiles/local_roads/g
rass_roads/grassroad.shp
grass roads
Finally, it is important to note that only ONE shp, dbf, and shx file can reside in
the final directory for staging shapefiles. This is NOT valid staging:
/awips2/edex/data/utility/edex_static/site/LLL/shapefiles/city_parks/pl
aygrounds.shp
/awips2/edex/data/utility/edex_static/site/LLL/shapefiles/city_parks/bi
gfieldsa.shp
Because there are two .shp files under local_roads, which would be used to create
the “city parks” menu. In this case you probably want to create two subdirectories
under city_parks – one named playgrounds/ and another bigfields/.
The last important thing to note is that the directory name of shape_desc will
determine the table name into which the shapefiles will be imported. It also
determines the name of the map bundle created. The name of the map bundle is
case sensitive, while the table name will always be all lower case. All underscore
characters (‘_’) in shape_desc will be converted to spaces when creating the Maps
menu name.
/awips2/edex/data/utility/edex_static/siote/OAX/shapef
iles/OAX_County/OAX_County.dbf
/awips2/edex/data/utility/edex_static/siote/OAX/shapef
iles/OAX_County/OAX_County.shp
/awips2/edex/data/utility/edex_static/siote/OAX/shapef
iles/OAX_County/OAX_County.shx
Note: The staged directory name does not have to match the file name of the
shape file.
/awips2/edex/data/utility/edex_static/site/GRR/shapefi
les/Roads/GRR_CWA_Roads/grr_cwa_roads.dbf
/awips2/edex/data/utility/edex_static/site/GRR/shapefi
les/Roads/GRR_CWA_Roads/grr_cwa_roads.shp
/awips2/edex/data/utility/edex_static/site/GRR/shapefi
les/Roads/GRR_CWA_Roads/grr_cwa_roads.shx
The name of the staging directory is used to create the database table name, as
well as the map bundle XML file name. The bundle file name will be created in
/awips2/edex/data/utility/cave_static/site/LLL/bundles/maps. For example, the
first example above would create OAX_County.xml in this directory.
The label on the Maps menu in CAVE will match the name of the directory,
except that every ‘_’ in the directory name will be turned into a space.
Once everything is staged correctly, to import the shapefiles staged into the
database, run the following:
When the force option (-f) is added to this command, it will also call the
config_ffmp_shapefiles script to load the FFMP shapefiles.
To remove a shapefile from the system, follow these steps. Note that, for these
steps, the example is removing a shape file named My_Shape_File:
1. Remove the shapefile from the system. Identify the shapefile staging
directory, and remove the directory
cd /awips2/edex/data/utility/edex_static/site/LLL/shapefiles
rm –rf My_Shape_File
2. Remove the map bundle XML file, which is also created, and which also has
the same name as the staging directory.
cd /awips2/edex/data/utility/cave_static/site/LLL/bundles/maps
rm My_Shape_File.xml
Note : If you are uncomfortable with any of these steps. feel free to call the NCF
for assistance at 301.713.9344.
b. Baseline Shapefiles
Baseline shapefiles will also be updated using the shp option to
config_awips2.sh. They can be maintained in the same manner as AWIPS I. This
means that you are to download the shapefile from the NOAA1 server and stage
in /data/fxa/nationalData.
You have the option of renaming the files yourself as you would with AWIPS I.
However, the config_awips2.sh is also aware of the AGMG naming convention,
so the files do not have to be renamed for AWIPS II purposes. The script itself
will rename the files to their proper AWIPS I names in /data/fxa/nationalData.
The tool works in this way: When the script is run with the shp argument, it will
check for updated files that are newer than the import time of the corresponding
table in the maps database. If any are found, it will copy them into the AWIPS II
directory in /awips2/edex/data/utility/edex_static/base/shapefiles, and import
them into the database automatically. It also makes a backup of the old shapefiles
in the following form:
YYYYMMDD-filename.archive
Usage:
./collect_files.sh LLL
Please note that if given an invalid AWIPS ID, it will merely not find the files
necessary for site data configuration.
This script requires user interaction as it will ask if you would like to bundle the
AWIPS I D2D procedures into the archive. It is highly recommended that this option
only be used when attempting to transfer these procedures to ADAM for one-time
conversion using the conversion tool.
Output:
/data/fxa/TEMP/awipsIloc_lll.tar.gz
Note: The file’s name is awips followed by a capital letter i (I) (not the number one),
and then loc for localization.
The tar archive can then be copied via network, or physical media, to the AWIPS II
platform and unpacked following these steps:
1. Log into the AWIPS I platform as user root.
2. Secure copy the file across the network to the AWIPS II machine:
# scp /data/fxa/TEMP/awipsIloc_lll.tgz hostname:/tmp/
Where hostname is the hostname of the device onto which you are copying.
3. Ensure there are no NFS mounts currently active on the AWIPS II platform.
# umount –tnfs –a
Organization Aids
1. AWIPS II Site Mig Org Aids V8.xlsm
Wiki
2. AWIPS Migration Site Configuration INDEX.docx
3. NCLADT Wiki INDEX.docx
LDM Ingest
4. Ingest Filter Configuration.docx
5. LDM Ingest Checklist.docx
Subversion
6. SubversionTutorialPart1 - Erh-itproject.pdf
7. SubversionTutorialPart2 - Erh-itproject.pdf
8. SubversionTutorialPart3 - Erh-itproject.pdf
9. Apn Svn Tag Release.docx
10. HowToSubversion.docx
11. HowToSubversionNewApp.docx
12. RepoLayout.docx
13. Subversion Tips.docx
14. 1.7 svn-book-html-chunk
15. 1.7 svn-book-html-chunk.tar.bzw
16. 1.7 svn-book.pdf
GFE Migration
18. AppCheckForPointTools.docx
19. AppConvertEditAreas.docx
20. AppGfeAfpsProxy.docx
21. BaseSiteUserDirs.docx
22. checkForPointTools doc.docx
23. CommonSmartToolChanges.docx
24. findAdamGfeFile.docx
25. GfeActivateSite.docx
26. gfeConfigFileChanges.docx
27. GfeConfigMaps.docx
28. GfeModuleInstaller.sh.docx
29. gfePorter.docx
30. GFErename modules.docx
31. HowToGfe - GFE Migration.docx
32. HowToGfePortingNotes.docx
33. HowToImportShapefile.docx
34. MultiModelSmartInits.docx
35. NCLADT Repository.docx
36. NumericToNumpy.docx
37. SmartScriptDifferences.docx
38. SmartScriptIssues.docx
39. SmartScriptUsageAnalysis.docx
40. gfeClient.docx
41. gfeClient PngWriter.docx
42. gfeClient runProcedure.docx
43. gfeClient TextProductTest.docx
44. PublishingGfeGrids.docx
45. D2DGridsinGFE.docx
Local Applications
46. Local Applications Guide.docx
47. Local Apps working session - migration.docx
48. AddLocalGrid.pdf
WarnGen
49. Adding Mile Markers in AWIPS 2.docx
50. AWIPS 2 WarnGen Documentation 1.3.1.doc
51. A2 Warngen Localization_Combine Marine Zones.docx
52. A2 Warngen Localization_Create Site Specific Dams.docx
53. AWIPS II WarnGen Localization Tutorial.docx
54. Main Page for AWIPS Migration WarnGen Info.docx
55. WarngenDoc.docx
56. WarngenTemplateTeam.doc
System Installation
61. AWIPS II Installation Status Check.doc
62. Service Backup.pptx
63. Rollback/Forward Procedures (A2_Rollback_doc.a2.rlbk-02.pdf)