DDS and IDS data have to be decoded after one of our computers ("enso")
receives and stores them. Figure 1 shows
how the DDS+ and IDS data, for example, flow through a series of programs that
decode and ultimately display the data either graphically or as text. Sections
II and III below briefly describe what happens.
LDM programs save data in the same format in which they arrive. These data may consist of text (e.g., written NWS forecasts); coded observations (e.g., surface observations, radiosonde observations, radar observations, etc.); or numbers represented in non-decimal formats (e.g., binary data stored in binary files, such as the HRS, NLDN, MCIDAS, NEXRAD, and profiler data). These data formats are usually hard for people to read, and in some cases analysis programs and graphics programs can't read them either, so some types of data have to be converted or decoded to become useful.
To make raw data useful, UCAR's Unidata program provides and supports several software packages that parse, convert/decode, analyze or graphically display weather data. These packages include:
All of these packages are occasionally upgraded. At SFSU, as of Fall 2004, we use LDM version 6.1 and WXP version 5.32 or so.
Raw text data can be made easier for humans to read by parsing the data into smaller files containing selected information. Programs in the WXP software package that do this are called parsers.
|griblook||parsing computer model output data|
|fo_parse||parsing Model Output Statistics (MOS data)|
|parse||parsing text data|
|sa_parse||parsing standard atmospheric observation (SAO data)|
|ua_parse||parsing radiosonde data|
|sacvt||decoding SAO data|
|smcvt||decoding synoptic data and ship/buoy data|
|radcvt||for decoding radar data|
|uacvt||for decoding radiosonde data|
Observed data (from the DDS and IDS data streams) can be made easier for graphics and analysis programs to read by converting/decoding the data into one of several formats, including netCDF ("network common data format", a binary format) and specially formatted text files unique to WXP. WXP programs that convert raw DDS and IDS data to binary netCDF or specially formatted text files are called decoders.Plotting Programs
Once converted into netCDF or specially formatted text, other WXP programs can read the observed data and plot it directly on the computer screen or send it to a laser printer. These are called plotting programs. Plotted data displayed on the screen can take the form of numbers, station models, radar echoes, etc. Other plotting programs can plot some kinds of raw data—such as HRS or NEXRAD data—directly, without converting/decoding it first. Examples include:
|fouswx||plotting Model Output Statistics (MOS)|
|mapplt||plotting geographic data, such as maps|
|rad||plotting radar data|
|sfcwx||plotting surface observations|
|uacalplt||plotting radiosonde soundings|
|upairwx||plotting radiosonde observations|
Observed data are distributed irregularly in space because weather stations and radiosonde launch sites are. However, for some purposes we can more easily analyze or display the data if the data are interpolated to a regular grid of points evenly distributed in space. WXP programs that draw contour analyses, for example, first interpolate data to a regular grid, and there are other programs that display wind vectors on a grid. These are examples of interpolating and contouring programs. Examples include:
|focalc||contouring Model Output Statistics (MOS)|
|grbcalc||contouring computer model output|
|sfccalc||contouring surface observations|
|uacalc||contouring radiosonde observations|
|xsat||contouring satellite image data|
Interpolating and contouring programs all display the gridded data on the
computer screen or send them to a laser printer, but first they create files
containing the gridded data. These files are called grid files.
Several WXP programs read netCDF data or specially formatted text files, perform calculations with it and print the numerical results. These are calculation programs, which produce no graphics or additional files. Examples include:
|heat||calculates heat index|
|moist||calculates various moisture variables|
|stdatms||relates pressure, height, and temperature in a standard atmosphere|
|suncalc||compute sunrise, sunset, twilight|
|wchill||calculate windchill factors|
General Purpose and Other Programs
WXP includes a number of other programs that allow you to overlay or animate multiple images; use other WXP programs more easily; access WXP database files containing names, latitude/longitude locations, and elevations of weather stations, as well as other non-weather information; perform mathematical functions on raw or gridded data; etc. Only the last-mentioned programs operate on weather data, and most of them do not create additional files.
For information about any WXP program, see the WXP Users Guide (http://wxp.unisys.com/wxp/wxp5/Users/index.php) and the WXP Program Reference (http://wxp.unisys.com/wxp/wxp5/Program/index.php).
In partial answer to question (1) above, the LDM software is kept in directories under the home directory of an account called "ldm". WXP programs are kept under the home directory of an account called "wxp". In partial answer to question (3), weather data are stored in subdirectories of directory "/data". Using UNIX commands on a workstation, you can explore the directory hierarchy to see more precisely where the files containing data and programs are stored and how they are named.
The answer to question (2) ("How do we run the programs?") depends on the program:
The Menu Approach
The easiest way to run WXP programs is first to run the interactive WXP program called "wxp", which presents you with menus from which you can make selections that in turn tell the computer to run WXP decoding, plotting, contouring or other programs for you:
norte_wxpuser% wxp WXP: The Weather Processor - version 5.32-Solaris-X11 Main Menu 1: Parsing Programs 2: Plotting Programs 3: Contouring Programs 4: Meteorological Calculations --------------------------- 0: Return to previous menu -1: Exit WXP shell WXP-main>
This approach is very easy because it doesn't require you to know anything
at all about other WXP programs or how they work. However, it's sometimes tedious
to work your way through all the menus. It's also not very flexible—there
are many things that WXP programs can do that you can't do using only the menus
presented by the wxp program. Hence, for some purposes you should consider use
of wxp just a "training wheels" approach to using WXP programs.
The Command Line Approach
The UNIX operating system always prompts you when it's ready to accept a command, and we have control over the appearance of the prompt. For someone logged onto account "wxpuser" on workstation "norte", we've designed the prompt to look something like this:
You would respond to this prompt by typing a command on the same line. This line is called the command line. (In examples of WXP commands below, the prompt is left out for convenience.)
You can run any single WXP program by typing the program name, followed by information needed by the program to do what you want, directly on the command line. For example,
sfcwx -current -region us -variable all -device d
will run the WXP program called sfcwx, which plots surface weather data. In particular, it will plot data from the current hour ("-current") for the whole U.S. ("-region us") in the compact form of station models ("-variable all"), and display it graphically on the computer screen, which is also called the display ("-device d").
For convenience, the same command above can be shortened to:
sfcwx -cu -reg us -var all -dev dor
sfcwx -cu -re=us -var=all -dev=d
If you don't provide on the command line a piece of information needed by the program, the program will either (a) choose default values for you; or else (b) prompt you for the information, often by presenting you with a "menu" of options from among which you select one.
The Script Approach
Two very powerful things that you might want to do with weather data include (a) overlaying two or more "fields" of information, such as wind vectors and contours of pressure or geopotential height, to see more clearly how they relate to each other; and (b) animating sequences of images in "loops". It is possible to generate loops of images using single WXP programs, but it's difficult to overlay fields using just one program. Instead, you usually have to run a sequence of WXP programs (possibly interspersed with UNIX commands), coordinated to achieve what you want. Sometimes it's better to generate animated loops this way, too.
Learning to use WXP programs this way is not easy, but there is a way to "package" sequences of WXP programs, UNIX commands and other programs into a single program, called a script, to create overlays and loops. At SFSU, Dr. Dempsey has written a variety of scripts to make creating overlays and loops relatively easy. You "run" these scripts on the command line, much as you would run a single WXP program. For example:
bayarea -cu 1
will run a script called bayarea, which overlays surface weather data (in the form of station models) on top of radar echoes, for a part of central and northern California that includes the S.F. Bay area. In this case in particular, data recorded one hour before the current standard observing time are displayed on the computer screen ("-cu 1").
xsatloop ir -nu 24
will run a script called xsatloop, which creates an animated loop of satellite images of a type that you specify for a length of time that you specify. In this case in particular, each image consists of an infrared satellite image from the GOES-West satellite ("ir"), which covers the eastern Pacific Ocean and western U.S., and there will be 24 images looped ("-nu 24") at intervals of one hour.
Lists of the WXP scripts that you can run are posted on the bulletin board in TH 607. You can find a similar list, with links to more information about how to run the scripts, at http://funnel.sfsu.edu/wgsl.
The Interactive Approach
The script approach makes creating overlays and loops relatively easy, but you have a limited choice of overlays and loops that you can create, and once you've created them you have no control over them.
However, it is possible to run from the command line, in sequence, the same WXP programs that appear in a script (if you know what they are!). This is more tedious and requires much more sophisticated understanding of how certain WXP programs work, but it does allow you to use certain WXP programs from the command line to delete images from or add them to a loop; overlay additional fields of data, remove fields already overlain; add or remove labels and titles; and so on.
In short, by typing individual WXP commands on the command line, you have much more control over the result that appears on the screen. You can "interact" with the computer to create your own overlays and loops and to change them after you've created them. But it takes a lot of practice to master this approach!
Parameters and Arguments
You have to give most WXP programs information about what precisely you want them to do. This is true even if you just use the wxp menu program to run WXP programs.
For example, if you want to plot surface data, you could run the sfcwx program from the command line, but at a minimum you have to tell sfcwx the particular variable that you want to plot (station model, temperature, pressure, or whatever); the region of the world you want; the observation time you want; and where to send the output (to the "display" or to a laser printer). In addition, if you want, you can tell sfcwx what size, color, font type, and location of labels and titles to use on the plot; whether you want all stations plotted or just some; the location of the file containing the decoded data; the location of files containing latitudes and longitudes of weather stations (so the program knows where on the plot to place the observations); the location of a file containing the map background; and so on. In fact, for the sfcwx program to work, it needs to know about three-dozen different things!
Pieces of information that programs need to work we call parameters or arguments (or resources, as WXP calls them). The values of various parameters needed by sfcwx and other WXP programs have to be specified for them to work.
Default Values for Parameters
It would not be practical for us to specify the values of all three-dozen parameters each time we wanted to run sfcwx. Fortunately, many of the parameters have values that are very common or standard and are not likely to change from one execution of the program to another. Such parameters can be given default values that the program will use unless you tell it otherwise.
We specify many of the default parameter values for WXP programs in a WXP "resource file" usually called "Wxp.res". When you run a WXP program, the program automatically checks the resource file for the default values of many parameters, and prompts you (often by presenting a menu of options) for the values of parameters for which there is no default and that you haven't otherwise specified.
We keep the main resource file, Wxp.res, in directory /usr/local/unidata/wxp/etc, but you can set up your own local version (with the same name) somewhere else, and tell WXP that yours will override the main one.
Specifying Parameter Values
If you don't specify the value of a parameter for which there is no default value, then WXP programs will prompt you for a value, often by presenting you with a menu of options from which to chose. For example, when you run the program "wxp" and thereby take the "menu" approach to running WXP programs, you'll always get menus of options for some parameter values. Your selections from the first two menus presented by wxp always determine which WXP program will run, but subsequent menus present options for parameter values required by the program you've selected. For example:
norte_wxpuser% wxp WXP: The Weather Processor - version 5.32-Solaris-X11 Main Menu 1: Parsing Programs 2: Plotting Programs 3: Contouring Programs 4: Meteorological Calculations --------------------------- 0: Return to previous menu -1: Exit WXP shell WXP-main> 2 Plot Data Menu 1: Plot Surface Data 2: Plot Ship Buoy Data 3: Plot Surface Meteograms 4: Plot Upper Air Data 5: Plot Soundings 6: Plot ETA Model Soundings 7: Plot RCM Radar Data 8: Plot USRad Data 9: Plot NOWRad Data 10: Plot Lightning Data 11: Plot MOS Data 12: Plot MOS Meteograms 13: Plot Maps --------------------------- 0: Return to previous menu -1: Exit WXP shell WXP-plot> 4 Running: upairwx UPPER AIR DATA PLOTTING (Ver 5.32-Solaris-X11) List of available files: /data/decoded/04092600_upa.asc /data/decoded/04092612_upa.asc /data/decoded/04092700_upa.asc /data/decoded/04092712_upa.asc /data/decoded/04092800_upa.asc /data/decoded/04092812_upa.asc /data/decoded/04092900_upa.asc /data/decoded/04092912_upa.asc /data/decoded/04093000_upa.asc /data/decoded/04093012_upa.asc /data/decoded/04100100_upa.asc /data/decoded/04100112_upa.asc /data/decoded/04100200_upa.asc /data/decoded/04100212_upa.asc /data/decoded/04100300_upa.asc /data/decoded/04100312_upa.asc /data/decoded/04100400_upa.asc /data/decoded/04100412_upa.asc Enter the upper air filename:
A drawback of using the "menu" approach to running WXP programs is that you can't change the default value of any parameter that actually has a default value.
However, you can to override the default values of parameters (as well as specify the values of parameters for which there is no default) using the "command line" approach. The general form for specifying parameter values on the command line is:
program_name -par_name1 val_par1 -par_name2 val_par2 ....or
program_name -par_name1=val_par1 -par_name2=val_par2 ....
where program_name is the name of a WXP program. The other stuff on the command line perform two functions:
The observation time can be an exception to the convention for specifying parameter values if you specify it in the form yymmddhh—it's an ordinary argument that doesn't need a parameter name preceding it. Like parameters, you can put it anywhere on the command line (as long as it doesn't come between a parameter and its value).
sfcwx 03030700 -var slp -re ep -dev dor
sfcwx -var slp 03030700 -re ep -dev dor
sfcwx -var=slp -re=ep -dev=d 03030700
All of three of these would plot the same thing—sea-level pressures at the location of each station in the Eastern Pacific region observed on March 7, 2003 at 00Z, plotted on the screen.
For a full summary and documentation of WXP parameters, see the WXP Resource Guide (http://wxp.unisys.com/wxp/wxp5/Resources/index.php).
2. The LDM programs follow instructions that we put in "configuration files" called "pqact.conf". They are located in directories ~ldm/etc and /usr/local/unidata/ldm/etc.