20000201: Unidata-Wisconsin products removed in July of 1999

>From: "Karli Lopez (McIDAS)" <address@hidden>
>Organization: University of Puerto Rico
>Keywords: 200002010754.AAA03412 datastreams ROUTE PP BATCH LDM

Karli,

>Although we have our main feeds back up and fully operational, some
>feeds have stopped working:
>
>1.  The following feeds have stopped on or around 99182 (July 1st) and
>as I understand they're now sent through XCD, which we have installed
>and configured.

Just to be precise (since others can read the email archives to help
them sort out their own questions/problems), XCD decoders convert data
in the NOAAPORT streams into McIDAS-usable data files.  What is sent
is the "raw" (undecoded) data.  XCD (X Conventional Decoders) just
do the decoding.

>How do we know if we're getting the latest equivalent
>data?

The XCD routines do not update the McIDAS routing table, so it is a
little harder to see if you have current data.

>Will it reflect in the routing tables?

No, but we modified the surface and upper air decoders to update the
System Key Table, SYSKEY.TAB.  In order for this to work, the 'mcidas'
account has to be setup to have REDIRECTions to SYSKEY.TAB, and the
user running the XCD decoders (this is the user that is running the
LDM) will have to be able to read and write SYSKEY.TAB.  You
would then check the data accessibility status by using SYSVAL LIST:

SYSVAL LIST 2001 2100

Look through the list for the various data types.  Which SYSKEY.TAB
entries are used for which data types can be seen in a listing of
SYSKEY.DOC:

 2001 I XXX    " UNIDATA: CURRENT ISFC MD FILE NUMBER
 2002 I XXX    " UNIDATA: CURRENT ISFC HOUR
 2003 I XXX    " UNIDATA: CURRENT ISFC DAY
 2004 I XXX    " UNIDATA: CURRENT SYN MD FILE NUMBER
 2005 I XXX    " UNIDATA: CURRENT SYN HOUR
 2006 I XXX    " UNIDATA: CURRENT SYN DAY
 2007 I XXX    " UNIDATA: CURRENT ISHP MD FILE NUMBER
 2008 I XXX    " UNIDATA: CURRENT ISHP HOUR
 2009 I XXX    " UNIDATA: CURRENT ISHP DAY
 2011 I XXX    " UNIDATA: CURRENT IRAB MD FILE NUMBER
 2012 I XXX    " UNIDATA: CURRENT IRAB HOUR
 2013 I XXX    " UNIDATA: CURRENT IRAB DAY
 2021 I XXX    " UNIDATA: CURRENT IRSG MD FILE NUMBER
 2022 I XXX    " UNIDATA: CURRENT IRSG HOUR
 2023 I XXX    " UNIDATA: CURRENT IRSG DAY
 ...

>They are:
>
>  MA Surface MD data            default  MDXX0002     99182 1431 SFC.BAT      3
>  NF Global Initialization Gri  101-102  GRID0101     99182 1038 GLOBAL.BAT   3
>  NG Early Domestic Products      1-2    GRID0002     99182  457 ADDGRID.BAT  3
>  RM Mandatory Upper Air MD da  default  MDXX0012     99182 1434 MAN.BAT      3
>  RS Significant Upper Air MD   default  MDXX0028     99178 1433 SIG.BAT      3
>  U4 Wind profiler              default  PROFILER.CDF 99181 2006 PROFILER.BAT 3
>  US Undecoded SAO Data         default  UNIDATAS     99182 1433 none     1
>
>The steps to activate GRID decoding under XCD were undertaken yet the
>routing table reflects no changes.

The GRID issue is a little more complicated.  XCD decodes all the model
data available in the datastream to McIDAS GRID files.  The NF and NG
products that were in the Unidata-Wisconsin datastream will not be
found in the set of decoded GRIDs.  It is not as though the data is
not there, it is and then some; the special thing about those GRID files
was the order of the grids in the GRID file was set.  The Unidata Fkey
menu system and MCGUI counted on the specific grid ordering.

One can setup a conversion of XCD-created GRID files into equivalents of
the Unidata-Wisconsin GRID files by using the Unidata McIDAS 7.60 UWGRID
process (see the online help for execution syntax).  Since one would
most likely want to have these GRID files automatically generated, I
included a Bourne shell script, uwgrid.sh (gets installed in ~mcidas/workdata)
that can be:

o edited to set McIDAS environment variables MCDATA, MCPATH, etc.
o kicked off from cron to recreate the NF and NG products that were
  removed from the Unidata-Wisconsin stream on July 1, 1999

The tricky part about running uwgrid.sh is figuring out when to run the
script.  The correct time is after the XCD-generated GRID files for
the particular product (AVN for the NF product; NGM for the NG product);
this will depend on your IDD connection, so you will have to play around.

>2. More importantly, as of December 31st (99365) we have stopped
>receiving all Topographical composite imagery and all GOES-8/9
>Composites as well.  (These would be feeds CI, CV, CW, and N1..N8)
>
>If these feeds have actually been discontinued (as I fear they have)
>where can we get replacements?

The composite imagery are not broadcast.  Each site generates the composites
from the images that are received in the Unidata-Wisconsin broadcast.
If you stopped creating these it is likely due to one or more of the
following:

o your routing table entries for products CI, CV, CW, N1..N8 are SUSpended
o your system is no longer setup to run McIDAS Route PostProces BATCH
  files upon receipt of Unidata-Wisconsin imagery

To check the first possibility, do the following:

o make sure that there is a copy of ROUTE.SYS in the directory in which
  the Unidata-Wisconsin imagery (AREA) files are created
o start a McIDAS-X session as the user 'mcidas'
o make sure that you have a REDIRECTion to the copy of ROUTE.SYS in
  the output directory
o do a listing of the routing table:

  ROUTE LIST

If you see entries like:

s CI GOES-E/W IR Composite       80-89       none        none        none     3
s CV GOES-E/W VIS Composite      90-99       none        none        none     3
s CW GOES-E/W H2O Composite      70-79       none        none        none     3

(the operative thing here is the 's' in the first column indicating that
the entry is SUSpended)

then you need to RELease the entries:

ROUTE REL C
ROUTE REL N

If the entries are already RELeased (i.e., not SUSpended), then the Route
PostProcess BATCH setup is not working (or is no longer in place).  To
troubleshoot this, you need to do several things as the user running
your LDM:

o make sure that you have an executable copy of the Bourne shell script,
  batch.k, that is distributed with the ldm-mcidas package in a directory
  in the PATH of the user running the LDM

o make sure that that copy of batch.k has been edited and that the
  McIDAS enviorment variables MCDATA, MCPATH, etc. have been set to
  match your system setup

o as the user 'mcidas' you will have had to make sure that there is a
  copy of ROUTE.SYS and SYSKEY.TAB in the directory in which the
  ldm-mcidas decoder, lwtoa3, writes its output data files; these files
  need to be both readable and writable by the user running the LDM

After these things are attended to, then the composites should once again
be created on your system.

>3. Another thing are there any Mollewide WV/TOPO Composite feeds being
>provided?

One can produce an IR Mollweide topography composite.  The reason that
no Water Vapor composites have ever been produced is that the water vapor
images basically have image data values for each display point. This means
that there would be no place for the topography to show through, so why
create the composite.

>Thanks for your help, : )

No problem.  It sounds like you are just about back to where you once
were on the data processing side.  Please don't hesitate to ask for
more help if you don't understand how the PostProcessing stuff works.
It is a little hard to understand since it is somewhat convoluted.

Tom Yoksas