[IDD #IER-713765]: MOS text

Hi Dana,

re:
> Apologies for sounding vague here, but we ceased creating MOS text
> products at Rutgers here quite some time ago and have a need to start up
> again..

OK.

> I *believe* we used to use Peter Neilley's 'weather' program to
> pull these out of the LDM

To be precise, the products received by the LDM would have to be written
into a gdbm database in a particular manner to be used by Peter's 'weather'
application.  Your FILE actions will simply write the products to disk
if/when they are received.

> as they were stored thusly:
> 
> ##WMO     ^FEX[E|C|S|W].. .... ([0-3][0-9])([0-2][0-9])   FILE
> /data/weather/txt/AVN_MOS/(\1:yy)(\1:mm)\1\2.AVN_MOS
> ##WMO     ^FEAK20 KWB[C-K] ([0-3][0-9])([0-2][0-9])       FILE
> /data/weather/txt/AVN_MOS/(\1:yy)(\1:mm)\1\2.AVN_MOS
> ##WMO     ^FOX[E|C|S|W][4-6]. .... ([0-3][0-9])([0-2][0-9])       FILE
> /data/Weather/txt/MRFMOS/(\1:yy)(\1:mm)\1\2.MRFMOS
> ##WMO     ^FOUS20 KWB[C-K] ([0-3][0-9])([0-2][0-9])       FILE
> /data/weather/txt/MRFMOS/(\1:yy)(\1:mm)\1\2.MRFMOS
> ##WMO     ^FOUS12 .... ([0-3][0-9])([0-2][0-9])   FILE
> /data/weather/txt/MOS/(\1:yy)(\1:mm)\1\2.MOS
> ##WMO     ^FOAK12 .... ([0-3][0-9])([0-2][0-9])   FILE
> /data/weather/txt/MOS/(\1:yy)(\1:mm)\1\2.MOS
> ##WMO     ^AXUS53 KWB[C-K] ([0-3][0-9])([0-2][0-9])       FILE
> /data/weather/txt/NEW_MOS/(\1:yy)(\1:mm)\1\2.NEW_MOS
> 
> Is this data still in the WMO feedtype, and are the FOUS20 and FOUS12
> patters still valid?

The easiest AND quickest way to figure this out yourself is by using
the LDM 'notifyme' utility.  For instance, here is the notifyme invocation
that will check for the availability of the FOUS12 products in the queue
of the top level IDD relay, idd.unidata.ucar.edu:

notifyme -vxl- -f ANY -o 10000 -h idd.unidata.ucar.edu -p '^FOUS12 .... 
([0-3][0-9])([0-2][0-9])'

Notes:

- '-o 10000' says to look back 10000 seconds from the present in the LDM
  queue on the host idd.unidata.ucar.edu ('-h idd.unidata.ucar.edu')

- -p '^FOUS12 .... ([0-3][0-9])([0-2][0-9])' specifies the pattern to search
  for in product's WMO header (note the single quotes around the pattern!)

This 'notifyme' invocation returned the following when I ran it a minute ago:

Mar 01 05:04:12 notifyme[16677] NOTE: Starting Up: idd.unidata.ucar.edu: 
20080301021732.915 TS_ENDT {{ANY,  "^FOUS12 .... ([0-3][0-9])([0-2][0-9])"}}
Mar 01 05:04:12 notifyme[16677] NOTE: LDM-5 desired product-class: 
20080301021732.915 TS_ENDT {{ANY,  "^FOUS12 .... ([0-3][0-9])([0-2][0-9])"}}
Mar 01 05:04:12 notifyme[16677] INFO: Resolving idd.unidata.ucar.edu to 
128.117.140.3 took 0.000344 seconds
Mar 01 05:04:12 DEBUG: NOTIFYME(idd.unidata.ucar.edu) returns OK
Mar 01 05:04:12 notifyme[16677] NOTE: NOTIFYME(idd.unidata.ucar.edu): OK
Mar 01 05:04:23 notifyme[16677] INFO: a8f3c6f0eb047bfa9a8d96caf3fa6321     5932 
20080301045136.675 IDS|DDPLUS 151221986  FOUS12 KWNO 010000 /pFTPSLR
Mar 01 05:04:23 notifyme[16677] INFO: b765b290b0b9774f4e57f2889186d1fa    11974 
20080301045136.677 IDS|DDPLUS 151221987  FOUS12 KWNO 010000 /pFTPKRF
Mar 01 05:04:23 notifyme[16677] INFO: d1c06a99da659be60f564239f7ce16f4     6265 
20080301045136.679 IDS|DDPLUS 151221988  FOUS12 KWNO 010000 /pFTPTUR
Mar 01 05:04:23 notifyme[16677] INFO: caae0e89950a5f69dcc13038f52cb991     3403 
20080301045136.682 IDS|DDPLUS 151221989  FOUS12 KWNO 010000 /pFTPHFD
Mar 01 05:04:23 notifyme[16677] INFO: a0707e48a301b027da5b4750f87146ce     7777 
20080301045136.685 IDS|DDPLUS 151221990  FOUS12 KWNO 010000 /pFTPMSR
Mar 01 05:04:23 notifyme[16677] INFO: 06865d203d7d5d79f7427312464d2ff0     3858 
20080301045138.611 IDS|DDPLUS 151222004  FOUS12 KWNO 010000 /pFTPRHA
Mar 01 05:04:23 notifyme[16677] INFO: ecaf783497946f5f2303b893108628a6     4080 
20080301045138.630 IDS|DDPLUS 151222005  FOUS12 KWNO 010000 /pFTPCIN
Mar 01 05:04:23 notifyme[16677] INFO: f3796ef7a60a0f2db907b39a3864e0c5     3489 
20080301045138.639 IDS|DDPLUS 151222006  FOUS12 KWNO 010000 /pFTPRSA
Mar 01 05:04:23 notifyme[16677] INFO: b513ee6f1dde958a841b20f2779e2b3d    44411 
20080301045138.668 IDS|DDPLUS 151222008  FOUS12 KWNO 010000 /pFTPPTR
Mar 01 05:04:27 notifyme[16677] NOTE: exiting

Note:

- the last line that says 'exiting' was written because I killed the invocation 
with CRTL-C

- the datastream containing the FOUS12 product is actually IDS|DDPLUS

  For reference, WMO is a compound feedtype:

  WMO == IDS|DDPLUS|HDS


A similar 'notifyme' invocation whose pattern expression ('-p') was the one you 
had for
FOUS20 products listed nothing. This implies that the products with an ID that 
would
match the pattern are no longer in the datastream, or none of those products 
have been
received in the 10000 seconds previous to when I ran 'notifyme'.

The 'notifyme' utility is _extremely_ useful in:

- checking for the availability of products in the IDD

- testing patterns for use in pqact.conf actions

- searching through your own LDM queue to see if you have received
  a particular product.  To search your own LDM queue, simply leave
  out the '-h hostname' specification.

Next:

A Google (tm) search using 'mos wmo header' as a search key yields a number of
useful documents describing MOS products.  For instance:

http://www.weather.gov/mdl/synop/stadrg.php

contains information on 'Current GFS and Eta MOS Stations and WMO Headers'.

To test if any of the headers listed in the www.weather.gov page are available 
in the
IDD (if the products are broadcast in NOAAPort, they will be in the IDD), use 
'notifyme'
in a manner similar to the example above.

For instance, look up the MOS header for New Jersey stations in the 
www.weather.gov
web page and you will find:

Northeast U.S. Region MOS text product WMO headers/AWIPS ids:
FOUS21 KWNO / MAVNE1 - GFS short-range MOS for Northeast U.S. {233}
FEUS21 KWNO / MEXNE1 - GFS Extended-range MOS for Northeast U.S. {233}
FOUS44 KWNO / METNE1 - Eta MOS for the Northeast U.S. {233}

So, try a notifyme using '-p FOUS21 KWNO' for GFS short-range MOS,
'-p FEUS21 KWNO' for GFS Extended-range MOS, and '-p FOUS44 KWNO' for
Eta MOS, all for the Northeast U.S.

One example of this is:

notifyme -vxl- -f ANY -o 10000 -h idd.unidata.ucar.edu -p 'FOUS21 KWNO'

etc.

> If not, is there another way to pull MOS text out of the LDM?

Please see my comments above and let me know if you have any questions.

> I spent quite a while searching the unidata support site
> but couldn't find much in the way of help for this one..

The MOS products are broadcast by the NWS, so they are the best source
for information on the WMO headers for the products.

> Thanks!

No worries.

Cheers,

Tom
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: IER-713765
Department: Support IDD
Priority: Normal
Status: Closed