[IDD #KSB-345648]: LDM Glider data. (notifyme)

Hi Chuan-Yuan,

re:
> I am now thinking the glider data what I am looking for is in BUFR format 
> (binary),
> which might be the reason why I cannot see it by notifyme.

'notifyme' just lists the LDM/IDD metadata we refer to as its Product ID for
products.  It doesn't care if the products themselves are in binary or ASCII
formats.

re:
> As what I checked with WMO report and BUFR category log, I found the 
> oceanographic
> observation data (bufr format) is beginning at “IO”, where IOB is the buoy, 
> IOS belongs
> to the sea-surface and below sounding, IOP is the sub-surface profile floats 
> (I believe
> the glider data is in this category).

OK, but the question still remains: is glider data being distributed in the IDD?
Please remember that the content of the IDS|DDPLUS and HDS feeds is what the
National Weather Service is sending to its forecast offices via the NOAAPort
Satellite Broadcast Network.  We neither add data nor remove products from the
data being delivered in datastreams that derive from NOAAPort.

re:
> Consequently, I used notifyme to check the results.
> 
> notifyme -vl- -f HDS -h idd.unidata.ucar.edu -o 8640000 -p '^IOP.* '
> Aug 23 19:01:31 notifyme[19999] NOTE: Starting Up: idd.unidata.ucar.edu: 
> 20170515190131.725 TS_ENDT {{HDS,  "^IOP.* "}}
> Aug 23 19:01:31 notifyme[19999] NOTE: LDM-5 desired product-class: 
> 20170515190131.725 TS_ENDT {{HDS,  "^IOP.* "}}
> Aug 23 19:01:31 notifyme[19999] INFO: Resolving idd.unidata.ucar.edu to 
> 128.117.140.3 took 0.001682 seconds
> Aug 23 19:01:31 notifyme[19999] NOTE: NOTIFYME(idd.unidata.ucar.edu): OK
> 
> It contains nothing after coupled minutes.

This says that no product in the HDS datastream with a Product ID that begins 
with 'IOP'
was seen on idd.unidata.ucar.edu during the time that you ran 'notifyme'.

re:
> But it does contain some info about the IOS and IOB. The results is as 
> following.
> 
> notifyme -vl- -f HDS -h idd.unidata.ucar.edu -o 86400 -p '^IO.* '
> Aug 23 18:58:45 notifyme[18586] NOTE: Starting Up: idd.unidata.ucar.edu: 
> 20170822185845.447 TS_ENDT {{HDS,  "^IO.* "}}
> Aug 23 18:58:45 notifyme[18586] NOTE: LDM-5 desired product-class: 
> 20170822185845.447 TS_ENDT {{HDS,  "^IO.* "}}
> Aug 23 18:58:45 notifyme[18586] INFO: Resolving idd.unidata.ucar.edu to 
> 128.117.140.3 took 0.001511 seconds
> Aug 23 18:58:45 notifyme[18586] NOTE: NOTIFYME(idd.unidata.ucar.edu): OK
> Aug 23 18:58:47 notifyme[18586] INFO:    24238 20170823163315.099     HDS 
> 159507651  IOSX01 KWBC 231633 /pBUFR
> Aug 23 18:58:50 notifyme[18586] INFO:      212 20170823172521.216     HDS 
> 27800443  IOBF08 KWNB 231700
> Aug 23 18:58:50 notifyme[18586] INFO:      216 20170823172523.118     HDS 
> 27800444  IOBF08 KWNB 231700 RRB
> Aug 23 18:58:51 notifyme[18586] INFO:      216 20170823172523.126     HDS 
> 27800445  IOBF08 KWNB 231700 RRA
> Aug 23 18:58:51 notifyme[18586] INFO:      216 20170823172529.122     HDS 
> 27800471  IOBF08 KWNB 231700 RRC
> Aug 23 18:58:51 notifyme[18586] INFO:      210 20170823172535.096     HDS 
> 27800525  IOBF08 KWNB 231700 RRG
> Aug 23 18:58:51 notifyme[18586] INFO:      210 20170823172535.098     HDS 
> 27800526  IOBF08 KWNB 231700 RRD
> Aug 23 18:58:51 notifyme[18586] INFO:      233 20170823172535.098     HDS 
> 27800527  IOBF08 KWNB 231700 RRE
> Aug 23 18:58:51 notifyme[18586] INFO:      216 20170823172535.120     HDS 
> 27800528  IOBF08 KWNB 231700 RRF
> Aug 23 18:58:51 notifyme[18586] INFO:      210 20170823172539.121     HDS 
> 27800568  IOBF08 KWNB 231700 RRI
> Aug 23 18:58:51 notifyme[18586] INFO:      216 20170823172539.139     HDS 
> 27800569  IOBF08 KWNB 231700 RRH
> Aug 23 18:58:51 notifyme[18586] INFO:     2242 20170823172542.686     HDS 
> 159628203  IOSX01 KWBC 231325
> Aug 23 18:58:51 notifyme[18586] INFO:     2200 20170823172542.687     HDS 
> 159628205  IOSX01 KWBC 231325

OK.

re:
> To further investigate whether the glider data is being included in the file, 
> I tried to
> modify my pqact.conf.

Please remember that 'notifyme' will show all of the products being received
by the machine it is interrogating.  Changing an action in a pattern-action
file (e.g., ~ldm/etc/pqact.conf), will have no effect on what 'notifyme' will
report.  Also remember that in order to process a product, a pattern-action
file action must match the extended regular expression you include in the
action.

re:
> Here is the code part.
> 
> HDS   ^IO.... KWNB (..)(..)
> FILE  -close   -overwrite  data/ldm/bufr/%y%m\1\2.bufr

OK.  One quick comment on your action:

- you action assumes that the day and time of the product match
  the day and time of your system's clock

  While this is true for most of the time, it will not be correct
  when your clock is running in local time and the product's day/time
  are in UTC.  A better way of writing this action is:

HDS<tab>^IO.... KWNB (..)(..)
<tab>FILE<tab>-close<tab>-overwrite data/ldm/bufr/(\1:yy)(\1:mm)\1\2.bufr

  This form uses the day contained in the Product ID (e.g., '23' of '231700')
  as a flag/clue to 'pqact' to create the correct date.

re:
> Here is my question.
> 
> 1. Why I cannot locate any IOP info by notifyme? Is it meaning that the 
> IOPXXX is not
>    in the system?

If you run 'notifyme -vl- -v ANY -p "^IOP" ...' for long enough (e.g., a full 
day)
and you do not see any indication of one being found, it means that there are no
products being delivered in the IDD that have a Product ID that matches.

re:
> 2. What is the meaning of the 159507651 in the line

> Aug 23 18:58:47 notifyme[18586] INFO:    24238 20170823163315.099     HDS 
> 159507651  IOSX01 KWBC 231633 /pBUFR

It is a sequence number.  It is not part of the Product ID which is 'IOSX01 
KWBC 231633 /pBUFR'

re:
> 3. I have check my pqact.conf by ldmadmin pqactcheck.  I didn’t see any error.

OK, good.

re:
> Do you know why can I not see any downloading data in my data directory 
> “data/ldm/bufr”.

It is likely that there are no products that match the extended regular 
expression
that you are using in your pattern-action file action(s).  Again, if 'notifyme'
does not show the receipt of a product you are interested in, then the 
action/actions
you have defined for that product/those products will never be executed.

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: KSB-345648
Department: Support IDD
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.