20000509: ldm at stc (cont.)

>From: alan anderson <address@hidden>
>Organization: St. Cloud State
>Keywords: 200003081631.JAA28926 IDD LDM notifyme feed IDD McIDAS DMRAOB addenda

Alan,

I am jumping in to answer (try to answer) this email since I think at
least part of it may relate to McIDAS-XCD.

>I have some more questions regarding the data ingested here at stc.
>
>As you may recall, we have noticed significant delays in receiving data, 
>and through Jeff W., we were provided with at least a temporary alternate
>feedsite, using host zelgadis at iowa state, rather than our old site 
>chinook at unl.

OK.  I looked through our tracking system and read the exchange you had
with Jeff.

>I have monitored data from both sites using  the  notifyme command in the ldm.

Good move.

>Latest problem is that notifyme seems to start ok, but I do not get any
>output to the screen.  This started today, although notifyme did run ok
>earlier this morning.  I then updated the clock time on the ldm machine,
>and it was after that when I recall having problems with notifyme.  

Sounds like the clock time on your machine is not correct.  This can cause
notifymes to not return information.

>I have stopped the ldm and restarted it but the problem persists.

I don't think that this would be the problem.

>A sample entry of my use of notifyme and the resulting response
>is listed below. 
>
>/usr/local/ldm/logs% notifyme -vxl - -f DDPLUS -o 3600 -h
>waldo.stcloudstate.edu
>May 09 20:15:37 notifyme[576]: Starting Up: waldo.stcloudstate.edu:
>20000509191537.010 TS_ENDT {{DDPLUS,  ".*"}}
        NOTIFYME(waldo.stcloudstate.edu) returns OK
>May 09 20:15:37 notifyme[576]: NOTIFYME(waldo.stcloudstate.edu): OK
>^CMay 09 20:15:42 notifyme[576]: Interrupt
>May 09 20:15:42 notifyme[576]: exiting

I just ran this exact command from a machine here at the UPC and got:

notifyme -vxl - -f DDPLUS -o 3600 -h waldo.stcloudstate.edu
May 09 21:55:51 notifyme[15707]: Starting Up: waldo.stcloudstate.edu: 
20000509205551.467 TS_ENDT {{DDPLUS,  ".*"}}
        NOTIFYME(waldo.stcloudstate.edu) returns OK
May 09 21:55:51 notifyme[15707]: NOTIFYME(waldo.stcloudstate.edu): OK
May 09 21:55:51 notifyme[15707]: 77222a5fecb1fa0efab7f256630c7461     2428 
20000509205552.848 IDS|DDPLUS 423  FXUS65 KLKN 092055 /pAFDEKO
May 09 21:55:52 notifyme[15707]: 1e30e96b0b1558c7ef631afc3a91e476     3752 
20000509205552.916 IDS|DDPLUS 425  FPUS53 KGLD 092055 /pZFPGLD
May 09 21:55:52 notifyme[15707]: 82bcab64d75eceef1e55b5408fef1f96      142 
20000509205553.010 IDS|DDPLUS 427  SAUS43 KEAX 092055 /pMTRMCI
May 09 21:55:52 notifyme[15707]: f9aefc8857172ee618d0bdf6404bd583      274 
20000509205553.436 IDS|DDPLUS 433  SRPO20 KWAL 092054
May 09 21:55:52 notifyme[15707]: 93652c3e796e20e13731c7f33971c0a2      132 
20000509205553.525 IDS|DDPLUS 434  SAUS44 KLIX 092055 /pMTRMSY
May 09 21:55:52 notifyme[15707]: 378b6bcf2945167d18738d9a8d1a7792      139 
20000509205553.531 IDS|DDPLUS 435  SAUS44 KMRX 092055 /pMTR6R6
May 09 21:55:52 notifyme[15707]: fb0ab714b5fe8e5f35015d8e058a6251      165 
20000509205553.540 IDS|DDPLUS 436  SAUS42 KMLB 092055 /pMTRMCO
May 09 21:55:52 notifyme[15707]: 875abc29a58b6b6b33791c2f9105cc74      132 
20000509205557.726 IDS|DDPLUS 440  SAUS42 KFFC 092055 /pMTRAHN
^CMay 09 21:55:53 notifyme[15707]: Interrupt
May 09 21:55:53 notifyme[15707]: exiting

So, there is apparently nothing wrong with your LDM.

>/usr/local/ldm/logs% notifyme -v -l - -h localhost
>May 09 20:13:18 notifyme[566]: Starting Up: localhost: 20000509201318.865
>TS_ENDT {{ANY,  ".*"}}
>May 09 20:13:18 notifyme[566]: NOTIFYME(localhost): OK
>^CMay 09 20:13:48 notifyme[566]: Interrupt
>May 09 20:13:48 notifyme[566]: exiting
>
>I used the ^C to interrupt notifyme after several seconds of waiting when it 
>was apparent nothing was being reported.  

I used it to kill my output since things are apparently working.

>Can you explain why notifyme does not seem to work  (what am I doing wrong) ?

I am betting on it having something to do with the timestamp of the products
in your queue and the new time on your machine.

>Also, when it has worked in the past,  when I use -o tttt as part of the 
>argument, I assume it looks back tttt seconds,

Yes, it starts tttt seconds ago.

>but once it has scanned 
>through that part of the file, does it just sit and wait as new data comes in?

Yes, exactly.  It has been told to 'notify me' when data comes in.

>i.e. can you tell me how it 'finishes' or does it keep running while waiting 
>for the latest data?

It never finishes.  It will keep reporting receipt of products for as long
as notifyme continues to run.

>Now, on the subject of our 'new' feedsite, zelgadis, our delay times seemed 
>shorter, but then zelgadis went down or something happened last week, as 
>we had no data and I got  "not responding" to an ldmping.  I emailed their
>site contact, Geff Underwood, but got no reply.

Unfortunately, getting a reply from a site sometimes takes awhile.

>Today, an ldmping showed 
>that zelgadis was again responding, so I switched our ldm back.  My ldm
>log file showed that things started ok.  

OK, good.

>Our data files, at least MD files, still are showing alot of missing data.
>i.e. only half or less of the raob sites show up on an upper air plot of
>temps. for this morning.  This was in spite of notifyme showing time delays
>of only 1/2 hour or less this morning when it worked for me initially.
>Any ideas?

Did you install the McIDAS-XCD addendum that fixed a problem with the
upper air decoding?  In looking back throuh our FTP logs, I see that you
have downloaded new McIDAS-X 7.6x distributions right along to different
machines, but I don't recall seeing you either reget the distribution
get the addenda for waldo.  Have you?  The addenda for upper air
decoding was made on February 12, 2000.  The Addendum page that covers
this is:

http://www.unidata.ucar.edu/packages/mcidas/mcx/addenda_000212.html

If you didn't get and install the addendum, then your upper air decoding
will be full of stations with missing data.

>I have not seen anyone else complaining about missing data.

McIDAS-XCD sites that noted missing upper air data have gotten past
the problem by installing the addendum.

>Is there any consistency to the size of raob MD files and if so, can 
>you tell me what a typical 'full' MD file for raobs is (size) ?

The file size will be more-or-less the same even if a number of stations
have no data since the format of the file is fixed by its schema.

>Well, that is enough.  It is finals week here.  I also have a joke or two, 
>but will wait.

Hey, why hold back on jokes?

>Regards
>Alan 

Tom