19990809: Beware, beware!

Bob,

Thanks for checking.

If Gilbert is seeing files on his disk twice as large as 158k,
then what is likely happening is that the product queue has been 
recreated- so that when the LDM is restarted, the same files are 
retransmitted to his LDM. If his pqact.conf file uses the
FILE action without the -overwrite option, subsequent files
that are received with the same name will be appended to the 
output file rather than overwriting, so the file size will
appear larger in multiples of the 158k size.

In general, when you stop and restart the LDM, you do not 
receive duplicate files since the MD5 check sum is compared.
When the product queue is deleted and remade, then there is nothing
to prevent duplication of FILE'ing unless the -overwrite is used.

Problems that are being seen with strange file names in FILE actions
are generally due to pqact.conf pattern errors, including mismatching
the escape patterns, inadvertant control characters in the
pqact.conf file (typically revealed with "od" on the file,
or multiple file actions to the same output file creating
stepping on the file.

Hopefully we can obtain the necessary information to locate the
problem.

Steve Chiswell
Unidata User Support





>From: NIMBUS Librarian <address@hidden>
>Organization: .
>Keywords: 199908092306.RAA00179

>
>I am not aware of anything that has changed at FSL.  It appears that
>we are generating netCDF files with the same size we expect,
>and that the transfers to thelma seem normal (over the last
>couple 6-min periods, anyway):
>
>-rw-r--r--   1 rtoper   fd        156168 Aug  9 22:28 9922122180006o
>-rw-r--r--   1 rtoper   fd        156168 Aug  9 22:34 9922122240006o
>-rw-r--r--   1 rtoper   fd        156168 Aug  9 22:40 9922122300006o
>-rw-r--r--   1 rtoper   fd        156168 Aug  9 22:46 9922122360006o
>-rw-r--r--   1 rtoper   fd        156168 Aug  9 22:51 9922122420006o
>
>shylock:/usr/local/ldm/logs> notifyme -h thelma.ucar.edu -v -l - -f FSL2
>Aug 09 22:51:46 notifyme[16891]: Starting Up: thelma.ucar.edu: 19990809225146.
> 271 TS_ENDT {{FSL2,  ".*"}}
>Aug 09 22:51:46 notifyme[16891]: NOTIFYME(thelma.ucar.edu): OK
>Aug 09 22:52:07 notifyme[16891]:   156168 19990809225159.787    FSL2 000  FSL.
> NetCDF.NOAAnet.windprofiler.06min.19992212242.*
>
>Unidata folks:  please let me know if there's anything else we might
>check on our end.
>
>       Regards,
>
>               Bob Lipschutz
>               NOAA/Forecast Systems Laboratory
>               Facility Division
>               325 Broadway  R/E/FS2
>               Boulder, CO 80303
>               303-497-6636
>
>> Hello all,
>> 
>> After a TON of searching, debugging, throwing everything into verbose mode
>> on my LDM, I have just figured out why my data feed as been out much of
>> the day. First, my apologies to my downstream sites.
>> 
>> FSL is sending some files with bizarre names and my pqact is choking
>> on it, giving a "signal 11" error (that's the equivalent of a segmentation
>> fault). My backup machine, which is running 5.0.7, apparently doesn't
>> give a rip about these files and saves them without error. For now, I have
>> turned off the FSL entries in my pqact (but my downstream sites should
>> still be getting FSL2 just fine). If you are having your LDM die,
>> and are running 5.0.8, that may be the reason. I will now ask the powers
>> that be why they are sending these files across the FSL feed (twice the
>> normal size of the hourly and six minute stuff), and why only 5.0.8 is
>> choking on it. These have the names of "Net.CD" in them. I haven't seen
>> these before. The other six minute and hourly files are coming in just
>> fine, and are being processed normally.
>> 
>> Gilbert
>> 
>