Re: Huge Files

Hi Brian,

With some help from others I've tracked the problem down to a problem 
with /home/ldm/util/ucnids.   Those very large files are filled not with 
data but with this:

ldm@ldm:/mnt/ldmdata/NNEXRAD_NCR/KOAX/20041123> tail 
KOAX_SDUS53_NCR_200411231638.raw

Read: 0 21 1
Cpy: 21
Read: 0 21 1
Cpy: 21
Read: 0 21 1
Cpy: 21
Read: 0 21 1
Cpy: 21
Read: 0 21 1
Cpy: 21

which is a ucnids issue.  If you execute this:
        strings /home/ldm/util/ucnids
you'll see that the above is ucnids output, i.e., output from this 
command includes these lines:
        Read: %d %d %d
        Cpy: %d
which are the lines that are generating the above.  Maybe ucnids is 
trying to simultaneously read and write the same file?  I would 
carefully examine the relevant pqact.conf entries to see if you're 
having some sort of a race condition that is causing this.

Please note that this has nothing to do with the upgrade.  It must have 
been occuring before the upgrade as well.

Are you using WXP?  We don't support WXP and haven't for some time. 
FWIW, our GEMPAK expert here pointed out that if you're using GEMPAK or 
MCIDAS you don't need to uncompress the radar data upon receipt as both 
those packages will uncompress for you when you ask them to display the 
data.

You could try to debug this problem by running ucnids on a good version 
of the problematic data.  We have the correct data files available on 
our archive - you can go to http://motherlode.ucar.edu/ and get the 
correct version of the corrupted files and try running ucnids on them, 
which might reveal the problem.  If ucnids runs properly on the good 
data from the command line then the problem is with interaction across 
or within pqact.conf entries.

If this is really a problem for you I may be able to help you after the 
holiday.  Unfortunately, I can't debug this problem for you now.  I was 
planning on starting my vacation over an hour ago.  For now could you 
deal with the problem by turning off the decompression?

Anne

Miller, Brian S wrote:
> Anne-
> I removed all of the huge files from /home, but there are still a few in
> /mnt/ldmdata/NNEXRAD_NCR
> In that directory there are 6 radars.  Look at the data for today in
> each radar folder and you will see what I am talking about.
> Thanks again.
> Brian
>
> -----Original Message-----
> From: Anne Wilson [mailto:address@hidden] 
> Sent: Tuesday, November 23, 2004 3:25 PM
> To: Miller, Brian S
> Subject: Re: Huge Files
>
> Hi Brian,
>
> Where are these files?  Please respond ASAP if you can.
>
> Anne
>
> Miller, Brian S wrote:
>
> > Anne-
> >
> > Since the upgrade there are random HUGE files that are being created
>
> on 
>
> > our LDM server.  Every now and then instead of having a 1MB file there
>
>
> > is a 2GB file in its place.
> >
> >
> >
> > Can you take a look at what it is doing and let me know if I can do 
> > anything to stop the "bad" data files?  It is filling up my storage
>
> fast.
>
> > Brian

--
***************************************************
Anne Wilson                     UCAR Unidata Program            
address@hidden                 P.O. Box 3000
                                  Boulder, CO  80307
----------------------------------------------------
Unidata WWW server       http://my.unidata.ucar.edu/
****************************************************