Re: 20010629: ewbie question

Unidata Support wrote:
> 
> ------- Forwarded Message
> 
> >To: address@hidden
> >From: Bunny Pfau <address@hidden>
> >Subject: ewbie question
> >Organization: UCAR/Unidata
> >Keywords: 200106292145.f5TLjw107630
> 
> Hello:
> 
> I'm using LDM 5.0.8 on Solaris 7.  The question I have
> I could not find answered in the documention, although
> I bet the answer IS there.
> 
> My question is, when a data file is arriving on a client
> machine, it appears that the datafile arrives ALL AT ONE
> TIME.  See, I'm afraid that a program that I have that
> attempts to read the newly arrived file *might* try
> to access thefile before it has completely arrived--
> like it's only HALF there.
> 
> Tests I've done seem to indicate that some sort of
> caching happens, thus the file that lands in my
> data directory, materializes all at once!  I hope
> I'm getting my message across here--it's kind of
> hard to put into work^R
> hard to put into words..
> 
> Anyway, does the way that LDM works, assure me
> that the requested data file arrives all at once?
> 
> Thanks,
> Bunny Pfau
> HAO
> 
> ------- End of Forwarded Message
Hi Bunny,

In the context of the LDM there are two possible "arrivals" for a
product.  First is when a product arrives and is placed into the product
queue.  Second is when pqact processes a product in the queue and writes
it to a file or to a decoder.  I assume you're talking about when pqact
processes a product in the queue and writes it to a file.

The answer to your question is: no.  You can't be sure that simply
because the file exists it is all there.  Figuring out when a file is
complete can be hard.  Programs invoked by pqact (part of the LDM) that
process the data usually (a) know something about the products so that
they can identify the end of the product, and/or (b) use timing in some
way to decide whether there will be more data coming or not.  If you're
interested let me know and I'll show you a few examples.

Anne
-- 
***************************************************
Anne Wilson                     UCAR Unidata Program            
address@hidden                 P.O. Box 3000
                                  Boulder, CO  80307
----------------------------------------------------
Unidata WWW server       http://www.unidata.ucar.edu/
****************************************************