Re: 20000502: Using pq_insert when LDM isn't running (fwd)

> ---------- Forwarded message ----------
> Date: Tue, 02 May 2000 10:05:39 -0600
> From: Unidata Support <address@hidden>
> Reply-To: Gregory Grosshans <address@hidden>
> To: address@hidden
> Subject: 20000502: Using pq_insert when LDM isn't running
>
> >To: Unidata Support <address@hidden>
> >cc: Gregory Grosshans <address@hidden>
> >From: Gregory Grosshans <address@hidden>
> >Subject: Using pq_insert when LDM isn't running
> >Organization: UCAR/Unidata
> >Keywords: 200005021556.e42FuLG27595
>
>
> I have a question about injecting NOAAPORT data into LDM, and
> minimizing the loss of NOAAPORT data when LDM needs to be restarted.
>
> I'm using modified acqserver code, obtained from FSL, to inject
> NOAAPORT data into LDM.  Basically, the code does:
>
> pq_open
> pq_insert
> checks status
> pq_close
>
> The acqserver code is a separate process from the rest of the 'LDM'
> processes.  Sometimes I need to copy a GEMPAK metar file from this
> computer to another computer where the metar file has been corrupted.
> Presently I stop the acqserver, then perform a 'ldmadmin stop'.
> Obtain the necessary file, then restart LDM and finally start the
> acqserver.  During the time the acqserver and LDM are stopped NOAAPORT
> data  received is potentially lost.
>
> What are the implications to the LDM queue if LDM is stopped for 2-5
> minutes but the 'acqserver' code continues to run and inserting data
> into the LDM queue?  Will the LDM queue become corrupted?  When LDM
> starts will the pqact process see the data inserted in the queue while
> LDM was down?  The easy answer is to not do this and make sure the
> NOAAPORT Receive System buffers/queues up the data.  I'm checking  into
> if and how much data can be buffered on the NOAAPORT Receive System.
>
> Thanks,
> Gregg
>
> ------- End of Forwarded Message

Hi Gregg,

You should be able to continue running the acqserver while shutting down the 
LDM.
The LDM code was written to allow multiple processes to access the queue.  Locks
are used to ensure the queue doesn't get corrupted.  The one thing you need to 
be
sure of is that you shut down the LDM gracefully ('ldmadmin stop' is fine) so 
that
locks are released properly.

Another possible issue is how far back in time the LDM processes will go in
scanning the queue upon restart, as the LDM might miss products inserted while 
it
was down.  However, I think you're ok there - by default: pqact scans from the
time stamp of the oldest product in the queue, pqsend scans from one hour before
its start time, and ldmd requests data at most one hour old.  Each of these
routines (and a few others) take arguments that change these default values 
should
that be necessary.   But I think that you should be able to do what you 
described
above using the default values.

There is a related issue, however, that may or may not be of interest to you.
When you start the acqserver then start the LDM, the resulting LDM processes
belong to one process group and the acqserver belongs to another.  The LDM
processes communicate by sending signals amongst themselves; this depends on 
them
belonging to the same process group.  Of particular interest here, when a new
product is inserted into the queue a signal is sent to inform other processes.
Your 'configuration' still works because processes will also poll to see if the
queue has changed.  However, this is inefficient ( the technical term 'clunky' 
has
been used here).  If efficiency is not a problem for you, great!  If it is, then
we'll need to talk further.   FYI.

Anne

--
***************************************************
Anne Wilson                     UCAR Unidata Program
address@hidden                  P.O. Box 3000
                                  Boulder, CO  80307
----------------------------------------------------
Unidata WWW server       http://www.unidata.ucar.edu/
****************************************************