19990616: July 1st Datastream changes

>From: "Jennie L. Moody" <address@hidden>
>Organization: UVa
>Keywords: 199906162038.OAA01382 McIDAS LDM cron ROUTE PP

Jennie,

>Well, I am just getting around to dealing with
>the impending changes in the datasteam.

Just in the nick of time ;-)

>We have McIDAS-XCD
>running, of course, but we have been using the arrival 
>time of these non-image products to trigger Postprocess
>batch files.  I guess there are two ways to redo our 
>post-processing.  One would be to revert to cron jobs
>(we used to do things this way).  The other way is to
>try to reconfigure our pqact to run scripts when a
>product (e.g., a model) arrives. (I presume we would have
>to put some "sleep" time in here, or something that peeks
>to see if the decoded product is available, since there must
>be some lag time between the first arrival of a header
>detected in pqact and the availability of the complete,
>decoded GRID, though I don't have a good sense for
>how long this might be.)

The time delay could be pretty variable depending on how loaded your
machine is and how long it takes for the entire product to come in.
There is no really good guidance here other than to allow yourself
a good buffer of time.

>Anyway, do you recommend
>one approach over the other (cron vs. pqact)?  

I would go with something based on pqact since it is tied with your
system actually receiving the data that you are probably interested
in.

>Another question that arises from this impending change is
>the following:  if we have post-process batch routines that
>run triggered by the arrival of an image product, will we
>be able to continue to run these.

Yes.

>I guess this should certainly
>continue to work for us running the current version of mcidas
>(for us this is still 7.4), but is this a feature (postprocessing)
>that will be gone in our upgrade to 7.6?  

Actually, the viability of running ROUTE PP BATCH routines depends on
how the data is actually sent, not on the version of McIDAS.  Currently,
the products in the UW stream are processed (compressed/packetized) so
that they need to be processed (uncompressed/depacketized) by routines
in the McIDAS distribution.  Some of the McIDAS routines are the ones
that support use of/update of the routing table and running of ROUTE
PP BATCH commands.  At some point we will probably want to change how
the imagery are sent.  At that point, I will have had to come up with
a routing table replacement since ROUTE PP BATCH files are really useful.

>Thanks.  Wish we weren't so late in dealing with this, but
>its been a busy period of proposal writing around here, and
>there hasn't been time for much else.  Hopefully we can make
>all the necessary changes in the next week.

Things have been kinda busy here as well.  I got 7.6 from SSEC and have
put together a distribution that appears to work (still in the testing
mode).  I have also been working on the NIDS and NOWrad ADDE servers
and have begun thinking about ADDE servers for TeraScan (tm) and GINI
imagery.

Tom