Re: 20040529: CRAFT archival

Inline comments!

Unidata Support wrote:
> > From: Gerry Creager N5JXS <address@hidden>
> > Organization: Texas A&M University -- AATLT
> > Keywords:  200405270011.i4R0BjtK010667 LDM Linux RAID JFS
>
>
> Gerry,
>
>
> > You may want to look at bigbird.  I'm not seeing any Level II data 
> > that's not staying bzip'd.
>
>
> I switched the pqact action for CRAFT data to filing bzip2 compressed
> products directly.  The reason I did this was to see the impact on CPU
> use and to see if it would help with the congestion on the RAID.  Also,
> the latest GEMPAK distribution 5.2.2p2 (which I loaded on bigbird) can
> use the bzip2 compressed images directly.

OK.  Then the amount of archive I can maintain is significantly greater. 
 Think we should change scour to, say, 15 days for now?

> > If that's the case, I can maintain a lot of 
> > CRAFT archive.
>
>
> Yes, a lot more than if all images were uncompresed.
>
>
> > In fact, I was thinking of adding a script that'd keep 
> > only the last 24 hrs of Level II uncompressed, and then keep perhaps 10 
> > days or more of compressed data as an archive.
>
>
> The current GEMPAK pqact.gempak_craft pattern action file has
> the commented-out entry for decoding the CRAFT images.  This could
> be turned back on and scouring could be setup to keep about a day
> of uncompressed images.

Duh! <Slaps head>  I should have noted that, but missed. it.  I'll play 
there now.

> > I'm also not seeing DDPLUS anywhere I can identify.
>
>
> IDS|DDPLUS data is being received and apparently decoded.  There is
> current data in:
>
> /data/ldm/gempak/surface
> /data/ldm/gempak/syn
> /data/ldm/gempak/upperair
> etc.
>
> Where are you looking, or, what am I missing?

One of the projects I'm working on is using the text watches and 
warningsparsing same and placing the info on the described polygons in a 
postgis database.  The .gem's don't get me there.  I also was looking 
for the METARs...

> > I was going to 
> > start looking at making ShapeFiles from watch/warn data but I can't use 
> > the info here 'til I can find the DDPLUS stuff.
>
>
> Are you looking for the raw IDS|DDPLUS data to be written to disk, or
> are you looking for the data to be decoded into GEMPAK format?  The
> data is being decoded into GEMPAK format, so if something else is
> missing, it is because there is no appropriate action(s) in
> ~ldm/etc/pqact.conf.  If this is the case, you could add the action
> running on mesodata and send a HUP signal to the pqact that is
> running to process actions in pqact.conf.

I'll take care of it.  THanks.

> > That's not a 
> > show-stopper.  I'll nfs-mount from mesodata, but that box is creaking 
> > and may need to be forklifted soon.
>
>
> If you want to have a day's worth of uncompressed CRAFT data online,
> let me know and I will turn the decoding back on.

If I don't need it for GPNEXR2 decoding, no.  The compressed form is 
more efficient!

Thanks, Gerry
--
Gerry Creager -- address@hidden
Texas Mesonet -- AATLT, Texas A&M University        
Cell: 979.229.5301 Office: 979.458.4020
FAX:  979.847.8578 Pager:  979.228.0173
Office: 903A Eller Bldg, TAMU, College Station, TX 77843