[McIDAS #KDN-271049]: /home/data/mcidas/images isn't scouring, filling disk

Hi Gilbert,

re: /home/data/mcidas/images isn't scouring, filling disk
> Says it all on weather, weather2 and weather3.

Yesterday afternoon I logged onto weather2 and started a hard look at the
setup there.  I found that the NEXRAD Level III images were being written
into /home/data/gempak/nexrad/NIDS, not /home/data/mcidas/images/NIDS.
Because of this difference, the directories were not being scoured by
the scourBYnumber entry in your crontab.  The /home/data/gempak/nexrad
directory structure was apparently setup to be scoured in ~ldm/etc/scour.conf,
but I found that some of the subdirectories had thousands of files.  Because
of this, I changed the crontab entry to scour /home/data/gempak/nexrad/NIDS
instead of /home/data/mcidas/images/NIDS.  I also ran the scour entry by
hand to verify that it worked correctly; it did.

> Isn't mcscour.sh supposed to do this?

No.  mcscour.sh is designed to scour XCD-produced data files (e.g., MD files,
*.IDX files; etc.).  mcscour.sh was running correctly on weather2 when I
just looked, but I changed it nonetheless:

- changed the output log file from /home/mcidas/workdata/scour.log to
  /home/ldm/logs/mcscour.log.  This makes your scouring match other site's
  scouring

I did not get a chance to finish looking at weather2 yesterday afternoon.  I
had intended to revisit the scouring setup on weather2 this evening.  Your
email reminds me that I need to visit all three machines and regularize
the cron-initiated scouring on each.

Question:

- do I have carte blanche for setting up the scouring?  Of course, I will
  not do anything drastic, but I think some tidying up is needed.

> Why hasn't it been updated since 2004? Just curious.

What, mcscour.sh?  If the setup on weather is the same as it was on weather2
when I logged on this morning, then it was running OK.  Like I said above, 
however,
I want to regularize the scouring and scour logging so that the log file for
mcscour.sh is ~ldm/logs/mcscour.log.

As for /home/data/mcidas/images not being scoured, it is on weather2.  If it
is not on weather/weather3 it would mean that the example crontab entries
created by the McIDAS configuration script mcxconfig have not been incorporated
into the crontab on those machines.  A quick look at weather shows that this is,
in fact, the case.  So, I just added the mcxconfig-generated scouring entries
to the crontab file on weather.

... a few minutes later ...

I just noticed that you added the use of the mcxconfig-generated 
pqact.conf_mcidas[AB]
files on weather.  The problem is that you did not comment out McIDAS actions 
out
of ~ldm/etc/pqact.conf.  The result of this is that several process are being 
run
redundantly.  This can/will cause excessive system use and potential corruption
of output data files.  I have taken the liberty of commenting out the redundant
actions from ~ldm/etc/pqact.conf.

... after a few more minutes ...

OK, I am confused.  Before proceeding I need to know what you really want to
do wrt decoding of imagery.  The reason I ask is I see that on weather you
have actions in ~ldm/etc/pqact.gempak that are decoding imagery using pnga2area
and storing it into a hierarchy under /home/data/gempak/images.  You also have
the mcxconfig-generated actions doing the same decoding into a different 
directory
structure.  I figure that the directory that is not getting scoured (gempak)
is the one that is filling up your disk.

Help!  I need a more clear picture before being able to proceed!!

Cheers,

Tom
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: KDN-271049
Department: Support McIDAS
Priority: Normal
Status: Closed