[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20030313: some more on waldo's disk overflow of Wed. (cont.)

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: 20030313: some more on waldo's disk overflow of Wed. (cont.)
  • Date: Thu, 13 Mar 2003 09:34:55 -0700
>From: "Anderson, Alan C. " <address@hidden>
>Organization: St. Cloud State
>Keywords: 200303111741.h2BHfFB2012143 McIDAS-X v2002 setup

Hi Alan,

>Just a few comments on what I found this morning.
>
>Waldo seems to be doing fine,  disk is at 55 %.  Looking at
>/var/data/mcidas, where most all of our decoded stuff is kept,  I find
>2  of those large DD*.XCD files,  one from yesterday and one for
>today.

This hints that the scouring is not working as it should.

>Looked at my cron entry for user ldm,  it is set for every three hours
>to run scour.

'scour' is the LDM scour routine.  Scouring of McIDAS data is best
handled by the 'mcscour.sh' script which must also be run from cron
once per day.

>I assume that scour.conf  located in ~ldm is what determines what is
>scoured, and that leads to some questions.  My scour.conf  looks
>puzzling to me in that the lines identifying the directories to be
>scoured are listed as   ~ldm/data/nnn  where  nnn  is represented  by
>forecasts, severe,  gempak,  surface and several others.

Right, scour is a general routine that scours directories.  Since it
doesn't understand McIDAS data files, and doesn't understand that some
files in the directory in which ldm-mcidas decoders create output
don't get updated but must not get deleted, scour is not the thing
to use to scour McIDAS data.

>I know that ~ldm/data/nnn   is linked to /var/data/ldm/nnn.   My puzzle
>is that I know that most of the data is in /var/data/mcidas, but I
>can't see how the scour.conf file points there.    The only part that
>seems consistent to me is that ~ldm/data/gempak does point to the place
>where the gempak data is going (/var/data/ldm/gempak).

I see from logging on as 'mcidas' that waldo is setup to run
/home/mcidas/workdata/mcscour.sh every day at 21:00 local time:

/home/mcidas% crontab -l
00 21 * * * /home/mcidas/workdata/mcscour.sh

This is as it should be.  I took a look in /home/mcidas/workdata/mcscour.sh
and see that it is also properly configured.  The time stamp on the
scour.log file for mcscour.sh scours looks like mcscour.sh was run
on time, and the entries in the scour.log file look like the scouring
routines were, in fact, run.  The only thing we might want to consider
is adjusting when the mcscour.sh script is run.  We can look more closely
at that later.

>Also, the file extensions listed in scour.conf don't seem to contain
>those used in the files in /var/data/mcidas.
>
>So how is /var/data/mcidas being scoured  ?

/home/mcidas/workdata/mcscour.sh being run from cron in the 'mcidas'
account.

>There must be a simple
>answer since it has been working fine  for a long time.

Yup.

I will keep an eye on things and see if I see anything suspicious.

Tom