20021001: refining how many days of McIDAS data are kept online (cont.)

>From: William C Klein <address@hidden>
>Organization: Valparaiso
>Keywords: 200208191558.g7JFw6K19855 McIDAS-X mcscour.sh

Bill,

>Here is the snibbit of the mcscour.sh that you were talking about from my
>machine:
>
>MCPATH=$MCPATH PATH=$PATH LD_LIBRARY_PATH=$LD_LIBRARY_PATH mcenv << EOF
>
>qrtmdg.k GRID 5001 6000 1
>doqtl.k  1  70 9
>doqtl.k 71  80 9
>doqtl.k 81  90 9
>doqtl.k 91 100 9
>delwxt.k 1 10
>igu.k DEL 132
>lwu.k DEL VIRT9001
>lwu.k DEL VIRT9002
>lwu.k DEL ROUTEPP.LOG
>exit
>
>EOF
>
># Done
>exit 0

If this really is the mcscour.sh file that is being used to scour McIDAS
data, you should be keeping 9 days of POINT source files online.  Since
I can point at your remote ADDE server (on aeolus.valpo.edu) and only
see two days worth of data, it is most likely that a different copy of
mcscour.sh is being used to scour the McIDAS data.  It is even possible
that you have two different invocations of mcscour.sh running from cron.
Perhaps one is running from the 'mcidas' account and a different one
is running from the 'ldm' account?  If this is the case, and the second
mcscour.sh is setup for the default number of days to keep (2), then
I could easily understand how there would be a discrepency between the
listing above and the number of files that we see on your machine.

>[ aeolus : mcidas : ~/bin ]
>[ 9 ] >
>
>It looks as though it's already setup for 9 days of data.  Should there be
>a change elsewhere?

No, not if you are only scouring using mcscour.sh.

>Like in:
>
>[ aeolus : ldm : ~/etc ] ???  There is a scour.conf.

I strongly recommend that the LDM scouring mechanism _NOT_ be used to scour
McIDAS data.  The main reason (unless one is really careful to setup things
up correctly) is that the directory containing McIDAS-XCD decoded files
should contain files that do not get updated (i.e., get old).  The LDM
scouring approach is to delete files from directories that are older
than a certain number of days.  What could happen if LDM scouring is
setup to scour McIDAS-XCD produced files is that these files (SCHEMA, in
particular) would get deleted after they got to be 'n' days old.  This
would be _bad_.  If one uses the mcscour.sh approach, this will not
happen.

>Does this also limit the # of days that are being held?

If you are scouring the directory that McIDAS-XCD files are being written
into, then yes, this would also limit the number of days being held.

I recommend that you check the following:

1) make sure that mcscour.sh is only being run from one account.  I recommend
   running it from the 'ldm' account.

2) make sure that the single copy of mcscour.sh that is being run is
   configured the way you want (like the listing above)

3) make sure that the LDM scouring is _not_ setup to scour the directory
   containing McIDAS-XCD produced data files.

Once you have only one copy of mcscour.sh being run by cron, you will
start keeping more days of data on line.

>Thanks,

Please keep me up to date with your problem.  I will be in and out
for the next couple of days as I am attending the GOES Users Conference
here in Boulder.

Tom

>From address@hidden Tue Oct  8 08:52:54 2002
>Subject: Re: 20021001: refining how many days of McIDAS data are kept online 
>(cont.)

Tom,

I think that the mcscour.sh is being run from a mounted directory and
that's why we are not getting the full amount of data.

Bill