documentation, and ingest script

Hi Youngsun,

I just reread an old email from you that said you couldn't get to the Site 
Manager's Guide on the web.  I guess I'd forgotten that - all this time I've 
been assuming that you could see it in some form.  Sorry for that 
misunderstanding.

I just talked with the developer who is updating the Guide.  Actually, it is 
changing form into a set of web based links.  Anyway, even though it is 
incomplete, he agreed to make it available so that you could use it.  So, go to
http://my.unidata.ucar.edu/content/software/ldm/ldm-6.0.14/index.html, and from 
there click on this link:

"Incomplete, volatile, unvetted, experimental documentation that I've put here 
as a favor for a friend. I don't want to hear anything about it."

(He has a sense of humor.)  I think this could be useful to you, but, again, 
please bear in mind that it is incomplete.  It is due to be completed in about 
three weeks (for the training workshop).

Also, I could send you a hard copy of the old Site Manager's Guide in the mail. 
 Would that be good for you?  If so, please send me an address where I should 
mail it.

Also, I found the old ingest script I promised you.  I wrote it three years ago 
when I was helping someone else ingest data.  I've appended the text to the 
bottom of this email.  I don't know if it will help you or not - it doesn't 
track when a new file
has arrived per se.  Instead, it just makes a list of files that it finds in 
the directory and pqinserts and removes each one.  My recollection is that it 
was to be called periodically, say via cron.   If it was called every minute, 
then it would pick up
new files every minute, as long as it didn't take more than a minute to 
pqinsert and rm all the files from the last invocation.  As an alternative, you 
could wrap it in a loop inside another script in which case it would just scan 
continuously.

Hope this helps!  Good luck!

Anne
-- 
***************************************************
Anne Wilson                     UCAR Unidata Program            
address@hidden                 P.O. Box 3000
                                  Boulder, CO  80307
----------------------------------------------------
Unidata WWW server       http://www.unidata.ucar.edu/
****************************************************


#!/bin/csh
#-----------------------------------------------------------------------
# Creates a list of all files in the directory.  For each file in the list,
# inserts it into the default LDM queue, then deletes it.
#
# usage: ingest.sh [-v] [-l]
#
# If the verbose option is given, a line will be displayed for each
# product inserted into the queue, plus some additional information
# (S0 far, this additionall information only consists of displaying
# the delivery directory.)  If the log option is given, this same
# information  will be logged to ldmd.log instead of to the screen
# (whether or not the -v option is given).
#
# Set DELIVERY_DIR, below, to the directory containing the files to be
# inserted into the queue.
#-----------------------------------------------------------------------
set LOGGER = echo
set DELIVERY_DIR = /imogene/data/NOGAPS/test

#
# Ensure that LDMHOME is set
#
if ( ! $?LDMHOME ) then
        $LOGGER "To use this script, the environment variable LDMHOME must be 
set."
        $LOGGER "Please set the variable appropriately and try again."
        exit 1
endif
#echo "LDMHOME: " $LDMHOME

#
# Now that LDMHOME is set, we can set PATH correctly
#
set PATH = /bin:/usr/bin:$LDMHOME/bin
#echo "path: " $PATH

#
# Process the arguments
#
while (  "$1" != "" )
        switch ( "$1" )
        case "-v":
                set VERBOSEFLAG = 1
                breaksw
        case "-l":                                              # logs to 
syslogd
                set LOGGER = "logger -t `basename $0` -p local0.debug"
                breaksw
        default:
                $LOGGER "unrecognized flag ($1)"
                set ERRS = 1
                breaksw
        endsw
        shift
end

#
# If we got a bad argument, log the problem and exit
#
if ( $?ERRS ) then
        $LOGGER "usage: `basename $0` [-v] [-l]"
        exit 1
endif

#
# Make the list of files to be processed
#
cd $DELIVERY_DIR
if ( $?VERBOSEFLAG ) then
   $LOGGER "Ingesting files from directory: $DELIVERY_DIR"
endif
set files = `ls -C1`


#
# Insert each file into the queue, then remove it
#
foreach file($files)
  # If the file starts with the string 'TEMP', don't ingest it
  echo $file | grep -v -E '^TEMP' > /dev/null
  if ( $status ) continue
  echo $file
  # Simply log insertion of products to ldmd.log
  if ( "$LOGGER" == "echo" ) then
      if ( $?VERBOSEFLAG  ) then
            # log each insertion to the screen
            pqinsert -v -f EXP $file
      else
            # no output
            pqinsert -f EXP $file
      endif # logger == echo
  else # other logger defined
      # log each insertion to ldmd.log
      pqinsert -v -l $LDMHOME/logs/ldmd.log -f EXP $file
  endif
  rm -f $file
end