[IDD #TTR-495389]: Slow access to local datasets through ADDE

Hi Samuel,

re:
> I started changing pqact.conf to try store the images in their own directory 
> based on
> day, however, this may not make a lot of sense for images that only update a 
> few times
> a day like Antarctic IR images.

Since it is easy to specify a regular expression that will match files in 
multiple directories
in the DSSERVE DIRFILE= keyword, I suggest that it is always a good idea to 
have some component
of the storage directory tree be date.  For instance, if you are storing the 
Antarctic IR
images in a directory tree that looks like:

/data/ldm/mcidas/images/sat/<date>/ANTARCTIC/IR/...

Then you can create an ADDE dataset that accesses the images from a single day:

DSSERVE ADD RTIMAGES/ANTARCTIC AREA 
DIRFILE=/data/ldm/mcidas/images/sat/\CURDAY/ANTARCTIC/IR/* "Antarctic IR 
Composite

or an ADDE dataset that access the images from all of the days you are keeping 
online:

DSSERVE ADD RTIMAGES/ANTARCTIC AREA 
DIRFILE=/data/ldm/mcidas/images/sat/*/ANTARCTIC/IR/* "Antarctic IR Composite

If you decided to keep 7 days of Antarctic IR images online, the dataset would 
be composed of 
56 images.


> I tried dividing up our RTIMAGES/ANTARCTIC images, but IDV no reports "No 
> data available for the selection"
> when trying to access RTIMAGES/Antarctic IR Composite.  There is imagery in 
> the child directory where
> the IR images used to be, and the dsserve.k command seemed to work.  I'm at a 
> loss as to why it didn't work.
> 
> Here is a log of what happened:
> 
> -bash-2.05b$ dsserve.k ADD RTIMAGES/ANTARCTIC AREA TYPE=IMAGE 
> DIRFILE=/data/ldm
> /mcidas/images/sat/ANTARCTIC/4km/IR/\\CURDAY/IR_* \"Antarctic IR Composite

In this dsserve.k invocation, you did not escape the '*'.  The invocation should
have been:

<as 'mcidas'>
cd $MCDATA
dsserve.k ADD RTIMAGES/ANTARCTIC AREA TYPE=IMAGE 
DIRFILE=/data/ldm/mcidas/images/sat/ANTARCTIC/4km/IR/\\CURDAY/IR_\* "Antarctic 
IR Composite

> Group/Descriptor         Type  Format & Range     RT Comment
> ------------------------ ----- ------------------ -- --------------------
 ...
> RTIMAGES/ANTARCTIC       IMAGE AREA                  Antarctic IR Composite
> DIRFILE=/data/ldm/mcidas/images/sat/ANTARCTIC/4km/IR/\CURDAY/IR_*
> dsserve.k: done

Hmm... This definition looks OK to me.  

> -bash-2.05b$ ls 
> /data/ldm/mcidas/images/sat/ANTARCTIC/4km/IR/20080206/IR_200802
> 06_1500 -l
> -rw-rw-r--    1 ldm      ldm        142425 Feb  6 12:44 
> /data/ldm/mcidas/images/
> sat/ANTARCTIC/4km/IR/20080206/IR_20080206_1500

This looks like it should have worked.  What is the result if you try to access
the RTIMAGES/ANTARCTIC images from McIDAS?

try:

<login as 'mcidas'>
cd $MCDATA
dataloc.k LIST RTIMAGES      <- verify that your McIDAS session will look for 
the
                                RTIMAGES dataset on shu either as a LOCAL-DATA
                                dataset or by pointing to the remote ADDE 
interface
                                on shu.cup.edu (I am assuming that you are 
serving
                                the data from shu.cup.edu; if not point to the 
machine
                                that is doing the serving)
imglist.k RTIMAGES/ANTARCTIC

What is the result?

> P.S. Is there a way to save these changes once we're happy with them, or are 
> they
> saved automaticaly on reboot?

The ADDE dataset definitions created by DSSERVE are stored in the disk file 
$MCDATA/RESOLV.SRV,
so the definitions are preserved across McIDAS invocations and system reboots.

By the way, it would help me in troubleshooting if I could either get SSH 
access to
shu as 'mcidas' again (I understand about there being login attempts by many 
hackers,
but shu's firewall can be configured to allow SSH access by machines in a 
single domain or,
if needed, by a single machine), and/or to open up access to the ADDE server on
shu.cup.edu so I can run McIDAS commands from my office.

Cheers,

Tom
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: TTR-495389
Department: Support McIDAS
Priority: Normal
Status: Closed