19990327: Defining datasets with DSSERVE

>From: "Jennie L. Moody" <address@hidden>
>Organization: UVa
>Keywords: 199903272230.PAA03345 McIDAS ADDE DSSERVE apostrophe

Jennie,

>I don't understand what I am doing wrong.  I was getting ready to
>test out a new batch file, and I wanted to just use my own mcidas
>directory for some scratch space.  I made up a new ADDE dataset 
>(group/descriptor) JENNIE/GRIDS.  I've done this a few other times as
>I try to get myself transitioned over to all ADDE commands, but I
>have looked at this long enough and cannot figure out what I have
>screwed up:  

The answer I found to your problem did not make a lot of sense to me,
but I found out more as I pursued the topic.  Please see my comments
immediately following your first set of DSSERVE, DATALOC, REDIRECT,
DMAP, and GRDLIST listings and at the end of this email.

>I had defined this dataset in a session with a command like:
>DSSERVE ADD JENNIE/GRIDS GRID 9901 9991 "Jennie's scratch grids
>I didn't turn on the tfile logging until I realized I couldn't get
>this to work, but here are some things to maybe allow you to
>see what I'm doing wrong. (I just went and read through all the 
>ADDE stuff about defining dataset etc, in the learning guide, 
>but it didn't seem to help).
>
>TFILE did OPEN on window 0
>DSSERVE LIST JENNIE
>
>Group/Descriptor         Type  Format & Range     RT Comment
>------------------------ ----- ------------------ -- --------------------
>JENNIE/GRIDS             GRID  GRID 9901-9991        JENNIE'S SCRATCH GRIDS
>DSSERVE: done
>DATALOC LIST
>
>Group Name                    Server IP Address
>--------------------         ----------------------------------------
 ...
>JENNIE                       <LOCAL-DATA>
 ...
><LOCAL-DATA> indicates that data will be accessed from the local data director
> y.
>DATALOC -- done
>
>REDIRECT LIST
>Number of active redirection entries=6
>ADMIN.MSG /incoming/logs
>AREA15* /p2/feb1599
>GRID990* /home/jlm8h/mcidas/data
>GRID99* /p0/users/jlm8h/mcidas/data
>GRID* /p1/FEB99
>MDXX* /p1/FEB99
>REDIRECT: Done
>
>I only had a small case study set of redirections defined,
>and I had messed with it a little, getting more specific
>and then realizing that maybe I needed to give the real
>location (/p0/users/jlm8h/mcidas/data) instead of the
>linked address (/home/).  But it didn't seem to matter.
>  
>DSINFO GRID JENNIE
>
>        Dataset Names of Type: GRID  in Group: JENNIE
>
>Name         NumPos   Content
>------------ ------   --------------------------------------
>GRIDS           91    JENNIE'S SCRATCH GRIDS
>
>DSINFO -- done

>DMAP GRID99*
>PERM      SIZE LAST CHANGED FILENAME DIRECTORY
>---- --------- ------------ -------- ---------
>-rw-  18965452 Mar 26 17:04 GRID9903 /home/jlm8h/mcidas/data
>
>This proves I did have a grid in this directory that I should be able
>to "see".
>
>18965452 bytes in 1 files
>GRDLIST JENNIE/GRIDS
>GRDLIST: Grid data server unable to resolve this dataset: JENNIE/GRIDS
>GRDLIST - done
>GRDLIST JENNIE/GRIDS.1
>GRDLIST: Grid data server unable to resolve this dataset: JENNIE/GRIDS
>GRDLIST - done
>GRDLIST JENNIE/GRIDS.ALL
>GRDLIST: Grid data server unable to resolve this dataset: JENNIE/GRIDS
>GRDLIST - done
>GRDLIST JENNIE/GRIDS.3
>GRDLIST: Grid data server unable to resolve this dataset: JENNIE/GRIDS
>GRDLIST - done

The DSINFO output shows that the dataset _is_ being found by the GRID
'list' server.  So, we both might figure that a GRDLIST should work
fine, but it didn't.  The reason it didn't:  the "'" (apostrophe) in
"JENNIE'S SCRATCH GRIDS" gums up something in ADDE servers.  Please see
my final comment on this at the end of this email.

I found this out by doing exactly what you did at Unidata.  I got the
exact same results as you: DSINFO worked, but a GRDLIST did not.

I verified my hypothesis by editing RESOLV.SRV and removing the apostrophe
and then rerunning GRDLIST JENNIE/GRIDS.  This time, I got the expected
output:

grdlist.k JENNIE/GRIDS.ALL FORM=FILE
DATASET NAME: JENNIE/GRIDS
 Dataset Position  Creation Date Max Grids         Directory Title
 ----------------  ------------- --------- -------------------------------
        3             1999083       2000
GRDLIST - done

>I realized that I have another dataset defined which defines grids in
>a similar range (This isn't the problem, is it?

No, ADDE datasets are designed to be able to handle arbitrary sets of
data files.  It doesn't matter if they overlap between datasets UNLESS
you attempt to copy from one dataset to the other in the position that
is the overlap.

>Am I supposed to locate 
>a set of grid numbers that couldn't possibly be resolved by any other
>dataset?)

No, this is not absolutely necessary.

>I tried to see if using this set of redirections,
>this dataset would see the 9903 grid in my mcidas directory:
>
>DSSERVE LIST TOMS97
>
>Group/Descriptor         Type  Format & Range     RT Comment
>------------------------ ----- ------------------ -- --------------------
>TOMS97/AREAS             IMAGE AREA 9000-9999        AREAS of Total Ozone
>                               Info=AREAS OF TOTAL OZONE
>TOMS97/GRIDS             GRID  GRID 9000-9999        TOMS Earth Probe Total Oz
> on
>                                                     e
>                               Info=GRIDS OF TOTAL OZONE
>DSSERVE: done

This demonstrates that the data file is viable.  By the way, it was this
listing that gave me the idea that the apostrophe was the cause of your
problem.

>GRDLIST TOMS97/GRIDS
>Dataset position 904    Directory Title= 12Z ETA    0   HR <= VT <=24  HR
>PAR    LEVEL       DAY        TIME    SRC FHOUR     FDAY       FTIME  GRID  PR
> O
>---- --------- ------------ -------- ---- ----- ------------ -------- ----- --
> --
>W     100 MB   15 FEB 99046 12:00:00  ETA     0 15 FEB 99046 12:00:00     1 LA
> MB
>Number of grids listed = 1
>GRDLIST - done
>
>It did resolve it?

Absolutely, yes.

>By why not this one:
>
>GRDLIST JENNIE/GRIDS
>GRDLIST: Grid data server unable to resolve this dataset: JENNIE/GRIDS
>GRDLIST - done
>TFILE CLOSE

The damn apostrophe!

>So then I thought about it a few minutes and decided I would start
>over,

This was totally unnecessary.  I would have been unfortunate if this
had worked _if_ you had left off the apostrophe on your DSSERVE invocation.
That would have left you with the incorrect impression that the overlapping
grid file numbers were to blame.

>so I deleted the group/descriptor, and renamed it with
>a unique set of grid numbers 
>
>DSSERVE ADD JENNIE/GRIDS 6001 6101 "JENNIE's SCRATCH GRIDS
>
>Then just to be on the safe side, I cleared the redirections and
>added just two:
>
>AREA15* /p1/FEB99
>GRID6* /p0/users/jlm8h/mcidas/data
>
>And I moved GRID9903 to GRID6003 (I'm doing this because the
>script had a simple design of working with 3 scripts at
>a time, reading in the raw gridfile (3), working on scratch grids
>in gridfile (2) (things like dtheta/dp), and ultimately making the 
>gridfile I want to keep (1): 
>
>But in the end, I get the same error message, 
>
>GRDLIST: Grid data server unable to resolve this dataset: JENNIE/GRIDS

Right.

>By now I'm frustrated,

I don't blame you one bit!

>then I thought, well, its not the way
>I was thinking about this, but let me just revert to the almost
>old way of doing things, and I will use the definition for
>MYDATA/GRIDS, so I tried to list the file that way.  Its defined
>for 9999 grids, so I should see my grid in location 6003 of MYDATA/GRIDS.
>I asked to see the 
>GRDLIST MYDATA/GRIDS
>
>but again, 
>GRDLIST: Grid data server unable to resolve this dataset: MYDATA/GRIDS

This one I really don't understand UNLESS there is an apostrophe (or
equal sign or comman) in the comment section for MYDATA/GRIDS in
RESOLV.SRV.  I have MYDATA/GRIDS setup as GRID files 1-9999 and I did
not get an error using GRDLIST on it:

/home/mcidas/workdata% grdlist.k MYDATA/GRIDS.132 FORM=FILE
DATASET NAME: MYDATA/GRIDS
 Dataset Position  Creation Date Max Grids         Directory Title
 ----------------  ------------- --------- -------------------------------
      132             1999083       2000
GRDLIST - done

On our server (which is the same as your's basically: Sun Ultra SPARC
running Solaris 2.6), the only dataset that screws up is the one with
an apostrophe in the comment section, i.e. your example.

>There is a file called RESOLV.SRV which was clearly
>updated after I made this change, and the last entry in it is:
>
>N1=JENNIE,N2=GRIDS,TYPE=GRID,RT=N,K=GRID,R1=6001,R2=6101,C=JENNIE'S SCRATCH GR
> IDS,

This was the same on my system (last entry that is) except that I was using
GRID file numbers 130 135.

>Similarly it contains an entry:
>
>N1=MYDATA,N2=GRIDS,TYPE=GRID,RT=N,K=GRID,R1=1,R2=9999,C=All gridded data in GR
> ID format,
>
>What is the problem?

The @#$%^&*()_+ apostrophe!

>I feel like I am just sitting here wasting
>time when I have this very clear objective.  I'm about to give up
>and go back and rewrite a script which used the old non-adde commands
>to do what I was going to do with this grid (calculate theta surfaces
>and ipv), but I would really like to make it work right with ADDE
>commands.  What is wrong here?

The "'".

I was going to send this one into SSEC for review, but I decided to read
the online help for DSSERVE before sending off the note.  Here is what I
found:

DSSERVE -- Manages and lists ADDE dataset names on the local server
   DSSERVE ADD dataset format bfile efile <keywords> "comment
   DSSERVE DEL dataset
   DSSERVE LIST group format
Parameters:
   ADD  | adds a new ADDE dataset name to the local server
   DEL  | deletes an ADDE dataset name from the local server
   LIST | lists the ADDE dataset names on the local server (def)
   dataset | ADDE dataset name to add or delete; must be specified in
             the group/descriptor format; see Remarks for restrictions
   format  | file format; valid formats include AREA, GRID, MD, NCDF,
             SYSN and TEXT (no def for ADD, def=all formats for LIST)
   bfile | beginning file number assigned to the ADDE dataset; 'bfile' and
           'efile' are required for AREA, GRID, MD, NCDF and SYSN formats
   efile | ending file number assigned to the ADDE dataset
   "comment | comment describing the ADDE dataset; max of 80 characters;
              the comment cannot contain an equal sign (=), apostrophe (')
              or comma (,) 

So, we were both caught by not reading the documentation for DSSERVE!
Equal signs, apostrophes, and commas are NOT allowed in the comment section!
This had totally escaped me, but there it is in black and white.

Tom

>From address@hidden  Sun Mar 28 08:26:12 1999
Man, its the little things that kill ya.

After I sent my message, I made one last ditch effort to run the batch
I was working on...I logged into the account which was set up for Kevin
as a user, MYDATA/GRIDS was resolved in his account, and I was able to
rename the group/descriptor to make the batch run, and generate a few
plots to ponder before I had to head home.

I will have to review sometime later today why MYDATA didn't work for
my session.

Anyway, thanks for pointing out what I should have noted up front,
since I read through the DSSERVE help a couple of times, but I sure
wasn't thinking about the comment line(!).

Anyway, thanks....you helped restore a small sense of sanity to my
weekend.

Jennie