20050224: ADDE on local host only (cont.)

>From: "Valentijn Venus" <address@hidden>
>Organization: ITC
>Keywords: 200502232310.j1NNAlv2007536 McIDAS ADDE compress gzip execl

Hi Valentijn,

I must admit that when I saw this email back on Feb 24, I only skimmed
it thinking it was intended for Don only (I saw saw the section
about the status bar in IDV and just assumed...).  I apologize!

>Tom, thanks for suggesting the importance of RESOLV.SRV in your last
>e-mail. For some reason it was completely empty on the machine i'm
>trying to serve METEOSAT-8 images from. copying the
>~mcidas/workdata/RESOLV.SRV from another machine which had the right
>entries sort of "solved" the issue.

OK.

>Tom, shouldn't DSSERVE ADD ... do the job,

Yes, DSSERVE is the application that adds entries to a RESOLV.SRV
file.  If you have more than one RESOLV.SRV file in the directories
included in your MCPATH set of directories, it will attempt to update
the first one found in searching the directories from left to right.  I
am uncertain of how your MCPATH was setup when you ran the DSSERVE
commands, so I would rather not speculate on how you could end up with
a zero-length RESOLV.SRV file.

>or is DSINFO IMAGE MSG really necessary to add the entries in
>RESOLV.SRV?

DSINFO's only job in life is to as an ADDE server about what it
datasets it has configured.  It could be the case that you had no
RESOLV.SRV file when the DSINFO command was first run.  McIDAS defaults
to opening a file if it exists or creating it if it doesn't even for
reads.  If a DSINFO was run when there was no RESOLV.SRV file, then an
empty one could have been created by this mechanism.  I'm shooting in
the dark here...

>Now that this ADDE datasource can successfully be accessed from a McIDAS
>client using MCCOMPRESS=3DGZIP, i started experimenting with accessing
>the same ADDE datasource from IDV. IDV sees the data, but when
>requesting an image i get a progress bar with no progress showing. After
>downloading and installing NCOMPRESS from the Fedora site (i'm running
>Fedora Core 2) on the machine serving the ADDE data (and adding the path
>of the binary to PATH in .mcenv, which now looks something like
>PATH=${MCGUI}:MCPATH:/usr/local/bin/), UNCOMPRESSed tranfers are also
>working, but COMPRESSed tranfers still not. Because of that the McIDAS
>client (in MCCOMPRESS=3DTRUE mode) nor IDV can see the data. I came
>across the e-mail below, in which the new GZIP tranfers are discussed
>between the two of you. Don, I read somewhere that IDV can only use
>COMPRESSed tranfers. Is that true, or can IDV be equipped with the
>necessary modules to support this new type of ADDE compression?

IDV works with 'compress' compressed transfers and 'gzip' compressed
transfers.

>Tom,
>what could be wrong with my McIDAS config, since after i added the
>COMPRESS/UNCOMPRESS program UNCOMPRESS seems to work fine, but COMPRESS
>not?

The problem is that adding the directory where 'compress' lives to the
PATH in .mcenv will not do what you would expect.  The ADDE server
module 'mcserv' needs to execl the actual 'compress' binary, not just
run it.  Because of this, it tries to exec it out of a standard set of
directories.  Here is the code from mcserv.cp that demonstrates the
process:

                                if (pre.servport==htonl(COMPRESS_PORT) ||
                                    pre.servport==htonl(COMPRESS_KEY))
                                {
                                        execl("/bin/compress",
                                              "compress", "-cf",0);
                                        execl("/usr/bsd/compress",
                                              "compress", "-cf",0);
                                        execl("/usr/bin/compress",
                                              "compress", "-cf",0);
                                        execl("/usr/contrib/bin/compress",
                                              "compress", "-cf",0);
                                }

So, since your installation of 'compress' was in a directory other than
one in this list (you put yours in /usr/local/bin), it was not found by
'mcserv'.  On Linux that support RPMs, it is always best to grab the
RPM that contains compress/uncompress and use 'rpm' to install the
executables in standard locations.

Now, you might ask why 'gzip' works.  Both 'gzip' and 'gunzip' are
built as part of the McIDAS package and are installed in known
locations.  Here is the code snippit form mcserv.cp that 'execl's
'gzip':

                                if (pre.servport==htonl(GZIP_PORT) ||
                                    pre.servport==htonl(GZIP_KEY))
                                {
                                        char gzspeed[8];
                                        strcpy(gzspeed,"-");
                                        if(getenv("MCGZIPCOMP") != (char *)0)
                                        {
                                          strcat(gzspeed,getenv("MCGZIPCOMP"));
                                        }
                                        else
                                        {
                                          strcat(gzspeed,GZIP_SPEED);
                                        }
                                        execlp("gzip", "gzip", "-cf",
                                              gzspeed, 0);
                                        execl("/usr/local/gzip/bin/gzip",
                                              "gzip", "-cf", gzspeed, 0);
                                }

Notice that 'execlp' is used to find 'gzip' first.  This will find the
executable in the ~mcidas/bin directory since it _is_ in the PATH
defined in .mcenv.  Why SSEC did not use 'execlp' for all its searches
is a bit of a mystery to me.

Again, I apologize for not reading your inquiry more carefully the
first time!  If I had, I would have seen that you were waiting for
a response from me, and I would have responded quickly.

Cheers,

Valentijn
--
NOTE: All email exchanges with Unidata User Support are recorded in the
Unidata inquiry tracking system and then made publicly available
through the web.  If you do not want to have your interactions made
available in this way, you must let us know in each email you send to us.

>From address@hidden  Wed Mar  9 01:27:18 2005

Hi, Tom

No need to apologize, it seems to be all working now. Thanx for helping!

Now uncompressed Meteosat data can succesfully be served to clients
using ADDE my first request becomes more urgent: making the McIDAS MSG
server-code capable of handling the wavelet compressed Meteosat-8
segmented files.

 I'm trying to evangelise McIDAS/ADDE and the concept of distributed
data services, but for some African co-universities and particulary a
regional training centre for remote sensing in Kenya data storage cost
is an issues. No need to explain that the requested modification would
reduce storage costs considerably.

Valentijn