20030123: Some more advice needed ...

>From:  "=?ISO-8859-1?Q?Marianne=20K=F6nig?=" <address@hidden>
>Organization:  EUMETSAT
>Keywords:  200301231030.h0NAUbx07842 McIDAS ADDE

Hi Marianne,

Sorry it has take me so long to get back to you on this, but things
have been extremely hectic around here lately.

>I have to come back to you with a question:
>
>I believe we discussed this last fall: I told you that we have big
>problems in reading in satellite data if we don't just go line by line
>(successive calls to mcalin), e.g. we want to read in line 10, then
>line 1, then line 135 and keep jumping around in the image. What the
>reading program has to do is to keep calling mcaget with a difference
>select clause w.r.t. LINELE ro whatever. On our system, this just
>completely slows down the program. Now, if I remember correctly you
>said it should not do this because several calls to mcaget should not
>be so time-consuming, but as a second thought you mentioned that that
>has to do with an NFS server (is this correct?).

I think I remember making the comment that _if_ your server is reading
the image data off of a file system that is NFS mounted, then the
access could be slow.  This is especially true on certain OSes where
NFS is not very efficient.

>Now, given the fact that we do have our data on a central server and
>need to access it via some sort of network server, would there be an
>alternative - i.e. a different NFS server or a different approach
>within mcidas I could do?

If you could run an ADDE server on the central server, then the
disk access to the data would be local, and therefore not slowed
by NFS.

>The reason why I am asking: So far I have circumvented the problem by
>having a general "read satellite data" subroutine that reads in an
>entire image (raw data) in the first call and then gives back whatever
>line the calling program requests (and does all this calibration mickey
>mousing via the kbx... routines). Of course that is rather memory
>consuming (if you think of processing different channels within one
>program). For MSG this will be a real killer because the images are so
>large and additionally go to 2 bytes instead of 1 byte Meteosat. So any
>sggestion you have on that would be very welcome.

The buffered read approach is one that I would take if I couldn't run
the ADDE server on a machine where the data resides locally.  I
probably wouldn't, however, try to read in the entire image, but, then
again,  I don't know exactly what you are trying to do with the data.
I understand that you jump around in the image, but do you really jump
all over the image?  What I am thinking about is whether or not you
could read a sector of the image in one go and then jump around in that
piece.  Also, depending on what you are trying to do, you could ask for
the entire image, but blown down by some factor.  If your server is
written like SSEC McIDAS ADDE servers, it will sample the original
image and not interpolate values.  The values you then have will be the
same as those from the original, but you won't have values that are
right next to one another.

>Sorry to "misuse" you as mcidas helpdesk, but the MUG Help Desk was
>never very helpful on this issue.

I am afraid that I am not being very helpful either, for that I
apologize.

>Thank you,
>Marianne

Tom
--
+-----------------------------------------------------------------------------+
* Tom Yoksas                                             UCAR Unidata Program *
* (303) 497-8642 (last resort)                                  P.O. Box 3000 *
* address@hidden                                   Boulder, CO 80307 *
* Unidata WWW Service                             http://www.unidata.ucar.edu/*
+-----------------------------------------------------------------------------+