20040818: MSG-1 through ADDE (cont.)

>From:  "Valentijn Venus" <address@hidden>
>Organization:  ITC
>Keywords:  200408161528.i7GFScaW025855

Hi Valentijn,

I just wanted to let you know that I received your email pointing
me to the wavelet code.  I will try to make some time to look into
the effort it would take to incorporate the decompression into
the MSG server, but this will probably have to wait until  I return
from Brazil.

One thing that I need to know, however.  You note:

"The licensing
agreement that we signed allows use for non-commercial and educational
purposes only."

Does your license then give me a right to look at the code and
incorporate it into the Unidata McIDAS releaes?  Or, do I need to
follow the same path and sign a license agreement with EUMETSAT?

Cheers,

Tom

>Clicking the following url
>(ftp://ftp.itc.nl/pub/venus/PublicDecompWT.zip) will provide you the
>sources and documentation to the Public Decompress Wavelet Transform
>library produced by VCS and published by EUMETSAT. The licensing
>agreement that we signed allows use for non-commercial and educational
>purposes only. The example provided in this WinZip archive is may not be
>usefull. Another, more ellaborate example is provided in the MSG
>HRIT/LRIT code (http://meteosatlib.sourceforge.net/) which depends on
>the same library for the MSG image decompression.
>
>Let me know if you need more help...
>
>
>Valenitjn
>
>________________________________
>
>From: Valentijn Venus=20
>Sent: Friday, August 13, 2004 10:13 PM
>To: Unidata Support
>Cc: address@hidden; Harald van der Werff;
>address@hidden
>Subject: RE: 20040809: MSG-1 through ADDE (cont.)
>
>
>for your convenience i'v included a better (easier) link to the set of
>example msg-1 data:
>http://oiswww.eumetsat.org/WEBOPS/msg_images/20030509/=20
>=20
>Don't bother about the "msg1_native.nat.bz2", which is the format
>Marianne is working with. The notes that EUMETSAT provides with the data
>are included below:
>=20
>
>HRIT (unencrypted)
>
>Each of these files has been bzip2
><http://sources.redhat.com/bzip2/index.html>  compressed and contains
>identical prologue and epilogue files (the prologue and epilogue files
>can be overwritten if all files are extracted into a single directory).
>Note that matching JPEG files are available above. =20
>
>________________________________
>
>From: Valentijn Venus
>Sent: Fri 8/13/2004 21:59
>To: Valentijn Venus; Unidata Support
>Cc: address@hidden; Harald van der Werff;
>address@hidden
>Subject: RE: 20040809: MSG-1 through ADDE (cont.)
>
>
>Hi Tom,
>=20
>just got word from a Cuban student a tornadao heading for the States did
>some serious damage to their Island. McIDAS provided proof of the event.
>Wonderful to have such a system in place!
>=20
>My problems with our MSG ADDE server disappeared partially after setting
>COMPRESS=3Dnone, and fully after equipping a faster machine with McIDAS.
>This answers my former questions, but a new one of more serious nature
>arised (i think). When displaying an image, the frame in question shows
>just the metadata (sensor name, time of observation, etc.). The image
>itself is not displayed. I believe it has to do with the fact that the
>EUMETCAST data is compressed using a proprietary wavelet transform. We
>have the sources (in c) from EUMETSAT to decompress them, and after
>looking in the MSGT server-files i found no similair occurences of the
>code.
>=20
>Miguel Angel told me the other day that they first decompress the
>segmented files, and then serve them to through their homebuild MSG ADDE
>server. This would however increase our storage requirements
>significantly.
>=20
>The example data provided at
>http://www.eumetsat.de/en/dps/news/msg-1_rect_image.html#hrit
><http://www.eumetsat.de/en/dps/news/msg-1_rect_image.html#hrit>  has the
>same compression applied i believe. Please let me know your thoughts on
>this.
>=20
>Cheers,
>Valentijn
>
>________________________________
>
>From: Valentijn Venus
>Sent: Tue 8/10/2004 17:02
>To: Unidata Support
>Cc: address@hidden
>Subject: RE: 20040809: MSG-1 through ADDE (cont.)
>
>
>
>So my questions are:
>
>-Do you observe the same problem with the "Work" and "Doesn't work"
>MSG-1 datasets on your McIDAS system?
>-Are other developers of McIDAS (e.g. those who worked on the MSG part)
>aware of the problem, and possibly willing to find a solution?
>-If negative, would you be willing to adjust the MSGX bit to get it to
>work for the "Doesn't work" set of example data?
>
>
>-----Original Message-----
>From: Valentijn Venus
>Sent: Tue 8/10/2004 16:49
>To: Unidata Support
>Cc: address@hidden
>Subject: RE: 20040809: MSG-1 through ADDE (cont.)
>
>Hi Tom,
>
>i believe i've found the problem. The format we receieve is XRIT alright
>(so the MSGT argument should work), but along the way EUMETCAST has made
>a change in the segmentation. E.g. the HRV 11134 segments were devided
>over 174 files, whereas the more recent segmentation sceme devides them
>over only 24 files. You can download corresponding sets of example data
>at the following adresses
>(http://www.eumetsat.de/en/dps/news/msg-1_rect_image.html#hrit):
>
>Works:
>http://oiswww.eumetsat.org/WEBOPS/msg_images/msg1_hrit_c12.tar.bz2 (7.3
>mb)
>Doesn't work:
>http://oiswww.eumetsat.org/WEBOPS/msg_images/20030509/msg1_hrit_ch12.tar
>.bz2 (40.3 mb)
>
>
>The McIDAS commands below seem to work with the "Works" dataset:
>DSSERVE ADD MSG/HRV MSGT TYPE=3DIMAGE
>DIRFILE=3D/home/mcidas/data/msg/*/*/*/H*{HRV,EPI,PRO}* "High resolution
>visible channel (band 12).
>DSSERVE ADD MSG/VISIR MSGT TYPE=3DIMAGE
>DIRFILE=3D/home/mcidas/data/msg/*/*/*/H*MSG1*{VIS,IR,WV,EPI,PRO}* "High
>resolution visible and infrared channel (bands 1-11).
>DSSERVE ADD MSG/ADMIN TEXT
>INFO=3D/home/mcidas/data/msg/2004/08/08/H*MSG1*ADMIN* "METEOSAT-8
>Administration Message.
>
>IMGDISP MSG/HRV.-1 1 MAG=3D-6 BAND=3D12
>
>Cheers! Valentijn
>
>-----Original Message-----
>From: Unidata Support [mailto:address@hidden]
>Sent: Mon 8/9/2004 22:09
>To: Valentijn Venus
>Cc: address@hidden
>Subject: 20040809: MSG-1 through ADDE (cont.)
>
>>From: "Valentijn Venus" <address@hidden>
>>Organization: ?
>>Keywords: 200408050006.i7506ZaW006360 McIDAS MSG
>
>Hi Valentijn,
>
>Well, after three abstracts and a LOT of other stuff, I finally have
>time
>to get to your inquiry.  Sorry for the wait...
>
>>another message to keep you up-to-date on my  latest developments. at
>the
>>moment the ADDE server is working well using a sample of MSG-1 images
>>provided by EUMETSAT.
>
>Which format are you using?  The McIDAS-X distribution supports two
>different file types for MSG data through two different servers:
>
>msgtadir.cp -- XRIT
>msgtaget.cp -- XRIT
>
>msgxadir.cp -- ?? (native?)
>msgxaget.cp -- ?? (native?)
>
>>The HRV data (band 12) are unencrypted and in Native format, and
>>2003/12/12 13.30 consists of 176 segmented files (excl. PRO and PERI
>>metadatailes).
>
>>A similair scene as we receive it contain only 24 (much bigger)
>segments
>>(filenaming looks similair), and when i try to serve this McIDAS tells
>me
>>tere are no bands/data in the group.
>>
>>Have you ever encountered this problem?
>
>No, I havn't.  I must add that I have not played around with the MSG
>data all that much, so my not having experienced the problem you are
>seeing means little or nothing.
>
>>From: "Valentijn Venus" <address@hidden>
>>Date: Thu, 5 Aug 2004 23:46:32 +0200
>
>>Btw, how is McIDAS compiling in 64-bit for an AMD64 system?
>
>I built McIDAS on our dual Opteron 64-bit machine back in late
>June/early
>July and had no problems that I can recall.  We have not keep this build
>around for testing since we have been experimenting with different OSes
>on the box (Fedora Core 1 for 64 bit; FreeBSD 5.2).
>
>>From: Valentijn Venus
>>Sent: Wed 8/4/2004 23:36
>
>>If i understood well the 2004 McIDAS upgrade will read the raw,
>transmitted
>>METEOSAT 8 files. These we receive from EUMETSAT by a satellite dish
>mounted
>>on the top of our roof.
>
>Is this through EUMETCAST?
>
>>The server runs on a  Windows XP OS and stores the
>>segmented files received through a DVB system on a shared drive.
>
>OK.  I have had two meeting with Volker Gartner, Head of EUMETSAT User
>Services, about setting up an ADDE server for the METEOSAT data at
>EUMETSAT.  The idea is to get the folks at EUMETSAT using remote access
>methods first, then get the member state's researchers interested, and
>then see if the service could be opened up.  There are many steps and
>pitfalls between the first effort and the goal, but I am hopeful that
>we can make progress.
>
>>since i
>>won't be able to participate in this years Workshop, i'm looking for
>some
>>documentation/guidance on how to set up an ADDE server for MSG-1 for
>internal
>>research/educational use.
>
>I think another of your emails shows that you got past this point even
>though your earlier email showed a bit of a misunderstanding:
>
>>>Just added a couple of EUMESAT MSG-1 images using:
>
>>>DDSERVE MSG/HRV MSGT TYPE=3DIMAGE
>DIRFILE=3D'/home/mcidas/data/H*MSG1*HRV*2004080112
>00*'
>
>OK, this creates the dataset with group name MSG and descriptor HRV.
>
>>>McIDAS tells me the command completed succesfully. The i added the
>ADDE to
>>>the client routing table using:
>
>>>DATALOC ADD MSG_LOC ADDESERVER.ITC.NL
>
>Two things:
>
>- if you created the dataset locally, then you don't have to run DATALOC
>  to locate it -- unless -- you want to access it through the remote
>  ADDE server interface
>
>- since the dataset you created has group name MSG, the DATALOC
>  command would be:
>
>DATALOC ADD MSG ADDESERVER.ITC.NL
>
>  This also assumes that the fully qualified hostname of the ADDE
>  server machine is ADDESERVER.ITC.NL.
>
>>>Then i tried to display the data using:
>
>>>IMGDISP MSG_LOC/HRV.-1 1 MAG=3D-6 BAND=3D4
>>>or
>>>IMGDISP MSG_LOC/FD
>
>The first thing to do is verify that the ADDE group name is recognized
>by the server:
>
>DSINFO IMAGE MSG_LOC
>
>If these return nothing, then you are asking for the wrong ADDE dataset
>group.
>
>>I could send you some sample files if that helps...
>
>At some point, I wouldn't mind getting sets of example data that works
>and doesn't work so I can better understand the formats.
>
>>I also came across a message in the archive that mentioned a way to
>>calculate satellite and sun angles for METEOSAT data (by Miguel Angel).
>>I would very much appreciate it for some radiative transfer
>modelling....
>
>Yes.  I believe that he sent along the program in his email.  If
>he didn't, it might be best to contact him directly to get his latest
>version of the code.
>
>>>Also an updated DATALOC listing (ADDELIST.TXT) would be very welcome.
>
>What would be more useful is the URL for the list of Publically
>Accessible
>ADDE servers:
>
>http://my.unidata.ucar.edu/content/software/mcidas/adde_servers.html
>
>Sending you the contents of ADDESITE.TXT would not give you a complete
>listing of what servers are out there.
>
>Cheers,
>
>Tom
>--
>************************************************************************
>Unidata User Support                                    UCAR Unidata
>(303)497-8643                                                  P.O. Box
>address@hidden                                   Boulder, CO
>------------------------------------------------------------------------
>Unidata WWW Service
>------------------------------------------------------------------------
>NOTE: All email exchanges with Unidata User Support are recorded in the
>Unidata inquiry tracking system and then made publically available
>through the web.  If you do not want to have your interactions made
>available in this way, you must let us know in each email you send to
>us.
>
>
>
>
>
>
>------_=_NextPart_001_01C483A5.B1EB5D89
>Content-Type: text/x-vcard;
>       name="Valentijn Venus.vcf"
>Content-Transfer-Encoding: base64
>Content-Description: Valentijn Venus.vcf
>Content-Disposition: attachment;
>       filename="Valentijn Venus.vcf"
>
>QkVHSU46VkNBUkQNClZFUlNJT046Mi4xDQpOOlZlbnVzO1ZhbGVudGlqbg0KRk46VmFsZW50aWpu
>IFZlbnVzDQpPUkc6O05SUw0KVElUTEU6TS5TYy4NClRFTDtXT1JLO1ZPSUNFOiszMSAoMCk1MyA0
>ODc0NTQ5DQpBRFI7V09SSzo7NC0xMzQ7UC5PLiBCb3ggNjtFbnNjaGVkZTs7NzUwMCBBQQ0KTEFC
>RUw7V09SSztFTkNPRElORz1RVU9URUQtUFJJTlRBQkxFOjQtMTM0PTBEPTBBUC5PLiBCb3ggNj0w
>RD0wQUVuc2NoZWRlIDc1MDAgQUENCkVNQUlMO1BSRUY7SU5URVJORVQ6dmVudXNAaXRjLm5sDQpS
>RVY6MjAwNDA1MjVUMDgzOTI2Wg0KRU5EOlZDQVJEDQo=
>
>------_=_NextPart_001_01C483A5.B1EB5D89--
--
+-----------------------------------------------------------------------------+
* Tom Yoksas                                             UCAR Unidata Program *
* (303) 497-8642 (last resort)                                  P.O. Box 3000 *
* address@hidden                                   Boulder, CO 80307 *
* Unidata WWW Service                             http://www.unidata.ucar.edu/*
+-----------------------------------------------------------------------------+