20001128: IMGREMAP: MAG= vs RES= (cont.)

>From: Jason Allard <address@hidden>
>Organization: PSU
>Keywords: 200010240345.e9O3jC425238 McIDAS-X IMGREMAP MAG RES

Jason,

>Oh, your knowledge of this software is incredible.

Thnaks for the compliment.

re: need to use AREA numbers, not ADDE dataset group/descriptor.position

>You were correct.  I had thought that this might have been a problem
>when I didn't see a .xx after the name.  Unfortunately, my solution was
>to do a DDSERVE for each individual image, instead of actually using
>the AREA name.  When I used the AREAxxxx, I got:
>
>NACOMP -- BEGIN
>  ELEMENT RESOLUTION=  8
>NACOMP -- DONE:  3005
>
>where, 3005 was the new AREA file created.  I used the IMGDISP command
>and it showed 'something' on the screen (it didn't give an error
>message, so it made something).  The only problem is that it's all
>black.

The values being stored in the resultant image are small numbers.
Using the IMAGE enhancement will most likely not stretch out the
limited range of values enough to see what is going on.

>The up side is that to display it, it requires a 'band'.  This
>was understandable since the NACOMP APPEND command created two numbers
>for each cell, one that has the number of times cumulus were detected
>and another that says how many times the test was performed.  When I
>use band 1 or 2, it displays a blank screen.

Are these the bands that are indicated from an IMGLIST ... FORM=EXP
listing?

>That's where I'm at now.
>I think the problem could be one of two things.  The thesholds I used
>might have been so narrow that no cumulus were detected... which means
>I have to broaden the thresholds.  The other is that I have to specify
>how to color code the results.  I don't know much about doing this in
>mcidas.  If mcidas is using a gray scale from 0 to 255, then of course
>I won't see the difference between 0 and 1 with my eyes.  Of course,
>that I just mentioned could be complete bull, and I don't know what I'm
>talking about.  I have to read up on that.

If the values are really only 0 and 1, then at most you will see a speckle
of dots in the display.  But first, you will have to stretch the values
so that you can see them.  There are a couple of ways of doing this:

EB - interactive black/white contrast stretch
SU - create a stretch table that you use when you load the image

>Either way, I'm excited that it's at least working this much.  I'm not
>sure if you have any idea how much you've helped me.  Thanks a lot.

I just browsed through the code in NACOMP, and I see what is probably a
show stopper:

**********************SUBROUTINE GETCOF*****************************
C
C  THIS SUBROUTINE WILL CALCULATE THE COEFFICIENTS NEEDED TO CONVERT
C  A DIGITAL BRIGHTNESS COUNT INTO ALN ALBEDO.  THE COEFFICIENTS WILL
C  BE STORED IN ARRAYS: ACONST AND BCONST.  IT IS *VERY* IMPORTANT TO
C  NOTE THAT THESE COEFFICIENTS ARE VALID ONLY FOR GOES-E VISIBLE
C  IMAGES.
C

...

C
C--- A, B, AND INOT ARE SATELLITE CONSTANTS FOR GOES-E *ONLY*
C     ALL UNITS ARE W/(M**2)
C
      DATA A/0.005393/, B/-2.67/, INOT/336.43/
C--------------------------------------------------------------------


Given that this program was written in 1992, what was then GOES-East
is no longer GOES-East.  This would mean that the values for A, B, and
INOT are not applicable to the current GOES-East platform.

This topic _has_ come up before from someone at Penn State.  The following
is a portion of an eamil exchange that I had with Jimmy Adegoke back in
June.  This should be of help to you in a couple of different ways:

o gives you Jimmy as a PSU contact
o cites web pages that probably have the coefficients that you need

You will notice that Jimmy quotes the calibration constant values for
GOES-7 (which was GOES-East in 1992), and the numbers he cites are the
same as the ones in the DATA statement above.

  >From: Jimmy Adegoke <address@hidden>
  >Organization: Penn State
  >Keywords: 200006271916.e5RJGgT14015 McIDAS GOES-8 VIS calibration
  
  Jimmy,
  
  >I'm need to find the calibration constants for converting GOES 8 VIS
  >brightness to albedo. The equation I'm using for the conversion is:
  >
  >A = (aB*B + b)/IcosE
  >
  >where   a, b and I are calibration constants
  >       B = brightness value
  >       E = solar elevation angle
  >
  >I have the calibration constants for GOES 7. They are 0.00539 W/meter sq.
  >for a, -2.67 W/meter sq. for b and 336 W/meter sq. for I, respectively.
  >Can someone kindly point me to where I can get similar constants for 
  >GOES 8 ?
  >
  >Greatful. 
  
  The following information was provided by Dejiang Han of NOAA/NESDIS to
  Craig Motell of the University of Hawaii.  It may be relevant to what
  you are looking for:
  
  > All the documents both on the GOES calibration web site
  > (http://www.oso.noaa.gov/goes/goescal.htm) and its
  > affiliated are the latest.
  > 
  > Regarding the GOES-10 imager calibration for visible channels, there are two
  > approached as follows. One approach is to apply the pre-launch calibration
  > coefficients. As you might know, Table 2 (visible-channel calibration
  > coefficients for GOES-10 imagers) on
  > http://www.nnic.noaa.gov/SOCC/vis_ch_calibration.html
  > contains the pre-launch calibration coefficients for all eight
  > imager detectors. The other approach is in-orbit star-based. For the
  > detailed information, please take a look at "GOES Imager Visible Channel
  > Responsivity Trending", which is located on
  > http://www.oso.noaa.gov/goes/visible_trending.htm.
  >
  > The way dealing with GOES-10 imager visible channel is
  > expected to be similar to those for GOES-8 and -9, although at this moment
  > there are no coefficients released due to insufficient data. However, in the
  > near future, we will release them and will keep you informed.
  
  An additional online reference that may be helpful is:
  
  http://www.nnic.noaa.gov/SOCC/gvarconv.html
  
  Please let me know if this does not get you headed in the right direction.
  
  >Jimmy Adegoke
  >Climate Program
  >Geog. Dept. & Earth System Sc.Center
  >302 Walker Building
  >The Pennsylvania State University
  >University Park PA 16802
  >e-mail: address@hidden
  >Fax: (814) 863-7943; Office Phone: (814) 865-7432

You can get information out of the Unidata McIDAS-X email archive by
visiting:

http://www.unidata.ucar.edu/glimpsedocs/ghmcidas.html

This is what I did to get the message above.  The search key I used was
'albedo'.

re: potential worry is the format of the date information and how it is
handled in NACOMP

>Would the answer to this question be in the code?  What would I look for?

Yes, the answer is in the code, but you have to be familiar with
programming in McIDAS to understand it.  I have a feeling that the date
stuff will not be a problem for you.  The calculation of the albedos
will be, however, since the constants above can not relate to the
current GOES-East unless someone at Penn State has updated them for the
current satellite.

>>   >I understand basically what it's suppose to do, and the logic
>>   >behind it.  It looks at both the VIS and IR images for a specified day
>>   >and time.  It calculated the albedo from the VIS brightness temps
>>   >(that's the main reason I wanted the program).  And then, depending
>>   >upon the user specifed VIS and IR thresholds, it'll determine whether
>>   >or not cumulus clouds are present.  It'll record the presence of
>>   >cumulus with another part of the new command, you can determine the
>>   >percentages of the data meeting the specified thresholds.
>
>Yes, I thought so, that's why I wanted to use it... unfortunately, the
>code takes the albedo it calculates and keeps going... as it is now, it
>doesn't give me the actual albedo.  I suppose it would be possible to
>re-write the code so that it stops after calculating the albedo, and
>provide that as an image.  Unfortunately, my limited knowledge almost
>prohibites me from doing that.  You don't know of any commands
>currently existing that converts the VIS brightness temp to an albedo,
>do you?

No, sorry.  Apparently, NACOMP will be useful to you if/when you
can update the A, B, and INOT values in the DATA statement (and sort
out the IB1 value).

re: changing the IB1 value from #0102 to 102 may not be good

>We were perplexed also (me more than anybody).  If the program ran, do
>you think that it's okay then?

No, not necessarily.

>Those of us over here really didn't
>know what it did... we only know that we couldn't rebuild mcidas with
>that there, so we removed it.

Unfortunately, it would take me much more time than I have right now
to fully understand NACOMP so that I could tell if this change is
OK or not.

re: it took a long time to get the code

>I have no idea other than I wasn't important enough... then again, once
>they did respond, they were helpful.

Do they know anything about the workings of the code?  If so, they may
be able to help sort out the albedo calcualtion and the IB1 mystery.

re: SSEC code author is still around

>Once again, thanks,

You are welcome.

Tom

>From address@hidden Wed Nov 29 21:40:50 2000
>Subject: Re: 20001128: IMGREMAP: MAG= vs RES= (cont.)
   
>   >Oh, your knowledge of this software is incredible.
>   
>   Thnaks for the compliment.

I was trying to think of an even stronger compliment... especially now
that you remember jimmy (I worked with him, he's gone now, more on that
below).

>   The values being stored in the resultant image are small numbers.
>   Using the IMAGE enhancement will most likely not stretch out the
>   limited range of values enough to see what is going on.

Yup, very small... each time the test is performed, 'band 2' increase
by one, 'band 1' only increases if it detects cumulus.  So, I'm going
to be dealing with numbers in the teens most likely.

>   
>   >The up side is that to display it, it requires a 'band'.  This
>   >was understandable since the NACOMP APPEND command created two numbers
>   >for each cell, one that has the number of times cumulus were detected
>   >and another that says how many times the test was performed.  When I
>   >use band 1 or 2, it displays a blank screen.
>   
>   Are these the bands that are indicated from an IMGLIST ... FORM=EXP
>   listing?

Yes... doing the IMGLIST shows two bands... but I knew it was suppose
to do that from the comments in the code and the conversation when I
got the program.  The NACOMP command creates two numbers for each
pixel... one that says the number of times cumulus were detected ('band
1') and the total number of times the test was performed ('band 2')

>   
>   >That's where I'm at now.
>   >I think the problem could be one of two things.  The thesholds I used
>   >might have been so narrow that no cumulus were detected... which means
>   >I have to broaden the thresholds.  The other is that I have to specify
>   >how to color code the results.  I don't know much about doing this in
>   >mcidas.  If mcidas is using a gray scale from 0 to 255, then of course
>   >I won't see the difference between 0 and 1 with my eyes.  Of course,
>   >that I just mentioned could be complete bull, and I don't know what I'm
>   >talking about.  I have to read up on that.

Since this time, I've discovered that the program was dumping out '0'
for every pixel in both bands... an error indeed.  I think it goes back
to the line were we removed the # symbol.  From what you said, and the
limited fortran on this end, we changed the line from:

DATA IB1/0102/, IB2/0/

to:

DATA IB1/Z'0102'/, IB2/0/

I guess that's the 'new' hexidecimal format... or something like that.
Either way, when I now do the NACOMP APPEND command, band 2 does show
the number of times the test is performed.  For example, if I perform
the test three times and use the same outarea, band 2 has 3s in the
array.  So, unless its an extremely odd coincidence, its keeping track
of the number of times the test was performed.  To get my hopes up
again, band 1 comes up with a mixture of 0s and 1s when the test was
performed once (I haven't done it more than once yet).  However, I had
streched VISMIN to 0 and VISMAX to 255 and IRMIN to 0 and IRMAX to
330... I did this just 'ensure' it would find 'cumulus'.  I now have to
come up with valid thresholds to perform the test with, and then look
to see if it detects cumulus in areas that have cumulus on the visible
image.

The problem now lies in the thresholds.  According the example in the
comments, and how the VISMIN and VISMAX are defined in the code, the
values can be between 0 and 255, however, an albedo is between 0 and 1,
or 0 and 100 if expressed as a percent, by definition.  In line 644
they multiply the albedo by 100, so, if I'm correct, the VIS range
should now be 0 to 100.  Even the IRMAX of 330 seems too high.  This,
of course, will be something I'll have to look into, if and only if the
program is indeed working.  I'm only going to believe its working when
I produce an image that has cumulus that matching the VIS image.

>   If the values are really only 0 and 1, then at most you will see a speckle
>   of dots in the display.  But first, you will have to stretch the values
>   so that you can see them.  There are a couple of ways of doing this:
>   
>   EB - interactive black/white contrast stretch
>   SU - create a stretch table that you use when you load the image

I'll look into these two ideas, thanks.  I'll have to do this shortly
so that I can visualize my results to see if they're making sense.  I
had been using:

EU make 0 10 black white


>   I just browsed through the code in NACOMP, and I see what is probably a
>   show stopper:

Finally, something I knew about... it's processes like these that have made
me realize how much I don't know.

>   
>   **********************SUBROUTINE GETCOF*****************************
>   C
>   C  THIS SUBROUTINE WILL CALCULATE THE COEFFICIENTS NEEDED TO CONVERT
>   C  A DIGITAL BRIGHTNESS COUNT INTO ALN ALBEDO.  THE COEFFICIENTS WILL
>   C  BE STORED IN ARRAYS: ACONST AND BCONST.  IT IS *VERY* IMPORTANT TO
>   C  NOTE THAT THESE COEFFICIENTS ARE VALID ONLY FOR GOES-E VISIBLE
>   C  IMAGES.
>   C
>   
>   ...
>   
>   C
>   C--- A, B, AND INOT ARE SATELLITE CONSTANTS FOR GOES-E *ONLY*
>   C     ALL UNITS ARE W/(M**2)
>   C
>         DATA A/0.005393/, B/-2.67/, INOT/336.43/
>   C--------------------------------------------------------------------
>   
>   
>   Given that this program was written in 1992, what was then GOES-East
>   is no longer GOES-East.  This would mean that the values for A, B, and
>   INOT are not applicable to the current GOES-East platform.

Yup, they're good for the GOES7 platform... and I'm currently trying
the program with data from 1991 (GOES7).

>   
>   This topic _has_ come up before from someone at Penn State.  The following
>   is a portion of an eamil exchange that I had with Jimmy Adegoke back in
>   June.  This should be of help to you in a couple of different ways:
>   
>   o gives you Jimmy as a PSU contact
>   o cites web pages that probably have the coefficients that you need
>   
>   You will notice that Jimmy quotes the calibration constant values for
>   GOES-7 (which was GOES-East in 1992), and the numbers he cites are the
>   same as the ones in the DATA statement above.

I'm amazed you remembered this.  Jimmy and I use to work together until
he left to take a job at the EROS Data Center.  He tried for a couple
of months to obtain the NACOMP program, but gave up.  He then 'reduced'
the size of his dissertation and calculated the albedo by hand for
small areas of the Midwest.  I kept trying to get the program, with
hopes that I could use it for the entire midwest.  The equation he used
to compute the albedo is the same one that the program uses (that's
where we got it).  That's why he needed the new calibration
coefficients... and so will I if the program works.  He didn't leave me
the coefficients, but with what you gave me or a quick e-mail to jimmy,
I could get those... hopefully... nothing seems as easy as it should be
these days.

>   You can get information out of the Unidata McIDAS-X email archive by
>   visiting:
>   
>   http://www.unidata.ucar.edu/glimpsedocs/ghmcidas.html
>   
>   This is what I did to get the message above.  The search key I used was
>   'albedo'.

Okay, I'll look into it.

>   
>   re: potential worry is the format of the date information and how it is
>   handled in NACOMP
>   
>   >Would the answer to this question be in the code?  What would I look for?
>   
>   Yes, the answer is in the code, but you have to be familiar with
>   programming in McIDAS to understand it.  I have a feeling that the date
>   stuff will not be a problem for you.  The calculation of the albedos
>   will be, however, since the constants above can not relate to the
>   current GOES-East unless someone at Penn State has updated them for the
>   current satellite.

It doesn't seem that it had a problem with the dates... it was that it
only wanted the area number, not the ADDE stuff (it I'm remembering
correctly what we were talking about).  The constants given in line 505
are only for GOES7, so I'll have to change those constants to the
appropriate ones for GOES8 (I'm only currently uses GOES7 and 8).  I'm
not sure if I'm confused, but all I should have to do is change those
three numbers to the ones for GOES8... and then rebuild mcidas with the
new numbers and it should run fine... and to get those, I'll read over
the e-mails from jimmy that you indicated and go to the appropriate
sites.  I'm not missing something that you're aware of, am I?

>   >Yes, I thought so, that's why I wanted to use it... unfortunately, the
>   >code takes the albedo it calculates and keeps going... as it is now, it
>   >doesn't give me the actual albedo.  I suppose it would be possible to
>   >re-write the code so that it stops after calculating the albedo, and
>   >provide that as an image.  Unfortunately, my limited knowledge almost
>   >prohibites me from doing that.  You don't know of any commands
>   >currently existing that converts the VIS brightness temp to an albedo,
>   >do you?
>   
>   No, sorry.  Apparently, NACOMP will be useful to you if/when you
>   can update the A, B, and INOT values in the DATA statement (and sort
>   out the IB1 value).

As mentioned above, I think that the coefficients for GOES8 are
available, although the e-mail from jimmy you included did mention that
10 was not available yet... and I think that I might have the IB1
problem figured out.

>>Those of us over here really didn't know what it did... we only know
>>that we couldn't rebuild mcidas with that there, so we removed it.

>   Unfortunately, it would take me much more time than I have right now to
>   fully understand NACOMP so that I could tell if his change is OK or
>   not.

That's quite alright... the information that you have provided me and
what people here have added might have made the program work.  I'm
going to have to play with it tomorrow, but it's at least looking good
at this time...

>   
>   Do they know anything about the workings of the code?  If so, they may
>   be able to help sort out the albedo calcualtion and the IB1 mystery.

Unfortunately, I don't think so... they mentioned that they only really
used it... and from the comments, I can see that it was written by
other people...  and they're nearly impossible to get ahold of.
However, as I mentioned above, it might be working... I'll just have to
figure out the appropriate thresholds and give it a go.

Thanks so much for the help,

Jason