20010718: HDF to AREA (cont.)

>From: Darren Gallant <address@hidden>
>Organization: UCAR/JOSS
>Keywords: 200107091620.f69GKI102721 McIDAS HDF AREA

Darren,

>I have consulted the docments you mentioned. If I created a pre-calibrated
>WV areafile and a raw WV areafile, would the resulting images be the same?

No.

>Is any information lost by creating a pre-calibrated image? 

Yes.  The difference is basically a quantization of data values in the
1-byte image.  The displays of the two images would be the same (since
the McIDAS image display is 8-bit), but math using the images would
probably be different.

I just checked the GOES-10 10.7 um AREAs that MSFC is creating (from
HDF files, I believe) and I see that they do NOT have calibration blocks
and store raw values:

imglist.k MSFC/GW-IR FORM=EXP
 Pos Satellite/         Date       Time      Center      Res (km)   Image_Size
     sensor                                 Lat  Lon    Lat   Lon
 --- -------------  ------------  --------  ---- ----  ----- ----- ------------
   3  G-10 IMG      18 JUL 01199  19:46:00    33  133
    Band: 4  10.7 um Surface temp; longwave window      5.31  2.37   983 x 2856
     proj:    0 created: 2001199 201543  memo:
     type:GVAR     cal type:RAW
     offsets:  data=    2816 navigation=  256 calibration=    0 auxillary=    0
     doc length:   0   cal length:   0   lev length:   0 PREFIX=   0
     valcod:          0 zcor:  1 avg-smp: A
     lcor: 2645  ecor: 10053  bytes per pixel: 2  ss: 74
     Resolution Factors (base=1):   Line=    4.0   Element=    4.0
imglist.k: done

lwu.k LIST G10-200107181830.ir4.ara 0 64
       0.          0           4 HEX:        0        4 ASCII:
       2.         74      101199 HEX:       4A    18B4F ASCII:    J    O
       4.     183000        2525 HEX:    2CAD8      9DD ASCII:
       6.       7845           1 HEX:     1EA5        1 ASCII:
       8.       1357        3312 HEX:      54D      CF0 ASCII:    M
      10.          2           4 HEX:        2        4 ASCII:
      12.          4           1 HEX:        4        1 ASCII:
      14.          0           0 HEX:        0        0 ASCII:
      16.     101199      195613 HEX:    18B4F    2FC1D ASCII:    O
      18.          8           0 HEX:        8        0 ASCII:
      20.          0           0 HEX:        0        0 ASCII:
      22.          0           0 HEX:        0        0 ASCII:
      24.          0           0 HEX:        0        0 ASCII:
      26.          0           0 HEX:        0        0 ASCII:
      28.          0           0 HEX:        0        0 ASCII:
      30.          0           0 HEX:        0        0 ASCII:
      32.          0        2816 HEX:        0      B00 ASCII:
      34.        256           0 HEX:      100        0 ASCII:
      36.          0           0 HEX:        0        0 ASCII:
      38.          0           0 HEX:        0        0 ASCII:
      40.          0           0 HEX:        0        0 ASCII:
      42.          0           0 HEX:        0        0 ASCII:
      44.          0           0 HEX:        0        0 ASCII:
      46.          0           0 HEX:        0        0 ASCII:
      48.          0           0 HEX:        0        0 ASCII:
      50.          0  1196835154 HEX:        0 47564152 ASCII:      GVAR
      52. 1380013856           0 HEX: 52415720        0 ASCII: RAW
      54.          0           0 HEX:        0        0 ASCII:
      56.          0           0 HEX:        0        0 ASCII:
      58.          0           0 HEX:        0        0 ASCII:
      60.          0           0 HEX:        0        0 ASCII:
      62.          0           1 HEX:        0        1 ASCII:
      64. 1196835154           0 HEX: 47564152        0 ASCII: GVAR
 --END OF LISTING

Furthermore, the result from IMGPROBE shows the calibrated value of
the 2-byte counts in image:

<output from IMGPROBE>

 Frame    Latitude    Longitude    Tvline    Tvelem    Line    Elem
    11    37:48:35    118:55:47      202       640     4253   17721
 
     Image Name           Day      Nominal Time   Scan Time    Band
  ----------------      -------    ------------   ---------    ----
  MSFC/GW-IR.3        18 Jul 01199   19:46:00      MISSING       4

                        File     Nominal  Image     RAW     RAD     TEMP    BRIT
       Lat/Lon      Line/Element  Line/Element               *       K
 37:48:35 / 118:55:47  402/ 1917    4253/17721     20928    122.08 306.50     48
 *   milliwatts/meter**2/steradian/(cm-1)

IMGPROBE: Done

So, it looks as if there is an interpretation of 2-byte counts to
things like Temperature for GVAR IR images even if a calibration block
does not exist.  I am suprised by this, but stand corrected.

I first figured that the temperature must be calculated after scaling
the 2-byte raw values to 1-byte brightnesses.  This, however, gives a
TEMP of 306 K, not 306.5 like is listed above.  Further reading of
area2.doc did not shed additional light on what is going on, so I
looked at the GVAR calibration module code, kbxgvar.dlm and see that
there is a calculation for Tempertature based on radiance which is
calculated (actually, a lookup table is generated for the range of raw
values and then temperatures are interpolated).

All of this seems to mean that you should create your AREAs with the
2-byte data and then _assume_ that the standard calculations apply.  I
believe that this is what Anthony G. meant by (the last sentence is the
operative one):

> Yes, we do have an HDF to McIDAS converter. It should in theory work for most
> datasets, except with the caveat that it expects navigation data to be in a
> separate file and different format (ascii). And as for calibration - it
> assumes it is GOES-8 so the cal. is internal to McIDAS so we don't do anything
> with that.

I hope that this helps...

Tom