[McIDAS #USW-595984]: 20200416: GLM imagery / McIDAS / CoD processing

Hi Mike,

re:
> I rebuilt the abin servers using the v2020a files (I think they may have
> been out of date before).  At that point I can confirm your results with
> SSEC's server (both IMGLIST and IMGDISP)... but not with local data still.

Hmm...

re:
> I run the following:
> DSSERVE ADD NPGOESRL/GLMGE2 ABIN TYPE=IMAGE 
> DIRFILE=/home/data/goes/GOES16/FullDisk/glm_vlab/Level2/*.nc "GOES-East 
> Level2 FullDisk GLM Vlab Products
> DSSERVE ADD NPGOESRL/GLMGE3 ABIN TYPE=IMAGE 
> DIRFILE=/home/data/goes/GOES16/FullDisk/glm_vlab/Level3/*.nc "GOES-East 
> Level3 FullDisk GLM Vlab Products
> DSSERVE ADD NPGOESRL/GLMGW2 ABIN TYPE=IMAGE 
> DIRFILE=/home/data/goes/GOES17/FullDisk/glm_vlab/Level2/*.nc "GOES-West 
> Level2 FullDisk GLM Vlab Products
> DSSERVE ADD NPGOESRL/GLMGW3 ABIN TYPE=IMAGE 
> DIRFILE=/home/data/goes/GOES17/FullDisk/glm_vlab/Level3/*.nc "GOES-West 
> Level3 FullDisk GLM Vlab Products

These look correct to me!

re:
> And then:
> mcidas@atlas-old:~/mcidas2020/src$ imglist.k NPGOESRL/GLMGE2 FOR=ALL
> Image file directory listing for:NPGOESRL/GLMGE2
> Pos Satellite/         Date       Time      Center      Res (km)
> Image_Size
> sensor                                 Lat  Lon    Lat   Lon
> --- -------------  ------------  --------  ---- ----  ----- ----- ------------
> 41  GOES-16 L2    30 JUN 22181  20:28:00     0   75
> Band: 28   L2: GLM - TTU Flash Extent Density        2.02  2.00  5424 x 5424
> Band: 29   L2: GLM - TTU Flash Centroid Density      2.02  2.00  5424 x 5424
> Band: 30   L2: GLM - TTU Minimum Flash Area          2.02  2.00  5424 x 5424
> Band: 31   L2: GLM - TTU Average Flash Area          2.02  2.00  5424 x 5424
> Band: 32   L2: GLM - TTU Total Energy                2.02  2.00  5424 x 5424
> Band: 33   L2: GLM - TTU Group Extent Density        2.02  2.00  5424 x 5424
> Band: 34   L2: GLM - TTU Group Centroid Density      2.02  2.00  5424 x 5424
> Band: 35   L2: GLM - TTU Average Group Area          2.02  2.00  5424 x 5424
> Band: 36   L2: GLM - VLAB Flash Extent Density       2.02  2.00  5424 x 5424
> Band: 37   L2: GLM - VLAB Total Optical Energy       2.02  2.00  5424 x 5424
> Band: 38   L2: GLM - VLAB Minimum Flash Area         2.02  2.00  5424 x 5424
> proj:    0 created: 2022181 202800  memo: GOES Full Disk - Mode 6
> type:ABIN     cal type:RAW
> offsets:  data=    5376 navigation=  256 calibration=  768 auxiliary= 0
> doc length:   0   cal length:   0   lev length:   0 PREFIX=   0
> valcod:          0 zcor:  0 avg-smp: N
> lcor:    1  ecor:     1  bytes per pixel: 2  ss:187
> Resolution Factors (base=1):   Line=    4.0   Element=    4.0
> imglist.k: done

This looks correct as well.

re:
> mcidas@atlas-old:~/mcidas2020/src$ imglist.k NPGOESRL/GLMGE3 FOR=ALL
> Image file directory listing for:NPGOESRL/GLMGE3
> imglist.k: No images satisfy the search criteria
> imglist.k: done
> 
> Same results for both the GOES17 sets.  So Level2 is seen, but not Level3.
> And as a sanity check:
> mcidas@atlas-old:~/mcidas2020/src$ ls -lh
> /home/data/goes/GOES16/FullDisk/glm_vlab/Level3/*.nc
> -rw-rw-r-- 1 ldm ldm 256K Jun 30 19:49
> /home/data/goes/GOES16/FullDisk/glm_vlab/Level3/OR_GLM-L3-GLMF-M6_G16_s20221811943000_e20221811948000_c20221811945300.nc
> -rw-rw-r-- 1 ldm ldm 256K Jun 30 19:50
> /home/data/goes/GOES16/FullDisk/glm_vlab/Level3/OR_GLM-L3-GLMF-M6_G16_s20221811944000_e20221811949000_c20221811946300.nc
> -rw-rw-r-- 1 ldm ldm 255K Jun 30 19:51
> /home/data/goes/GOES16/FullDisk/glm_vlab/Level3/OR_GLM-L3-GLMF-M6_G16_s20221811945000_e20221811950000_c20221811947300.nc
> -rw-rw-r-- 1 ldm ldm 255K Jun 30 19:52
> /home/data/goes/GOES16/FullDisk/glm_vlab/Level3/OR_GLM-L3-GLMF-M6_G16_s20221811946000_e20221811951000_c20221811948300.nc

The next thing to try is turning on 'trace'ing to see what the server is
complaining about:

imglist.k NPGOESRL/GLMGE3 FOR=ALL TRACE=1

A file named 'trce' should be created in the first writable directory included 
in
the MCPATH definition that was in-scope when the 'imglist.k' was run.  Do a 
'less trce'
to see what the server has to say.  If there is not enough infomation in the 
abbreviated
listing, you will need to edit abinaget.cp and change the setting of DEBUG from 
'0'
to '1', '2', or '3'.  Each debug level will result in more output in the 'trce' 
file.

NB: you should delete the 'trce' file between runs where TRACE=1 is specified,
otherwise the new output will be written to the end of the file, and it will
get harder and harder to wade through the output to find the clue that you
need.

re:
> I even tried mapping that to L2GOESR/VLABGLML3FD just to replicate what
> works remotely, but no difference (wasn't expecting one though).  FWIW,
> same results when run from the GUI.

The ADDE dataset group and descriptor names are not important.  What is
important is how the set of files (DIRFILE= regular expression) is defined.

re:
> I'm going to scratch my head on this a bit more, but if nothing else jumps
> out at me I might try a full reinstall next just to make sure I didn't mess
> up my dev environment too much.

I can login and poke around to see what I can see if you like.  Recall that
I have SSH access to climate.

re:
> Oh I agree!  I was just looking for differences and tested a theory, and
> now I realize it was much more than that anyhow.

OK.

re:
> My tests here match your results.
> However, I do not seem to have the SLMVLAB SU table.  Ignoring that bit I
> was able to display though.

The stretch table is not crucial at all.  I just used it to accentuate
the display.

re:
> As a side note, on a v2020 install (not 'a') IMGLIST and IMGDISP still
> seemed to work, however the metadata for the variables was incorrect (and I
> didn't interrogate data values, just that they plotted). So a partial
> success.  This might be note-worthy because that's still farther than I'm
> getting with v2020a and local data.

v2020 should not work correctly (fully?) with the Vlab GLM products as
it did not have the needed definitions in abinparm.inc, etc.

re: providing your set of min/max values for Level2 products

> Okay, I'm down for that.  I'll work on that after I get the Vlab data
> working.  I only have operational experience with FED though, so the others
> may take a little longer.

OK, sounds good.

re: We have no resources or intentions to pick up the activity internally
especially since the Vlab GLM L2/L3 products are very usable, prehaps even
more than the TTU images. The thing that is missing (and will NOT be added)
is CONUS/PACUS coverage.

> This is exactly what I expected.  It's just nice to have that confirmed.
> :)
> 
> 
> I'll put face-to-desk on the Vlab local data a while longer.  I think a
> fresh install someplace will be my next move.

I think that the next move(s) is(are) to get some debugging output from
the ABIN AGET server.  Hopefully, this will tell us what is going on
(or not going on as the case may be).

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: USW-595984
Department: Support McIDAS
Priority: Normal
Status: Open
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.