[Datastream #LLB-351971]: Question about FNEXRAD datasets

Hi Munsung,

re:
> - what software are you using to extract calibrated values from the
>   N0R and DHR images?
> 
> I looked at the map (DHR) with two ways:
> 
> 1) using NOAA's weather and climate toolkit and
> 2) OPeNDAP Dataset in Unidata THREDDS Data server
> 
> (For OPeNDAP: 
> http://thredds.ucar.edu/thredds/dodsC/nexrad/composite/gini/dhr/1km/20180206/Level3_Composite_dhr_1km_20180206_2120.gini.ascii?Reflectivity[0:1:0][1000:1:2999][2000:1:4000])

It is not surprising to me that NOAA's weather and climate toolkit does not
work/work correctly with our NEXRAD Level III national composites since the
image portion of the files are PNG compressed.  I am a bit surprised that
the TDS did not work ** especially if it works for the N0R composites since
all of the composites use PNG compression on the image portion of the files **.

Question:

- have you been using the TDS for access to the N0R composites?

re:
> The unreasonable values which I mentioned is like -204749.36 and -272418.47, 
> and the
> range of reflectivity values in the map is about from -375940 to -204750. 
> Those values
> are obviously wrong.

Yes indeed.  This is undoubtedly a result of the package not unPNG compressing
the image values.

re:
> That's why I was thinking that the values are converted from dBZ values with 
> certain
> offset and scale parameters.

The PNG compression of the image portion of the files should explain everything.

re:
> Also, I tried to read the gini file by using Metpy python package, but it 
> didn't work
> with the following error message.
> 
> (Navigation/Calibration unhandled: -128
> Leftover unprocessed data beyond EOF marker: PNG)

This indicates that the PNG compression is not being handled by the MetPy code.

re:
> I will try to use McIDAS and check it.
> 
> Now, however, I wonder if there is a way to read the DHR national composite 
> data
> provided with the gini format in python.

The first thing I would try is running the images through the ldm-mcidas
'pngg2gini' "decoder" as this will unPNG compress the image portion of the
image files, and then try using MetPy on the uncompressed images.

You can download the ldm-mcidas decoders in source form from our anonymous
FTP server as follows:

machine:    ftp.unidata.ucar.edu
user:       anonymous
pass:       your email address
directory:  pub/ldm-mcidas
file:       ldm-mcidas-2012.tar.gz

If you are running your scripting in Linux, I can cut a binary distribution
that should work for you and put it out in the ldm-mcidas area of our
anonymous FTP.  I offer this since a number of users have had a problem
building the ldm-mcidas decoders from source (even though I never seem
to).  If you are interested in this option, please give me the particulars
for your environment (which version of Linux; 32-bit or 64-bit) I will try
to get to it later today.

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: LLB-351971
Department: Support Datastream
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.