[netCDFJava #RGN-330338]: Different grib2 decodings between NCL and netCDF Java library

Hi Brian:

This is what Ive found so far:

file: https://aws1.pemdastech.com/tests/1420921000300.grib2

variable: CICEP_P0_L1_GLC0 aka "Categorical_Ice_Pellets_surface"

record: only has 1 record, starts at pos 498311562

a dump of the drs( section 5 ) type 3 has these values (note that type 3 
inherits type 2 and type 0 values):

Type0{referenceValue=0.0, binaryScaleFactor=0, decimalScaleFactor=0, 
numberOfBits=0, originalType=0}
Type2{splittingMethod=1, missingValueManagement=0, primaryMissingValue=0.0, 
secondaryMissingValue=0.0, numberOfGroups=0, referenceGroupWidths=4, 
bitsGroupWidths=1, referenceGroupLength=1905141, lengthIncrement=1, 
lengthLastGroup=-112001040, bitsScaledGroupLength=0}
Type3{orderSpatial=2, descriptorSpatial=1}

my code has the logic that if numberOfGroups=0, then set all data equal to 
missing value, in this case primaryMissingValue=0.0. This would explain why NCL 
gives all zeroes. 

however, my code has the additional operation that if the data is missing, set 
them to NaNs. So, it may be possible that "set to missing value" is the way 
that GRIB-2 drs=3 sets a constant field. For sure, 0.0 is a valid value, given 
that the unit description is "Categorical ice pellets (yes=1 no=0)"

Another possibility is that numberOfGroups=0 should not mean "set to missing 
values", but rather "this is a constant field, set to the reference value", 
which is also 0.0. Im leaning towards this at the moment.

I will investigate more tommorrow.

John


> Thanks John,
> Here's a link to the HRRR forecast model output grib2 (500MB) file:
> https://aws1.pemdastech.com/tests/1420921000300.grib2
> 
> When I decode the file using NCL or wgrib2 I get "0" for the following fields 
> (NCL names):
> CICEP_P0_L1_GLC0
> CFRZR_P0_L1_GLC0
> 
> I also notice that I get the same issue for some lower sigma levels of cloud 
> cover(NCL names):
> TCDC_P0_L100_GLC0
> 
> When I decode either of those with netCDF-Java, I get NaN.
> 
> I didn't expect to have explicitly missing data for the HRRR model output 
> and, using wgrib2, I looked at the grib2 file code table 5.5 to examine the 
> missing value management and it returned a value indicating there were no 
> explictly encoded missing values in the grib2 file.
> 
> Thanks for your help!
> ---
> Brian Griffith, Ph.D.
> Senior Research Scientist & Site Lead
> PEMDAS Technologies & Innovations
> Phone: 828-398-8000
> address@hidden<mailto:address@hidden>
> 
> [cid:3FE592C7-C041-4CEB-8A81-855478D05A7E@hsd1.co.comcast.net.]
> CONFIDENTIALITY NOTICE: This email message and all attachments transmitted 
> with it may contain legally privileged and confidential information intended 
> solely for the use of the addressee. If the reader of this message is not the 
> intended recipient, you are hereby notified that any reading, dissemination, 
> distribution, copying, or other use of this message or its attachments is  
> strictly prohibited. If you have received this message in error, please 
> notify the sender immediately and delete this message from your system. Thank 
> you.
> 
> On Aug 20, 2014, at 1:56 PM, Unidata netCDF Java Support 
> <address@hidden<mailto:address@hidden>> wrote:
> 
> Hi Brian:
> 
> Netcdf-Java typically sets missing values to NaNs, sounds like NCL does not .
> 
> If you can send me a sample file (and as much detail as needed to reproduce) 
> I can double check if anything more nefarious is going on.
> 
> John
> 
> Dear Sir,
> 
> I am decoding NOAA ESRL HRRR model output in grib2 format and have 
> encountered a conflict.
> 
> I am decoding the categorical precipitation fields:
> CRAIN_P0_L1_GLC0
> CFRZR_P0_L1_GLC0
> CICEP_P0_L1_GLC0
> CSNOW_P0_L1_GLC0
> 
> In particular, I am decoding CFRZR and CICEP and am getting an issue that 
> when I use NCL to view the values of these parameters, I get values of "O".
> 
> When I use netCDF-Java library 4.3.x or 4.5.2 (or NASA's Panoply tool which 
> employs netCDF-Java libraries) to view the values of these parameters, I get 
> values of "NaN".
> 
> Can you explain what is happening here? Are the values in the grib2 file 
> actually missing and being decoded in two different ways? How can I determine?
> 
> Thanks,
> Brian
> ---
> Brian Griffith, Ph.D.
> Senior Research Scientist & Site Lead
> PEMDAS Technologies & Innovations
> Phone: 828-398-8000
> address@hidden<mailto:address@hidden><mailto:address@hidden>
> 
> [cid:3FE592C7-C041-4CEB-8A81-855478D05A7E@hsd1.co.comcast.net<mailto:address@hidden>.]
> CONFIDENTIALITY NOTICE: This email message and all attachments transmitted 
> with it may contain legally privileged and confidential information intended 
> solely for the use of the addressee. If the reader of this message is not the 
> intended recipient, you are hereby notified that any reading, dissemination, 
> distribution, copying, or other use of this message or its attachments is  
> strictly prohibited. If you have received this message in error, please 
> notify the sender immediately and delete this message from your system. Thank 
> you.
> 
> 
> 
> 
> 
> Ticket Details
> ===================
> Ticket ID: RGN-330338
> Department: Support netCDF Java
> Priority: Normal
> Status: Open
> 
> 
> 
> 

Ticket Details
===================
Ticket ID: RGN-330338
Department: Support netCDF Java
Priority: Normal
Status: Open