This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
> Thanks for the response, John. Perhaps you or Russ could talk to the > NCL people and convince them to use missing_value instead of _FillValue > as their preferred attribute for missing data since NCL can read > multiple formats (including GrADS binary) and might run into the same > issues. > > Don > > On 7/29/11 2:55 PM, Unidata netCDF Java Support wrote: > > Hi Don: > > > > I think Russ and I have talked about this and decided that _FillValue shoul > d match the bit pattern exactly (for portability?). valid_range, valid_min, v > alid_max, and missing value are intended to match data values and so should b > e allowed some slop. Im cc'ing Russ in case he remembers different. The way ncdump handles it (which seems to differ from the documented requirement for exact bit-for-bit equality), is to treat any floating point value that's within one machine epsilon of the _FillValue (that is one bit in the least significant bit) as equal to _FillValue, and display it as "_". The code for this is in ncdump/dumplib.c, in the ncfloat_val_equals() and corresponding double precision functions: /* * Return ( *(float* )v1p == *(float* )v2p); * except use floating epsilon to compare very close vals as equal * and handle IEEE NaNs and infinities. */ boolean ncfloat_val_equals(const nctype_t *this, const void *v1p, const void *v2p) { float v1 = *(float* )v1p; float v2 = *(float* )v2p; if((v1 > 0.0f) != (v2 > 0.0f)) /* avoid overflow */ return false; if(isfinite(v1) && isfinite(v2)) return (absval(v1 - v2) <= absval(float_eps * v2)) ; if(isnan(v1) && isnan(v2)) return true; if(isinf(v1) && isinf(v2)) return true; return false; } It looks like I should fix the documentation to match the behavior. I think it was necessary to add the 1-bit fuzz to make the ncdump tests pass on every tested platform, but it may have been an artifact of bad compiler code that's fixed now. I may try taking it out to see if all the ncdump and ncgen tests pass without it. > > > > My own opinion is that the trouble stems from overloading _FillValue with m > ultiple meanings, plus the lack of clarity to tell people what they should do > . We should probably make another pass at > > > > http://www.unidata.ucar.edu/software/netcdf/docs/BestPractices.html Yup. --Russ > > John > > > >> John- > >> > >> In the GrADS IOSP, I added a _FillValue attribute for the GrADS Data > >> Descriptor File "undef" value: > >> > >> http://www.iges.org/grads/gadoc/descriptorfile.html#UNDEF > >> > >> I used _FillValue instead of missing_value because it seemed more > >> accepted in the CF conventions until the recent change to undeprecate > >> missing_value. Also, NCL prefers _FillValue over missing_value and I > >> was working with NCL at the time I wrote the IOSP. > >> > >> I came across a file today where the undef was set to 9.99E8, but the > >> "missing" values in the file ranged from 9.9899994E8 to 9.99E8 (We're > >> looking into why it is inconsistent - the file is written by GrADS). > >> Talking to one of the developers, they always use 32-bit values (floats) > >> for the data when working with GrADS and I'm using float for the > >> variable type. > >> > >> The logic in EnhanceScaleMissingImpl is different for _FillValue vs. > >> missing_value. isMissingValue uses ucar.nc2.util.Misc.closeEnough > >> whereas isFillValue uses straight equality. > >> > >> I can change the IOSP to have both _FillValue and missing_value > >> attributes, or change _FillValue to missing_value and I get the results > >> I expect. However, it seems like there may be some slop even in fill > >> values, so perhaps isFillValue should use Misc.closeEnough instead of > >> straight equality which also solves this problem. That seems more > >> prudent for the typically large values that are used as _FillValues. > >> > >> Thoughts? > >> > >> Don > >> -- > >> Don Murray > >> NOAA/ESRL/PSD and CIRES > >> 303-497-3596 > >> http://www.esrl.noaa.gov/psd/people/don.murray/ > >> > >> > > > > > > Ticket Details > > =================== > > Ticket ID: KJR-472449 > > Department: Support netCDF Java > > Priority: Critical > > Status: Open > > > > -- > Don Murray > NOAA/ESRL/PSD and CIRES > 303-497-3596 > http://www.esrl.noaa.gov/psd/people/don.murray/