Turns out the problem is missing_value, which must be in packed units:
from http://www.unidata.ucar.edu/software/netcdf-java/v4.1/javadoc/index.html
Implementation rules for missing data with scale/offset
- Always converted to a float or double type.
- _FillValue and missing_value values are always in the units of
the external (packed) data.
- If valid_range is the same type as scale_factor (actually the
wider of scale_factor and add_offset) and this is wider than the
external data, then it will be interpreted as being in the units of the
internal (unpacked) data. Otherwise it is in the units of the external
(packed) data.
- The dataType is set to float if all attributes used are float
(scale_factor, add_offset valid_min, valid_max, valid_range,
missing_data and _FillValue) otherwise the dataType is set to double
technically, valid_min and max should also be in packed units acording
to the NUG, but we relaxed that in netcdf-java to accomodate some
existing datasets that misread the manual.
also, he is using floats, and is encountering roundoff error, looking
in the debugger i see:
valid_min = -140.00045776367188
which will also cause problems.
so:
1: dont use missing_value or valid min/max unless you need them, eg
you actually have missing data.
2. missing_value must be in packed units.
3. make valid_min and valid_max a little wider than you need to
accomodate roundoff
On 3/23/2010 4:35 PM, Don Murray wrote:
John-
I'm trying to respond to the ticket that's been flopping back and forth
between netcdf-java and IDV (ticket QAW-317402). In the file
(testdata/grid/netcdf/G11.20090602.0715.nc), LONGITUDE is defined as:
int LONGITUDE(lines, samples) ;
LONGITUDE:units = "" ;
LONGITUDE:missing_value = -94.f ;
LONGITUDE:valid_min = -140.0005f ;
LONGITUDE:valid_max = -95.0994f ;
LONGITUDE:scale = 1.e-007f ;
LONGITUDE:offset = -140.0005f ;
LONGITUDE:scale_factor = 1.e-007f ;
LONGITUDE:add_offset = -140.0005f ;
The first longitude value is 0, but it comes up through the netCDF-Java
stack as a NaN. I would think that it would be 0*1E-7+-140.0005 =
-140.0005 which is the valid min and shouldn't be set to NaN. The last
latitude is also set to NaN, but there, we have 24902*.001+24.999 =
49.901 which is beyond the valid_max of 49.90084.
A bug? Is the missing value being scaled and offsetted? That would
make it -94*1E-7+-140.0005 = -140.0005094 which might be "close enough"
in EnhanceScaleMissingImpl.isMissingValue(). I would think that the
missing value would only be scaled and offset if it was narrower than
the converted data. In the toolsUI, I notice that the LONGITUDE looks
like:
float LONGITUDE(lines=505, samples=909);
:units = "";
:scale = 1.0E-7f; // float
:offset = -140.00046f; // float
:_CoordinateAxisType = "Lon";
a roundoff error for the offset? Maybe that's the problem?
Don
*************************************************************
Don Murray UCAR Unidata Program
address@hidden P.O. Box 3000
(303) 497-8628 Boulder, CO 80307
http://www.unidata.ucar.edu/staff/donm
*************************************************************
|