This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
> From: address@hidden [mailto:address@hidden] > Sent: Tuesday, January 25, 2000 8:17 PM > To: John Caron > Subject: missing values Hi Brian, > I'm thinking that you're a good person to address this question to > since I assume you've dealt with missing values in the applications > you're building. But feel free to pass it on if someone else is a > better resource. > > I have been reviewing the attribute conventions in the NetCDF User's > Guide (NUG) in section 8.1. There seems to be a substantial change > in how missing values are determined between version-2.3 and > version-3 of the NUG. The reason I'm writing is that I would like > to make sure that I'm interpreting the convention correctly. The material on missing values was rewritten by Harvey Davies, because he had found the version-2.3 description to be ambiguous and inadequate for use in writing a generic package. He's the developer of the FAN package <http://www.unidata.ucar.edu/packages/netcdf/software.html#FAN>, which he used as a test bed to make sure the new missing-value description was implementable. It was not intended to be an incompatible change to the version 2.3 description, but rather a clarification and further elaboration of conventions that would be useful in generic netCDF packages. > In version-2.3 there were two ways to express missing data. Either > use the scalar _FillValue attribute, or use the possibly vector > valued missing_value attribute. A generic application should treat > all the values specified by _FillValue and missing_value as missing. > The NUG assigns no interpretation to multiple missing values. > > In version-3 things seem to have changed due to interpreting all > values outside the valid range as missing, and by having the > _FillValue attribute imply a valid range if one is not specified. > According to the NUG: "Generic applications should treat values > outside the valid range as missing." This implies that missing > values are no longer a set of discrete values specified by > _FillValue and missing_value, but now the missing values cover a > range that is defined to be outside the valid range. > > Is that the correct interpretation of the current convention? It > concerns me somewhat because I don't know of any applications that > implement the convention that way. In the few applications that > I've tested, if I define _FillValue = 1.e36, and put a value of > 2.e36 into the variable, that value won't be recognized as missing. I think your interpretation of the current convention is correct. I also think you're right about it not being implemented much; as far as I know, FAN was the only package that actually implemented the complete missing value semantics as described for version 3.x. Part of the justification for the valid range additions was to permit reasonable interpretations for fractional values of an index and in-between values of a coordinate variable to imply interpolation. I can probably dig up some more notes on justification for the changes, but it will have to wait until next week. If you want to ask Harvey about it, he's also reachable at <address@hidden>. I also think it would be OK to write a generic package that didn't follow these conventions exactly, if they seem unreasonable for what you need. Since they are not widely observed, you wouldn't be making your package incompatible with everyone else ... --Russ _____________________________________________________________________ Russ Rew UCAR Unidata Program address@hidden http://www.unidata.ucar.edu