[netCDFJava #MRA-346602]: NetCDF file that used to work now failing

Greetings Bob,

(comments in-line)

> Hello -
> 
> We have a NetCDF file that worked in McIDAS-V 1.5 which uses NetCDF
> version 4.3.22 and now fails in McIDAS-V 1.6 which uses NetCDF version
> 4.6.5.  The error when trying to load the file is:
> > There was an error loading the data:
> > Error creating data source:file.grid with:
> > C:\Users\rcarp\Data\UserFunctions\H8\HS_H08_20150415_0420_B14_FLDK.subset.nc
> > Grid data source failed making data set:
> > C:\Users\rcarp\Data\UserFunctions\H8\HS_H08_20150415_0420_B14_FLDK.subset.nc
> > Must specify semi_minor_axis
> 
> We created a NcML file that fixes this by adding a semi_minor_axis
> attribute to the projection variable, but we wanted to get your thoughts
> on this.
> 
> * Do you think this problem came about because of more strict rules
> with the new NetCDF version?

The problem came about when we merged a pull request from Tom Rink in 2014, 
which made
netCDF-Java's handle of Geostationary projections more general:

https://github.com/Unidata/thredds/commit/0b0b49ef0527e2a1b4d9aef7e169bfdaa9db1980

The Geostationary projection isn't defined in a standard that I know of (for 
sure not CF). The 
original code in netCDF-Java was implemented by John Caron following this 
document:

http://www.cgms-info.org/documents/pdf_cgms_03.pdf

and was originally obtained via eumetsat, but the link in the code is now dead. 
Rather than a standard,
this looks to be one way of doing a geostationary projection, and was used for 
a particular mission.

It looks like Tom's code generalized the code a bit, and there wasn't a default 
set if semi_minor_axis
was undefined (previously these were hardcoded values). Would it be reasonable 
to have a default 
value in the case that it isn't defined? Maybe the same as semi_major_axis?

> * Assuming this is a NetCDF issue, is this something that you are
> concerned with (backward compatibility with files that worked with
> previous NetCDF versions)?  Or do you feel it is more on the data
> producer to keep their files up with NetCDF standards?

As long as standards are being followed and versioned appropriately, we do our 
best to make sure
things keep working. If there is a fundamental flaw in the way the something 
was being defined
(for example, is semi_minor_axis absolutely required to get the projection 
correct?), then in an ideal
world, data providers should update the files to have the correct metadata to 
ensure all libraries 
have access to the required parameters. In this case, maybe some default values 
should have been set?
Not sure, but we should have had a test file in our test suite to detect this 
situation. Is it ok if I used the
example you sent for that purpose (again, assuming the correct thing to do is 
set a default value)?

> * Just as a reference, the original NetCDF file works in IDV 5.0
> (NetCDF 4.3.22) and fails the same way McIDAS-V currently does in
> IDV 5.1 (NetCDF 4.5.5).

Yes, 4.3.22 used the old code, which was before Tom's additions, and so set 
values of major and minor
axes were used. Were they correct? Maybe not.

Please let us know what you guys think in terms of whether or not we should 
have a default value for
the projection parameters in the Geostationary projection.

Cheers,

Sean

> 
> I posted the following files to our anonymous ftp location:
> 
> * Geographically subsetted NetCDF file that is currently failing:
> ftp://ftp.ssec.wisc.edu/pub/incoming/HS_H08_20150415_0420_B14_FLDK.subset.nc
> * NcML file that adds a semi_minor_axis attribute to the projection
> variable, which works:
> ftp://ftp.ssec.wisc.edu/pub/incoming/HS_H08_20150415_0420_B14_FLDK.ncml
> 
> These files should remain in there for a week before they are removed,
> but if you need access to them after they are removed let me know and I
> can re-post them.
> 
> Thanks -
> Bob Carp
> 
> 
> 


Ticket Details
===================
Ticket ID: MRA-346602
Department: Support netCDF Java
Priority: Normal
Status: Open
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.