[netCDF #KGN-827851]: libnetcdff ??

Howdy Charlie!

> And HOW am I supposed to know about this ??

Actually Russ intended that comment for me, not you, and you would be supposed 
to know it when I put together the email posting he discussed, and I would 
reference it. With issues such as this it is our custom to work out a solution, 
and then post a summary of the issue and its solution to the netCDF mailing 
list.

The solution is, in fact, the one you proposed, allowing the fortran code to be 
included in the C library. Since the libraries are now separated, this will 
require an extra build step for those who wish to use it. It is not recommended 
practice for netCDF users at large, but only those who have lost control of 
their build systems to the extent that they can no longer support modern netCDF 
releases.

> 
> I challenge you to find where all this is properly documented on the
> netCDF web site -- you WILL NOT find it there.
> 
> You say, "mailing list" but (to quote
> <http://www.unidata.ucar.edu/support/index.html#mailinglists>)
> 
> Unidata provides topical mailing lists to enable Unidata users
> to exchange information freely on topics of mutual interest.
> These lists are provided as a service to user community and
> <STRONG> are of little or no interest to anyone outside the
> UCAR/Unidata organization.</STRONG>  [emphasis mine]
> 
> How is that at all useful to me, or to the community at large ??

Charlie, Russ and I are working very hard to resolve this issue. The netCDF 
team is not some vast building of programming drones, but three guys who also 
have other tasks that we are required to do. It seems that your issue is 
getting about as much priority as I can manage. There is no one doing support 
but the developers, and your issue is one of 47 open issues at the moment. 

Meanwhile, the netCDF-4.2 release is due in two days, and not ready. The 
Fortran and C++ releases are overdue, but are ready to go as soon as I get a 
minute to build the distribution and put them on the website. So we are not 
just sitting here, pouring snacks down our pie-holes. We are working hard to 
attempt to meet the needs of an incredibly diverse, global community of 
(usually) very nice and supportive users, many of whom, like you, are involved 
in projects of vital scientific importance.

Perhaps I am interpreting your emails incorrectly, as is prone to happen in 
email communication. (Did you really accuse me of "stealing" your time?)

> 
> Meanwhile, I (or US EPA; the Models-3 I/O API (built on top
> of netCDF and various other software) is used by the official
> EPA Eulerian atmospheric chemistry models, together with their
> emissions models and other supporting software) have several
> hundred naive users who will have to manually "fix" ten or more
> build-systems each, to deal with this deliberate upward-
> incompatibility in the link-interfaces to netCDF, since they
> are using a mix of netCDF 3.x.7y and now 4.x.y.  It will be
> a nightmare.

It is a bad idea to be using a mix of library versions, and this is true for 
any library, not just netCDF. Our new releases are fully backward code 
compatible with older releases, so all users should be using netCDF-4.1.3, 
building with --disable-netcdf-4 if they don't want the netCDF-4/HDF5 features. 

New versions of netCDF do not just contain improvements such as netCDF-4/HDF5 
interoperability, or remote data access via the built-in DAP server. They also 
include occasional (but very important) bug fixes to the classic netCDF 
library. If you really have not changed your code in 18 years, then you are 
probably using the V2 API. I happen to recall fixing a V2 bug sometime in the 
last few releases. There is also the dreaded no-fill bug. Perhaps your users 
never hit these bugs. Perhaps they do. The safe course is to upgrade to the 
latest netCDF release.

If it is really necessary that your build system handle multiple versions of 
the netCDF, or other, libraries, then you need to use a tool that is 
appropriate to the task, like autoconf/automake. These tools take the guesswork 
out of your build, and make it widely portable, with less effort than it takes 
to keep makefiles working on one machine. There is little need to accuse me of 
stealing your time when you are using plain old "make"! Problems such as those 
you are experiencing are exactly the reason that netCDF, and most other complex 
free software packages, use these more modern tools.

Models of the importance cited by you should be upgraded to use the modern 
tools. If you catch me at a conference I can explain how to do it, it's not 
hard. There are dozens of tutorials on the web as well.

> 
> Note that the ONLY failure in upwards compatibility for this library
> in the last EIGHTEEN YEARS happened in 1999, due to the changes
> in the "magic number" flags used to open files, between netCDF 2.x
> and 3.x.

On behalf of the netCDF team: you're welcome. (The new magic number in 1999 
also did not break backward code compatibility.)

Were you to upgrade to include netCDF-4 features, you would also find that 
merely changing the create mode flag will allow you to produce HDF5 files with 
the same 18-year-old code, and a few more lines would allow you to turn on 
automatic compression/uncompression of the data. All without changing a line of 
code which is writing metadata, or reading or writing data. Pretty neat 
backward compatibility, I think. We take pride in that around here.

But this is no reason that you should not upgrade your build system once every 
couple of decades. We state that we are backward compatible for code, and for 
data. *BUT* we explicitly say that you must link to a modern version of the 
library. We do not claim (and, in general, cannot claim) that you will never 
have to modify your makefiles due to changes in the netCDF libraries. We are 
only in charge of our build system, you are in charge of your build system.

Ed

Ticket Details
===================
Ticket ID: KGN-827851
Department: Support netCDF
Priority: Normal
Status: Closed