This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
>To: address@hidden >cc: address@hidden >From: Jim Edwards <address@hidden> >Subject: netcdf-4 >Organization: UCAR/Unidata >Keywords: 200505041602.j44G2eP3008413 Hi Jim, > You are probably aware that NCAR has a single rack bluegene rack on > the floor. One of the main applications running on this system is > the SCD/CSS developed homme <http://www.homme.ucar.edu/> model. > This model currently uses an embarrassingly parallel I/O model with > each processor writing an individual netcdf (3) file to disk. The > files are later combined in a post processing step using a > combination of ncl and nco tools. > > How is the netcdf-4 effort going? Is it at a point where we could > consider replacing our i/o model and get rid of the postprocessing > step? Is there a possibilty of colaboration in development of an > i/o interface of homme to netcdf-4? The netCDF-4 effort is going pretty well, but we don't plan to have the first release out until late July or August. There will be one or more beta releases before that, and we already have an alpha release with parallel I/O implemented, but it is still undergoing testing and not much documentation is available for it yet. We recently got a similar question from someone at Sandia Labs who is writing out 1500 netCDF files in parallel from an application and then running a big job at the end to read them all in and combine them. Here's the advice I gave him, which is also relevant to your situation: http://my.unidata.ucar.edu/cgi-bin/getfile?file=/content/support/help/MailArchives/netcdf/msg03033.html and there were two follow-ups: http://my.unidata.ucar.edu/cgi-bin/getfile?file=/content/support/help/MailArchives/netcdf/msg03034.html http://my.unidata.ucar.edu/cgi-bin/getfile?file=/content/support/help/MailArchives/netcdf/msg03037.html In short, if you need something right now in which the interfaces and format probably won't change, I'd recommend pnetcdf 1.0.0-pre2 from http://www-unix.mcs.anl.gov/parallel-netcdf/ If you're willing to take a chance on our alpha release, which may not build easily on BlueGene, which requires first building a current snapshot of HDF5, in which the interfaces and format may change before final release, and for which the documentation is minimal, you could try our alpha3 release of netcdf-4 from ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-4 Or you might wait until our beta release or contact Greg Sjaardema to see if he's come up with a good solution yet. --Russ