[netCDF #UUS-259413]: pnetcdf and netcdf

> Russ, we are using pnetcdf from ANL. We are producing data in parallel, and 
> using using pnetcdf to perform parallel writes. But we have analysis tools 
> that read the data serially, and are currently linked with netCDF-4. Early on 
> we tried parallel netCDF-4 for the writes and ran into some performance 
> issues, but this was a while ago.
> 
> It gets very confusing keeping straight ANL's pnetcdf package, which I think 
> is now referred to as "parallel-netcdf", and Unidata's netCDF with support 
> for parallel I/O. Am I correct in thinking that the Unidata netCDF library 
> provides two different paths to parallel I/O: one that uses ANL's pnetcdf 
> under the hood, and another one that sits directly on top of MPI I/O? I 
> believe our earlier Unidata netCDF parallel tests were done with MPI I/O 
> because the file was opened with:
> 
> nc_create_par(path.c_str(), NC_MPIIO | NC_NETCDF4, _IO_Comm, info, &ncid);
> 
> not with
> 
> nc_create_par(file_name, NC_PNETCDF, comm, info, &ncid)

Yes, the Unidata's netCDF-4 provides two paths to parallel I/O:

  1. For netCDF-4 files that the use HDF5 storage layer with the first 
nc_create_par() call above, 
     the parallel I/O is just a layer over HDF5 parallel I/O, which uses MPI 
I/O.

  2. For netCDF classic format or 64-bit offset format files that use the 
second nc_create_par() 
     call above, the parallel I/O uses pnetcdf aka parallel-netcdf, but 
provides the same API
     (and thus API parameter types) as in the serial netCDF API.  This 
restricts it to arrays of
     primitive types for data, rather than the structured MPI types of data 
that pnetcdf can also 
     access, but makes it easier to convert a serial netCDF program to use 
parallel I/O.

> In any event, maybe I'm not understanding but it would seem that if we used 
> the NC_PNETCDF flag we would also have to turn on NC_64BIT_DATA and would get 
> a pnetcdf generated file that would be in CDF-5 format. So this would't help 
> with the Unidata netCDF readback problem.

Right, I understand that now.

> >
> > Ed Hartnett did that integration before he left, and we don't know much 
> > more about
> > it, but I'm just integrating some pnetcdf fixes from Greg Sjaardema at 
> > Sandia into
> > our trunk for a netcdf-4.3 release, so I'm trying to get the pnetcdf tests 
> > working.
> >
> > On the subject of CDF-5, Rob Latham contributed the code needed to 
> > integrate support
> > for CDF-5 into netCDF-4, as described in this Jira ticket, but the work 
> > hasn't been
> > done yet:
> >
> >  https://www.unidata.ucar.edu/jira/browse/NCF-163
> >
> > It would require that we also change netCDF-Java to read (and eventually 
> > write) the
> > new forat as well.  I've discussed that with John Caron, and he thought it 
> > wouldn't
> > require too much work, so we should eventually do it.  We're a little short 
> > of resources
> > now, with only two of us working on netCDF C and Fortran, but I suspect it 
> > won't get
> > any easier to integrate Rob Latham's changes into netCDF as we continue to 
> > refactor it,
> > though recent work on the dispatch layer might actually make it easier.
> >
> > If it would be of any use to you, I could pass on the CDF-5 patches ...
> 
> I'd like to at least take a look at the patches to see how substantial the 
> changes are.

OK, I'm forwarding the patches in a separate email.

--Russ

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: UUS-259413
Department: Support netCDF
Priority: Normal
Status: Closed