[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #ONT-426275]: Parallel-netcdf integration and nc_var_par_access

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [netCDF #ONT-426275]: Parallel-netcdf integration and nc_var_par_access
  • Date: Wed, 13 Mar 2013 09:38:10 -0600
Greg,

Sorry it's taken so long to respond to your bug report.

And thanks for the suggestions for how to fix the problem.  We'll be
considering which approach to use for a fix.

--Russ

> I think there is a mismatch of intent in the parallel-netcdf
> implementation of the nc_var_par_access() function.
> 
> As I read the documentation for nc_var_par_access, it allows setting of
> the parallel access mode on a variable by variable basis and that is how
> the hdf5 version is implemented.
> 
> However, looking at the parallel-netcdf implementation, it instead sets
> the access mode for the entire model and not just a particular variable.
> It also always assumes that the mode was NC_COLLECTIVE and calls
> begin_indep_data() or end_indep_data()  as shown below.
> 
> #ifdef USE_PNETCDF
> /* Handle files opened/created with parallel-netcdf library. */
> if (nc->pnetcdf_file)
> {
> if (par_access == nc->pnetcdf_access_mode)
> return NC_NOERR;
> 
> *nc->pnetcdf_access_mode = par_access;*
> if (par_access == NC_INDEPENDENT)
> returnncmpi_begin_indep_data(nc->int_ncid);
> else
> returnncmpi_end_indep_data(nc->int_ncid);
> }
> #endif /* USE_PNETCDF */
> 
> Looking at the implementation, it looks like there are some possible fixes:
> 
> 1. Use the NC_VAR_INFO_T structure to also store pnetcdf variables.
> Since this also assumes the use of groups, some additional coding
> would be needed to return only a base group for pnetcdf and some
> other issues...  Possible, but may not be easily implemented in a
> short time.
> 2. Add a "int pnetcdf_par_access[NC_MAX_VARS];" to nc4internal.h and
> set/get the parallel access setting from it.  Disadvantages are
> another array dimensioned out to NC_MAX_VARS (some of us have a
> modified netcdf with very large NC_MAX_VARS settings); however, it
> would be very quick to implement.
> 3. Document that nc_var_par_access() is not supported for
> parallel-netcdf files.
> 4. Similar to (1), but use a different structure for parallel-netcdf so
> there isn't a big overhead of unused hdf5-specific data.
> 5. Similar to (1), but store both pnetcdf_ndims and parallel_access in
> the same integer since pnetcdf_dims <= NC_MAX_VAR_DIMS (typically 8)
> and parallel_access would be 0/1
> 
> --Greg
> 
> 
> 

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: ONT-426275
Department: Support netCDF
Priority: High
Status: Closed