[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #CRF-113718]: Garbage values in parallel netcdf write

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [netCDF #CRF-113718]: Garbage values in parallel netcdf write
  • Date: Wed, 12 Sep 2018 20:51:10 -0600
Sorry, I am still unsure of what you are doing
Are you doing a single call to nc_put_vara_float
with the following values:
start = (0,0,0,0,0)
count = (10,100,6,5,5)
and a chunk of memory of size
sizeof(float) * 10 * 100 * 6 * 5 *5 bytes?



> Yes that is correct, I am using parallel netcdf, create the file with 
> nc_open_par, and set the variable to collective access.
> ________________________________
> From: Unidata netCDF Support <address@hidden>
> Sent: 12 September 2018 19:45:44
> To: Alexandre Szenicer
> Cc: address@hidden
> Subject: [netCDF #CRF-113718]: Garbage values in parallel netcdf write
> 
> Let me make sure I understand what you are doing.
> When you say
> > Each core has to write a vector of size [10][25][6][5][5]
> do you mean that you are accessing the netcdf library in
> a multi-threaded way?
> 
> >
> >
> > I'm writing you with regards to an unexpected behaviour I encountered when 
> > writing a variable in parallel with netcdf.
> >
> >
> > The situation is the following. I have a 5 dimensional vector, say of 
> > dimensions [10][100][6][5][5]. I want to write it in parallel with four 
> > cores.
> >
> > Each core has to write a vector of size [10][25][6][5][5]. It is necessary 
> > in my problem that the dimension split between the processes be in second 
> > position.
> >
> > When this variable is written, it appears that there are a lot of corrupted 
> > values in the resulting netcdf file.
> >
> > The alternative, which I am using at the moment, is to loop over the first 
> > dimension, and perform write operations on 4D vectors instead. This works 
> > fine, but becomes very time consuming for larger vectors, as I need to 
> > perform many write operations.
> >
> >
> > I am not sure if this behaviour is normal (something to do with contiguity 
> > in memory) or not?
> >
> >
> > Many thanks for your help.
> >
> >
> > Best regards,
> >
> >
> > Alexandre Szenicer.
> >
> >
> >
> >
> >
> 
> =Dennis Heimbigner
> Unidata
> 
> 
> Ticket Details
> ===================
> Ticket ID: CRF-113718
> Department: Support netCDF
> Priority: Normal
> Status: Open
> ===================
> NOTE: All email exchanges with Unidata User Support are recorded in the 
> Unidata inquiry tracking system and then made publicly available through the 
> web.  If you do not want to have your interactions made available in this 
> way, you must let us know in each email you send to us.
> 
> 
> 
> 

=Dennis Heimbigner
  Unidata


Ticket Details
===================
Ticket ID: CRF-113718
Department: Support netCDF
Priority: Normal
Status: Open
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.