[netCDF #RTA-885842]: Writing several netcdf files simultaneously by multiple threads

Hello Christian,

It is possible to do what you're describing; you will need to pass a non-null 
'path' to specify the dataset filename, but other than that you will not need 
to worry about any disk I/O.

The function you are interested in is `nc_open_mem()`, and the documentation 
can be found here:

* 
http://www.unidata.ucar.edu/software/netcdf/docs/group__datasets.html#gac12fdf7579a2619b2aeb238cea2e7377

Regarding thread safety; netCDF is not thread safe, as you've discovered; this 
is on our list of upcoming features, but it is not available in the current 
release.  

Sorry I can't provide a more immediately useful solution for the second 
question, but hopefully you will be able to use nc_open_mem() for your purposes.

Thanks!

-Ward
> Hi there
> 
> I have two questions regarding the netcdf c library and would highly
> appreciate a quick answer.
> 
> We want to generate netcdf files upon user requests and deliver the file
> via a Rest-API. At the moment we do that by writing a file to disk with
> the netcdf C library and then read the file again to send it to the
> user. We would prefer directly generating the netcdf into a memory
> location or a stream, so that we do not need to do the detour via
> writing an actual file to disk. Is there a way to do that?
> Unfortunately, I couldn’t find any netcdf-write-methods that do
> not work with filenames but with streams, pointers, o.s. instead.
> 
> The second question is regarding thread safety. I read all the
> documentation about it I could find, but couldn’t find a
> solution to this specific case: We get segmentation faults when multiple
> independent files (different data, different filenames) are written by
> multiple threads at the same time (one file is written by each thread).
> I did build  HDF5 with the enable-threadsafe option and used the
> manually built HDF5 when compiling netcdf from source.
> I did not however enable parallel-IO, but if I understand it right, this
> is not necessary (or even worse) in our use case, as we do not want to
> write one single file in a multi-threaded way, but several independent
> files.
> 
> Thank you very much in advance and best regards,
> Christian Schluchter
> 
> 
> 


Ticket Details
===================
Ticket ID: RTA-885842
Department: Support netCDF
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.