[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #BUM-177161]: Streaming netCDF4 files

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [netCDF #BUM-177161]: Streaming netCDF4 files
  • Date: Tue, 29 Jun 2021 16:50:11 -0600
Hi James,

We don't really provide a mechanism for this; the closest I can think of might 
be to use 'nc_open_mem' which takes a pointer to a memory location; once 
opened, this can be read by software linked against libnetcdf.  This only works 
for reading, however.  You can disable fill values when writing the file, so 
perhaps that would result in a file that works with the process you've 
described?

-Ward

> Hello,
> 
> Do you know how to stream data as it’s written to a netCDF4 file? That is, I 
> want to write data to a stream so that a process can read from that stream 
> and write the bytes to disk. The results should be identical to netcdf4 file 
> ‘written normally.’ One way to do this _might_ be to write data to a tmp file 
> and stream the data from the tmp file using a second thread or process. 
> However, I looked at this and it seems that the netCDF API writes fill values 
> into the file, so looking at growth of the file size results in a streamed 
> file with all the right metadata, but fill values for data.
> 
> Thanks,
> James
> --
> James Gallagher
> address@hidden
> 
> 
> 


Ticket Details
===================
Ticket ID: BUM-177161
Department: Support netCDF
Priority: Emergency
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.