Dan Schmitt: Re: netcdf c++ class

Tom,

I'm forwarding a note from Dan Schmitt on another possible solution to the
memory-leak problem in NcFile::sync() that preserves the ability to detect
if another process adds new dimensions or variables.  If you are using this
capability, I'd appreciate it if you could eventually test this solution and
let me know if you think it would be useful enough to keep in the C++
interface.  Dan's first message is prefixed with "> > ", my reply with "> ",
and his reply to that with "".

Thanks.

--Russ

------- Forwarded Message

From: address@hidden (Dan Schmitt)
Subject: Re: netcdf c++ class
To: address@hidden (Russ Rew)
Date: Wed, 1 Feb 1995 10:54:51 -0600 (CST)

> 
> Hi Dan,
> 
> In preparing another patch file for the netCDF C++ interface, I finally got
> around to looking at the problem with NcFile::sync() you reported almost two
> months ago:
> 
> > I just hit a show stopper with NcFile::sync().
> > 
> >     for (int i = 0; i < num_dims(); i++) {
> >       dimensions[i] = new NcDim(this, i);
> >     }
> >     for (i = 0; i < num_vars(); i++) {
> >       variables[i] = new NcVar(this, i);
> >     }
> > 
> > is a memory leak (or are memory leaks).  The
> > easy fix of "delete dimensions[i];" before
> > the new hoses anyone who got an NcDim* from
> > you before (similary for the NcVars).  I think
> > the easy way out is an NcDim::sync() and an
> > NcVar::Sync() <thinking as I write>.  I'll give
> > these a shot, if you see something else
> > give me a holler.
> 
> I've think the best way to deal with this problem now is to just delete the
> two loops entirely. ..................................................

Unless its been implemented? (please please)

> .................. The sync() member function will still synchronize
> writes to disk and subsequent reads will even detect if more records have
> been added by another writer of the open file (since I don't cache this
> information in the C++ interface).  The only thing lost (besides the memory
> leaks) will be the ability to use new dimensions and variables that have
> been added by another writer to the open file.  This is better handled by
> requiring the user to close and reopen the file to see new variables and
> dimensions.  Maybe there should be an NcSchema class to make schema changes
> visible instead of trying to hide them under the interface.
> 

I have implemented syncs for var's and dims. Here is the patch (I tested
heavily for memory leaks.)  It breaks if someone else deletes a var
or dim that has been seen, but it finds things as they are added.
I use this to avoid race conditions (e.g. 2 process
are going to talk through the netcdf file, one of them writes
it, the other waits for the file to exist and opens the file
and waits for the the record dimension to get to a certain place
and then reads the file. I'd like to avoid file close/reopen
if I could.)

- ----------------------Begin netcdf.cc diffs---------------------
224,225d222
<     if (!data_mode())
<       return 0;
228,241c225,228
<     for (int i = 0; i < num_dims(); i++) {
<       if (dimensions[i]->is_valid() != 0) {
<       dimensions[i]->sync();
<       } else {
<       dimensions[i] = new NcDim(this,i);
<       }
<     }
<     for (i = 0; i < num_vars(); i++) {
<       if (variables[i]->is_valid()) {
<       variables[i]->sync();
<       } else {
<       variables[i] = new NcVar(this,i);
<       }
<     }
- ---
>     for (int i = 0; i < num_dims(); i++)
>       dimensions[i] = new NcDim(this, i);
>     for (i = 0; i < num_vars(); i++)
>       variables[i] = new NcVar(this, i);
363,376d350
< NcBool NcDim::sync(void) 
< {    
<   char nam[MAX_NC_NAME];
<   if (the_name) {
<     delete [] the_name;
<   }
<   if (the_file && ncdiminq(the_file->id(), the_id, nam, 0) != ncBad) {
<     the_name = new char[strlen(nam) + 1]; 
<     strcpy(the_name, nam);
<   } else {
<     the_name = 0;
<   }
< }
< 
934,953d773
< NcBool NcVar::sync(void)
< {
<   if (the_name) {
<     delete [] the_name;
<   }
<   if (the_cur) {
<     delete [] the_cur;
<   }
<   char nam[MAX_NC_NAME];
<   if (the_file 
<       && ncvarinq(the_file->id(), the_id, nam, 0, 0, 0, 0) != ncBad) {
<     the_name = new char[1 + strlen(nam)];
<     strcpy(the_name, nam);
<   } else {
<     the_name = 0;
<   }
<   cur_rec = 0;
<   init_cur(); 
< }
< 
- -------------------------end netcdf.cc diffs--------------------------

The diffs to the headers are not included as they seemed simple enough.

It looks like I may be doing a rpc layer for the netCDF function
in the next couple of months (open a connection to a netCDF server
rather than a file have the netCDF server do all the file I/O.)


> By the way, I just went to a two-day presentation on the C++ standard
> library and the Standard Template Library (STL) by Michael Vilot and Pete
> Becker.  I learned a lot about library implementation and especially about
> the great new STL algorithms and containers that will be part of the
> standard.  When I get time, I might try designing a set of cooperating or
> nested STL container classes as an interface to a netCDF dataset to see if
> it works.
> 

Cool.

        Dan

Dan Schmitt
Systems Analyst                                 IEEE Member #3779402

Ranching Systems Group                          Phone # (409) 845-7333
Department of Rangeland Ecology and Management  Fax   # (409) 847-9366
Texas A&M University                            Home  # (409) 846-3914
College Station Texas 77843-2126                ICBM: 30.67 N, 96.37 W



------- End of Forwarded Message