Re: 20040330:unaligned access pid=##### va =0x

>To: <address@hidden>,
>To: <address@hidden>
>From: "Joseph Benckert" <address@hidden>
>Subject: Re: 20040330:unaligned access pid=##### va =0xnetcdf troubles
>Organization: UCAR/Unidata
>Keywords: 200403291443.i2TEhsrV014206

Joseph,

> > Yes, there are a couple of examples in the "Large File Support"
> > section of the F90 User's Guide (which I've got to get into the C and
> > Fortran-77 User's Guides soon):
> >
> >
> http://www.unidata.ucar.edu/packages/netcdf/f90/Documentation/f90-html-docs/guide9.html#2236524
> >
> > The second example shows how to use record variables to get lots of
> > data in a file, but the first example is only relevant for one very
> > large variable.
> 
> Yes, I've read this.  This is CDL right?  I wanted to see an example
> "program" using unlimited dimensions and large file support, especailly
> after reading the warning about creating at least one record before actually
> writing one....

Yes, it's CDL.  If you have a file foo.cdl, you can get an example
C program that generates the corresponding binary netCDF file by using
the "ncgen" utility, as in :

  ncgen -c foo.cdl > foo.c

and similarly for Fortran (although the generated Fortran programs are
convoluted if the unlimited dimension is used, in order to only make
one pass over the CDL).

I've appended a C example that creates a 3 GB file, "big.nc", which
has been modified a bit from the output of ncgen.  This example only
has one record variable, x(r, n), but I think you can try it with more
variables.  No guarantees that this example even compiles, but I think
I remember using this to test on large file systems.

--Russ

#include <stdio.h>
#include <stdlib.h>
#include <netcdf.h>

void
check_err(const int stat, const int line, const char *file) {
    if (stat != NC_NOERR) {
           (void) fprintf(stderr, "line %d of %s: %s\n", line, file, 
nc_strerror(stat));
        exit(1);
    }
}

#define NFIXED 1000000
#define NRECS  750 /* 100 -> 400 MB, 500 -> 2GB, 750 -> 3GB */

int
main() {                        /* create big.nc */

   int  ncid;                   /* netCDF id */

   /* dimension ids */
   int n_dim;
   int r_dim;

   /* dimension lengths */
   size_t n_len = NFIXED;
   size_t r_len = NC_UNLIMITED;

   /* variable ids */
   int x_id;

   /* rank (number of dimensions) for each variable */
#  define RANK_x 2

   /* variable shapes */
   int x_dims[RANK_x];

   /* enter define mode */
   int stat = nc_create("big.nc", NC_CLOBBER, &ncid);
   check_err(stat,__LINE__,__FILE__);

   /* define dimensions */
   stat = nc_def_dim(ncid, "n", n_len, &n_dim);
   check_err(stat,__LINE__,__FILE__);
   stat = nc_def_dim(ncid, "r", r_len, &r_dim);
   check_err(stat,__LINE__,__FILE__);

   /* define variables */

   x_dims[0] = r_dim;
   x_dims[1] = n_dim;
   stat = nc_def_var(ncid, "x", NC_FLOAT, RANK_x, x_dims, &x_id);
   check_err(stat,__LINE__,__FILE__);

   /* leave define mode */
   stat = nc_enddef (ncid);
   check_err(stat,__LINE__,__FILE__);

   {                    /* store x */
    static size_t x_start[RANK_x];
    static size_t x_count[RANK_x];
    float x[NFIXED];
    int i;
    int r;

    r_len = NRECS;                      /* number of records of x data */
    
    for (i=0; i<NFIXED; i++) {
      x[i] = i;
    }    
    
    for (r = 0; r < NRECS; r++) {
      x_start[0] = r;
      x_start[1] = 0;
      x_count[0] = 1;
      x_count[1] = n_len;
      
      stat = nc_put_vara_float(ncid, x_id, x_start, x_count, x);
      check_err(stat,__LINE__,__FILE__);
    }
   }
   stat = nc_close(ncid);
   check_err(stat,__LINE__,__FILE__);
   return 0;
}