Re: 20001129: netCDF offset limitations

>From: "maseland j.e.j." <address@hidden>
>Subject: netCDF offset limitations
>Organization: National Aerospace Lab. NLR
>Keywords: 200011291622.eATGMro00850 netCDF 3.4 size offset limitations

Hi Hans,

> we encounter read errors for netCDF Version 3.4 files larger than 2.1 
> Gbyte for SUPER UX-10.2.  We are aware of the offset limitations for 
> netCDF file containing fixed length variables. However the formulation
> of the limitations is in my view ambiguous:
>
> "On systems that support very large files (exceeding 2 Gbytes), such as
>  IRIX 6.x or SunOS 5.6 with the large file compilation environment, it
>  is possible to create and access very large netCDF files with version
>  3.4.  The remaining size constraints are that the file offset to the
>  beginning of the record variables (if any) must be less than 2 Gbytes,
>  and the relative offset to the start of each fixed length variable or
>  each record variable within a record must be less than 2 Gbytes."
>
> My confusion originates from the word "relative". Does relative mean here
> with respect to the start of the previous date part (visualised as
> method 1). 
> Or is the offset to the position of the start of fixed-size data 
> part computed relative to the header (visualised as method 2)? 
>
>          Method 1:                        Method 2:
>        ---------------                  -------------
>    --->|  header     |                  |  header   |<---------
>    |   ---------------                  -------------  |  |   |
>    --->|             |                  |           |<--  |   |
>    |   |  data1      |                  |  data1    |     |   |
>    |   |             |                  |           |     |   |
>    |   ---------------                  -------------     |   |
>    --->|             |                  |           |<-----   |
>    |   |  data2      |                  |  data2    |         |
>    |   |             |                  |           |         |
>    |   ---------------                  -------------         |
>    --->|             |                  |           |<---------
>        |  data3      |                  |  data3    |
>        |             |                  |           |
>        ---------------                  -------------
> In case of method 1, the offset value for each fixed size variable
> must be smaller than 2.1 Gbyte.
> In case of method 2, the offset value for the last fixed size variable
> must be smaller than 2.1 Gbtye.
>
> To which method is referred to in your text?

Method 2 was the intended interpretation.  Sorry for the confusion,
I'll try to make this clearer in the next revision.

I wish it were easy to remove this limitation, but it requires a
change to the format, which for netCDF would be a very big change.

--Russ

_____________________________________________________________________

Russ Rew                                         UCAR Unidata Program
address@hidden                     http://www.unidata.ucar.edu