20030926: Problem generate binary netCDF with ncgen

Loreyna Yeung,

> To: address@hidden
> From: Loreyna Yeung <address@hidden>
> Subject: Problem generate binary netCDF with ncgen
> Organization: NASA Langley Research Center
> Keywords: netCDF large file

The above message contained the following:

> I am currently using the pre-compiled version 3.5 netCDF libraries
> running on IRIX 6.x. I followed the CDL format and generated a 6.7G CDL
> data file using a C program.

You used a C program to generate a 6.7 gigabyte CDL (i.e., text) file?
If so, then you should know that most netCDF users would use the C
program to generate the binary netCDF file directly -- that's what the
netCDF library is for.

> When I tried to use ncgen utility to
> generate a binary format for this large data set, an error message said
> "file.cdl   line10221: out of memory" How can I fix this problem?

If the above assumption is correct, can you modify the program to
generate the netCDF file directly?

> Do I
> need to recompile the libraries and utilities with LFS (Large File
> Support) with the source? Thank you for your help.

I believe that you're running into a limitation within the ncgen(1)
utility that is not related to large-file support but, rather, deals
with excessive memory usage.

> below is my CDL file:
> 
> netCDF  MOLA {
> dimensions:
>         lat = unlimited; // did not work with value 23042
>         lon = 46082;
> variables:
>        float    P(lat, lon);
>                  P: name = "Topographic height";
>                  P: units = "meters";
> //global attributes
>                  :history = "Topographic data for 1/128 degree per pixel
> resolution";
>                  :title = "MOLA topographic data";
> 
> data:  ALL THE  DATA ARE HERE in 23042 x 46082 table format

The data lines might be too long for ncgen(1) to handle.

> 
> }
> 
> Loreyna Yeung
> Swales Aerospace
> NASA Langley Research Center
> Hampton, VA

Regards,
Steve Emmerson