[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20040609: _LARGE_FILES in netCDF

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: 20040609: _LARGE_FILES in netCDF
  • Date: Wed, 09 Jun 2004 13:31:04 -0600
Charlie,

>Date: Tue, 08 Jun 2004 13:10:16 -0700
>From: Charlie Zender <address@hidden>
>Organization: University of California at Irvine
>To: Steve Emmerson <address@hidden>
>Subject: Re: 20040608: _LARGE_FILES in netCDF
> Keywords: 200406081710.i58HATtK011254Q netCDF large file

The above message contained the following:

> I am starting to create large files (> 2 GB) on 64-bit systems.
> I've encoutered problems (and successes) accessing them (with NCO) on
> both 32 and 64 bit systems. I'm trying to understand what's causing
> the behavior.
...
> 1. As I understand it, only the netCDF library needs to be compiled with
> _LARGE_FILES (or equivalent for other compilers/OS's).
> Application programs (e.g., NCO) which use netCDF need not be compiled
> with _LARGE_FILES as long as they link to a netCDF library that was.
> Is this correct?

I believe that's correct -- assuming that the application, itself,
doesn't try to access a "large" file.

> 2. If netCDF is compiled with the large file macro on a 32bit system,
> e.g., a typical linux pentium system, then will applications on that
> system (e.g., NCO) successfully access contents of large files
> assuming that the application stays within physical memory boundaries?
> i.e., Will 32-bit systems work with 10 GB files as long as they don't
> try to pull all 10 GB into memory at once and thus exceed physical
> memory limitations?

That should work OK -- assuming that you can build a netCDF library with
large file support.

> 3. Regarding the large file flags for linux/GCC:
> 
> >    CC=/usr/bin/gcc
> >   CFLAGS='-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE'
> 
> Are these tokens specific for GCC, and thus should they be used by GCC
> on any platform?
> are they somehow generic to Linux (i.e., glibc) and 
> thus equally valid for other Linux compilers? Or are they specifically
> for GCC on Linux?

The above C macros are specific to the combination of the gcc(1) compiler
and a Linux system with support for large files.

> Any guidance appreciated,
> Charlie
> -- 
> 
> Charlie Zender, address@hidden, (949) 824-2987, Department of Earth 
> System Science, University of California, Irvine CA 92697-3100 

Regards,
Steve Emmerson