[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 20020123: large files in NETCDF 3.4 under HPUX 11

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: Re: 20020123: large files in NETCDF 3.4 under HPUX 11
  • Date: Wed, 23 Jan 2002 13:18:01 -0700
>To: address@hidden
>From: Tim Davis <address@hidden>
>Subject: large files in NETCDF 3.4 under HPUX 11
>Organization: The Goodyear Tire & Rubber Co.
>Keywords: 200201231302.g0ND2Yx07542 netCDF 3.4 HP-UX 11.0 large files

Hi Tim,

> I work in the area of finite element structural mechanics for tire
> applications.  Our in-house analysis code uses a finite-element-specific
> data format which is built on top of NETCDF 3.4 . As our analysis needs
> have grown, we are encountering larger and larger data sets.  It is
> becoming more common for us to generate files that exceed 2 gig.
> 
> Generally, we have no difficulty with this limitation on DEC, SUN, and
> LINUX platforms.  For example, using the 'ext3' file system on LINUX, we
> build NETCDF with the additional gcc flags '-D_LARGEFILE_SOURCE
> -D_FILE_OFFSET_BITS=64' and our code can then accomodate files that
> exceed 2 gig.
> 
> Do you know of comparable flags for HPUX 11.0 ?  This is the one
> platform for which we have not been able to overcome the file size
> limitation.   Both our NFS installation (for networked file systems) and
> our HPUX 11 local file systems have been configured to accomodate the 64
> bit (rather than 32 bit) file size limitations.

We have the ansic compiler for HP-UX 11, and I thought using the
CFLAGS=+DD64 would do it, since that uses the LP64 data model.  But I
just tried it and got an assertion violation just as it was trying to
write past the 2 Gbyte limit:

  $ ./big0
  Assertion failed: *posp == OFF_NONE || *posp == lseek(nciop->fd, 0, 
SEEK_CUR), file posixio.c, line 189
  Abort(coredump)
  /test/russ$ ls -l big.nc
  total 4212216
  -rw-r--r--   1 russ       ustaff     2147483647 Jan 23 12:49 big.nc

I also tried

  CFLAGS='+DD64 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE'

with the same result, though I had no reason to believe those were the
right defines to use.

So although I suspect HP-UX 11.0 has Large File Support, I don't know
how to use it.  If you find out, please let me know.

--Russ