Re: 20030521: Netcdf files appear corrupt when read through Bulk Data Service

>To: address@hidden
>From: "Spencer L. Swift" <address@hidden>
>Subject: Netcdf files appear corrupt when read through Bulk Data Service
>Organization: NASA/GSFC
>Keywords: 200305092353.h49Nr57U012369

Hi Spencer,

> Are you aware of any issues with NetCDF 3.5 and the Bulk Data Service
> available under IRIX?
> 
> http://www.sgi.com/peripherals/networking/software_protocol.html
> 
> If you aren't aware of BDS, it is an extension to NFS that increases
> the throughput performance of cross mounted partitions for large
> transfers. 
> 
> The problem we are seeing is that NetCDF files appear corrupt if they
> are accessed from partitions that are using BDS, but work fine from
> partitions w/o BDS.
> 
> This can be shown simply by doing 'make all test' on a BDS mounted
> partition.  The compilation will succeed, but the testing phase will
> fail immediately with errors like this:
> 
> ================================================================
> 
> Making `test' in directory /home/sswift/netcdf-3.5.0/src/nc_test
> 
> make[2]: Entering directory `/home/sswift/netcdf-3.5.0/src/nc_test'
> c89 -c -O -I../libsrc  -DNDEBUG nc_test.c
> "nc_test.c": Warning: Olimit was exceeded on function main; will not optimize.
>       To optimize use -OPT:Olimit=0 (off) or -OPT:Olimit=2199
> "nc_test.c": Warning: To override Olimit for all functions in file, use 
> -OPT:Olimit=2199
> c89 -c -O -I../libsrc  -DNDEBUG error.c
> c89 -c -O -I../libsrc  -DNDEBUG test_get.c
> c89 -c -O -I../libsrc  -DNDEBUG test_put.c
> c89 -c -O -I../libsrc  -DNDEBUG test_read.c
> c89 -c -O -I../libsrc  -DNDEBUG test_write.c
> c89 -c -O -I../libsrc  -DNDEBUG util.c
> c89 -o nc_test -O  nc_test.o error.o test_get.o test_put.o test_read.o 
> test_write.o util.o -L../libsrc -lnetcdf -lm 
> ./nc_test -c
> ./nc_test
> *** Testing nc_strerror ... ok
> *** Testing nc_open ... 
>       FAILURE at line 104 of test_read.c: nc_open: Not a netCDF file
>       FAILURE at line 107 of test_read.c: nc_redef of read-only file should 
> fail
>       FAILURE at line 111 of test_read.c: nc_open: Not a netCDF file
>       FAILURE at line 136 of test_read.c: nc_close: Not a netCDF id
>       ### 4 FAILURES TESTING nc_open! ###
> *** Testing nc_close ... 
>       FAILURE at line 153 of test_read.c: nc_open: Not a netCDF file
>       FAILURE at line 158 of test_read.c: nc_close failed: Not a netCDF id
>       FAILURE at line 171 of test_read.c: nc_open: Not a netCDF file
>       FAILURE at line 174 of test_read.c: nc_close in data mode failed: Not a 
> netCDF id
>       ### 4 FAILURES TESTING nc_close! ###
>       :
>       :
>       :
> ====================================================================
> 
> 
> 
> The local systems are configured thus:
> IRIX 6.5.17f
> MIPSpro 7.3.1.2m & 7.2.1.3m
> netCDF 3.5.0
> BDS 2.4 (patch 4817)
> 
> The NFS BDS options are:
> rw,nodefxattr,bg,bds,bdsauto=65536

No, this is the first report we've seen of this problem.  I'm trying
to see if we can reproduce the problem locally, but first we have to
configure a BDS file system.

If you have time, you might try the netCDF 3.5.1 beta 10 release, to
see if it still exhibits the same problem"

  ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-beta.tar.Z

--Russ

_____________________________________________________________________

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://my.unidata.ucar.edu