951204: electronic transfer of netcdf files

Steve,

> To:       address@hidden
> From: address@hidden
> Organization: University of California, Davis
> Keywords: 199512050436.AA20898

In the above message you wrote:

>               I am an Atmospheric Science grad student here at U.C.D,
>       and my work involves the analysis of  seasonal sequences
>         of wind and cloud state data. After years of using
>       VAX/VMS exclusively, we're now in the process of gradually 
>       switching over to UNIX, so NetCDF is very helpful to us. The 
>       question I have for you regards the use of FTP to transfer
>       NetCDF files from  UNIX to VMS. I've recently attempted to
>       do this after setting the FTP transfer mode to binary, and 
>       there seems to be a problem in the transfer. I say this be-
>       cause ncdumping the NetCDF file on the UNIX machine (where
>       it was generated) reveals the non-zero I*2 integer in which 
>       I encoded my data. But after the file is transfered to our
>       VAX via FTP, the same dump reveals the dimensions & attributes
>       correctly, but the data is almost all zeros. I understand that
>       FTP requires either UNIX binary or ASCII to work correctly, but
>       as NetCDF is neither, I can see where this could be what's
>       causing the problem. Is that the problem, and if so, how can
>       you electronically transfer a NetCDF file?   

This is a known problem.  Here's something that's very relevant from the
support email archive:

    Date: Tue, 16 Nov 1993 15:19:14 -0700
    From: Russ Rew <address@hidden>
    Message-Id: <address@hidden>
    To: address@hidden (Marie Schall)
    Subject: vms and netcdf at PMEL

    > Organization: NOAA/PMEL
    > Keywords: 199311130253.AA03345

    Hi Marie,

    > Awhile ago (July) I sent you mail about being unable to read
    > netCDF files on vms which have been binary FTP'd from unix to vms.
    > You suggested that I look at options on FTP, but, I couldn't find
    > any options on our FTP to force "stream_LF" output rather than
    > "fixed length 512 byte records" for image transfers.  I have a
    > program to modify the RMS attributes after transferring the file,
    > which fixes the problem, but it is kind of a pain to always have
    > to run modify after FTP.  I have recently had to look at this
    > problem again.  In xdrposix.c, is there any reason not to use the
    > "ctx=stm" keyword in the file open statement?  This will fix the
    > problem at the source. e.g.
    > 
    >         fd = open(path, fmode, 0666, "ctx=stm") ;
    > 
    > I could not think of a reason when you would want file access
    > other than stream mode, but, I am not familiar with your routines.
    > I wanted to check with you that stream mode was correct for all
    > cases.  Thanks for your help.

    That sounds right.  In the previous major release (2.02), the call
    to fopen for VMS included all the following optional file attribute
    options:

            "ctx=stm","mbf=2","mbc=16"

    but for some reason these were left out of the current release.

    We haven't had a VMS system around here to test on for a long time,
    so things like this can easily fall through the cracks.

As I see it, you have three options:

    * Convert the RMS attribute on the file from fixed, 512-byte records
      to stream_LF (I don't recall how to do this).

    * Add a fourth argument "ctx=stm" to the call to open() in the
      netCDF source file libsrc/xdrposix.c and rebuild and reinstall the
      netCDF package.

    * Obtain and install the netCDF 2.4beta release.  It already has the
      above modification.

Please let me know how this works out.

--------
Steve Emmerson   <address@hidden>