[netCDF #YYX-933412]: overwirte nc4 variable issues.

> This file comes from a user in Germany, which clearly the data is generate
> from WRF model,
> used for restart.
> 
> coincidently, I am the person who wrote the WRF NetCDF4 code,
> which def_endian part is never called explicitly.
> address@hidden> wrote:

OK, thanks for the info.  In that case, I think a workaround
for the bug would be:

 1. Use nccopy to copy the original file, written on a big-endian
    platform, to the native platform of your computer, which must
    be little endian, specifying that the destination file is to be
    netCDF-4 format:

    nccopy -k netCDF-4 source.nc destination.nc

  
 2. Run your program to overwrite LU_INDEX on the destination file
    resulting from nccopy above.  Don't worry about endianness, just
    write values to the destination.nc file as you normally would and
    let the library take care of encodingg the data.

 3. Check that the values of LU_INDEX are now OK.

If you decide to try this workaround, please let me know if it works.
Thanks!

--Russ

> > Wei,
> >
> > I've looked at the destination file you made available,
> >
> >   wrfrst_d01_2010-01-20_00_00_00_destination
> >
> > and it's a netCDF-4 file (not a netCDF-4 classic model format).
> >
> > The integer variables in that file are all defined with
> > NC_ENDIAN_BIG, defined in "netcdf.h" to be 2.  But all the
> > floating-point variables in the file, including LU_INDEX,
> > seem to have been defined with NC_ENDIAN_NATIVE, defined in
> > "netcdf.h" to be 0.  Do you know if the function
> >
> >   nc_def_var_endian(int ncid, int varid, int endian)
> >
> > was explicitly called when defining any of the variables?
> > Or was the destionation file generated by nccopy?
> >
> > I'm asking because there may be another bug involved, if the
> > endianness of floating-point variables isn't properly
> > represented in the destination file of nccopy ...
> >
> > > This sounds like the open bug report, described in this Jira
> > > ticket:
> > >
> > > https://bugtracking.unidata.ucar.edu/browse/NCF-291
> > >
> > > The byte order for a variable is fixed when the variable is defined,
> > > but if values of the variable are later written on a platform with a
> > > different byte order, the values are not converted to the byte order
> > > of the platform on which the variable was first created.  This is a
> > > bug in netCDF-4 that is assigned for fixing, but I have just changed
> > > its priority from "Major" to "Critical", as you are the second person
> > > to report the bug.
> > >
> > > The only workaround currently is for the user to check the endianness
> > > of the data in the netCDF-4 file using the nc_inq_var_endian()
> > > function. If it differs from the endianness of the platform on which
> > > the data will be overwritten or appended, the user must change the
> > > endianness of the in-memory array before writing the data.  That's
> > > clearly not the intended behavior, but a consequence of taking the
> > > "reader makes write" rule of HDF5 too far.
> > >
> > > > I have two files:
> > > > met_em.d01.2010-01-20_12_00_00.nc
> > > > wrfrst_d01_2010-01-20_00_00_00_destination
> > > > Both have a variable named "LU_INDEX",
> > > > I want to read from first file, and then overwrite the one in the
> > second.
> > > >
> > > > When tried to read back this variable from second file.
> > > > Then value is not right.
> > > > Attached is a C program which demonstrates this behavior.
> > > >
> > > > The above two file can get from:
> > > > ftp ftp.cgd.ucar.edu
> > > > login: anonymous
> > > > passwd: your-email
> > > > cd incoming
> > > > get file-name
> > > > ...
> > > > quit
> > > >
> > > > We have also found out that the overwritten variable is somehow
> > > > byte swapped. The following NCL script demonstrates this.
> > > >
> > > > f = addfile("met_em.d01.2010-01-20_12_00_00.nc", "r")
> > > > lu = f->LU_INDEX
> > > > lug = ndtooned(lu)
> > > > print(lug(::100))
> > > >
> > > > f = addfile("wrfrst_d01_2010-01-20_00_00_00_destination", "r")
> > > > lu := f->LU_INDEX
> > > > lut = ndtooned(lu)
> > > > print(lut(::100))
> > > >
> > > > setfileoption("bin","WriteByteOrder","BigEndian")
> > > > cbinwrite("test.bin",lu)
> > > > lu1 = cbinread("test.bin",-1,"float")
> > > > print(lu1(::100))
> > > >
> > > >
> > > > Could please check this issue, and see if we have done anything wrong,
> > > > or if there is any way we can control the byte order?
> > > Russ Rew                                         UCAR Unidata Program
> > > address@hidden                      http://www.unidata.ucar.edu
> > >
> > >
> > Russ Rew                                         UCAR Unidata Program
> > address@hidden                      http://www.unidata.ucar.edu
> >
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: YYX-933412
> > Department: Support netCDF
> > Priority: Normal
> > Status: Closed
> >
> >
> 
> 

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: YYX-933412
Department: Support netCDF
Priority: Normal
Status: Closed