[netCDF #AWC-509858]: H5Fflush not found

Gabe,

> Thank you very much for going through that for me. Adding the LIBS=-ldl
> flag let configure find HDFflush. I also had to add the disable-shared
> option. With those two changes, everything worked fine.
> 
> Unfortunately, this is all leadup to the part I actually need - the
> parallel IO fortran interface. Following the instructions here (
> http://www.unidata.ucar.edu/software/netcdf/docs/netcdf-fortran-install.html)
> I see that not using the shared libraries makes life harder.

Yes, and it turns out that shared libraries are not supported for parallel HDF5,
as explained here:

  
http://mail.hdfgroup.org/pipermail/hdf-forum_hdfgroup.org/2013-February/015393.html

>                                         I tried adding
> all these flags, but make check returned an error:
> 
> $  LIBS="-lnetcdf -lhdf5_hl -lhdf5 -lz -ldl" CC=mpicc
> CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib
> LD_LIBRARY_PATH=/usr/local/lib CFLAGS=-DgFortran ./configure
> --enable-parallel-tests --prefix=/usr/local --disable-shared
> $ make check
> 
> ends with undefined references to all the MPI commands such as:
> 
> H5Smpio.c:(.text+0x12c5): undefined reference to `MPI_Error_string'

I think you needed F90=mpif90 to get those MPI functions, for the same
reason you needed CC=mpicc to get the C MPI functions.

Here's what worked for me:

  $ H5DIR=/machine/russ/installs/hdf5_1811-par # parallel hdf5_1.8.11, static
  $ NCDIR=/machine/russ/installs/nc430-par     # parallel netCDF-4_4.3.0, static
  $ CPPFLAGS="-I${NCDIR}/include" F90=mpif90 CC=mpicc LDFLAGS="-L${NCDIR}/lib 
-L${H5DIR}/lib" LIBS="-lnetcdf -lz -ldl -lm" ./configure --disable-shared 
--enable-parallel-tests --prefix=${INDIR}
  $ make check install

That was using the current netcdf-fortran svn trunk:

  https://sub.unidata.ucar.edu/netcdf-fortran/trunk/

but I expect it works for the released netcdf-fortran-4.2 as well.

> This is such a painful process... installing four programs from source just
> to get easier parallel IO...

Yup!  I'd be interested in a way to make it easier, if you come up with one.  
Maybe
using cmake tools instead of the autoconf toolchain, but I haven't mastered 
those ...

--Russ

> Gabriel D Weymouth
> SMMI Lecturer
> The University of Southampton
> Highfield Campus, Building 28, Room 2021
> Southampton, SO17 1BJ, UK
> 
> 
> address@hidden> wrote:
> 
> > Hi Gabe,
> >
> > > I'm trying to get some parallel IO support through netCDF but the
> > > installation is killing me. I am running Ubuntu 13.04 and couldn't find a
> > > package online that supports parallel netCDF (there is libhdf5-mpich2-dev
> > > but nothing like that for netCDF).
> > >
> > > So, I tried to build it myself. First I tried to use the HDF5 package and
> > > just build netCDF but it could find the HDF5 libraries. So I removed the
> > > package and built HDF5 (and zlib) but the netCDF configure script still
> > > can't find the hdf5 library. Here is the command I used:
> >
> > Note that HDF5 libraries have to be built to support parallel I/O in order
> > for
> > parallel I/O to be enabled for netCDF-4.  This is typically accomplished by
> > configuring with CC=mpicc, where for me mpicc is a wrapper around gcc:
> >
> >   http://www.unidata.ucar.edu/software/netcdf/docs/build_parallel.html
> >
> > As an example, here's what I just used for hdf5-1.8.11:
> >
> >    $ H5DIR=/machine/russ/installs/hdf5-1811-par
> >    $ CC=mpicc ./configure --prefix=${H5DIR} && make check install
> >
> > The resulting installs/lib directory has the libraries as well as a
> > libhdf5.settings file that lists features of the library and contains this
> > line when
> > parallel I/O is enabled:
> >
> >                   Parallel HDF5: mpicc
> >
> > Then I built netcdf-4.3.0 thusly, (which only found H5Fflush when I
> > included the
> > LIBS=-ldl setting, for some reason I don't understand):
> >
> >    $ LIBS=-ldl CC=mpicc CPPFLAGS=-I${H5DIR}/include LDFLAGS=-L${H5DIR}/lib
> > ./configure --enable-parallel-tests
> > --prefix=/machine/russ/installs/nc430-par && make check install
> >
> > I'm adding a mention of the necessity for "LIBS=-ldl" to the document
> > above, but
> > I would really like to know why it seems to be necessary and is not
> > automatically
> > supplied by the configure script ...
> >
> > The above may work for you, but still didn't complete successfully for me,
> > instead
> > resulting in a linking error having to do with trying to use shared
> > libraries:
> >
> >   /usr/bin/ld:
> > /machine/russ/installs/hdf5-1811-par/lib/libhdf5_hl.a(H5DS.o): relocation
> > R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared
> > object; recompile with -fPIC
> >   /machine/russ/installs/hdf5-1811-par/lib/libhdf5_hl.a: could not read
> > symbols: Bad value
> >
> > It turns out the HDF5 configure disables shared libraries when enabling
> > parallel I/O,
> > so I finally had to add "--disable-shared" as a configure option to get a
> > successful
> > parallel I/O build:
> >
> >   $ LIBS=-ldl CC=mpicc CPPFLAGS=-I${H5DIR}/include LDFLAGS=-L${H5DIR}/lib
> > ./configure --disable-shared --enable-parallel-tests
> > --prefix=/machine/russ/installs/nc430-par && make check install
> >
> > By the way, if you get that working, you should also be able to use
> > parallel-I/O
> > on netCDF classic format files if you also install the pnetcdf library from
> > Argonne/Northwestern in a place where configure can find it and add
> > "--enable-pnetcdf" to the above configure command.
> >
> > Also, we're currently experiencing some sort of test server failure for the
> > remote tests, so if you're going to try this right away, you may want to
> > add
> > --disable-dap-remote-tests to the configure options.
> >
> > --Russ
> >
> > > $ CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib
> > > LD_LIBRARY_PATH=/usr/local/lib CXX=null FC=null ./configure
> > > --prefix=/usr/local --enable-netcdf-4 --enable-parallel-tests
> > >
> > > and here is the configure script response:
> > >
> > > checking for library containing H5Fflush... no
> > > configure: error: Can't find or link to the hdf5 library. Use
> > > --disable-netcdf-4, or see config.log for errors.
> > >
> > > I've attached the config.log. Any help would be appreciated.
> > >
> > > Thanks,
> > >
> > > Gabriel D Weymouth
> > > SMMI Lecturer
> > > The University of Southampton
> > > Highfield Campus, Building 28, Room 2021
> > > Southampton, SO17 1BJ, UK
> > >
> > >
> > > Hello,
> > >
> > > I'm trying to get some parallel IO support through netCDF but the
> > > installation is killing me. I am running Ubuntu 13.04 and couldn't find a
> > > package online that supports parallel netCDF (there is libhdf5-mpich2-dev
> > > but nothing like that for netCDF).
> > >
> > > So, I tried to build it myself. First I tried to use the HDF5 package and
> > > just build netCDF but it could find the HDF5 libraries. So I removed the
> > > package and built HDF5 (and zlib) but the netCDF configure script still
> > > can't find the hdf5 library. Here is the command I used:
> > >
> > > $ CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib
> > > LD_LIBRARY_PATH=/usr/local/lib CXX=null FC=null ./configure
> > > --prefix=/usr/local --enable-netcdf-4 --enable-parallel-tests
> > >
> > > and here is the configure script response:
> > >
> > > checking for library containing H5Fflush... no
> > > configure: error: Can't find or link to the hdf5 library. Use
> > > --disable-netcdf-4, or see config.log for errors.
> > >
> > > I've attached the config.log. Any help would be appreciated.
> > >
> > > Thanks,
> > >
> > > Gabriel D Weymouth
> > > SMMI Lecturer
> > > The University of Southampton
> > > Highfield Campus, Building 28, Room 2021
> > > Southampton, SO17 1BJ, UK
> > >
> > >
> > Russ Rew                                         UCAR Unidata Program
> > address@hidden                      http://www.unidata.ucar.edu
> >
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: AWC-509858
> > Department: Support netCDF
> > Priority: Normal
> > Status: Closed
> >
> >
> 
> 
Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: AWC-509858
Department: Support netCDF
Priority: Normal
Status: Closed