This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
Gabe, > Thank you very much for going through that for me. Adding the LIBS=-ldl > flag let configure find HDFflush. I also had to add the disable-shared > option. With those two changes, everything worked fine. > > Unfortunately, this is all leadup to the part I actually need - the > parallel IO fortran interface. Following the instructions here ( > http://www.unidata.ucar.edu/software/netcdf/docs/netcdf-fortran-install.html) > I see that not using the shared libraries makes life harder. Yes, and it turns out that shared libraries are not supported for parallel HDF5, as explained here: http://mail.hdfgroup.org/pipermail/hdf-forum_hdfgroup.org/2013-February/015393.html > I tried adding > all these flags, but make check returned an error: > > $ LIBS="-lnetcdf -lhdf5_hl -lhdf5 -lz -ldl" CC=mpicc > CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib > LD_LIBRARY_PATH=/usr/local/lib CFLAGS=-DgFortran ./configure > --enable-parallel-tests --prefix=/usr/local --disable-shared > $ make check > > ends with undefined references to all the MPI commands such as: > > H5Smpio.c:(.text+0x12c5): undefined reference to `MPI_Error_string' I think you needed F90=mpif90 to get those MPI functions, for the same reason you needed CC=mpicc to get the C MPI functions. Here's what worked for me: $ H5DIR=/machine/russ/installs/hdf5_1811-par # parallel hdf5_1.8.11, static $ NCDIR=/machine/russ/installs/nc430-par # parallel netCDF-4_4.3.0, static $ CPPFLAGS="-I${NCDIR}/include" F90=mpif90 CC=mpicc LDFLAGS="-L${NCDIR}/lib -L${H5DIR}/lib" LIBS="-lnetcdf -lz -ldl -lm" ./configure --disable-shared --enable-parallel-tests --prefix=${INDIR} $ make check install That was using the current netcdf-fortran svn trunk: https://sub.unidata.ucar.edu/netcdf-fortran/trunk/ but I expect it works for the released netcdf-fortran-4.2 as well. > This is such a painful process... installing four programs from source just > to get easier parallel IO... Yup! I'd be interested in a way to make it easier, if you come up with one. Maybe using cmake tools instead of the autoconf toolchain, but I haven't mastered those ... --Russ > Gabriel D Weymouth > SMMI Lecturer > The University of Southampton > Highfield Campus, Building 28, Room 2021 > Southampton, SO17 1BJ, UK > > > address@hidden> wrote: > > > Hi Gabe, > > > > > I'm trying to get some parallel IO support through netCDF but the > > > installation is killing me. I am running Ubuntu 13.04 and couldn't find a > > > package online that supports parallel netCDF (there is libhdf5-mpich2-dev > > > but nothing like that for netCDF). > > > > > > So, I tried to build it myself. First I tried to use the HDF5 package and > > > just build netCDF but it could find the HDF5 libraries. So I removed the > > > package and built HDF5 (and zlib) but the netCDF configure script still > > > can't find the hdf5 library. Here is the command I used: > > > > Note that HDF5 libraries have to be built to support parallel I/O in order > > for > > parallel I/O to be enabled for netCDF-4. This is typically accomplished by > > configuring with CC=mpicc, where for me mpicc is a wrapper around gcc: > > > > http://www.unidata.ucar.edu/software/netcdf/docs/build_parallel.html > > > > As an example, here's what I just used for hdf5-1.8.11: > > > > $ H5DIR=/machine/russ/installs/hdf5-1811-par > > $ CC=mpicc ./configure --prefix=${H5DIR} && make check install > > > > The resulting installs/lib directory has the libraries as well as a > > libhdf5.settings file that lists features of the library and contains this > > line when > > parallel I/O is enabled: > > > > Parallel HDF5: mpicc > > > > Then I built netcdf-4.3.0 thusly, (which only found H5Fflush when I > > included the > > LIBS=-ldl setting, for some reason I don't understand): > > > > $ LIBS=-ldl CC=mpicc CPPFLAGS=-I${H5DIR}/include LDFLAGS=-L${H5DIR}/lib > > ./configure --enable-parallel-tests > > --prefix=/machine/russ/installs/nc430-par && make check install > > > > I'm adding a mention of the necessity for "LIBS=-ldl" to the document > > above, but > > I would really like to know why it seems to be necessary and is not > > automatically > > supplied by the configure script ... > > > > The above may work for you, but still didn't complete successfully for me, > > instead > > resulting in a linking error having to do with trying to use shared > > libraries: > > > > /usr/bin/ld: > > /machine/russ/installs/hdf5-1811-par/lib/libhdf5_hl.a(H5DS.o): relocation > > R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared > > object; recompile with -fPIC > > /machine/russ/installs/hdf5-1811-par/lib/libhdf5_hl.a: could not read > > symbols: Bad value > > > > It turns out the HDF5 configure disables shared libraries when enabling > > parallel I/O, > > so I finally had to add "--disable-shared" as a configure option to get a > > successful > > parallel I/O build: > > > > $ LIBS=-ldl CC=mpicc CPPFLAGS=-I${H5DIR}/include LDFLAGS=-L${H5DIR}/lib > > ./configure --disable-shared --enable-parallel-tests > > --prefix=/machine/russ/installs/nc430-par && make check install > > > > By the way, if you get that working, you should also be able to use > > parallel-I/O > > on netCDF classic format files if you also install the pnetcdf library from > > Argonne/Northwestern in a place where configure can find it and add > > "--enable-pnetcdf" to the above configure command. > > > > Also, we're currently experiencing some sort of test server failure for the > > remote tests, so if you're going to try this right away, you may want to > > add > > --disable-dap-remote-tests to the configure options. > > > > --Russ > > > > > $ CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib > > > LD_LIBRARY_PATH=/usr/local/lib CXX=null FC=null ./configure > > > --prefix=/usr/local --enable-netcdf-4 --enable-parallel-tests > > > > > > and here is the configure script response: > > > > > > checking for library containing H5Fflush... no > > > configure: error: Can't find or link to the hdf5 library. Use > > > --disable-netcdf-4, or see config.log for errors. > > > > > > I've attached the config.log. Any help would be appreciated. > > > > > > Thanks, > > > > > > Gabriel D Weymouth > > > SMMI Lecturer > > > The University of Southampton > > > Highfield Campus, Building 28, Room 2021 > > > Southampton, SO17 1BJ, UK > > > > > > > > > Hello, > > > > > > I'm trying to get some parallel IO support through netCDF but the > > > installation is killing me. I am running Ubuntu 13.04 and couldn't find a > > > package online that supports parallel netCDF (there is libhdf5-mpich2-dev > > > but nothing like that for netCDF). > > > > > > So, I tried to build it myself. First I tried to use the HDF5 package and > > > just build netCDF but it could find the HDF5 libraries. So I removed the > > > package and built HDF5 (and zlib) but the netCDF configure script still > > > can't find the hdf5 library. Here is the command I used: > > > > > > $ CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib > > > LD_LIBRARY_PATH=/usr/local/lib CXX=null FC=null ./configure > > > --prefix=/usr/local --enable-netcdf-4 --enable-parallel-tests > > > > > > and here is the configure script response: > > > > > > checking for library containing H5Fflush... no > > > configure: error: Can't find or link to the hdf5 library. Use > > > --disable-netcdf-4, or see config.log for errors. > > > > > > I've attached the config.log. Any help would be appreciated. > > > > > > Thanks, > > > > > > Gabriel D Weymouth > > > SMMI Lecturer > > > The University of Southampton > > > Highfield Campus, Building 28, Room 2021 > > > Southampton, SO17 1BJ, UK > > > > > > > > Russ Rew UCAR Unidata Program > > address@hidden http://www.unidata.ucar.edu > > > > > > > > Ticket Details > > =================== > > Ticket ID: AWC-509858 > > Department: Support netCDF > > Priority: Normal > > Status: Closed > > > > > > Russ Rew UCAR Unidata Program address@hidden http://www.unidata.ucar.edu Ticket Details =================== Ticket ID: AWC-509858 Department: Support netCDF Priority: Normal Status: Closed