[netCDF #OOT-688821]: 1) Netcdf on Redhat Linux 6 has glitches 2) Polite questions

Hi Dr. Smolka,

> to avoid wasting your time:
> 
> The issue "where did the script place the files (below)?" is solved.
> 
> The first run I did as "normal user", e.g. $HOME/netcdfinst/netcdf4.2.1.1
> 
> There it had the problem.
> 
> Then I ran it as "su": The script put the files into a /PeterPaul/netcdf42
> -directory -as the script was told.
> 
> The "user" is "PeterPaul" ($HOME) - same as the directory.

Right, it is not necessary to run the configure script or to run "make" or 
"make check"
as su, if you only want to install the executables and libraries in directories 
where
you have write permissions.  If you want to install the libraries and 
executables in a
directory where other users expect to find them, such as /usr/local/lib and 
/usr/local/bin,
then you need to either accept the default "/usr/local" location, or specify a 
different 
location using the "--prefix=/where/ever" option to the configure script.

The only command you should run as su is "make install", after the libraries 
are built and 
you have run "make check" to test that they work.

> ***************
> 
> The compiler-question below:
> 
> What I want is that it is compatible with the NCAR CESM1_1_1-scripts and
> the compiler-flags there.
> 
> How the text on p. 1/2 is to be understood accordingly I have to find out
> (for the Intel 13.x Compiler).

Intel has published instructions for building the older netCDF 4.1.3 version
with their compilers here:

  
http://software.intel.com/en-us/articles/performance-tools-for-software-developers-building-netcdf-with-the-intel-compilers

After version 4.1.3, we separated the Fortran software from the C software to 
simplify
maintenance and installation.  With version 4.2.1.1 that you have, only the C 
libraries
and utility programs like ncdump are built.  After version 4.2.1.1 is 
installed, you can
build the Fortran libraries separately from the netcdf-fortran-4.2 release, 
telling its
configure script where you installed the C libraries, as described here:

  http://www.unidata.ucar.edu/software/netcdf/docs/netcdf-fortran-install.html

> CESM got "so script-laden" that I decided to "do it as NCAR does"
> regarding the technology and focus on adding the paleoclimatology.
> 
> Therefore: The script-compatibility with NCAR.

I'm not very familiar with the CESM scripts or procedures, so I'm not sure
what that means.  Nevertheless, I'll try to answer the questions I understand 
below.

> ---------- Forwarded message ----------
 ...
> Dear Members of the Netcdf-Group,
> 
> to run cesm (the new) with netcdf (I downloaded 4.2.1.something) on a new and
> fully standard (I got six physically shipped Workstation-CDs from Redhat to
> make sure
> "everything is the standard").
> 
> The Redhat 6.x  I installed as IA32.
> 
> The new Intel-Compiler (with "everything", e.g. "Cluster Pack or so") I
> installed as IA32 as well.
> 
> It is a machine that is dedicated to CESM and related paleoclimatology.
> 
> 
> I ran your the as indicated under:
> 
> netcdf: Building Netcdf with Classical Library.
> 
> First part: Logged on to the system as "normal" user with the name "PeterPaul"
> and the home-directory $HOME/PeterPaul
> 
> I made a directory under /PeterPaul called netcdf42, e.g.
> $HOME/PeterPaul/netcdf42
> 
> This is the directory where netcdf42 should be after installation.
> 
> The install-directory has the name .../netcdf42inst
> 
> Being inside the (...)/netcdf42inst/netcdf4.2.1.1 -directory:
> 
> The glitches:
> 
> using
> 
> ./configure --prefix=/PeterPaul/netcdf42 -disable-dap
> 
> 1) The script noted an "old" system date (2011, intentionally) and stopped.
> 
> OK: Changed to today.
> 
> 2) same: The script "complained" about a missing (something) (directory,
> library etc.) and recommended to disaple netcdf4.
> 
> 3) Done, e.g. dap disabled, netcdf-4 disable as written by you.
> 
> 4) Result
> 
> It ran quite far and passed many tests.
> 
> Then he (the script) said (see below near "the end"):
> 
> while creating a directory: "permission denied".
> 
> (continuing below the stars ******* )
> 
> netcdf error msg:
> 
> PASS: do_comps.sh ================== All 2 tests passed ==================
> make[3]: Leaving directory
> `/home/PeterPaul/netcdf42inst/netcdf-4.2.1.1/examples/CDL' make[2]: Leaving
> directory `/home/PeterPaul/netcdf42inst/netcdf-4.2.1.1/examples/CDL' make[2]:
> Entering directory `/home/PeterPaul/netcdf42inst/netcdf-4.2.1.1/examples'
> make[2]: Nothing to be done for `check-am'. make[2]: Leaving directory
> `/home/PeterPaul/netcdf42inst/netcdf-4.2.1.1/examples' make[1]: Leaving
> directory `/home/PeterPaul/netcdf42inst/netcdf-4.2.1.1/examples' make[1]:
> Entering directory `/home/PeterPaul/netcdf42inst/netcdf-4.2.1.1' make[1]:
> Leaving directory `/home/PeterPaul/netcdf42inst/netcdf-4.2.1.1' Making install
> in include make[1]: Entering directory
> `/home/PeterPaul/netcdf42inst/netcdf-4.2.1.1/include' make[2]: Entering
> directory `/home/PeterPaul/netcdf42inst/netcdf-4.2.1.1/include' make[2]:
> Nothing to be done for `install-exec-am'. test -z 
> "/PeterPaul/netcdf42/include"
> || /bin/mkdir -p "/PeterPaul/netcdf42/include" /bin/mkdir: cannot create
> directory `/PeterPaul': Permission denied make[2]: *** 
> [install-includeHEADERS]
> Error 1 make[2]: Leaving directory
> `/home/PeterPaul/netcdf42inst/netcdf-4.2.1.1/include' make[1]: *** 
> [install-am]
> Error 2 make[1]: Leaving directory
> `/home/PeterPaul/netcdf42inst/netcdf-4.2.1.1/include' make: ***
> [install-recursive] Error 1

I think the errors were mostly from specifying --prefix=/PeterPaul/netcdf42 
instead of 
/home/PeterPaul/netcdf42, in case that's what you intended.  Linux doesn't 
permit
normal users to create new directories at the top level.

> *********************
> 
> 5) Bypass:
> 
> Changed to su (set user, superuser, root)
> 
> Restarted: It ran through and displayed congratulations.
> 
> Thus it should work - except the questions:
> 
> 6) Reset to "normal user".
> 
> 7) In the /PeterPaul/netcdf42 -directory: Nothing found.
> 
> 8) Then again changed back to "su".
> 
> Found under "/usr/bin" quite a lot of files that look like netcdf (I know
> netcdf since ccm3.6).
> 
> So: Of course I can run the computer with "su" (administrator) but with Linux 
> I
> have really an uneasy feeling with it, except for short times (IT since 1979,
> IBM-mainframe).

Right, as explained above you should only run "make install" as root and 
everything,
and then only if you are installing in directories that require root 
permissions.

> Now the questions:
> 
> Q1) Is it, as intended, a 32-Bit netcdf as I installed the Intel-Compiler
> with IA32?

Yes, IA32 means you have installed 32-bit libraries.  That should be OK for 
most uses,
as you can still write and read netCDF classic files (with 32-bit file offsets) 
and
64-bit offset netCDF files.

If it becomes necessary to write or read netCDF-4 files that are compressed or 
chunked,
or that have multiple variables larger than 4GB, you will need to rebuild your 
libraries 
to use netCDF-4.

By the way, with RedHat Linux (or Fedora), you can get prebuilt binary 
installations of 
netCDF, using the "yum" command, including netCDF-4 versions.

> Reason for IA32: All the "rest" is with IA32 and with 64-Bit all arrays are
> "twice that large".

It's good to be compatible with other 32-bit libraries and programs that link 
with shared 
netCDF libraries at run time.  However the on-disk arrays for netCDF-4 files 
are typically
about the same size as for netCDF-3 files, beyond a small fixed-size overhead.  
Only the
slots in the file header that identify where variables start in the file are 64 
bits instead
of 32-bits with 64-bit offset files.

> Q2) Is it correct that it installs (as it appears) in usr/bin?

The default installation location is /usr/local, so the libraries would be in 
/usr/local/lib
and the programs like ncdump, ncgen, nccopy, and nc-config would be in 
/usr/local/bin.

> Or: How can I force it to $HOME/netcdf42? (and does it make sense in context 
> of
> CESM1_1_1?

It sounds like you figured out the answer to this one.

> Q3) Should I have written instead:
> 
> ./configure --prefix=(the full directory path as root for PeterPaul, e.g.
> /..../PeterPaul/netcdf42 starting at root (real root in the admin sense)?

Yes.

> I must admit: Running scripts as "root" creates a little bit an uneasy 
> feeling.
> 
> Is there a different way to enable scripts (applies also to CESM) to make
> directories?

Yes, don't run scripts as root.  Only run "make install" as root if you need to.

> Q4) The Fortran-part (on page 1/2) appears a little unclear.
> 
> The C-Part was compiled (apparently) with the gcc of Redhat, which means:
> The Intel C-Compiler was not(!) used.

The default is to use the first compiler found in your PATH, usually gcc.

You can force use of the Intel compiler by setting the CC environment variable
before running the configure script, for example if you are using bash or sh or 
ksh

  CC=icc FC=ifort ./configure --prefix= ...

With csh or tcsh shells, you use a different syntax to set environment 
variables.

Or you can just give the settings as arguments to configure at the end with 
either 
kind of shell as in:

  ./configure --prefix=... --disable-... CC=icc ...   

> As I compiler normally Fortran with Intel (and as you have the CESM-group near
> you): Will the Intel-Compiler be compatible with the settings of gcc?
> (appears likely by the many settings you did, but I ask for experiences).

I think it's best to use icc and ifort together, or gcc and gfortran together.
We don't test with mixed combinations.

> Into which directory should I place the netcdf-Fortran-Files (by NCARS
> standard) to be compatible with the scripts?
> 
> Email:
> address@hidden (science)
> address@hidden (private)

I don't know what NCAR's standard is.  I would use the same directory in 
--prefix=... for both C and Fortran libraries, or just accept the /usr/local
default if you're willing to run "make install" as root.

> I want: Just to run it compatible to NCAR and after it: Adding 
> paleoclimatology
> and the respective things (already partially tested).
> 
> The "technical" things (IA32, netcdf) should be "1:1 to the NCAR-standard".

To be sure you're compatible with the NCAR standard, I would ask someone from 
the
CESM group or the NCAR CISL consulting group.

> Thankyou very much for providing netcdf.

You're welcome, thank you for doing great science!

> Peter Smolka
> (Dr. Peter P. Smolka)

--Russ

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: OOT-688821
Department: Support netCDF
Priority: Normal
Status: Closed