[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[ldmMcidas #KQH-987099]: Make error: ldm-mcidas on Mac OS X 10.5.1
- Subject: [ldmMcidas #KQH-987099]: Make error: ldm-mcidas on Mac OS X 10.5.1
- Date: Sat, 09 Feb 2008 13:55:33 -0700
Hi Dave,
re:
> Tom (probably),
Yup, its me :-)
> I've got a nice new Mac Pro Quad-core running Mac OS X 10.5.1 that
> sooner (with good luck) or later (with poor luck) is going to be my
> local data server, running LDM and some decoders.
Sounds like fun!
> It seems to be
> running LDM OK (thanks to some intervention by Steve Emmerson), other
> than the fact that it's not logging to logs/ldm.log correctly (though
> it is logging to system.log).
>
> In any case, one of the decoders that I need is pnga2area, which is
> probably the only one in the ldm-mcidas package that I need. I'm
> having problems compiling ldm-mcidas, though.
OK.
> I've attached config.log, configure.log, and make.log files that
> testify to the difficulty.
Thanks. You giving me a login was even more helpful.
> The results don't seem to be sensitive to the compiler I specify in my
> CUSTOMIZE file (c89, cc, or gcc)--the make attempt fails at the same
> point, regardless.
>
> I know you don't have a Mac (at least of this sort) available at
> Unidata, so if you need to intervene directly, you can do so as follows:
Thanks for the access to your machine!
> ldm-mcidas is located in /usr/local/unidata on my new machine.
I logged on to virga (using SSH, of course) this morning and started poking
around. Here is what I found and what I did to get most of ldm-mcidas to
build:
- first, I created an ldm-mcidas subdirectory under /usr/local/unidata and
I moved the ldm-mcidas-2004 tree and distribution under the new directory
- next, I copied over an as yet unreleased version of ldm-mcidas, v2006, to
your /usr/local/unidata/ldm-mcidas and unpacked it creating the
/usr/local/unidata/ldm-mcidas/ldm-mcidas-2006 directory tree
- in preparation for running 'configure', I defined a set of environment
variables in your Tcsh defintion file ~ldm/.cshrc:
setenv CC /usr/bin/gcc
setenv FC /usr/local/bin/g95
setenv CPP_LDM -I/Users/ldm/include
setenv LD_LDM '-L/Users/ldm/lib -lldm'
setenv CPP_MCIDAS -I/Users/mcidas/mcidas2007/src
setenv LD_MCIDAS '-L/Users/mcidas/lib -lmcidas'
setenv CPP_NETCDF -I/Users/mcidas/mcidas2007/netcdf/libsrc
setenv LD_NETCDF '-L/Users/mcidas/mcidas2007/netcdf/libsrc -lnetcdf'
Comments (mainly for myself):
- defines for all flags was reasonably straightforward
- CPP_MCIDAS, LD_MCIDAS, CPP_NETCDF, and LD_NETCDF point to non-existent
locations. This was done just to be able to produce a Makefile.
- it took me a couple of minutes to determine that the only Fortran
compiler installed was 'g95' -- no biggie as pnga2area is an all-C
application
- after making the environment variable definitions active in ~ldm/.cshrc,
I ran 'configure' as follows:
<as 'ldm'>
cd /usr/local/unidata/ldm-mcidas/ldm-mcidas-2006/src
./configure
- I then ran make. This is when I ran into the problems you reported:
1) the configure step defined a macro named UD_SYSTEM_LIMITS_H
as ///usr/lib/gcc/i686-apple-darwin9/4.0.1/include/limits.h.
This file exists and contains the appropriate limits.h stuff
for the installation/compiler (gcc is version 4.0.1)
2) gcc complained as follows:
gcc -c -O -I/usr/local/src/mcidas/src -I/usr/local/unidata/ldm/include
-I../libpng -I../zlib -I../port/misc -I../port/cfortran -I/usr/local/include
commonsubs.c
In file included from ../port/misc/limits.h:12,
from commonsubs.c:10:
///usr/lib/gcc/i686-apple-darwin9/4.0.1/include/limits.h:10:25: error: no
include path in which to search for limits.h
make[2]: *** [commonsubs.o] Error 1
make[1]: *** [decode/current] Error 1
Evidently, gcc did not like the code produced by 'configure' in
src/port/misc/limits.h:
#define UD_SYSTEM_LIMITS_H
"///usr/lib/gcc/i686-apple-darwin9/4.0.1/include/limits.h"
#include UD_SYSTEM_LIMITS_H
Experimentation showed that the following handmade modification allowed
the build to
proceed further (but not completely)
#define UD_SYSTEM_LIMITS_H <limits.h>
#include UD_SYSTEM_LIMITS_H
I need to talk to Steve Emmerson about this issue.
- after fixing-up the port/misc/limits.h file (again, created by 'configure'), I
continued with the make, but ran into the cfortran code not knowing about this
platform. Reviewing the cfortran code (port/cfortran/cfortran.h) showed that
it had no knowledge of Macs in general.
A Google search using 'cfortran MacOS' resulted in a number of hits the most
relevant/useful of which was:
https://setisvn.ssl.berkeley.edu/svn/lib/cfitsio/cfortran.h
That page showed some modifications made for Macintosh:
#if defined(macintosh) /* 11/1999 */
#define f2cFortran
#endif
#if defined(__APPLE__) /* 11/2002 */
#define f2cFortran
#endif
#if defined(__hpux) /* 921107: Use __hpux instead of __hp9000s300 */
#define hpuxFortran /* Should also allow hp9000s7/800 use.*/
#endif
I added the two defines related to Macs (macintosh and __APPLE_) to
port/cfortran/cfortran.h and was then able to build all ldm-mcidas executables
that did not rely on the McIDAS library (including proftomd which needs to
link against the netCDF library since netCDF is installed on the machine!)
- since building of 'nldn2md' failed, I was unable to fully run 'make install':
<still as 'ldm'>
cd /usr/local/unidata/ldm-mcidas/ldm-mcidas-2006/src
make install <- this fails at some point, but it does create the needed
output bin, lib, and etc directories
- I tried running 'make pnga2area.test' from the decode directory, but I ran
into
a problem where MacOS does not differentiate between upper and lower case
file names.
This will result in 'route.sys' and 'syskey.tab' being deleted when a 'make
clean'
or 'make distclean' is run from the src or decode directories. This, in turn,
forced me to re-extract these files from the compressed tar file. Argh!@#$%
In order to get 'make pnga2area.test' to run, I was forced to edit the
Makefile:
<as 'ldm'>
cd /usr/local/unidata/ldm-mcidas/ldm-mcidas-2006/src/decode
-- edit Makefile and comment out the copying of route.sys to ROUTE.SYS and
syskey.tab to SYSKEY.TAB
I was then able to run the test:
ldm_virga% make pnga2area.test
# cp route.sys ROUTE.SYS
# cp syskey.tab SYSKEY.TAB
rm -f pngarea.test AREA0200;
././pnga2area -vl- -a SATANNOT -b SATBAND -r U3 < pngarea
Feb 09 19:54:03 pnga2area[45786] NOTE: Starting Up
Feb 09 19:54:03 pnga2area[45786] INFO: output file pathname: ./AREA0200
Feb 09 19:54:03 pnga2area[45786] NOTE: unPNG:: 5549 225088 40.5637
Feb 09 19:54:03 pnga2area[45786] NOTE: Exiting
rm -f AREA0200;
././pnga2area -vl- -a SATANNOT -b SATBAND -r ZZ,200 < pngarea
./batch.k: line 121: cd: ~mcidas: No such file or directory
Feb 09 19:54:03 pnga2area[45795] NOTE: Starting Up
Feb 09 19:54:03 pnga2area[45795] INFO: output file pathname: ./AREA0200
Feb 09 19:54:03 pnga2area[45795] NOTE: unPNG:: 5549 225088 40.5637
Feb 09 19:54:03 pnga2area[45795] NOTE: Exiting
././area2png -vl- -a SATANNOT -b SATBAND -f AREA0200 pngarea.test
Feb 09 19:54:03 area2png[45798] NOTE: Starting Up
./batch.k: line 168: cd:
/Users/ldm/unidata/ldm-mcidas/ldm-mcidas-2006/src/decode/workdata: No such file
or directory
Feb 09 19:54:03 area2png[45798] INFO: output file pathname: pngarea.test
Feb 09 19:54:03 area2png[45798] NOTE: doPNG:: 225088 5549 0.0247
PNGA2AREA test successful
- since I had a working version of pnga2area, I hand-copied the executables
that had
been built to the ldm-mcidas installation bin directory:
<still as 'ldm' in the src/decode directory>
cp area2png pnga2area proftomd zlibg2gini batch.k pngg2gini uwgrid.sh
../../bin
- I then made some symbolic links:
cd /usr/local/unidata/ldm-mcidas
ln -s ldm-mcidas-2006 runtime
ln -s runtime/* .
- next, I copied the newly created version of pnga2area to the decoders
directory
defined in 'ldm's PATH:
cp bin/pnga2area /usr/local/unidata/decoders/bin
- since Dave had mentioned some problems with LDM logging, I decided to modify
the pqact.conf file to log pnga2area transactions to ~ldm/logs/ldm-mcidas.log.
N.B., logging to ldm-mcidas.log will require that you keep an eye on this
file.
The best approach is to rotate the file from a crontab entry like:
#
# Rotate ldm-mcidas logs
#
00 16 * * * bin/newlog logs/ldm-mcidas.log 7
N.B., I did not add this entry to your crontab.
- since I did not see any UNIWISC (aka MCIDAS) images arrive while requesting
from freshair.atmos.washington.edu, I added a redundant request for
DDPLUS|IDS|UNIWISC to the ~ldm/etc/ldmd.conf file and restarted the LDM.
- after a bit of time, I saw UNIWISC images being received and processed. Here
is a listing of pnga2area processing in ~ldm/logs/ldm-mcidas.log:
<cd ~ldm>
less logs/ldm-mcidas.log
Feb 09 19:16:55 pnga2area[27354] NOTE: Starting Up
Feb 09 19:16:55 pnga2area[27354] INFO: output file pathname:
/data/mcidas/gewv/08020918_swe.mca
Feb 09 19:16:55 pnga2area[27354] NOTE: unPNG:: 738391 4724488 6.3984
Feb 09 19:16:55 pnga2area[27354] NOTE: Exiting
Feb 09 19:16:58 pnga2area[27463] NOTE: Starting Up
Feb 09 19:16:58 pnga2area[27463] INFO: output file pathname:
/data/mcidas/geir/0802091800_sie.mca
Feb 09 19:16:58 pnga2area[27463] NOTE: unPNG:: 1389722 4724488 3.3996
Feb 09 19:16:58 pnga2area[27463] NOTE: Exiting
...
I believe at this point that you have a functional pnga2area decoder running on
virga.
However, I have some questions/comments:
- the UNIWISC actions defined in your ~ldm/etc/pqact.conf file all decode to the
/data/mcidas directory hierarchy. Since there are lots of subdirectories
under /data/mcidas (e.g., gevis, etc.), and since these subdirectories have
lots of current images in them, I am left wondering:
Is another machine decoding the same images into the same directory hierarchy?
If the answer is yes, you need to either change your output directory on virga
or turn off decoding on whatever other machine is writing to the same
directories.
If you don't change this setup, the images being decoded may well end up
corrupted.
- a 'locate pnga2area' shows that there is a copy in /Volumes:
ldm_virga% locate pnga2area
/Volumes/virgabackup/usr/local/src/decoders-4.1.0/bin/pnga2area
...
This decoder is for the PPC architecture:
ldm_virga% file /Volumes/virgabackup/usr/local/src/decoders-4.1.0/bin/pnga2area
/Volumes/virgabackup/usr/local/src/decoders-4.1.0/bin/pnga2area: Mach-O
executable ppc
The newly-built version of pnga2area is for Intel hardware as expected:
ldm_virga% file `which pnga2area`
/usr/local/unidata/decoders/bin/pnga2area: Mach-O executable i386
Interestingly, your Intel-based MacOS can run PPC executables:
ldm_virga% /Volumes/virgabackup/usr/local/src/decoders-4.1.0/bin/pnga2area
Example Usage:
/Volumes/virgabackup/usr/local/src/decoders-4.1.0/bin/pnga2area <-a
SATANNOT> <-b SATBAND> <-vxl-> IMGfile < infile
/Volumes/virgabackup/usr/local/src/decoders-4.1.0/bin/pnga2area <-a
SATANNOT> <-b SATBAND> <-vxl-> -d datadir -r pcode < infile
/Volumes/virgabackup/usr/local/src/decoders-4.1.0/bin/pnga2area <-a
SATANNOT> <-b SATBAND> <-vxl-> -f infile IMGfile
/Volumes/virgabackup/usr/local/src/decoders-4.1.0/bin/pnga2area <-a
SATANNOT> <-b SATBAND> <-vxl-> -f infile -d datadir -r pcode
Flags:
Help:
-h Print the help, then exit the program
Satellite definitions:
-a SATANNOT McIDAS SATANNOT pathname (def: /home/mcidas/data/SATANNOT)
-b SATBAND McIDAS SATBAND pathname (def: /home/mcidas/data/SATBAND)
-f infile Pathname for input PNG compressed file (default is stdin)
Logging:
-l logname Log file name (default uses local0; use '-' for stdout)
-v Verbose logging
-x Debug logging
...
So, building pnga2area on virga was apparently not needed at all!
Mind you, this exercise was valuable since:
- it pointed out some work that is needed in the 'configure' part of the
ldm-mcidas
package
- the v2006 version of pnga2area fixes some shortcomings in the v2004 version so
it is preferable to run the newer version
Final comment:
The current versions of McIDAS, GEMPAK, and _I believe_ the IDV can all use the
PNG-compressed AREA files directly. Because of this, we have turned off
decoding
of the images by pnga2area and replaced the actions by FILEs on a number of the
systems under our control. You may want to do the same _if_ you are not using
the
images for other applications that do not know how to deal with the compressed
imagery.
Please let me know if you have any questions and/or comments about the above.
Again, most of what I wrote was designed to be useful to me, not you, so it
may seem a bit obtuse. The possible issue of decoders on different machines
writing the same images to the same directory needs to be addressed.
Cheers,
Tom
****************************************************************************
Unidata User Support UCAR Unidata Program
(303) 497-8642 P.O. Box 3000
address@hidden Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage http://www.unidata.ucar.edu
****************************************************************************
Ticket Details
===================
Ticket ID: KQH-987099
Department: Support ldm-mcidas
Priority: Normal
Status: Closed