20010123: Unidata-Wisconsin satellite image decode problem

>From:  Robert Ballentine <address@hidden>
>Organization:  SUNY Oswego
>Keywords:  200101231555.f0NFtue06122 LDM pqact.conf pnga2area Unidata-Wisconsin

Bob,

>Back in September you installed McIdas on met-05.oswego.edu which is our
>server running the LDM under Red Hat linux 5.2.  It was working beautifully
>until Sunday (day before yesterday).

I saw your message in support, but I'll have to admit that I only scanned
it.  Things have been pretty hectic here as we just moved back in after
being out of our offices for 7 weeks (remodel).  Things are slowly getting
back to normal...

>Somehow, I managed to overwrite my
>pqact.conf file, and when I restarted the LDM, we stopped getting satellite
>imagery and radar.  I had a backup pqact.conf from September 12 which looks
>OK to me.  Everything else seems to be coming in and I get no error messages
>in ldmd.log except for the RECLASS stuff when our network is slow.  But we're
>not even getting sat pix in the middle of the night when network traffic is
>much lighter.

OK.

>   The LDM is not even creating a png2area.log file in ~/data/
>gempak/logs directory.  My ldmd.stats file looks like:
>
>20010123144813     HDS    ingestor.wunderground.com        345    1896624
>3445.64 3606@3022 20010123154801
>20010123144949     HDS    noaaport.unidata.ucar.edu       4891   77949265
>2159.65 3663@4718 20010123155043
>20010123144944     HDS       sunshine.ssec.wisc.edu       1589   29072707
>2179.42 3661@3320 20010123155041
>20010123144807 IDS|DDPLUS    ingestor.wunderground.com        309     282990
>   3214.76 3606@3022 20010123154740
>20010123144950 IDS|DDPLUS    noaaport.unidata.ucar.edu       3087    3422153
>   2133.85 3660@3048 20010123155046
>20010123144950 IDS|DDPLUS       sunshine.ssec.wisc.edu       1248    1321222
>   2039.49 3656@2915 20010123155046
>20010123145030  MCIDAS        unidata.ssec.wisc.edu         13    6317493
>2900.47 3594@3320 20010123154945
>20010123144228    FSL2            hero.fsl.noaa.gov          8    1446924
>2634.36 3637@3154 20010123154141
>20010123145832     WSI        sysu1.uni.wsicorp.com        139    1175336
> 338.20  340@5658 20010123150410
>
>   Would you be able to check my LDM to see if there is anything wrong? I
>am enclosing my ldmd.conf, pqact.conf, and ldmd.log files from today.

OK.  I took you up on your offer of loggin on (from home no less; I have
DSL which gives me 1284 Kb down!).

>   Sorry to be a pest, but we really love having the sat pix for our student
>forecasters and for met labs.

You are not a pest at all.  I apologize for not being able to jump on this
when I first saw it yesterday.

OK, so I did the following:

o added an allow for Unidata machines in your ldmd.conf file and then
  stopped and restarted your LDM.  This allows us to run notifyme from
  here and monitor what you are getting from your upstream sites

o edited pqact.conf.  I made a couple of changes, but only one should
  have any effect on your decoding of Unidata-Wisconsin imagery:

  o changed the pqact.conf actions for pnga2area from something like:

MCIDAS  ^pnga2area Q. UI ... (.*) (.*) (.*) (........) (....)

    to:

MCIDAS  ^pnga2area Q. UI .* (.*) (.*) (.*) (........) (....)

    This will allow for cases where the default AREA file number
    in the header varies from 4 digits

  o found that the first pnga2area action had one bad line:

# GOES East North America Infrared - McIDAS product code UI

>MCIDAS ^pnga2area Q. UI ... (.*) (.*) (.*) (........) (....)
>       PIPE    -close
>       /apps/ldm/util/pnga2area
>        -a /apps/ldm/etc/SATANNOT -b /apps/ldm/etc/SATBAND
>       -l data/gempak/logs/pnga2area.log 
>       data/gempak/images/sat/GOES-8/\3/IR/IR_\4_\5

    The line '-a /apps/ldm/etc...' was prefixed with blanks, not a tab.
    This could have the effect of making every action past this unusable,
    and that would explain why decoding stopped on met-05.

As I write this, I am waiting for Unidata-Wisconsin imagery to be received
by your machine to see if the single error in pqact.conf was the cause
of all of your problems.

<later>

Well, those changes had no effect!?

I reedited ~ldm/etc/pqact.conf to see what I could see (again).  I
tried modifying some inconsequential lines and then did a save from
vi when I noticed the following message:

"pqact.conf" [dos format] 683 lines, 25748 characters written

So, the backup copy of pqact.conf was in DOS format?  Now grasping at
straws, I FTPed your pqact.conf back to Unidata (in binary mode) and
did an octal dump on it.  I noticed that every line ended with '\r\n'
which is apparently indicative of a DOS format file.  I converted the
file on our Solaris SPARC machine to Unix format (using dos2unix)
and then redid the od.  This time the lines end with '\n'.

I FTPed the converted pqact.conf file back to met-05 and did:

<login as 'ldm'>
cd etc
cp pqact.conf pqact.dos
mv pqact.unix pqact.conf
ldmadmin pqactcheck
<no errors>
kill -HUP 4313   (4313 was the process ID of pqact)

After waiting a couple of minutes, I saw a PNG compressed image come in
and get decoded!  Also, the log file pnga2area.log is now correctly being
updated in the ~ldm/data/gempak/logs directory.

So, it looks like the culprit was that your backup copy pqact.conf was
in a format that was confusing pnga2area!

Tom

>From: Robert Ballentine <address@hidden>
>Date: Tue, 23 Jan 2001 16:22:58 -0500 (EST)
>Subject: Satellite pix are back

>Tom:

>Thank-you very much for restoring our satellite imagery.  I guess I won't
>back up any more configuration files under DOS.

>Bob