20020223: McIDAS dataflow (cont.)

>From: James Frysinger <address@hidden>
>Organization: College of Charleston
>Keywords: 200202210415.g1L4Fbx02565 McIDAS dataflow

Jim,

re: hectic
>Been kinda like that here, too. Two of the IEEE standards boards I'm on are 
>putting forth standards this spring. I'm the webmaster for both committees 
>and two standards have my name on them as chair. Can't remember if I've 
>mentioned it, but one might interest you folks. I have a "preview" site at
>   http://www.cofc.edu/~frysingj/binprefixes.html
>on one of them -- prefixes for binary multiples, as in gibibytes.

Interesting read!  I passed the URL along to our internal 'techies'
email list for wider consumption.

>The last couple of days we've been looking at a possible substitute for 
>X-window displaly. The need is being pushed by our Mac-lovers who still 
>haven't gotten their Mac OS-X setups to link to mcidas. Turns out that OS-X 
>apparently does not provide X-window.

Russ Rew, the head of our Systems and Development group, runs MacOS-X
on a laptop, and he does have X.  I know that he grabbed the X code
he is using off of the net, and has had no complaints about it yet
so I know that there are freely available solutions.

>So we're looking at vcn which is available at

>   http://www.uk.research.att.com/vnc/sshvnc.html

>I've seen a non-mcidas demonstration of it on Mac-to-Mac and it looks solid, 
>with two-way communications, cursor sensing, etc. Of course my suggestion is 
>to stop buying $3000 Macs and to buy $1200 Dells. Have you folks looked at 
>vnc?

Yes, several folks use it in house.  It is funny that you mention using
it on a Mac to view McIDAS since this is the exact same thing I did
to test VNC out a little more than a year ago (McIDAS was running on
a RedHat Linux box; I was driving it from a non-MacOS-X Mac).

re: NEXRCOMP data

>Neat! I was even able to blow it up and center it on KCHS!

Did you try the 1KN0R-NAT descriptor set (NEXRCOMP/1KN0R-NAT)?  This is
the 1 km National Composite.  I am still working on it and BAR since
that product contains reflectivities that range from -28 dBz up to
75 dBz.

>A couple of months ago you mentioned that raw GINI data would be coming out 
>on the servers. I haven't noticed it in the ldm-users mail (or any of the 
>others) but I won't claim to have read them very closely. How's that coming?

I am a little suprised by your comment since the GINI data has been there
all along.  The datasets with group names GINICOMP, GINIEAST, and GINIWEST
are all GINI data from NOAAPORT.  The MCGUI has access to these datasets
built in, and the Client Routing Table configuration widget allows one to
choose from among 7 different servers for those data.

>I spent several hours this afternoon (finally!) playing with IMGCOPY and 
>renaming files, like you mentioned in your 18 Dec 2001 13:27:22 -0700 
>message. I did notice that the image size defaults to 480 by 640 as you 
>warned me about so I allowed for that with a SIZ=600 800 argument on IMGCOPY. 
>Thanks for the heads up!

You can also copy entire images using the SIZE=ALL keyword.  Be careful
when doing this with the GINI 1 km VIS products and NEXRCOMP 1 km N0R
composites as they are LARGE: the VIS product is 26 MB and the N0R
composite is 14.4 MB!

>Now that I've done some image downloading, copying, and renaming manually 
>I'm starting to think about expediting the process. Looks like I can build a 
>repeatable string to IMGCOPY satellite image files.

Yup.

>At the end of a day 
>(eventually having the scheduler do this) I can batch copy a day's worth of 
>images. I can write a short bash script (that can be put into crontab) to 
>rename and relocate those AREA files to the non-AREA directories.

Everything can be run from cron.  I included a couple of skeleton scripts
in the 7.8 distribution that can be used (copied and renamed and edited)
to do this:

mcbatch.sh
mcrun.sh

These can be found in the ~mcidas/data directory after McIDAS is installed.

>(I set up 
>the DSSERVE BREEZE system you suggested for that -- thanks!) I haven't 
>figured out a way yet to pull the image times off the files to use as file 
>name clues for the user (me!) but that's not a biggie at the moment since 
>IMGLIST reads the image timestamps.

Getting the image times is not that hard.  IMGLIST gives it to you, and
you can send the output from IMGLIST to a file and then parse it to
get relevant info.  For broadcast of the Unidata-Wisconsin imagery
(LDM feedtype MCIDAS), I wrote a compression routine (area2png) and
uncompression routine (pnga2area) that have the smarts built in to
extract date/time/satellite/band info from the image and be able to use
it to name output files.  You may want to check out these routines.
Online help can be had from either by simply running the command with
no arguments.  Both routines are included with the ldm-mcidas package.
You are using pnga2area to decode the UW images from actions in your
~ldm/etc/pqact.conf file.

>After getting my repeatable string written for IMGCOPY i need to go read up 
>on scheduler (in the Users Manual?).

Yes, SKE, SKU are in the online Users Guide.  I would suggest, howeer,
use of cron with either of the two shell scripts I listed above.

>I'm curious as to whether it will run 
>after I exit mcidas and even log off as user, as long as I don't reboot or 
>shutdown the computer.

No, the McIDAS scheduler will not.  It runs in a McIDAS-X session.  This
is why I am recommending using cron.

>Anyway, that's where I am at the moment, Tom. Better quit playing and start 
>thinking about writing the test for Monday's Physics test and doing some lab 
>report grading.
>
>Take care and don't slip on the ice like those skaters in SLC keep doing!

Later...

Tom