20040610: importing and contouring data on GOES imagery (cont.)

>From: Owen Cooper <address@hidden>
>Organization: Aeronomy Laboratory/NOAA
>Keywords: 200406080008.i5808gtK000586 McIDAS SCHEMA Flexpart TXT2MD

Hi Owen,

re: I will look at the data this afternoon or tomorrow
Well, I was able to get to this quicker than I thought...

>Well, I'm just not having any luck at all.
>I changed the format of  my text files so that the date is in ccyyddd 
>format and the time is in hh:mm:ss format.

These should be OK.

>And I created a new schema file but I ran into several problems
>1) Using PTLIST, the date is listed as 0.000000
>2) the maximum number of columns allowed is 3500 but my data file has 
>   6000+ entries.
>3) When I reduced the number of entries to below 3500 the data did plot 
>   with PTCON but the date was listed as 0.0 and the time was listed as 
>   22.37Z even though it should have been 00Z.
>
>I have attached a sample data file and the SCHEMA file.  Could I ask you 
>the favor of trying to modify the schema so that the data will plot?
>I would really appreciate it!

Here goes:

>1) Using PTLIST, the date is listed as 0.000000
1) I took your schema, DCFPOZ and noticed that its format was screwed
   up, and you had specified SCALE factors for DAY and TIME which should
   have none.

   Second, to make life easier, the schema file no longer has to be
   in McIDAS LW format; it can be an ASCII text file.

   Given these two things, I recreated your schema as an ASCII text
   file, FPOZ.TXT:

"       NAME    VSN     DATE    ID      TEXTID
"       ----    ---     ----    --      ------
SCHEMA  FPOZ    1       04156   0       "FLEXPART COLUMN O3
 
"KEY    SCALE   UNITS   DESCRIPTION
"---    -----   -----   -----------
 
ROWS    1               "1 time per file
 
DAY             CYD     "Year and julian day
TIME            HMS     "nominal time
 
COLUMNS 3500            "# ! OF COLUMNS
 
DATA                    "START OF DATA SECTION
 
LAT     4       DEG     "LATITUDE (DECIMAL DEG)
LON     4       DEG     "LONGITUDE (DECIMAL DEG)
O3      4       CON     "COLUMN O3
 
ENDSCHEMA

  I then put the schema and your data file in the ~mcidas/workdata
  directory (since I was working as the user 'mcidas'; if I was not the
  user 'mcidas', I would have put them in ~/mcidas/data).

  Next, I added the schema to the SCHEMA file.  I did this after
  copying SCHEMA from ~mcidas/data to ~mcidas/workdata (so I wouldn't
  change the schema file sent in the McIDAS distribution), again
  in workdata since I was working as the user 'mcidas'.

  Next, I registered the FPOZ schema:

SCHE FPOZ.TXT
FPOZ.TXT     will be treated as a text file
"       NAME    VSN     DATE    ID      TEXTID
"       ----    ---     ----    --      ------
SCHEMA  FPOZ    1       04156   0       "FLEXPART COLUMN O3
*--SCHEMA * FPOZ *, VERSION * 1 * ADDED TO SCHEMA FILE *  KEYS=5
 --END OF SCHEMA REGISTRATION PROGRAM
 
  Then I verified that it had been correctly registered:
 
LSCHE FPOZ

NAME: FPOZ  VERSION:  1  DATE: 2004156  TEXTID: "FLEXPART COLUMN O3
----        -------
   DEFAULT NUMBER OF ROWS:    1                        INTEGER ID: 0
                     COLS:   3500                   MISSING DATA VALUE:  
-2139062144
   REPEAT GROUP:  NUMBER OF REPETITIONS:   1
                      STARTING POSITION:   3
                                   SIZE:   3
   NUMBER OF KEYS IN ROW HEADER:     2
                     COL HEADER:     0   STARTING AT POSITION   3
                    DATA RECORD:     3   STARTING AT POSITION   3
                                  ----
                                     5 TOTAL

        KEY  SCALE UNIT     KEY  SCALE UNIT     KEY  SCALE UNIT
        ---- ----- ----     ---- ----- ----     ---- ----- ----
        DAY    0   CYD      TIME   0   HMS      LAT    4   DEG
        LON    4   DEG      O3     4   CON
LSCHE: DONE

   Next, I made sure that the intended output MD file did not exist:

DMAP MDXX1234
PERM      SIZE LAST CHANGED FILENAME DIRECTORY
---- --------- ------------ -------- ---------
0 bytes in 0 files

  If it had, I would have either deleted it or tried a different
  output MD file.

  Next, I create the output file from the registered schema and
  input data file:

TXT2MD ST_7100.FLX 1234 FPOZ
 TXT2MD -- BEGIN
NEW MD file: 1234
 TXT2MD -- DONE:  1234

  So, the output data file exists as MDXX1234.

  Next, I made sure that I had defined the MYDATA/PTSRCS dataset
  so I can look at the data using ADDE access:

DATALOC ADD MYDATA LOCAL-DATA
BATCH MYDATA.BAT

  Next, since I noticed that the data lies in a LAT,LON band, I
  drew a map that covered the desired area:

MAP DEF 1 LAT=0 70 LON=-30 130

  Finally, I contoured the data put into MDXX1234 using PTCON:

PTCON MYDATA/PTSRCS.1234 NAV=C CINT=5 COL=3 PAR=O3 SEL='TIME 0'

  The plot worked fine:

http://my.unidata.ucar.edu/content/staff/tom/gifs/flex_o3.gif

>2) the maximum number of columns allowed is 3500 but my data file has 
>   6000+ entries.
2) the maximum number of columns in an MD file was 3500 when TXT2MD
   was first written.  This fact was hardcoded into the source file:

~mcidas/mcidas2003/src/txt2md.pgm

   in the line:

      parameter(MAXCOL = 3500 )

   You can go in and change this definition to a larger value.  I
   see that the SSEC v2004 version of txt2md.pgm uses 7000, so
   I would recommend changing it to 7000.  After making the change
   you have to rebuild and install the TXT2MD executable, txt2md.k:

<as 'mcidas'>
cd ~mcidas/mcidas2003/src
<modify txt2md.pgm as per recommendation>
make txt2md.k                <- VENDOR compilers
make txt2md.K VENDOR=-g77    <- gcc/g77
make txt2md.K VENDOR=-gcc    <- gcc/f2c/mcfc
rm ~mcidas/bin/txt2md.k
ln txt2md.k ~/mcidas/bin

>3) When I reduced the number of entries to below 3500 the data did plot 
>   with PTCON but the date was listed as 0.0 and the time was listed as 
>   22.37Z even though it should have been 00Z.
3) I assume that the problem you report in 3) is related to the schema
   being dinged.  I did not see this problem in the data I plotted
   as you can see from the GIF (tm) file I created (URL above).

By the way, did you notice that your 1) had one line; 2) had two lines;
and 3) had 3 lines.  Very interesting ;-)

Cheers,

Tom
--
NOTE: All email exchanges with Unidata User Support are recorded in the
Unidata inquiry tracking system and then made publically available
through the web.  If you do not want to have your interactions made
available in this way, you must let us know in each email you send to us.