[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20010626: Surface Obs - Data Availability and Request from SCRIPPS

------- Forwarded Message

From: address@hidden
To: address@hidden
Subject: Surface Obs - Data Availability and Request
Date: Tue, 26 Jun 2001 13:24:59 -0600
Organization: UCAR/Unidata

I want to make the community aware of the availability of some data here at 
Scripps and to ask for some data.  If you're not interested in the 
availability of flat ASCII observation files and don't have archives of raw 
"surface airways" format obs, you'll be bored by the remainder of this message.

First, my request.  I've processed all of my METAR obs files (Jul 1996, et 
seq) and as many SAO format files as I have (May 1993 to Jun 1996, with 
many holes).  Does anyone have an archive of raw SAO format obs they'd be 
will to share with CRD?

Now, the data files.  Since the Climate Research Division's "mission" is 
almost totally research, we look at data somewhat differently than do most 
teaching departments.  We need long time series of data at single or 
multiple selected stations.  In my limited (truthfully, very limited) 
experience with GEMPAK, it appears that a synoptic view is easy, but a time 
series view (especially multi-year) can be difficult.

It's also difficult to share data with non-GEMPAK collaborators.  Most of 
our collaborators have never even heard of GEMPAK or NAWIPS.  So, I needed 
surface obs I could manipulate easily and share with anyone.

I wrote some PERL scripts that take the raw METAR and SAO obs and process 
them so that all headers, trailers, and non-printable (sub-32) characters 
are stripped.  Then multi-line observations are concatenated to single 
lines.  (Most mail programs will wrap lines in the mid-process example 
below, but each obs is in a single record (or line).)  After that, the obs 
are converted to constant position fields.

The raw METAR obs:

KELY 261153Z AUTO 00000KT 10SM -RA OVC075 14/04 A3009 RMK AO2
      RAB43 SLP084 P0000 60000 T01390044 10194 20133 50005=
KMSY 261153Z 00000KT 6SM HZ FEW020TCU SCT060 SCT110 SCT250 24/20
      A3016 RMK AO2 SLP214 TCU DSNT S T02440200 10244 20222
      53018=
KSTC 261153Z AUTO 18011KT 10SM TS SCT060 22/19 A3007 RMK AO2 RAE32
      SLP184 P0001 60001 70001 T02220194 10267 20222 53021=
KMSP 261153Z 20009KT 8SM FEW120 SCT200 BKN250 25/19 A3010 RMK AO2
      SLP185 T02500194 10272 20239 53015=

Mid-process (single line) obs:

KELY 261153Z AUTO 00000KT 10SM -RA OVC075 14/04 A3009 RMK AO2 RAB43 SLP084 
P0000 60000 T01390044 10194 20133 50005=
KMSY 261153Z 00000KT 6SM HZ FEW020TCU SCT060 SCT110 SCT250 24/20 A3016 RMK 
AO2 SLP214 TCU DSNT S T02440200 10244 20222 53018=
KSTC 261153Z AUTO 18011KT 10SM TS SCT060 22/19 A3007 RMK AO2 RAE32 SLP184 
P0001 60001 70001 T02220194 10267 20222 53021=
KMSP 261153Z 20009KT 8SM FEW120 SCT200 BKN250 25/19 A3010 RMK AO2 SLP185 
T02500194 10272 20239 53015=

Decoded constant position obs:

ICAO  YYMMDDHHMM  Dir  Spd  Gst    Vsby     Prcp   TAir   TDew    SLP 
SkyCvr  PrWX
KELY  0106261153  000    0    0   10.00    0.001   13.9    4.4  1008.40 
OVC075  -RA
KMSY  0106261153  000    0    0    6.00   -9.999   24.0   20.0  1021.40 
SCT060  HZ
KSTC  0106261153  180   11    0   10.00    0.254   22.2   19.4  1018.40 
SCT060  TS
KMSP  0106261153  200    9    0    8.00   -9.999   25.0   19.4  1018.50 
BKN250

It is then easy to GREP a single station out of multiple hourly, daily, 
monthly, or annual files to see a time series view:

ICAO  YYMMDDHHMM  Dir  Spd  Gst    Vsby     Prcp   TAir   TDew    SLP 
SkyCvr  PrWX
KSTC  0106260753  190    9    0   10.00   -9.999   25.6   17.2  1015.90 
CLR
KSTC  0106260853  180    7    0   10.00   -9.999   26.0   17.0  1016.00 
OVC095
KSTC  0106260953  190    7    0   10.00   -9.999   25.0   18.0  1016.20 
BKN075  TS
KSTC  0106261053  110    4    0   10.00    0.001   23.9   18.3  1017.50 
BKN070  -TSRA
KSTC  0106261153  180   11    0   10.00    0.254   22.2   19.4  1018.40 
SCT060  TS
KSTC  0106261253  190    6    0   10.00   -9.999   22.2   19.4  1018.20 
SCT085  TS
KSTC  0106261353  180    6    0   10.00    0.001   23.0   19.0  1018.90 
CLR

The scripts are not elegant, almost certainly are buggy, and I've made some 
decisions that will set some peoples teeth on edge, but they do 
work.  FORTRAN and other PERL scripts have little trouble reading them.  An 
example of one of my decisions, in the decoded files, the wind gust is the 
speed in excess of the sustained, where 15G25 will show as 
"  15  10".)  The SkyCvr column shows the lowest layer with greatest coverage.

That means that not all of the data in the complete obs will be in the 
decoded record.  One of the immutable laws of the universe is "Something's 
lost when something's gained."  That's why I retain the mid-process files.

The current months files are available from CRD's web site.  Prior months 
are available by request.  I've written two versions of each script.  One 
processes METAR and the other does SAO obs.  If you're interested in these 
data files (or have any questions) please contact me.

Larry

      ---===---=-=-=-=-=-=-=-=-=-=-=====[\/]=====-=-=-=-=-=-=-=-=-=-=---===---
    -----===(*  Climate's what we expect, but weather's what we 
get.  *)===-----
   Larry Riddle : Climate Research Division : Scripps Institution of 
Oceanography
       University of California, San Diego : La Jolla, California  92093-0224
       Phone: (858) 534-1869 : Fax: (858) 534-8561 : E-Mail: address@hidden


------- End of Forwarded Message