[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[IDD #QVQ-842701]: LDM of Seoul National University has problem
- Subject: [IDD #QVQ-842701]: LDM of Seoul National University has problem
- Date: Fri, 16 Jan 2009 15:06:24 -0700
Hi Euihyun,
I apologize for taking so long to respond to your email. A combination
of a small problem with our inquiry tracking system and my being away
at the AMS Annual meeting this week prevented a response before now.
re:
> I appreciated that your effort to set our LDM system up.
> My professor is Gyu-Ho Lim, who visited NCAR on last December.
No worries.
> However, our LDM system has some problems.
> We can find no data from upstream.
> And We find some error message from log file in data folder.
> I attached log file of my LDM system.
I took a look at your log file to see what the problem could
be. In order to more fully investigate the problem, I decided to
logon to your machine, ssmgwdss.snu.ac.kr as user 'ldm'.
I see that you are receiving COSMIC data on the EXP feed in spite
of the errors you reported in from ~ldm/logs/ldmd.log file:
<as 'ldm'>
cd ~ldm
notifyme -vl- -f EXP -o 10000
Jan 16 21:00:36 notifyme[15777] NOTE: Starting Up: localhost:
20090116181356.200 TS_ENDT {{EXP, ".*"}}
Jan 16 21:00:36 notifyme[15777] NOTE: LDM-5 desired product-class:
20090116181356.200 TS_ENDT {{EXP, ".*"}}
Jan 16 21:00:36 notifyme[15777] INFO: Resolving localhost to 127.0.0.1 took
0.00049 seconds
Jan 16 21:00:36 notifyme[15777] NOTE: NOTIFYME(localhost): OK
Jan 16 21:00:39 notifyme[15777] INFO: 15044 20090116195004.988 EXP 000
/pub_out/cosmicrt/level2/wetPrf/2009.016/wetPrf_C005.2009.016.16.10.G11_0001.0002_nc
Jan 16 21:00:39 notifyme[15777] INFO: 15044 20090116195005.032 EXP 000
/pub_out/cosmicrt/level2/wetPrf/2009.016/wetPrf_C005.2009.016.16.12.G23_0001.0002_nc
Jan 16 21:00:39 notifyme[15777] INFO: 15044 20090116195005.038 EXP 000
/pub_out/cosmicrt/level2/wetPrf/2009.016/wetPrf_C005.2009.016.16.13.G15_0001.0002_nc
Jan 16 21:00:39 notifyme[15777] INFO: 15044 20090116195005.067 EXP 000
/pub_out/cosmicrt/level2/wetPrf/2009.016/wetPrf_C005.2009.016.16.15.G10_0001.0002_nc
...
The problem is that there is no action defined for what to do with the
products once they are received.
A little bit of background is needed. The LDM performs two primary
functions:
- move products from one machine to another
- do something with the products once they are received
The configuration needed to get products from an "upstream" (feeder) machine
is to add an appropriate line in the ~ldm/etc/ldmd.conf configuration file.
This was done on ssmgwdss.snu.ac.kr for two different streams of data:
- NOAAPort-delivered global observational data and low to moderate resolution
model output (this was done mainly for testing purposes)
- GPS (radio occultation) data from COSMIC
The ~ldm/etc/ldmd.conf entries that make the appropriate requests are:
REQUEST WMO ".*" idd.unidata.ucar.edu
REQUEST EXP ".*" cosmic-io.cosmic.ucar.edu
The first entry requests the NOAAPort-delivered global observational data and
low to moderate resolution model output. The second entry requests all of
the data provided by COSMIC.
I did not setup any processing actions for either of these types of data
when I installed the LDM when Dr. Gyu-Ho Lim was in Boulder because I did not
know what he/you wanted to do with the data.
Question:
- what would you like to do with the COSMIC GPS data once it is received
at SNU?
In order to demonstrate that the data are being received, I just added
a simple FILE action to the "standard" pattern-action file, ~ldm/etc/pqact.conf.
The action I added is at the very end of the file and looks like:
#
# Example action added to write the COSMIC GPS data products to disk files
#
# Typical Product Headers:
#
#
/pub_out/cosmicrt/level2/wetPrf/2009.016/wetPrf_C005.2009.016.19.12.G15_0001.0002_nc
#
/pub_out/cosmicrt/level2/bfrPrf/2009.016/bfrPrf_C005.2009.016.16.10.G11_0001.0006_bufr
#
# History: 20090116 - Added FILE action for EXP-delivered products
# (Tom Yoksas, Unidata User Support)
#
EXP (...Prf)_(.*)_(bufr|nc)
FILE -close data/COSMIC/\1/\2_\3
NB: certain whitespace in pattern-action file entries MUST be tabs, not spaces:
- the whitespace between EXP and (...Prf) is a tab
- the whitespace before FILE is a tab
- the whitespace between FILE and -close is a tab
- the whitespace between -close and data/COSMIC is a tab
This creates the directories:
~ldm/data/COSMIC/wetPrf
~ldm/data/COSMIC/bfrPrf
In the ~ldm/data/COSMIC/wetPrf directory will be files that look like:
wetPrf_C004.2009.016.19.30.G24_0001.0002_nc
wetPrf_C004.2009.016.19.31.G29_0001.0002_nc
wetPrf_C004.2009.016.19.32.G19_0001.0002_nc
wetPrf_C004.2009.016.19.37.G02_0001.0002_nc
wetPrf_C004.2009.016.19.52.G07_0001.0002_nc
wetPrf_C004.2009.016.19.52.G25_0001.0002_nc
wetPrf_C004.2009.016.20.07.G05_0001.0002_nc
...
These files are in netCDF format.
In the ~ldm/data/COSMIC/bfrPrf directory there will be fills that look like:
bfrPrf_C004.2009.016.18.25.G26_0001.0006_bufr
bfrPrf_C004.2009.016.18.35.G05_0001.0006_bufr
bfrPrf_C004.2009.016.18.39.G19_0001.0006_bufr
bfrPrf_C004.2009.016.18.41.G10_0001.0006_bufr
bfrPrf_C004.2009.016.18.42.G30_0001.0006_bufr
...
These files are in BUFR format.
Because new data files will be written as data are received, I also turned on
data scouring so your disk would not eventually fill. I did this by
adding an entry to 'ldm's crontab file that runs the LDM 'scour' utility once
per day:
MAILTO=""
#
# 20090116 - Run the LDM 'scour' utility at 9:30 local time (currently 0:30 UTC)
#
30 9 * * * bin/ldmadmin scour
I configured 'scour' by editing ~ldm/etc/scour.conf. The scour.conf
configuration
lines I added are:
#
# COSMIC data - keep 2 days of data online
#
~ldm/data/COSMIC 2
The LDM pattern-action file entries can be configured to do almost anything
you can imagine. For more information on this and other LDM topics, please
read through the LDM documenation at:
http://www.unidata.ucar.edu/software/ldm
> Thank you for your help.
No worries.
Please let us know if/when you have questions on the LDM or on processing of
products received through actions in LDM pattern-action files.
Cheers,
Tom
--
****************************************************************************
Unidata User Support UCAR Unidata Program
(303) 497-8642 P.O. Box 3000
address@hidden Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage http://www.unidata.ucar.edu
****************************************************************************