[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
20051129: some AREA file not being updated (cont.)
- Subject: 20051129: some AREA file not being updated (cont.)
- Date: Tue, 29 Nov 2005 17:03:15 -0700
>From: Myron Kowalski <address@hidden>
>Organization: Moravian
>Keywords: 200511281523.jASFNF7s008982 ldm-mcidas McIDAS
Hi Myron,
>All the data we needed for the class was there this morning.
Excellent!
>Everything but 30 some files were upgraded, so I didn't
>touch anyting until after the class.
OK. I just jumped onto batboat and took a look at AREA files in
/mcidasdata/mcidas that are not being updated:
-rw-rw-rw- 1 ldm unidata 607856 Aug 8 12:18 AREA0064
-rw-rw-rw- 1 ldm unidata 607856 Aug 8 12:18 AREA0164
-rw-rw-rw- 1 ldm unidata 608096 Aug 8 11:04 AREA0063
-rw-rw-rw- 1 ldm unidata 607856 Aug 8 11:04 AREA0163
-rw-rw-rw- 1 ldm unidata 607856 Aug 8 09:37 AREA0062
-rw-rw-rw- 1 ldm unidata 607856 Aug 8 09:36 AREA0162
-rw-rw-rw- 1 ldm unidata 607856 Aug 8 08:37 AREA0061
-rw-rw-rw- 1 ldm unidata 607856 Aug 8 08:36 AREA0161
-rw-rw-rw- 1 ldm unidata 607856 Aug 8 07:37 AREA0060
-rw-rw-rw- 1 ldm unidata 607856 Aug 8 07:36 AREA0160
-rw-rw-rw- 1 ldm unidata 607856 Aug 8 07:04 AREA0069
-rw-rw-rw- 1 ldm unidata 607856 Aug 8 07:03 AREA0169
-rw-rw-rw- 1 ldm unidata 607856 Aug 8 05:37 AREA0068
-rw-rw-rw- 1 ldm unidata 607856 Aug 8 05:36 AREA0168
-rw-rw-rw- 1 ldm unidata 607856 Aug 8 04:37 AREA0067
-rw-rw-rw- 1 ldm unidata 608096 Aug 8 04:36 AREA0167
-rw-rw-rw- 1 ldm unidata 607856 Aug 8 03:37 AREA0066
-rw-rw-rw- 1 ldm unidata 607856 Aug 8 03:36 AREA0166
-rw-rw-rw- 1 ldm unidata 607856 Aug 8 02:37 AREA0065
-rw-rw-rw- 1 ldm unidata 607856 Aug 8 02:36 AREA0165
None of these will get upgraded since the Educational Floaters have
been removed from the Unidata-Wisconsin (IDD UNIWISC (aka MCIDAS))
datastream. That accounts for 20 of the files not being updated. They
should be deleted from /mcidasdata/mcidas.
re:
-rw-rw-rw- 1 ldm unidata 308528 Nov 27 13:05 AREA0199
-rw-rw-rw- 1 ldm unidata 308528 Nov 27 10:08 AREA0198
-rw-rw-rw- 1 ldm unidata 310576 Nov 27 07:05 AREA0197
These files will get updated as new Antarctic composite IR images
are received.
re:
-rw-r--r-- 1 ldm unidata 4506816 Nov 28 19:44 AREA0024
-rw-r--r-- 1 ldm unidata 4506096 Nov 1 11:40 AREA0020
-rw-r--r-- 1 ldm unidata 4506256 Jul 11 14:00 AREA0012
There are no products in the Unidata-Wisconsin datastream that will be
unpacked into AREA files of these names. These files must have been
created by someone for something specific or are a result of the messed
up ROUTE.SYS file. They should be deleted from /mcidasdata/mcidas.
re:
-rw-rw-rw- 1 ldm unidata 897280 Jun 19 2004 AREA0279
This will be updated when enough Mollweide global IR images have been
received. The image itself is a composite of the Mollweide global IR
images and topography. It is created by a McIDAS post process upon
receipt of the Mollweide global IR images.
re:
-rw-r--r-- 1 ldm unidata 2439056 Nov 26 00:41 AREA-13428820
-rw-r--r-- 1 ldm unidata 4506256 Nov 25 15:30 AREA-12728432
-rw-r--r-- 1 ldm unidata 4506256 Jul 10 20:21 AREA-13363388
-rw-r--r-- 1 ldm unidata 4506256 Jun 19 2004 AREA-13164684
-rw-r--r-- 1 ldm unidata 2439056 May 30 2004 AREA-13164680
-rw-r--r-- 1 ldm unidata 2439056 May 8 2004 AREA-13164664
-rw-r--r-- 1 ldm unidata 1155888 Oct 5 2000 AREA4194304
All of these files are BAD (the names do not conform to any
McIDAS convention). They should be deleted from /mcidasdata/mcidas.
re: *00271.IDX
-rw-rw-r-- 1 ldm unidata 353024 Sep 27 2000 MET00271.IDX
-rw-rw-r-- 1 ldm unidata 1060864 Sep 27 2000 SA00271.IDX
-rw-rw-r-- 1 ldm unidata 847872 Sep 27 2000 SM00271.IDX
-rw-rw-r-- 1 ldm unidata 421888 Sep 27 2000 ZZ00271.IDX
...
-rw-rw-r-- 1 ldm unidata 128 Sep 26 2000 MAT00271.IDX
-rw-rw-r-- 1 ldm unidata 192 Sep 26 2000 TON00271.IDX
-rw-rw-r-- 1 ldm unidata 128 Sep 26 2000 FWO00271.IDX
These files were created by XCD decoders back in 2000. Since they will
never be deleted by the McIDAS scour process (mcscour.sh run from
cron), they should be deleted by hand from /mcidasdata/mcidas.
re:
-rw-rw-r-- 1 ldm unidata 432640 Oct 31 2000 SCHEMA.103100
-rw-rw-r-- 1 ldm unidata 24000 Oct 11 2000 SYSKEY.TAB.1011
-rwxrwxr-- 1 ldm unidata 16384 Oct 11 2000 ROUTE.SYS.1011
-rw-rw-r-- 1 ldm unidata 24000 Oct 5 2000 SYSKEY.TAB.org
-rw-r--r-- 1 ldm unidata 1155888 Oct 5 2000 AREA4194304
-rw-rw-r-- 1 ldm unidata 14080 Oct 5 2000 ROUTE.SYS.org
-rw-rw-r-- 1 ldm unidata 7312 Nov 15 1999 MDRDEC.IDT
-rw-rw-r-- 1 ldm unidata 73420 Aug 7 1999 PIRPDEC.IDT
-rw-rw-r-- 1 mcidas unidata 24804 Aug 7 1999 IDXALIAS.DAT
These files should be deleted from /mcidasdata/mcidas when upgrading
to the current McIDAS release.
re:
-rw-rw-r-- 1 mcidas unidata 465408 Oct 31 2000 SCHEMA
This file will be replaced by a current station database file
when upgrading to the McIDAS release.
>I upgraded to the new ldm, just now and data is certainly
>streaming in now.
Sounds good!
>So now the ldm and the ldm-mcidas decoders are upgraded. I guess I
>still need to upgrade mcidas itself.
Sounds good.
>Thanks for your help on short notice.
No worries.
>I brought up a few things
>in mcidas and thought everything was ok until Joe came and said
>nothing he needed for his class was working. Oh, well. That the
>way schools work. Wait until there's just isn't enough time in
>the day left--then work a miracle.
:-)
One last comment (that I can make now that you are running LDM-6).
You can see real time statistics for the LDM ingestion on catwoman
in the following links:
Real time statistics - Site Index
http://www.unidata.ucar.edu/software/idd/rtstats/siteindex.php
catwoman.cs.moravian.edu
http://www.unidata.ucar.edu/software/idd/rtstats/siteindex.php?catwoman.cs.moravian.edu
IDS|DDPLUS latency
http://www.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?IDS|DDPLUS+catwoman.cs.moravian.edu
HDS latency
http://www.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?HDS+catwoman.cs.moravian.edu
A comparison of the HDS products latency (which is high) and the other
feeds (which are not at zero) indicates one of two potential problems:
- the feed from SUNY Brockport (vortex.esc.brockport.edu) is slow
- there is packet shaping (artificial volume limiting) being done on your
network connection
To test to see if the problem you are experiencing is the latter
possibility, I just changed the feed of HDS data from
vortex.esc.brockport.edu to atm.geo.nsf.gov.
If the latencies drop to zero, the problem is the feed from SUNY
Brockport.
If the latencies remain high, the problem is packet shaping somewhere
on your campus. If you intend to continue to ingest all of the HDS
data (NOAAPORT model output from NCEP), you will need to contact your
campus IT folks and get the packet shaping removed for port 388
traffic.
My suspicion is that the problem is being caused by packet shaping, but
I will be more confident in my theory after a couple of hours.
One last comment: Since you (Moravian) do not access data on an
ongoing basis, it may be advisable to stop ingesting data through the
IDD and switch to reliance on the remote access of data by McIDAS.
Another site did this several years back and has never regretted the
change. The concept is that McIDAS can access data from any open
access ADDE server that is reachable by TCP/IP Ethernet. There are
several such servers in the Unidata community that are open to all and
maintained with current data. One user that was using McIDAS in much
the same way that you appear to be switched to this setup several years
ago and has been happy ever since: all he has to do now is maintain
McIDAS. Please bounce this idea off of Joe to see what he thinks.
Cheers,
Tom
--
NOTE: All email exchanges with Unidata User Support are recorded in the
Unidata inquiry tracking system and then made publicly available
through the web. If you do not want to have your interactions made
available in this way, you must let us know in each email you send to us.