[IDV #WRY-586577]: IDV - Error loading MADIS METARS

Hi Bill-

> Thanks for the effort on this. I've been playing around with my data files as 
> well and came up with the attached file that works. When I saved the attached 
> file as a .csv from Excel it doesn't have any spaces between the commas but 
> it works fine.

The test file you sent today doesn't have any missing fields.  It's
only when you have missing fields that this is a problem which was the
case in the previous file you sent.  An alternative is to have
a missing value in empty cells and use the miss= keyword in the header.

> Looks like I need to separate the original data file into multiple files 
> based on sensor height. That's easy to do with a macro in Excel. It's 
> important for me to be able to display one sensor height on the map at one 
> time based on the way the forecasters issue and verify warnings.

I figured that.  My approach was just a kludge.

> I was just getting to the date/time issue - IDV needs to have the zeros added 
> to each time so there are 8 digits per time (for HHmmss). I haven't yet 
> figured out the easiest way to do a mass conversion of the times using an 
> Excel macro. Also, the only way Excel will display zeros in front of other 
> digits is to change the format of the number from "Number" to "Text" but 
> that's easy to do as well. Did you "pad" the times with zeros manually just 
> for the test or did you do it with a script - are you using Linux or Windows 
> to manipulate the data?

vi is a wonderful tool. ;-)  I just did it manually to test.

> I also wrote a macro to add lat/lon. That's my last step in the process once 
> I have the rest of the data file setup. I copy/paste once manually in Excel 
> while recording the macro and then I can just run it as the last macro on the 
> rest of the data files. The station ID's are a bit flaky. As you noticed in 
> the "SIDE" column some say "SE" or "NW". That's because there are dual 
> sensors on opposing sides of the towers on some towers because the tower 
> itself will disturb the flow to the sensor. So if the winds are easterly, the 
> forecasters will use the sensors on the SE side, not the NW side. So towers 
> 20 and 21 are virtual - they are really tower 2 where 20 is the NW side and 
> 21 is the SE side (or vice versa). So tower locations 20 and 21 will use the 
> lat/lon of tower 2. Same with tower 110: it has two virtual towers as 1101 
> and 1102. And there are others.
> 
> The data is originally in ASCII text files. I'm just using Excel to 
> manipulate the files with macros and then save them as .csv. It seems to be 
> working okay - I just need to figure out how to convert the times to 8 digits.

As your other note said, it's 6 digit time.  I just typed too many 00's. ;-)
You can use the Excel date/time formatting to get something like 0:00:00 and
then change the fmt string in the header to match what you use.  That worked for
me.

> Data retrieval, manipulation and display is always a long pole for the work 
> we do. Once all the data is acquired, nicely formatted and displayed, the 
> analysis doesn’t take so long. I've asked the rest of my staff to download 
> and install IDV to start working with it. I intend to make that our primary 
> display software rather than GEMPAK or GrADS. Can IDV be setup to 
> automatically ftp real time data, produce maps and then be uploaded to a 
> location (say a web site)?

Glad to hear that you are finding IDV useful.

One last note on this importing.  If you save the file off as a CSV without
the header info and read it in to the IDV, you will be prompted with a dialog
that allows you to set the header information.  The advantage here is that you
can then save this off and use it as a resource to apply to other files of
similar format.   This "wizard" still needs some work, but it might help
you out.

Don

Ticket Details
===================
Ticket ID: WRY-586577
Department: Support IDV
Priority: Normal
Status: Open