[IDV #WRY-586577]: IDV - Error loading MADIS METARS

hey Bill-

> Thought I had it figured out. See attached .csv file. After opening the file 
> with the Data Chooser and then choosing Point Data Plot in the Field Selector 
> I get this error. What did I do wrong? Is this the issue with missing fields 
> and needing an extra space between commas?

Yup.  I'm working on a fix for this but don't know when I'll have
it done.  Any way you can work your Excel magic to put in a 
missing value (e.g. -9999) for the blank cells?  Then just add
it to the header.  

Don

> An error has occurred:
> Creating obs
> java.util.NoSuchElementException
> 
> java.util.NoSuchElementException
> at java.util.StringTokenizer.nextToken(StringTokenizer.java:332)
> at visad.data.text.TextAdapter.readit(TextAdapter.java:844)
> at visad.data.text.TextAdapter.<init>(TextAdapter.java:202)
> at 
> ucar.unidata.data.point.TextPointDataSource.makeObs(TextPointDataSource.java:211)
> at 
> ucar.unidata.data.point.TextPointDataSource.makeObs(TextPointDataSource.java:168)
> at 
> ucar.unidata.data.point.PointDataSource.getDataInner(PointDataSource.java:425)
> at 
> ucar.unidata.data.point.TextPointDataSource.getDataInner(TextPointDataSource.java:1188)
> at ucar.unidata.data.DataSourceImpl.getData(DataSourceImpl.java:2005)
> at ucar.unidata.data.DirectDataChoice.getData(DirectDataChoice.java:333)
> at ucar.unidata.data.DataChoice.getData(DataChoice.java:618)
> at ucar.unidata.data.DataInstance.getData(DataInstance.java:239)
> at ucar.unidata.data.DataInstance.getData(DataInstance.java:203)
> at ucar.unidata.data.DataInstance.dataOk(DataInstance.java:282)
> at ucar.unidata.data.point.PointDataInstance.init(PointDataInstance.java:88)
> at ucar.unidata.data.point.PointDataInstance.<init>(PointDataInstance.java:74)
> at 
> ucar.unidata.idv.control.ObsDisplayControl.doMakeDataInstance(ObsDisplayControl.java:766)
> at 
> ucar.unidata.idv.control.DisplayControlImpl.initializeDataInstance(DisplayControlImpl.java:2871)
> at 
> ucar.unidata.idv.control.DisplayControlImpl.setData(DisplayControlImpl.java:2853)
> at 
> ucar.unidata.idv.control.StationModelControl.setData(StationModelControl.java:1405)
> at 
> ucar.unidata.idv.control.StationModelControl.init(StationModelControl.java:423)
> at 
> ucar.unidata.idv.control.DisplayControlImpl.init(DisplayControlImpl.java:1225)
> at 
> ucar.unidata.idv.control.DisplayControlImpl.init(DisplayControlImpl.java:963)
> at ucar.unidata.idv.ControlDescriptor.initControl(ControlDescriptor.java:948)
> at ucar.unidata.idv.ControlDescriptor$1.run(ControlDescriptor.java:875)
> at ucar.unidata.util.Misc$3.run(Misc.java:1011)
> 
> -----Original Message-----
> From: Unidata IDV Support [mailto:address@hidden]
> Sent: Thursday, November 08, 2007 11:18 AM
> To: Bauman.Bill
> Cc: address@hidden; address@hidden
> Subject: [IDV #WRY-586577]: IDV - Error loading MADIS METARS
> 
> Bill-
> 
> > As for the time converter in Excel, if the time in the cell is 500 (which 
> > is 00:05:00, or 5 minutes after 00 UTC) and you change the Excel format to 
> > time or custom (h:mm:ss), it returns 00:00:00, not 00:05:00. Any 
> > suggestions?
> 
> Instead of Time, use Custom and an a Type of 000000.  That seems
> to work.  Then you can use my original fmt of HHmmss
> 
> Don
> 
> > -----Original Message-----
> > From: Unidata IDV Support [mailto:address@hidden]
> > Sent: Thursday, November 08, 2007 10:55 AM
> > To: Bauman.Bill
> > Cc: address@hidden; address@hidden
> > Subject: [IDV #WRY-586577]: IDV - Error loading MADIS METARS
> >
> > Hi Bill-
> >
> > > Thanks for the effort on this. I've been playing around with my data 
> > > files as well and came up with the attached file that works. When I saved 
> > > the attached file as a .csv from Excel it doesn't have any spaces between 
> > > the commas but it works fine.
> >
> > The test file you sent today doesn't have any missing fields.  It's
> > only when you have missing fields that this is a problem which was the
> > case in the previous file you sent.  An alternative is to have
> > a missing value in empty cells and use the miss= keyword in the header.
> >
> > > Looks like I need to separate the original data file into multiple files 
> > > based on sensor height. That's easy to do with a macro in Excel. It's 
> > > important for me to be able to display one sensor height on the map at 
> > > one time based on the way the forecasters issue and verify warnings.
> >
> > I figured that.  My approach was just a kludge.
> >
> > > I was just getting to the date/time issue - IDV needs to have the zeros 
> > > added to each time so there are 8 digits per time (for HHmmss). I haven't 
> > > yet figured out the easiest way to do a mass conversion of the times 
> > > using an Excel macro. Also, the only way Excel will display zeros in 
> > > front of other digits is to change the format of the number from "Number" 
> > > to "Text" but that's easy to do as well. Did you "pad" the times with 
> > > zeros manually just for the test or did you do it with a script - are you 
> > > using Linux or Windows to manipulate the data?
> >
> > vi is a wonderful tool. ;-)  I just did it manually to test.
> >
> > > I also wrote a macro to add lat/lon. That's my last step in the process 
> > > once I have the rest of the data file setup. I copy/paste once manually 
> > > in Excel while recording the macro and then I can just run it as the last 
> > > macro on the rest of the data files. The station ID's are a bit flaky. As 
> > > you noticed in the "SIDE" column some say "SE" or "NW". That's because 
> > > there are dual sensors on opposing sides of the towers on some towers 
> > > because the tower itself will disturb the flow to the sensor. So if the 
> > > winds are easterly, the forecasters will use the sensors on the SE side, 
> > > not the NW side. So towers 20 and 21 are virtual - they are really tower 
> > > 2 where 20 is the NW side and 21 is the SE side (or vice versa). So tower 
> > > locations 20 and 21 will use the lat/lon of tower 2. Same with tower 110: 
> > > it has two virtual towers as 1101 and 1102. And there are others.
> > >
> > > The data is originally in ASCII text files. I'm just using Excel to 
> > > manipulate the files with macros and then save them as .csv. It seems to 
> > > be working okay - I just need to figure out how to convert the times to 8 
> > > digits.
> >
> > As your other note said, it's 6 digit time.  I just typed too many 00's. ;-)
> > You can use the Excel date/time formatting to get something like 0:00:00 and
> > then change the fmt string in the header to match what you use.  That 
> > worked for
> > me.
> >
> > > Data retrieval, manipulation and display is always a long pole for the 
> > > work we do. Once all the data is acquired, nicely formatted and 
> > > displayed, the analysis doesn’t take so long. I've asked the rest of my 
> > > staff to download and install IDV to start working with it. I intend to 
> > > make that our primary display software rather than GEMPAK or GrADS. Can 
> > > IDV be setup to automatically ftp real time data, produce maps and then 
> > > be uploaded to a location (say a web site)?
> >
> > Glad to hear that you are finding IDV useful.
> >
> > One last note on this importing.  If you save the file off as a CSV without
> > the header info and read it in to the IDV, you will be prompted with a 
> > dialog
> > that allows you to set the header information.  The advantage here is that 
> > you
> > can then save this off and use it as a resource to apply to other files of
> > similar format.   This "wizard" still needs some work, but it might help
> > you out.
> >
> > Don
> >
> > Ticket Details
> > ===================
> > Ticket ID: WRY-586577
> > Department: Support IDV
> > Priority: Normal
> > Status: Open
> >
> >
> >
> > ......................................................................
> > The information contained in this email message is intended only for the 
> > use of the individuals to whom it is
> > addressed and may contain information that is privileged and sensitive. If 
> > the reader of this message is not
> > the intended recipient, you are hereby notified that any dissemination, 
> > distribution or copying of this
> > communication is strictly prohibited. If you have received this 
> > communication in error, please notify the
> > sender immediately by email at the above referenced address. Thank you.
> >
> 
> 
> Ticket Details
> ===================
> Ticket ID: WRY-586577
> Department: Support IDV
> Priority: Normal
> Status: Open
> 
> 
> 


Ticket Details
===================
Ticket ID: WRY-586577
Department: Support IDV
Priority: Normal
Status: Open