WinGap Sketch Conversion Process

_______________________________________________ Brent Kastor
GIS Coordinator Coweta County, GA Newnan, GA 30265 Ph: 678.854.0029 E‐mail: [email protected]
WinGap Sketch Conversion Process An analysis of the specific steps in Mr. Chad Rupert and Mr. Jimmy Nolan’s – A New Look at Planimetrics. Abstract: Coweta County originally received the idea of converting the Computer Assisted Mass Appraisal (CAMA) property sketches into a Geographic Information System (GIS) from an article written by Chad Rupert and Jimmy Nolan of the Carl Vinson Institute of Government (ITOS). In their publication, A New Look at Planimetrics, they go into some detail about how to convert WinGAP (a CAMA program) sketches into a GIS, but certain extremely important steps are skipped. Coweta County staff traveled through the ITOS process and figured out the missing steps – with some programming assistance from a former colleague of the author, Frank Doud. Here is the introduction to the ITOS article: On page three of the article, they go into great detail about the building sketch process. They ultimately move from the WinGAP Sketch Screen to a Final Product. Figure 1 The ITOS article continues to explain the benefits/uses of building footprints, and discusses the creation and maintenance of building footprints. Rupert and Nolan then examine the traditional building footprint with the WinGAP converted footprint. The WinGAP footprint clearly contains more detail – since you can distinguish between an open porch, garage, and/or first floor. On page six of the article, they get into “The Sketch Conversion Process”. These are some excellent details of the overall process and I will list the details here: You will need to acquire a free tool (ET GeoWizard) to complete the conversion process. Whether you install it now or later makes no difference. The tool is really not used until very late in the process (converting points – vertices ‐ into polygons). You can simply browse to http://www.ian‐ko.com/ and find ET GeoWizards download. Here is the summary of the conversion process as listed by ITOS. Please continue reading after the summary for the details that were not listed by ITOS. Figure 2 1. Pack your WinGAP tables (this removes records that have been marked for deletion) This was completed by Keith Knight in the Tax Assessor Office. Keith is very familiar with the WinGAP tables, and executed this step without a problem. Keith used the FoxPro command line pack command to pack realprop, reprop, and wgsketch. 2. Import WinGAP tables to SQL Server (reorganize data to acceptable format for ET GeoWizard) Prior to importing the tables to SQL Server, Keith used the FoxPro COPYFIELDS command and specified fields from the [wgsketch] table and the [realprop] table. Since the wgsketch table included the “realkey” field and not the “parcel_no” field, the joining allowed each sketch record to have a Parcel ID. The COPYFIELDS FoxPro command is very similar to a JOIN command. The ultimate goal being to have a single table containing the vertices from [wgsketch] and the parcel ID’s from [realprop]. Keith actually created a new FoxPro table eloquently named “test”. Here is a screen capture of our new [dbo.test] database: Figure 3 To import the data from WinGAP, I used “Tasks Æ”, “Import Data…” in SQL Server 2005. Figure 4 In order to import data from Visual FoxPro, you will probably need to install the Visual FoxPro driver. Once you have installed the driver, you can simply select the Data source from the drop down list in the SQL Server 2005 Import Data wizard. Figure 5 Once you have chosen Visual FoxPro as your data source, you will need to click on “Properties…”. This will open the “Data Link Properties” window. You will then need to click on “…” under option 1. “Select or enter a database name:”. In the “Configure Connection” window, you will then need to choose the “Free Table directory” radio button, and then click “Browse…” in the “Path:” text box. Simply browse to the location of your FoxPro table(s). Our WinGAP table happens to be named “wgsketch.dbf”. I imported both [wgsketch] and the table Keith created – [test]. Here is a screen capture of the [wgsketch] table import: Figure 6 Continue through the SQL Server import process with the defaults selected (unless you have something else in mind). At this point, we have the [test] table – with a new field added – “parcel_no”. This will allow us to join each record to a parcel in GIS. Prior to doing this though, we need to perform the portion we had the most difficulty with – splitting out each “;” delimited vertex. 3. Export a single reorganized table – easier said than done… Take a moment to look at the table. You will notice the “impkey” field and the “vertices” field. The “impkey” tells the user what feature – 1st floor, 2nd floor, open porch, deck, garage, etc. The “vertices” field lists the coordinates (based on 0,0) of each feature – ‐14, 22; 21, 22; etc. Our goal is to split out each vertex based on the “;” into a new record. This new record will contain the original parcel number, “repropkey”, “impkey”, etc. In order to split out the records, our method of choice is to use MS Access 2003. MS Access has a database size limitation of 2 GB – which would be a very large database. Our final Access DB was ~500MB for >40,000 structures. If you have more that 200,000 structures (not parcels), this might pose a problem? So, now we need to export our SQL [test] table to MS Access. Simply create a new Access .mdb file and use the SQL export wizard. The MS Access drivers are built into SQL Server 2005. Here is the table in MS Access: Take notice that I added the “AutoNum” field. This will ensure that when I split out the records that each record will have a unique ID – each Open Patio will have a unique ID and each Deck for each property will have a unique ID. A real problem could arise when you have multiple decks on one property, but that is handled by incorporating the “repropkey” into your unique ID. Notice in the table above how parcel number “001 4176 002” has multiple OP’s (Open Porches). By combining the “AutoNum” field with the “repropkey” field, you guarantee that each sketch feature is unique. We will see this later. Once your table is in MS Access, you can run a program (Access Module) Frank Doud wrote for us. He is a good friend of mine, and was able to figure out my programming problem in minutes. A big thanks to Mr. Doud! MAKE CERTAIN TO ADD THE AUTONUM FIELD PRIOR TO RUNNING THE MODULE! Access is really where all of the magic happens. Prior to working any magic, you will need to clean up your table – or remove the orphan files. You can do this simply by sorting your table on “parcel_no” A‐Z. You need to delete all of the records containing 0 or NULL for the “parcel_no” field. We heard orphan files sometimes get created in WinGAP? If there are “0” or “NULL” values in the “vertices” field, feel free to delete these as well. We will also clean up the “vertices” field after running the module. Create a new module in your MS Access project and paste the following code into it. The name of the module is irrelevant. ' First: create a new module in Access and paste the entire contents of this subroutine ' ' Second: Create your second database where the new values will be stored. Make sure the fields ' match to what the code calls for below. (realkey; number, impkey; text, and vertices; text) I also ' use an autonumber key field (Key; autonumber), this is your discretion but with large amounts of ' data it is strongly recommended. ' ' Third make sure you change the names of the databases as instructed below. ' ' When finished, simply hit F5 to run, or click Run ‐> Run Sub/UserForm and whalla. ' ' You should have a whole lot of parsed records which look like this: 'Key realkey impkey vertices '1815 82 1st ‐14,22 '1816 82 1st 21,22 '1817 82 1st 21,2 '1818 82 1st 13,2 '1819 82 1st 13,‐28 '1820 82 1st ‐7,‐28 '1821 82 1st ‐7,2 '1822 82 1st ‐14,2 '1823 82 OP ‐12,22 '1824 82 OP 19,22 '1825 82 OP 19,30 '1826 82 OP ‐12,30 '1827 82 OP 13,‐28 '1828 82 OP 21,‐28 '1829 82 OP 21,2 Sub Convert() Dim sAutoNum As Long Dim sParcel_no As String Dim sRepropkey As String Dim sRealKey As String Dim sImKey As String Dim sVertices As String Dim sVertArray() As String Dim iVert As Integer Dim dbTaxAccessor As Database Dim rsTaxAccessor As Recordset Set dbTaxAccessor = CurrentDb ' This is the tax accessors database where we get the data. ' Remember to replace this database name with the correct one! ' ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐v Set rsTaxAccessor = dbTaxAccessor.OpenRecordset("TEST", dbOpenDynaset) ' ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐^ ' Dim rsNewRecordset As Recordset ' This is the new database where we put the new data. ' Remember to replace this database name with the correct one! ' ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐v Set rsNewRecordset = dbTaxAccessor.OpenRecordset("NewTable", dbOpenDynaset) ' ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐^ ' rsTaxAccessor.MoveFirst Do While Not rsTaxAccessor.EOF With rsTaxAccessor sRealKey = .Fields("RealKey") sImKey = .Fields("ImpKey") sParcel_no = .Fields("parcel_no") sRepropkey = .Fields("repropkey") sAutoNum = .Fields("AutoNum") ' ' Create the vertices array by splitting it between the simi‐colons. ' sVertArray = SPLIT(.Fields("Vertices"), ";") ' ' Loop through the elements in the sVertArray array and append a new record. For iVert = 0 To UBound(sVertArray) ‐ 1 If Len(sVertArray(iVert)) >= 1 Then rsNewRecordset.AddNew rsNewRecordset.Fields("realkey") = sRealKey rsNewRecordset.Fields("impkey") = sImKey rsNewRecordset.Fields("vertices") = sVertArray(iVert) rsNewRecordset.Fields("parcel_no") = sParcel_no rsNewRecordset.Fields("repropkey") = sRepropkey rsNewRecordset.Fields("AutoNum") = sAutoNum rsNewRecordset.Update End If Next End With rsTaxAccessor.MoveNext Loop ' ' Clean up the recordset and close rsTaxAccessor.Close dbTaxAccessor.Close End Sub At this point, you should have your main table and an Access module in your project. Take some time to read through the Visual Basic for Applications (VBA) code. Mr. Doud details the steps you need to follow in order to make everything work. Pay special attention to the fact that the user needs to create a new empty table (he calls it “NewTable”). We used the same naming conventions as Mr. Doud. After running the module, you should have a table that looks like this: We now have a new record for each pair of vertices. I suggest sorting the vertices field A‐Z and deleting any “0” or “NULL” records. These records will just create problems later. These bad records give you a good opportunity to quality check some of your sketches in WinGAP. It definitely poses the question – “Why are these records bad?” The next step is to separate the X value and the Y value and to create a Unique ID field. Since I am most familiar with running VBA within ArcMap, this step will utilize ArcMap. Close your Access database and Launch ArcMap – if it is not running already –create a new empty map. We will begin with creating three new fields – “UniqueID”, “X_val”, and “Y_val”. Add your Access [NewTable] as a new data layer by clicking on the Add Data button in ArcMap. Open the attributes of [NewTable] by right clicking the layer in the table of contents and clicking “Open”. Create a New Field by clicking “Options” and then selecting “Add Field…” In the Add Field window, choose Long Integer for Type and type UniqueID for “Name:” – leave the other items default. Follow the previous steps to create the “X_val” and “Y_val” fields. For “Type:” use “Short Integer”. We will use the Field Calculator to calculate all of the values for our new fields. We will begin with “UniqueID”. Right click on the field name – “UniqueID” and choose “Field Calculator”. “UniqueID” will be a combination of “repropkey” and “AutoNum”. Simply select [repropkey] & [AutoNum]. Your “UniqueID” field should be populated. Next, we will use the field calculator to separate the X value from the Y value. This will require checking the “Advanced” check box within the field calculator and typing the following code: ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ Dim cs As String Cs = LEFT([vertices], InStr([vertices], “,”)) ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ This is for the “X_val” field. The “Y_val” uses a different script. For the “Y_val”, use the following code: ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ Dim cs As String Cs = MID([vertices], InStr([vertices], “,”)+1) ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ If your WinGAP sketch data is anything like Coweta County’s data, you will need to multiply the “Y_val” field by ‐1 in order to flip the sketch accordingly. Notice how the “Y_val” field is “‐22”. Doing this will make positives negative, and negatives positive. The ITOS data looked the same way. You should now have a table that looks like this: 4. Create a table of Parcel numbers and their real world X, Y centroid coordinates The next step is to get the X, Y locations from a data layer that contains the parcel ID. ITOS uses the parcel centroids. Coweta County has a very good address data layer we call SiteAddresses. The SiteAddresses contain a parcel number field. If it did not, you can use a spatial join and join the parcel data to the address data. Our point data layer did not have X and Y locations though, so we had to use ArcToolbox to generate the two new fields. We exported our points to a new shapefile and kept only the parcel number field, and then ran the ArcToolbox command – “Add XY Coordinates”. This created a shapefile of address points containing the parcel ID and the X and Y coordinates. Notice my shapefile is named “TaxAssessorAddresCoord” and I now have “POINT_X” and “POINT_Y”. We will combine steps 5, 6, and 7… 5. Join SQL Server exported table to the parcel centroid table based on Parcel # 6. Add X values from two different tables together 7. Add Y values from two different tables together To join the SQL Server exported table (MS Access [NewTable]) to the parcel centroid table (address point shapefile), you will need to first export the attribute table from ArcMap to a dbf file, and then import that dbf into MS Access. We need to do this so we can join the tables using an Access query and run some expressions on the X and Y data fields. To export the attribute table, simply click “Options” and then choose “Export…”. Remember where you saved the file, and then close ArcMap and Open MS Access.
In Access click the “File” drop‐down menu and choose “Get External Data ‐>” and finally click on “Import…” Browse to the location of your dbf file and click “Import”. You should now have an Access database with three tables – [test], [NewTable], and [TaxAssessorAddressCoord]. You will need to create a query to join the necessary tables together and use a few expressions to add x and y fields together.
Click on “Queries” and then double‐click “Create query in Design view”. Notice that I already have a query created in this database – that is what you will end up with when you are completed. We have two goals with this Query. The first is to Join the tables, and the second is to add the X and Y fields together into two new fields. You will need to ADD two tables to this Select Query – [NewTable] and [TaxAssessorAddressCoord]. To join the two tables together, simply drag the field containing the Parcel ID from the [TaxAssessorAddresCoord] table to the field containing the Parcel ID in [NewTable]. Notice how the X and Y fields have different names. I happened to use “Real_X” and “Real_Y” in place of “POINT_X” and “POINT_Y” . This is important to note, since you will write an Expression using these fields. Type the following expression into the third field place holder: Expr1: [POINT_X]+[X_VAL] ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ For my table, I would type: “Expr1: [REAL_X]+[X_VAL]” Type the following expression into the third field place holder: Expr2: [POINT_Y]+[Y_VAL] These expressions are simply adding the two fields together to come up with the exact location for each vertex in the sketch. Click on the “Run” button. Your table should now have all the fields from [NewTable] and [TaxAssessorAddresCoord] and the two new expression fields. It should look something like this: 8. Convert X, Y coordinate points to polygons The next step is to bring all of this X and Y data into ArcMap as an Event layer and use ET GeoWizard to convert the points to polygons. First, you will need to save your query and export it to a dbf. Make sure to specify “Save as type:” to be dBASE IV (*.dbf) – and remember where you saved the file. I saved mine as “WG_SKTCH.dbf” – keeping the name to 8 characters or fewer. You can now launch ArcMap and bring in the dbf file as an Event layer. I like to add some data to ArcMap first – something small – like a county boundary.
Under the “Tools” drop down menu, select “Add XY Data…” This will open the “Add XY Data” window. Open the dbf you created when you exported your query. Mine was called “WG_SKTCH.dbf”. For the “X Field:” choose “EXPR1”. For the “Y Field:” choose “EXPR2”. You will also need to choose a Coordinate System. I just imported one from another data layer using the “edit…” button and then chose “import…” and browsed to a data layer I knew had good Coordinate System information. Finally, Click “OK”. It should take a while to add all of the points – the table is extremely large. You should now have data that looks like this: At this point, you will need to have installed ET GeoWizard. ET GeoWizard is a free install that will allow the conversion of point data to polygons. Simply click on the ET GeoWizards button to get started. When you click on the ETGW button, the “ET GeoWizards” window opens. Notice all of the available functions and tabs. We are specifically interested in the “Convert” tab and the “Point To Polygon” function. Select the “Point To Polygon” radio button and click “GO”. The first step is to select your input ‐ point layer ‐ and specify where you want to save your output. The second step involves selecting your “UNIQUEID” field and what attribute field you would like to include. Finally, click “Finish”. The process will take quite a bit of time to complete – depending on the speed of your PC and the location of the files. ET GeoWizards does have a nice progress bar – for those who like to watch the progress. 9. Move and rotate each sketch into place Finally, you get to start editing each sketch. You can simply use ArcMap to rotate and move each sketch into position. It is a time consuming process, but could be worth quite a few tax dollars by helping to locate missing or improperly sized sketches. Now that you have all of your sketches on the map and are spending the time to rotate and move them, you probably want to know how to update the sketches. This will be another article.