Monday, August 31, 2009

Free GIS Data for Washington, D.C.

Whether you are working on a GIS project for a class assignment or setting up a demo to convey functionality in ArcMap, the biggest challenge is usually finding (free) spatial data. Even if you are able to find the data required to present the main theme of your project/demo, it is often difficult to find the surrounds necessary for a cartographic display of the results.

One great source for spatial data is the DC GIS Data Inventory provided by the Washington, D.C. local government. Their website contains hundreds of datasets from A to Z (literally) in both shapefile and keyhole markup language formats. Data exists for a broad range of categories for the entire District. I have listed only a fraction of available data below to point out some of the most common/useful datasets, but you can visit the DC GIS Data Inventory website for a complete directory.

• Streets
• Bike Routes
• Bus Routes
• Bus Stops
• Metro Lines
• Metro Stops
• Sidewalks

• Soil Polygons
• Wetlands
• Wooded Areas
• Watersheds
• Air Emissions

• Utility Poles
• Electrical Substations
• Fire Hydrants
• Buildings

• Retail Sites
• Gas Stations

• City Boundary
• Census Boundaries
• Streets
• Railroads
• Parks
• Water Bodies

The above is merely a snapshot of what is available on the site. You could easily create a mulit-modal transportation network, complete an environmental research project, present a detailed cartographic display of your project/demo results, and much more. Additionally, each dataset contains highly detailed metadata in Federal Geographic Data Committee (FGDC) and ESRI standards. The metadata for each dataset can be viewed prior to downloading the data.

So if you have a project idea for a class assignment or demo that isn’t location specific, establishing Washington, D.C. as the setting for your analysis may be a pliable option given the broad range of available data.

Tuesday, August 18, 2009

Get XYZM - Populate X, Y, Z, and M Attributes

So here it is, as promised - a tool for populating attributes with values for the X, Y, Z, and/or M dimensions of point features. It is compatible with ArcGIS Desktop 9.x at the ArcView, ArcEditor, and ArcInfo license levels. Download it for free here (right click and select Save Target As...):


The zip file contains the tool (GetXYZM.dll) and a read me file (readme.txt). Unzip the folder to your hard drive. To use the Get XYZM tool, open ArcCatalog and select Customize from the Tools menu. Under the Commands tab, click Add from file… and navigate to the directory containing the unzipped files.

Select the GetXYZM.dll and click Open. Click OK on the Added Objects form. The tool is now listed under the Commands window. Select the Get XYZM command and drag it to the toolbar. You can create a new toolbar or add it to an existing toolbar. Click Close in the Customize dialogue box.

To activate the tool, you must select a point shapefile or feature class in the Catalog tree. Select the point data to populate with dimension values and click the Get XYZM tool button that you just added to the toolbar to bring up the Get XYZM dialogue box.

If your point data already contains fields to populate, ensure the Use existing fields option is selected and choose the appropriate fields in the dropdown menu for the corresponding dimension. You can choose to populate any combination of dimensions – you don’t need to provide a field for each. Once the existing fields for dimension values to populate are selected, click OK. The dialogue box closes automatically when the tool is finished running.

If your point data does not contain existing fields for storing the values of the desired dimensions, select the Create new fields option and enter the field names in the corresponding text boxes provided for each dimension. Once the desired field names to add and populate are entered, click OK. The dialogue box closes automatically when the tool is finished running.

After you select/enter the appropriate fields and click OK - VoilĂ ! - the fields are populated with the chosen dimensions!

Thursday, August 13, 2009

Easy Calculate 5.0 - Free Tools for Calculating Shape Geometries

I'm writing this entry in response to StuGotz's comment on my last post (Batch Edit String Fields - Free Tools for Calculating String Fields). StuGotz currently uses VB to populate fields with X, Y, Z, and M values but is looking for a more simplistic method for calculating these fields. This is a good opportunity for a post on Easy Calcuate 5.0 - a set of files (.cal) containing code for calculating a wide range of geometric properties for vector data (points, lines, and polygons) that can be easily loaded into the Field Calculator. The files are available for free here:

Easy Calculate 5.0

Just download the zip file and unzip the file to a directory on your hard drive. These .cal files are extremely easy to use and come in very handy for common geospatial calculations including the very properties that StuGotz is looking for - the X, Y, Z, and M dimensions for point features. Also included are files for calculating the from, to, and midpoint X and Y coordinates of a line, the maximum and minimum X, Y, Z, and M values for points, multipoints, and polygons, the X and Y coordinates for the center of polygons, the thickness ratio of each polygon, and many, many more (see above link for a complete list).

Okay, so I mentioned Easy Calculate 5.0 is easy to use, right? I didn't lie. Once you've downloaded and unzipped the file, here's all you have to do:

1. Right click on a field in the attribute table in ArcMap and select Calculate Values.
2. In the Field Calculator dialogue, click the Load button, navigate to the directory where you unzipped the files, and select the appropriate .cal file for your calculation.
3. Make any changes to the code *if needed* (adjust tolerance, etc).
4. Click OK.

Now is that easy or is that easy? I can see why they called it "Easy Calculate"!

In addition to geospatial calculations on attribute fields, Easy Calculate 5.0 also provides methods for editing the actual geometry of vector features including offsetting lines by a specified distance, generalizing lines and polygons using the Douglas Poiker algorithm, converting polygons to their bounding rectangles (envelopes), reversing the direction of lines, and more. Simply run the Field Calculator on the Shape field when using the files for editing feature geometry.

Field calculations using these files can be made at any of the ArcGIS license levels (ArcView, ArcEditor, and ArcInfo) with ArcGIS 8.x and 9.x. Changes to the fields and feature geometry are applied directly to the original data. If the .cal files are applied out of an Edit session (not available with ArcView), it is a good idea to make a copy of the original data, as the changes are irreversible. Calculations and geometry edits are applied only to selected features. If no features are selected, values for all records will be calculated.

Now, it's not Christmas or anything but I was feeling pretty generous and decided to work on a custom tool for StuGotz so that the fields for X, Y, Z, and M values can be populated with the click of a button (as if Easy Calculate weren't easy enough). So stay tuned and subscribe and/or follow this blog. Once I finish the tool, I'll write an entry on how to use it and upload it to the geospatial database so that it can be downloaded for free.

Saturday, August 8, 2009

Batch Edit String Fields - Free Tools for Calculating String Fields

If you’ve ever tried to make batch edits on all or a selection of string values in an attribute table, you’ve probably spent a significant amount of time searching for the correct VBScript to plug into the Field Calculator dialogue box to get the desired results. Well search no more! A few years back I wrote the following Python scripts to simplify common batch edits made to string fields:

  • Capitalize: Capitalizes the first letter of a string. Ex. Changes "hello world" to "Hello world"
  • Capitalize Words: Capitalizes the first letter of every word in a string. Ex. Changes "hello world" to "Hello World"
  • Lower Case to Upper Case: Converts lower case letters of a string to upper case. Ex. Changes "hello world" to "HELLO WORLD"
  • LStrip: Strips a specified number of preceding characters of a string.
  • LTrim: Removes preceding white spaces from the end of a string.
  • Replace: Replaces current string with a new string.
  • RStrip: Strips a specified number of trailing characters of a string.
  • RTrim: Removes trailing white spaces from the end of a string.
  • Upper Case to Lower Case: Converts upper case letters to lower case. Ex. Changes "HELLO WORLD" to "hello world"

Download the toolbox with all nine Python scripts for free! The link below retrieves a zip file containing a toolbox (.tbx), nine Python scripts (.py) listed above, and a readme.txt from the geospatial database. Right click the link and select Save Target As... Unzip the folder and be sure to store the toolbox in the same location as all of the Python scripts – relative paths are used to locate the scripts.

Batch Edit String Fields

Minimum Requirements:
ArcView license of ArcGIS 9.2
Python 2.4

The scripts can be run in either ArcMap or ArcCatalog. Edits can be made on all records in a feature layer, feature class, or shapefile. If a selection is used, the scripts will only run on the selected records. A SQL expression can be applied to the script to run on only those features that meet the conditions of the SQL expression.

*Important Note*
All changes made using these scripts are permanent unless they are used in an Edit session (ArcEditor or ArcInfo license required).

In this example, the Capitalize Words tool converts the first letter in each word in the ADDRESS field from lower to upper case.

Tuesday, August 4, 2009

What is GIS?

Okay so here’s a good topic to start my blog with: What is GIS? How do you really define and explain it to someone who doesn’t know what it is?

As a Geospatial Analyst I know the ins and outs of GIS, as I use it in my job every day. Though I’m not so sure you’d get that from me if you heard me try to explain what GIS is to someone who has never heard of it before. Instead of giving a brief, understandable definition of GIS, I find myself painfully stuttering through what ends up becoming a boring, broken monologue to which I eventually receive a blank stare. This is the point at which I usually just awkwardly stop talking, as I end with “Yeah…it’s just kinda hard to explain”.

Here’s my problem: GIS is so broad of a topic and so powerful of a tool, I want people to understand and RECOGNIZE its capabilities. So instead of explaining it at a high level, I find myself overwhelmed trying to figure out how I can briefly explain all of the capabilities and applications of GIS to someone who probably doesn’t know much about databases in general – let alone spatial databases. After talking to my fellow GIS geek friends, this seems to be a pretty common problem among all of us in the GIS field – explaining what it is that we do every day. So, I’ve put together some thoughts on best practices for approaching this topic and explaining what GIS is at a high level.

First tip: You’re never going to be able to briefly explain GIS to someone in a way that they will be able to understand its power and applications if they have never used it before – don’t waste your time or theirs with details.

Okay, so what IS GIS anyway? *At a high level*, GIS is a combination of a few things:
1. Hardware: workstations, servers, and handheld devices for collecting and storing spatial data.
2. Software: GIS application for viewing, analyzing, and transforming spatial data.
3. Spatial data: data with a geographic location.
4. People: analysts well-trained in spatial analysis and skilled in using GIS software.

Let’s practice. Someone asks you, “What is GIS?”. Instead of stuttering your way through the details, you calmly and collectively say:

“GIS stands for Geographic Information Systems. It’s a computer-based system for collecting, viewing, analyzing, storing, and transforming spatial data, or data with a geographic location.”

You might want to give a *few* descriptive examples, so that people have a better idea of what you are talking about. So here are some applications of GIS with examples that you may want to mention:

  • Commercial: Use spatial models and geodemographics to target markets and improve profitability.
  • Law Enforcement: Use incident mapping and spatial statistics to analyze crime.
  • Transportation: Collect roadway pavement conditions to determine funding and plan pavement schedules.
  • Utilities: Model electric networks to determine customers effected by power outages.
  • Logistics Management: Vehicle routing for school buses, delivery services, emergency vehicles, etc.
  • Environmental: Locate ideal habitat locations for studying wildlife.

So there you have it – my very own simplified explanation of GIS. Post a comment or shoot me an email if you have another definition. I could throw a list of good definitions together for a later blog.