Monday, February 22, 2010

Inserting Characters into a String - The Latest Addition to Batch Edit String Fields Toolbox

This post serves as a response to a comment posted by Josh on the Find and Replace entry. Josh commented:

Let me ask a variant of Ivar's question. Ultimately I can resolve my situation the same way he identified above, however, is there a way to insert specified characters at a specific position. For instance; insert "ap" at position 2 from left for the field "Hpy" to create "Happy"? Any Ideas? Thanks. Nice site. Josh

Thank you so much for your question and for visiting my blog. There are a couple ways of doing this:

1) You can use the field calculator in the layers attribute table using the following expression:
Left( [FieldName], 1) & "ap" & Right( [FieldName], Len( [FieldName]) -1)
So in this example, "ap" is added to the first character of the string (the Left function with the number 1 returns the first character), and then the characters of the original string minus the first character (the Right function with the number -1 returns all characters from the right of the original string minus the first character) are added to the end. So if the string in your field is "Hpy", "ap" will be added after the first character ("H"), and then the rest of the original string ("py") is added to the end. See the post on String functions for ArcMap Field Calculator for more VB examples for altering strings.

2) Thanks to your question, I added a new script to the Batch Edit String Fields toolbox I blogged about last September. Below is a graphic of this new tool.

The toolbox and scripts can be downloaded from the Geospatial Database. Use the link below to download the zip file. Ensure that the files are unzipped to the same folder (the scripts use relative paths).

Batch Edit String

Sunday, February 21, 2010

Using Python to Generate Status Emails for Lengthy Geoprocessing Tasks

In honor of PyCon (Python conference in downtown Atlanta) this weekend, I've decided to dedicate this post to a very valuable Python capability that can be extremely useful when running daily, behind-the-scenes scripts or scripts with lengthy processing times.

If there is a script that you run daily for some process such as an overnight synchronization, or an overnight cache, it can be a hassle checking the script's log file every morning to make sure it ran successfully. Or if there is a script you are running on an extremely large dataset with hundreds of thousands, or even millions of features, it would be nice to receive an email when the script completed that contained a subject providing information on whether the script completed successfully or stopped running due to some error, and in the message body see the error that caused the script to fail.

This can be done very easily with Python. I have uploaded a script to the Geospatial Database called and it can be downloaded by right clicking on the link below and selecting "Save Target as...".

The uploaded script is written to work so that an email is generated once the script finishes running. If the script runs successfully, the email's subject displays "Script Completed Successfully!" and the message body is left blank. However, if the script fails, the subject displays "Script Failed", a log file with error messages is attached to the email, and the message body of the email notifies the user to check the attached log files for errors. The script can be easily modified to meet your own personal needs - maybe you only want to receive an email if the script fails, maybe you want the error messages to just print to the message body instead of having a log file, etc. All of these things are pretty simple changes within the script.

What the script requires from you:
1) Your own code which you will paste into the PyEmail script.
2) The path to your log file (if you are using a log file to catch error messages).
3) The email address(es) of the recipient(s).
4) The name and email address of the sender.
5) IP or name of the mail server used to send the email.
6) SMTP user name and password (if you are using SMTP authentification).

Wednesday, February 10, 2010

ET GeoWizards – A suite of tools extending ArcMap functionality

ET GeoWizards is a suite of GIS tools that extends ArcMap functionality and is available at all license levels. Additionally, the tools allow ArcView license holders to perform analysis tasks that would otherwise require an ArcEditor or ArcInfo license, or require the 3D Analyst extension. The full ET GeoWizards tool set without restrictions or limitations can be purchased here, however much of the functionality is absolutely free. Click here to download the tools for free (with some restrictions and data processing limitations without a license key) from ET SpatialTechniques. Installation includes a DLL that can be added to an ArcMap toolbar and provides an easy-to-use GUI with tabs of tools grouped by theme.

The installation package also includes an ET GeoWizards toolbox (.tbx) that contains tools that can be used in Model Builder and Command Line. For a complete list of free tools on both the GUI and in the toolbox, see the Free functions of ET GeoWizards page.

Below are *just a few* functions with descriptions from each tabbed group on the GUI that I personally have found useful. Definitely check out the complete list to see all of the free tools available.

• Clean Point Layer – Deletes duplicate points.
• Measure Points – Assigns measures (M) or elevations (Z) to points based on interpolated values from the closest polyline within a user specified search tolerance.
• Reverse Geocoding – Uses a reference polyline layer to assign assign addresses to points of a point layer.

• Clean Polyline Layer – Ensures topological correctness of polyline features.
• Get Z Characteristics – Calculates several Z dimension characteristics and stores them in fields added to the attribute table. Includes characteristics such as 2D and 3D length of the polyline, min and max Z values, and average and max slow uphill and downhill among others.
• Flip Polylines – Assigns the node closest to a specified start point as the from node so that the direction of all polyline features point away from the start point.

• Clean Polygons – Ensures topological correctness of polygon features.
• Create Centerlines – Creates centerlines from polygon features. Can be created inside polygons (from polygon rivers and streets) our outside polygons (from parcel data).
• Get Adjacent Polygons – Stores the adjacent polygon names/IDs (user specified) as a comma delimited string as well as the count (number of adjacent polygons) for each polygon in the attribute table.

• Polyline to Point – Converts a polyline dataset to a point dataset at 1) vertices, 2) nodes, or 3) mid points.
• Point to Multipoint – Converts a point layer to a multipoint dataset.
• Point to Point Z (M) – Uses a numeric field to assign values to the Z or M dimensions.
• Polygon Z (M) to Point – Converts polygon vertices to points and adds attributes to store Z and/or M values of each vertex.
• Polyline Z (M) to Point – Converts polyline vertices to points and adds attributes to store Z and/or M values of each vertex.

• Build TIN – Builds a Triangulated Irregular Network from a point, polyline, or polygon feature layer.
• Features to 3D – Creates a 3D dataset from point, polyline, or polygon features by deriving the values from a surface (Raster or TIN) layer.

• Transfer Attributes – Transfers attributes from one polygon layer (source) to another (target) based on their spatial location.
• Spider Diagram (Spatial Location) – Creates a Spider Diagram representing the shortest distance between the points of a Center Points layer and the features in the destination layer (points, polylines, or polygons).
• Spider Diagram (Attribute Link) – Creates a Spider Diagram between the points of a Center Points layer and the features in the destination layer (points, polylines, or polygons) based on a common value in a user specified field in both layers.

• Redefine Fields – Change field names, length, precision, and scale.
• Order Fields – Exports a feature layer to a new feature class. The user selects the fields to be exported and the order in which they will appear in the attribute table.
• Vector Grid – Creates a polygon or polyline grid with user defined extents and cell size.

• Create routes from existing polylines – Creates routes from existing polyline features. Input polylines that share a common identifier are merged to create a single route.
• Calibrate routes with points – Calibrates existing routes using the measures of existing points. The measures collected from the points can be M dimension values or can be stored in a ield of the point attribute table.
• Locate point features along routes – Finds the route and measure information of point features and creates a point event table. Search tolerance is used to correctly identify the closest routes to each point.
• Locate polygon features along routes – Calculates route and measure information at the geometric intersection of polygon data and route data and creates a line event table.

• Features To Envelopes – Creates a polygon from the envelope of each feature in the input feature class. Attributes of the original features are transferred to the envelope polygons.
• Features To Minimum Bounding Circles – Creates a circular bounding polygon from each feature in the input feature class. Attributes of the original features are transferred to the resulting polygons.
• Create Station Lines – Creates station lines along the polyline features of the input layer. The station lines are perpendicular to the original polylines with user specified station distance and length. The length can be constant or calculated from Z or M values (if the input layer has Z or M values).

• Export To Google Earth – Converts all visible layers in a data frame to a Google Earth KML file.
• Import From Google Earth – Converts Google Earth (KML or KMZ) data in to feature classes.