Wednesday, January 9, 2013

Esri Press Publishes Python Scripting for ArcGIS

From Esri Press:

Redlands, California—January 9, 2013Python Scripting for ArcGIS is a technical guide that provides a clear, step-by-step process for the creation of Python scripts to enhance ArcGIS 10.1 capabilities.

"Python scripting is a fundamental tool for GIS professionals to extend the functionality of ArcGIS and automate workflows," says author Paul A. Zandbergen. "You can use it to modify the existing capabilities of ArcGIS and create new ones."

Python Scripting for ArcGIS is a guide for experienced users of ArcGIS and students that are interested in learning how to create Python scripts. No previous programming experience is necessary. Each chapter in the book has a corresponding exercise on the accompanying DVD, and a free 180-day trial of ArcGIS 10.1 for Desktop is included.

Python is the scripting language of choice for ArcGIS users because of these characteristics:

• Simple and easy to learn
• Free and open source
• Cross-platform compatible
• Interpreted, not compiled
• Object oriented

Zandbergen is an associate professor of geography at the University of New Mexico in Albuquerque where he teaches classes in GIS and spatial analysis. His areas of expertise include GIS applications in criminology, economics, health, and ecology, as well as spatial and statistical analysis techniques using GIS.

Python Scripting for ArcGIS (ISBN: 978-1-58948-282-1, 368 pages, US$79.95) is available at online retailers worldwide, at esri.com/esripress, or by calling 1-800-447-9778. Outside the United States, visit esri.com/esripressorders for complete ordering options, or visit esri.com/distributors to contact your local Esri distributor. Interested retailers can contact Esri Press book distributor Ingram Publisher Services.

Sunday, November 25, 2012

Tools for Comparing and Analyzing Geodatabase Schemas

Here is a suite of tools written in Python for analyzing components of a geodatabase and for finding the differences between geodatabase schemas. The tools are accessible through an ArcGIS Toolbox called SchemaInspector.tbx. They are grouped into three categories:
1. Analyzer Tools
2. Diff & Compare Tools
3. Diff Tools

There are five tools under the Analyzer Tools toolset. These tools are for printing properties of a single geodatabase component. Each tool has parameters for the input workspace and the output text file (the output text file does not have to pre-exist).
1. Analyze Domains
- Prints properties to a text file for each domain in a workspace including Domain Type, Range, Field Type, Merge Policy, and Split Policy.

2. Analyze Feature Classes
 - Prints properties to a text file for each feature class in a workspace (geodatabase or feature dataset) including Shape Type, Shape Field Name, Feature Type, Spatial Index, Has M, and Has Z.

3. Analyze Fields
- Prints properties to a text file for each selected field in a feature class including Type, Alias, Base Name, Domain, Editable, Nullable, Length, Precision, Required, and Scale.

4. Analyze Relationship Classes
- Prints properties to a text file for each relationship class in a workspace including Backward Path Label, Cardinality, Class Key, Destination Class Names, Forward Path Label, Attachment Relationship, Attributed, Composite, Reflexive, Key Type, Notification, and Origin Class Names.

5. Analyze Tables
- Prints properties to a text file for each table in a workspace including Has OID and OID Field Name.


There are six tools under the Diff & Compare Tools toolset. These tools are for comparing particular geodatabase components between two workspaces. Each tool has parameters for the two schemas to compare and the output text file.
 1. Domain Diff & Compare
- Prints the names of domains that are not common between workspaces to a text file. Searches common domains and prints domain names and properties (same properties as the Analyze Domains tool) for domains with the same name but with different properties.

2. Feature Class Diff & Compare
- Prints the names of feature classes that are not common between workspaces to a text file. Searches both standalone feature classes and feature classes in feature datasets. Searches common feature classes and prints feature class names and properties (same properties as the Analyze Feature Classes tool) for feature classes with the same name but with different properties.

3. Field Diff & Compare (by Feature Class)
- Prints the names of fields that are not common between feature classes to a text file. Searches common fields and prints field names and properties (same properties as the Analyze Fields tool) for fields with the same name but with different properties.

4. Field Diff & Compare (by Workspace)
- Prints the names of fields that are not common between all feature classes in the workspaces to a text file. Searches both standalone feature classes and feature classes in feature datasets. Searches common fields and prints field anmes and properties (same properties as the Analyze Fields tool) for fields with the same name but with different properties.

5. Relationship Class Diff & Compare
- Prints the names of relationship classes that are not common between workspaces to a text file. Searches common relationship classes and prints relationship class names and properties (same properties as the Analyze Relationship Classes tool) for relationship classes with the same name but with different properties.

6. Table Diff & Compare
- Prints the names of tables that are not common between workspaces to a text file. Searches common tables and prints table names and properties (same properties as the Analyze Tables tool) for tables with the same name but with different properties.


There is one tool under the Diff Tools toolset. This tool is called Workspace Diff and combines all of the tools under Diff & Compare toolset into a single tool. There are parameters for the two schemas to compare and the output text file.


All of the tools can be used with Basic, Standard, and Advanced license levels. The tools can be used with ArcGIS 10.0 or higher, with the exception of the Analyze Domains tool, the Domain Diff & Compare tool, and the Workspace Diff tool which require ArcGIS 10.1 or higher (to use the Workspace Diff tool on 10.0, simply comment out the lines that compare domains [361 - 387] in the ws_diff.py file).

The SchemaInspector.tbx and all of the tools described above can be downloaded from the Geospatial Database for free. Right click on the link below and select Save As... to download the zip containing the .tbx and .py files. Unzip all of the files to the same folder location (toolbox and scripts use relatives paths). The tools can be accessed and executed from both ArcMap and ArcCatalog.

Schema Inspector Toolbox

Below is an image of the Workspace Diff tool and the output text file that is created when the tool is executed.

Wednesday, April 11, 2012

Yup.

This makes me laugh.

Wednesday, March 9, 2011

Remove Leading Zeros From a String

I get a lot of email from people wanting to find/replace specific patterns in string fields, so I'm writing this post to *briefly* introduce regular expressions (commonly called regex), and answer a question that was posted by Chris earlier today on the Batch Edit String Fields post.

The term "regular expression" is used to describe a flexible method for searching for patterns within a string. Regular expressions use special characters to match any number and/or combination of characters. You can read a little more about them on Wikipedia here:

http://en.wikipedia.org/wiki/Regular_expression

I'll explain the concept further and give some examples in my next post, but for now I want to answer the question that was posted by Chris. Chris needs to trim leading zeros from a string. I created a .cal file (for both Python and VB Script) that uses a regex and can be loaded into the Field Calculator to run on any string field. The regex I used for this is "^0+", where ^ denotes the beginning of the string to match, and + matches the preceding character (in this case 0) one or more times. I use a replace function to replace this pattern with an empty string, to completely remove all zeros at the beginning of the string.

The .cal files can be downloaded from the Geospatial Database:

TrimLeadingZeros_VB.cal

TrimLeadingZeros_Python.cal

*Note: The .cal file for Python can only be used in ArcGIS 10.x. If you are using any earlier ArcGIS version, you will need to use the .cal file for VB Script (TrimLeadingZeros_VB.cal).


How to use:
Open the Field Calculator either through the Geoprocessing tool in Data Management Tools > Fields > Calculate Field or by right clicking the field you want to edit in the attribute table and selecting Field Calculator... Select the Load... button and browse to the .cal file. If you use the VB Script .cal file, change the field within the Replace function in the Code Block to the name of the field containing the string you want to search on, leaving the field name in brackets. If you are using the Python .cal file (in ArcGIS 10.x only), be sure to change the Parser option at the top of the dialog to Python and change the name of the field in the Expression to the name of your field, leaving the field name within the exclamation points. When you click OK, the field will update and remove all leading zeros within each record.




More on regex later. Stay tuned!

Wednesday, December 1, 2010

From/To Replace String - Another Addition to the Batch Edit String Fields Toolbox

From/To String Replace is the latest script added to the Batch Edit String Fields Toolbox (see the Batch Edit String Fields - Free Tools for Calculating String Fields entry). A coworker needed to edit a field for labeling River polygons on a cartographic product in ArcMap. The field was a Name field but contained extra unwanted verbiage in parentheses to further explain the Name. For example:

New River (North Carolina portion)

and

Roanoke River (Lake Gaston below normal full power pool elevation 200 MSL)

She only wanted the labeling to show the river name, and not the explanation contained in parentheses. The Find and Replace script in the Batch Edit String Fields Toolbox is handy for replacing known substrings in a text field, but in this case, anything could be within the parentheses, so using the Find and Replace script wouldn’t work. [Enter the new From/To String Replace script!]

See the image of the script below. With this script, the starting character as well as the ending character of the substring you’d like to replace/remove is taken into account. The Replace With parameter is for the new string that will replace the characters to be removed. If there is nothing to replace the removed string with (like in the example above), the Replace With parameter can be left blank, and the substring will simply be removed from the field.

The From/To String Replace script can be downloaded as part of the Batch Edit String Fields Toolbox from the Geospatial Database. Feel free to provide feedback if you found this useful or if you have ideas for further enhancements.

Batch Edit String Fields Toolbox

*Note: These scripts were written in ArcGIS Desktop 9.3 and 9.3.1. They will work in ArcGIS Desktop 9.2. 9.3, 9.3.1, and 10.0 at the ArcView, ArcEditor, and ArcInfo license levels. Python 2.4, 2.5, or 2.6 is also required.

Sunday, November 21, 2010

Stressful day at the office? Not for us...

Check this out. A friend sent me this link about a month and a half ago. CNNMoney.com and Payscale.com conducted a survey on more than 40,000 workers and compiled a list of the “10 Least Stressful Jobs”. Geographic Information Systems Analyst is number five on the list!

CNNMoney.com/Payscale.com Survey

This got me thinking – we (GIS/Geospatial Analysts), generally speaking, really are in a great field for so many reasons. Our jobs are challenging and can be so rewarding. We often have the opportunity to work with domain experts in other fields, allowing us to learn about other industries. Because the technology is always advancing, it’s essential to stay on top of the current, cutting-edge solutions. Moreover, it just seems like people who work in GIS are so passionate and excited about the work that they do. Positive attitudes coupled with low stress… now THAT makes it easier to get up and go to work every day. We’ve got it good.

Tuesday, June 22, 2010

Access your Oracle Database from Python

cx_Oracle is a free, downloadable Python extension module that enables Python to access an Oracle database. This can come in handy when you need to streamline or automate tasks that require the use of SQL statements to access components of your spatial database, such as feature classes and domains. The module is available for all combinations of a variety of Operating Systems, Oracle versions (9.2.0, 10.2.0, and 11.1.0), and Python versions (2.0 and up). *Note: An Oracle client (or server) installation is required in order to use cx_Oracle. If you do not require the tools that come with a full client installation, it is recommended to install the Instant Client which is far easier to install.

I found this module to be useful when deleting feature classes with domains from the database. In ArcGIS 9.3.1 and earlier versions, unused domains are left lingering in the database after feature classes have been deleted. What’s more, default domains have to be unregistered before they can be deleted. I wrote a script in Python using the cx_Oracle module to loop through all feature classes in an ArcSDE database, remove default domains from fields, delete each feature class, and then delete the domains remaining in the database. I’ve added this script to the Geospatial Database. If you would like to see an example of how to use cx_Oracle or are looking to delete feature classes with domains from a database like I described above, you can download the script here.

The script takes four arguments:
1) Owner: Data owner.
2) Password: password@NET SERVICE NAME
3) Workspace: The folder location of the sde database (and feature dataset, if applicable) containing feature classes to run the script on (ex. C:\Documents and Settings\user\Application Data\ESRI\ArcCatalog\example.sde)
4) SDE Owner: Owner of the SDE repository in the database (ex. sde)

Friday, April 30, 2010

What Is A Geospatial Analyst?

I came across this article that gives a good description of the roles of a Geospatial Analyst. I know that a lot of the traffic on my blog stems from people searching for a description of what a Geospatial Analyst does, so I thought I would share this. It mentions the skill sets required for a Geospatial Analyst position are environmental sciences, computer science, geography, and database design. I think this is a pretty inclusive list at a very high level (other than remote sensing, which is brushed upon later on in the article). Check it out here:

The Role of a Geospatial Analyst

Tuesday, March 23, 2010

"ezree"

Here’s an interesting tidbit: ESRI employees received an email from corporate last week regarding the pronunciation of ESRI. We will now be calling ourselves “ezree” instead of E-S-R-I to ensure “consistent name recognition around the world”. This makes sense as most non-ESRI employees know the industry’s leading GIS software company as “ezree”. Even I called it "ezree" prior to becoming an employee. Though now the pronunciation sort of makes me cringe. How do you feel about it?

Monday, February 22, 2010

Inserting Characters into a String - The Latest Addition to Batch Edit String Fields Toolbox

This post serves as a response to a comment posted by Josh on the Find and Replace entry. Josh commented:

"Brooke,
Let me ask a variant of Ivar's question. Ultimately I can resolve my situation the same way he identified above, however, is there a way to insert specified characters at a specific position. For instance; insert "ap" at position 2 from left for the field "Hpy" to create "Happy"? Any Ideas? Thanks. Nice site. Josh
"

Josh,
Thank you so much for your question and for visiting my blog. There are a couple ways of doing this:

1) You can use the field calculator in the layers attribute table using the following expression:
Left( [FieldName], 1) & "ap" & Right( [FieldName], Len( [FieldName]) -1)
So in this example, "ap" is added to the first character of the string (the Left function with the number 1 returns the first character), and then the characters of the original string minus the first character (the Right function with the number -1 returns all characters from the right of the original string minus the first character) are added to the end. So if the string in your field is "Hpy", "ap" will be added after the first character ("H"), and then the rest of the original string ("py") is added to the end. See the post on String functions for ArcMap Field Calculator for more VB examples for altering strings.

2) Thanks to your question, I added a new script to the Batch Edit String Fields toolbox I blogged about last September. Below is a graphic of this new tool.



The toolbox and scripts can be downloaded from the Geospatial Database. Use the link below to download the zip file. Ensure that the files are unzipped to the same folder (the scripts use relative paths).

Batch Edit String Fields.zip