Introduction to the ArcPy library for querying, analyzing, and manipulating geospatial data in ArcGIS Pro

The purpose of this article is to provide a comprehensive overview of the core functionalities offered by the ArcGIS Pro Desktop application, highlighting not only its graphical interface but also the potential for interaction through Python scripting. This programmatic interaction is made possible via ArcPy, a Python library specifically designed for the analysis and manipulation of geospatial data. To facilitate a clearer understanding of this library, a detailed explanation of its principal modules will be presented. Following the theoretical framework, a practical Python script will be developed and integrated into ArcGIS Pro through the creation of a custom tool, accessible to end users via a dedicated dialog interface. To demonstrate its practical utility, two use cases of the implemented tool will be illustrated.

Overview of ArcGIS Pro

ArcGIS Pro is an application based on the ArcGIS license. ArcGIS is a software product developed by Esri, a leading multinational company in the field of Geographic Information Systems (GIS). Several desktop applications are available under the ArcGIS license, with ArcMap and the more recent ArcGIS Pro being the most prominent. These were initially released in 1995 and 2015, respectively. ArcGIS Pro introduces a number of significant differences compared to ArcMap, offering both advantages and limitations. Among the primary drawbacks is its limited portability, as it is designed exclusively for the Windows operating system. Additionally, it has more demanding hardware requirements, which may pose challenges for users operating on older machines. Nonetheless, for users equipped with modern, high-performance Windows systems, the benefits of ArcGIS Pro are substantial and often sufficient to persuade even the most hesitant users. Key advantages include:

• Enhanced performance through optimized hardware utilization and reduced execution times, enabled by multi-threading capabilities and an architecture free from RAM capacity limitations.
• A modern, intuitive, and user-friendly interface
• Seamless integration of both 2D and 3D maps and scenes, enabling more comprehensive spatial data visualization and analysis
• Improved map rendering performance through a dedicated graphics engine powered by GPU acceleration
• A significantly improved approach to project management and organizational structure.

While multiple versions of ArcGIS Pro exist, this article focuses on the most recent release, version 3.4, launched in November 2024.

With the main advantages of ArcGIS Pro outlined, attention can now be turned to the structure of a project file within the application. Upon launching ArcGIS Pro, a new project is created from a predefined template, which serves as the foundational workspace for incorporating visual representations of the geospatial data to be analyzed and manipulated. Among the available templates, the most commonly used is the Map template. Once the project is initialized, a file with the .aprx extension is generated. This file represents the actual project and is saved in a user-specified directory.

ArcGIS Pro’s opening interface, where users can select a project template.

Below is an image showing the ArcGIS Pro interface within the newly created project.

ArcGIS Pro graphical user interface.

From a graphical standpoint, an ArcGIS Pro project follows a hierarchical tree structure. Within the ArcGIS Pro interface, each project is represented as a collection of one or more maps, each of which is composed of multiple layers, each with its own symbology. This graphical structure is accessible via the Contents pane on the left side of the interface.

Similarly, from a file system perspective, an ArcGIS Pro project also exhibits a tree-like organization, which can be explored through the Catalog pane on the right side of the interface. At the top level is the project root folder, labeled Folders, which contains all files intended for use within the project. By default, a subfolder is created within this root directory, bearing the same name as the project. This is referred to as the default project folder. Within this folder resides a key component: the default geodatabase, which — although technically a file — is treated as a folder due to its internal structure. It shares the project’s name, carries the .gdb extension, and serves as the central repository for the project’s data. The default geodatabase is also the default workspace, meaning that any data generated without an explicitly defined output path is automatically stored here. At the same hierarchical level as the default geodatabase, another file is created by default: it also shares the project’s name but has an .atbx extension. This file type was introduced to store executable objects, specifically Python Script Tools and Models:

• Models allow users to visually chain together multiple geoprocessing tools in a flowchart-like interface.
• Python Script Tools, as will be discussed later, enable the integration of custom tools developed using Python code.

In addition to the default geodatabase and the toolbox file, users can incorporate other folders from the local file system into the project by creating Folder Connections. As a best practice, it is common to connect the folder containing all project-related resources — such as input shapefiles, external datasets, and Python scripts — to streamline access and organization.

Structure of the project root folder, consisting of the default project folder and an additional folder named LPA, which has been incorporated by the user.

The data contained within a project’s root folders can be manipulated using the suite of tools provided by ArcGIS Pro through the Geoprocessing pane. This pane offers a wide range of applications that users can interact with either via a tool dialog interface or through the Python Command Window integrated into the application. As will be discussed in more detail later, each geoprocessing tool corresponds to a specific function within the ArcPy library. This library is pre-installed and automatically imported within the default virtual environment of ArcGIS Pro, enabling seamless integration between graphical operations and Python-based scripting.

In the two images below, an example of using the Select function is shown, with the aim of filtering only the capital cities from all cities in the world. In the where clause, a field determining whether the corresponding city is a capital has been set to true. The Select tool was dragged from the Geoprocessing Pane and dropped into the Python Command Window.

Before running the command.

After running the command.

The following section provides a description of the typical data structure found within the default geodatabase of an ArcGIS Pro project.

Introduction to geospatial data

Geospatial data imported into an ArcGIS project are typically provided in the form of shapefiles.

But what exactly are shapefiles?

Shapefiles are files that contain geospatial data and are characterized by a set of properties. These data are always associated with geometric elements of one of three types: polygons, polylines, or points. Due to the shared properties, all data within a single shapefile are organized in a single attribute table. Each row of the table, also called feature, represents a geometric element and includes attributes that describe its characteristics. The type of geometry used depends on the nature of the data. For instance:

• A shapefile representing world cities will consist of a set of points.
• A shapefile representing countries will consist of polygons corresponding to their physical boundaries.
• A road or railway network is more appropriately represented using polylines.

There are several free online sources for obtaining shapefiles. Among these, the most reliable and frequently updated is Natural Earth, a comprehensive database providing downloadable vector and raster data at various scales. It encompasses both cultural themes — such as political boundaries, urban centers, transportation networks, and infrastructure — and physical or natural features. For the purposes of this article, only vector data pertaining to cultural themes will be considered, available in the shapefile format previously introduced.

Example of an attribute table associated with a shapefile of countries obtained from Natural Earth.

Using a tool called Copy Features, a shapefile can be converted into another type of file known as a Feature Class. In most cases, shapefiles and Feature Classes are interchangeable, and the concepts previously introduced for shapefiles apply equally to Feature Classes. The main difference lies in the fact that Feature Classes can be stored within the project’s default geodatabase and can be fully manipulated using all tools available in the Geoprocessing pane as well as through ArcPy functionalities, whereas shapefiles are typically read-only.

A geodatabase can contain three types of elements:

• Feature Classes
• Standalone Tables
• Feature Datasets, which are containers for Feature Classes

Two important constraints apply:

• A Feature Dataset can only contain Feature Classes
• The name of each Feature Class must be unique within a geodatabase

To keep things simple, this article will not cover the creation of Feature Datasets, focusing instead on the creation of Feature Classes and Standalone Tables.

The main tools for data analysis

ArcGIS Pro provides both general-purpose functionality through the toolbar and specialized tools via the Geoprocessing pane. Through the Geoprocessing pane, users can search for and select various tools capable of reading from and writing to the data stored within the project folder. In practice, it is often useful to combine multiple tools into a workflow that takes input data (typically shapefiles or Feature Classes) and produces output data (Feature Classes or Standalone Tables).

Below is a description of some of the tools available in the Geoprocessing pane:

• Copy Features: Takes as input a shapefile or a Feature Class and produces a new Feature Class containing identical data.
• Count: Given a shapefile or a Feature Class, returns the number of records (or features) it contains.
• Select: Similar to Copy Features, but includes the ability to filter the output based on a user-defined Boolean condition.
• Buffer: Given a shapefile or a Feature Class, generates a buffer area around each geometric feature at a specified distance.
• Clip: Takes a Feature Class and a buffer as input and returns only those features from the Feature Class whose geometries fall entirely or partially within the buffer area.

Each of these tools corresponds to a specific function within the ArcPy library, as will be demonstrated in the following section.

Overview of the ArcPy library

As previously mentioned, ArcPy is a Python library that enables querying and manipulation of geospatial data through Python scripts that interact directly with ArcGIS applications. While ArcPy is compatible with both Python 2 and Python 3, ArcGIS Pro requires Python 3 to function properly. Like all Python libraries, ArcPy is organized into a modular structure. Its modules can be broadly categorized into two main groups:

• Modules that provide access to the tools available in the Geoprocessing pane (e.g., arcpy.analysis, arcpy.management)
• Modules that offer additional functionality beyond what is available through the Geoprocessing pane (e.g., arcpy.da, arcpy.mp)

With regard to the tools previously described, the following ArcPy functions correspond directly to them:

• arcpy.management.CopyFeatures
• arcpy.management.GetCount
• arcpy.analysis.Select
• arcpy.analysis.Buffer
• arcpy.analysis.Clip

Some of these functions will be revisited later when we introduce the code for creating a custom tool within the Geoprocessing pane. Among the modules offering extended functionality, the most significant are:

• The Data Access module (arcpy.da)
• The Mapping module (arcpy.mp)

Querying and manipulating data using the Data Access Module

The Data Access module allows for efficient access to both attribute data and geometries within Feature Classes by creating a special iterable object known as a cursor. There are three primary cursor types provided by the arcpy.da module:

• arcpy.da.SearchCursor: Creates a read-only cursor that can be used to iterate over records in a Feature Class, filtered by a subset of fields and an optional Boolean condition.
• arcpy.da.UpdateCursor: Creates a cursor that allows for searching, updating, and deleting records, given a Feature Class, a subset of fields, and an optional Boolean expression.
• arcpy.da.InsertCursor: Creates a cursor that enables the insertion of new records into a table or Feature Class.

Each record accessed by a cursor is returned as a Python list, with one element per field specified in the cursor’s constructor. Depending on the cursor type, different methods can be applied to modify or insert data.

In addition to these newer constructors introduced with the .da module, ArcPy also includes older cursor constructors at the top level of the library (arcpy.SearchCursor, arcpy.UpdateCursor, and arcpy.InsertCursor). However, these legacy versions are significantly slower — up to ten times less efficient. This is why it became necessary to develop highly optimized solutions, which ultimately converged into the Data Access module. This module enables efficient access to and manipulation of large volumes of data, while maintaining fast processing times. Examples of cursor usage will be provided in the development section.

Working with symbology via the Mapping Module

The arcpy Mapping Module was introduced with the aim of manipulating the layers belonging to the maps and their symbology, as an alternative to the functionalities already provided by the interface. This module gives the possibility to create an object associated with the project .aprx through the ArcGISProject constructor. Starting from the definition of this object, it is possible to access all the elements that make up the project and its properties. Among the main constituent elements, we have the layouts, useful for the creation of PDFs to document geospatial data, and maps. In particular, maps are composed of a set of layers, such as standalone tables, Feature Classes, and shapefiles dropped into the map. Each layer is associated with a symbology, which is the set of all the graphic and visual elements that, together, make it possible to represent a layer on a map. The symbology of a layer can be directly manipulated using a particular object defined in its properties, called renderer. Through the renderer, you can manipulate lines, borders, styles, and colors. For example, for nation shapefiles it can be very useful to associate individual polygons with different colors based on the values taken by a specific field, such as name, continent, or estimated population. There are different techniques for rendering layers, regarding the labeling of their geometric elements by color.

The main renderer classes are:

• Single Symbol: all instances associated with a layer have the same color with the same intensity, regardless of the values taken by their fields
• Unique Value: each instance has a different color that varies depending on the value of a specific nominal attribute (such as the name or the continent of reference)
• Graduated Colors: all instances associated with a layer have the same color, whose intensity level increases or decreases depending on the value of a numerical attribute (for example, the estimated population of different countries). It is possible to establish, either via interface or code, the palette of shades associated with a single color, as well as the number of intensity levels.

Below are two maps containing the shapefile layer of the world’s nations tagged with two color rendering techniques: Unique Value (based on name) and Graduated Colors (based on the estimated population field by nation).

Example of unique value rendering based on name.

Example of graduated color rendering based on estimated population per nation.

Building a Python Script Tool from scratch

The creation from scratch of a generic Python Script Tool takes place through the following steps:

1. Instantiation of a new tool using the New Script function of the .atbx file. This allows the automatic insertion of the tool into the Geoprocessing pane once the creation is completed.
2. Definition and configuration of the tool’s input parameters.
3. Implementation of the Python script and its integration into the tool.

Steps 2 and 3 have been further developed by defining a specific use case.

Description of the workflow

The defined Python Script Tool calculates the list of all cities located at a distance less than or equal to a given radius from a selected city. The list of cities is returned in tabular form, sorted in ascending order by distance from the selected city, and includes the respective distances. The end user provides the input parameters to the tool through a dialog box, which is automatically generated based on the parameter definitions. The values are entered using the following widgets:

• Two file uploaders for loading the shapefile of countries and the shapefile of world cities (both required, but already preset to a default path)
• A combo box containing the pick list of country names, extracted from the countries shapefile
• A combo box containing the pick list of city names belonging to the selected country (extracted from the cities shapefile and filtered based on the selected country)
• A custom widget for defining the buffer radius centered on the selected city, preset to a default value (in Linear Unit format, allowing the user to specify both the magnitude and the unit of measurement).
• A custom widget for defining a SQL expression that enables filtering of cities based on a Boolean condition specified by the user.

Parameter setup

In order to configure the parameters, it was necessary to correctly set the Properties tab of the implemented Python Script Tool. This tab includes several sections. In particular, it was first necessary to complete the Parameters section, followed by modifications to the Validation section. The Parameters section is used to define the type and order of the parameters, which will be reflected in the final tool dialog. In this section, six input parameters were defined, labeled from 0 to 5. Some of them were assigned default values or configured with dependency relationships to other parameters.

Definition table listing all parameters that configure the tool dialog.

The Validation section was modified to define dependencies between input parameters and the pick lists associated with the Country and City parameters. In particular, it was necessary to modify the updateParameters method of the ToolValidator class. This method is used to adjust the values and properties of parameters before the internal validation process is executed. The following code is required to define the Country parameter based on a shapefile that includes countries identified by the NAME field.

The updateParameters method, handling parameter dependencies and initialization logic.

Within the updateParameters method, logic was also implemented to manage duplicate city names within the same country (numerous cases of this type were found for the United States of America). In the presence of duplicate names, it was decided to include, within the corresponding items of the pick list, the associated province as well (indicated by the ADM1NAME field in the input shapefile).

Description of the Python code

Once the parameters were configured, the Python script was written. The implementation made use of all the necessary functionalities provided by the ArcPy library. The function arcpy.GetParameterAsText(index) was used to retrieve the parameters entered by the user through the tool dialog. This function returns the value of the parameter at the specified index in text format, as defined in the Parameters section. To define the workflow, some of the previously described functions were used. The following steps were implemented:

1. Creation of two Feature Classes from the two input shapefiles, named Countries and Cities, using the arcpy.management.CopyFeatures function.

#Insert all input data to be converted into Feature Classes 
inputCountries = arcpy.GetParameterAsText(0)
arcpy.management.CopyFeatures(inputCountries, "Countries")

#Retrieve the selected country from the country PickList
country = arcpy.GetParameterAsText(1)

inputCities = arcpy.GetParameterAsText(2)
arcpy.management.CopyFeatures(inputCities, "Cities")

#Retrieve the selected city from the city PickList
city = arcpy.GetParameterAsText(3)

arcpy.AddMessage("Selected Country: {0}".format(country))
arcpy.AddMessage("Selected City: {0}".format(city))

2. Creation of a Feature Class named SelectedCity, obtained by filtering the Cities Feature Class to include only the selected city, using the arcpy.analysis.Select function.

3. Creation of a buffer named CityBuffer, based on the radius specified by the user and generated from the SelectedCity Feature Class. The GEODESIC method was chosen to account for the curvature of the Earth in distance calculations. The arcpy.analysis.Buffer function was used.

4. Creation of a Feature Class named FilteredCity, obtained by filtering cities using the optional Boolean condition defined by the user. The arcpy.analysis.Select function was used again.

arcpy.analysis.Select("Cities", "SelectedCity", condSelectedCity)

#Retrieve the buffer radius selected by the user
radius = arcpy.GetParameterAsText(4)

#Constructing the buffer centered on the city
arcpy.analysis.Buffer(
    in_features="SelectedCity",
    out_feature_class="CityBuffer",
    buffer_distance_or_field= radius,
    line_side="FULL",
    line_end_type="ROUND",
    dissolve_option="NONE",
    dissolve_field=None,
    method="GEODESIC"
)

#Retrieve the optional condition used for filtering cities 
sqlExpression = arcpy.GetParameterAsText(5)

#Constructing the Feature Class that includes only cities filtered by the optional condition    
arcpy.analysis.Select("Cities", "FilteredCities", sqlExpression)

5. Creation of the CitiesWithinBuffer Feature Class, containing all FilteredCity instances located within the buffer. The arcpy.analysis.Clip function was applied.

6. Creation of a temporary table derived from CitiesWithinBuffer, containing the names of the cities within the buffer, their respective countries, and their distances from the center point. The geometries of the selected city and the other cities within the buffer were reconstructed using coordinates based on the WGS 1984 reference system. WGS 1984 (World Geodetic System 1984) is a global geodetic reference system primarily used for GPS navigation. It is based on a reference ellipsoid that approximates the shape of the Earth and provides a mathematical model for describing geographic positions on the Earth’s surface. Once the geometries were obtained, the angleAndDistanceTo function was used to calculate distances between them, with the method parameter set to GEODESIC to ensure consistency with the buffer radius. Read access to the CitiesWithinBuffer data was handled using a Search Cursor, while write access to the temporary table was managed using an Insert Cursor.

7. Creation of the final table, derived from the temporary table, by sorting the cities in ascending order of distance. The arcpy.management.Sort function was used.

workspace = arcpy.env.workspace
outputTable = "TempTable"
tablePath = r"{0}\{1}".format(workspace, outputTable)
arcpy.management.CreateTable(workspace, outputTable)
arcpy.management.AddField(tablePath, "City", "TEXT", field_length = 80)
arcpy.management.AddField(tablePath, "Country", "TEXT")
arcpy.management.AddField(tablePath, "DistanceInKm", "FLOAT")

fieldList = ["City", "Country", "DistanceInKm"]

outputCursor =  arcpy.da.InsertCursor(tablePath, fieldList)

#Set the geographic coordinate system to WGS 1984 for geometry creation.
#This setting is necessary to ensure the accurate calculation of geodetic distances

coordSystem = arcpy.SpatialReference(4326)

if(mainLongitude != None and mainLatitude != None):

    #Construct the geometry associated with the selected city, conventionally referred to as mainCityGeometry   
    mainCityGeometry = arcpy.PointGeometry(arcpy.Point(mainLongitude, mainLatitude), coordSystem) 

    for row in arcpy.da.SearchCursor("CitiesWithinBuffer", ["NAME", "ADM0NAME", "ADM1NAME", "SHAPE@XY"]):
        otherLongitude = row[3][0]
        otherLatitude = row[3][1]

        #Construct the geometry associated with each city
        otherCityGeometry = arcpy.PointGeometry(arcpy.Point(otherLongitude, otherLatitude), coordSystem)


        _, distance = mainCityGeometry.angleAndDistanceTo(otherCityGeometry, 'GEODESIC')
        distanceInKm = round(distance/1000)

        otherName = row[0]
        if row[0] in duplicatedNamesInSameState:
           otherName = "{0} ({1})".format(row[0], row[2])        

        outputCursor.insertRow([otherName, row[1], distanceInKm])        

del outputCursor       

#Create the output table by sorting the temporary table from the nearest to the farthest city within the buffer

arcpy.management.Sort('TempTable', 'OutputTable', [['DistanceInKm', 'ASCENDING']])

Running two use cases

The following are two distinct use cases of the developed Python Script Tool.

First use case

The objective is to display and store all cities located within 300 kilometers of the Swedish city of Malmö. In this particular case, it was not necessary to enter any SQL expression, as the goal was to retrieve the complete list of cities within the specified radius.

Parameter configuration in the Tool Dialog Interface for the first use case.

Graphical visualization of cities within 300 km of Malmö.

Visualization of the final table presenting cities located within 300 km of Malmö, sorted by distance.

Second use case

The objective is to display and store all capital cities located within 600 kilometers of the Swedish city of Malmö. In this specific case, to retrieve only the capital cities, it was necessary to enter an SQL expression that sets the Boolean field ADM0CAP to 1 (True).

Parameter configuration in the Tool Dialog Interface for the second use case.

Graphical visualization of capitals within 600 km of Malmö.

Visualization of the final table presenting capitals located within 600 km of Malmö, sorted by distance.

Conclusions and future developments

As we reach the conclusion of this article, we have progressively explored the universe of ArcGIS Pro, starting with introductory concepts and gradually moving toward its more advanced functionalities. After analyzing project organization and the hierarchical structure of vector data, we focused on the ArcPy library, highlighting the role of its functions in programmatically translating predefined tools and the importance of its modules dedicated to data analysis and processing. Following the theoretical discussion, we presented a practical case of implementing a fully customized tool, integrated into ArcGIS Pro and usable in black box mode through an intuitive interface.

This study has exclusively focused on vector data, whose tabular representation facilitates structured queries and analyses. However, the ArcPy ecosystem is not limited to this category: just as a python wraps around prey of different types, a Python script can leverage ArcPy to manage and process raster data, expanding the possibilities for spatial analysis. A potential future development could be the application of advanced techniques for raster image processing, utilizing ArcPy’s dedicated modules that harness artificial intelligence and deep learning for automatic feature extraction.

References

1. https://gis.oneteam.it/

2. https://www.udemy.com/course/arcpy-for-python-developers/?srsltid=AfmBOop8nmun18j_GrK5B6UdhnG00slh7lDn0fsoBYgKfhKK7nv0V8P5&couponCode=CP130525

3. https://pro.arcgis.com/en/pro-app/latest/arcpy/main/arcgis-pro-arcpy-reference.htm