Make your arcpy script faster and minimal writing intermediate-data to geodatabase

Faster and Faster, Photo by Chander R on Unsplash

As a GIS specialist, I was wondering how to save an output of geoprocessing tools as a memory not as a feature class or object that hard-coded in a harddisk i.e. file geodatabase. This is an interesting topic, since some of my collegues still thinking about it.

So, I found in Esri documentation about saving data to the memory instead of shapefile or feature class. Actually it is very simple. It uses only os library. The full syntax is os.path.join(“in_memory”, “name_of_the_feature”).

As you can see in the picture above, the output of the feature will be saved as a memory in during the process.

You may wonder why is this matter. This method is very effective when we create a geoprocessing services. We do not need all the intermediate data to be stored either in the enterprise geodatabase or file-geodatabase. This intermediate data is only be used if there is a process that required them to do next analyses.

However, there are also some drawbacks using this method.

This method are not likely to cover all spatial analyses. I have tried some spatial analyses with the output in_memory method. The drawbacks are

1. It cannot do selection spatial analyses tool. Select by attribute or select by location. If you want to select attribute or select by location, you must create the feature class first.
2. In_memory method cannot save feature dataset such as Network Dataset, Topology, Geometric Network, etc. It also applies to the spatial analyses depend on the dataset. You must create the feature class first.

Overall, this method is very usefull. In the normal script of a project where I always saved intermediate table, it takes about 5–10 minutes. However, when I used in_memory, the process cuts to 80–95% which is about 1–3 minutes. Now that is how you make your script simple and minimalist output.