GCP IAM Permissions — From Design to Revocation — Part 2) Automated Revocation

Introduction

Photo by Vadim Bogulov on Unsplash

Managing IAM permissions in Google Cloud Platform (GCP) is not just about granting access — it’s also about ensuring that permissions are revoked when they are no longer needed. However, revoking permissions effectively presents two major challenges:

1. Verifying Unused Permissions is Difficult: It is not always clear whether a user truly does not need a permission anymore. Some permissions might only be used infrequently, making it difficult to determine if they are genuinely unnecessary.
2. Manual Revocation is Operationally Expensive: Even if we assume that permissions can be revoked after a certain period of inactivity, manually reviewing and removing them is impractical at scale.

Photo by Danielle Rice on Unsplash

As a result, organizations often struggle to maintain least privilege access, leading to excessive permissions that increase security risks. To solve this, I built a system that automatically tracks IAM role usage daily and revokes roles that have not been used for 30 days.

With this automation in place, I no longer have to worry about tracking and removing unused GCP permissions manually. In this post, I will explain how this system works in detail.

Essential Concepts Before We Begin

Before diving into the implementation, let’s clarify some key GCP IAM concepts to ensure a solid understanding of how permissions work in GCP.

1. What is a GCP IAM Role?

A GCP IAM Role is a collection of permissions that define what actions a user or service account can perform on a resource. Instead of assigning individual permissions directly, GCP groups them into roles for easier management.

For example, here are some common roles and their associated permissions:

• Viewer Role (roles/viewer) → Read-only access to resources (compute.instances.get, storage.buckets.get)
• Editor Role (roles/editor) → Read and write access to most resources (compute.instances.start, bigquery.jobs.create)
• Owner Role (roles/owner) → Full control, including the ability to modify IAM policies
• Custom Role (e.g., custom_role_for_dev) → A role defined by the organization with a specific set of permissions (e.g., compute.instances.create, bigquery.tables.create)

2. What is IAM Role Binding in GCP?

Since IAM Roles are collections of permissions, GCP grants permissions by assigning IAM Roles to users through a mechanism called Role Binding. A Role Binding associates:

• A GCP Project (or a folder or organization, but for simplicity, we will focus on projects)
• A User (e.g., derek@gmail.com)
• An IAM Role (a set of permissions)

A Role Binding could look like this:

GCP Project: feed-service
User: derek@gmail.com
IAM Role: Custom Role for Dev

This means that Derek has been granted a specific IAM Role within the feed-service project, giving him the associated permissions.

System Architecture Overview

If it’s too small to read, enlarge the image

The architecture of this system is designed to automate the tracking and revocation of IAM roles. Below is the high-level architecture diagram that illustrates how different components interact. In the following sections, I will explain each component in detail.

Tech Stack

This system is built using the following technologies:

• Golang → Used to develop the core services handling log querying, Redis updates, and role revocations.
• Cloud Run Jobs → Schedules and executes periodic tasks to track IAM role usage.
• GCP Memorystore (Redis) → Serves as a lightweight and fast storage solution for tracking last activity timestamps of IAM roles.

Setting Up the System

1. Building an Integrated Audit Log

To determine whether a user has used a specific IAM Role within a project, we need a centralized Audit Log system. This means consolidating audit logs from multiple projects into a single log aggregation project.

We need to ensure that audit logs are collected at the folder or organization level so that all projects report their IAM activity to one place. For details on setting this up, refer to this guide:

With this setup, we can efficiently query Log Explorer to track IAM Role usage across all projects.

[embed]Build an Integrated Audit Log System for Multiple GCP Projects “Streamline your GCP audit logs with a unified system. Centralize Admin Activity and Data Access logs from multiple…medium.com

2. Defining IAM Roles for Our Organization

Once audit logging is in place, the next step is to define which IAM Roles will be used in our GCP environment.

• Google provides predefined roles like roles/owner, roles/editor, and roles/viewer.
• However, organizations often require Custom Roles tailored to their specific needs.
• Custom Roles should be created at the Organization level, ensuring that they are inherited by all sub-projects for easier Role Binding management.

For a deeper dive into designing GCP IAM Roles effectively, check out this blog post:

[embed]GCP IAM Permissions: From Design to Revocation — Part 1) Design Roles Solve GCP IAM challenges! Learn to design roles, enable self-service & revoke permissions effectively with a DevSecOps…medium.com

3. Setting Up the Required Infrastructure

Now, we need to build the infrastructure components that power the automated role revocation system.

Cloud Run Jobs: Since this system does not require always-on servers, we use Cloud Run Jobs to periodically execute tasks. This job runs once per day to:

1. Query the audit logs to check which users have used their assigned IAM Roles.
2. Record activity timestamps in GCP Memorystore (Redis).
3. Identify users who have not used their IAM Roles in the past 30 days.
4. Revoke unused roles automatically.

GCP Memorystore (Redis): This serves as our fast lookup storage for tracking role usage. We store:

• Key: iam-role-binding
• Value: project_id&role&user
• Score: Timestamp (float64 format) The reason for choosing Redis is its ZRANGEBYSCORE command, which allows us to efficiently filter out entries older than 30 days without iterating through all data.

Implementing the Automated Role Revocation System

0. Preprocessing

a) Retrieve the list of Roles to be checked.

• Example: organizations/{ORG_ID}/roles/custom_role_for_dev

b) Fetch the set of permissions associated with each Role. Since querying all roles across GCP can be time-consuming, it is more efficient to limit the check to only the roles that are actively assigned to users in the organization. This ensures we only fetch and monitor the relevant permission sets, reducing unnecessary overhead.

• Example: [bigquery.bireservations.get, bigquery.capacityCommitments.get, bigquery.connections.create, ...]

c) Retrieve all GCP Projects in the organization.

d) Retrieve Role Binding information for each project.

• Example: &{feed-service organizations/{ORG_ID}/roles/custom_role_for_dev [derek@gmail.com]}

1. Recording Recent IAM Role Usage

• Query Cloud Logging API to check whether a user has utilized any permission associated with their IAM Role within the last 24 hours.
• Example Query:

resource.labels.project_id="feed-service" 
protoPayload.authenticationInfo.principalEmail="derek@gmail.com" 
protoPayload.authorizationInfo.permission=("resourcemanager.projects.get" OR "iam.serviceAccounts.get" OR ...) 
timestamp>="2025-03-14T00:00:00Z" AND timestamp<="2025-03-15T00:00:00Z"

• If a Role was used, store the data in Redis (Sorted Set):
• Key: iam-role-binding
• Value: feed-service&organizations/{ORG_ID}/roles/custom_role_for_dev&derek@gmail.com
• Score: Unix timestamp (float64 format)
• Redis Command Example:

ZADD iam-role-binding 1716172107 "feed-service&organizations/{ORG_ID}/roles/custom_role_for_dev&derek@gmail.com"

2. Revoking Unused IAM Roles

• Use ZRANGEBYSCORE to fetch entries older than 30 days.
• Example Command:

ZRANGEBYSCORE iam-role-binding -inf 1713572107

• Convert retrieved Role Binding data back into the appropriate IAM Role Binding format.
• Example: &{feed-service organizations/{ORG_ID}/roles/custom_role_for_dev [derek@gmail.com]}
• Use GCP IAM API to revoke the Role Binding.
• Delete the corresponding Redis entry after successful revocation.
• Example Command:

ZREM iam-role-binding "feed-service&organizations/{ORG_ID}/roles/custom_role_for_dev&derek@gmail.com"

Considerations and Best Practices

While this system effectively automates IAM Role revocation, there are a few important considerations to keep in mind:

1. Handling Service Accounts Carefully

• Unlike human users, Service Accounts often run critical workloads. Automatically revoking their permissions could cause unintended disruptions.
• Instead of revoking Service Account roles immediately, a safer approach is to send notifications or alerts when a Service Account has an unused role for 30+ days.

2. Optimizing for Large-Scale Organizations

• Checking IAM Role Bindings for an entire organization can be overwhelming. Organizations often have many projects, each containing multiple users, with each user potentially having several roles.
• To avoid performance bottlenecks, we utilize Goroutines (Concurrency) in Golang to efficiently process Role Binding checks in parallel.
• Since Role Binding checks are independent, we don’t need to run them sequentially. Instead, we leverage concurrent processing to significantly improve performance.

3. Cloud Logging API Rate Limits

• The Cloud Logging API has a strict quota, allowing only 200 requests per minute.
• Since Google rarely increases API quotas, we implement a rate-limiting mechanism to avoid hitting these limits.
• A simple approach is to introduce sleep intervals between requests, ensuring we stay within the allowed quota.

4. Cloud Logging Query Length Limitations

• A single query string sent to Cloud Logging must be under 20,000 characters.
• Since each IAM Role can contain hundreds of permissions, querying too many permissions in one request can exceed this limit.
• To handle this, we split queries into smaller chunks, ensuring we don’t exceed the length limit.
• Example: If custom_role_for_dev contains 1,200 permissions, we split them into batches of 300 permissions per query.
• Without this, Cloud Logging queries will fail entirely, making it impossible to retrieve Role usage data.

5. Cloud Run Jobs Failure Handling

• If a Cloud Run Job fails, no role usage data will be recorded for that day, potentially leading to incorrect revocations.
• It’s essential to set up error alerts and failure notifications to detect and respond to job failures promptly.
• Use Cloud Monitoring and Alerting to notify relevant teams when a job execution fails.

Photo by Jakub Żerdzicki on Unsplash

Have we ever truly managed IAM role assignments and revocations properly? It seems like a simple task, but expecting humans to do this manually is unrealistic. Building an automated system may sound complex, but with the approach outlined above, creating an automated permission revocation system is not difficult.

Furthermore, this system can be customized to fit different security policies — whether roles should be revoked after 60 days, 15 days, or any other threshold can be adjusted based on your organization’s needs.

With this in place, security audits will no longer reveal shocking cases like ex-employees from a year ago still having access to critical systems. It’s time to ensure our access control remains tight and our security posture remains strong!