Fast, Secure, Embedded — Running Apache Superset in Production the Right Way

How we deployed a fully embedded, multi-tenant analytics platform on GCP Cloud Run — and finally got dashboards that don’t make users stare at a loading spinner.

Our analytics stack was simple — BigQuery datasets piped into Looker dashboards, embedded inside our web application via iframes. It worked — until it didn’t.

Dashboards took 15–30 seconds to load. Filters were sluggish. Users started screenshotting charts instead of using the live dashboards because “it’s faster to just look at yesterday’s screenshot.” And the worst part? We needed row-level security — each client should only see their data — and Looker didn’t support it natively for embedded dashboards. We ended up building a fragile Apps Script workaround that was a nightmare to maintain.

That’s when we decided to explore Apache Superset.

Why Superset?

Apache Superset is an open-source data exploration and visualization platform. It connects directly to BigQuery, supports advanced caching, has a rich REST API, and — critically — supports embedded dashboards with guest tokens and row-level security (RLS) out of the box.

But “out of the box” is doing some heavy lifting in that sentence. Getting Superset production-ready on Google Cloud with proper multi-tenant isolation took real engineering. This article walks through exactly how we did it — every component, every decision, every lesson.

The Architecture: What We’re Building

Before diving into code, let’s understand the full picture — what components we need, why each one exists, and how data flows between them.

That’s six components. Let’s understand why each one is here.

The Components: What and Why

1. BigQuery — Your Analytics Data

This is where your actual business data lives. In our case, it’s a BigQuery dataset with tables like orders, revenue, user_activity — the data our clients want to see on dashboards. Superset connects to BigQuery as a data source using the sqlalchemy-bigquery driver, which lets it run SQL directly against your warehouse with no ETL or extraction needed.

BigQuery is the only database your end users’ queries touch. Everything else in this architecture exists to make that interaction fast, secure, and embeddable.

2. PostgreSQL (Cloud SQL) — Superset’s Metadata Store

Here’s something that surprises people: Superset needs its own database, separate from your analytics data. This PostgreSQL instance stores Superset’s internal state — think of it as Superset’s brain:

• Dashboard definitions (layout, chart positions, filter configurations)
• Chart configurations (which table, which columns, which visualization type)
• User accounts and roles
• Saved SQL queries
• Row-level security rules
• Audit logs

When you design a dashboard in Superset’s UI and hit “Save,” that configuration gets written to PostgreSQL. When a user loads that dashboard, Superset reads the config from PostgreSQL, then uses it to build SQL queries that run against BigQuery.

Why PostgreSQL and not just SQLite? Superset supports SQLite for local development, but it falls apart in production. Cloud Run can scale to multiple container instances, and SQLite is a local file — each instance would have its own separate database with no shared state. PostgreSQL gives us a single source of truth that all instances read from, plus proper ACID transactions, concurrent access, and backup/restore capabilities.

What the data looks like inside PostgreSQL:

 Schema |        Name        | Type  
--------+--------------------+-------
 public | dashboards         | table   ← Dashboard layouts and metadata
 public | slices             | table   ← Individual charts
 public | tables             | table   ← Registered data source tables
 public | sql_metrics        | table   ← Custom metric definitions
 public | row_level_security | table   ← RLS filter rules
 public | ab_user            | table   ← User accounts (Flask-AppBuilder)
 public | ab_role            | table   ← Roles (Admin, Gamma, etc.)
 public | logs               | table   ← Audit trail
 ...

3. Redis (Memorystore) — The Caching Layer

This is the single biggest performance win in the entire setup. Without Redis, here’s what happens every time a user loads a dashboard:

1. Superset reads dashboard config from PostgreSQL
2. Superset fetches table metadata (column names, types) from BigQuery
3. Superset loads filter options by querying BigQuery for distinct values
4. Superset executes the actual chart queries against BigQuery
5. Results render in the browser

Steps 2, 3, and 4 hit BigQuery — which is powerful but not fast for interactive queries. Each one can take 2–5 seconds. Multiply by 6 charts on a dashboard, and you’re looking at 15–30 seconds of loading.

With Redis, the flow becomes:

1. Superset checks Redis for cached table metadata → HIT (cached for 7 days)
2. Superset checks Redis for cached filter options → HIT (cached for 7 days)
3. Superset checks Redis for cached query results → HIT (cached for 1 hour)
4. Results render in the browser

Dashboard loads go from 15–30 seconds to 2–4 seconds. Filter interactions (which previously triggered fresh BigQuery queries) drop to under 1 second.

We use Google Cloud Memorystore for Redis with TLS encryption. It sits on a private IP inside our VPC, so only services with VPC access (like our Cloud Run instances) can reach it.

Why not just use Superset’s built-in SimpleCache? SimpleCache stores data in-process memory. With 4 Gunicorn workers per Cloud Run instance and potentially multiple instances, each worker has its own isolated cache. User A’s request warms the cache in Worker 2, but User B’s request hits Worker 3 and gets a cache miss. Redis is a shared, external cache that all workers and all instances read from — one cache warm-up benefits everyone.

4. Apache Superset — The Visualization Engine

Superset is the core of the setup. It’s an open-source BI platform that:

• Connects to BigQuery (and 40+ other databases) as data sources
• Provides a drag-and-drop UI for building charts and dashboards
• Supports SQL Lab for ad-hoc queries
• Has a REST API for programmatic access (which our backend uses)
• Supports embedded dashboards via guest tokens
• Enforces row-level security at the SQL query level

We deploy it as a Docker container on Cloud Run, configured with our custom superset_config.py that wires it to PostgreSQL, Redis, and BigQuery.

5. Backend Authentication Service — The Trust Bridge

This is a lightweight FastAPI app that solves a critical problem: how does Superset know who your users are?

Your application has its own authentication system — users log in, get a JWT, and interact with your app. Superset has its own, completely separate authentication system. The backend service bridges these two worlds:

1. It receives a JWT from your frontend (signed by your app)
2. It validates the JWT and extracts the user’s identity
3. It logs into Superset’s API as admin (server-to-server, never exposed to users)
4. It requests a guest token — a short-lived token that says “this user can view this specific dashboard with these data restrictions”
5. It returns the guest token to your frontend

The user never sees Superset credentials. They never interact with Superset’s login page. They just see a dashboard — their dashboard, with their data.

6. Your Frontend — The Embedding Layer

Your existing web application embeds Superset dashboards using the @superset-ui/embedded-sdk. It renders an iframe that loads the dashboard, authenticated with the guest token from the backend. To the end user, it looks like a native part of your application.

The Implementation

Now that we understand what each component does and why it exists, let’s build them.

Part 1: Deploying Superset on Cloud Run

The container is built from python:3.10-slim. The key dependencies to install are:

• apache-superset — the core application
• psycopg2-binary — connects to PostgreSQL
• sqlalchemy-bigquery — connects to BigQuery
• gunicorn + gevent — production WSGI server with async workers
• redis — connects to Memorystore
• google-cloud-secret-manager — fetches secrets at runtime

Set SUPERSET_CONFIG_PATH to point at your superset_config.py.

The entrypoint runs four things in sequence every time the container starts:

1. superset db upgrade — applies any pending database migrations to PostgreSQL (safe to run repeatedly; Alembic skips already-applied migrations)
2. Fetch the admin password from Secret Manager (never hardcode it)
3. superset fab create-admin — creates the admin user if it doesn't exist
4. superset init — initializes default roles and permissions
5. Start Gunicorn with 4 workers × 20 threads for 80 concurrent request slots

Why pull the password from Secret Manager at boot? No secrets in environment variables (they show up in Cloud Run revision diffs), and nothing baked into the Docker image.

Part 2: Superset Configuration (superset_config.py)

This file is the brain of the deployment. Superset reads it at startup to configure all connections and security settings.

Secret fetching — a utility function checks environment variables first (for local dev), then falls back to Secret Manager using the service account attached to the Cloud Run instance. No credentials files needed in production.

PostgreSQL connection — store credentials as a JSON blob in Secret Manager and construct the SQLAlchemy URI at startup. One important detail: URL-encode the password using urllib.parse.quote_plus(). Special characters like @, !, or # in passwords silently break the connection string in hard-to-debug ways.

Redis connection — Memorystore requires TLS. Pull the CA certificate and connection address from Secret Manager, write the cert to disk (the Redis client needs a file path, not a string), and use the rediss:// scheme (double-s) to enable SSL.

Cache layers — Superset has five independent cache layers, each tuned separately:

**CACHE_CONFIG** (24 hours) handles general internal metadata — permission checks, miscellaneous lookups.

**FILTER_STATE_CACHE_CONFIG** (7 days) stores each user's filter selections per dashboard. Without this, every visit resets all their filters — infuriating for power users who've set up 10+ filters.

**DATA_CACHE_CONFIG** (1 hour) caches actual query results from BigQuery. Shorter TTL because data freshness matters — you don't want users seeing yesterday's numbers all day. Even so, 1 hour eliminates redundant queries when multiple users load the same dashboard in the same window.

**TABLE_NAMES_CACHE_CONFIG** (7 days) is the most critical one for filter loading speed. Without it, Superset queries BigQuery's INFORMATION_SCHEMA on every dashboard load to fetch column names and types. Schema changes are rare, so 7 days is safe — and the speed difference is dramatic.

**EXPLORE_FORM_DATA_CACHE_CONFIG** (2 days) caches chart builder form metadata used in Superset's Explore view.

Give each cache a distinct key prefix (sp_data_, sp_filter_, sp_tables_) so you can selectively invalidate individual layers after a data refresh without flushing everything.

The TABLE_NAMES_CACHE_CONFIG is critical for filter loading speed — without it, Superset queries BigQuery's INFORMATION_SCHEMA on every dashboard load. Give each cache a distinct key prefix (sp_data_, sp_filter_, etc.) so you can selectively invalidate individual layers without flushing everything.

Embedding and security settings:

FEATURE_FLAGS = {
    "EMBEDDED_SUPERSET": True,          # Enables the /embedded/ endpoint
    "ENABLE_TEMPLATE_PROCESSING": True, # Enables Jinja2 in SQL (needed for RLS)
}
GUEST_ROLE_NAME = "Gamma"              # Read-only dashboard access
GUEST_TOKEN_JWT_EXP_SECONDS = 300      # 5-minute tokens — short by design
TALISMAN_ENABLED = False               # Disables X-Frame-Options: DENY
HTTP_HEADERS = {"X-Frame-Options": "ALLOWALL"}
SESSION_COOKIE_SAMESITE = "Lax"
ENABLE_CORS = True
CORS_OPTIONS = {
    "supports_credentials": True,
    "origins": ["https://your-app.com"],  # Lock down in production
}

Security note: ALLOWALL means any site could embed your Superset instance. In production, restrict CORS_OPTIONS["origins"] to your actual domain and add Content-Security-Policy headers.

Part 3: The Authentication Backend

This FastAPI service handles one job: exchange your app’s JWT for a Superset guest token.

On startup, load SUPERSET_SERVICE_URL (the internal Cloud Run URL of your Superset instance) and BACKEND_JWT_SECRET (the shared signing key between your app and this backend) from Secret Manager. Load them once at boot, reuse on every request. Include retry logic with a short delay — Cloud Run service account credentials sometimes aren't fully propagated during cold starts.

The /get-guest-token endpoint does four things in sequence:

Step 1 — Validate your app’s JWT. Extract the externalId (your client identifier) and role from the token. Reject anything expired, malformed, or unsigned.

Step 2 — Log into Superset as admin. POST to /api/v1/security/login with the admin credentials. This is server-to-server — the user never sees it. Fetch the admin password from Secret Manager on each request rather than caching it (login is ~5–10ms on the internal network; the simplicity is worth it).

Step 3 — Fetch a CSRF token. Superset requires CSRF tokens even for API-to-API calls. Use a requests.Session() object so the cookies from step 2 are carried through automatically.

Step 4 — Request the guest token. POST to /api/v1/security/guest_token/ with:

{
    "user": {
        "username": user_id,   # ← This becomes current_username() in RLS rules
        "first_name": "Embed",
        "last_name": "User",
    },
    "resources": [{"type": "dashboard", "id": dashboard_id}],
    "rls": []   # Covered in Part 4
}

Whatever you set as username here becomes the value of {{ current_username() }} inside Superset's RLS rules. This is the key link between your identity system and Superset's row-level security.

Part 4: Row-Level Security — The Secret Sauce

This is what made the entire migration worth it. Let me explain the problem first.

Say you run a SaaS platform where multiple client companies log in to see their analytics. Your BigQuery table looks something like this:

order_id client_id product revenue date 1001 acme_corp Widget A 5,200 2024–11–01 1002 globex_inc Widget B 3,800 2024–11–01 1003 acme_corp Widget C 7,100 2024–11–02 1004 initech Widget A 2,400 2024–11–02

When someone from Acme Corp opens the dashboard, they should see rows 1001 and 1003 — and nothing else. They shouldn’t even know Globex or Initech exist.

Approach 1: Static RLS Rules in the Superset UI

In Superset’s admin panel under Settings → Row Level Security, you create a rule:

Table:    orders
Clause:   client_id = '{{ current_username() }}'
Role:     Gamma

Now here’s the trick — remember the guest token we generate in the backend? We set the username to the client's identifier:

guest_payload = {
    "user": {
        "username": "acme_corp",  # This becomes current_username()
        "first_name": "Acme",
        "last_name": "User",
    },
    "resources": [{"type": "dashboard", "id": req.dashboard_id}],
    "rls": [],
}

When this user loads any chart built on the orders table, Superset silently rewrites the query:

-- What the chart defines:
SELECT product, SUM(revenue) FROM orders GROUP BY product
-- What actually runs (RLS injected automatically):
SELECT product, SUM(revenue) FROM orders
WHERE client_id = 'acme_corp'
GROUP BY product

Acme sees their $12,300 in revenue. They have no idea Globex or Initech exist. This isn’t a frontend filter that can be inspected or bypassed — it’s enforced at the SQL layer before results ever leave the database.

Approach 2: Dynamic RLS via the Guest Token

Static rules work great when access maps cleanly to a single column. But what if your access logic is more complex? Say a regional manager should see data for multiple clients in their territory.

Instead of maintaining static rules in the Superset UI, you can inject RLS clauses directly through the guest token payload:

# In your backend, after validating the user's JWT...
user_clients = get_clients_for_user(user_id)
# e.g., ["acme_corp", "initech"] for a regional manager
client_list = ", ".join(f"'{c}'" for c in user_clients)
guest_payload = {
    "user": {
        "username": user_id,
        "first_name": "Regional",
        "last_name": "Manager",
    },
    "resources": [{"type": "dashboard", "id": req.dashboard_id}],
    "rls": [
        {"clause": f"client_id IN ({client_list})"}
    ],
}

This generates a guest token that applies WHERE client_id IN ('acme_corp', 'initech') to every query — no static rules needed in Superset. Your backend is the single source of truth for who can see what, and the access logic can be as complex as your business requires: org hierarchy, role-based access, time-based permissions, whatever you need. The RLS clause is just SQL — if you can express it as a WHERE condition, you can enforce it.

Part 5: Deploying to Cloud Run

Both services deploy with VPC egress so they can reach PostgreSQL and Redis on private IPs.

Superset service needs more resources:

• --cpu 2 --memory 4Gi — Superset loads a full Flask application with SQLAlchemy connection pools
• --min-instances 1 — Superset's cold start involves database migrations and takes 30–60 seconds; keep at least one instance warm at all times
• --timeout 300s — some dashboard renders take time
• VPC connector with --vpc-egress private-ranges-only to reach PostgreSQL and Redis

Backend service is lightweight — default CPU and memory are fine. Cold starts are under 5 seconds, so --min-instances 0 is acceptable.

Service accounts — give each service only what it needs. The Superset service account needs secretmanager.secretAccessor and bigquery.user. The backend service account needs only secretmanager.secretAccessor. Neither needs more.

The Complete Request Flow

Here’s the end-to-end journey when a user opens an embedded dashboard:

The Results

After migrating from Looker to this setup:

Metric Looker Superset Dashboard load time 15–30s 2–4s Filter interaction 5–10s < 1s (cached) Embedding approach iframe (both use iframes) iframe via Embedded SDK Row-level security Required custom Apps Script workaround Native RLS with dynamic rules via guest tokens Caching Limited / opaque Multi-layer Redis (fully configurable) Cost Free (part of Google Cloud) Free + open source (infra costs for Cloud Run, Redis, Postgres)

Let’s be fair to Looker — it’s a free tool within the Google Cloud ecosystem, and it works well for straightforward reporting. Both tools use iframes for embedding. The problem wasn’t that Looker is a bad product. The problems were specific to our use case:

Caching was the biggest pain point. Looker’s caching model felt opaque to us — every filter interaction seemed to trigger a fresh BigQuery query, and we had limited control over what got cached and for how long. Superset’s multi-layer Redis cache gives us explicit, granular control. Filter options and metadata are served from memory in milliseconds, and even query results survive for an hour.

Row-level security was the dealbreaker. Looker doesn’t offer native row-level security for embedded dashboards out of the box. We had to build a custom Apps Script-based workaround to enforce per-user data access — fragile, hard to maintain, and a nightmare to debug. Superset’s RLS is a first-class feature that integrates directly into the guest token flow, letting us inject access rules dynamically from our backend.

Lessons Learned

1. Secret management matters more than you think. Early on we had secrets in environment variables, which showed up in Cloud Run revision diffs. Moving everything to Secret Manager with runtime fetching was one of the best decisions we made.

2. Superset’s filter cache is everything. Without FILTER_STATE_CACHE_CONFIG and TABLE_NAMES_CACHE_CONFIG properly configured, filters will be just as slow as what you're replacing. Configure these first.

3. Guest tokens expire fast — and that’s good. A 5-minute TTL means even if a token leaks, the blast radius is tiny. Your frontend should request a fresh token on each dashboard load.

4. Test RLS rules with SQL Lab. Before embedding anything, use SQL Lab to verify that RLS clauses are being applied correctly for different user contexts.

5. Cold starts are real. Superset on Cloud Run with 0 minimum instances means a 30–60 second cold start. Keep at least one instance warm.

6. PostgreSQL > SQLite, always in production. We tried SQLite initially to save costs. It worked — until Cloud Run scaled to 2 instances and each had its own database with different state. The migration to PostgreSQL was painful but necessary.

7. Redis key prefixes save you. When you need to debug cache issues or invalidate specific cache layers after a data refresh, having distinct prefixes (sp_data_, sp_filter_, sp_tables_) lets you target exactly what needs clearing without flushing everything.

Wrapping Up

Apache Superset isn’t a drop-in replacement for Looker or any other BI tool. It requires real infrastructure work — containerization, a metadata database, a caching layer, secret management, an auth backend, and careful configuration across all of them.

But what you get in return is a fully open-source, embeddable, fast analytics platform with flexible row-level security that you control completely. The six-component architecture might seem heavy at first, but each piece solves a specific problem, and the result is a system that’s faster, more secure, and more maintainable than our previous setup.

If your dashboards are slow, your embedding experience is clunky, or you need multi-tenant data isolation that your current tool can’t deliver natively — it might be time to give Apache Superset a serious look.