Building a Beauty Brand Intelligence Dashboard with Bright Data

A Step-by-Step Guide Using OpenAI and Streamlit

In this guide, you will see:

• Everything you need to know to take your first steps with Bright Data for e-commerce monitoring.
• The most popular and effective approaches to gathering structured data from Amazon, Google Shopping, TikTok, and Instagram.
• How to build a custom brand ad monitoring dashboard.
• How to use Bright Data’s pre-built datasets and SERP APIs for real-time insights.
• How to integrate AI-powered analysis with OpenAI.
• How to deploy your dashboard for team collaboration.

Let’s dive in!

Getting Started with Bright Data for E-Commerce Intelligence

Bright Data is a comprehensive web data platform that provides enterprise-grade tools for scraping and collecting structured data from across the internet. It is particularly valued for its robust proxy infrastructure, compliance-focused scraping solutions, and ready-to-use datasets, which help businesses avoid the complexities of custom scrapers while ensuring data reliability and ethical sourcing. Over the years, Bright Data has become a go-to resource for marketers and analysts seeking actionable insights from e-commerce sites, social platforms, and search engines.

For e-commerce monitoring in the beauty and skincare sector, Bright Data offers two primary variants of data access:

• Pre-built Datasets: Static, high-volume collections of historical data, ideal for initial analysis and prototyping.
• SERP APIs and Custom Scrapers: Dynamic, real-time feeds that capture live changes, such as price fluctuations or trending content.

Depending on your needs — whether batch processing for trend analysis or live updates for competitive alerts — you can select the appropriate method, as outlined in this summary table:

With the growing emphasis on data-driven marketing in the beauty industry — where trends shift rapidly and competitive pricing can make or break campaigns — I turned to Bright Data to streamline a project: a real-time intelligence dashboard for tracking brand advertising, pricing, and social engagement. Scraping manually would have been inefficient and prone to errors, but Bright Data’s structured datasets provided clean, up-to-date information across multiple platforms, eliminating the need for fragile custom logic or constant maintenance.

In this post, I’ll walk you through how I built this tool using Bright Data datasets, Streamlit, Pandas, Plotly, and OpenAI.

What We’re Building

The core idea is straightforward: create an interactive dashboard that aggregates data from e-commerce and social sources to deliver key insights for beauty brand managers. Users can load datasets, filter by brand, visualize ad frequency and pricing trends, and receive AI-generated recommendations — all in a single, shareable interface.

For instance, a marketing team could identify that a competitor is ramping up sponsored Amazon listings while discounting 20% on Google Shopping, then use the dashboard’s insights to adjust their strategy accordingly.

To illustrate, here’s a short demo video of the dashboard in action, showcasing data loading, chart interactions, and AI output generation:

[embed]

This tool not only saves time but also empowers data-informed decisions in a fast-paced market.

How to Obtain Data from Bright Data: Pre-Built Datasets vs. SERP APIs

Bright Data excels in two main data acquisition methods: pre-built datasets for immediate use and SERP APIs for on-demand scraping. Pre-built datasets are curated collections covering platforms like Amazon and TikTok, available via a marketplace with previews and bulk downloads. SERP APIs, on the other hand, allow programmatic requests for fresh results, integrating seamlessly into applications like our dashboard.

For beauty brand monitoring, datasets provide comprehensive historical views (e.g., product prices over time), while APIs enable real-time checks (e.g., current sponsored listings). Both ensure compliance with terms of service through managed proxies and structured outputs.

To get started, sign up for a Bright Data account at brightdata.com and explore the free tier for testing.

Tools and Technology Stack

This project leverages a minimal, efficient stack focused on rapid development and visualization:

• Streamlit: For building the interactive web dashboard.
• Pandas and NumPy: For data processing and cleaning.
• Plotly: For dynamic charts (histograms, bar graphs, pie charts).
• OpenAI API: For generating actionable insights from analyzed data.
• Requests: For API calls to Bright Data’s SERP endpoints.
• Python 3.12: As the core runtime environment.

These tools combine to create a lightweight application that runs locally or deploys easily to the cloud.

Getting the Data: Using Bright Data Datasets

Here’s how I obtained the datasets for Amazon products, Google Shopping results, TikTok posts, and Instagram profiles:

Step 1: Sign In to Bright Data Head over to Bright Data and log in to your dashboard. If you don’t have an account, you can create one for free.

Step 2: Open the Dataset Marketplace On the dashboard sidebar, click on Web Datasets, then select **Dataset Marketplace**.

Step 3: Search for the Amazon Products Dataset In the search bar, type “Amazon Products”. Click on the result that matches, preview the dataset (including fields like price, brand, and reviews), and proceed to purchase it.

Once purchased, download the dataset in CSV format.

Step 4: Do the Same for Google Shopping Repeat the process — this time searching for “Google Shopping Dataset”. Preview listings with pricing and ratings, then purchase and download.

Step 5: Explore TikTok and Instagram Datasets For TikTok, search “TikTok Posts” to access video descriptions, hashtags, and engagement metrics. For Instagram, query “Instagram Profiles” for bios, follower counts, and business details. Purchase, download, and preview each.

Even search for the “**Social Media**” all the relevant datasets will appear —

Alternatively, you can obtain live data via Bright Data’s SERP APIs or web scrapers, available in both code and no-code formats for ongoing monitoring.

Step 6: Explore the CSV Files Each CSV provides a structured format with fields like product title, price, engagement counts, and timestamps — ideal for direct integration into analysis tools.

Step-by-Step Implementation: From Virtual Environment to Project Structure

Project Structure Organize your files as follows for clarity:

text

brand-ad-monitor/
├── app.py                  # Main Streamlit application
├── requirements.txt        # Dependencies
├── Amazon products.csv     # Downloaded dataset
├── Google Shopping.csv
├── TikTok - Posts.csv
├── Instagram - Profiles.csv
└── .env                    # Environment variables (API keys)

Step 1: Set Up a Virtual Environment

python -m venv venv
# Activate: source venv/bin/activate (macOS/Linux) or venv\Scripts\activate (Windows)

Step 2: Install Dependencies: Create requirements.txt with the listed packages, then run:

pip install -r requirements.txt

Step 3: Configure Environment Variables: In .env, add:

OPENAI_API_KEY=your_openai_key
BRIGHTDATA_BEARER_TOKEN=your_brightdata_token
DATASET_ID=your_dataset_token

Step 4: Implement the Core Application: Use the provided app.py script (from the repository). Key components include:

• Data loading functions that handle CSV parsing with encoding fallbacks.
• An analyzer class for extracting brands, pricing, and categories.
• AI integration for insight generation.
• Interactive tabs for visualizations.

import streamlit as st
import pandas as pd
import plotly.express as px
import plotly.graph_objects as go
from datetime import datetime
import requests
import json
from typing import List, Dict
import numpy as np

# OpenAI import - compatible with both old and new versions
try:
    from openai import OpenAI
    OPENAI_NEW_VERSION = True
except ImportError:
    import openai
    OPENAI_NEW_VERSION = False

# Page config
st.set_page_config(
    page_title="Brand Ad Monitor",
    page_icon="📊",
    layout="wide",
    initial_sidebar_state="expanded"
)

# Custom CSS
st.markdown("""
<style>
    .main-header {
        font-size: 2.5rem;
        font-weight: bold;
        color: #8b5cf6;
        text-align: center;
        margin-bottom: 1rem;
    }
    .metric-card {
        background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
        padding: 1.5rem;
        border-radius: 10px;
        color: white;
        text-align: center;
    }
    .insight-box {
        background-color: #f3f4f6;
        padding: 1rem;
        border-radius: 8px;
        border-left: 4px solid #8b5cf6;
        margin: 0.5rem 0;
    }
</style>
""", unsafe_allow_html=True)

# Initialize session state
if 'amazon_data' not in st.session_state:
    st.session_state.amazon_data = None
if 'google_data' not in st.session_state:
    st.session_state.google_data = None
if 'tiktok_data' not in st.session_state:
    st.session_state.tiktok_data = None
if 'instagram_profiles_data' not in st.session_state:
    st.session_state.instagram_profiles_data = None
if 'api_key' not in st.session_state:
    st.session_state.api_key = None
if 'openai_key' not in st.session_state:
    st.session_state.openai_key = None

# Bright Data API Configuration
BRIGHTDATA_API_URL = "https://api.brightdata.com/datasets/v3/trigger"
BRIGHTDATA_BEARER_TOKEN = "<bearer_token>"
DATASET_ID = "<dataset_id>"

class BrightDataScraper:
    """Handle Bright Data API interactions"""

    @staticmethod
    def scrape_amazon_products(urls: List[Dict]) -> Dict:
        """Scrape Amazon products using Bright Data API"""
        headers = {
            "Authorization": f"Bearer {BRIGHTDATA_BEARER_TOKEN}",
            "Content-Type": "application/json"
        }

        params = {
            "dataset_id": DATASET_ID,
            "include_errors": "true"
        }

        try:
            response = requests.post(
                BRIGHTDATA_API_URL,
                headers=headers,
                params=params,
                json=urls,
                timeout=30
            )

            if response.status_code == 200:
                return {"success": True, "data": response.json()}
            else:
                return {"success": False, "error": f"API Error: {response.status_code}"}
        except Exception as e:
            return {"success": False, "error": str(e)}

class DataAnalyzer:
    """Analyze data and extract patterns"""

    def __init__(self, amazon_df, google_df, tiktok_df=None, instagram_profiles_df=None):
        self.amazon_df = amazon_df
        self.google_df = google_df
        self.tiktok_df = tiktok_df
        self.instagram_profiles_df = instagram_profiles_df

    def get_beauty_brands(self) -> List[str]:
        """Extract beauty/skincare brands"""
        beauty_keywords = ['beauty', 'skincare', 'skin', 'serum', 'cream', 
                          'moisturizer', 'cleanser', 'cosmetic', 'lotion', 'facial']
        brands = set()

        # From Amazon
        if self.amazon_df is not None and not self.amazon_df.empty:
            for idx, row in self.amazon_df.iterrows():
                if pd.notna(row.get('brand')) and pd.notna(row.get('categories')):
                    cat_lower = str(row['categories']).lower()
                    if any(kw in cat_lower for kw in beauty_keywords):
                        brands.add(row['brand'])

        # From Google Shopping
        if self.google_df is not None and not self.google_df.empty:
            for idx, row in self.google_df.iterrows():
                if pd.notna(row.get('title')):
                    title_lower = str(row['title']).lower()
                    if any(kw in title_lower for kw in beauty_keywords):
                        words = str(row['title']).split()
                        if len(words) > 0:
                            brands.add(words[0])

        # From TikTok (using descriptions/hashtags)
        if self.tiktok_df is not None and not self.tiktok_df.empty:
            for idx, row in self.tiktok_df.iterrows():
                desc = str(row.get('description', '')).lower()
                if any(kw in desc for kw in beauty_keywords):
                    words = str(row.get('description', '')).split()
                    if len(words) > 0:
                        brands.add(words[0] if len(words) > 0 else 'TikTok Brand')

        # From Instagram Profiles (using biography)
        if self.instagram_profiles_df is not None and not self.instagram_profiles_df.empty:
            for idx, row in self.instagram_profiles_df.iterrows():
                bio = str(row.get('biography', '')).lower()
                if any(kw in bio for kw in beauty_keywords):
                    profile_name = str(row.get('profile_name', ''))
                    if profile_name:
                        brands.add(profile_name)

        return sorted(list(brands))

    def analyze_pricing(self, brand_filter=None):
        """Analyze pricing patterns with robust data type handling"""
        data = []

        # Amazon data
        if self.amazon_df is not None:
            df = self.amazon_df.copy()
            if brand_filter and brand_filter != "All Brands":
                df = df[df['brand'] == brand_filter]

            for idx, row in df.iterrows():
                if pd.notna(row.get('final_price')):
                    try:
                        price = float(row['final_price'])
                        discount = float(row.get('discount', 0)) if pd.notna(row.get('discount')) else 0

                        data.append({
                            'source': 'Amazon',
                            'brand': str(row.get('brand', 'Unknown')),
                            'price': price,
                            'discount': discount
                        })
                    except (ValueError, TypeError):
                        continue

        # Google data
        if self.google_df is not None:
            df = self.google_df.copy()
            for idx, row in df.iterrows():
                if pd.notna(row.get('item_price')):
                    try:
                        price_str = str(row['item_price']).replace('$', '').replace(',', '').strip()
                        price = float(price_str)
                        brand = str(row.get('title', '')).split()[0] if pd.notna(row.get('title')) else 'Unknown'

                        if not brand_filter or brand_filter == "All Brands" or brand == brand_filter:
                            data.append({
                                'source': 'Google Shopping',
                                'brand': brand,
                                'price': price,
                                'discount': 0
                            })
                    except (ValueError, TypeError, IndexError):
                        continue

        return pd.DataFrame(data)

    def analyze_ad_frequency(self):
        """Analyze sponsored vs organic listings with robust data handling"""
        frequency_data = {}

        if self.amazon_df is not None:
            for idx, row in self.amazon_df.iterrows():
                brand = row.get('brand', 'Unknown')
                if pd.notna(brand) and brand != 'Unknown':
                    brand_str = str(brand)

                    if brand_str not in frequency_data:
                        frequency_data[brand_str] = {
                            'sponsored': 0,
                            'organic': 0,
                            'total_reviews': 0
                        }

                    sponsored_val = str(row.get('sponsered', '')).lower()
                    if sponsored_val in ['true', '1', 'yes']:
                        frequency_data[brand_str]['sponsored'] += 1
                    else:
                        frequency_data[brand_str]['organic'] += 1

                    try:
                        reviews = float(row.get('reviews_count', 0))
                        if pd.notna(reviews):
                            frequency_data[brand_str]['total_reviews'] += int(reviews)
                    except (ValueError, TypeError):
                        pass

        if not frequency_data:
            return pd.DataFrame(columns=['brand', 'sponsored', 'organic', 'total_reviews'])

        df = pd.DataFrame([
            {'brand': k, **v} for k, v in frequency_data.items()
        ])
        return df.sort_values('sponsored', ascending=False)

    def analyze_category_distribution(self):
        """Analyze product category distribution"""
        categories = {}

        if self.amazon_df is not None:
            for idx, row in self.amazon_df.iterrows():
                if pd.notna(row.get('bs_category')):
                    cat = str(row['bs_category']).split('>')[0].strip()
                    categories[cat] = categories.get(cat, 0) + 1

        return pd.DataFrame([
            {'category': k, 'count': v} 
            for k, v in sorted(categories.items(), key=lambda x: x[1], reverse=True)
        ])

class AIInsightGenerator:
    """Generate AI insights using OpenAI - supports both old and new API versions"""

    def __init__(self, api_key):
        self.api_key = api_key
        self.client = None

        if api_key:
            if OPENAI_NEW_VERSION:
                self.client = OpenAI(api_key=api_key)
            else:
                import openai
                openai.api_key = api_key

    def generate_insights(self, analysis_data: Dict) -> List[str]:
        """Generate marketing insights using GPT"""
        if not self.api_key:
            return self._generate_basic_insights(analysis_data)

        try:
            prompt = f"""
            As a marketing intelligence analyst, analyze this e-commerce data for beauty/skincare brands:

            Total Brands: {analysis_data['total_brands']}
            Total Products: {analysis_data['total_products']}
            Top Sponsored Brand: {analysis_data['top_sponsored_brand']} ({analysis_data['sponsored_count']} ads)
            Average Discount: {analysis_data['avg_discount']}%
            Price Range Distribution: {analysis_data['price_distribution']}

            Provide 5 actionable marketing insights and recommendations. Focus on:
            Ad strategy effectiveness
            Pricing positioning
            Competitive opportunities
            Market trends
            Strategic recommendations

            Format each insight as a clear, concise bullet point.
            """

            if OPENAI_NEW_VERSION:
                response = self.client.chat.completions.create(
                    model="gpt-3.5-turbo",
                    messages=[
                        {"role": "system", "content": "You are a marketing intelligence expert specializing in e-commerce and beauty/skincare brands."},
                        {"role": "user", "content": prompt}
                    ],
                    max_tokens=500,
                    temperature=0.7
                )
                content = response.choices[0].message.content
            else:
                import openai
                response = openai.ChatCompletion.create(
                    model="gpt-3.5-turbo",
                    messages=[
                        {"role": "system", "content": "You are a marketing intelligence expert specializing in e-commerce and beauty/skincare brands."},
                        {"role": "user", "content": prompt}
                    ],
                    max_tokens=500,
                    temperature=0.7
                )
                content = response.choices[0].message.content

            insights = content.strip().split('\n')
            return [i.strip() for i in insights if i.strip()]

        except Exception as e:
            st.warning(f"OpenAI API error: {str(e)}. Using basic insights.")
            return self._generate_basic_insights(analysis_data)

    def _generate_basic_insights(self, data: Dict) -> List[str]:
        """Generate basic insights without AI"""
        return [
            f"🎯 {data['top_sponsored_brand']} dominates with {data['sponsored_count']} sponsored listings - aggressive market positioning",
            f"💰 Average discount of {data['avg_discount']}% indicates competitive pricing pressure in the market",
            f"📊 {data['total_brands']} active beauty brands tracked across {data['total_products']} products",
            f"🔍 Most products concentrated in {data['price_distribution']} range - optimal pricing sweet spot",
            f"📈 Recommendation: Monitor top performers' ad frequency and pricing strategy for competitive advantage"
        ]

def load_data():
    """Load CSV data with proper error handling and data cleaning"""
    try:
        amazon_df = pd.read_csv('Amazon products.csv', encoding='utf-8', on_bad_lines='skip')

        if 'discount' in amazon_df.columns:
            amazon_df['discount'] = pd.to_numeric(amazon_df['discount'], errors='coerce')

        if 'final_price' in amazon_df.columns:
            amazon_df['final_price'] = pd.to_numeric(amazon_df['final_price'], errors='coerce')
        if 'initial_price' in amazon_df.columns:
            amazon_df['initial_price'] = pd.to_numeric(amazon_df['initial_price'], errors='coerce')

        if 'reviews_count' in amazon_df.columns:
            amazon_df['reviews_count'] = pd.to_numeric(amazon_df['reviews_count'], errors='coerce').fillna(0)

        st.session_state.amazon_data = amazon_df
        st.success(f"✅ Loaded {len(amazon_df)} Amazon products")
    except Exception as e:
        st.error(f"Error loading Amazon data: {str(e)}")
        st.session_state.amazon_data = None

    try:
        google_df = pd.read_csv('Google Shopping.csv', encoding='utf-8', on_bad_lines='skip')
        st.session_state.google_data = google_df
        st.success(f"✅ Loaded {len(google_df)} Google Shopping products")
    except Exception as e:
        st.error(f"Error loading Google data: {str(e)}")
        st.session_state.google_data = None

    # Load TikTok data (replacing Instagram posts)
    try:
        tiktok_df = None
        for encoding in ['utf-8', 'latin-1', 'iso-8859-1', 'cp1252']:
            try:
                tiktok_df = pd.read_csv('TikTok - Posts.csv', encoding=encoding, on_bad_lines='skip')
                if not tiktok_df.empty:
                    break
            except:
                continue

        if tiktok_df is not None and not tiktok_df.empty:
            st.session_state.tiktok_data = tiktok_df
            st.success(f"✅ Loaded {len(tiktok_df)} TikTok posts")
        else:
            st.info("🎵 TikTok data file is empty or not available")
            st.session_state.tiktok_data = None
    except Exception as e:
        st.info(f"🎵 TikTok data not available: {str(e)}")
        st.session_state.tiktok_data = None

    # Load Instagram Profiles data
    try:
        instagram_profiles_df = None
        for encoding in ['utf-8', 'latin-1', 'iso-8859-1', 'cp1252']:
            try:
                instagram_profiles_df = pd.read_csv('Instagram - Profiles.csv', encoding=encoding, on_bad_lines='skip')
                if not instagram_profiles_df.empty:
                    break
            except:
                continue

        if instagram_profiles_df is not None and not instagram_profiles_df.empty:
            st.session_state.instagram_profiles_data = instagram_profiles_df
            st.success(f"✅ Loaded {len(instagram_profiles_df)} Instagram profiles")
        else:
            st.info("📷 Instagram profiles data file is empty or not available")
            st.session_state.instagram_profiles_data = None
    except Exception as e:
        st.info(f"📷 Instagram profiles data not available: {str(e)}")
        st.session_state.instagram_profiles_data = None

def main():
    st.markdown('<h1 class="main-header">🛍️ Beauty & Skincare Ad Monitor</h1>', unsafe_allow_html=True)
    st.markdown("#### AI-Powered Brand Advertising Intelligence Platform")

    with st.sidebar:
        st.image("https://img.icons8.com/fluency/96/000000/analytics.png", width=80)
        st.title("⚙️ Configuration")

        openai_key = st.text_input("OpenAI API Key (Optional)", type="password", 
                                   help="For AI-powered insights")
        if openai_key:
            st.session_state.openai_key = openai_key

        st.divider()

        if st.button("📥 Load Data", use_container_width=True):
            with st.spinner("Loading data..."):
                load_data()

        st.divider()

        st.subheader("🔍 Scrape New Products")
        urls_input = st.text_area(
            "Amazon URLs (one per line)",
            placeholder="https://www.amazon.com/product1\nhttps://www.amazon.com/product2",
            height=150
        )

        zipcode = st.text_input("Zipcode (Optional)", "94107")

        if st.button("🚀 Scrape Products", use_container_width=True):
            if urls_input.strip():
                urls = []
                for url in urls_input.strip().split('\n'):
                    if url.strip():
                        urls.append({
                            "url": url.strip(),
                            "zipcode": zipcode,
                            "language": ""
                        })

                with st.spinner("Scraping products via Bright Data API..."):
                    scraper = BrightDataScraper()
                    result = scraper.scrape_amazon_products(urls)

                    if result['success']:
                        st.success("✅ Scraping job submitted! Data will be available soon.")
                        st.json(result['data'])
                    else:
                        st.error(f"❌ Error: {result['error']}")
            else:
                st.warning("Please enter at least one URL")

        st.divider()
        st.info("💡 **Tip**: Load existing data first, then optionally scrape new products")

    if st.session_state.amazon_data is None and st.session_state.google_data is None:
        st.warning("⚠️ Please load data using the sidebar to begin analysis")
        st.stop()

    analyzer = DataAnalyzer(
        st.session_state.amazon_data,
        st.session_state.google_data,
        st.session_state.tiktok_data,
        st.session_state.instagram_profiles_data
    )

    brands = analyzer.get_beauty_brands()

    col1, col2, col3 = st.columns([2, 1, 1])
    with col1:
        selected_brand = st.selectbox("🎯 Select Brand", ["All Brands"] + brands)
    with col2:
        st.metric("Total Brands", len(brands))
    with col3:
        total_products = len(st.session_state.amazon_data) if st.session_state.amazon_data is not None else 0
        total_products += len(st.session_state.google_data) if st.session_state.google_data is not None else 0
        st.metric("Total Products", total_products)

    st.divider()

    st.subheader("📊 Key Performance Indicators")

    ad_freq_df = analyzer.analyze_ad_frequency()
    pricing_df = analyzer.analyze_pricing(selected_brand)
    category_df = analyzer.analyze_category_distribution()

    top_sponsored = ad_freq_df.iloc[0] if not ad_freq_df.empty else None

    avg_discount = 0
    if st.session_state.amazon_data is not None and 'discount' in st.session_state.amazon_data.columns:
        discount_col = pd.to_numeric(st.session_state.amazon_data['discount'], errors='coerce')
        avg_discount = discount_col.mean()
        if pd.isna(avg_discount):
            avg_discount = 0

    avg_price = pricing_df['price'].mean() if not pricing_df.empty else 0
    total_reviews = ad_freq_df['total_reviews'].sum() if not ad_freq_df.empty else 0

    col1, col2, col3, col4 = st.columns(4)

    with col1:
        st.markdown("""
        <div class="metric-card">
            <h3 style="margin:0;">🏆 Top Advertiser</h3>
            <h2 style="margin:0.5rem 0;">{}</h2>
            <p style="margin:0;">{} sponsored ads</p>
        </div>
        """.format(
            top_sponsored['brand'] if top_sponsored is not None else "N/A",
            int(top_sponsored['sponsored']) if top_sponsored is not None else 0
        ), unsafe_allow_html=True)

    with col2:
        st.markdown("""
        <div class="metric-card" style="background: linear-gradient(135deg, #f093fb 0%, #f5576c 100%);">
            <h3 style="margin:0;">💰 Avg Discount</h3>
            <h2 style="margin:0.5rem 0;">{:.1f}%</h2>
            <p style="margin:0;">Across all products</p>
        </div>
        """.format(avg_discount if pd.notna(avg_discount) else 0), unsafe_allow_html=True)

    with col3:
        st.markdown("""
        <div class="metric-card" style="background: linear-gradient(135deg, #4facfe 0%, #00f2fe 100%);">
            <h3 style="margin:0;">💵 Avg Price</h3>
            <h2 style="margin:0.5rem 0;">${:.2f}</h2>
            <p style="margin:0;">Average product price</p>
        </div>
        """.format(avg_price if pd.notna(avg_price) else 0), unsafe_allow_html=True)

    with col4:
        st.markdown("""
        <div class="metric-card" style="background: linear-gradient(135deg, #43e97b 0%, #38f9d7 100%);">
            <h3 style="margin:0;">⭐ Total Reviews</h3>
            <h2 style="margin:0.5rem 0;">{:,}</h2>
            <p style="margin:0;">Customer reviews</p>
        </div>
        """.format(int(total_reviews)), unsafe_allow_html=True)

    st.divider()

    st.subheader("🤖 AI-Generated Insights")

    price_ranges = pricing_df['price'].describe() if not pricing_df.empty else pd.Series()
    analysis_data = {
        'total_brands': len(brands),
        'total_products': total_products,
        'top_sponsored_brand': top_sponsored['brand'] if top_sponsored is not None else "N/A",
        'sponsored_count': int(top_sponsored['sponsored']) if top_sponsored is not None else 0,
        'avg_discount': f"{avg_discount:.1f}" if pd.notna(avg_discount) else "0",
        'price_distribution': f"${price_ranges.get('25%', 0):.0f}-${price_ranges.get('75%', 0):.0f}" if not pricing_df.empty else "N/A"
    }

    ai_generator = AIInsightGenerator(st.session_state.openai_key)
    insights = ai_generator.generate_insights(analysis_data)

    for insight in insights:
        st.markdown(f'<div class="insight-box">{insight}</div>', unsafe_allow_html=True)

    st.divider()

    tab1, tab2, tab3, tab4 = st.tabs(["📈 Ad Frequency", "💰 Pricing Analysis", "📊 Categories", "🔍 Detailed Data"])

    with tab1:
        st.subheader("Sponsored vs Organic Listings by Brand")

        if not ad_freq_df.empty:
            fig = go.Figure()
            fig.add_trace(go.Bar(
                name='Sponsored',
                x=ad_freq_df['brand'][:10],
                y=ad_freq_df['sponsored'][:10],
                marker_color='#8b5cf6'
            ))
            fig.add_trace(go.Bar(
                name='Organic',
                x=ad_freq_df['brand'][:10],
                y=ad_freq_df['organic'][:10],
                marker_color='#ec4899'
            ))

            fig.update_layout(
                barmode='group',
                height=500,
                xaxis_title="Brand",
                yaxis_title="Number of Listings",
                hovermode='x unified'
            )
            st.plotly_chart(fig, use_container_width=True)

            st.dataframe(
                ad_freq_df.head(10).style.background_gradient(cmap='Purples'),
                use_container_width=True
            )
        else:
            st.info("No ad frequency data available")

    with tab2:
        st.subheader("Price Distribution Analysis")

        if not pricing_df.empty:
            col1, col2 = st.columns(2)

            with col1:
                fig = px.histogram(
                    pricing_df,
                    x='price',
                    nbins=30,
                    title="Price Distribution",
                    color_discrete_sequence=['#8b5cf6']
                )
                fig.update_layout(height=400)
                st.plotly_chart(fig, use_container_width=True)

            with col2:
                fig = px.box(
                    pricing_df,
                    x='source',
                    y='price',
                    title="Price Range by Source",
                    color='source',
                    color_discrete_sequence=['#8b5cf6', '#ec4899']
                )
                fig.update_layout(height=400)
                st.plotly_chart(fig, use_container_width=True)

            if 'discount' in pricing_df.columns:
                discount_data = pricing_df[pricing_df['discount'] > 0].groupby('brand')['discount'].mean().sort_values(ascending=False).head(10)

                if not discount_data.empty:
                    st.subheader("Top Discounting Brands")
                    fig = px.bar(
                        x=discount_data.index,
                        y=discount_data.values,
                        labels={'x': 'Brand', 'y': 'Average Discount (%)'},
                        color=discount_data.values,
                        color_continuous_scale='Reds'
                    )
                    fig.update_layout(height=400)
                    st.plotly_chart(fig, use_container_width=True)
        else:
            st.info("No pricing data available")

    with tab3:
        st.subheader("Category Distribution")

        if not category_df.empty:
            col1, col2 = st.columns(2)

            with col1:
                fig = px.pie(
                    category_df.head(8),
                    values='count',
                    names='category',
                    title="Top Categories Distribution"
                )
                fig.update_layout(height=400)
                st.plotly_chart(fig, use_container_width=True)

            with col2:
                fig = px.bar(
                    category_df.head(10),
                    x='count',
                    y='category',
                    orientation='h',
                    title="Category Product Count",
                    color='count',
                    color_continuous_scale='Viridis'
                )
                fig.update_layout(height=400)
                st.plotly_chart(fig, use_container_width=True)
        else:
            st.info("No category data available")

    with tab4:
        st.subheader("Raw Data Explorer")

        data_source = st.radio("Select Data Source", ["Amazon", "Google Shopping", "TikTok", "Instagram Profiles"])

        if data_source == "Amazon" and st.session_state.amazon_data is not None:
            st.dataframe(st.session_state.amazon_data, use_container_width=True, height=400)

            csv = st.session_state.amazon_data.to_csv(index=False)
            st.download_button(
                label="📥 Download Amazon Data",
                data=csv,
                file_name="amazon_products.csv",
                mime="text/csv"
            )

        elif data_source == "Google Shopping" and st.session_state.google_data is not None:
            st.dataframe(st.session_state.google_data, use_container_width=True, height=400)

            csv = st.session_state.google_data.to_csv(index=False)
            st.download_button(
                label="📥 Download Google Shopping Data",
                data=csv,
                file_name="google_shopping.csv",
                mime="text/csv"
            )

        elif data_source == "TikTok" and st.session_state.tiktok_data is not None:
            st.dataframe(st.session_state.tiktok_data, use_container_width=True, height=400)

            csv = st.session_state.tiktok_data.to_csv(index=False)
            st.download_button(
                label="📥 Download TikTok Data",
                data=csv,
                file_name="tiktok_posts.csv",
                mime="text/csv"
            )

        elif data_source == "Instagram Profiles" and st.session_state.instagram_profiles_data is not None:
            st.dataframe(st.session_state.instagram_profiles_data, use_container_width=True, height=400)

            csv = st.session_state.instagram_profiles_data.to_csv(index=False)
            st.download_button(
                label="📥 Download Instagram Profiles Data",
                data=csv,
                file_name="instagram_profiles.csv",
                mime="text/csv"
            )
        else:
            st.info(f"No {data_source} data available")

    st.divider()
    st.markdown("""
    <div style="text-align: center; color: #6b7280; padding: 2rem;">
        <p><strong>Brand Ad Monitor</strong> • Powered by Bright Data & OpenAI</p>
        <p style="font-size: 0.9rem;">Real-time brand intelligence platform for e-commerce marketers</p>
    </div>
    """, unsafe_allow_html=True)

if __name__ == "__main__":
    main()

Step 5: Run the Application: Execute:

streamlit run app.py

Access at http://localhost:8501. Use the sidebar to load data and initiate scrapes.

Results

The resulting dashboard features KPI metrics (e.g., top advertiser, average discount), interactive charts for ad frequency and pricing, and AI-driven recommendations.

For example, analyzing a dataset might reveal a brand’s 15% discount trend, prompting a strategic alert. Outputs are exportable as CSVs for further reporting.

What’s Next: Extensions You Can Build Yourself

• Automated Alerts: Integrate email notifications for price changes exceeding 10%.
• Social Sentiment Analysis: Layer in NLP to gauge TikTok comment positivity.
• Multi-Channel Expansion: Add YouTube or Pinterest datasets for broader coverage.
• Team Collaboration: Deploy to Streamlit Cloud with user authentication.

These additions can evolve the tool into a full marketing operations hub.

Conclusion

By combining Bright Data’s reliable datasets with Streamlit’s ease of use, this dashboard provides a powerful foundation for beauty brand intelligence. It transforms raw data into strategic insights, helping teams stay agile in competitive markets.

Resources

Where to Go From Here?

• Explore Bright Data’s documentation for advanced API usage.
• Deploy your app on Streamlit Cloud.
• Experiment with custom prompts in the OpenAI integration for domain-specific advice.

A message from our Founder

Hey, Sunil here. I wanted to take a moment to thank you for reading until the end and for being a part of this community.

Did you know that our team run these publications as a volunteer effort to over 3.5m monthly readers? We don’t receive any funding, we do this to support the community. ❤️

If you want to show some love, please take a moment to follow me on LinkedIn, TikTok, **Instagram. You can also subscribe to our [weekly newsletter](https://newsletter.plainenglish.io/)**.

And before you go, don’t forget to clap and follow the writer️!