MongoDB Documents, CRUD Operations - Evolution

Master MongoDB fundamentals: BSON document structure, ObjectId generation, CRUD operations (insertOne, insertMany, find, findOne, updateOne, updateMany, deleteOne, deleteMany), query operators ($eq, $gt, $lt, $in, $and, $or), write concerns, error handling, and validation.

MongoDB Documents, CRUD Operations — Evolution

**The Complete MongoDB Evolution Series — From Documents to Distributed Clusters — *Story 1: MongoDB Documents, CRUD Operations — Evolution***

Story Navigation

• 1. **MongoDB Documents, CRUD Operations — Evolution** (Current Story)
• 2. MongoDB Projection, Sorting, Pagination — Evolution — Coming Soon
• 3. MongoDB Mongoose ODM, Bulk Operations, Best Practices — Evolution — Coming Soon
• 4. MongoDB Indexing Strategy and Query Optimization — Evolution — Coming Soon
• 5. MongoDB Aggregation Pipeline — Evolution — Coming Soon
• 6. MongoDB Data Modeling and Schema Design Patterns — Evolution — Coming Soon
• 7. MongoDB Replication and High Availability — Evolution — Coming Soon
• 8. MongoDB Sharding and Horizontal Scaling — Evolution — Coming Soon
• 9. MongoDB Transactions and ACID Compliance — Evolution — Coming Soon
• 10. MongoDB Change Streams and Real-time Integration — Evolution — Coming Soon
• 11. MongoDB Atlas Cloud Operations — Evolution — Coming Soon
• 12. MongoDB Performance Optimization — Evolution — Coming Soon
• 13. MongoDB Security and Compliance — Evolution (Bonus) — Coming Soon
• 14. MongoDB Backup, Disaster Recovery, and Migration — Evolution (Bonus) — Coming Soon

Table of Contents

1. Introduction
2. MongoDB Architecture Overview
3. BSON Documents and Data Types
4. ObjectId Structure and Generation
5. Create Operations
6. Read Operations
7. Query Operators Deep Dive
8. Update Operations
9. Delete Operations
10. Write Concerns and Read Concerns
11. Error Handling and Validation
12. Database Profiling and Slow Query Logging
13. What We Learned from Story 1
14. Looking Ahead to Story 2

1. Introduction

MongoDB, released by MongoDB Inc. (formerly 10gen) in 2009, introduced a revolutionary approach to data persistence using document-oriented storage with dynamic schemas. Unlike traditional relational databases that require predefined schemas and normalized tables, MongoDB stores data as flexible BSON (Binary JSON) documents, allowing developers to evolve schemas naturally as applications grow.

Why MongoDB?

MongoDB Evolution Timeline

CRUD Operations Evolution Timeline

2. MongoDB Architecture Overview

Understanding MongoDB’s architecture helps you make better decisions about data modeling and query patterns.

MongoDB Architecture Diagram

Data Model Comparison: SQL vs MongoDB

Document vs Relational Data Modeling

3. BSON Documents and Data Types

MongoDB stores data as documents in collections. A document is a set of key-value pairs with dynamic schema, meaning documents in the same collection can have different fields and structures. Documents are stored in BSON (Binary JSON) format, which extends JSON with additional data types.

BSON Document Structure

BSON Data Types Reference

BSON vs JSON Comparison

4. ObjectId Structure and Generation

ObjectId is MongoDB’s default primary key type, designed to be unique across machines and time without centralized coordination.

ObjectId Structure Diagram

ObjectId Generation Flow

ObjectId Methods and Properties

// Creating ObjectIds
const id1 = new ObjectId();                    // New unique ID
const id2 = ObjectId();                        // Alternative syntax
const id3 = new ObjectId("6727a8e0d4f1a23a7b9c4e2b"); // From hex string
const id4 = ObjectId.fromTimestamp(1699876543); // From timestamp

// Extracting information
const timestamp = id1.getTimestamp();           // ISODate of creation
const creationDate = id1.getTimestamp().toISOString();
const hexString = id1.toString();               // 24-char hex string
const idValue = id1.valueOf();                  // Hexadecimal string

// Comparison
const isEqual = id1.equals(id2);                // Compare ObjectIds
const isGreater = id1 > id2;                    // Chronological comparison

// Generating without ObjectId class
const generated = new ObjectId().toHexString();  // Just the hex string

5. Create Operations

Create operations insert new documents into a collection. MongoDB provides multiple methods for document insertion with different behaviors and performance characteristics.

Create Operations Flow Diagram

Insert Methods Comparison

Complete Create Operations Implementation

// services/createService.js - Complete create operations
const { MongoClient, ObjectId } = require('mongodb');

class CreateOperationService {
  constructor(db) {
    this.db = db;
    this.collection = db.collection('courses');
  }

  // ============ SINGLE DOCUMENT INSERT ============

  /**
   * insertOne - Insert a single document
   * 
   * Behavior:
   * - Generates _id if not provided
   * - Validates document against schema
   * - Returns InsertOneResult with insertedId
   * - Atomic operation
   */
  async insertOneCourse(courseData) {
    try {
      // Generate slug if not provided
      if (!courseData.slug && courseData.title) {
        courseData.slug = courseData.title
          .toLowerCase()
          .replace(/[^a-z0-9]+/g, '-')
          .replace(/^-|-$/g, '');
      }

      // Add timestamps
      const now = new Date();
      courseData.createdAt = now;
      courseData.updatedAt = now;

      const result = await this.collection.insertOne(courseData);

      return {
        success: true,
        insertedId: result.insertedId,
        insertedCount: 1
      };
    } catch (error) {
      return this.handleInsertError(error);
    }
  }

  /**
   * insertOne with custom _id
   */
  async insertCourseWithCustomId(customId, courseData) {
    try {
      const document = {
        _id: customId,
        ...courseData,
        slug: courseData.slug || customId.toLowerCase().replace(/\s/g, '-'),
        createdAt: new Date(),
        updatedAt: new Date()
      };

      const result = await this.collection.insertOne(document);

      return {
        success: true,
        insertedId: result.insertedId,
        insertedCount: 1
      };
    } catch (error) {
      if (error.code === 11000) {
        return {
          success: false,
          error: 'Duplicate key error: A document with this ID or slug already exists',
          insertedCount: 0
        };
      }
      return this.handleInsertError(error);
    }
  }

  // ============ MULTIPLE DOCUMENTS INSERT ============

  /**
   * insertMany - Insert multiple documents
   * 
   * Options:
   * - ordered: true (default) - Stop on first error
   * - ordered: false - Continue even if errors occur
   * 
   * Performance: InsertMany is significantly faster than multiple insertOne calls
   */
  async insertManyCourses(coursesData, ordered = true) {
    try {
      // Add timestamps to each document
      const now = new Date();
      const documents = coursesData.map(course => ({
        ...course,
        slug: course.slug || (course.title ? course.title.toLowerCase().replace(/[^a-z0-9]+/g, '-') : undefined),
        createdAt: now,
        updatedAt: now
      }));

      const options = { ordered };
      const result = await this.collection.insertMany(documents, options);

      return {
        success: true,
        insertedIds: Object.values(result.insertedIds),
        insertedCount: result.insertedCount
      };
    } catch (error) {
      if (error.writeErrors) {
        // Some documents succeeded, some failed in unordered mode
        return {
          success: false,
          insertedCount: error.result?.insertedCount || 0,
          writeErrors: error.writeErrors,
          error: `${error.writeErrors.length} documents failed to insert`
        };
      }
      return this.handleInsertError(error);
    }
  }

  /**
   * Bulk insert with progress tracking
   * Splits large arrays into batches to avoid memory issues
   */
  async bulkInsertWithProgress(coursesData, batchSize = 1000, onProgress = null) {
    const results = {
      totalInserted: 0,
      totalFailed: 0,
      errors: []
    };

    for (let i = 0; i < coursesData.length; i += batchSize) {
      const batch = coursesData.slice(i, i + batchSize);

      try {
        const now = new Date();
        const documents = batch.map(course => ({
          ...course,
          createdAt: now,
          updatedAt: now
        }));

        const result = await this.collection.insertMany(documents, { ordered: false });
        results.totalInserted += result.insertedCount;

        if (onProgress) {
          onProgress(Math.min(i + batchSize, coursesData.length), coursesData.length);
        }
      } catch (error) {
        // In unordered mode, some may succeed
        if (error.result?.insertedCount) {
          results.totalInserted += error.result.insertedCount;
        }
        results.totalFailed += batch.length - (error.result?.insertedCount || 0);

        if (error.writeErrors) {
          results.errors.push(...error.writeErrors);
        }
      }
    }

    return results;
  }

  // ============ UPSERT OPERATIONS ============

  /**
   * Upsert - Update if exists, insert if not
   * 
   * Use cases:
   * - Idempotent operations
   * - Synchronizing external data
   * - Counter updates with initialization
   */
  async upsertCourseBySlug(slug, updateData) {
    const filter = { slug };
    const update = {
      $set: {
        ...updateData,
        updatedAt: new Date()
      },
      $setOnInsert: {
        createdAt: new Date(),
        slug: slug
      }
    };

    const options = {
      upsert: true,
      returnDocument: 'after'
    };

    const result = await this.collection.findOneAndUpdate(filter, update, options);

    return {
      course: result,
      wasInserted: result && !result.lastErrorObject?.updatedExisting
    };
  }

  /**
   * Counter-based upsert - Increment or initialize counter
   */
  async incrementCourseViewCount(courseId) {
    const result = await this.collection.updateOne(
      { _id: new ObjectId(courseId) },
      { 
        $inc: { 'analytics.totalViews': 1 },
        $setOnInsert: { 
          'analytics.uniqueViewers': 0,
          'analytics.averageWatchTimeSeconds': 0,
          'analytics.lastViewedAt': new Date()
        }
      },
      { upsert: false }
    );

    // Get updated count
    const course = await this.collection.findOne({ _id: new ObjectId(courseId) });
    return course?.analytics?.totalViews || 0;
  }

  // ============ NESTED DOCUMENT INSERT ============

  /**
   * Insert nested documents (subdocuments) into arrays using $push
   */
  async addModuleToCourse(courseId, moduleData) {
    // First get current module count to set order
    const course = await this.collection.findOne(
      { _id: new ObjectId(courseId) },
      { projection: { modules: 1 } }
    );

    const newOrder = (course?.modules?.length || 0) + 1;

    const result = await this.collection.updateOne(
      { _id: new ObjectId(courseId) },
      {
        $push: {
          modules: {
            order: newOrder,
            title: moduleData.title,
            description: moduleData.description,
            videos: moduleData.videos,
            totalDurationSeconds: moduleData.videos.reduce((sum, v) => sum + v.durationSeconds, 0),
            isPublished: true,
            createdAt: new Date()
          }
        },
        $inc: {
          totalDurationSeconds: moduleData.videos.reduce((sum, v) => sum + v.durationSeconds, 0),
          totalVideosCount: moduleData.videos.length
        }
      }
    );

    return result.modifiedCount > 0;
  }

  /**
   * Add multiple nested documents using $push with $each
   * More efficient for adding many items
   */
  async addMultipleModulesToCourse(courseId, modules) {
    const course = await this.collection.findOne(
      { _id: new ObjectId(courseId) },
      { projection: { modules: 1 } }
    );

    const startIndex = (course?.modules?.length || 0) + 1;

    const modulesWithOrder = modules.map((module, index) => ({
      ...module,
      order: startIndex + index,
      totalDurationSeconds: module.videos.reduce((sum, v) => sum + v.durationSeconds, 0),
      isPublished: true,
      createdAt: new Date()
    }));

    const totalDurationAdded = modulesWithOrder.reduce((sum, m) => sum + m.totalDurationSeconds, 0);
    const totalVideosAdded = modulesWithOrder.reduce((sum, m) => sum + m.videos.length, 0);

    const result = await this.collection.updateOne(
      { _id: new ObjectId(courseId) },
      {
        $push: {
          modules: { $each: modulesWithOrder }
        },
        $inc: {
          totalDurationSeconds: totalDurationAdded,
          totalVideosCount: totalVideosAdded
        }
      }
    );

    return result.modifiedCount > 0;
  }

  // ============ ERROR HANDLING ============

  handleInsertError(error) {
    // Duplicate key error (code 11000)
    if (error.code === 11000) {
      const field = Object.keys(error.keyPattern)[0];
      return {
        success: false,
        error: `Duplicate key error: A document with this ${field} already exists`,
        insertedCount: 0
      };
    }

    // Validation error
    if (error.name === 'ValidationError') {
      return {
        success: false,
        error: `Validation failed: ${error.message}`,
        insertedCount: 0
      };
    }

    // Generic error
    return {
      success: false,
      error: error.message || 'Unknown error occurred',
      insertedCount: 0
    };
  }
}

// Usage Examples
async function createExamples() {
  const client = await MongoClient.connect('mongodb://localhost:27017');
  const db = client.db('videoPortal');
  const createService = new CreateOperationService(db);

  // Example 1: Simple insert
  const result1 = await createService.insertOneCourse({
    title: "MongoDB Fundamentals",
    shortDescription: "Learn MongoDB from scratch",
    price: 49.99,
    primaryCategory: "development",
    difficulty: "beginner",
    instructor: {
      name: "John Doe",
      email: "john@example.com",
      avatar: "john.jpg",
      bio: "MongoDB Expert"
    },
    thumbnailUrl: "https://example.com/thumb.jpg",
    isPublished: true
  });
  console.log('Inserted course:', result1.insertedId);

  // Example 2: Bulk insert
  const batchResult = await createService.bulkInsertWithProgress(
    courseBatchData,
    500,
    (completed, total) => console.log(`Progress: ${completed}/${total}`)
  );
  console.log(`Inserted ${batchResult.totalInserted} courses`);

  // Example 3: Upsert
  const upsertResult = await createService.upsertCourseBySlug(
    "advanced-mongodb",
    {
      price: 99.99,
      isPublished: true,
      $inc: { totalEnrollments: 1 }
    }
  );
}

6. Read Operations

Read operations query documents from a collection. MongoDB’s find methods support filtering, projection, sorting, and pagination.

Read Operations Execution Flow

Read Methods Comparison Table

Complete Read Operations Implementation

// services/readService.js - Complete read operations
const { ObjectId } = require('mongodb');

class ReadOperationService {
  constructor(db) {
    this.db = db;
    this.collection = db.collection('courses');
  }

  // ============ BASIC FIND OPERATIONS ============

  /**
   * find() - Returns a cursor, not executed until .toArray() or iteration
   * The cursor is lazy - query executes only when you start iterating
   */
  async findAllCourses(options = {}) {
    let query = this.collection.find({ isPublished: true });

    if (options.sort) query = query.sort(options.sort);
    if (options.skip) query = query.skip(options.skip);
    if (options.limit) query = query.limit(options.limit);
    if (options.projection) query = query.project(options.projection);

    return await query.toArray();
  }

  /**
   * findOne() - Returns first matching document or null
   */
  async findOneCourse(filter) {
    return await this.collection.findOne(filter);
  }

  /**
   * findById() - Convenience method for { _id: id }
   */
  async findCourseById(id) {
    if (!ObjectId.isValid(id)) {
      throw new Error('Invalid ObjectId format');
    }
    return await this.collection.findOne({ _id: new ObjectId(id) });
  }

  // ============ ADVANCED FILTERING ============

  /**
   * Find courses with multiple filter conditions
   */
  async findCoursesWithFilters(filters) {
    const query = { isPublished: true };

    // Category filter (single)
    if (filters.category) {
      query.primaryCategory = filters.category;
    }

    // Difficulty filter (multiple values with $in)
    if (filters.difficulty && filters.difficulty.length > 0) {
      query.difficulty = { $in: filters.difficulty };
    }

    // Price range filter
    if (filters.minPrice !== undefined || filters.maxPrice !== undefined) {
      query.price = {};
      if (filters.minPrice !== undefined) query.price.$gte = filters.minPrice;
      if (filters.maxPrice !== undefined) query.price.$lte = filters.maxPrice;
    }

    // Free filter
    if (filters.isFree !== undefined) {
      query.isFree = filters.isFree;
    }

    // Rating filter
    if (filters.minRating) {
      query.averageRating = { $gte: filters.minRating };
    }

    // Tags filter ($all ensures all tags are present)
    if (filters.tags && filters.tags.length > 0) {
      query.tags = { $all: filters.tags };
    }

    // Text search using $regex (for simple search)
    if (filters.searchTerm && filters.searchTerm.trim()) {
      query.$or = [
        { title: { $regex: filters.searchTerm, $options: 'i' } },
        { shortDescription: { $regex: filters.searchTerm, $options: 'i' } },
        { tags: { $in: [new RegExp(filters.searchTerm, 'i')] } }
      ];
    }

    // Instructor filter
    if (filters.instructorId) {
      query['instructor.userId'] = new ObjectId(filters.instructorId);
    }

    // Date range filter
    if (filters.publishedAfter || filters.publishedBefore) {
      query.publishedAt = {};
      if (filters.publishedAfter) query.publishedAt.$gte = new Date(filters.publishedAfter);
      if (filters.publishedBefore) query.publishedAt.$lte = new Date(filters.publishedBefore);
    }

    return await this.collection.find(query).toArray();
  }

  // ============ COUNT OPERATIONS ============

  /**
   * countDocuments() - Accurate count with filters
   * Slower for large collections, but respects filters
   */
  async countCourses(filters = {}) {
    return await this.collection.countDocuments(filters);
  }

  /**
   * estimatedDocumentCount() - Fast approximate count
   * Uses collection metadata, doesn't respect filters
   * Good for large collections when exact count not needed
   */
  async getEstimatedCourseCount() {
    return await this.collection.estimatedDocumentCount();
  }

  /**
   * Count with grouping (using aggregation)
   */
  async countByDifficulty() {
    return await this.collection.aggregate([
      { $match: { isPublished: true } },
      {
        $group: {
          _id: '$difficulty',
          count: { $sum: 1 }
        }
      },
      { $sort: { count: -1 } },
      { $project: { difficulty: '$_id', count: 1, _id: 0 } }
    ]).toArray();
  }

  // ============ DISTINCT OPERATIONS ============

  /**
   * distinct() - Get unique values for a field
   */
  async getDistinctCategories() {
    return await this.collection.distinct('primaryCategory', { isPublished: true });
  }

  async getDistinctTags() {
    return await this.collection.distinct('tags', { isPublished: true });
  }

  /**
   * Get distinct values with count (using aggregation)
   */
  async getTagsWithCounts() {
    return await this.collection.aggregate([
      { $match: { isPublished: true } },
      { $unwind: '$tags' },
      {
        $group: {
          _id: '$tags',
          count: { $sum: 1 }
        }
      },
      { $sort: { count: -1 } },
      { $limit: 20 },
      { $project: { tag: '$_id', count: 1, _id: 0 } }
    ]).toArray();
  }

  // ============ EXISTENCE CHECK ============

  /**
   * exists() - Check if any document matches filter
   * More efficient than find() when you only need existence
   */
  async courseExists(slug) {
    const result = await this.collection.findOne(
      { slug },
      { projection: { _id: 1 } }
    );
    return result !== null;
  }

  async hasPublishedCourses() {
    const result = await this.collection.findOne(
      { isPublished: true },
      { projection: { _id: 1 } }
    );
    return result !== null;
  }

  // ============ CURSOR OPERATIONS ============

  /**
   * Working with cursors for large result sets
   * Cursors allow processing large datasets without loading all into memory
   */
  async processAllCoursesBatch(batchSize = 100, processor) {
    let processed = 0;
    let errors = 0;

    const cursor = this.collection.find({ isPublished: true })
      .sort({ createdAt: -1 })
      .batchSize(batchSize);

    for await (const course of cursor) {
      try {
        await processor(course);
        processed++;
      } catch (error) {
        console.error(`Error processing course ${course._id}:`, error);
        errors++;
      }
    }

    return { processed, errors };
  }

  /**
   * Manual cursor iteration with next()
   */
  async iterateWithManualCursor() {
    const cursor = this.collection.find().sort({ title: 1 });

    let doc = await cursor.next();
    while (doc !== null) {
      // Process document
      console.log(`Processing: ${doc.title}`);
      doc = await cursor.next();
    }
  }

  // ============ PAGINATED READS ============

  /**
   * Paginated read with total count
   */
  async getPaginatedCourses(page = 1, limit = 10, filters = { isPublished: true }) {
    const skip = (page - 1) * limit;

    const [data, total] = await Promise.all([
      this.collection.find(filters)
        .sort({ createdAt: -1 })
        .skip(skip)
        .limit(limit)
        .toArray(),
      this.collection.countDocuments(filters)
    ]);

    const totalPages = Math.ceil(total / limit);

    return {
      data,
      pagination: {
        total,
        page,
        limit,
        totalPages,
        hasNextPage: page < totalPages,
        hasPrevPage: page > 1
      }
    };
  }

  /**
   * Cursor-based pagination for infinite scroll
   * More efficient than skip-based pagination
   */
  async getCursorPaginatedCourses(cursor = null, limit = 10, sortField = '_id', sortOrder = -1) {
    const query = { isPublished: true };

    if (cursor) {
      if (sortField === '_id') {
        const comparison = sortOrder === -1 ? '$lt' : '$gt';
        query._id = { [comparison]: new ObjectId(cursor) };
      } else {
        const comparison = sortOrder === -1 ? '$lt' : '$gt';
        query[sortField] = { [comparison]: cursor };
      }
    }

    const sortObject = { [sortField]: sortOrder };
    if (sortField !== '_id') {
      sortObject._id = sortOrder;
    }

    // Fetch one extra to determine if more exist
    const data = await this.collection.find(query)
      .sort(sortObject)
      .limit(limit + 1)
      .toArray();

    const hasMore = data.length > limit;
    const items = hasMore ? data.slice(0, limit) : data;
    const nextCursor = hasMore && items.length > 0
      ? String(items[items.length - 1][sortField])
      : null;

    return {
      data: items,
      nextCursor,
      hasMore
    };
  }

  // ============ READ WITH EXPLAIN (Analysis) ============

  /**
   * Get query execution plan for optimization
   */
  async explainQuery(filters) {
    return await this.collection.find(filters).explain('executionStats');
  }

  /**
   * Compare index performance
   */
  async compareQueryPerformance() {
    // Query 1: Using category index
    const plan1 = await this.collection.find({ primaryCategory: 'development' })
      .explain('executionStats');

    // Query 2: Using compound index
    const plan2 = await this.collection.find({ 
      primaryCategory: 'development',
      difficulty: 'intermediate'
    }).explain('executionStats');

    console.log('Query 1 - Category only:', {
      stage: plan1.queryPlanner.winningPlan.stage,
      docsExamined: plan1.executionStats.totalDocsExamined,
      executionTime: plan1.executionStats.executionTimeMillis
    });

    console.log('Query 2 - Compound:', {
      stage: plan2.queryPlanner.winningPlan.stage,
      docsExamined: plan2.executionStats.totalDocsExamined,
      executionTime: plan2.executionStats.executionTimeMillis
    });
  }
}

// Usage Examples
async function readExamples() {
  const client = await MongoClient.connect('mongodb://localhost:27017');
  const db = client.db('videoPortal');
  const readService = new ReadOperationService(db);

  // Example 1: Find with filters
  const popularCourses = await readService.findCoursesWithFilters({
    minRating: 4.5,
    minPrice: 0,
    maxPrice: 100,
    difficulty: ['beginner', 'intermediate'],
    tags: ['mongodb', 'database']
  });

  // Example 2: Pagination
  const page1 = await readService.getPaginatedCourses(1, 20);
  console.log(`Page 1: ${page1.data.length} of ${page1.pagination.total} courses`);

  // Example 3: Cursor pagination (infinite scroll)
  let cursor = null;
  let allCourses = [];

  for (let i = 0; i < 5; i++) {
    const result = await readService.getCursorPaginatedCourses(cursor, 10);
    allCourses.push(...result.data);
    cursor = result.nextCursor;
    if (!result.hasMore) break;
  }
  console.log(`Loaded ${allCourses.length} items via cursor pagination`);

  // Example 4: Process large dataset with cursor
  const results = await readService.processAllCoursesBatch(50, async (course) => {
    console.log(`Processed: ${course.title}`);
  });
  console.log(`Processed ${results.processed} courses with ${results.errors} errors`);
}

7. Query Operators Deep Dive

MongoDB provides rich query operators for complex filtering conditions. Understanding these operators is crucial for building efficient queries.

Query Operator Hierarchy Diagram

Query Operator Decision Flow

Complete Query Operators Implementation

// services/queryOperatorsService.js - Comprehensive query operators
const { ObjectId } = require('mongodb');

class QueryOperatorsService {
  constructor(db) {
    this.db = db;
    this.collection = db.collection('courses');
  }

  // ============ 1. COMPARISON OPERATORS ============

  /**
   * $eq - Matches values equal to a specified value
   * $ne - Matches values not equal to a specified value
   */
  async comparisonEqualityExamples() {
    // $eq - exact match
    const beginnerCourses = await this.collection.find({ difficulty: { $eq: 'beginner' } }).toArray();
    // Equivalent to: { difficulty: 'beginner' }

    // $ne - not equal
    const nonFreeCourses = await this.collection.find({ isFree: { $ne: true } }).toArray();

    // $eq with nested field
    const specificInstructor = await this.collection.find({ 
      'instructor.name': { $eq: 'Jane Smith' } 
    }).toArray();

    return { beginnerCourses, nonFreeCourses, specificInstructor };
  }

  /**
   * $gt, $gte, $lt, $lte - Range operators
   */
  async comparisonRangeExamples() {
    // $gt (greater than)
    const expensiveCourses = await this.collection.find({ price: { $gt: 100 } }).toArray();

    // $gte (greater than or equal)
    const highRatedCourses = await this.collection.find({ averageRating: { $gte: 4.5 } }).toArray();

    // $lt (less than)
    const shortCourses = await this.collection.find({ totalDurationSeconds: { $lt: 3600 } }).toArray();

    // $lte (less than or equal)
    const affordableCourses = await this.collection.find({ price: { $lte: 49.99 } }).toArray();

    // Range between two values
    const midRangeCourses = await this.collection.find({ 
      price: { $gte: 50, $lte: 150 },
      totalEnrollments: { $gt: 1000, $lt: 50000 }
    }).toArray();

    return { expensiveCourses, highRatedCourses, shortCourses, affordableCourses, midRangeCourses };
  }

  /**
   * $in - Matches any of the values in an array
   * $nin - Matches none of the values in an array
   */
  async comparisonInExamples() {
    // $in
    const popularCategories = await this.collection.find({
      primaryCategory: { $in: ['development', 'data-science', 'cloud'] }
    }).toArray();

    const preferredDifficulties = await this.collection.find({
      difficulty: { $in: ['beginner', 'intermediate'] }
    }).toArray();

    // $in with array field
    const coursesWithTags = await this.collection.find({
      tags: { $in: ['mongodb', 'react', 'nodejs'] }
    }).toArray();

    // $nin - Exclude
    const notInCategories = await this.collection.find({
      primaryCategory: { $nin: ['business', 'marketing'] }
    }).toArray();

    return { popularCategories, preferredDifficulties, coursesWithTags, notInCategories };
  }

  // ============ 2. LOGICAL OPERATORS ============

  /**
   * $and - Joins query clauses with a logical AND
   * Multiple top-level conditions are implicitly ANDed
   */
  async logicalAndExamples() {
    // Explicit $and
    const explicitAnd = await this.collection.find({
      $and: [
        { isPublished: true },
        { price: { $lt: 100 } },
        { averageRating: { $gte: 4.0 } }
      ]
    }).toArray();

    // Implicit AND (same as above)
    const implicitAnd = await this.collection.find({
      isPublished: true,
      price: { $lt: 100 },
      averageRating: { $gte: 4.0 }
    }).toArray();

    // $and with same field (must use explicit $and)
    const rangeWithSameField = await this.collection.find({
      $and: [
        { price: { $gte: 50 } },
        { price: { $lte: 150 } }
      ]
    }).toArray();

    return { explicitAnd, implicitAnd, rangeWithSameField };
  }

  /**
   * $or - Joins query clauses with a logical OR
   */
  async logicalOrExamples() {
    // Any of these conditions
    const highValueCourses = await this.collection.find({
      $or: [
        { totalEnrollments: { $gt: 10000 } },
        { averageRating: { $gte: 4.8 } },
        { isFeatured: true }
      ]
    }).toArray();

    // Complex $or with multiple conditions per clause
    const complexOr = await this.collection.find({
      $or: [
        { 
          $and: [
            { price: { $gt: 100 } },
            { averageRating: { $gte: 4.5 } }
          ]
        },
        {
          $and: [
            { isFree: true },
            { totalEnrollments: { $gt: 5000 } }
          ]
        }
      ]
    }).toArray();

    return { highValueCourses, complexOr };
  }

  /**
   * $nor - Joins query clauses with a logical NOR
   * Returns documents that fail all conditions
   */
  async logicalNorExamples() {
    // Exclude beginner free courses
    const advancedPaidCourses = await this.collection.find({
      $nor: [
        { difficulty: 'beginner' },
        { isFree: true },
        { price: { $eq: 0 } }
      ]
    }).toArray();

    // NOR with nested conditions
    const excludeCheapAndPopular = await this.collection.find({
      $nor: [
        { $and: [{ price: { $lt: 20 } }, { totalEnrollments: { $gt: 1000 } }] },
        { isFree: true }
      ]
    }).toArray();

    return { advancedPaidCourses, excludeCheapAndPopular };
  }

  /**
   * $not - Inverts the effect of a query expression
   */
  async logicalNotExamples() {
    // Not equal to a value
    const notBeginner = await this.collection.find({
      difficulty: { $not: { $eq: 'beginner' } }
    }).toArray();

    // Not greater than
    const notExpensive = await this.collection.find({
      price: { $not: { $gt: 200 } }
    }).toArray();

    // Not matching pattern
    const notReact = await this.collection.find({
      title: { $not: /react/i }
    }).toArray();

    return { notBeginner, notExpensive, notReact };
  }

  // ============ 3. ELEMENT OPERATORS ============

  /**
   * $exists - Matches documents that have the specified field
   * $type - Selects documents if a field is of the specified type
   */
  async elementOperatorExamples() {
    // $exists
    const withPreview = await this.collection.find({ previewVideoUrl: { $exists: true } }).toArray();
    const withDiscount = await this.collection.find({ discountPrice: { $exists: true } }).toArray();

    // $exists with null check (field exists but is null)
    const withNullDiscount = await this.collection.find({ 
      discountPrice: { $exists: true, $eq: null } 
    }).toArray();

    // $type - BSON type codes (string or number)
    const stringTitles = await this.collection.find({ title: { $type: 'string' } }).toArray();
    const decimalPrices = await this.collection.find({ exactPrice: { $type: 'decimal' } }).toArray();

    // $type with multiple types
    const numericOrString = await this.collection.find({
      price: { $type: ['double', 'int', 'string'] }
    }).toArray();

    return { withPreview, withDiscount, stringTitles, decimalPrices };
  }

  // ============ 4. ARRAY OPERATORS ============

  /**
   * $all - Matches arrays that contain all specified elements
   * $size - Matches arrays with specified number of elements
   */
  async arrayAllSizeExamples() {
    // $all - Must contain ALL tags
    const fullStackCourses = await this.collection.find({
      tags: { $all: ['react', 'nodejs', 'mongodb'] }
    }).toArray();

    // $size - Exact array length
    const shortCourses = await this.collection.find({ modules: { $size: 3 } }).toArray();

    // $size with $gte using $expr (since $size only does exact match)
    const comprehensiveCourses = await this.collection.find({
      $expr: { $gte: [{ $size: '$modules' }, 10] }
    }).toArray();

    return { fullStackCourses, shortCourses, comprehensiveCourses };
  }

  /**
   * $elemMatch - Matches documents that contain an array element matching all conditions
   */
  async arrayElemMatchExamples() {
    // Find courses where ANY module has long videos
    const coursesWithLongVideos = await this.collection.find({
      modules: {
        $elemMatch: {
          'videos.durationSeconds': { $gt: 3600 },
          isPublished: true
        }
      }
    }).toArray();

    // Multiple conditions on same array element
    const coursesWithHighQualityVideos = await this.collection.find({
      'modules.videos': {
        $elemMatch: {
          quality: '4k',
          sizeMB: { $lt: 1000 }
        }
      }
    }).toArray();

    return { coursesWithLongVideos, coursesWithHighQualityVideos };
  }

  /**
   * Array indexing - Access specific array positions
   */
  async arrayIndexingExamples() {
    // First module must be published
    const coursesWithPublishedFirstModule = await this.collection.find({
      'modules.0.isPublished': true
    }).toArray();

    // Second video's quality must be 4k
    const coursesWith4kSecondVideo = await this.collection.find({
      'modules.0.videos.1.quality': '4k'
    }).toArray();

    return { coursesWithPublishedFirstModule, coursesWith4kSecondVideo };
  }

  // ============ 5. EVALUATION OPERATORS ============

  /**
   * $regex - Provides regular expression capabilities
   */
  async evaluationRegexExamples() {
    // Case-insensitive search
    const reactCourses = await this.collection.find({
      title: { $regex: 'react', $options: 'i' }
    }).toArray();

    // Start with pattern
    const startsWithReact = await this.collection.find({
      title: { $regex: '^React', $options: 'i' }
    }).toArray();

    // End with pattern
    const endsWithCourse = await this.collection.find({
      title: { $regex: 'Course$', $options: 'i' }
    }).toArray();

    // Contains word (with word boundaries)
    const containsNode = await this.collection.find({
      title: { $regex: '\\bnode\\b', $options: 'i' }
    }).toArray();

    return { reactCourses, startsWithReact, endsWithCourse, containsNode };
  }

  /**
   * $expr - Allows use of aggregation expressions within query
   * Powerful for comparing fields within the same document
   */
  async evaluationExprExamples() {
    // Compare two fields
    const moreEnrollmentsThanReviews = await this.collection.find({
      $expr: { $gt: ['$totalEnrollments', '$reviewCount'] }
    }).toArray();

    // Calculate and compare
    const highEngagement = await this.collection.find({
      $expr: {
        $gte: [
          { $multiply: ['$averageRating', '$totalEnrollments'] },
          10000
        ]
      }
    }).toArray();

    // Compare with constant
    const expensivePerStudent = await this.collection.find({
      $expr: {
        $gt: [{ $divide: ['$price', '$totalEnrollments'] }, 0.01 ]
      }
    }).toArray();

    // Date comparison
    const recentAndPublished = await this.collection.find({
      $expr: {
        $and: [
          { $gte: ['$publishedAt', { $dateSubtract: { startDate: new Date(), unit: 'month', amount: 1 } }] },
          { $eq: ['$isPublished', true] }
        ]
      }
    }).toArray();

    return { moreEnrollmentsThanReviews, highEngagement, expensivePerStudent, recentAndPublished };
  }

  /**
   * Complex real-world query combining multiple operators
   */
  async complexCourseSearch(searchParams) {
    const query = { isPublished: true };

    // Price range
    if (searchParams.minPrice !== undefined || searchParams.maxPrice !== undefined) {
      query.price = {};
      if (searchParams.minPrice !== undefined) query.price.$gte = searchParams.minPrice;
      if (searchParams.maxPrice !== undefined) query.price.$lte = searchParams.maxPrice;
    }

    // Categories using $in
    if (searchParams.categories && searchParams.categories.length > 0) {
      query.primaryCategory = { $in: searchParams.categories };
    }

    // Difficulties using $in
    if (searchParams.difficulties && searchParams.difficulties.length > 0) {
      query.difficulty = { $in: searchParams.difficulties };
    }

    // Rating
    if (searchParams.minRating) {
      query.averageRating = { $gte: searchParams.minRating };
    }

    // Tags using $all
    if (searchParams.tags && searchParams.tags.length > 0) {
      query.tags = { $all: searchParams.tags };
    }

    // Text search using $regex on multiple fields
    if (searchParams.query && searchParams.query.trim()) {
      query.$or = [
        { title: { $regex: searchParams.query, $options: 'i' } },
        { shortDescription: { $regex: searchParams.query, $options: 'i' } },
        { tags: { $in: [new RegExp(searchParams.query, 'i')] } }
      ];
    }

    // Build sort object
    let sortObj = { totalEnrollments: -1 };
    switch (searchParams.sortBy) {
      case 'price_asc':
        sortObj = { price: 1 };
        break;
      case 'price_desc':
        sortObj = { price: -1 };
        break;
      case 'rating':
        sortObj = { averageRating: -1, reviewCount: -1 };
        break;
      case 'newest':
        sortObj = { publishedAt: -1 };
        break;
      default:
        sortObj = { totalEnrollments: -1 };
    }

    let cursor = this.collection.find(query).sort(sortObj);

    if (searchParams.limit) {
      cursor = cursor.limit(searchParams.limit);
    }

    return await cursor.toArray();
  }
}

8. Update Operations

Update operations modify existing documents in a collection. MongoDB provides powerful update operators for atomic field modifications.

Update Operators Reference Table

Update Operation Flow Diagram

Complete Update Operations Implementation

// services/updateService.js - Complete update operations
const { ObjectId } = require('mongodb');

class UpdateOperationService {
  constructor(db) {
    this.db = db;
    this.collection = db.collection('courses');
  }

  // ============ BASIC UPDATE METHODS ============

  /**
   * updateOne() - Update first matching document
   */
  async updateOneCourse(filter, updateData) {
    try {
      // Always add updated timestamp
      if (!updateData.$set) {
        updateData.$set = {};
      }
      updateData.$set.updatedAt = new Date();

      const result = await this.collection.updateOne(filter, updateData);

      return {
        success: result.acknowledged,
        matchedCount: result.matchedCount,
        modifiedCount: result.modifiedCount,
        upsertedId: result.upsertedId
      };
    } catch (error) {
      return {
        success: false,
        matchedCount: 0,
        modifiedCount: 0,
        error: error.message
      };
    }
  }

  /**
   * updateMany() - Update all matching documents
   */
  async updateManyCourses(filter, updateData) {
    try {
      if (!updateData.$set) {
        updateData.$set = {};
      }
      updateData.$set.updatedAt = new Date();

      const result = await this.collection.updateMany(filter, updateData);

      return {
        success: result.acknowledged,
        matchedCount: result.matchedCount,
        modifiedCount: result.modifiedCount
      };
    } catch (error) {
      return {
        success: false,
        matchedCount: 0,
        modifiedCount: 0,
        error: error.message
      };
    }
  }

  /**
   * findOneAndUpdate() - Update and return the document
   * Returns the updated document (unlike updateOne)
   */
  async findAndUpdateCourse(filter, updateData, options = {}) {
    const update = {
      $set: {
        ...updateData,
        updatedAt: new Date()
      }
    };

    const result = await this.collection.findOneAndUpdate(
      filter,
      update,
      {
        returnDocument: options.new ? 'after' : 'before',
        upsert: options.upsert || false
      }
    );

    return result;
  }

  // ============ FIELD UPDATE OPERATORS ============

  /**
   * $set - Set field values
   * $unset - Remove fields
   */
  async fieldUpdateExamples(courseId) {
    // $set - Update specific fields
    await this.collection.updateOne(
      { slug: 'mongodb-fundamentals' },
      {
        $set: {
          price: 79.99,
          isPublished: true,
          publishedAt: new Date(),
          'instructor.bio': 'Updated bio text',
          tags: ['mongodb', 'database', 'nosql']
        }
      }
    );

    // $unset - Remove fields
    await this.collection.updateMany(
      { isArchived: true },
      { $unset: { discountPrice: "", promoVideoUrl: "" } }
    );

    // Combined $set and $unset
    await this.collection.updateOne(
      { _id: new ObjectId(courseId) },
      {
        $set: { status: 'updated' },
        $unset: { temporaryFlag: "" }
      }
    );
  }

  /**
   * $inc - Increment numeric fields
   * $mul - Multiply numeric fields
   * $min/$max - Conditional updates
   */
  async numericUpdateExamples() {
    // $inc - Increment
    await this.collection.updateOne(
      { slug: 'mongodb-advanced' },
      { $inc: { totalEnrollments: 1, totalViews: 1 } }
    );

    // $inc can also decrement (use negative number)
    await this.collection.updateOne(
      { slug: 'outdated-course' },
      { $inc: { relevanceScore: -5 } }
    );

    // $mul - Multiply (apply 10% discount)
    await this.collection.updateMany(
      { isOnSale: true },
      { $mul: { price: 0.9 } }
    );

    // $min - Only update if value is smaller (price reduction only)
    await this.collection.updateOne(
      { slug: 'course-with-discount' },
      { $min: { price: 49.99 } }
    );

    // $max - Only update if value is larger (track peak students)
    await this.collection.updateOne(
      { slug: 'popular-course' },
      { $max: { peakEnrollments: 5000 } }
    );
  }

  /**
   * $rename - Rename field names
   */
  async renameFieldExamples() {
    // Simple rename
    await this.collection.updateMany(
      {},
      { $rename: { 'totalEnrollments': 'totalStudents' } }
    );

    // Rename nested field
    await this.collection.updateMany(
      {},
      { $rename: { 'instructor.biography': 'instructor.bio' } }
    );
  }

  // ============ ARRAY UPDATE OPERATORS ============

  /**
   * $push - Add elements to array
   */
  async arrayPushExamples(courseId) {
    // Simple $push
    await this.collection.updateOne(
      { slug: 'web-dev-bootcamp' },
      { $push: { tags: 'fullstack' } }
    );

    // $push with $each - multiple items
    await this.collection.updateOne(
      { slug: 'web-dev-bootcamp' },
      {
        $push: {
          tags: {
            $each: ['react', 'nodejs', 'mongodb']
          }
        }
      }
    );

    // $push with $position - insert at specific index
    await this.collection.updateOne(
      { slug: 'web-dev-bootcamp' },
      {
        $push: {
          tags: {
            $each: ['typescript'],
            $position: 0
          }
        }
      }
    );

    // $push with $slice - limit array size (keep only last 10 reviews)
    await this.collection.updateOne(
      { _id: new ObjectId(courseId) },
      {
        $push: {
          recentReviews: {
            $each: [newReview],
            $slice: -10
          }
        }
      }
    );
  }

  /**
   * $addToSet - Add unique elements to array (no duplicates)
   */
  async arrayAddToSetExamples() {
    // Simple $addToSet - only adds if not already present
    await this.collection.updateOne(
      { slug: 'mongodb-course' },
      { $addToSet: { tags: 'database' } }
    );

    // $addToSet with $each - add multiple unique items
    await this.collection.updateOne(
      { slug: 'mongodb-course' },
      {
        $addToSet: {
          tags: {
            $each: ['nosql', 'scalability', 'replication']
          }
        }
      }
    );
  }

  /**
   * $pull - Remove elements from array
   * $pullAll - Remove multiple elements
   * $pop - Remove first or last element
   */
  async arrayRemoveExamples() {
    // $pull - Remove specific value
    await this.collection.updateOne(
      { slug: 'web-dev' },
      { $pull: { tags: 'outdated' } }
    );

    // $pull with conditions
    await this.collection.updateOne(
      { _id: courseId },
      {
        $pull: {
          modules: {
            totalDurationSeconds: { $lt: 600 }
          }
        }
      }
    );

    // $pullAll - Remove multiple specific values
    await this.collection.updateOne(
      { slug: 'web-dev' },
      { $pullAll: { tags: ['legacy', 'deprecated', 'old'] } }
    );

    // $pop - Remove first element (-1) or last element (1)
    await this.collection.updateOne(
      { _id: courseId },
      { $pop: { recentReviews: 1 } }
    );
  }

  /**
   * Positional Array Operators
   * $ - First matching element
   * $[] - All elements
   * $[<identifier>] - Filtered positional operator
   */
  async positionalOperatorsExamples(courseId) {
    // $ - Update first matching array element
    await this.collection.updateOne(
      { 
        _id: new ObjectId(courseId),
        'modules.title': 'Introduction'
      },
      {
        $set: {
          'modules.$.isPublished': true,
          'modules.$.description': 'Updated description'
        }
      }
    );

    // $[] - Update all array elements
    await this.collection.updateOne(
      { _id: new ObjectId(courseId) },
      { $set: { 'modules.$[].isPublished': true } }
    );

    // $[<identifier>] - Update specific elements matching filter
    await this.collection.updateOne(
      { _id: new ObjectId(courseId) },
      {
        $set: {
          'modules.$[module].isPremium': true
        }
      },
      {
        arrayFilters: [
          { 'module.difficulty': 'advanced' }
        ]
      }
    );

    // Multiple array filters
    await this.collection.updateOne(
      { _id: new ObjectId(courseId) },
      {
        $set: {
          'modules.$[module].videos.$[video].isProcessed': true
        }
      },
      {
        arrayFilters: [
          { 'module.order': { $lte: 2 } },
          { 'video.quality': '4k' }
        ]
      }
    );
  }

  // ============ REAL-WORLD UPDATE SCENARIOS ============

  /**
   * Enroll a student in a course
   */
  async enrollStudent(courseId, userId) {
    const result = await this.collection.updateOne(
      { 
        _id: new ObjectId(courseId),
        'enrollments.userId': { $ne: new ObjectId(userId) }
      },
      {
        $push: {
          enrollments: {
            userId: new ObjectId(userId),
            enrolledAt: new Date(),
            progressPercentage: 0,
            lastAccessedAt: new Date(),
            completed: false
          }
        },
        $inc: { totalEnrollments: 1 }
      }
    );

    return result.modifiedCount > 0;
  }

  /**
   * Update student progress
   */
  async updateProgress(courseId, userId, progressPercentage) {
    const update = {
      $set: {
        'enrollments.$.progressPercentage': progressPercentage,
        'enrollments.$.lastAccessedAt': new Date()
      }
    };

    // If completed, set completed flag
    if (progressPercentage === 100) {
      update.$set['enrollments.$.completed'] = true;
      update.$set['enrollments.$.completedAt'] = new Date();
      update.$inc = { totalStudentsCompleted: 1 };
    }

    const result = await this.collection.updateOne(
      {
        _id: new ObjectId(courseId),
        'enrollments.userId': new ObjectId(userId)
      },
      update
    );

    return result.modifiedCount > 0;
  }

  /**
   * Apply bulk discount to courses based on criteria
   */
  async applyBulkDiscount(discountPercentage, criteria) {
    const filter = {
      isPublished: true,
      isFree: false
    };

    if (criteria.minEnrollments) {
      filter.totalEnrollments = { $gte: criteria.minEnrollments };
    }

    if (criteria.minRating) {
      filter.averageRating = { $gte: criteria.minRating };
    }

    if (criteria.categories && criteria.categories.length > 0) {
      filter.primaryCategory = { $in: criteria.categories };
    }

    const discountMultiplier = (100 - discountPercentage) / 100;

    const result = await this.collection.updateMany(
      filter,
      [
        {
          $set: {
            originalPrice: '$price',
            price: { $multiply: ['$price', discountMultiplier] },
            discountApplied: {
              percentage: discountPercentage,
              appliedAt: new Date()
            }
          }
        }
      ]
    );

    return result.modifiedCount;
  }
}

9. Delete Operations

Delete operations remove documents from a collection. MongoDB provides methods for deleting single documents, multiple documents, or all documents in a collection.

Delete Operations Flow Diagram

Delete Methods Comparison Table

Complete Delete Operations Implementation

// services/deleteService.js - Complete delete operations
const { ObjectId } = require('mongodb');

class DeleteOperationService {
  constructor(db) {
    this.db = db;
    this.collection = db.collection('courses');
  }

  // ============ BASIC DELETE METHODS ============

  /**
   * deleteOne() - Delete first matching document
   */
  async deleteOneCourse(filter) {
    try {
      const result = await this.collection.deleteOne(filter);

      return {
        success: result.acknowledged,
        deletedCount: result.deletedCount
      };
    } catch (error) {
      return {
        success: false,
        deletedCount: 0,
        error: error.message
      };
    }
  }

  /**
   * deleteMany() - Delete all matching documents
   */
  async deleteManyCourses(filter) {
    try {
      const result = await this.collection.deleteMany(filter);

      return {
        success: result.acknowledged,
        deletedCount: result.deletedCount
      };
    } catch (error) {
      return {
        success: false,
        deletedCount: 0,
        error: error.message
      };
    }
  }

  /**
   * findOneAndDelete() - Delete and return the document
   * Useful for archiving before deletion
   */
  async findAndDeleteCourse(filter) {
    const deletedCourse = await this.collection.findOneAndDelete(filter);
    return deletedCourse;
  }

  // ============ CONDITIONAL DELETE ============

  /**
   * Delete courses that haven't been updated in specified days
   */
  async deleteInactiveCourses(daysInactive) {
    const cutoffDate = new Date();
    cutoffDate.setDate(cutoffDate.getDate() - daysInactive);

    const result = await this.collection.deleteMany({
      isPublished: false,
      updatedAt: { $lt: cutoffDate },
      createdAt: { $lt: cutoffDate }
    });

    console.log(`Deleted ${result.deletedCount} inactive courses`);
    return result.deletedCount;
  }

  /**
   * Delete courses with zero enrollments and older than specified date
   */
  async deleteUnpopularCourses(olderThanMonths = 6) {
    const cutoffDate = new Date();
    cutoffDate.setMonth(cutoffDate.getMonth() - olderThanMonths);

    const result = await this.collection.deleteMany({
      totalEnrollments: 0,
      createdAt: { $lt: cutoffDate },
      isPublished: false
    });

    return result.deletedCount;
  }

  /**
   * Soft delete - mark as deleted instead of removing
   * Allows recovery and maintains referential integrity
   */
  async softDeleteCourse(courseId) {
    const result = await this.collection.updateOne(
      { _id: new ObjectId(courseId) },
      {
        $set: {
          isDeleted: true,
          deletedAt: new Date(),
          isPublished: false
        },
        $unset: { slug: "" }
      }
    );

    return result.modifiedCount > 0;
  }

  /**
   * Restore a soft-deleted course
   */
  async restoreSoftDeletedCourse(courseId, newSlug) {
    const result = await this.collection.updateOne(
      { 
        _id: new ObjectId(courseId),
        isDeleted: true 
      },
      {
        $set: {
          isDeleted: false,
          slug: newSlug
        },
        $unset: { deletedAt: "" }
      }
    );

    return result.modifiedCount > 0;
  }

  /**
   * Permanent deletion of soft-deleted courses older than retention period
   */
  async purgeSoftDeletedCourses(retentionDays = 30) {
    const cutoffDate = new Date();
    cutoffDate.setDate(cutoffDate.getDate() - retentionDays);

    const result = await this.collection.deleteMany({
      isDeleted: true,
      deletedAt: { $lt: cutoffDate }
    });

    console.log(`Permanently deleted ${result.deletedCount} soft-deleted courses`);
    return result.deletedCount;
  }
}

10. Write Concerns and Read Concerns

Write Concerns and Read Concerns control the durability, consistency, and isolation guarantees of operations.

Write Concern Levels Diagram

Write Concern Implementation

// Write Concern Examples

// w: 0 - Unacknowledged (fastest, no guarantee)
const result0 = await collection.insertOne(
  { title: "Fast Insert" },
  { writeConcern: { w: 0 } }
);

// w: 1 - Acknowledged by primary (default)
const result1 = await collection.insertOne(
  { title: "Standard Insert" },
  { writeConcern: { w: 1 } }
);

// w: majority - Acknowledged by majority of replica set members
const resultMajority = await collection.insertOne(
  { title: "Critical Data" },
  { writeConcern: { w: 'majority', j: true } }
);

// w: majority with timeout
const resultWithTimeout = await collection.insertOne(
  { title: "Timed Write" },
  { writeConcern: { w: 'majority', wtimeout: 5000 } }
);

// w: custom tag set (write to specific data center)
const resultTagSet = await collection.insertOne(
  { title: "Regional Data" },
  { writeConcern: { w: 'east-coast' } }
);

Read Concern Levels

// Read Concern Examples

// readConcern: 'local' (default) - Returns most recent data available
const localRead = await collection.find(
  { status: 'active' },
  { readConcern: { level: 'local' } }
).toArray();

// readConcern: 'majority' - Returns data that has been acknowledged by majority
const majorityRead = await collection.find(
  { status: 'active' },
  { readConcern: { level: 'majority' } }
).toArray();

// readConcern: 'linearizable' - Strongest consistency (for critical reads)
const linearizableRead = await collection.findOne(
  { _id: criticalId },
  { readConcern: { level: 'linearizable' } }
);

// readConcern: 'available' - For sharded clusters, returns available data
const availableRead = await collection.find(
  {},
  { readConcern: { level: 'available' } }
).toArray();

// readConcern: 'snapshot' - For transactions
const session = client.startSession();
session.startTransaction({
  readConcern: { level: 'snapshot' },
  writeConcern: { w: 'majority' }
});

11. Error Handling and Validation

Proper error handling is crucial for production MongoDB applications.

Error Types and Handling

// Error handling utilities
class ErrorHandler {
  // Duplicate Key Error (code 11000)
  handleDuplicateKeyError(error) {
    if (error.code === 11000) {
      const field = Object.keys(error.keyPattern)[0];
      const value = error.keyValue[field];
      return {
        type: 'DUPLICATE_KEY',
        message: `Duplicate value '${value}' for field '${field}'`,
        field,
        value
      };
    }
    return null;
  }

  // Validation Error (Mongoose)
  handleValidationError(error) {
    if (error.name === 'ValidationError') {
      const errors = {};
      for (const field in error.errors) {
        errors[field] = error.errors[field].message;
      }
      return {
        type: 'VALIDATION_ERROR',
        message: 'Document validation failed',
        errors
      };
    }
    return null;
  }

  // Cast Error (invalid ObjectId, etc.)
  handleCastError(error) {
    if (error.name === 'CastError') {
      return {
        type: 'CAST_ERROR',
        message: `Invalid value '${error.value}' for field '${error.path}'`,
        field: error.path,
        value: error.value
      };
    }
    return null;
  }

  // Write Concern Error
  handleWriteConcernError(error) {
    if (error.writeConcernError) {
      return {
        type: 'WRITE_CONCERN_ERROR',
        message: error.writeConcernError.errmsg,
        code: error.writeConcernError.code
      };
    }
    return null;
  }

  // Network/Timeout Error
  handleNetworkError(error) {
    if (error.message?.includes('timed out') || 
        error.message?.includes('socket') ||
        error.message?.includes('connection')) {
      return {
        type: 'NETWORK_ERROR',
        message: error.message,
        retryable: true
      };
    }
    return null;
  }

  // Comprehensive error handler
  handleError(error) {
    const handlers = [
      this.handleDuplicateKeyError,
      this.handleValidationError,
      this.handleCastError,
      this.handleWriteConcernError,
      this.handleNetworkError
    ];

    for (const handler of handlers) {
      const result = handler(error);
      if (result) return result;
    }

    return {
      type: 'UNKNOWN_ERROR',
      message: error.message,
      originalError: error
    };
  }
}

// Retry logic for transient errors
async function withRetry(operation, maxRetries = 3, delay = 1000) {
  let lastError;

  for (let i = 0; i < maxRetries; i++) {
    try {
      return await operation();
    } catch (error) {
      lastError = error;

      // Only retry on specific error types
      const retryableErrors = ['NetworkTimeout', 'ConnectionError', 'TransientTransactionError'];
      const isRetryable = retryableErrors.some(type => error.message?.includes(type));

      if (!isRetryable && i < maxRetries - 1) {
        console.log(`Retry attempt ${i + 1} for operation...`);
        await new Promise(resolve => setTimeout(resolve, delay * Math.pow(2, i)));
        continue;
      }

      throw error;
    }
  }

  throw lastError;
}

12. Database Profiling and Slow Query Logging

Monitoring and optimizing query performance is essential for production deployments.

Profiling Levels

// Database Profiler Configuration

// Level 0 - Profiler off (default)
db.setProfilingLevel(0);

// Level 1 - Log only slow operations (over slowms threshold)
db.setProfilingLevel(1, { slowms: 100 });

// Level 2 - Log all operations (for debugging)
db.setProfilingLevel(2);

// Get current profiling status
const status = db.getProfilingStatus();
console.log(status);

// View profiler data
const slowQueries = db.system.profile.find({ millis: { $gt: 100 } })
  .sort({ ts: -1 })
  .limit(10)
  .toArray();

// Enable slow query logging for specific collection
db.setLogLevel(1, 'query');

// Create indexes from slow queries
db.system.profile.aggregate([
  { $match: { op: 'query', millis: { $gt: 200 } } },
  { $group: { _id: '$query', count: { $sum: 1 }, avgTime: { $avg: '$millis' } } },
  { $sort: { avgTime: -1 } },
  { $limit: 10 }
]).toArray();

13. What We Learned from Story 1

Congratulations on completing Story 1 of the MongoDB Evolution Series! Let’s recap the key concepts we mastered:

Key Takeaways

Practical Skills Gained

1. Document Design: Understanding BSON types and when to use each for optimal storage and performance
2. CRUD Mastery: Confidently performing create, read, update, and delete operations with proper options
3. Query Building: Constructing complex queries using the full range of MongoDB operators
4. Bulk Operations: Efficiently handling large datasets with batch operations
5. Error Resilience: Implementing proper error handling and retry logic
6. Performance Monitoring: Using database profiler to identify and fix slow queries

Code Patterns Mastered

// The complete CRUD pattern we learned
async function completeCRUDExample() {
  // CREATE
  const result = await collection.insertOne(courseData);

  // READ with operators
  const course = await collection.findOne({
    price: { $gte: 50, $lte: 200 },
    tags: { $in: ['mongodb', 'database'] }
  });

  // UPDATE with operators
  await collection.updateOne(
    { _id: course._id },
    { $inc: { views: 1 }, $set: { lastViewed: new Date() } }
  );

  // DELETE
  await collection.deleteOne({ _id: course._id });
}

14. Looking Ahead to Story 2

In Story 2: MongoDB Projection, Sorting, Pagination — Evolution, we will build upon the CRUD foundations we’ve established and explore advanced data shaping and retrieval techniques.

What’s Coming in Story 2

Story 2 Preview: Key Concepts

Example: What You’ll Be Able to Build After Story 2

// After Story 2, you'll build efficient APIs like this:
app.get('/api/courses', async (req, res) => {
  const { page, limit, sortBy, fields } = req.query;

  // Efficient cursor pagination
  const result = await courseService.getCursorPaginatedCourses(
    req.cursor, 25, 'createdAt', -1
  );

  // Smart projection based on client needs
  const projection = buildProjection(fields);

  // Multi-field sorting
  const sort = { difficulty: 1, averageRating: -1 };

  res.json(result);
});

Transition Note

“Story 1 gave us the ability to create, read, update, and delete data. In Story 2, we’ll learn how to shape, order, and paginate that data for production-ready APIs that can handle millions of users efficiently.”

Series Progress

About the Author

Vineet Sharma

• Medium: mvineetsharma.medium.com
• LinkedIn: linkedin.com/in/vineet-sharma-architect

In-depth React, TypeScript, Node.js, MongoDB, Next.js, and System Design. New articles weekly.

If you found this story helpful, please clap 👏 and share with your network. Your support helps more developers discover this content.

📌 Save this story to your reading list — it helps other developers discover it.

❓ Questions? Feedback? Leave a response below. If you’re implementing something similar and want to discuss architectural tradeoffs, I’m always happy to connect with fellow engineers tackling these challenges.

Happy Coding! 🚀

[embed]