Implementation of AWS CloudWatch Synthetics for Real-Time Application URL Monitoring

The challenge

In this hands-on project, I explored the advanced aspects of AWS CloudWatch, focusing on the implementation of CloudWatch Synthetics for real-time monitoring of application URLs, integrated with CloudWatch Alarms, SNS, and AWS Chatbot for instant alerts on Slack. The goal was to establish a robust monitoring system for the HumanGov application, ensuring high availability and performance.

To begin, I created CloudWatch Synthetics canaries. These canaries were designed to continuously monitor the HumanGov application URLs, simulating user interactions to verify availability and response times.

After configuring the canaries, I integrated CloudWatch Alarms. These alarms were set up to trigger based on specific metrics, such as high response times or error rates, ensuring immediate detection of any application issues.

The alarm thresholds were carefully fine-tuned to balance sensitivity and practicality — preventing false positives while ensuring that no critical issue would go unnoticed.

To enhance responsiveness to these alarms, I integrated AWS SNS (Simple Notification Service). This allowed the creation of notification topics that would broadcast any changes in alarm states to designated users, ensuring a continuous flow of information regarding application health.

The highlight of this project was the integration of AWS Chatbot with Slack. By linking SNS topics to AWS Chatbot, I enabled real-time alerts to be sent directly to a dedicated Slack channel. This ensured that both the development and SRE (Site Reliability Engineering) teams were instantly informed of any application issues, allowing for swift action.

The project culminated in a series of rigorous tests to validate the entire configuration. By simulating various application issues, I confirmed that the canaries correctly detected the problems, the alarms were triggered as expected, and the notifications were promptly delivered by AWS Chatbot to Slack.

This intensive project not only strengthened my understanding of AWS CloudWatch and its integrations but also highlighted the importance of proactive monitoring in maintaining cloud application reliability and performance.

Architecture

In this project, I configured a proactive monitoring system for distributed applications.

The journey included:

o Creating an SNS topic to centralize notifications.

o Configuring CloudWatch Synthetics Canaries (Heartbeat Canary) to monitor critical endpoints and validate real-time availability.

o Creating a workspace and Slack channel for the SRE team.

o Integrating Amazon Q Developer with Slack, connecting AWS monitoring directly to the team’s communication channel.

o Validating the setup by simulating failures and monitoring alerts automatically sent to the channel.

As a result, I achieved a practical observability and incident response solution, ensuring that critical applications remain under constant watch and that the team receives alerts quickly and in a centralized way.

Step 1 — Creating an SNS Topic (Amazon SNS)

I created a Standard SNS Topic through which alert notifications would be published: Amazon SNS >> Topics >> Create Topic >> Type: Standard >> Name: humangov-chatbot-notifications.

Step 2 — Creating a Heartbeat Canary (Amazon CloudWatch)

This step involved setting up monitoring for the application URLs (Sites: California and Florida):

Amazon CloudWatch

o Application Signals (APM)

o Synthetics Canaries >> Create Canary

o Use Blueprint

o Heartbeat Monitoring

o Name: humangov-canary-XXXXXXXX (Create one for each site: California and Florida)

o Application or endpoint URL: https://xxxxxxxx.domain (Create one for each site: California and Florida)

o Screenshots: Take screenshots (captures a screenshot automatically when an error occurs)

o Runtime version: Default selected.

o Schedule:

§ Run continuously;

§ Run Canary every 2 minutes (adjust as needed);

§ Start immediately after creation.

o Data Retention:

§ Failure data retention: Custom value — 1 day (adjust as needed; set to 1 day for educational purposes).

§ Success data retention: Custom value — 1 day (adjust as needed; set to 1 day for educational purposes).

o CloudWatch Alarms:

§ Add new alarm:

§ Metric name: Failed → Greater/Equal → 1 minute.

§ Set notifications for this canary: select the topic created in Step 1.

Canaries running

The Amazon CloudWatch Synthetics Canary script monitors URLs by simulating real user access, validating HTTP status codes, recording performance metrics, and capturing screenshots to proactively identify failures.

Step 3 — Creating a Slack Workspace and Communication Channel

If you don’t already have a Slack account, you’ll need to register for one.

In Slack, create a new workspace (Create a new workspace) and give it the name of your organization or department. Within the newly created workspace, create a new channel: Channels >> Create >> Create channel. Assign a name of your choice to this channel.

Step 4 — Integrating Amazon Q with Slack

Navigate to: Amazon Q Developer >> Amazon Q in chat applications >> Configure chat client >> Chat Client: Slack >> Configure Client

You will be redirected to grant permission for Amazon Q Developer to access your Slack workspace.

Permission Amazon Q to access Slack

Step 5 — Configuring the Slack Channel in Amazon Q

Navigate to: Amazon Q in chat applications >> Configure new channel

• Configuration name: (name of your workspace)
• Slack channel: (name of your Slack channel)

Permissions: Select the IAM Role created to link with Slack.

Notifications:

• SNS Topics
• **Region 1: **<Your AWS Cloud region>
• **Topic 1: **<Name of the Topic created in Step 1>

Step 6 — Integrating the Slack Channel with Amazon Q

This step links your Slack communication channel directly with Amazon Q, enabling real-time notifications and interactions within your workspace.

Installing Client

• Add apps to <Workspace>
• Amazon Q Developer >> Install

Note that we now have the ‘Amazon Q’ application installed in Slack.

Amazon Q Client

Navigate to: Amazon Q in chat applications

• Slack

• Name of your <Amazon Q instance>

• Configuration name: Name of the Slack channel

Press

You should receive a message similar to the one shown below:

Step 8 — Validating CloudWatch Synthetics Canaries

At this stage, simulate the application’s unavailability to start observing how your Canary detects and collects failure data.

Example: Stopping the application on the California site.

kubectl scale deployment humangov-nginx-reverse-proxy-california — replicas=0

kubectl get deploy

kubectl get pods

After 2 minutes, you will see the following:

Cloudwatch received Data captured by the Canaries

Alarms dispatched

In Slack, you will receive a notification like the one shown below:

Notification received in the Slack