How SageMaker Improve Productivity — An AWS Pipeline

Unraveling the Essence of Compute Resources in the Creation of an End-to-End ML Pipeline

Photo by Alain Duchateau on Unsplash

In the dynamic landscape of modern machine learning, the cloud emerges as an indispensable ally, transforming the way we bring machine learning models to life. The seamless integration of cloud services facilitates the entire lifecycle of machine learning systems, from development and training to deployment and scalability. Cloud platforms, such as Amazon Web Services (AWS), provide a versatile and scalable infrastructure that not only accelerates the development process but also ensures the seamless transition of machine learning models into production. With the cloud’s elastic compute capabilities, practitioners can efficiently manage resources, handle complex computations, and deploy models at scale, ultimately unlocking the full potential of machine learning applications. In the realm of deploying machine learning in production (MLOps), the cloud stands as a catalyst, propelling innovation, agility, and efficiency into the heart of the machine learning workflow.

Navigating the construction of a resilient machine learning infrastructure resembles orchestrating a symphony, where diverse layers seamlessly collaborate to bring forth innovation. The foundation is laid by data warehouses, storing the essence of insights. Compute resources flex their muscles, providing the raw power to transform data into real value. A virtuoso job scheduler conducts the flow, orchestrating tasks fluidly. Versioning emerges as the silent guardian, ensuring models speak the same language across time. Architectural prowess defines the blueprint, a structured symphony of interconnected elements. Model operations take center stage, ensuring models perform with precision in real-world scenarios. Feature engineering crafts the melody, shaping raw data into refined patterns. Finally, model development is the composer’s act, where algorithms become the music of intelligent systems. In this ensemble, each layer plays a unique role, contributing to the crescendo of a well-tuned machine learning infrastructure.

If you’re intrigued by the intricacies of this infrastructure, I highly recommend checking out Ville Tuulos’ excellent book, available here. Ville is also the Co-Founder and CEO of Outerbounds, the company responsible for Metaflow — a powerful framework for constructing machine learning workflows initially developed at Netflix.

What about compute resources

The compute layer in a machine learning infrastructure is the backbone that propels scalability, efficiency, and adaptability. Scalability, a central tenet in this context, involves handling increased workloads seamlessly by efficiently utilizing additional resources. It’s not merely about optimizing performance for a fixed amount of work but rather accommodating more data processing or model training by adding resources like more computers or memory.

The compute layer allows projects to be more scalable (Effective Data Science Infrastructure, Ville Tuulos)

Its focus is on the ‘where,’ simply deciding where to run a task. For data scientists and machine learning practitioners, this means a hassle-free playground for experiments, endpoint instances, and whatever else needs to keep running smoothly. It simplifies the technical nitty-gritty, allowing users to focus on their work without getting bogged down by the intricacies of execution. In essence, the compute layer provides the necessary stage for tasks to shine, taking care of the logistics behind the scenes.

The cloud provides a semblance of unlimited compute capacity (Ville Tuulos).

Various compute resources services, such as Kubernetes, AWS Batch, AWS Lambda, Apache Spark, and distributed training platforms like SageMaker, offer different workload support, latency, workload management, cost efficiency, and operational complexity. The choice of compute layer depends on specific project needs.

Now that we’ve explored the fundamental role of the compute layer in orchestrating machine learning workflows, let’s now dive into the AWS SageMaker pipeline that I’ve crafted — a streamlined and efficient system that encapsulates the power of SageMaker.

SageMaker in Action

Essentially, SageMaker stands as AWS’s comprehensive machine learning platform, offering the tools to construct an end-to-end machine learning system. It seamlessly integrates with various AWS features and accommodates the use of different frameworks, enabling flexibility in development. Whether utilizing a Jupyter notebook instance or the SageMaker Studio, the core concept remains constant — efficiently constructing the architecture, from data processing to model monitoring.

Within SageMaker, the SageMaker Pipeline emerges as the singular method for crafting a workflow within the SageMaker infrastructure. It involves leveraging the SageMaker SDK to define and sequence steps, intricately specifying each required element. This approach optimizes every stage of the workflow by harnessing the power of the AWS environment that underlies SageMaker.

To provide a practical demonstration, I’ve constructed a pipeline with the goal of developing a model to address a binary classification challenge related to hotel bookings. You can access the dataset here. While the model itself, an XGBoost with a set of hyperparameters, may not be groundbreaking, the primary aim is to showcase the workflow — understanding its functionalities and assessing potential enhancements and additions for future refinement.

Let’s take a closer look at the stack behind this workflow.

The Stack Architecture

Sagemaker plays a central role in this architecture. The goal was to maximize its potential by integrating other components to ensure smooth operation. The entirety of the right stack forms the Sagemaker pipeline, while the left stack is dedicated to versioning elements, encompassing code and all aspects related to artifacts and data.

This is how the code behind the pipeline looks like

The main element of the stack are those one:

• S3 as the Central Storage:
• Serves as the primary storage, facilitating the transit of every element within the pipeline.
• The code utilized in the pipeline is stored within S3.
• Versioning extends to everything surrounding the model, ensuring readiness for deployment.
• Github for Version Control: Utilized for effective version control, managing changes and updates.
• SageMaker:
• Employs instances for processing each step with parameterized instances, providing the computational resources discussed earlier.
• Offers key features, including pre-built models (such as XGBoost), a model registry, and inference endpoints.
• Introduces “Clarify,” an additional SageMaker component enhancing model evaluation beyond conventional metrics like Area Under the Curve (AUC). It examines bias and provides insights into model explainability using SHAP.
• Lambda Function for Lightweight Model Deployment:
• Utilized for deploying the model in a lightweight manner. By “lightweight,” I refer to deployments for development or test endpoints, or for internal endpoints with lower traffic volumes, excluding customer-facing high-traffic scenarios.

A pivotal parameter in the pipeline is the Condition Step, designed to assess conditions and trigger subsequent steps accordingly. Within the stack, I implemented a fundamental condition checking for the AUC metric to exceed 70%. When the condition evaluates to True, the model is registered and deployed to the endpoint.

Improvements

This example, while simple, addresses several problems and boasts the advantage of being entirely cloud-based, versioned, reproducible, and more. All four V’s — Volume, Velocity, Validity and Variety — are accounted for.

To enhance robustness and refine the engineering:

1. Enhance CI/CD by incorporating AWS CodeBuild or Github Actions.
2. Integrate unit tests for added reliability.
3. Implement a dedicated orchestrator to schedule jobs, particularly useful for handling new data arrivals.
4. Introduce flexibility by enabling experimentation with alternative models.
5. Implement monitoring and logging using AWS CloudWatch, for instance.

Thanks for taking the time to read this article, I hope you enjoyed reading about the computer layer and SageMaker Pipeline. Please feel free to share this post with your friends, and if you are interested in data science or machine learning, check out my other articles here.

Resources

[embed]Effective Data Science Infrastructure Simplify data science infrastructure to give data scientists an efficient path from prototype to production.www.manning.com

[embed]GitHub - npogeant/hotel-booking-aws-pipeline: An AWS Sagemaker Pipeline for Hotel Bookings… An AWS Sagemaker Pipeline for Hotel Bookings Classification using s3, Lambda. - GitHub …github.com

[embed]What is Amazon SageMaker? Amazon SageMaker is a fully managed machine learning service. With SageMaker, data scientists and developers can…docs.aws.amazon.com