Lecture 4 - API driven Cloud Native Solutions - By Animesh Giri

1. Serverless Computing and Architecture Serverless computing is an execution model where cloud providers dynamically manage the allocation of machine resources.
   • The Paradigm Shift: In a serverless architecture, applications run on-demand and consume only the resources specifically required for their execution. The primary benefit is that developers no longer need to manage or provision underlying physical infrastructure, servers, storage, or networks.
   • Developer Focus: Cloud providers handle the "heavy lifting" of the physical infrastructure and runtime orchestration, allowing developers to focus entirely on writing business logic, building APIs, and delivering customer value.
   • Function as a Service (FaaS): This model executes functions automatically in response to certain events or triggers, automatically scaling up based on demand. Popular examples include AWS Lambda, Google Cloud Functions, and Azure Functions.
2. The AWS Serverless Services Stack Serverless encompasses much more than just computation; it involves a fully managed stack of services. Key components in the AWS ecosystem include:
   • Compute: AWS Lambda runs the core application code in response to events, while AWS Fargate provides a serverless computing engine specifically designed for running containers.
   • Storage & Databases: Amazon S3 provides highly scalable object storage. Amazon DynamoDB serves as a fully managed NoSQL database, and Amazon Aurora Serverless acts as an on-demand, auto-scaling relational database.
   • Integration & Orchestration:
     • Amazon API Gateway: Routes HTTP requests and acts as the interface to trigger backend microservices.
     • Amazon SQS (Simple Queue Service): Decouples microservices to manage message queuing.
     • Amazon SNS (Simple Notification Service): Handles messaging and application-to-person notifications.
     • AWS Step Functions: Orchestrates multiple AWS services into a single unified workflow.
3. Application Workflow Example (AWS Stack) The lecture featured a practical demonstration of how these components interact to form a functional API-driven backend:
   • A client application sends HTTP GET or POST requests to the Amazon API Gateway.
   • The gateway routes the request and triggers the appropriate AWS Lambda function.
   • Inside the Lambda function, developers use tools like JSON (for formatting request/response bodies) and Boto3 (the Python SDK for AWS) to execute backend logic without needing a continuously running standard HTTP server.
   • The Lambda function then communicates with Amazon DynamoDB to either scan and retrieve existing user information (GET request) or put_item to insert new user data (POST request) into the database.
4. Drawbacks and Limitations of Serverless While highly efficient and cost-effective, serverless computing introduces several significant trade-offs:
   • Not Ideal for Long-Running Tasks: FaaS platforms like AWS Lambda are optimized only for short-lived tasks (e.g., resizing an image, handling an API request, or chatting with a bot). They are heavily constrained by time and memory limitations, making them entirely unsuitable for long-running processes such as continuous video rendering, real-time data streaming (like Kafka), or training large machine learning models.
   • Vendor Lock-in: Serverless architectures rely heavily on proprietary services specific to a single cloud provider. Transitioning from AWS to Azure or Google Cloud is highly difficult and usually requires completely rewriting large portions of the application configuration.
   • Cold Start Latency: Serverless containers do not stay "warm" indefinitely. If a function is invoked after a period of inactivity, the cloud provider must initialize a brand new container, causing a noticeable delay or latency before the code actually executes.
5. The Shift to DevOps Traditional Software Development Life Cycles (SDLC) force Development and Operations teams to work in isolated silos. This creates a "wall of conflict" because Developers focus heavily on agility (pushing new features), while Operations focus on stability (maintaining system reliability). DevOps bridges this gap by merging agile development processes with operational stability to enable continuous value delivery. DevOps fundamentally revolves around three pillars:
   • People: Fostering a cultural shift toward collaboration and shared responsibility.
   • Process: Adopting methodologies to automate repetitive tasks and streamline lifecycles.
   • Tools: Utilizing the right technologies for source control, continuous integration, and infrastructure management.
6. CI/CD Pipelines A critical process implementation of DevOps is the CI/CD pipeline, which automates different stages of software delivery:
   • Continuous Integration (CI): Developers frequently merge code updates into a shared central repository. The system then automatically builds the code and runs unit testing to quickly detect bugs and provide fast feedback.
   • Continuous Delivery (CD): Extends CI by automating the preparation of the software build for staging and production environments. However, Continuous Delivery always requires a manual approval step by a manager or an authoritative figure before the final deployment to the live production server.
   • Continuous Deployment (CD): Takes automation to the absolute maximum by eliminating all manual intervention. If the code successfully passes all automated acceptance tests, it is deployed straight to production automatically.
   • Smoke Testing: After deployment to the live production environment, smoke tests are executed to validate functionality and catch configuration errors or missing dependencies.
7. GitOps GitOps is a set of best practices explicitly tailored for cloud-native applications and Kubernetes environments.
   • It utilizes Git (tools like GitHub or GitLab) as the single source of truth for both application code and declarative infrastructure configurations.
   • Any configuration update pushed to the Git repository automatically triggers a pipeline that updates the runtime environment to match, simplifying continuous deployment and making system rollbacks incredibly easy.

Real-World API Automation Case Study

A student (Ravi) demonstrated a real-world scenario from 2020 where an organization needed to rapidly collect the hardware configurations (RAM, HDD capacity) of over 2,000 newly hired remote trainees.

• Instead of manually asking trainees, the team built a lightweight Python Flask web server and deployed it on an Azure Virtual Machine to act as a listener and API endpoint.
• They provided a PowerShell script to the remote trainees running Windows 10.
• Upon execution, the script automatically collected the system's hardware specs and sent an HTTP POST/PUT request over the internet to the Flask API endpoint.
• The Flask server received this data dynamically and wrote it all directly into a centralized CSV file, allowing the organization to easily filter and identify which trainees lacked sufficient computing power to begin training.