Skip to content

robindevelops/Full-Stack-DevOps-Pipeline

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

3 Commits
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

πŸš€ Full-Stack DevOps Pipeline β€” Amazon Prime Video Clone

A fully automated, production-grade CI/CD pipeline for deploying an Amazon Prime Video clone using modern DevOps practices and cloud-native tooling.


Architecture CI/CD IaC Container Orchestration GitOps Monitoring


Full-Stack DevOps Pipeline Architecture

πŸ“‹ Table of Contents


🎯 Project Overview

This project demonstrates a complete end-to-end DevOps pipeline for an Amazon Prime Video clone application. The entire workflow β€” from code commit to production deployment β€” is fully automated with industry-standard tools.

What This Project Covers

Area Tools Used
Source Control GitHub
CI/CD Automation Jenkins
Code Quality SonarQube
Dependency Management npm
Security Scanning Aqua Trivy
Containerization Docker
Container Registry AWS ECR
Infrastructure as Code Terraform
Kubernetes Orchestration AWS EKS
Package Management Helm
GitOps / Continuous Delivery ArgoCD
Monitoring Prometheus & Grafana

πŸ— Architecture

Developer β†’ GitHub β†’ Jenkins Pipeline
                         β”‚
                         β”œβ”€β”€ 1. Code Checkout
                         β”œβ”€β”€ 2. SonarQube Analysis (Code Quality)
                         β”œβ”€β”€ 3. npm Install & Build
                         β”œβ”€β”€ 4. Trivy Scan (Security)
                         β”œβ”€β”€ 5. Docker Build & Tag
                         β”œβ”€β”€ 6. Push to AWS ECR
                         β”‚
                         β–Ό
                    AWS ECR (Container Registry)
                         β”‚
                         β–Ό
                    ArgoCD (GitOps)
                         β”‚
                         β”œβ”€β”€ Watches Helm chart changes
                         β”œβ”€β”€ Syncs desired state
                         β”‚
                         β–Ό
                    AWS EKS Cluster
                         β”‚
                         β”œβ”€β”€ Pulls image from ECR
                         β”œβ”€β”€ Deploys Pods via Helm
                         β”œβ”€β”€ Manages Services & Ingress
                         β”‚
                         β–Ό
                    Prometheus + Grafana (Monitoring)

High-Level Flow

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  GitHub  │───▢│ Jenkins  │───▢│ SonarQube  │───▢│  npm  │───▢│ Trivy │───▢│ Docker β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”¬β”€β”€β”€β”˜
                                                                                 β”‚
                     β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                     β–Ό
              β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
              β”‚  AWS ECR   │───▢│  ArgoCD  │───▢│ AWS EKS │───▢│ Prometheus  β”‚
              β”‚  (Registry)β”‚    β”‚ (GitOps) β”‚    β”‚  (K8s)  β”‚    β”‚ + Grafana   β”‚
              β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

πŸ›  Tech Stack

Application

Layer Technology Purpose
Frontend React.js Amazon Prime Video clone UI
Backend Node.js REST API server
Styling CSS Responsive design & animations

DevOps & Infrastructure

Category Tool Purpose
Version Control GitHub Source code management & webhooks
CI/CD Jenkins Pipeline orchestration & automation
Code Quality SonarQube Static code analysis & quality gates
Security Aqua Trivy Container & filesystem vulnerability scan
Containerization Docker Application containerization
Registry AWS ECR Private container image registry
IaC Terraform AWS infrastructure provisioning
Orchestration AWS EKS Managed Kubernetes cluster
Packaging Helm Kubernetes manifest templating & management
GitOps ArgoCD Declarative continuous delivery
Monitoring Prometheus Metrics collection & alerting
Visualization Grafana Dashboards & observability

πŸ”„ Pipeline Flow

End-to-End CI/CD Pipeline Stages

 β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
 β”‚                        JENKINS CI/CD PIPELINE                               β”‚
 β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
 β”‚  Stage 1β”‚  Stage 2  β”‚ Stage 3  β”‚ Stage 4  β”‚  Stage 5  β”‚     Stage 6        β”‚
 β”‚         β”‚           β”‚          β”‚          β”‚           β”‚                    β”‚
 β”‚  Clone  β”‚ SonarQube β”‚   npm    β”‚  Trivy   β”‚  Docker   β”‚  Push to ECR      β”‚
 β”‚  Repo   β”‚  Analysis β”‚  Build   β”‚  Scan    β”‚  Build    β”‚  + Deploy via     β”‚
 β”‚         β”‚           β”‚          β”‚          β”‚           β”‚    ArgoCD          β”‚
 β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
  1. πŸ“₯ Clone β€” Pull latest code from GitHub
  2. πŸ” SonarQube Analysis β€” Static code analysis for bugs, vulnerabilities, code smells
  3. πŸ“¦ npm Install & Build β€” Install dependencies and build the application
  4. πŸ›‘οΈ Trivy Security Scan β€” Scan filesystem and Docker images for CVEs
  5. 🐳 Docker Build & Tag β€” Build production Docker image with proper tagging
  6. πŸ“€ Push to AWS ECR β€” Push image to private container registry
  7. πŸš€ ArgoCD Sync β€” ArgoCD detects Helm chart changes and deploys to EKS

πŸ“ Prerequisites

Before setting up this pipeline, ensure you have the following:

Local Development

CI/CD & Infrastructure

AWS Resources

  • AWS Account with IAM permissions for EKS, ECR, VPC, and EC2
  • An S3 bucket for Terraform state (recommended)

πŸ“ Project Structure

full_stack_devops_pipeline/
β”‚
β”œβ”€β”€ πŸ“„ server.js                    # Node.js application entry point
β”œβ”€β”€ πŸ“„ package.json                 # Node.js dependencies & scripts
β”œβ”€β”€ πŸ“„ Dockerfile                   # Multi-stage Docker build
β”œβ”€β”€ πŸ“„ docker-compose.yml           # Local development orchestration
β”œβ”€β”€ πŸ“„ Jenkinsfile                  # CI/CD pipeline definition
β”œβ”€β”€ πŸ“„ .dockerignore                # Docker build exclusions
β”œβ”€β”€ πŸ“„ .gitignore                   # Git exclusions
β”œβ”€β”€ πŸ“„ README.md                    # Project documentation (this file)
β”‚
β”œβ”€β”€ πŸ“‚ src/                         # Application source code
β”‚   β”œβ”€β”€ πŸ“‚ components/              # React components
β”‚   β”œβ”€β”€ πŸ“‚ pages/                   # Page-level components
β”‚   β”œβ”€β”€ πŸ“‚ assets/                  # Static assets (images, icons)
β”‚   └── πŸ“„ App.js                   # Main React application
β”‚
β”œβ”€β”€ πŸ“‚ terraform/                   # Infrastructure as Code
β”‚   β”œβ”€β”€ πŸ“„ main.tf                  # Main Terraform configuration
β”‚   β”œβ”€β”€ πŸ“„ variables.tf             # Input variables
β”‚   β”œβ”€β”€ πŸ“„ outputs.tf               # Output values
β”‚   β”œβ”€β”€ πŸ“„ provider.tf              # AWS provider configuration
β”‚   β”œβ”€β”€ πŸ“‚ modules/
β”‚   β”‚   β”œβ”€β”€ πŸ“‚ vpc/                 # VPC networking module
β”‚   β”‚   β”œβ”€β”€ πŸ“‚ eks/                 # EKS cluster module
β”‚   β”‚   └── πŸ“‚ ecr/                 # ECR repository module
β”‚   └── πŸ“„ terraform.tfvars         # Variable values
β”‚
β”œβ”€β”€ πŸ“‚ helm/                        # Helm chart for Kubernetes deployment
β”‚   β”œβ”€β”€ πŸ“„ Chart.yaml               # Chart metadata
β”‚   β”œβ”€β”€ πŸ“„ values.yaml              # Default configuration values
β”‚   └── πŸ“‚ templates/
β”‚       β”œβ”€β”€ πŸ“„ deployment.yaml      # K8s Deployment manifest
β”‚       β”œβ”€β”€ πŸ“„ service.yaml         # K8s Service manifest
β”‚       β”œβ”€β”€ πŸ“„ ingress.yaml         # K8s Ingress manifest
β”‚       └── πŸ“„ hpa.yaml             # Horizontal Pod Autoscaler
β”‚
β”œβ”€β”€ πŸ“‚ monitoring/                  # Monitoring stack configuration
β”‚   β”œβ”€β”€ πŸ“„ prometheus.yml           # Prometheus configuration
β”‚   └── πŸ“‚ grafana/
β”‚       └── πŸ“‚ dashboards/          # Grafana dashboard JSON files
β”‚
└── πŸ“‚ scripts/                     # Utility scripts
    β”œβ”€β”€ πŸ“„ setup-jenkins.sh         # Jenkins setup automation
    β”œβ”€β”€ πŸ“„ install-tools.sh         # Tool installation script
    └── πŸ“„ cleanup.sh               # Resource cleanup script

πŸš€ Getting Started

1. Clone the Repository

git clone https://github.com/<your-username>/full-stack-devops-pipeline.git
cd full-stack-devops-pipeline

2. Local Development

# Install dependencies
npm install

# Start the development server
npm run dev

# Application will be available at http://localhost:3000

3. Docker Build

# Build the Docker image
docker build -t prime-video-clone .

# Run the container
docker run -d -p 3000:3000 --name prime-clone prime-video-clone

# Or use Docker Compose
docker-compose up -d

4. Infrastructure Provisioning (Terraform)

cd terraform/

# Initialize Terraform
terraform init

# Preview the infrastructure changes
terraform plan

# Apply β€” provisions VPC, EKS Cluster, ECR, and IAM Roles
terraform apply -auto-approve

# Get EKS kubeconfig
aws eks update-kubeconfig --name prime-clone-cluster --region us-east-1

5. CI/CD Pipeline (Jenkins)

Jenkins Setup

  1. Install Jenkins on an EC2 instance or run locally via Docker:

    docker run -d --name jenkins \
      --restart=on-failure \
      -p 8080:8080 \
      -v jenkins_home:/var/jenkins_home \
      -v /var/run/docker.sock:/var/run/docker.sock \
      jenkins/jenkins:lts
  2. Install required Jenkins plugins:

    • Docker Pipeline
    • SonarQube Scanner
    • AWS Credentials
    • Pipeline: AWS Steps
    • NodeJS Plugin
  3. Configure credentials in Jenkins:

    • github-credentials β€” GitHub personal access token
    • sonarqube-token β€” SonarQube authentication token
    • aws-credentials β€” AWS Access Key & Secret Key
    • dockerhub-credentials β€” Docker Hub credentials (if using)
  4. Create a Pipeline job pointing to the Jenkinsfile in this repository.

  5. Set up GitHub Webhook β†’ http://<jenkins-url>:8080/github-webhook/

Jenkinsfile Overview

pipeline {
    agent any

    environment {
        AWS_ACCOUNT_ID   = credentials('aws-account-id')
        AWS_REGION       = 'us-east-1'
        ECR_REPO         = "${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/prime-clone"
        SONAR_HOST       = 'http://<sonarqube-server>:9000'
    }

    stages {
        stage('Checkout')          { steps { git branch: 'main', url: '...' } }
        stage('SonarQube Analysis'){ steps { /* SonarQube scanner */ } }
        stage('npm Build')         { steps { sh 'npm install && npm run build' } }
        stage('Trivy FS Scan')     { steps { sh 'trivy fs --severity HIGH,CRITICAL .' } }
        stage('Docker Build')      { steps { sh 'docker build -t prime-clone .' } }
        stage('Trivy Image Scan')  { steps { sh 'trivy image prime-clone' } }
        stage('Push to ECR')       { steps { /* ECR login & push */ } }
        stage('Deploy via ArgoCD') { steps { /* Update Helm values, ArgoCD syncs */ } }
    }

    post {
        success { echo 'βœ… Pipeline completed successfully!' }
        failure { echo '❌ Pipeline failed. Check logs.' }
    }
}

6. GitOps Deployment (ArgoCD)

# Install ArgoCD on EKS
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

# Access ArgoCD UI
kubectl port-forward svc/argocd-server -n argocd 8080:443

# Get initial admin password
kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d

# Create ArgoCD Application (points to Helm chart in this repo)
argocd app create prime-clone \
  --repo https://github.com/<your-username>/full-stack-devops-pipeline.git \
  --path helm/ \
  --dest-server https://kubernetes.default.svc \
  --dest-namespace production \
  --sync-policy automated

7. Monitoring (Prometheus & Grafana)

# Install Prometheus & Grafana via Helm
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

helm install monitoring prometheus-community/kube-prometheus-stack \
  --namespace monitoring \
  --create-namespace

# Access Grafana dashboard
kubectl port-forward svc/monitoring-grafana -n monitoring 3000:80
# Default credentials β€” admin / prom-operator

πŸ”’ Security

This pipeline implements security at every stage:

Layer Tool/Practice Description
Code Quality SonarQube Detects bugs, code smells & vulnerabilities
Dependency Scan Trivy (filesystem mode) Scans node_modules for known CVEs
Image Scan Trivy (image mode) Scans built Docker image for OS/library vulns
Registry AWS ECR (private) Images stored in private, encrypted registry
Secrets Management Jenkins Credentials Store No hardcoded secrets in code
Network AWS VPC + Security Groups Isolated network with least-privilege access
Cluster EKS with RBAC Role-based access control for Kubernetes

πŸ“Š Monitoring & Observability

Component Purpose Access
Prometheus Metrics collection, alerting rules http://<cluster-ip>:9090
Grafana Visualization dashboards http://<cluster-ip>:3000
ArgoCD UI Deployment status & sync health http://<cluster-ip>:8080
Jenkins Pipeline logs & build history http://<jenkins-ip>:8080

Key Metrics Monitored

  • 🟒 Application β€” Request latency, error rates, throughput
  • πŸ”΅ Kubernetes β€” Pod health, resource utilization, node status
  • 🟠 Infrastructure β€” CPU, memory, disk, network I/O
  • πŸ”΄ Alerts β€” Pipeline failures, pod crashes, high error rates

🀝 Contributing

Contributions are welcome! Please follow these steps:

  1. Fork the repository
  2. Create a feature branch: git checkout -b feature/amazing-feature
  3. Commit your changes: git commit -m 'Add amazing feature'
  4. Push to the branch: git push origin feature/amazing-feature
  5. Open a Pull Request

Commit Convention

feat: add new feature
fix: bug fix
docs: documentation update
ci: CI/CD pipeline changes
infra: infrastructure changes
refactor: code refactoring
test: add or update tests

πŸ“„ License

This project is licensed under the MIT License β€” see the LICENSE file for details.


πŸ™ Acknowledgments


⭐ Star this repo if you find it helpful!

Built with ❀️ using modern DevOps practices

About

This project implements a highly resilient, production-grade DevOps ecosystem designed for high availability and security. The architecture transitions from a traditional CI/CD model to a modern GitOps approach, utilizing ArgoCD to synchronize the state of the AWS EKS cluster with the source code. By integrating Aqua Trivy and SonarQube, the pipeli

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors