How to Improve CI/CD Pipelines for Multi-Tenant Environments with Capsule and Jenkins

Introduction

In modern DevOps practices, Continuous Integration and Continuous Deployment (CI/CD) pipelines are essential for delivering software quickly and reliably. However, managing CI/CD pipelines in a multi-tenant Kubernetes environment introduces new challenges:

• Isolating pipelines for different tenants
• Managing resource allocation across tenants
• Securing access to secrets, logs, and deployment environments

Capsule solves the multi-tenancy problem by allowing you to create and manage tenant-specific namespaces with strict resource and access control.
Jenkins, a widely used CI/CD tool, can be configured to run jobs in a multi-tenant Kubernetes cluster managed by Capsule.

This blog will guide you through setting up Jenkins in a multi-tenant Kubernetes environment with Capsule to create isolated and secure CI/CD pipelines.

---

Why Capsule and Jenkins Work Well Together

✅ Capsule for Multi-Tenancy:

• Creates tenant-specific namespaces in Kubernetes.
• Enforces resource quotas and RBAC for tenants.
• Ensures secure isolation between tenants.

✅ Jenkins for CI/CD:

• Automates build, test, and deployment processes.
• Natively supports Kubernetes for dynamic agent provisioning.
• Can be configured to use tenant-specific permissions and resources.

---

Step 1: Install Capsule and Jenkins

Install Capsule

Add the Capsule Helm repository and install it:

helm repo add clastix https://clastix.github.io/charts
helm repo update
helm install capsule clastix/capsule

Verify that Capsule is running:

kubectl get pods -n capsule-system

Install Jenkins

Install Jenkins using Helm:

helm repo add jenkins https://charts.jenkins.io
helm repo update
helm install jenkins jenkins/jenkins

Retrieve the Jenkins admin password:

kubectl get secret --namespace default jenkins -o jsonpath="{.data.jenkins-admin-password}" | base64 --decode

Access Jenkins UI:

kubectl port-forward svc/jenkins 8080:8080

Open a browser and navigate to:

http://localhost:8080

---

Step 2: Create a Capsule Tenant for Jenkins Pipelines

Define a tenant in Capsule to isolate Jenkins pipelines for different teams:

tenant.yaml:

apiVersion: capsule.clastix.io/v1beta1
kind: Tenant
metadata:
  name: ci-team
spec:
  owner:
    kind: User
    name: ci-user
  namespaceOptions:
    quota:
      hard:
        pods: "20"
        requests.cpu: "8"
        requests.memory: "16Gi"
        limits.cpu: "16"
        limits.memory: "32Gi"

Apply the tenant definition:

kubectl apply -f tenant.yaml

Create a namespace for the Jenkins agent pods under the tenant:

kubectl create namespace ci-team-pipelines
kubectl label namespace ci-team-pipelines capsule.clastix.io/tenant=ci-team

---

Step 3: Configure Jenkins to Use Capsule Namespaces

Install Kubernetes Plugin in Jenkins:

1. Go to Jenkins Dashboard → Manage Jenkins → Plugins.
2. Install the Kubernetes Plugin.

Configure Kubernetes Cloud in Jenkins:

1. Go to Manage Jenkins → Configure System → Cloud → Kubernetes.
2. Set the following values:
   • Kubernetes URL: https://kubernetes.default.svc.cluster.local
   • Jenkins URL: http://jenkins.default.svc.cluster.local:8080
   • Namespace: ci-team-pipelines
3. Enable dynamic provisioning of agents:
   • Retention Timeout: 5 minutes
   • Pod Template: Add the following template:

kind: Pod
metadata:
  labels:
    jenkins-agent: "true"
spec:
  containers:
    - name: jnlp
      image: jenkins/inbound-agent
      args:
        - "$(JENKINS_SECRET)"
        - "$(JENKINS_AGENT_NAME)"
      resources:
        requests:
          memory: "256Mi"
          cpu: "500m"
        limits:
          memory: "512Mi"
          cpu: "1"

---

Step 4: Create a CI/CD Pipeline in Jenkins

Example Pipeline:

Create a Jenkins pipeline job using the following Jenkinsfile:

pipeline {
    agent {
        kubernetes {
            cloud 'kubernetes'
            yaml """
apiVersion: v1
kind: Pod
metadata:
  labels:
    app: jenkins-pipeline
spec:
  containers:
    - name: builder
      image: maven:3.8.5-openjdk-17
      command:
        - cat
      tty: true
"""
        }
    }
    environment {
        K8S_NAMESPACE = 'ci-team-pipelines'
    }
    stages {
        stage('Build') {
            steps {
                container('builder') {
                    sh 'mvn clean install'
                }
            }
        }
        stage('Test') {
            steps {
                container('builder') {
                    sh 'mvn test'
                }
            }
        }
        stage('Deploy') {
            steps {
                container('builder') {
                    sh '''
                    kubectl apply -f k8s/deployment.yaml -n $K8S_NAMESPACE
                    '''
                }
            }
        }
    }
}

Explanation:

• Agent: Uses a Kubernetes pod as the build agent.
• Build Stage: Builds the project using Maven.
• Test Stage: Runs unit tests.
• Deploy Stage: Deploys the application to the tenant namespace using kubectl.

---

Step 5: Isolate Jenkins Workloads with Capsule

To ensure that Jenkins builds are restricted to tenant namespaces:

1. Create an RBAC Role and RoleBinding:

role.yaml:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: ci-team-pipelines
  name: jenkins-build-role
rules:
  - apiGroups: [""]
    resources: ["pods", "services", "deployments"]
    verbs: ["create", "get", "list", "watch", "delete"]

rolebinding.yaml:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  namespace: ci-team-pipelines
  name: jenkins-build-rolebinding
subjects:
  - kind: ServiceAccount
    name: jenkins
    namespace: default
roleRef:
  kind: Role
  name: jenkins-build-role
  apiGroup: rbac.authorization.k8s.io

Apply the Role and RoleBinding:

kubectl apply -f role.yaml
kubectl apply -f rolebinding.yaml

---

Step 6: Monitor Build Performance and Logs

Use Jenkins UI to monitor builds:

• ✅ Successful builds should display logs for each stage.
• ✅ Failed builds should output logs for debugging.

Check running Jenkins agent pods:

kubectl get pods -n ci-team-pipelines

Sample output:

NAME                        READY   STATUS    RESTARTS   AGE
jenkins-agent-xyz           1/1     Running   0          2m

---

Real-World Use Cases

1. Enterprise SaaS Providers

• Capsule creates tenant-specific build environments.
• Jenkins automates deployment to production.

2. Multi-Product Development

• Isolate CI/CD pipelines for different products using Capsule.
• Separate logs and build environments for each product.

3. Cloud-Native Startups

• Set up separate CI/CD pipelines for different microservices.
• Capsule ensures secure and efficient resource allocation.

---

Best Practices

1. Use Capsule quotas to prevent resource overuse.
2. Create separate RBAC policies for build agents.
3. Monitor CI/CD performance using Grafana and Prometheus.
4. Store Jenkins secrets securely in Kubernetes.

---

Conclusion

By combining Capsule and Jenkins, you can create a scalable and secure CI/CD pipeline infrastructure for multi-tenant Kubernetes environments. Capsule ensures resource isolation, while Jenkins automates the build, test, and deployment lifecycle—empowering teams to deliver high-quality software faster.

That’s it for now. I hope this article gave you some useful insights on the topic. Please feel free to drop a comment, question or suggestion.