How to Implement Custom Admission Policies with Capsule

Kubernetes admission policies control how and what objects can be created or modified within a cluster. In a multi-tenant environment, it’s crucial to have granular control over tenant permissions and resource configurations. Capsule enhances Kubernetes’ built-in admission control by allowing you to define custom admission policies at the tenant level.

In this blog, we’ll explore how to implement custom admission policies using Capsule, with real-world examples and best practices.

---

🚀 Why Custom Admission Policies Matter in Multi-Tenancy

In a multi-tenant Kubernetes cluster, different tenants may have varying levels of trust, requirements, and constraints. Kubernetes’ built-in admission controllers (like LimitRange, NamespaceLifecycle, and ResourceQuota) are useful but not flexible enough for complex multi-tenant scenarios.

✅ Custom Admission Policies enable you to:

1. Restrict the creation of certain types of objects.
2. Define security-related restrictions (e.g., block privileged containers).
3. Control naming conventions, labels, and annotations.
4. Ensure you enforce certain policies consistently across all tenants.

🔥 Example Use Cases:

• Prevent tenants from using the hostNetwork setting.
• Restrict the use of specific Kubernetes versions or images.
• Enforce container resource limits and requests.
• Ensure you define certain environment variables in each pod.

---

🏗️ How Capsule Handles Admission Policies

Capsule implements admission policies using MutatingAdmissionWebhook and ValidatingAdmissionWebhook Kubernetes objects. These webhooks allow Capsule to intercept and validate API requests before objects are created or updated.

Capsule manages two types of policies:

1. Validation Policies – Block resource creation or updates if they don’t match the defined rules.
2. Mutation Policies – Automatically modify or inject missing attributes into objects.

Capsule’s webhook-based architecture ensures that policies are enforced consistently at the tenant level, without modifying native Kubernetes behavior.

---

🔎 Step-by-Step Guide: Creating Custom Admission Policies with Capsule

1. Install Capsule (If Not Already Installed)

If you haven’t installed Capsule, install it using Helm:

helm repo add clastix https://clastix.github.io/charts
helm install capsule clastix/capsule

Verify the installation:

kubectl get pods -n capsule-system

---

2. Create a Tenant

Let’s create a tenant named team-b:

apiVersion: capsule.clastix.io/v1beta1
kind: Tenant
metadata:
  name: team-b
spec:
  owners:
    - kind: User
      name: "developer@example.com"
  namespaceQuota: 5

Apply the tenant:

kubectl apply -f tenant.yaml

---

3. Define a Custom Admission Policy

Let’s create a policy to prevent privileged containers within the tenant. This is a common security measure to prevent container breakout attacks.

Create a ClusterPolicy:

apiVersion: capsule.clastix.io/v1beta1
kind: ClusterPolicy
metadata:
  name: deny-privileged-containers
spec:
  failurePolicy: Fail
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]
  rules:
    - operation: CREATE
      context: |
        request.object.spec.containers[*].securityContext.privileged == true
      message: "Privileged containers are not allowed within this tenant."

Apply the policy:

kubectl apply -f deny-privileged-containers.yaml

✅ What this does:

• The policy intercepts CREATE operations on Pod objects.
• If any container attempts to set privileged: true, the request is denied.
• The failure message ("Privileged containers are not allowed within this tenant.") is returned to the user.

---

4. Test the Admission Policy

Let’s try to create a privileged pod within the tenant namespace:

apiVersion: v1
kind: Pod
metadata:
  name: privileged-pod
  namespace: team-b
spec:
  containers:
    - name: test-container
      image: nginx
      securityContext:
        privileged: true

Apply the pod:

kubectl apply -f privileged-pod.yaml

You should get an error like:

Error from server (Privileged containers are not allowed within this tenant.)

✅ Why?

• The policy defined by Capsule intercepted the request.
• Since the container requested privileged: true, the admission webhook rejected it.

---

5. Add Multiple Rules in a Single Policy

You can define multiple rules within the same policy. Let’s create a policy that:

• Prevents privileged containers.
• Enforces a naming convention for pods (team-b-*).

Create a combined policy:

apiVersion: capsule.clastix.io/v1beta1
kind: ClusterPolicy
metadata:
  name: combined-policy
spec:
  failurePolicy: Fail
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]
  rules:
    - operation: CREATE
      context: |
        request.object.spec.containers[*].securityContext.privileged == true
      message: "Privileged containers are not allowed within this tenant."
    - operation: CREATE
      context: |
        !request.object.metadata.name.startsWith("team-b-")
      message: "Pod names must start with 'team-b-'"

Apply the policy:

kubectl apply -f combined-policy.yaml

✅ What this does:

• Blocks privileged containers.
• Enforces a naming convention on pods within the tenant namespace.

---

6. Test the Combined Policy

Now try creating a pod with an invalid name:

apiVersion: v1
kind: Pod
metadata:
  name: invalid-name
  namespace: team-b
spec:
  containers:
    - name: test-container
      image: nginx

Apply the pod:

kubectl apply -f invalid-name-pod.yaml

You should get:

Error from server (Pod names must start with 'team-b-')

---

🚨 Troubleshooting Common Issues

1. Policy Not Applied

• Ensure the policy is correctly applied to the tenant using:

kubectl get clusterpolicy

• Check the logs of the Capsule webhook pod:

kubectl logs -n capsule-system -l app=capsule-webhook

2. Conflict Between Policies

• If multiple policies conflict, Kubernetes will process them in the order it defines them.
• Combine related rules into a single policy to avoid conflicts.

3. Pod Still Being Created Despite Policy

• Ensure that the failurePolicy is set to Fail instead of Ignore.
• Ensure the webhook is active and running.

---

🏆 Best Practices for Admission Policies

1. Start with minimal policies and expand gradually.
2. Keep policies as granular as possible.
3. Group similar rules into a single policy for better organization.
4. Use Capsule logging to monitor policy enforcement.
5. Regularly test policies to avoid conflicts with Kubernetes updates.

---

🌍 Real-World Use Case

A SaaS company uses Capsule to manage multiple tenants in a single Kubernetes cluster:

• Each tenant has strict security requirements.
• Capsule policies prevent privileged containers and enforce secure labels.
• Tenant namespaces are monitored, and violations trigger alerts.
• Automation ensures that policies are updated in real time.

This setup ensures secure and consistent multi-tenant environments with minimal administrative effort.

---

🎯 Conclusion

Custom admission policies with Capsule allow you to define and enforce complex tenant-level rules, improving security and consistency across your Kubernetes cluster. Capsule’s webhook-based architecture makes it easy to implement, manage, and extend these policies as your multi-tenant environment evolves.

That’s it for now. I hope this article gave you some useful insights on the topic. Please feel free to drop a comment, question or suggestion.