CI/CD for Secure Microservices with Spring Boot, Kafka, and gRPC on AWS/GCP/Azure

Microservices are now the industry standard for building scalable and stand-alone services. However, designing the microservices is not enough; deploying them safely and carefully is also an important part. Without proper automation and security, a team can face problems like manual deployment errors, a lack of consistency across different environments, long release cycles, and security risks. These things can occur due to exposed secrets or untested code.

In this blog, we will explore how to solve these challenges by creating a CI/CD pipeline for secure microservices with Spring Boot, Kafka, and gRPC on AWS/GCP/Azure. We will look at how to deploy these services on cloud platforms. This will assure the automated service delivery from code to production.

Below are some real-world use cases where Spring Boot, Kafka, and gRPC are deployed with CI/CD on cloud platforms:

Problem: Users want real-time updates on their orders from placement to delivery without any delay or data inconsistency.
Solution: Spring Boot REST handles order requests. While Kafka ensures real-time event flow across different services, gRPC enables fast communication.
CI/CD Benefit: This enables automated testing and quick deployment of new order features without downtime.

Problem: The bank processes a huge number of transactions securely, detects fraud instantly, and alerts users without delay.
Solution: Spring Boot microservices process transactions and detect fraud. Streaming the data using Kafka and gRPC ensures fast service-to-service communication.
CI/CD Benefit: Regular security patches and fast rollouts of compliance updates using automated pipelines.

Problem: Hospitals need an efficient way to manage different appointments and keep patients informed.
Solution: A booking service schedules appointments and publishes events through Kafka to inform doctors and patients. gRPC fetches required data instantly.
CI/CD Benefit: Seamless integration of new services (e.g., video consultation) and secure rollouts via pipelines.

Security is non-negotiable when different microservices communicate with each other over the network. Here’s how we can protect the microservices using Spring Boot, Kafka, and gRPC.

• For security, we can use OAuth2 and JWT for authentication and authorisation.
• Enable TLS encryption in REST and gRPC services.
• We can use SASL/TLS and ACLs for securing Kafka.
• Cloud secret managers store sensitive data like DB credentials.
• Trivy and OWASP: These tools can be used for scanning the code and containers.

name: CI/CD Pipeline

on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]

env:
  AWS_REGION: ap-south-1
  IMAGE_NAME: order-service
  CLUSTER_NAME: cluster-c1
  K8S_NAMESPACE: default

As we know, the CI/CD pipeline, or we can say the GitHub Actions pipeline, is initiated on every push or pull request to the main branch. For reuse across all jobs, it sets global environment variables like CLUSTER_NAME, AWS_REGION, IMAGE_NAME, and K8S_NAMESPACE. Configuring AWS and Kubernetes deployment settings is simplified by these variables.

jobs:
  build-and-push:
    name: Build & Push Docker Image
    runs-on: ubuntu-latest

    steps:
      - name: Checkout repo
        uses: actions/checkout@v4

      - name: Set up JDK
        uses: actions/setup-java@v4
        with:
          java-version: '21'

      - name: Build with Maven
        run: mvn clean package -DskipTests

      - name: Login to DockerHub
        run: echo "${{ secrets.DOCKER_PASSWORD }}" | docker login -u ${{ secrets.DOCKER_USERNAME }} --password-stdin

      - name: Build Docker image
        run: docker build -t ${{ secrets.DOCKER_USERNAME }}/${{ env.IMAGE_NAME }}:latest .

      - name: Push Docker image
        run: docker push ${{ secrets.DOCKER_USERNAME }}/${{ env.IMAGE_NAME }}:latest

In this part of the yml, the job builds the Spring Boot project using mvn clean package. It authenticates to DockerHub using stored secrets, then builds and tags the Docker image as latest. Finally, it pushes the image to the configured DockerHub repository.

  deploy:
    name: Deploy to K8s
    needs: build-and-push
    runs-on: ubuntu-latest

    steps:
      - name: Checkout repo
        uses: actions/checkout@v4

      - name: Setup kubectl
        uses: azure/setup-kubectl@v3
        with:
          version: 'latest'

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v2
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_KEY }}
          aws-region: ${{ env.AWS_REGION }}

      - name: Update kubeconfig for EKS
        run: aws eks update-kubeconfig --name ${{ env.CLUSTER_NAME }} --region ${{ env.AWS_REGION }}

      - name: Deploy to Kubernetes
        run: |
          kubectl apply -f k8s/
          kubectl rollout restart deployment/${{ env.IMAGE_NAME }} -n ${{ env.K8S_NAMESPACE }}

This current job depends on the last job using needs to build and push the image. It connects to the EKS cluster, sets up kubectl, and configures AWS credentials from secrets. Lastly, it restarts the deployment using the modified image and applies the Kubernetes manifests from the k8s/ directory.

SPRING_KAFKA_BOOTSTRAP_SERVERS=your-kafka-broker:9092
GRPC_SERVER_PORT=9090
DB_USERNAME=admin
DB_PASSWORD=secret

These variables are used for the configuration part of the application. It gives the Kafka broker address and the server port for the gRPC server. The username and password variable is required to get connected to the database.

server:
  port: 8080
  ssl:
    enabled: true
    key-store: classpath:keystore.p12
    key-store-password: ${KEYSTORE_PASSWORD}
    key-store-type: PKCS12

We all know this configuration part sets up the Spring Boot application server to run on port 8080. It uses an SSL certificate stored in a keystore.p12 file with the keystore type as PKCS12. The password for the keystore is securely injected using the KEYSTORE_PASSWORD environment variable. This procedure ensures secure communication over SSL.

spring:
  kafka:
    bootstrap-servers: ${SPRING_KAFKA_BOOTSTRAP_SERVERS}
    consumer:
      group-id: order-service
    security:
      protocol: SSL

We can configure the properties of Apache Kafka for the Spring Boot application using this configuration. It establishes a connection with the Kafka broker and assigns a consumer group ID with the help of this variable, “SPRING_KAFKA_BOOTSTRAP_SERVERS.” It also enables the SSL to ensure secure communication with the Kafka server.

Faster Releases: CI/CD helps in faster releases from automatically building, testing, and deploying code within minutes.

Fewer Errors: Automation testing reduces manual errors and boosts overall code quality and reliability during the CI stage. It also helps to detect the defects and flaws early in the pipeline.

Better Security: Secrets are managed securely using encrypted cloud vaults, and code/images are scanned automatically. This security ensures flaws are caught quickly, and the data stays safe and protected.

Scalability: Cloud-native deployments provide an easy service to scale beyond geographical boundaries. Consistently rolling out updates across any number of clusters is made easy using CI/CD.

As per the above blog, we can say, in today’s fast-paced development landscape, microservices succeed not just on how well they are constructed but also on how safely and swiftly they are deployed. You can reduce the risks involved in manual deployments and speed up release cycles. With this, your services are always production-ready by integrating CI/CD pipelines with Spring Boot, Kafka, and gRPC on AWS, GCP, or Azure.

The CI for secure microservices using Spring Boot, Kafka, and gRPC on AWS, Azure, and GCP not only increases developer productivity but also ensures system robustness and scalability.

https://medium.com/@bubu.tripathy/building-a-ci-cd-pipeline-for-a-spring-boot-application-763a2dec1ac4