A quick guide for integrating Pub-Sub in GCP.

Introduction

In today’s fast-paced world, real-time data processing and event-driven architectures have become the norm for many modern applications. Google Cloud Platform (GCP) offers a powerful and scalable solution for implementing publish-subscribe (Pub/Sub) messaging systems, enabling you to build robust and efficient event-driven applications. In this blog post, we will explore the key components of GCP’s Pub/Sub service and walk through the steps to create a Pub/Sub system.

What is Publish-Subscribe (Pub/Sub)?

Publish-Subscribe, often abbreviated as Pub/Sub, is a messaging pattern where senders (publishers) of messages do not need to know who or what will receive the messages (subscribers). Instead, messages are categorized into topics, and subscribers express interest in receiving messages from specific topics. This decoupling of publishers and subscribers makes it a versatile pattern for building event-driven systems.

Key Components of Google Cloud Pub/Sub

Before diving into creating a Pub/Sub system on GCP, it’s essential to understand the core components involved:

1. Topics: Topics are the channels through which messages are sent by publishers. You create topics to categorize and organize your messages.
2. Subscriptions: Subscriptions represent the interest of subscribers in receiving messages from specific topics. Each subscription is associated with a single topic.
3. Publishers: Publishers are responsible for sending messages to topics. These can be applications, services, or devices that generate events.
4. Subscribers: Subscribers are the entities that consume messages from subscriptions. They process and respond to the messages they receive.
5. Message: A message is the data payload sent by a publisher to a topic. It can be any structured data, such as JSON, XML, or plain text.
6. Acknowledgments: Subscribers acknowledge the receipt of messages, indicating that they have successfully processed the message.

Now, let’s dive into the steps to create a Pub/Sub system on Google Cloud Platform:

1: Set Up a Google Cloud Project

1. **Create a Google Cloud Platform (GCP) account if you don’t have one already.
2. Create a new GCP project or use an existing one for your Pub/Sub system.

2: Enable the Pub/Sub API

1. Go to the GCP Console and select your project.
2. Navigate to APIs & Services > Dashboard.
3. Click on Enable APIs and Services.
4. Search for “Cloud Pub/Sub API” and enable it for your project.

3: Create a Pub/Sub Topic

1. Navigate to Cloud Pub/Sub from the GCP Console.
2. Click on Create a Topic.
3. Give your topic a unique name and click Create.

4: Create a Subscription

1. After creating a topic, click on it to open its details.
2. Under “Subscriptions,” click on Create Subscription.
3. Provide a name for your subscription.
4. Configure other settings as needed (e.g., delivery type, retention duration).
5. Click Create.

5: Publish Messages

1. To publish a message to your topic, go to the topic’s details page.
2. Click on Publish Message.
3. Enter the message data and any attributes.
4. Click Publish.

6: Create a Subscriber

1. Create an application or service that subscribes to the subscription you created.
2. Use the Google Cloud Pub/Sub client libraries to connect to the subscription and process messages.

7: Process Messages

1. Implement the logic in your subscriber to process the messages it receives.
2. Acknowledge the message after processing to remove it from the subscription.

To create a Java application for publishing and subscribing to Google Cloud Pub/Sub, you’ll need to use the Google Cloud Client Libraries for Java. Below, I’ll provide you with a basic example of how to set up a Pub/Sub publisher and subscriber in Java.

Note: Before you start, make sure you have set up a GCP project and enabled the Pub/Sub API.

Java Code for Pub Sub

1. Set Up Your Project Dependencies

You need to include the Google Cloud Pub/Sub Java Client Library in your project’s dependencies. You can do this using Maven:

<dependency>
    <groupId>com.google.cloud</groupId>
    <artifactId>google-cloud-pubsub</artifactId>
    <version>2.0.1</version> <!-- Use the latest version -->
</dependency>

2. Create a Pub/Sub Publisher

import com.google.api.gax.core.FixedCredentialsProvider;
import com.google.auth.oauth2.GoogleCredentials;
import com.google.cloud.pubsub.v1.Publisher;
import com.google.pubsub.v1.ProjectTopicName;
import com.google.pubsub.v1.PubsubMessage;

public class PubSubPublisher {
    public static void main(String[] args) throws Exception {
        // Replace with your GCP project ID and Pub/Sub topic name
        String projectId = "your-project-id";
        String topicId = "your-topic-id";
        
        // Initialize Google Cloud credentials
        GoogleCredentials credentials = GoogleCredentials.getApplicationDefault();
        
        // Create a publisher
        ProjectTopicName topicName = ProjectTopicName.of(projectId, topicId);
        Publisher publisher = Publisher.newBuilder(topicName)
                .setCredentialsProvider(FixedCredentialsProvider.create(credentials))
                .build();
        
        // Publish a message
        String message = "Hello, Pub/Sub!";
        PubsubMessage pubsubMessage = PubsubMessage.newBuilder()
                .setData(message.getBytes("UTF-8"))
                .build();
        
        publisher.publish(pubsubMessage);
        
        // Shutdown the publisher when done
        publisher.shutdown();
    }
}

3. Create a Pub/Sub Subscriber

import com.google.api.gax.core.FixedCredentialsProvider;
import com.google.auth.oauth2.GoogleCredentials;
import com.google.cloud.pubsub.v1.*;
import com.google.pubsub.v1.ProjectSubscriptionName;
import com.google.pubsub.v1.PubsubMessage;
import com.google.pubsub.v1.Subscriber;
import com.google.pubsub.v1.SubscriptionName;
import com.google.pubsub.v1.MessageReceiver;
import com.google.pubsub.v1.MessageReceiverException;
import java.io.IOException;

public class PubSubSubscriber {
    public static void main(String[] args) throws Exception {
        // Replace with your GCP project ID and Pub/Sub subscription name
        String projectId = "your-project-id";
        String subscriptionId = "your-subscription-id";
        
        // Initialize Google Cloud credentials
        GoogleCredentials credentials = GoogleCredentials.getApplicationDefault();
        
        // Create a subscriber
        SubscriptionName subscriptionName = SubscriptionName.of(projectId, subscriptionId);
        MessageReceiver messageReceiver = (message, consumer) -> {
            System.out.println("Received message: " + message.getData().toStringUtf8());
            consumer.ack();
        };
        
        Subscriber subscriber = Subscriber.newBuilder(subscriptionName, messageReceiver)
                .setCredentialsProvider(FixedCredentialsProvider.create(credentials))
                .build();
        
        // Start the subscriber
        subscriber.startAsync().awaitRunning();
        
        // Allow the subscriber to run indefinitely
        Thread.sleep(Long.MAX_VALUE);
    }
}

In these examples, replace "your-project-id", "your-topic-id", and "your-subscription-id" with your actual GCP project ID, Pub/Sub topic name, and subscription name. These Java programs use the Google Cloud Pub/Sub client library to publish and subscribe to messages in a Pub/Sub topic.

Make sure to handle exceptions and errors appropriately in a production environment. This code provides a basic starting point for working with Google Cloud Pub/Sub in Java.

Conclusion

Google Cloud Pub/Sub provides a reliable and scalable platform for building event-driven and real-time applications. By following the steps outlined in this blog post, you can create a Pub/Sub system on Google Cloud Platform and leverage the power of publish-subscribe messaging to build efficient, decoupled, and responsive applications. Pub/Sub’s flexibility and integration with other GCP services make it a valuable tool for modern cloud-native architectures.

Reference link :- https://cloud.google.com/pubsub/docs/overview