Hello folks ! I hope you all are doing well. So, today I am here with a new and interesting topic for you all. In this blog we’ll see how we can get the Gateway Logs using Logs Explorer in apigee. Apigee API Gateway logs can provide valuable insights into the behaviour and performance of your APIs. 

So, let’s get started !!!

Introduction

As part of the Next Gen API gateway platform migration project, we need to get Apigee API Gateway call logs Using the Logs Explorer for better observability. Apigee API Gateway logs can provide valuable insights into the behaviour and performance of your APIs. This guide will walk you through the process of enabling message logging in shared flows and flow hooks and retrieving those logs using Logs Explorer.

Prerequisites

Before you begin, make sure you have the following:

1. Access to the Apigee Management UI or API
2. Permissions to create or modify shared flows and flow hooks
3. Access to the Cloud Console for the project associated with your Apigee organisation
4. Service Account Access for the role – roles/logging.logWriter for the service account we are using to add the message logging policy..
5. Familiarity with Logs Explorer and querying logs in the Cloud Logging interface

Enabling Message Logging in Shared Flows

Shared Flow:Shared flow is a reusable component which can be created at one place and used by multiple proxies during their execution flow as long as the proxies and shared flow lie in the same environment.

To create the Message-Logging Policy(Sample-Message-Logging-Policy). Go to  Apigee Management UI and navigate to the Shared Flows section as above.

1. Under Develop in Apigee, select the Shared Flows tab.
2. Click Create New, enter the name (Sample-Message-Logging-Policy), and click create.
3. Under the Develop tab, click + under policies.
4. Scroll down, and select Message Logging under the Extension group.
5. Set the name and display name to ML-CloudLogging Policy.
6. From the Script File option, select Import new script.
7. Select Choose File and select the Sample-Message-Logging-Policy.xml file from the decompressed policy package.
8. When complete, select Add (no extra configuration is needed for this policy)
9. In the shared flow, under Shared Flows click default.
10. Click + to add a policy step and choose the existing policy ML-CloudLogging.
11. Once finished, click Deploy to deploy the request shared flow and also save the changes to the shared flow.   

Add the following policy to the policy or step configuration

Message Logging Policy file

Replace the below xml file with the sample cloudlogging.xml in order to the the apigee API gateway call logs.

<? version="1.0" encoding="UTF-8" standalone="yes"?>
<MessageLogging continueOnError="false" enabled="true" name="ML-CloudLogging">
  <DisplayName>ML-CloudLogging</DisplayName>
  <CloudLogging>
    <LogName>projects/{organization.name}/logs/apigee</LogName>
    <Message contentType="application/json">{
        "organization": "{organization.name}",
        "environment": "{environment.name}",
        "proxy": "{apiproxy.name}",
        "verb": "{request.verb}",
        "response.code":"{response.status.code}",
        "response.reason":"{message.reason.phrase}",
        "response.content":"{escapeJSON(message.content)}",
        "literal_value":"Hi! I am a literal value",
        "OrgName": "{organization.name}",
        "Proxy_Name": "{apiproxy.name}",
        "Environment_Name": "{environment.name}",
        "User_Agent": "{request.header.User-Agent.values}",
        "Host": "{request.header.Host}",
        "Proxy_URL": "{proxy.url}",
        "API_Proxy_Revision": "{apiproxy.revision}",
        "Target_IP": "{target.ip}",
        "Target_Host": "{target.host}",
        "Target_URL": "{target.url}",
        "Target_Endpoint_Name": " {route.target}",
        "Proxy_Base_Path": "{proxy.basepath}",
        "Proxy_IP": "{proxy.client.ip}",
    }</Message>
  </CloudLogging>
</MessageLogging>

you can add the variables you want to see the logs for Click Here for the variables list.

Adding Shared Flows to Flow Hooks

Flow Hooks:  “Flow Hooks” are pre-defined points within the request/response flow of an API proxy where you can inject custom policies or logic. These hooks allow you to extend the functionality of the API proxy and perform additional processing or modifications.

1. In the Apigee Management UI, under Environment, select the Flow Hooks.
2. choose “PreFlow” or “PostFlow” depending on where you want to add the shared flow.
3. Click on the “Edit” button to add a Shared Flow in which your policy is attached.
4. Select “Shared Flow” from the drop-down menu and choose the shared flow you modified in the previous section.
5. Save the changes to the API Proxy.

Retrieving Apigee API Gateway Logs using Logs Explorer

• Go to the Cloud Console for the project associated with your Apigee organisation.
• Navigate to the Logs Explorer page.
• Use the following filter in the query editor to retrieve the Apigee API Gateway logs:

logName="projects/[PROJECT_ID]/logs/apigee"

Replace [PROJECT_ID] with your actual Cloud project ID.

• Optionally, you can further refine the query using additional filters like time range, severity level, or specific APIs.
• Click on the “Run Query” button to execute the query and retrieve the logs.

You will see the output as below:

So this is how you can get the Apigee API Gateway Logs using the Logs Explorer.

Conclusion

So, in this blog you have seen How to get Apigee API Gateway Logs using Logs Explorer. By following the steps outlined in this guide, you can enable message logging in shared flows and flow hooks in Apigee API Gateway and retrieve the logs using Logs Explorer. Analysing these logs can help you gain insights into the behaviour and performance of your APIs, enabling you to identify and address any issues effectively. For more information feel free to contact me.

Keep Learning !!!

Keep Growing !!!

Thank You 🙂

Reference

https://github.com/GoogleCloudPlatform/apigee-samples/tree/main/cloud-logging