How to store Python log inside a file

Introduction

As we all know, logging is a crucial part of running applications in a production environment. Logs help us quickly identify the root causes of issues, errors, and events. You can think of logs as evidence of what is happening inside your system. Managing and storing these logs becomes very important when deploying your application in production. In today’s blog, we’ll explore how to store Python log inside a file. So, without further delay, let’s get started.

What are the use cases of storing logs inside a file

Storing logs in a file has various use cases. For instance, you can replicate logs to another file, back up the log file, and store it in different locations. From a DevOps perspective, you can attach volumes when running applications inside containers or Kubernetes. This way, even if the container or pod is recreated, your logs remain safe and intact.

Logging Module in Python

Python provides build in module for logs which is logging module that can easily integrate with our application, this module provides various methods that we can use to generate logs.

Storing logs inside a file

So to store logs inside a file we need to provide file configuration details for logging inside logging.basicConfig() method. This method bascConfig() takes various arguments some important arguments are filename, filemode, level etc.

Note: By default filemode parameter is set to ‘a’ which stands for appends that means when we get a new log then that log will get append in the file with the previous content. I will explain more about that with an example below.

Example

In the below example we will make an API call and generate logs based on connection. Create a file named as myapp.py and paste below code

# myapp.py
import logging
import requests

logger = logging.getLogger(__name__)

def logdemo():
    logging.basicConfig(filename='myapp.log', level=logging.INFO)
    
    try:
        logger.info('Going to make an API call')
        x = requests.get('https://w3schools.com/python/demopage.htm')
        print(x.text)
        logger.info('Request Successfull')
    except Exception as e:
        logger.error('Some error occured')

if __name__ == '__main__':
    logdemo()

Now run this program

$ python3 myapp.py 

After running the above command you will notice a log file gets created automatically and when you look into that you will find these content

# myapp.log
INFO:__main__:Going to make an API call
INFO:__main__:Request Successfull       

As you can see, we get our logs inside our file now try to rerun that program again, what you will notice is that the file myapp.log content is appending with new logs

# myapp.log
INFO:__main__:Going to make an API call
INFO:__main__:Request Successfull    
INFO:__main__:Going to make an API call
INFO:__main__:Request Successfull    

But what if we want to change this behavior and store only the new logs each time the application restarts? To achieve this, you can use an additional parameter, filemode, in the basicConfig() function. Setting filemode to ‘w’ will ensure that the log file is overwritten each time the application starts, rather than appending new logs to the existing file.

Conclusion

Logging not only helps to track events within our application but also makes it easier to identify problems quickly and efficiently. That concludes this blog on how to store python log inside a file.

Reference: https://docs.python.org/3/howto/logging.html