Python Logging: A Guide

You’ve just built an application and have noticed something is not working as intended. The feature you just implemented is behaving in a way you did not expect. What should you do? How are you going to address this problem?

In software development, programmers rely on logging to track events while their software is running. This helps them effectively track down the source of problems should they arise.

In this guide, we’re going to talk about why you should keep application logs and how you can use the Python logging module to keep track of events in your programs.

Why You Should Keep Logs

Keeping logs helps you write more maintainable code. When you use a module like logging, you can keep an accurate record of all the events that occur in your programs. This means you can see which lines of code have run and which failed to execute.

In Python, developers often rely on the print() statement to log their code. It’s a practice everyone – from beginners to experts – is guilty of using and with good reason. A print() statement is easy to use; it’s simple.

However, the print() statement is not the best way to keep logs of your code. For one thing, the print() statement can be used to print any output to the console. This means it can be confusing to distinguish which output is to be stored as a log and which output is part of your main program. What’s more, the print() statement doesn’t save your logs by default.

That’s where the Python logging module comes in handy. Using the logging system module, you can keep more accurate records of what events have run in your code. This will help you debug your code more effectively and fix errors.

How to Use the Python Logging Module

In this tutorial, we’re going to be adding basic logging messages to a program that goes through a list of student grades and calculates whether they have failed or passed their exam. Consider the following code:

students = ["Martin", "Lewis", "Terri", "Bart"]
grades = [92, 78, 72, 38]
pass_fail = []

for i in range(0, len(students)):
	print("Calculating grade for {}".format(students[i]))
	if grades[i] > 55:
		pass_fail.append(True)
		print("{} has passed their exam.".format(students[i)))
	else:
		pass_fail.append(False)
		print("{} has failed their exam.".format(students[i)))

print("Student grades have been calculated. Moving on to send emails to parents of students who have failed their exams.")

The code above calculates whether each of five students in a fourth grade class have passed their exam. In our program, we declare three lists. One list stores the name of each student; another list stores the grade each student earned; the final list stores whether a student has passed or failed their exam.

When we run our program, the following is returned:

Calculating grade for Martin.
Martin has passed their exam.
Calculating grade for Lewis.
Lewis has passed their exam.
Calculating grades for Terri.
Terri has passed their exam.
Calculating grades for Bart.
Bart has failed their exam.

The print() statement shows our code is working, but we could use the logging() module to show this data instead. This will allow us to distinguish our debugging messages from the output of our program.

To start, let’s add the logging standard library at the top of our program:

import logging

Now that we’ve imported the logging library, we can start to keep track of logs in our code. For this example, we are going to keep track of logs using the DEBUG configuration. This is because we are not interested in tracking warnings at this time. For reference, here are the logging options you can specify:

• CRITICAL: Used to display serious errors (logging.critical())
• ERROR: Used to show a problem (logging.error())
• WARNING: Used to show unexpected behavior (logging.warning())
• INFO: Used to show a program is working (logging.info())
• DEBUG: Used to debug code (logging.debug())

The default configuration for the logging library is WARNING, so we’ll have to reset it using this code:

import logging

logging.basicConfig(level=logging.DEBUG)

Now we’re ready to start debugging our code. Let’s replace our print() statements that tell us that a grade is being calculated or that our program has finished with debugging statements:

import logging

logging.basicConfig(level=logging.DEBUG)

students = ["Martin", "Lewis", "Terri", "Bart"]
grades = [92, 78, 72, 38]
pass_fail = []

for i in range(0, len(students)):
	logging.debug("Calculating grade for {}".format(students[i]))
	if grades[i] > 55:
		pass_fail.append(True)
		print("{} has passed their exam.".format(students[i]))
	else:
		pass_fail.append(False)
		print("{} has failed their exam.".format(students[i]))

logging.debug("Student grades have been calculated. Moving on to send emails to parents of students who have failed their exams.")

When we run our code, the following is returned:

DEBUG:root:Calculating grade for Martin
Martin has passed their exam.
DEBUG:root:Calculating grade for Lewis
Lewis has passed their exam.
DEBUG:root:Calculating grade for Terri
Terri has passed their exam.
DEBUG:root:Calculating grade for Bart
Bart has failed their exam.
DEBUG:root:Student grades have been calculated. Moving on to send emails to parents of students who have failed their exams.

You can see that our output contains the same content as earlier. However, the message “Calculating grade for…” and the message informing us that the grades have been calculated appear differently. The text “DEBUG:root:” appears before those statements.

This allows us to keep track of what our program is doing at any given time. Because the logging events module adds the aforementioned text to our program, it is easy to figure out what text has been outputted by our program and what text is for debugging.

In this case, the message “[student]” has failed their exam” is crucial information that tells us whether each student has passed or failed their exam. Everything else is useful to see how our program is running, but is not useful to the user. Hence, we track these statements using logging calls.

How to Keep Logs in a File

The logging module allows you to keep track of your logs in a file. This is useful because it means you won’t lose your logs once you close your Python shell. You’ll have a permanent record of how your program executed on a particular occasion.

All you need to do is include a filename argument in your logging handler configuration line and your program will automatically save logs to a file:

import logging

logging.basicConfig(level=logging.DEBUG, filename="student_data.log")

…

When we run our code, our logs will be added to the filename we specify. If we open up the “student_data.log” file, the following is returned:

DEBUG:root:Calculating grade for Martin
DEBUG:root:Calculating grade for Lewis
DEBUG:root:Calculating grade for Terri
DEBUG:root:Calculating grade for Bart
DEBUG:root:Student grades have been calculated. Moving on to send emails to parents of students who have failed their exams.

Notice that our debug output only contains the messages we have specified as logs using the logging.debug() method. This is useful because it helps us distinguish program output – which is denoted using a print() statement – from debug logs.

You can also add a parameter to keep track of when an entry has been added to your log. We can do so using the following code:

logging.basicConfig(
level=logging.DEBUG,
filename="student_data.log",
format="%(asctime)s:%(levelname)s:%(message)s"
)

This code adds the following to our student_data.log file:

2020-06-18 08:27:50,123:DEBUG:Calculating grade for Martin
2020-06-18 08:27:50,123:DEBUG:Calculating grade for Lewis
2020-06-18 08:27:50,123:DEBUG:Calculating grade for Terri
2020-06-18 08:27:50,123:DEBUG:Calculating grade for Bart
2020-06-18 08:27:50,124:DEBUG:Student grades have been calculated. Moving on to send emails to parents of students who have failed their exams.

Now we know when each line of our code was executed. In a longer program, this data would be especially useful because it would help us understand the order in which our code is running and the speed of our code.

Conclusion

The Python logging module is an incredibly useful tool for debugging. It helps you track all the events that run in your program and gives you the option to save those events as a separate file. This will help you more effectively debug your code and understand what events have executed when you run a program. 

"Career Karma entered my life when I needed it most and quickly helped me match with a bootcamp. Two months after graduating, I found my dream job that aligned with my values and goals in life!"

Venus, Software Engineer at Rockbot

Find Your Bootcamp Match