Python Subprocess Module for System Commands

The subprocess module in Python allows you to start and interact with subprocesses, which are separate processes running on your computer. It provides functions for creating, controlling, and communicating with subprocesses, as well as for handling the output and error streams of the subprocess.

A programme can start new processes by calling library functions available in the os or subprocess modules such as os.fork(), subprocess.Popen(), and so on. These processes, known as subprocesses, however, operate as totally independent entities, each with their own private system state and main thread of execution.

Using Python subprocess as an alternative to the os module was first suggested and approved for Python 2.4. Even as recently as 3.8, certain changes were formally recorded. Although Python 3.10.4 was used to test the examples in this article, you just need version 3.8 or higher to follow along with this lesson.

The run() function will be the primary means of communication between you and the Python subprocess module. This blocking function will launch a new process and wait for it to complete before resuming.

Because a subprocess is self-contained, it runs alongside the original process. That is, the process that produced the subprocess can continue working on other things while the subprocess works behind the scenes.

Here’s an example of how to use the subprocess module to execute a command and print the results:

import subprocess

result = subprocess.run(["ls", "-l"], capture_output=True)
print(result.stdout.decode())

This will run the ls -l command and print the output as a string.

You can also use the subprocess module to communicate with the subprocess using pipes. For example:

import subprocess

process = subprocess.Popen(["wc"], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
output, _ = process.communicate(input=b"hello\nworld\n")
print(output.decode())

This will start the wc command and send it the input “hello\nworld\n” through a pipe. The communicate function will then wait for the command to finish and return the output as a string.

The subprocess module in Python provides a powerful and flexible way to interact with subprocesses, and can be useful for tasks such as running external commands, capturing the output and error streams of the subprocess, and communicating with the subprocess through pipes.