python subprocess

Python hosting: Host, run, and code Python in the cloud!

The subprocess module in Python allows users to spawn new applications, connect to their input/output/error pipes, and obtain their return codes. It’s a powerful tool that lets you interact with the system and other applications directly from your Python code.

If you’re aspiring to become proficient in Python programming, consider the Python Programming Bootcamp: Go from zero to hero.

Using Popen to Start a Process:

Popen is a fundamental method within the subprocess module. It helps you start external programs. The example below demonstrates how to start the Unix program ‘cat’ to read the contents of ‘test.py’. This is similar to executing ‘cat test.py’ in a terminal. In Python, you can initiate any program with any parameter using this approach.

#!/usr/bin/env python
from subprocess import Popen, PIPE
process = Popen(['cat', 'test.py'], stdout=PIPE, stderr=PIPE)
stdout, stderr = process.communicate()
print(stdout)

Here, process.communicate() captures both the output and any errors from the process. The stdout variable contains the process’s output. If there’s an error, it’s captured in stderr. If you want to ensure your script waits for the subprocess to finish before proceeding, you can use Popen.wait().

The call() Method in Subprocess:

subprocess provides another method named call(). This function is used to execute a program, wait for it to finish, and then return the return code. Below is its full definition:

1	subprocess.call(args, *, stdin=None, stdout=None, stderr=None, shell=False)

For instance, to execute the command “ls -l” using this method:

1
2
3

#!/usr/bin/env python
import subprocess
subprocess.call(["ls", "-l"])

Capturing Process Output with check_output:

If you want to capture the output of an executed command directly into a string, you can use check_output. Here’s its method definition:

1	subprocess.check_output(args, *, stdin=None, stderr=None, shell=False, universal_newlines=False)

Here’s how you can use it:

#!/usr/bin/env python
import subprocess
s = subprocess.check_output(["echo", "Hello World!"])
print("s = " + str(s))

By mastering the subprocess module, you’ll be better equipped to create Python scripts that can seamlessly interact with other programs and the operating system.

Back | Next

Posted in beginner

2015-08-29

Leave a Reply:

Kheenan • Thu, 16 Jul 2015

I just want to say These are the best tutorials of any anywhere. I use tutorials all the time and these are just so concise and effective. Thanks a lot and keep at it!

Frank • Thu, 16 Jul 2015

Thanks Kheenan!

Benben • Tue, 01 Sep 2015

Hi,Frank

Could tell me why?

I've copy it and run it . it show me:

from subprocess import Popen, PIPE
 
process = Popen(['cat', 'test.py'], stdout=PIPE, stderr=PIPE)
stdout, stderr = process.communicate()
print stdout

Traceback (most recent call last):
  File "C:\Python27\learn\subprocess.py", line 5, in 
    process = Popen(['cat', 'test.py'], stdout=PIPE, stderr=PIPE)
  File "C:\Python27\lib\subprocess.py", line 710, in __init__
    errread, errwrite)
  File "C:\Python27\lib\subprocess.py", line 958, in _execute_child


   startupinfo)
WindowsError: [Error 2]

Frank • Tue, 01 Sep 2015

Hi,
The first parameter of Popen() is 'cat', this is a unix program.
I'm not sure if this program is available on windows, try changing it to notepad.exe

Hina • Mon, 11 Jan 2016

Hi Frank,
i have found your website very useful as i am python learner. please if you could guide me the way to read pdf file in python. Secondly, i want tutorials about natural language processing in python. can you help in this regard.
Many Thanks

Frank • Thu, 28 Jan 2016

Hi Hina,
Python has no standard method to read pdf. To read pdf you need to use a module. You can find related projects here: https://github.com/search?l...
Alternatively you can convert the pdf document to a txt method, using the Pdftotext tool: https://en.wikipedia.org/wi...

Francisco Javier Martin • Tue, 30 May 2017

If you start notepad.exe as a windowed app then python will not get the output.
The MSDOS command similar to "cat" is "type".

process = Popen(['type', 'test.py'], stdout=PIPE, stderr=PIPE)

Frank • Wed, 31 May 2017

Thanks Francisco!

M.d Rehman • Mon, 10 Jul 2017

Hi frank. I wanted to know if i could store many values using (check_output). I want to use ping operation in cmd as subprocess and store the ping statistics in the variable to use them.

Frank • Tue, 11 Jul 2017

Hi Rehman, you can store the entire output like this:


#!/usr/bin/env python
import subprocess
s = subprocess.check_output(["ping", "-c 1", "google.com"])
print("s = " + str(s))

Python 3 returns this as bytes, so we convert it to string with the function decode.
Then we split on the newline character:


#!/usr/bin/env python
import subprocess
s = subprocess.check_output(["ping", "-c 4", "google.com"])
output = s.decode("utf-8")
lines = output.split('\n')
for line in lines:
    print(line)

Now we have a few strings. We can parse the line or lines we want. We'll split that string into parts.
I'll take the statistics summary.


#!/usr/bin/env python
import subprocess
s = subprocess.check_output(["ping", "-c 4", "google.com"])
output = s.decode("utf-8")
lines = output.split('\n')
statistics = (lines[7])
values = statistics.split(", ")
for v in values:
    print(v)

All variables are stored in the list values. You can then store them in variables if you want. v[0] contains the first variable of the line.

Laura • 2021-10-21T14:12:48.544Z

Hey Frank !
Thank you for the tutorial. It has been quite helpfull in understanding the subprocess module, but now
i found a new problem and am wondering if you might have an idea for a solution ? I want to open an exe and pass it a .tcl file filled with commands for the .exe to use. By searching the whole internet I still havent ound a way that works. Do you might have an idea ?

Kind regards,

Laura

Frank • 2021-10-21T14:13:42.544Z

You might have luck with command line arguments and include them in the subprocess call

Norm • 2021-08-21T13:31:05.089Z

There are two coding errors on this page:

It should be 'print(stdout)' not 'print stdout'

'print('s = ' + s)' throws an error message 'TypeError: can only concatenate str (not 'bytes') to str'

Frank • 2021-08-21T13:32:05.089Z

Thanks, Python is not backwards compatible and thats why you got these errors. I have updated the page