Python Subprocess


The subprocess module enables you to start new applications from your Python program. How cool is that?

Related course:
Complete Python Masterclass

Start a process in Python:

You can start a process in Python using the Popen function call. The program below starts the unix program ‘cat’ and the second parameter is the argument. This is equivalent to ‘cat test.py’.  You can start any program with any parameter.

#!/usr/bin/env python 
 
from subprocess import Popen, PIPE
 
process = Popen(['cat', 'test.py'], stdout=PIPE, stderr=PIPE)
stdout, stderr = process.communicate()
print stdout

The process.communicate() call reads input and output from the process.  stdout is the process output. stderr will be written only if an error occurs.  If you want to wait for the program to finish you can call Popen.wait().

Subprocess call():

Subprocess has a method call() which can be used to start a program. The parameter is a list of which the first argument must be the program name. The full definition is:

subprocess.call(args, *, stdin=None, stdout=None, stderr=None, shell=False)
# Run the command described by args. 
# Wait for command to complete, then return the returncode attribute.

In the example below the full command would be “ls -l”

#!/usr/bin/env python 
 
import subprocess
subprocess.call(["ls", "-l"])

Save process output (stdout)

We can get the output of a program and store it in a string directly using check_output. The method is defined as:

 subprocess.check_output(args, *, stdin=None, stderr=None, shell=False, universal_newlines=False)
 # Run command with arguments and return its output as a byte string.

Example usage:

#!/usr/bin/env python 
import subprocess
 
s = subprocess.check_output(["echo", "Hello World!"])
print("s = " + s)

12 thoughts on “Python Subprocess

  1. Lukasz Szczepaniak - October 30, 2017

    Hi,

    I am struggle with provide a pipe ‘|’ between command execution. Let’s say I try to make “ifconfig ixgbe | grep ether”. How to provide pipe to the code? When I try something like this:

    from subprocess import Popen, PIPE
    ...
            if interface==ixgbe0:
                interface = Popen(['ifconfig', 'ixgbe0'], stdout=PIPE, stderr=PIPE)
                stdout, stderr = interface.communicate()

    It works like ‘ifconfig ixgbe’ but when I add pipe, like this:

            if interface==ixgbe0:
                interface = Popen(['ifconfig', 'ixgbe0',  '|', 'grep', 'ether], stdout=PIPE, stderr=PIPE)
                stdout, stderr = interface.communicate()

    It does not show any error, but there is no output for ether.

    1. Frank - November 4, 2017

      I could use the pipe operator with ifconfig using this way (I used wlp7s0 instead of ixgbe0):

      from subprocess import check_output
       
      out = check_output('ifconfig wlp7s0 | grep ether', shell=True)
      print(out)

      You can also do it this way:

      from subprocess import PIPE,Popen
       
      p1 = Popen(["ifconfig", "wlp7s0"],stdout=PIPE)
      p2 = Popen(["grep", "ether"],stdin=p1.stdout,stdout=PIPE,stderr=PIPE)
      p1.stdout.close()
      out, err = p2.communicate()
      print(out)

      We link p1 and p2.
      (Be sure to change the device to ixgbe0).

  2. M.d Rehman - July 10, 2017

    Hi frank. I wanted to know if i could store many values using (check_output). I want to use ping operation in cmd as subprocess and store the ping statistics in the variable to use them.

    1. Frank - July 11, 2017

      Hi Rehman, you can store the entire output like this:

      #!/usr/bin/env python
      import subprocess
       
      s = subprocess.check_output(["ping", "-c 1", "google.com"])
      print("s = " + str(s))

      Python 3 returns this as bytes, so we convert it to string with the function decode.
      Then we split on the newline character:

      #!/usr/bin/env python
      import subprocess
       
      s = subprocess.check_output(["ping", "-c 4", "google.com"])
      output = s.decode("utf-8")
       
      lines = output.split('\n')
       
      for line in lines:
          print(line)

      Now we have a few strings. We can parse the line or lines we want. We’ll split that string into parts.
      I’ll take the statistics summary.

      #!/usr/bin/env python
      import subprocess
       
      s = subprocess.check_output(["ping", "-c 4", "google.com"])
      output = s.decode("utf-8")
       
      lines = output.split('\n')
       
      statistics = (lines[7])
      values = statistics.split(", ")
      for v in values:
          print(v)

      All variables are stored in the list values. You can then store them in variables if you want. v[0] contains the first variable of the line.

  3. Hina - January 11, 2016

    Hi Frank,
    i have found your website very useful as i am python learner. please if you could guide me the way to read pdf file in python. Secondly, i want tutorials about natural language processing in python. can you help in this regard.
    Many Thanks

    1. Frank - January 28, 2016

      Hi Hina,
      Python has no standard method to read pdf. To read pdf you need to use a module. You can find related projects here: https://github.com/search?l=Python&q=pdf&type=Repositories&utf8=%E2%9C%93
      Alternatively you can convert the pdf document to a txt method, using the Pdftotext tool: https://en.wikipedia.org/wiki/Pdftotext

  4. Benben - September 1, 2015

    Hi,Frank

    Could tell me why?

    I’ve copy it and run it . it show me:

    from subprocess import Popen, PIPE
     
    process = Popen(['cat', 'test.py'], stdout=PIPE, stderr=PIPE)
    stdout, stderr = process.communicate()
    print stdout
     
    Traceback (most recent call last):
      File "C:\Python27\learn\subprocess.py", line 5, in 
        process = Popen(['cat', 'test.py'], stdout=PIPE, stderr=PIPE)
      File "C:\Python27\lib\subprocess.py", line 710, in __init__
        errread, errwrite)
      File "C:\Python27\lib\subprocess.py", line 958, in _execute_child
     
     
       startupinfo)
    WindowsError: [Error 2]
    1. Frank - September 1, 2015

      Hi,
      The first parameter of Popen() is ‘cat’, this is a unix program.
      I’m not sure if this program is available on windows, try changing it to notepad.exe

      1. Francisco Javier Martin - May 30, 2017

        If you start notepad.exe as a windowed app then python will not get the output.
        The MSDOS command similar to “cat” is “type”.

        process = Popen([‘type’, ‘test.py’], stdout=PIPE, stderr=PIPE)

        1. Frank - May 31, 2017

          Thanks Francisco!

  5. Kheenan - July 16, 2015

    I just want to say These are the best tutorials of any anywhere. I use tutorials all the time and these are just so concise and effective. Thanks a lot and keep at it!

    1. Frank - July 16, 2015

      Thanks Kheenan!