Python - How do I pass a string into subprocess.Popen (using the stdin argument)?












246















If I do the following:



import subprocess
from cStringIO import StringIO
subprocess.Popen(['grep','f'],stdout=subprocess.PIPE,stdin=StringIO('onentwonthreenfournfivensixn')).communicate()[0]


I get:



Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/build/toolchain/mac32/python-2.4.3/lib/python2.4/subprocess.py", line 533, in __init__
(p2cread, p2cwrite,
File "/build/toolchain/mac32/python-2.4.3/lib/python2.4/subprocess.py", line 830, in _get_handles
p2cread = stdin.fileno()
AttributeError: 'cStringIO.StringI' object has no attribute 'fileno'


Apparently a cStringIO.StringIO object doesn't quack close enough to a file duck to suit subprocess.Popen. How do I work around this?










share|improve this question




















  • 3





    Instead of disputing my answer with this being deleted, I'm adding it as a comment... Recommended reading: Doug Hellmann's Python Module of the Week blog post on subprocess.

    – Daryl Spitzer
    Jun 18 '13 at 22:43








  • 3





    the blog post contains multiple errors e.g., the very first code example: call(['ls', '-1'], shell=True) is incorrect. I recommend to read common questions from subprocess' tag description instead. In particular, Why subprocess.Popen doesn't work when args is sequence? explains why call(['ls', '-1'], shell=True) is wrong. I remember leaving comments under the blog post but I don't see them now for some reason.

    – jfs
    Mar 17 '16 at 14:24
















246















If I do the following:



import subprocess
from cStringIO import StringIO
subprocess.Popen(['grep','f'],stdout=subprocess.PIPE,stdin=StringIO('onentwonthreenfournfivensixn')).communicate()[0]


I get:



Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/build/toolchain/mac32/python-2.4.3/lib/python2.4/subprocess.py", line 533, in __init__
(p2cread, p2cwrite,
File "/build/toolchain/mac32/python-2.4.3/lib/python2.4/subprocess.py", line 830, in _get_handles
p2cread = stdin.fileno()
AttributeError: 'cStringIO.StringI' object has no attribute 'fileno'


Apparently a cStringIO.StringIO object doesn't quack close enough to a file duck to suit subprocess.Popen. How do I work around this?










share|improve this question




















  • 3





    Instead of disputing my answer with this being deleted, I'm adding it as a comment... Recommended reading: Doug Hellmann's Python Module of the Week blog post on subprocess.

    – Daryl Spitzer
    Jun 18 '13 at 22:43








  • 3





    the blog post contains multiple errors e.g., the very first code example: call(['ls', '-1'], shell=True) is incorrect. I recommend to read common questions from subprocess' tag description instead. In particular, Why subprocess.Popen doesn't work when args is sequence? explains why call(['ls', '-1'], shell=True) is wrong. I remember leaving comments under the blog post but I don't see them now for some reason.

    – jfs
    Mar 17 '16 at 14:24














246












246








246


72






If I do the following:



import subprocess
from cStringIO import StringIO
subprocess.Popen(['grep','f'],stdout=subprocess.PIPE,stdin=StringIO('onentwonthreenfournfivensixn')).communicate()[0]


I get:



Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/build/toolchain/mac32/python-2.4.3/lib/python2.4/subprocess.py", line 533, in __init__
(p2cread, p2cwrite,
File "/build/toolchain/mac32/python-2.4.3/lib/python2.4/subprocess.py", line 830, in _get_handles
p2cread = stdin.fileno()
AttributeError: 'cStringIO.StringI' object has no attribute 'fileno'


Apparently a cStringIO.StringIO object doesn't quack close enough to a file duck to suit subprocess.Popen. How do I work around this?










share|improve this question
















If I do the following:



import subprocess
from cStringIO import StringIO
subprocess.Popen(['grep','f'],stdout=subprocess.PIPE,stdin=StringIO('onentwonthreenfournfivensixn')).communicate()[0]


I get:



Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/build/toolchain/mac32/python-2.4.3/lib/python2.4/subprocess.py", line 533, in __init__
(p2cread, p2cwrite,
File "/build/toolchain/mac32/python-2.4.3/lib/python2.4/subprocess.py", line 830, in _get_handles
p2cread = stdin.fileno()
AttributeError: 'cStringIO.StringI' object has no attribute 'fileno'


Apparently a cStringIO.StringIO object doesn't quack close enough to a file duck to suit subprocess.Popen. How do I work around this?







python subprocess stdin






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Jun 5 '09 at 2:56









Nikhil Chelliah

4,7732630




4,7732630










asked Oct 2 '08 at 17:25









Daryl SpitzerDaryl Spitzer

53.5k60144162




53.5k60144162








  • 3





    Instead of disputing my answer with this being deleted, I'm adding it as a comment... Recommended reading: Doug Hellmann's Python Module of the Week blog post on subprocess.

    – Daryl Spitzer
    Jun 18 '13 at 22:43








  • 3





    the blog post contains multiple errors e.g., the very first code example: call(['ls', '-1'], shell=True) is incorrect. I recommend to read common questions from subprocess' tag description instead. In particular, Why subprocess.Popen doesn't work when args is sequence? explains why call(['ls', '-1'], shell=True) is wrong. I remember leaving comments under the blog post but I don't see them now for some reason.

    – jfs
    Mar 17 '16 at 14:24














  • 3





    Instead of disputing my answer with this being deleted, I'm adding it as a comment... Recommended reading: Doug Hellmann's Python Module of the Week blog post on subprocess.

    – Daryl Spitzer
    Jun 18 '13 at 22:43








  • 3





    the blog post contains multiple errors e.g., the very first code example: call(['ls', '-1'], shell=True) is incorrect. I recommend to read common questions from subprocess' tag description instead. In particular, Why subprocess.Popen doesn't work when args is sequence? explains why call(['ls', '-1'], shell=True) is wrong. I remember leaving comments under the blog post but I don't see them now for some reason.

    – jfs
    Mar 17 '16 at 14:24








3




3





Instead of disputing my answer with this being deleted, I'm adding it as a comment... Recommended reading: Doug Hellmann's Python Module of the Week blog post on subprocess.

– Daryl Spitzer
Jun 18 '13 at 22:43







Instead of disputing my answer with this being deleted, I'm adding it as a comment... Recommended reading: Doug Hellmann's Python Module of the Week blog post on subprocess.

– Daryl Spitzer
Jun 18 '13 at 22:43






3




3





the blog post contains multiple errors e.g., the very first code example: call(['ls', '-1'], shell=True) is incorrect. I recommend to read common questions from subprocess' tag description instead. In particular, Why subprocess.Popen doesn't work when args is sequence? explains why call(['ls', '-1'], shell=True) is wrong. I remember leaving comments under the blog post but I don't see them now for some reason.

– jfs
Mar 17 '16 at 14:24





the blog post contains multiple errors e.g., the very first code example: call(['ls', '-1'], shell=True) is incorrect. I recommend to read common questions from subprocess' tag description instead. In particular, Why subprocess.Popen doesn't work when args is sequence? explains why call(['ls', '-1'], shell=True) is wrong. I remember leaving comments under the blog post but I don't see them now for some reason.

– jfs
Mar 17 '16 at 14:24












10 Answers
10






active

oldest

votes


















286














Popen.communicate() documentation:




Note that if you want to send data to
the process’s stdin, you need to
create the Popen object with
stdin=PIPE. Similarly, to get anything
other than None in the result tuple,
you need to give stdout=PIPE and/or
stderr=PIPE too.



Replacing os.popen*




    pipe = os.popen(cmd, 'w', bufsize)
# ==>
pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin



Warning Use communicate() rather than
stdin.write(), stdout.read() or
stderr.read() to avoid deadlocks due
to any of the other OS pipe buffers
filling up and blocking the child
process.




So your example could be written as follows:



from subprocess import Popen, PIPE, STDOUT

p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
grep_stdout = p.communicate(input=b'onentwonthreenfournfivensixn')[0]
print(grep_stdout.decode())
# -> four
# -> five
# ->




On the current Python 3 version, you could use subprocess.run, to pass input as a string to an external command and get its exit status, and its output as a string back in one call:



#!/usr/bin/env python3
from subprocess import run, PIPE

p = run(['grep', 'f'], stdout=PIPE,
input='onentwonthreenfournfivensixn', encoding='ascii')
print(p.returncode)
# -> 0
print(p.stdout)
# -> four
# -> five
# ->





share|improve this answer





















  • 3





    I missed that warning. I'm glad I asked (even though I thought I had the answer).

    – Daryl Spitzer
    Oct 3 '08 at 16:02






  • 11





    This is NOT a good solution. In particular, you cannot asynchronously process p.stdout.readline output if you do this since you'd have to wait for the entire stdout to arrive. It's is also memory-inefficient.

    – OTZ
    Aug 20 '10 at 21:59






  • 5





    @OTZ What's a better solution?

    – Nick T
    Nov 17 '10 at 21:27






  • 10





    @Nick T: "better" depends on context. Newton's laws are good for the domain they are applicable but you need special relativity to design GPS. See Non-blocking read on a subprocess.PIPE in python.

    – jfs
    Oct 18 '11 at 20:25






  • 9





    But note the NOTE for communicate: "do not use this method if the data size is large or unlimited"

    – Owen
    Jan 22 '14 at 0:21



















42














I figured out this workaround:



>>> p = subprocess.Popen(['grep','f'],stdout=subprocess.PIPE,stdin=subprocess.PIPE)
>>> p.stdin.write(b'onentwonthreenfournfivensixn') #expects a bytes type object
>>> p.communicate()[0]
'fournfiven'
>>> p.stdin.close()


Is there a better one?






share|improve this answer





















  • 8





    That's not a workaround - that's the correct way to do it!

    – Moe
    Oct 2 '08 at 18:10






  • 21





    @Moe: stdin.write() usage is discouraged, p.communicate() should be used. See my answer.

    – jfs
    Oct 3 '08 at 14:26






  • 9





    Per the subprocess documentation: Warning - Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.

    – Jason Mock
    Aug 25 '10 at 20:49






  • 1





    I think this is good way to do it if you're confident that your stdout/err won't ever fill up (for instance, it's going to a file, or another thread is eating it) and you have an unbounded amount of data to be sent to stdin.

    – Lucretiel
    May 9 '16 at 19:24






  • 1





    In particular, doing it this way still ensures that stdin is closed, so that if the subprocesses is one that consumes input forever, the communicate will close the pipe and allow the process to end gracefully.

    – Lucretiel
    May 9 '16 at 19:25



















22














I'm a bit surprised nobody suggested creating a pipe, which is in my opinion the far simplest way to pass a string to stdin of a subprocess:



read, write = os.pipe()
os.write(write, "stdin input here")
os.close(write)

subprocess.check_call(['your-command'], stdin=read)





share|improve this answer



















  • 2





    The os and the subprocess documentation both agree that you should prefer the latter over the former. This is a legacy solution which has a (slightly less concise) standard replacement; the accepted answer quotes the pertinent documentation.

    – tripleee
    May 4 '16 at 11:53






  • 1





    I'm not sure that's correct, tripleee. The quoted documentation says why it is hard to use the pipes created by the process, but in this solution it creates a pipe and passes it in. I believe it avoids the potential deadlock problems of managing the pipes after the process has already started.

    – Graham Christensen
    May 7 '16 at 15:09













  • os.popen is deprecated in favour of subprocess

    – hd1
    Feb 18 '17 at 17:30






  • 2





    -1: it leads to the deadlock, it may loose data. This functionality is already provided by the subprocess module. Use it instead of reimplementing it poorly (try to write a value that is larger than an OS pipe buffer)

    – jfs
    Aug 18 '17 at 6:24



















17














There's a beatiful solution if you're using Python 3.4 or better. Use the input argument instead of the stdin argument, which accepts a bytes argument:



output = subprocess.check_output(
["sed", "s/foo/bar/"],
input=b"foo",
)





share|improve this answer



















  • 1





    does not work for call

    – vidstige
    Oct 5 '17 at 13:46






  • 3





    @vidstige You're right, that's weird. I would consider filing this as an Python bug, I don't see any good reason in why check_output should have an input argument, but not call.

    – Flimm
    Oct 5 '17 at 14:35











  • This is the best answer for Python 3.4+ (using it in Python 3.6). It indeed does not work with check_call but it works for run. It also works with input=string as long as you pass an encoding argument too according to the documentation.

    – Nikolaos Georgiou
    Feb 22 at 2:48



















14














I am using python3 and found out that you need to encode your string before you can pass it into stdin:



p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=PIPE)
out, err = p.communicate(input='onentwonthreenfournfivensixn'.encode())
print(out)





share|improve this answer



















  • 4





    You don't specifically need to encode the input, it just wants a bytes-like object (e.g. b'something'). It will return err and out as bytes also. If you want to avoid this, you can pass universal_newlines=True to Popen. Then it will accept input as str and will return err/out as str also.

    – Six
    Jan 17 '16 at 15:00






  • 2





    But beware, universal_newlines=True will also convert your newlines to match your system

    – Nacht
    Feb 25 '16 at 12:35






  • 1





    If you're using Python 3, see my answer for an even more convenient solution.

    – Flimm
    Dec 8 '16 at 10:05



















13














"Apparently a cStringIO.StringIO object doesn't quack close enough to a file duck to suit subprocess.Popen"



:-)



I'm afraid not. The pipe is a low-level OS concept, so it absolutely requires a file object that is represented by an OS-level file descriptor. Your workaround is the right one.






share|improve this answer































    8














    from subprocess import Popen, PIPE
    from tempfile import SpooledTemporaryFile as tempfile
    f = tempfile()
    f.write('onentwonthreenfournfivensixn')
    f.seek(0)
    print Popen(['/bin/grep','f'],stdout=PIPE,stdin=f).stdout.read()
    f.close()





    share|improve this answer



















    • 3





      fyi, tempfile.SpooledTemporaryFile.__doc__ says: Temporary file wrapper, specialized to switch from StringIO to a real file when it exceeds a certain size or when a fileno is needed.

      – Doug F
      Aug 9 '13 at 20:18





















    6














    """
    Ex: Dialog (2-way) with a Popen()
    """

    p = subprocess.Popen('Your Command Here',
    stdout=subprocess.PIPE,
    stderr=subprocess.STDOUT,
    stdin=PIPE,
    shell=True,
    bufsize=0)
    p.stdin.write('STARTn')
    out = p.stdout.readline()
    while out:
    line = out
    line = line.rstrip("n")

    if "WHATEVER1" in line:
    pr = 1
    p.stdin.write('DO 1n')
    out = p.stdout.readline()
    continue

    if "WHATEVER2" in line:
    pr = 2
    p.stdin.write('DO 2n')
    out = p.stdout.readline()
    continue
    """
    ..........
    """

    out = p.stdout.readline()

    p.wait()





    share|improve this answer





















    • 4





      Because shell=True is so commonly used for no good reason, and this is a popular question, let me point out that there are a lot of situations where Popen(['cmd', 'with', 'args']) is decidedly better than Popen('cmd with args', shell=True) and having the shell break the command and arguments into tokens, but not otherwise providing anything useful, while adding a significant amount of complexity and thus also attack surface.

      – tripleee
      Oct 29 '14 at 13:43





















    5














    Beware that Popen.communicate(input=s)may give you trouble ifsis too big, because apparently the parent process will buffer it before forking the child subprocess, meaning it needs "twice as much" used memory at that point (at least according to the "under the hood" explanation and linked documentation found here). In my particular case,swas a generator that was first fully expanded and only then written tostdin so the parent process was huge right before the child was spawned,
    and no memory was left to fork it:



    File "/opt/local/stow/python-2.7.2/lib/python2.7/subprocess.py", line 1130, in _execute_child
    self.pid = os.fork()
    OSError: [Errno 12] Cannot allocate memory






    share|improve this answer

































      3














      p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)    
      p.stdin.write('onen')
      time.sleep(0.5)
      p.stdin.write('twon')
      time.sleep(0.5)
      p.stdin.write('threen')
      time.sleep(0.5)
      testresult = p.communicate()[0]
      time.sleep(0.5)
      print(testresult)





      share|improve this answer


























        Your Answer






        StackExchange.ifUsing("editor", function () {
        StackExchange.using("externalEditor", function () {
        StackExchange.using("snippets", function () {
        StackExchange.snippets.init();
        });
        });
        }, "code-snippets");

        StackExchange.ready(function() {
        var channelOptions = {
        tags: "".split(" "),
        id: "1"
        };
        initTagRenderer("".split(" "), "".split(" "), channelOptions);

        StackExchange.using("externalEditor", function() {
        // Have to fire editor after snippets, if snippets enabled
        if (StackExchange.settings.snippets.snippetsEnabled) {
        StackExchange.using("snippets", function() {
        createEditor();
        });
        }
        else {
        createEditor();
        }
        });

        function createEditor() {
        StackExchange.prepareEditor({
        heartbeatType: 'answer',
        autoActivateHeartbeat: false,
        convertImagesToLinks: true,
        noModals: true,
        showLowRepImageUploadWarning: true,
        reputationToPostImages: 10,
        bindNavPrevention: true,
        postfix: "",
        imageUploader: {
        brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
        contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
        allowUrls: true
        },
        onDemand: true,
        discardSelector: ".discard-answer"
        ,immediatelyShowMarkdownHelp:true
        });


        }
        });














        draft saved

        draft discarded


















        StackExchange.ready(
        function () {
        StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f163542%2fpython-how-do-i-pass-a-string-into-subprocess-popen-using-the-stdin-argument%23new-answer', 'question_page');
        }
        );

        Post as a guest















        Required, but never shown

























        10 Answers
        10






        active

        oldest

        votes








        10 Answers
        10






        active

        oldest

        votes









        active

        oldest

        votes






        active

        oldest

        votes









        286














        Popen.communicate() documentation:




        Note that if you want to send data to
        the process’s stdin, you need to
        create the Popen object with
        stdin=PIPE. Similarly, to get anything
        other than None in the result tuple,
        you need to give stdout=PIPE and/or
        stderr=PIPE too.



        Replacing os.popen*




            pipe = os.popen(cmd, 'w', bufsize)
        # ==>
        pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin



        Warning Use communicate() rather than
        stdin.write(), stdout.read() or
        stderr.read() to avoid deadlocks due
        to any of the other OS pipe buffers
        filling up and blocking the child
        process.




        So your example could be written as follows:



        from subprocess import Popen, PIPE, STDOUT

        p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
        grep_stdout = p.communicate(input=b'onentwonthreenfournfivensixn')[0]
        print(grep_stdout.decode())
        # -> four
        # -> five
        # ->




        On the current Python 3 version, you could use subprocess.run, to pass input as a string to an external command and get its exit status, and its output as a string back in one call:



        #!/usr/bin/env python3
        from subprocess import run, PIPE

        p = run(['grep', 'f'], stdout=PIPE,
        input='onentwonthreenfournfivensixn', encoding='ascii')
        print(p.returncode)
        # -> 0
        print(p.stdout)
        # -> four
        # -> five
        # ->





        share|improve this answer





















        • 3





          I missed that warning. I'm glad I asked (even though I thought I had the answer).

          – Daryl Spitzer
          Oct 3 '08 at 16:02






        • 11





          This is NOT a good solution. In particular, you cannot asynchronously process p.stdout.readline output if you do this since you'd have to wait for the entire stdout to arrive. It's is also memory-inefficient.

          – OTZ
          Aug 20 '10 at 21:59






        • 5





          @OTZ What's a better solution?

          – Nick T
          Nov 17 '10 at 21:27






        • 10





          @Nick T: "better" depends on context. Newton's laws are good for the domain they are applicable but you need special relativity to design GPS. See Non-blocking read on a subprocess.PIPE in python.

          – jfs
          Oct 18 '11 at 20:25






        • 9





          But note the NOTE for communicate: "do not use this method if the data size is large or unlimited"

          – Owen
          Jan 22 '14 at 0:21
















        286














        Popen.communicate() documentation:




        Note that if you want to send data to
        the process’s stdin, you need to
        create the Popen object with
        stdin=PIPE. Similarly, to get anything
        other than None in the result tuple,
        you need to give stdout=PIPE and/or
        stderr=PIPE too.



        Replacing os.popen*




            pipe = os.popen(cmd, 'w', bufsize)
        # ==>
        pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin



        Warning Use communicate() rather than
        stdin.write(), stdout.read() or
        stderr.read() to avoid deadlocks due
        to any of the other OS pipe buffers
        filling up and blocking the child
        process.




        So your example could be written as follows:



        from subprocess import Popen, PIPE, STDOUT

        p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
        grep_stdout = p.communicate(input=b'onentwonthreenfournfivensixn')[0]
        print(grep_stdout.decode())
        # -> four
        # -> five
        # ->




        On the current Python 3 version, you could use subprocess.run, to pass input as a string to an external command and get its exit status, and its output as a string back in one call:



        #!/usr/bin/env python3
        from subprocess import run, PIPE

        p = run(['grep', 'f'], stdout=PIPE,
        input='onentwonthreenfournfivensixn', encoding='ascii')
        print(p.returncode)
        # -> 0
        print(p.stdout)
        # -> four
        # -> five
        # ->





        share|improve this answer





















        • 3





          I missed that warning. I'm glad I asked (even though I thought I had the answer).

          – Daryl Spitzer
          Oct 3 '08 at 16:02






        • 11





          This is NOT a good solution. In particular, you cannot asynchronously process p.stdout.readline output if you do this since you'd have to wait for the entire stdout to arrive. It's is also memory-inefficient.

          – OTZ
          Aug 20 '10 at 21:59






        • 5





          @OTZ What's a better solution?

          – Nick T
          Nov 17 '10 at 21:27






        • 10





          @Nick T: "better" depends on context. Newton's laws are good for the domain they are applicable but you need special relativity to design GPS. See Non-blocking read on a subprocess.PIPE in python.

          – jfs
          Oct 18 '11 at 20:25






        • 9





          But note the NOTE for communicate: "do not use this method if the data size is large or unlimited"

          – Owen
          Jan 22 '14 at 0:21














        286












        286








        286







        Popen.communicate() documentation:




        Note that if you want to send data to
        the process’s stdin, you need to
        create the Popen object with
        stdin=PIPE. Similarly, to get anything
        other than None in the result tuple,
        you need to give stdout=PIPE and/or
        stderr=PIPE too.



        Replacing os.popen*




            pipe = os.popen(cmd, 'w', bufsize)
        # ==>
        pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin



        Warning Use communicate() rather than
        stdin.write(), stdout.read() or
        stderr.read() to avoid deadlocks due
        to any of the other OS pipe buffers
        filling up and blocking the child
        process.




        So your example could be written as follows:



        from subprocess import Popen, PIPE, STDOUT

        p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
        grep_stdout = p.communicate(input=b'onentwonthreenfournfivensixn')[0]
        print(grep_stdout.decode())
        # -> four
        # -> five
        # ->




        On the current Python 3 version, you could use subprocess.run, to pass input as a string to an external command and get its exit status, and its output as a string back in one call:



        #!/usr/bin/env python3
        from subprocess import run, PIPE

        p = run(['grep', 'f'], stdout=PIPE,
        input='onentwonthreenfournfivensixn', encoding='ascii')
        print(p.returncode)
        # -> 0
        print(p.stdout)
        # -> four
        # -> five
        # ->





        share|improve this answer















        Popen.communicate() documentation:




        Note that if you want to send data to
        the process’s stdin, you need to
        create the Popen object with
        stdin=PIPE. Similarly, to get anything
        other than None in the result tuple,
        you need to give stdout=PIPE and/or
        stderr=PIPE too.



        Replacing os.popen*




            pipe = os.popen(cmd, 'w', bufsize)
        # ==>
        pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin



        Warning Use communicate() rather than
        stdin.write(), stdout.read() or
        stderr.read() to avoid deadlocks due
        to any of the other OS pipe buffers
        filling up and blocking the child
        process.




        So your example could be written as follows:



        from subprocess import Popen, PIPE, STDOUT

        p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
        grep_stdout = p.communicate(input=b'onentwonthreenfournfivensixn')[0]
        print(grep_stdout.decode())
        # -> four
        # -> five
        # ->




        On the current Python 3 version, you could use subprocess.run, to pass input as a string to an external command and get its exit status, and its output as a string back in one call:



        #!/usr/bin/env python3
        from subprocess import run, PIPE

        p = run(['grep', 'f'], stdout=PIPE,
        input='onentwonthreenfournfivensixn', encoding='ascii')
        print(p.returncode)
        # -> 0
        print(p.stdout)
        # -> four
        # -> five
        # ->






        share|improve this answer














        share|improve this answer



        share|improve this answer








        edited Sep 13 '17 at 9:27

























        answered Oct 3 '08 at 4:11









        jfsjfs

        271k815721127




        271k815721127








        • 3





          I missed that warning. I'm glad I asked (even though I thought I had the answer).

          – Daryl Spitzer
          Oct 3 '08 at 16:02






        • 11





          This is NOT a good solution. In particular, you cannot asynchronously process p.stdout.readline output if you do this since you'd have to wait for the entire stdout to arrive. It's is also memory-inefficient.

          – OTZ
          Aug 20 '10 at 21:59






        • 5





          @OTZ What's a better solution?

          – Nick T
          Nov 17 '10 at 21:27






        • 10





          @Nick T: "better" depends on context. Newton's laws are good for the domain they are applicable but you need special relativity to design GPS. See Non-blocking read on a subprocess.PIPE in python.

          – jfs
          Oct 18 '11 at 20:25






        • 9





          But note the NOTE for communicate: "do not use this method if the data size is large or unlimited"

          – Owen
          Jan 22 '14 at 0:21














        • 3





          I missed that warning. I'm glad I asked (even though I thought I had the answer).

          – Daryl Spitzer
          Oct 3 '08 at 16:02






        • 11





          This is NOT a good solution. In particular, you cannot asynchronously process p.stdout.readline output if you do this since you'd have to wait for the entire stdout to arrive. It's is also memory-inefficient.

          – OTZ
          Aug 20 '10 at 21:59






        • 5





          @OTZ What's a better solution?

          – Nick T
          Nov 17 '10 at 21:27






        • 10





          @Nick T: "better" depends on context. Newton's laws are good for the domain they are applicable but you need special relativity to design GPS. See Non-blocking read on a subprocess.PIPE in python.

          – jfs
          Oct 18 '11 at 20:25






        • 9





          But note the NOTE for communicate: "do not use this method if the data size is large or unlimited"

          – Owen
          Jan 22 '14 at 0:21








        3




        3





        I missed that warning. I'm glad I asked (even though I thought I had the answer).

        – Daryl Spitzer
        Oct 3 '08 at 16:02





        I missed that warning. I'm glad I asked (even though I thought I had the answer).

        – Daryl Spitzer
        Oct 3 '08 at 16:02




        11




        11





        This is NOT a good solution. In particular, you cannot asynchronously process p.stdout.readline output if you do this since you'd have to wait for the entire stdout to arrive. It's is also memory-inefficient.

        – OTZ
        Aug 20 '10 at 21:59





        This is NOT a good solution. In particular, you cannot asynchronously process p.stdout.readline output if you do this since you'd have to wait for the entire stdout to arrive. It's is also memory-inefficient.

        – OTZ
        Aug 20 '10 at 21:59




        5




        5





        @OTZ What's a better solution?

        – Nick T
        Nov 17 '10 at 21:27





        @OTZ What's a better solution?

        – Nick T
        Nov 17 '10 at 21:27




        10




        10





        @Nick T: "better" depends on context. Newton's laws are good for the domain they are applicable but you need special relativity to design GPS. See Non-blocking read on a subprocess.PIPE in python.

        – jfs
        Oct 18 '11 at 20:25





        @Nick T: "better" depends on context. Newton's laws are good for the domain they are applicable but you need special relativity to design GPS. See Non-blocking read on a subprocess.PIPE in python.

        – jfs
        Oct 18 '11 at 20:25




        9




        9





        But note the NOTE for communicate: "do not use this method if the data size is large or unlimited"

        – Owen
        Jan 22 '14 at 0:21





        But note the NOTE for communicate: "do not use this method if the data size is large or unlimited"

        – Owen
        Jan 22 '14 at 0:21













        42














        I figured out this workaround:



        >>> p = subprocess.Popen(['grep','f'],stdout=subprocess.PIPE,stdin=subprocess.PIPE)
        >>> p.stdin.write(b'onentwonthreenfournfivensixn') #expects a bytes type object
        >>> p.communicate()[0]
        'fournfiven'
        >>> p.stdin.close()


        Is there a better one?






        share|improve this answer





















        • 8





          That's not a workaround - that's the correct way to do it!

          – Moe
          Oct 2 '08 at 18:10






        • 21





          @Moe: stdin.write() usage is discouraged, p.communicate() should be used. See my answer.

          – jfs
          Oct 3 '08 at 14:26






        • 9





          Per the subprocess documentation: Warning - Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.

          – Jason Mock
          Aug 25 '10 at 20:49






        • 1





          I think this is good way to do it if you're confident that your stdout/err won't ever fill up (for instance, it's going to a file, or another thread is eating it) and you have an unbounded amount of data to be sent to stdin.

          – Lucretiel
          May 9 '16 at 19:24






        • 1





          In particular, doing it this way still ensures that stdin is closed, so that if the subprocesses is one that consumes input forever, the communicate will close the pipe and allow the process to end gracefully.

          – Lucretiel
          May 9 '16 at 19:25
















        42














        I figured out this workaround:



        >>> p = subprocess.Popen(['grep','f'],stdout=subprocess.PIPE,stdin=subprocess.PIPE)
        >>> p.stdin.write(b'onentwonthreenfournfivensixn') #expects a bytes type object
        >>> p.communicate()[0]
        'fournfiven'
        >>> p.stdin.close()


        Is there a better one?






        share|improve this answer





















        • 8





          That's not a workaround - that's the correct way to do it!

          – Moe
          Oct 2 '08 at 18:10






        • 21





          @Moe: stdin.write() usage is discouraged, p.communicate() should be used. See my answer.

          – jfs
          Oct 3 '08 at 14:26






        • 9





          Per the subprocess documentation: Warning - Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.

          – Jason Mock
          Aug 25 '10 at 20:49






        • 1





          I think this is good way to do it if you're confident that your stdout/err won't ever fill up (for instance, it's going to a file, or another thread is eating it) and you have an unbounded amount of data to be sent to stdin.

          – Lucretiel
          May 9 '16 at 19:24






        • 1





          In particular, doing it this way still ensures that stdin is closed, so that if the subprocesses is one that consumes input forever, the communicate will close the pipe and allow the process to end gracefully.

          – Lucretiel
          May 9 '16 at 19:25














        42












        42








        42







        I figured out this workaround:



        >>> p = subprocess.Popen(['grep','f'],stdout=subprocess.PIPE,stdin=subprocess.PIPE)
        >>> p.stdin.write(b'onentwonthreenfournfivensixn') #expects a bytes type object
        >>> p.communicate()[0]
        'fournfiven'
        >>> p.stdin.close()


        Is there a better one?






        share|improve this answer















        I figured out this workaround:



        >>> p = subprocess.Popen(['grep','f'],stdout=subprocess.PIPE,stdin=subprocess.PIPE)
        >>> p.stdin.write(b'onentwonthreenfournfivensixn') #expects a bytes type object
        >>> p.communicate()[0]
        'fournfiven'
        >>> p.stdin.close()


        Is there a better one?







        share|improve this answer














        share|improve this answer



        share|improve this answer








        edited Jan 18 '18 at 0:31









        tobsecret

        1,384516




        1,384516










        answered Oct 2 '08 at 17:27









        Daryl SpitzerDaryl Spitzer

        53.5k60144162




        53.5k60144162








        • 8





          That's not a workaround - that's the correct way to do it!

          – Moe
          Oct 2 '08 at 18:10






        • 21





          @Moe: stdin.write() usage is discouraged, p.communicate() should be used. See my answer.

          – jfs
          Oct 3 '08 at 14:26






        • 9





          Per the subprocess documentation: Warning - Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.

          – Jason Mock
          Aug 25 '10 at 20:49






        • 1





          I think this is good way to do it if you're confident that your stdout/err won't ever fill up (for instance, it's going to a file, or another thread is eating it) and you have an unbounded amount of data to be sent to stdin.

          – Lucretiel
          May 9 '16 at 19:24






        • 1





          In particular, doing it this way still ensures that stdin is closed, so that if the subprocesses is one that consumes input forever, the communicate will close the pipe and allow the process to end gracefully.

          – Lucretiel
          May 9 '16 at 19:25














        • 8





          That's not a workaround - that's the correct way to do it!

          – Moe
          Oct 2 '08 at 18:10






        • 21





          @Moe: stdin.write() usage is discouraged, p.communicate() should be used. See my answer.

          – jfs
          Oct 3 '08 at 14:26






        • 9





          Per the subprocess documentation: Warning - Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.

          – Jason Mock
          Aug 25 '10 at 20:49






        • 1





          I think this is good way to do it if you're confident that your stdout/err won't ever fill up (for instance, it's going to a file, or another thread is eating it) and you have an unbounded amount of data to be sent to stdin.

          – Lucretiel
          May 9 '16 at 19:24






        • 1





          In particular, doing it this way still ensures that stdin is closed, so that if the subprocesses is one that consumes input forever, the communicate will close the pipe and allow the process to end gracefully.

          – Lucretiel
          May 9 '16 at 19:25








        8




        8





        That's not a workaround - that's the correct way to do it!

        – Moe
        Oct 2 '08 at 18:10





        That's not a workaround - that's the correct way to do it!

        – Moe
        Oct 2 '08 at 18:10




        21




        21





        @Moe: stdin.write() usage is discouraged, p.communicate() should be used. See my answer.

        – jfs
        Oct 3 '08 at 14:26





        @Moe: stdin.write() usage is discouraged, p.communicate() should be used. See my answer.

        – jfs
        Oct 3 '08 at 14:26




        9




        9





        Per the subprocess documentation: Warning - Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.

        – Jason Mock
        Aug 25 '10 at 20:49





        Per the subprocess documentation: Warning - Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.

        – Jason Mock
        Aug 25 '10 at 20:49




        1




        1





        I think this is good way to do it if you're confident that your stdout/err won't ever fill up (for instance, it's going to a file, or another thread is eating it) and you have an unbounded amount of data to be sent to stdin.

        – Lucretiel
        May 9 '16 at 19:24





        I think this is good way to do it if you're confident that your stdout/err won't ever fill up (for instance, it's going to a file, or another thread is eating it) and you have an unbounded amount of data to be sent to stdin.

        – Lucretiel
        May 9 '16 at 19:24




        1




        1





        In particular, doing it this way still ensures that stdin is closed, so that if the subprocesses is one that consumes input forever, the communicate will close the pipe and allow the process to end gracefully.

        – Lucretiel
        May 9 '16 at 19:25





        In particular, doing it this way still ensures that stdin is closed, so that if the subprocesses is one that consumes input forever, the communicate will close the pipe and allow the process to end gracefully.

        – Lucretiel
        May 9 '16 at 19:25











        22














        I'm a bit surprised nobody suggested creating a pipe, which is in my opinion the far simplest way to pass a string to stdin of a subprocess:



        read, write = os.pipe()
        os.write(write, "stdin input here")
        os.close(write)

        subprocess.check_call(['your-command'], stdin=read)





        share|improve this answer



















        • 2





          The os and the subprocess documentation both agree that you should prefer the latter over the former. This is a legacy solution which has a (slightly less concise) standard replacement; the accepted answer quotes the pertinent documentation.

          – tripleee
          May 4 '16 at 11:53






        • 1





          I'm not sure that's correct, tripleee. The quoted documentation says why it is hard to use the pipes created by the process, but in this solution it creates a pipe and passes it in. I believe it avoids the potential deadlock problems of managing the pipes after the process has already started.

          – Graham Christensen
          May 7 '16 at 15:09













        • os.popen is deprecated in favour of subprocess

          – hd1
          Feb 18 '17 at 17:30






        • 2





          -1: it leads to the deadlock, it may loose data. This functionality is already provided by the subprocess module. Use it instead of reimplementing it poorly (try to write a value that is larger than an OS pipe buffer)

          – jfs
          Aug 18 '17 at 6:24
















        22














        I'm a bit surprised nobody suggested creating a pipe, which is in my opinion the far simplest way to pass a string to stdin of a subprocess:



        read, write = os.pipe()
        os.write(write, "stdin input here")
        os.close(write)

        subprocess.check_call(['your-command'], stdin=read)





        share|improve this answer



















        • 2





          The os and the subprocess documentation both agree that you should prefer the latter over the former. This is a legacy solution which has a (slightly less concise) standard replacement; the accepted answer quotes the pertinent documentation.

          – tripleee
          May 4 '16 at 11:53






        • 1





          I'm not sure that's correct, tripleee. The quoted documentation says why it is hard to use the pipes created by the process, but in this solution it creates a pipe and passes it in. I believe it avoids the potential deadlock problems of managing the pipes after the process has already started.

          – Graham Christensen
          May 7 '16 at 15:09













        • os.popen is deprecated in favour of subprocess

          – hd1
          Feb 18 '17 at 17:30






        • 2





          -1: it leads to the deadlock, it may loose data. This functionality is already provided by the subprocess module. Use it instead of reimplementing it poorly (try to write a value that is larger than an OS pipe buffer)

          – jfs
          Aug 18 '17 at 6:24














        22












        22








        22







        I'm a bit surprised nobody suggested creating a pipe, which is in my opinion the far simplest way to pass a string to stdin of a subprocess:



        read, write = os.pipe()
        os.write(write, "stdin input here")
        os.close(write)

        subprocess.check_call(['your-command'], stdin=read)





        share|improve this answer













        I'm a bit surprised nobody suggested creating a pipe, which is in my opinion the far simplest way to pass a string to stdin of a subprocess:



        read, write = os.pipe()
        os.write(write, "stdin input here")
        os.close(write)

        subprocess.check_call(['your-command'], stdin=read)






        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Nov 2 '15 at 16:34









        Graham ChristensenGraham Christensen

        68456




        68456








        • 2





          The os and the subprocess documentation both agree that you should prefer the latter over the former. This is a legacy solution which has a (slightly less concise) standard replacement; the accepted answer quotes the pertinent documentation.

          – tripleee
          May 4 '16 at 11:53






        • 1





          I'm not sure that's correct, tripleee. The quoted documentation says why it is hard to use the pipes created by the process, but in this solution it creates a pipe and passes it in. I believe it avoids the potential deadlock problems of managing the pipes after the process has already started.

          – Graham Christensen
          May 7 '16 at 15:09













        • os.popen is deprecated in favour of subprocess

          – hd1
          Feb 18 '17 at 17:30






        • 2





          -1: it leads to the deadlock, it may loose data. This functionality is already provided by the subprocess module. Use it instead of reimplementing it poorly (try to write a value that is larger than an OS pipe buffer)

          – jfs
          Aug 18 '17 at 6:24














        • 2





          The os and the subprocess documentation both agree that you should prefer the latter over the former. This is a legacy solution which has a (slightly less concise) standard replacement; the accepted answer quotes the pertinent documentation.

          – tripleee
          May 4 '16 at 11:53






        • 1





          I'm not sure that's correct, tripleee. The quoted documentation says why it is hard to use the pipes created by the process, but in this solution it creates a pipe and passes it in. I believe it avoids the potential deadlock problems of managing the pipes after the process has already started.

          – Graham Christensen
          May 7 '16 at 15:09













        • os.popen is deprecated in favour of subprocess

          – hd1
          Feb 18 '17 at 17:30






        • 2





          -1: it leads to the deadlock, it may loose data. This functionality is already provided by the subprocess module. Use it instead of reimplementing it poorly (try to write a value that is larger than an OS pipe buffer)

          – jfs
          Aug 18 '17 at 6:24








        2




        2





        The os and the subprocess documentation both agree that you should prefer the latter over the former. This is a legacy solution which has a (slightly less concise) standard replacement; the accepted answer quotes the pertinent documentation.

        – tripleee
        May 4 '16 at 11:53





        The os and the subprocess documentation both agree that you should prefer the latter over the former. This is a legacy solution which has a (slightly less concise) standard replacement; the accepted answer quotes the pertinent documentation.

        – tripleee
        May 4 '16 at 11:53




        1




        1





        I'm not sure that's correct, tripleee. The quoted documentation says why it is hard to use the pipes created by the process, but in this solution it creates a pipe and passes it in. I believe it avoids the potential deadlock problems of managing the pipes after the process has already started.

        – Graham Christensen
        May 7 '16 at 15:09







        I'm not sure that's correct, tripleee. The quoted documentation says why it is hard to use the pipes created by the process, but in this solution it creates a pipe and passes it in. I believe it avoids the potential deadlock problems of managing the pipes after the process has already started.

        – Graham Christensen
        May 7 '16 at 15:09















        os.popen is deprecated in favour of subprocess

        – hd1
        Feb 18 '17 at 17:30





        os.popen is deprecated in favour of subprocess

        – hd1
        Feb 18 '17 at 17:30




        2




        2





        -1: it leads to the deadlock, it may loose data. This functionality is already provided by the subprocess module. Use it instead of reimplementing it poorly (try to write a value that is larger than an OS pipe buffer)

        – jfs
        Aug 18 '17 at 6:24





        -1: it leads to the deadlock, it may loose data. This functionality is already provided by the subprocess module. Use it instead of reimplementing it poorly (try to write a value that is larger than an OS pipe buffer)

        – jfs
        Aug 18 '17 at 6:24











        17














        There's a beatiful solution if you're using Python 3.4 or better. Use the input argument instead of the stdin argument, which accepts a bytes argument:



        output = subprocess.check_output(
        ["sed", "s/foo/bar/"],
        input=b"foo",
        )





        share|improve this answer



















        • 1





          does not work for call

          – vidstige
          Oct 5 '17 at 13:46






        • 3





          @vidstige You're right, that's weird. I would consider filing this as an Python bug, I don't see any good reason in why check_output should have an input argument, but not call.

          – Flimm
          Oct 5 '17 at 14:35











        • This is the best answer for Python 3.4+ (using it in Python 3.6). It indeed does not work with check_call but it works for run. It also works with input=string as long as you pass an encoding argument too according to the documentation.

          – Nikolaos Georgiou
          Feb 22 at 2:48
















        17














        There's a beatiful solution if you're using Python 3.4 or better. Use the input argument instead of the stdin argument, which accepts a bytes argument:



        output = subprocess.check_output(
        ["sed", "s/foo/bar/"],
        input=b"foo",
        )





        share|improve this answer



















        • 1





          does not work for call

          – vidstige
          Oct 5 '17 at 13:46






        • 3





          @vidstige You're right, that's weird. I would consider filing this as an Python bug, I don't see any good reason in why check_output should have an input argument, but not call.

          – Flimm
          Oct 5 '17 at 14:35











        • This is the best answer for Python 3.4+ (using it in Python 3.6). It indeed does not work with check_call but it works for run. It also works with input=string as long as you pass an encoding argument too according to the documentation.

          – Nikolaos Georgiou
          Feb 22 at 2:48














        17












        17








        17







        There's a beatiful solution if you're using Python 3.4 or better. Use the input argument instead of the stdin argument, which accepts a bytes argument:



        output = subprocess.check_output(
        ["sed", "s/foo/bar/"],
        input=b"foo",
        )





        share|improve this answer













        There's a beatiful solution if you're using Python 3.4 or better. Use the input argument instead of the stdin argument, which accepts a bytes argument:



        output = subprocess.check_output(
        ["sed", "s/foo/bar/"],
        input=b"foo",
        )






        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Dec 8 '16 at 10:04









        FlimmFlimm

        55.1k23141162




        55.1k23141162








        • 1





          does not work for call

          – vidstige
          Oct 5 '17 at 13:46






        • 3





          @vidstige You're right, that's weird. I would consider filing this as an Python bug, I don't see any good reason in why check_output should have an input argument, but not call.

          – Flimm
          Oct 5 '17 at 14:35











        • This is the best answer for Python 3.4+ (using it in Python 3.6). It indeed does not work with check_call but it works for run. It also works with input=string as long as you pass an encoding argument too according to the documentation.

          – Nikolaos Georgiou
          Feb 22 at 2:48














        • 1





          does not work for call

          – vidstige
          Oct 5 '17 at 13:46






        • 3





          @vidstige You're right, that's weird. I would consider filing this as an Python bug, I don't see any good reason in why check_output should have an input argument, but not call.

          – Flimm
          Oct 5 '17 at 14:35











        • This is the best answer for Python 3.4+ (using it in Python 3.6). It indeed does not work with check_call but it works for run. It also works with input=string as long as you pass an encoding argument too according to the documentation.

          – Nikolaos Georgiou
          Feb 22 at 2:48








        1




        1





        does not work for call

        – vidstige
        Oct 5 '17 at 13:46





        does not work for call

        – vidstige
        Oct 5 '17 at 13:46




        3




        3





        @vidstige You're right, that's weird. I would consider filing this as an Python bug, I don't see any good reason in why check_output should have an input argument, but not call.

        – Flimm
        Oct 5 '17 at 14:35





        @vidstige You're right, that's weird. I would consider filing this as an Python bug, I don't see any good reason in why check_output should have an input argument, but not call.

        – Flimm
        Oct 5 '17 at 14:35













        This is the best answer for Python 3.4+ (using it in Python 3.6). It indeed does not work with check_call but it works for run. It also works with input=string as long as you pass an encoding argument too according to the documentation.

        – Nikolaos Georgiou
        Feb 22 at 2:48





        This is the best answer for Python 3.4+ (using it in Python 3.6). It indeed does not work with check_call but it works for run. It also works with input=string as long as you pass an encoding argument too according to the documentation.

        – Nikolaos Georgiou
        Feb 22 at 2:48











        14














        I am using python3 and found out that you need to encode your string before you can pass it into stdin:



        p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=PIPE)
        out, err = p.communicate(input='onentwonthreenfournfivensixn'.encode())
        print(out)





        share|improve this answer



















        • 4





          You don't specifically need to encode the input, it just wants a bytes-like object (e.g. b'something'). It will return err and out as bytes also. If you want to avoid this, you can pass universal_newlines=True to Popen. Then it will accept input as str and will return err/out as str also.

          – Six
          Jan 17 '16 at 15:00






        • 2





          But beware, universal_newlines=True will also convert your newlines to match your system

          – Nacht
          Feb 25 '16 at 12:35






        • 1





          If you're using Python 3, see my answer for an even more convenient solution.

          – Flimm
          Dec 8 '16 at 10:05
















        14














        I am using python3 and found out that you need to encode your string before you can pass it into stdin:



        p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=PIPE)
        out, err = p.communicate(input='onentwonthreenfournfivensixn'.encode())
        print(out)





        share|improve this answer



















        • 4





          You don't specifically need to encode the input, it just wants a bytes-like object (e.g. b'something'). It will return err and out as bytes also. If you want to avoid this, you can pass universal_newlines=True to Popen. Then it will accept input as str and will return err/out as str also.

          – Six
          Jan 17 '16 at 15:00






        • 2





          But beware, universal_newlines=True will also convert your newlines to match your system

          – Nacht
          Feb 25 '16 at 12:35






        • 1





          If you're using Python 3, see my answer for an even more convenient solution.

          – Flimm
          Dec 8 '16 at 10:05














        14












        14








        14







        I am using python3 and found out that you need to encode your string before you can pass it into stdin:



        p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=PIPE)
        out, err = p.communicate(input='onentwonthreenfournfivensixn'.encode())
        print(out)





        share|improve this answer













        I am using python3 and found out that you need to encode your string before you can pass it into stdin:



        p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=PIPE)
        out, err = p.communicate(input='onentwonthreenfournfivensixn'.encode())
        print(out)






        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Jul 27 '14 at 15:29









        qedqed

        10.1k1269119




        10.1k1269119








        • 4





          You don't specifically need to encode the input, it just wants a bytes-like object (e.g. b'something'). It will return err and out as bytes also. If you want to avoid this, you can pass universal_newlines=True to Popen. Then it will accept input as str and will return err/out as str also.

          – Six
          Jan 17 '16 at 15:00






        • 2





          But beware, universal_newlines=True will also convert your newlines to match your system

          – Nacht
          Feb 25 '16 at 12:35






        • 1





          If you're using Python 3, see my answer for an even more convenient solution.

          – Flimm
          Dec 8 '16 at 10:05














        • 4





          You don't specifically need to encode the input, it just wants a bytes-like object (e.g. b'something'). It will return err and out as bytes also. If you want to avoid this, you can pass universal_newlines=True to Popen. Then it will accept input as str and will return err/out as str also.

          – Six
          Jan 17 '16 at 15:00






        • 2





          But beware, universal_newlines=True will also convert your newlines to match your system

          – Nacht
          Feb 25 '16 at 12:35






        • 1





          If you're using Python 3, see my answer for an even more convenient solution.

          – Flimm
          Dec 8 '16 at 10:05








        4




        4





        You don't specifically need to encode the input, it just wants a bytes-like object (e.g. b'something'). It will return err and out as bytes also. If you want to avoid this, you can pass universal_newlines=True to Popen. Then it will accept input as str and will return err/out as str also.

        – Six
        Jan 17 '16 at 15:00





        You don't specifically need to encode the input, it just wants a bytes-like object (e.g. b'something'). It will return err and out as bytes also. If you want to avoid this, you can pass universal_newlines=True to Popen. Then it will accept input as str and will return err/out as str also.

        – Six
        Jan 17 '16 at 15:00




        2




        2





        But beware, universal_newlines=True will also convert your newlines to match your system

        – Nacht
        Feb 25 '16 at 12:35





        But beware, universal_newlines=True will also convert your newlines to match your system

        – Nacht
        Feb 25 '16 at 12:35




        1




        1





        If you're using Python 3, see my answer for an even more convenient solution.

        – Flimm
        Dec 8 '16 at 10:05





        If you're using Python 3, see my answer for an even more convenient solution.

        – Flimm
        Dec 8 '16 at 10:05











        13














        "Apparently a cStringIO.StringIO object doesn't quack close enough to a file duck to suit subprocess.Popen"



        :-)



        I'm afraid not. The pipe is a low-level OS concept, so it absolutely requires a file object that is represented by an OS-level file descriptor. Your workaround is the right one.






        share|improve this answer




























          13














          "Apparently a cStringIO.StringIO object doesn't quack close enough to a file duck to suit subprocess.Popen"



          :-)



          I'm afraid not. The pipe is a low-level OS concept, so it absolutely requires a file object that is represented by an OS-level file descriptor. Your workaround is the right one.






          share|improve this answer


























            13












            13








            13







            "Apparently a cStringIO.StringIO object doesn't quack close enough to a file duck to suit subprocess.Popen"



            :-)



            I'm afraid not. The pipe is a low-level OS concept, so it absolutely requires a file object that is represented by an OS-level file descriptor. Your workaround is the right one.






            share|improve this answer













            "Apparently a cStringIO.StringIO object doesn't quack close enough to a file duck to suit subprocess.Popen"



            :-)



            I'm afraid not. The pipe is a low-level OS concept, so it absolutely requires a file object that is represented by an OS-level file descriptor. Your workaround is the right one.







            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered Oct 2 '08 at 18:33









            Dan LenskiDan Lenski

            50.8k95695




            50.8k95695























                8














                from subprocess import Popen, PIPE
                from tempfile import SpooledTemporaryFile as tempfile
                f = tempfile()
                f.write('onentwonthreenfournfivensixn')
                f.seek(0)
                print Popen(['/bin/grep','f'],stdout=PIPE,stdin=f).stdout.read()
                f.close()





                share|improve this answer



















                • 3





                  fyi, tempfile.SpooledTemporaryFile.__doc__ says: Temporary file wrapper, specialized to switch from StringIO to a real file when it exceeds a certain size or when a fileno is needed.

                  – Doug F
                  Aug 9 '13 at 20:18


















                8














                from subprocess import Popen, PIPE
                from tempfile import SpooledTemporaryFile as tempfile
                f = tempfile()
                f.write('onentwonthreenfournfivensixn')
                f.seek(0)
                print Popen(['/bin/grep','f'],stdout=PIPE,stdin=f).stdout.read()
                f.close()





                share|improve this answer



















                • 3





                  fyi, tempfile.SpooledTemporaryFile.__doc__ says: Temporary file wrapper, specialized to switch from StringIO to a real file when it exceeds a certain size or when a fileno is needed.

                  – Doug F
                  Aug 9 '13 at 20:18
















                8












                8








                8







                from subprocess import Popen, PIPE
                from tempfile import SpooledTemporaryFile as tempfile
                f = tempfile()
                f.write('onentwonthreenfournfivensixn')
                f.seek(0)
                print Popen(['/bin/grep','f'],stdout=PIPE,stdin=f).stdout.read()
                f.close()





                share|improve this answer













                from subprocess import Popen, PIPE
                from tempfile import SpooledTemporaryFile as tempfile
                f = tempfile()
                f.write('onentwonthreenfournfivensixn')
                f.seek(0)
                print Popen(['/bin/grep','f'],stdout=PIPE,stdin=f).stdout.read()
                f.close()






                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Apr 13 '12 at 3:36









                Michael WaddellMichael Waddell

                8912




                8912








                • 3





                  fyi, tempfile.SpooledTemporaryFile.__doc__ says: Temporary file wrapper, specialized to switch from StringIO to a real file when it exceeds a certain size or when a fileno is needed.

                  – Doug F
                  Aug 9 '13 at 20:18
















                • 3





                  fyi, tempfile.SpooledTemporaryFile.__doc__ says: Temporary file wrapper, specialized to switch from StringIO to a real file when it exceeds a certain size or when a fileno is needed.

                  – Doug F
                  Aug 9 '13 at 20:18










                3




                3





                fyi, tempfile.SpooledTemporaryFile.__doc__ says: Temporary file wrapper, specialized to switch from StringIO to a real file when it exceeds a certain size or when a fileno is needed.

                – Doug F
                Aug 9 '13 at 20:18







                fyi, tempfile.SpooledTemporaryFile.__doc__ says: Temporary file wrapper, specialized to switch from StringIO to a real file when it exceeds a certain size or when a fileno is needed.

                – Doug F
                Aug 9 '13 at 20:18













                6














                """
                Ex: Dialog (2-way) with a Popen()
                """

                p = subprocess.Popen('Your Command Here',
                stdout=subprocess.PIPE,
                stderr=subprocess.STDOUT,
                stdin=PIPE,
                shell=True,
                bufsize=0)
                p.stdin.write('STARTn')
                out = p.stdout.readline()
                while out:
                line = out
                line = line.rstrip("n")

                if "WHATEVER1" in line:
                pr = 1
                p.stdin.write('DO 1n')
                out = p.stdout.readline()
                continue

                if "WHATEVER2" in line:
                pr = 2
                p.stdin.write('DO 2n')
                out = p.stdout.readline()
                continue
                """
                ..........
                """

                out = p.stdout.readline()

                p.wait()





                share|improve this answer





















                • 4





                  Because shell=True is so commonly used for no good reason, and this is a popular question, let me point out that there are a lot of situations where Popen(['cmd', 'with', 'args']) is decidedly better than Popen('cmd with args', shell=True) and having the shell break the command and arguments into tokens, but not otherwise providing anything useful, while adding a significant amount of complexity and thus also attack surface.

                  – tripleee
                  Oct 29 '14 at 13:43


















                6














                """
                Ex: Dialog (2-way) with a Popen()
                """

                p = subprocess.Popen('Your Command Here',
                stdout=subprocess.PIPE,
                stderr=subprocess.STDOUT,
                stdin=PIPE,
                shell=True,
                bufsize=0)
                p.stdin.write('STARTn')
                out = p.stdout.readline()
                while out:
                line = out
                line = line.rstrip("n")

                if "WHATEVER1" in line:
                pr = 1
                p.stdin.write('DO 1n')
                out = p.stdout.readline()
                continue

                if "WHATEVER2" in line:
                pr = 2
                p.stdin.write('DO 2n')
                out = p.stdout.readline()
                continue
                """
                ..........
                """

                out = p.stdout.readline()

                p.wait()





                share|improve this answer





















                • 4





                  Because shell=True is so commonly used for no good reason, and this is a popular question, let me point out that there are a lot of situations where Popen(['cmd', 'with', 'args']) is decidedly better than Popen('cmd with args', shell=True) and having the shell break the command and arguments into tokens, but not otherwise providing anything useful, while adding a significant amount of complexity and thus also attack surface.

                  – tripleee
                  Oct 29 '14 at 13:43
















                6












                6








                6







                """
                Ex: Dialog (2-way) with a Popen()
                """

                p = subprocess.Popen('Your Command Here',
                stdout=subprocess.PIPE,
                stderr=subprocess.STDOUT,
                stdin=PIPE,
                shell=True,
                bufsize=0)
                p.stdin.write('STARTn')
                out = p.stdout.readline()
                while out:
                line = out
                line = line.rstrip("n")

                if "WHATEVER1" in line:
                pr = 1
                p.stdin.write('DO 1n')
                out = p.stdout.readline()
                continue

                if "WHATEVER2" in line:
                pr = 2
                p.stdin.write('DO 2n')
                out = p.stdout.readline()
                continue
                """
                ..........
                """

                out = p.stdout.readline()

                p.wait()





                share|improve this answer















                """
                Ex: Dialog (2-way) with a Popen()
                """

                p = subprocess.Popen('Your Command Here',
                stdout=subprocess.PIPE,
                stderr=subprocess.STDOUT,
                stdin=PIPE,
                shell=True,
                bufsize=0)
                p.stdin.write('STARTn')
                out = p.stdout.readline()
                while out:
                line = out
                line = line.rstrip("n")

                if "WHATEVER1" in line:
                pr = 1
                p.stdin.write('DO 1n')
                out = p.stdout.readline()
                continue

                if "WHATEVER2" in line:
                pr = 2
                p.stdin.write('DO 2n')
                out = p.stdout.readline()
                continue
                """
                ..........
                """

                out = p.stdout.readline()

                p.wait()






                share|improve this answer














                share|improve this answer



                share|improve this answer








                edited Oct 8 '14 at 16:15









                Arne Babenhauserheide

                1,9722220




                1,9722220










                answered Jun 14 '13 at 13:20









                Lucien HercaudLucien Hercaud

                7911




                7911








                • 4





                  Because shell=True is so commonly used for no good reason, and this is a popular question, let me point out that there are a lot of situations where Popen(['cmd', 'with', 'args']) is decidedly better than Popen('cmd with args', shell=True) and having the shell break the command and arguments into tokens, but not otherwise providing anything useful, while adding a significant amount of complexity and thus also attack surface.

                  – tripleee
                  Oct 29 '14 at 13:43
















                • 4





                  Because shell=True is so commonly used for no good reason, and this is a popular question, let me point out that there are a lot of situations where Popen(['cmd', 'with', 'args']) is decidedly better than Popen('cmd with args', shell=True) and having the shell break the command and arguments into tokens, but not otherwise providing anything useful, while adding a significant amount of complexity and thus also attack surface.

                  – tripleee
                  Oct 29 '14 at 13:43










                4




                4





                Because shell=True is so commonly used for no good reason, and this is a popular question, let me point out that there are a lot of situations where Popen(['cmd', 'with', 'args']) is decidedly better than Popen('cmd with args', shell=True) and having the shell break the command and arguments into tokens, but not otherwise providing anything useful, while adding a significant amount of complexity and thus also attack surface.

                – tripleee
                Oct 29 '14 at 13:43







                Because shell=True is so commonly used for no good reason, and this is a popular question, let me point out that there are a lot of situations where Popen(['cmd', 'with', 'args']) is decidedly better than Popen('cmd with args', shell=True) and having the shell break the command and arguments into tokens, but not otherwise providing anything useful, while adding a significant amount of complexity and thus also attack surface.

                – tripleee
                Oct 29 '14 at 13:43













                5














                Beware that Popen.communicate(input=s)may give you trouble ifsis too big, because apparently the parent process will buffer it before forking the child subprocess, meaning it needs "twice as much" used memory at that point (at least according to the "under the hood" explanation and linked documentation found here). In my particular case,swas a generator that was first fully expanded and only then written tostdin so the parent process was huge right before the child was spawned,
                and no memory was left to fork it:



                File "/opt/local/stow/python-2.7.2/lib/python2.7/subprocess.py", line 1130, in _execute_child
                self.pid = os.fork()
                OSError: [Errno 12] Cannot allocate memory






                share|improve this answer






























                  5














                  Beware that Popen.communicate(input=s)may give you trouble ifsis too big, because apparently the parent process will buffer it before forking the child subprocess, meaning it needs "twice as much" used memory at that point (at least according to the "under the hood" explanation and linked documentation found here). In my particular case,swas a generator that was first fully expanded and only then written tostdin so the parent process was huge right before the child was spawned,
                  and no memory was left to fork it:



                  File "/opt/local/stow/python-2.7.2/lib/python2.7/subprocess.py", line 1130, in _execute_child
                  self.pid = os.fork()
                  OSError: [Errno 12] Cannot allocate memory






                  share|improve this answer




























                    5












                    5








                    5







                    Beware that Popen.communicate(input=s)may give you trouble ifsis too big, because apparently the parent process will buffer it before forking the child subprocess, meaning it needs "twice as much" used memory at that point (at least according to the "under the hood" explanation and linked documentation found here). In my particular case,swas a generator that was first fully expanded and only then written tostdin so the parent process was huge right before the child was spawned,
                    and no memory was left to fork it:



                    File "/opt/local/stow/python-2.7.2/lib/python2.7/subprocess.py", line 1130, in _execute_child
                    self.pid = os.fork()
                    OSError: [Errno 12] Cannot allocate memory






                    share|improve this answer















                    Beware that Popen.communicate(input=s)may give you trouble ifsis too big, because apparently the parent process will buffer it before forking the child subprocess, meaning it needs "twice as much" used memory at that point (at least according to the "under the hood" explanation and linked documentation found here). In my particular case,swas a generator that was first fully expanded and only then written tostdin so the parent process was huge right before the child was spawned,
                    and no memory was left to fork it:



                    File "/opt/local/stow/python-2.7.2/lib/python2.7/subprocess.py", line 1130, in _execute_child
                    self.pid = os.fork()
                    OSError: [Errno 12] Cannot allocate memory







                    share|improve this answer














                    share|improve this answer



                    share|improve this answer








                    edited May 23 '17 at 11:33









                    Community

                    11




                    11










                    answered May 19 '14 at 14:56









                    Lord Henry WottonLord Henry Wotton

                    1,044810




                    1,044810























                        3














                        p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)    
                        p.stdin.write('onen')
                        time.sleep(0.5)
                        p.stdin.write('twon')
                        time.sleep(0.5)
                        p.stdin.write('threen')
                        time.sleep(0.5)
                        testresult = p.communicate()[0]
                        time.sleep(0.5)
                        print(testresult)





                        share|improve this answer






























                          3














                          p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)    
                          p.stdin.write('onen')
                          time.sleep(0.5)
                          p.stdin.write('twon')
                          time.sleep(0.5)
                          p.stdin.write('threen')
                          time.sleep(0.5)
                          testresult = p.communicate()[0]
                          time.sleep(0.5)
                          print(testresult)





                          share|improve this answer




























                            3












                            3








                            3







                            p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)    
                            p.stdin.write('onen')
                            time.sleep(0.5)
                            p.stdin.write('twon')
                            time.sleep(0.5)
                            p.stdin.write('threen')
                            time.sleep(0.5)
                            testresult = p.communicate()[0]
                            time.sleep(0.5)
                            print(testresult)





                            share|improve this answer















                            p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)    
                            p.stdin.write('onen')
                            time.sleep(0.5)
                            p.stdin.write('twon')
                            time.sleep(0.5)
                            p.stdin.write('threen')
                            time.sleep(0.5)
                            testresult = p.communicate()[0]
                            time.sleep(0.5)
                            print(testresult)






                            share|improve this answer














                            share|improve this answer



                            share|improve this answer








                            edited Jun 27 '12 at 14:01









                            dusan

                            7,53822753




                            7,53822753










                            answered Apr 9 '09 at 4:39







                            gedwarp





































                                draft saved

                                draft discarded




















































                                Thanks for contributing an answer to Stack Overflow!


                                • Please be sure to answer the question. Provide details and share your research!

                                But avoid



                                • Asking for help, clarification, or responding to other answers.

                                • Making statements based on opinion; back them up with references or personal experience.


                                To learn more, see our tips on writing great answers.




                                draft saved


                                draft discarded














                                StackExchange.ready(
                                function () {
                                StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f163542%2fpython-how-do-i-pass-a-string-into-subprocess-popen-using-the-stdin-argument%23new-answer', 'question_page');
                                }
                                );

                                Post as a guest















                                Required, but never shown





















































                                Required, but never shown














                                Required, but never shown












                                Required, but never shown







                                Required, but never shown

































                                Required, but never shown














                                Required, but never shown












                                Required, but never shown







                                Required, but never shown







                                Popular posts from this blog

                                Create new schema in PostgreSQL using DBeaver

                                Deepest pit of an array with Javascript: test on Codility

                                Costa Masnaga