Python - How do I pass a string into subprocess.Popen (using the stdin argument)?
If I do the following:
import subprocess
from cStringIO import StringIO
subprocess.Popen(['grep','f'],stdout=subprocess.PIPE,stdin=StringIO('onentwonthreenfournfivensixn')).communicate()[0]
I get:
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/build/toolchain/mac32/python-2.4.3/lib/python2.4/subprocess.py", line 533, in __init__
(p2cread, p2cwrite,
File "/build/toolchain/mac32/python-2.4.3/lib/python2.4/subprocess.py", line 830, in _get_handles
p2cread = stdin.fileno()
AttributeError: 'cStringIO.StringI' object has no attribute 'fileno'
Apparently a cStringIO.StringIO object doesn't quack close enough to a file duck to suit subprocess.Popen. How do I work around this?
python subprocess stdin
add a comment |
If I do the following:
import subprocess
from cStringIO import StringIO
subprocess.Popen(['grep','f'],stdout=subprocess.PIPE,stdin=StringIO('onentwonthreenfournfivensixn')).communicate()[0]
I get:
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/build/toolchain/mac32/python-2.4.3/lib/python2.4/subprocess.py", line 533, in __init__
(p2cread, p2cwrite,
File "/build/toolchain/mac32/python-2.4.3/lib/python2.4/subprocess.py", line 830, in _get_handles
p2cread = stdin.fileno()
AttributeError: 'cStringIO.StringI' object has no attribute 'fileno'
Apparently a cStringIO.StringIO object doesn't quack close enough to a file duck to suit subprocess.Popen. How do I work around this?
python subprocess stdin
3
Instead of disputing my answer with this being deleted, I'm adding it as a comment... Recommended reading: Doug Hellmann's Python Module of the Week blog post on subprocess.
– Daryl Spitzer
Jun 18 '13 at 22:43
3
the blog post contains multiple errors e.g., the very first code example:call(['ls', '-1'], shell=True)
is incorrect. I recommend to read common questions from subprocess' tag description instead. In particular, Why subprocess.Popen doesn't work when args is sequence? explains whycall(['ls', '-1'], shell=True)
is wrong. I remember leaving comments under the blog post but I don't see them now for some reason.
– jfs
Mar 17 '16 at 14:24
add a comment |
If I do the following:
import subprocess
from cStringIO import StringIO
subprocess.Popen(['grep','f'],stdout=subprocess.PIPE,stdin=StringIO('onentwonthreenfournfivensixn')).communicate()[0]
I get:
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/build/toolchain/mac32/python-2.4.3/lib/python2.4/subprocess.py", line 533, in __init__
(p2cread, p2cwrite,
File "/build/toolchain/mac32/python-2.4.3/lib/python2.4/subprocess.py", line 830, in _get_handles
p2cread = stdin.fileno()
AttributeError: 'cStringIO.StringI' object has no attribute 'fileno'
Apparently a cStringIO.StringIO object doesn't quack close enough to a file duck to suit subprocess.Popen. How do I work around this?
python subprocess stdin
If I do the following:
import subprocess
from cStringIO import StringIO
subprocess.Popen(['grep','f'],stdout=subprocess.PIPE,stdin=StringIO('onentwonthreenfournfivensixn')).communicate()[0]
I get:
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/build/toolchain/mac32/python-2.4.3/lib/python2.4/subprocess.py", line 533, in __init__
(p2cread, p2cwrite,
File "/build/toolchain/mac32/python-2.4.3/lib/python2.4/subprocess.py", line 830, in _get_handles
p2cread = stdin.fileno()
AttributeError: 'cStringIO.StringI' object has no attribute 'fileno'
Apparently a cStringIO.StringIO object doesn't quack close enough to a file duck to suit subprocess.Popen. How do I work around this?
python subprocess stdin
python subprocess stdin
edited Jun 5 '09 at 2:56
Nikhil Chelliah
4,7732630
4,7732630
asked Oct 2 '08 at 17:25
Daryl SpitzerDaryl Spitzer
53.5k60144162
53.5k60144162
3
Instead of disputing my answer with this being deleted, I'm adding it as a comment... Recommended reading: Doug Hellmann's Python Module of the Week blog post on subprocess.
– Daryl Spitzer
Jun 18 '13 at 22:43
3
the blog post contains multiple errors e.g., the very first code example:call(['ls', '-1'], shell=True)
is incorrect. I recommend to read common questions from subprocess' tag description instead. In particular, Why subprocess.Popen doesn't work when args is sequence? explains whycall(['ls', '-1'], shell=True)
is wrong. I remember leaving comments under the blog post but I don't see them now for some reason.
– jfs
Mar 17 '16 at 14:24
add a comment |
3
Instead of disputing my answer with this being deleted, I'm adding it as a comment... Recommended reading: Doug Hellmann's Python Module of the Week blog post on subprocess.
– Daryl Spitzer
Jun 18 '13 at 22:43
3
the blog post contains multiple errors e.g., the very first code example:call(['ls', '-1'], shell=True)
is incorrect. I recommend to read common questions from subprocess' tag description instead. In particular, Why subprocess.Popen doesn't work when args is sequence? explains whycall(['ls', '-1'], shell=True)
is wrong. I remember leaving comments under the blog post but I don't see them now for some reason.
– jfs
Mar 17 '16 at 14:24
3
3
Instead of disputing my answer with this being deleted, I'm adding it as a comment... Recommended reading: Doug Hellmann's Python Module of the Week blog post on subprocess.
– Daryl Spitzer
Jun 18 '13 at 22:43
Instead of disputing my answer with this being deleted, I'm adding it as a comment... Recommended reading: Doug Hellmann's Python Module of the Week blog post on subprocess.
– Daryl Spitzer
Jun 18 '13 at 22:43
3
3
the blog post contains multiple errors e.g., the very first code example:
call(['ls', '-1'], shell=True)
is incorrect. I recommend to read common questions from subprocess' tag description instead. In particular, Why subprocess.Popen doesn't work when args is sequence? explains why call(['ls', '-1'], shell=True)
is wrong. I remember leaving comments under the blog post but I don't see them now for some reason.– jfs
Mar 17 '16 at 14:24
the blog post contains multiple errors e.g., the very first code example:
call(['ls', '-1'], shell=True)
is incorrect. I recommend to read common questions from subprocess' tag description instead. In particular, Why subprocess.Popen doesn't work when args is sequence? explains why call(['ls', '-1'], shell=True)
is wrong. I remember leaving comments under the blog post but I don't see them now for some reason.– jfs
Mar 17 '16 at 14:24
add a comment |
10 Answers
10
active
oldest
votes
Popen.communicate()
documentation:
Note that if you want to send data to
the process’s stdin, you need to
create the Popen object with
stdin=PIPE. Similarly, to get anything
other than None in the result tuple,
you need to give stdout=PIPE and/or
stderr=PIPE too.
Replacing os.popen*
pipe = os.popen(cmd, 'w', bufsize)
# ==>
pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin
Warning Use communicate() rather than
stdin.write(), stdout.read() or
stderr.read() to avoid deadlocks due
to any of the other OS pipe buffers
filling up and blocking the child
process.
So your example could be written as follows:
from subprocess import Popen, PIPE, STDOUT
p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
grep_stdout = p.communicate(input=b'onentwonthreenfournfivensixn')[0]
print(grep_stdout.decode())
# -> four
# -> five
# ->
On the current Python 3 version, you could use subprocess.run
, to pass input as a string to an external command and get its exit status, and its output as a string back in one call:
#!/usr/bin/env python3
from subprocess import run, PIPE
p = run(['grep', 'f'], stdout=PIPE,
input='onentwonthreenfournfivensixn', encoding='ascii')
print(p.returncode)
# -> 0
print(p.stdout)
# -> four
# -> five
# ->
3
I missed that warning. I'm glad I asked (even though I thought I had the answer).
– Daryl Spitzer
Oct 3 '08 at 16:02
11
This is NOT a good solution. In particular, you cannot asynchronously process p.stdout.readline output if you do this since you'd have to wait for the entire stdout to arrive. It's is also memory-inefficient.
– OTZ
Aug 20 '10 at 21:59
5
@OTZ What's a better solution?
– Nick T
Nov 17 '10 at 21:27
10
@Nick T: "better" depends on context. Newton's laws are good for the domain they are applicable but you need special relativity to design GPS. See Non-blocking read on a subprocess.PIPE in python.
– jfs
Oct 18 '11 at 20:25
9
But note the NOTE for communicate: "do not use this method if the data size is large or unlimited"
– Owen
Jan 22 '14 at 0:21
|
show 9 more comments
I figured out this workaround:
>>> p = subprocess.Popen(['grep','f'],stdout=subprocess.PIPE,stdin=subprocess.PIPE)
>>> p.stdin.write(b'onentwonthreenfournfivensixn') #expects a bytes type object
>>> p.communicate()[0]
'fournfiven'
>>> p.stdin.close()
Is there a better one?
8
That's not a workaround - that's the correct way to do it!
– Moe
Oct 2 '08 at 18:10
21
@Moe:stdin.write()
usage is discouraged,p.communicate()
should be used. See my answer.
– jfs
Oct 3 '08 at 14:26
9
Per the subprocess documentation: Warning - Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.
– Jason Mock
Aug 25 '10 at 20:49
1
I think this is good way to do it if you're confident that your stdout/err won't ever fill up (for instance, it's going to a file, or another thread is eating it) and you have an unbounded amount of data to be sent to stdin.
– Lucretiel
May 9 '16 at 19:24
1
In particular, doing it this way still ensures that stdin is closed, so that if the subprocesses is one that consumes input forever, thecommunicate
will close the pipe and allow the process to end gracefully.
– Lucretiel
May 9 '16 at 19:25
add a comment |
I'm a bit surprised nobody suggested creating a pipe, which is in my opinion the far simplest way to pass a string to stdin of a subprocess:
read, write = os.pipe()
os.write(write, "stdin input here")
os.close(write)
subprocess.check_call(['your-command'], stdin=read)
2
Theos
and thesubprocess
documentation both agree that you should prefer the latter over the former. This is a legacy solution which has a (slightly less concise) standard replacement; the accepted answer quotes the pertinent documentation.
– tripleee
May 4 '16 at 11:53
1
I'm not sure that's correct, tripleee. The quoted documentation says why it is hard to use the pipes created by the process, but in this solution it creates a pipe and passes it in. I believe it avoids the potential deadlock problems of managing the pipes after the process has already started.
– Graham Christensen
May 7 '16 at 15:09
os.popen is deprecated in favour of subprocess
– hd1
Feb 18 '17 at 17:30
2
-1: it leads to the deadlock, it may loose data. This functionality is already provided by the subprocess module. Use it instead of reimplementing it poorly (try to write a value that is larger than an OS pipe buffer)
– jfs
Aug 18 '17 at 6:24
add a comment |
There's a beatiful solution if you're using Python 3.4 or better. Use the input
argument instead of the stdin
argument, which accepts a bytes argument:
output = subprocess.check_output(
["sed", "s/foo/bar/"],
input=b"foo",
)
1
does not work forcall
– vidstige
Oct 5 '17 at 13:46
3
@vidstige You're right, that's weird. I would consider filing this as an Python bug, I don't see any good reason in whycheck_output
should have aninput
argument, but notcall
.
– Flimm
Oct 5 '17 at 14:35
This is the best answer for Python 3.4+ (using it in Python 3.6). It indeed does not work withcheck_call
but it works forrun
. It also works with input=string as long as you pass an encoding argument too according to the documentation.
– Nikolaos Georgiou
Feb 22 at 2:48
add a comment |
I am using python3 and found out that you need to encode your string before you can pass it into stdin:
p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=PIPE)
out, err = p.communicate(input='onentwonthreenfournfivensixn'.encode())
print(out)
4
You don't specifically need to encode the input, it just wants a bytes-like object (e.g.b'something'
). It will return err and out as bytes also. If you want to avoid this, you can passuniversal_newlines=True
toPopen
. Then it will accept input as str and will return err/out as str also.
– Six
Jan 17 '16 at 15:00
2
But beware,universal_newlines=True
will also convert your newlines to match your system
– Nacht
Feb 25 '16 at 12:35
1
If you're using Python 3, see my answer for an even more convenient solution.
– Flimm
Dec 8 '16 at 10:05
add a comment |
"Apparently a cStringIO.StringIO object doesn't quack close enough to a file duck to suit subprocess.Popen"
:-)
I'm afraid not. The pipe is a low-level OS concept, so it absolutely requires a file object that is represented by an OS-level file descriptor. Your workaround is the right one.
add a comment |
from subprocess import Popen, PIPE
from tempfile import SpooledTemporaryFile as tempfile
f = tempfile()
f.write('onentwonthreenfournfivensixn')
f.seek(0)
print Popen(['/bin/grep','f'],stdout=PIPE,stdin=f).stdout.read()
f.close()
3
fyi, tempfile.SpooledTemporaryFile.__doc__ says: Temporary file wrapper, specialized to switch from StringIO to a real file when it exceeds a certain size or when a fileno is needed.
– Doug F
Aug 9 '13 at 20:18
add a comment |
"""
Ex: Dialog (2-way) with a Popen()
"""
p = subprocess.Popen('Your Command Here',
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
stdin=PIPE,
shell=True,
bufsize=0)
p.stdin.write('STARTn')
out = p.stdout.readline()
while out:
line = out
line = line.rstrip("n")
if "WHATEVER1" in line:
pr = 1
p.stdin.write('DO 1n')
out = p.stdout.readline()
continue
if "WHATEVER2" in line:
pr = 2
p.stdin.write('DO 2n')
out = p.stdout.readline()
continue
"""
..........
"""
out = p.stdout.readline()
p.wait()
4
Becauseshell=True
is so commonly used for no good reason, and this is a popular question, let me point out that there are a lot of situations wherePopen(['cmd', 'with', 'args'])
is decidedly better thanPopen('cmd with args', shell=True)
and having the shell break the command and arguments into tokens, but not otherwise providing anything useful, while adding a significant amount of complexity and thus also attack surface.
– tripleee
Oct 29 '14 at 13:43
add a comment |
Beware that Popen.communicate(input=s)
may give you trouble ifs
is too big, because apparently the parent process will buffer it before forking the child subprocess, meaning it needs "twice as much" used memory at that point (at least according to the "under the hood" explanation and linked documentation found here). In my particular case,s
was a generator that was first fully expanded and only then written tostdin
so the parent process was huge right before the child was spawned,
and no memory was left to fork it:
File "/opt/local/stow/python-2.7.2/lib/python2.7/subprocess.py", line 1130, in _execute_child
self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory
add a comment |
p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
p.stdin.write('onen')
time.sleep(0.5)
p.stdin.write('twon')
time.sleep(0.5)
p.stdin.write('threen')
time.sleep(0.5)
testresult = p.communicate()[0]
time.sleep(0.5)
print(testresult)
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f163542%2fpython-how-do-i-pass-a-string-into-subprocess-popen-using-the-stdin-argument%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
10 Answers
10
active
oldest
votes
10 Answers
10
active
oldest
votes
active
oldest
votes
active
oldest
votes
Popen.communicate()
documentation:
Note that if you want to send data to
the process’s stdin, you need to
create the Popen object with
stdin=PIPE. Similarly, to get anything
other than None in the result tuple,
you need to give stdout=PIPE and/or
stderr=PIPE too.
Replacing os.popen*
pipe = os.popen(cmd, 'w', bufsize)
# ==>
pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin
Warning Use communicate() rather than
stdin.write(), stdout.read() or
stderr.read() to avoid deadlocks due
to any of the other OS pipe buffers
filling up and blocking the child
process.
So your example could be written as follows:
from subprocess import Popen, PIPE, STDOUT
p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
grep_stdout = p.communicate(input=b'onentwonthreenfournfivensixn')[0]
print(grep_stdout.decode())
# -> four
# -> five
# ->
On the current Python 3 version, you could use subprocess.run
, to pass input as a string to an external command and get its exit status, and its output as a string back in one call:
#!/usr/bin/env python3
from subprocess import run, PIPE
p = run(['grep', 'f'], stdout=PIPE,
input='onentwonthreenfournfivensixn', encoding='ascii')
print(p.returncode)
# -> 0
print(p.stdout)
# -> four
# -> five
# ->
3
I missed that warning. I'm glad I asked (even though I thought I had the answer).
– Daryl Spitzer
Oct 3 '08 at 16:02
11
This is NOT a good solution. In particular, you cannot asynchronously process p.stdout.readline output if you do this since you'd have to wait for the entire stdout to arrive. It's is also memory-inefficient.
– OTZ
Aug 20 '10 at 21:59
5
@OTZ What's a better solution?
– Nick T
Nov 17 '10 at 21:27
10
@Nick T: "better" depends on context. Newton's laws are good for the domain they are applicable but you need special relativity to design GPS. See Non-blocking read on a subprocess.PIPE in python.
– jfs
Oct 18 '11 at 20:25
9
But note the NOTE for communicate: "do not use this method if the data size is large or unlimited"
– Owen
Jan 22 '14 at 0:21
|
show 9 more comments
Popen.communicate()
documentation:
Note that if you want to send data to
the process’s stdin, you need to
create the Popen object with
stdin=PIPE. Similarly, to get anything
other than None in the result tuple,
you need to give stdout=PIPE and/or
stderr=PIPE too.
Replacing os.popen*
pipe = os.popen(cmd, 'w', bufsize)
# ==>
pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin
Warning Use communicate() rather than
stdin.write(), stdout.read() or
stderr.read() to avoid deadlocks due
to any of the other OS pipe buffers
filling up and blocking the child
process.
So your example could be written as follows:
from subprocess import Popen, PIPE, STDOUT
p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
grep_stdout = p.communicate(input=b'onentwonthreenfournfivensixn')[0]
print(grep_stdout.decode())
# -> four
# -> five
# ->
On the current Python 3 version, you could use subprocess.run
, to pass input as a string to an external command and get its exit status, and its output as a string back in one call:
#!/usr/bin/env python3
from subprocess import run, PIPE
p = run(['grep', 'f'], stdout=PIPE,
input='onentwonthreenfournfivensixn', encoding='ascii')
print(p.returncode)
# -> 0
print(p.stdout)
# -> four
# -> five
# ->
3
I missed that warning. I'm glad I asked (even though I thought I had the answer).
– Daryl Spitzer
Oct 3 '08 at 16:02
11
This is NOT a good solution. In particular, you cannot asynchronously process p.stdout.readline output if you do this since you'd have to wait for the entire stdout to arrive. It's is also memory-inefficient.
– OTZ
Aug 20 '10 at 21:59
5
@OTZ What's a better solution?
– Nick T
Nov 17 '10 at 21:27
10
@Nick T: "better" depends on context. Newton's laws are good for the domain they are applicable but you need special relativity to design GPS. See Non-blocking read on a subprocess.PIPE in python.
– jfs
Oct 18 '11 at 20:25
9
But note the NOTE for communicate: "do not use this method if the data size is large or unlimited"
– Owen
Jan 22 '14 at 0:21
|
show 9 more comments
Popen.communicate()
documentation:
Note that if you want to send data to
the process’s stdin, you need to
create the Popen object with
stdin=PIPE. Similarly, to get anything
other than None in the result tuple,
you need to give stdout=PIPE and/or
stderr=PIPE too.
Replacing os.popen*
pipe = os.popen(cmd, 'w', bufsize)
# ==>
pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin
Warning Use communicate() rather than
stdin.write(), stdout.read() or
stderr.read() to avoid deadlocks due
to any of the other OS pipe buffers
filling up and blocking the child
process.
So your example could be written as follows:
from subprocess import Popen, PIPE, STDOUT
p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
grep_stdout = p.communicate(input=b'onentwonthreenfournfivensixn')[0]
print(grep_stdout.decode())
# -> four
# -> five
# ->
On the current Python 3 version, you could use subprocess.run
, to pass input as a string to an external command and get its exit status, and its output as a string back in one call:
#!/usr/bin/env python3
from subprocess import run, PIPE
p = run(['grep', 'f'], stdout=PIPE,
input='onentwonthreenfournfivensixn', encoding='ascii')
print(p.returncode)
# -> 0
print(p.stdout)
# -> four
# -> five
# ->
Popen.communicate()
documentation:
Note that if you want to send data to
the process’s stdin, you need to
create the Popen object with
stdin=PIPE. Similarly, to get anything
other than None in the result tuple,
you need to give stdout=PIPE and/or
stderr=PIPE too.
Replacing os.popen*
pipe = os.popen(cmd, 'w', bufsize)
# ==>
pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin
Warning Use communicate() rather than
stdin.write(), stdout.read() or
stderr.read() to avoid deadlocks due
to any of the other OS pipe buffers
filling up and blocking the child
process.
So your example could be written as follows:
from subprocess import Popen, PIPE, STDOUT
p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
grep_stdout = p.communicate(input=b'onentwonthreenfournfivensixn')[0]
print(grep_stdout.decode())
# -> four
# -> five
# ->
On the current Python 3 version, you could use subprocess.run
, to pass input as a string to an external command and get its exit status, and its output as a string back in one call:
#!/usr/bin/env python3
from subprocess import run, PIPE
p = run(['grep', 'f'], stdout=PIPE,
input='onentwonthreenfournfivensixn', encoding='ascii')
print(p.returncode)
# -> 0
print(p.stdout)
# -> four
# -> five
# ->
edited Sep 13 '17 at 9:27
answered Oct 3 '08 at 4:11
jfsjfs
271k815721127
271k815721127
3
I missed that warning. I'm glad I asked (even though I thought I had the answer).
– Daryl Spitzer
Oct 3 '08 at 16:02
11
This is NOT a good solution. In particular, you cannot asynchronously process p.stdout.readline output if you do this since you'd have to wait for the entire stdout to arrive. It's is also memory-inefficient.
– OTZ
Aug 20 '10 at 21:59
5
@OTZ What's a better solution?
– Nick T
Nov 17 '10 at 21:27
10
@Nick T: "better" depends on context. Newton's laws are good for the domain they are applicable but you need special relativity to design GPS. See Non-blocking read on a subprocess.PIPE in python.
– jfs
Oct 18 '11 at 20:25
9
But note the NOTE for communicate: "do not use this method if the data size is large or unlimited"
– Owen
Jan 22 '14 at 0:21
|
show 9 more comments
3
I missed that warning. I'm glad I asked (even though I thought I had the answer).
– Daryl Spitzer
Oct 3 '08 at 16:02
11
This is NOT a good solution. In particular, you cannot asynchronously process p.stdout.readline output if you do this since you'd have to wait for the entire stdout to arrive. It's is also memory-inefficient.
– OTZ
Aug 20 '10 at 21:59
5
@OTZ What's a better solution?
– Nick T
Nov 17 '10 at 21:27
10
@Nick T: "better" depends on context. Newton's laws are good for the domain they are applicable but you need special relativity to design GPS. See Non-blocking read on a subprocess.PIPE in python.
– jfs
Oct 18 '11 at 20:25
9
But note the NOTE for communicate: "do not use this method if the data size is large or unlimited"
– Owen
Jan 22 '14 at 0:21
3
3
I missed that warning. I'm glad I asked (even though I thought I had the answer).
– Daryl Spitzer
Oct 3 '08 at 16:02
I missed that warning. I'm glad I asked (even though I thought I had the answer).
– Daryl Spitzer
Oct 3 '08 at 16:02
11
11
This is NOT a good solution. In particular, you cannot asynchronously process p.stdout.readline output if you do this since you'd have to wait for the entire stdout to arrive. It's is also memory-inefficient.
– OTZ
Aug 20 '10 at 21:59
This is NOT a good solution. In particular, you cannot asynchronously process p.stdout.readline output if you do this since you'd have to wait for the entire stdout to arrive. It's is also memory-inefficient.
– OTZ
Aug 20 '10 at 21:59
5
5
@OTZ What's a better solution?
– Nick T
Nov 17 '10 at 21:27
@OTZ What's a better solution?
– Nick T
Nov 17 '10 at 21:27
10
10
@Nick T: "better" depends on context. Newton's laws are good for the domain they are applicable but you need special relativity to design GPS. See Non-blocking read on a subprocess.PIPE in python.
– jfs
Oct 18 '11 at 20:25
@Nick T: "better" depends on context. Newton's laws are good for the domain they are applicable but you need special relativity to design GPS. See Non-blocking read on a subprocess.PIPE in python.
– jfs
Oct 18 '11 at 20:25
9
9
But note the NOTE for communicate: "do not use this method if the data size is large or unlimited"
– Owen
Jan 22 '14 at 0:21
But note the NOTE for communicate: "do not use this method if the data size is large or unlimited"
– Owen
Jan 22 '14 at 0:21
|
show 9 more comments
I figured out this workaround:
>>> p = subprocess.Popen(['grep','f'],stdout=subprocess.PIPE,stdin=subprocess.PIPE)
>>> p.stdin.write(b'onentwonthreenfournfivensixn') #expects a bytes type object
>>> p.communicate()[0]
'fournfiven'
>>> p.stdin.close()
Is there a better one?
8
That's not a workaround - that's the correct way to do it!
– Moe
Oct 2 '08 at 18:10
21
@Moe:stdin.write()
usage is discouraged,p.communicate()
should be used. See my answer.
– jfs
Oct 3 '08 at 14:26
9
Per the subprocess documentation: Warning - Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.
– Jason Mock
Aug 25 '10 at 20:49
1
I think this is good way to do it if you're confident that your stdout/err won't ever fill up (for instance, it's going to a file, or another thread is eating it) and you have an unbounded amount of data to be sent to stdin.
– Lucretiel
May 9 '16 at 19:24
1
In particular, doing it this way still ensures that stdin is closed, so that if the subprocesses is one that consumes input forever, thecommunicate
will close the pipe and allow the process to end gracefully.
– Lucretiel
May 9 '16 at 19:25
add a comment |
I figured out this workaround:
>>> p = subprocess.Popen(['grep','f'],stdout=subprocess.PIPE,stdin=subprocess.PIPE)
>>> p.stdin.write(b'onentwonthreenfournfivensixn') #expects a bytes type object
>>> p.communicate()[0]
'fournfiven'
>>> p.stdin.close()
Is there a better one?
8
That's not a workaround - that's the correct way to do it!
– Moe
Oct 2 '08 at 18:10
21
@Moe:stdin.write()
usage is discouraged,p.communicate()
should be used. See my answer.
– jfs
Oct 3 '08 at 14:26
9
Per the subprocess documentation: Warning - Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.
– Jason Mock
Aug 25 '10 at 20:49
1
I think this is good way to do it if you're confident that your stdout/err won't ever fill up (for instance, it's going to a file, or another thread is eating it) and you have an unbounded amount of data to be sent to stdin.
– Lucretiel
May 9 '16 at 19:24
1
In particular, doing it this way still ensures that stdin is closed, so that if the subprocesses is one that consumes input forever, thecommunicate
will close the pipe and allow the process to end gracefully.
– Lucretiel
May 9 '16 at 19:25
add a comment |
I figured out this workaround:
>>> p = subprocess.Popen(['grep','f'],stdout=subprocess.PIPE,stdin=subprocess.PIPE)
>>> p.stdin.write(b'onentwonthreenfournfivensixn') #expects a bytes type object
>>> p.communicate()[0]
'fournfiven'
>>> p.stdin.close()
Is there a better one?
I figured out this workaround:
>>> p = subprocess.Popen(['grep','f'],stdout=subprocess.PIPE,stdin=subprocess.PIPE)
>>> p.stdin.write(b'onentwonthreenfournfivensixn') #expects a bytes type object
>>> p.communicate()[0]
'fournfiven'
>>> p.stdin.close()
Is there a better one?
edited Jan 18 '18 at 0:31
tobsecret
1,384516
1,384516
answered Oct 2 '08 at 17:27
Daryl SpitzerDaryl Spitzer
53.5k60144162
53.5k60144162
8
That's not a workaround - that's the correct way to do it!
– Moe
Oct 2 '08 at 18:10
21
@Moe:stdin.write()
usage is discouraged,p.communicate()
should be used. See my answer.
– jfs
Oct 3 '08 at 14:26
9
Per the subprocess documentation: Warning - Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.
– Jason Mock
Aug 25 '10 at 20:49
1
I think this is good way to do it if you're confident that your stdout/err won't ever fill up (for instance, it's going to a file, or another thread is eating it) and you have an unbounded amount of data to be sent to stdin.
– Lucretiel
May 9 '16 at 19:24
1
In particular, doing it this way still ensures that stdin is closed, so that if the subprocesses is one that consumes input forever, thecommunicate
will close the pipe and allow the process to end gracefully.
– Lucretiel
May 9 '16 at 19:25
add a comment |
8
That's not a workaround - that's the correct way to do it!
– Moe
Oct 2 '08 at 18:10
21
@Moe:stdin.write()
usage is discouraged,p.communicate()
should be used. See my answer.
– jfs
Oct 3 '08 at 14:26
9
Per the subprocess documentation: Warning - Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.
– Jason Mock
Aug 25 '10 at 20:49
1
I think this is good way to do it if you're confident that your stdout/err won't ever fill up (for instance, it's going to a file, or another thread is eating it) and you have an unbounded amount of data to be sent to stdin.
– Lucretiel
May 9 '16 at 19:24
1
In particular, doing it this way still ensures that stdin is closed, so that if the subprocesses is one that consumes input forever, thecommunicate
will close the pipe and allow the process to end gracefully.
– Lucretiel
May 9 '16 at 19:25
8
8
That's not a workaround - that's the correct way to do it!
– Moe
Oct 2 '08 at 18:10
That's not a workaround - that's the correct way to do it!
– Moe
Oct 2 '08 at 18:10
21
21
@Moe:
stdin.write()
usage is discouraged, p.communicate()
should be used. See my answer.– jfs
Oct 3 '08 at 14:26
@Moe:
stdin.write()
usage is discouraged, p.communicate()
should be used. See my answer.– jfs
Oct 3 '08 at 14:26
9
9
Per the subprocess documentation: Warning - Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.
– Jason Mock
Aug 25 '10 at 20:49
Per the subprocess documentation: Warning - Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.
– Jason Mock
Aug 25 '10 at 20:49
1
1
I think this is good way to do it if you're confident that your stdout/err won't ever fill up (for instance, it's going to a file, or another thread is eating it) and you have an unbounded amount of data to be sent to stdin.
– Lucretiel
May 9 '16 at 19:24
I think this is good way to do it if you're confident that your stdout/err won't ever fill up (for instance, it's going to a file, or another thread is eating it) and you have an unbounded amount of data to be sent to stdin.
– Lucretiel
May 9 '16 at 19:24
1
1
In particular, doing it this way still ensures that stdin is closed, so that if the subprocesses is one that consumes input forever, the
communicate
will close the pipe and allow the process to end gracefully.– Lucretiel
May 9 '16 at 19:25
In particular, doing it this way still ensures that stdin is closed, so that if the subprocesses is one that consumes input forever, the
communicate
will close the pipe and allow the process to end gracefully.– Lucretiel
May 9 '16 at 19:25
add a comment |
I'm a bit surprised nobody suggested creating a pipe, which is in my opinion the far simplest way to pass a string to stdin of a subprocess:
read, write = os.pipe()
os.write(write, "stdin input here")
os.close(write)
subprocess.check_call(['your-command'], stdin=read)
2
Theos
and thesubprocess
documentation both agree that you should prefer the latter over the former. This is a legacy solution which has a (slightly less concise) standard replacement; the accepted answer quotes the pertinent documentation.
– tripleee
May 4 '16 at 11:53
1
I'm not sure that's correct, tripleee. The quoted documentation says why it is hard to use the pipes created by the process, but in this solution it creates a pipe and passes it in. I believe it avoids the potential deadlock problems of managing the pipes after the process has already started.
– Graham Christensen
May 7 '16 at 15:09
os.popen is deprecated in favour of subprocess
– hd1
Feb 18 '17 at 17:30
2
-1: it leads to the deadlock, it may loose data. This functionality is already provided by the subprocess module. Use it instead of reimplementing it poorly (try to write a value that is larger than an OS pipe buffer)
– jfs
Aug 18 '17 at 6:24
add a comment |
I'm a bit surprised nobody suggested creating a pipe, which is in my opinion the far simplest way to pass a string to stdin of a subprocess:
read, write = os.pipe()
os.write(write, "stdin input here")
os.close(write)
subprocess.check_call(['your-command'], stdin=read)
2
Theos
and thesubprocess
documentation both agree that you should prefer the latter over the former. This is a legacy solution which has a (slightly less concise) standard replacement; the accepted answer quotes the pertinent documentation.
– tripleee
May 4 '16 at 11:53
1
I'm not sure that's correct, tripleee. The quoted documentation says why it is hard to use the pipes created by the process, but in this solution it creates a pipe and passes it in. I believe it avoids the potential deadlock problems of managing the pipes after the process has already started.
– Graham Christensen
May 7 '16 at 15:09
os.popen is deprecated in favour of subprocess
– hd1
Feb 18 '17 at 17:30
2
-1: it leads to the deadlock, it may loose data. This functionality is already provided by the subprocess module. Use it instead of reimplementing it poorly (try to write a value that is larger than an OS pipe buffer)
– jfs
Aug 18 '17 at 6:24
add a comment |
I'm a bit surprised nobody suggested creating a pipe, which is in my opinion the far simplest way to pass a string to stdin of a subprocess:
read, write = os.pipe()
os.write(write, "stdin input here")
os.close(write)
subprocess.check_call(['your-command'], stdin=read)
I'm a bit surprised nobody suggested creating a pipe, which is in my opinion the far simplest way to pass a string to stdin of a subprocess:
read, write = os.pipe()
os.write(write, "stdin input here")
os.close(write)
subprocess.check_call(['your-command'], stdin=read)
answered Nov 2 '15 at 16:34
Graham ChristensenGraham Christensen
68456
68456
2
Theos
and thesubprocess
documentation both agree that you should prefer the latter over the former. This is a legacy solution which has a (slightly less concise) standard replacement; the accepted answer quotes the pertinent documentation.
– tripleee
May 4 '16 at 11:53
1
I'm not sure that's correct, tripleee. The quoted documentation says why it is hard to use the pipes created by the process, but in this solution it creates a pipe and passes it in. I believe it avoids the potential deadlock problems of managing the pipes after the process has already started.
– Graham Christensen
May 7 '16 at 15:09
os.popen is deprecated in favour of subprocess
– hd1
Feb 18 '17 at 17:30
2
-1: it leads to the deadlock, it may loose data. This functionality is already provided by the subprocess module. Use it instead of reimplementing it poorly (try to write a value that is larger than an OS pipe buffer)
– jfs
Aug 18 '17 at 6:24
add a comment |
2
Theos
and thesubprocess
documentation both agree that you should prefer the latter over the former. This is a legacy solution which has a (slightly less concise) standard replacement; the accepted answer quotes the pertinent documentation.
– tripleee
May 4 '16 at 11:53
1
I'm not sure that's correct, tripleee. The quoted documentation says why it is hard to use the pipes created by the process, but in this solution it creates a pipe and passes it in. I believe it avoids the potential deadlock problems of managing the pipes after the process has already started.
– Graham Christensen
May 7 '16 at 15:09
os.popen is deprecated in favour of subprocess
– hd1
Feb 18 '17 at 17:30
2
-1: it leads to the deadlock, it may loose data. This functionality is already provided by the subprocess module. Use it instead of reimplementing it poorly (try to write a value that is larger than an OS pipe buffer)
– jfs
Aug 18 '17 at 6:24
2
2
The
os
and the subprocess
documentation both agree that you should prefer the latter over the former. This is a legacy solution which has a (slightly less concise) standard replacement; the accepted answer quotes the pertinent documentation.– tripleee
May 4 '16 at 11:53
The
os
and the subprocess
documentation both agree that you should prefer the latter over the former. This is a legacy solution which has a (slightly less concise) standard replacement; the accepted answer quotes the pertinent documentation.– tripleee
May 4 '16 at 11:53
1
1
I'm not sure that's correct, tripleee. The quoted documentation says why it is hard to use the pipes created by the process, but in this solution it creates a pipe and passes it in. I believe it avoids the potential deadlock problems of managing the pipes after the process has already started.
– Graham Christensen
May 7 '16 at 15:09
I'm not sure that's correct, tripleee. The quoted documentation says why it is hard to use the pipes created by the process, but in this solution it creates a pipe and passes it in. I believe it avoids the potential deadlock problems of managing the pipes after the process has already started.
– Graham Christensen
May 7 '16 at 15:09
os.popen is deprecated in favour of subprocess
– hd1
Feb 18 '17 at 17:30
os.popen is deprecated in favour of subprocess
– hd1
Feb 18 '17 at 17:30
2
2
-1: it leads to the deadlock, it may loose data. This functionality is already provided by the subprocess module. Use it instead of reimplementing it poorly (try to write a value that is larger than an OS pipe buffer)
– jfs
Aug 18 '17 at 6:24
-1: it leads to the deadlock, it may loose data. This functionality is already provided by the subprocess module. Use it instead of reimplementing it poorly (try to write a value that is larger than an OS pipe buffer)
– jfs
Aug 18 '17 at 6:24
add a comment |
There's a beatiful solution if you're using Python 3.4 or better. Use the input
argument instead of the stdin
argument, which accepts a bytes argument:
output = subprocess.check_output(
["sed", "s/foo/bar/"],
input=b"foo",
)
1
does not work forcall
– vidstige
Oct 5 '17 at 13:46
3
@vidstige You're right, that's weird. I would consider filing this as an Python bug, I don't see any good reason in whycheck_output
should have aninput
argument, but notcall
.
– Flimm
Oct 5 '17 at 14:35
This is the best answer for Python 3.4+ (using it in Python 3.6). It indeed does not work withcheck_call
but it works forrun
. It also works with input=string as long as you pass an encoding argument too according to the documentation.
– Nikolaos Georgiou
Feb 22 at 2:48
add a comment |
There's a beatiful solution if you're using Python 3.4 or better. Use the input
argument instead of the stdin
argument, which accepts a bytes argument:
output = subprocess.check_output(
["sed", "s/foo/bar/"],
input=b"foo",
)
1
does not work forcall
– vidstige
Oct 5 '17 at 13:46
3
@vidstige You're right, that's weird. I would consider filing this as an Python bug, I don't see any good reason in whycheck_output
should have aninput
argument, but notcall
.
– Flimm
Oct 5 '17 at 14:35
This is the best answer for Python 3.4+ (using it in Python 3.6). It indeed does not work withcheck_call
but it works forrun
. It also works with input=string as long as you pass an encoding argument too according to the documentation.
– Nikolaos Georgiou
Feb 22 at 2:48
add a comment |
There's a beatiful solution if you're using Python 3.4 or better. Use the input
argument instead of the stdin
argument, which accepts a bytes argument:
output = subprocess.check_output(
["sed", "s/foo/bar/"],
input=b"foo",
)
There's a beatiful solution if you're using Python 3.4 or better. Use the input
argument instead of the stdin
argument, which accepts a bytes argument:
output = subprocess.check_output(
["sed", "s/foo/bar/"],
input=b"foo",
)
answered Dec 8 '16 at 10:04
FlimmFlimm
55.1k23141162
55.1k23141162
1
does not work forcall
– vidstige
Oct 5 '17 at 13:46
3
@vidstige You're right, that's weird. I would consider filing this as an Python bug, I don't see any good reason in whycheck_output
should have aninput
argument, but notcall
.
– Flimm
Oct 5 '17 at 14:35
This is the best answer for Python 3.4+ (using it in Python 3.6). It indeed does not work withcheck_call
but it works forrun
. It also works with input=string as long as you pass an encoding argument too according to the documentation.
– Nikolaos Georgiou
Feb 22 at 2:48
add a comment |
1
does not work forcall
– vidstige
Oct 5 '17 at 13:46
3
@vidstige You're right, that's weird. I would consider filing this as an Python bug, I don't see any good reason in whycheck_output
should have aninput
argument, but notcall
.
– Flimm
Oct 5 '17 at 14:35
This is the best answer for Python 3.4+ (using it in Python 3.6). It indeed does not work withcheck_call
but it works forrun
. It also works with input=string as long as you pass an encoding argument too according to the documentation.
– Nikolaos Georgiou
Feb 22 at 2:48
1
1
does not work for
call
– vidstige
Oct 5 '17 at 13:46
does not work for
call
– vidstige
Oct 5 '17 at 13:46
3
3
@vidstige You're right, that's weird. I would consider filing this as an Python bug, I don't see any good reason in why
check_output
should have an input
argument, but not call
.– Flimm
Oct 5 '17 at 14:35
@vidstige You're right, that's weird. I would consider filing this as an Python bug, I don't see any good reason in why
check_output
should have an input
argument, but not call
.– Flimm
Oct 5 '17 at 14:35
This is the best answer for Python 3.4+ (using it in Python 3.6). It indeed does not work with
check_call
but it works for run
. It also works with input=string as long as you pass an encoding argument too according to the documentation.– Nikolaos Georgiou
Feb 22 at 2:48
This is the best answer for Python 3.4+ (using it in Python 3.6). It indeed does not work with
check_call
but it works for run
. It also works with input=string as long as you pass an encoding argument too according to the documentation.– Nikolaos Georgiou
Feb 22 at 2:48
add a comment |
I am using python3 and found out that you need to encode your string before you can pass it into stdin:
p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=PIPE)
out, err = p.communicate(input='onentwonthreenfournfivensixn'.encode())
print(out)
4
You don't specifically need to encode the input, it just wants a bytes-like object (e.g.b'something'
). It will return err and out as bytes also. If you want to avoid this, you can passuniversal_newlines=True
toPopen
. Then it will accept input as str and will return err/out as str also.
– Six
Jan 17 '16 at 15:00
2
But beware,universal_newlines=True
will also convert your newlines to match your system
– Nacht
Feb 25 '16 at 12:35
1
If you're using Python 3, see my answer for an even more convenient solution.
– Flimm
Dec 8 '16 at 10:05
add a comment |
I am using python3 and found out that you need to encode your string before you can pass it into stdin:
p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=PIPE)
out, err = p.communicate(input='onentwonthreenfournfivensixn'.encode())
print(out)
4
You don't specifically need to encode the input, it just wants a bytes-like object (e.g.b'something'
). It will return err and out as bytes also. If you want to avoid this, you can passuniversal_newlines=True
toPopen
. Then it will accept input as str and will return err/out as str also.
– Six
Jan 17 '16 at 15:00
2
But beware,universal_newlines=True
will also convert your newlines to match your system
– Nacht
Feb 25 '16 at 12:35
1
If you're using Python 3, see my answer for an even more convenient solution.
– Flimm
Dec 8 '16 at 10:05
add a comment |
I am using python3 and found out that you need to encode your string before you can pass it into stdin:
p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=PIPE)
out, err = p.communicate(input='onentwonthreenfournfivensixn'.encode())
print(out)
I am using python3 and found out that you need to encode your string before you can pass it into stdin:
p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=PIPE)
out, err = p.communicate(input='onentwonthreenfournfivensixn'.encode())
print(out)
answered Jul 27 '14 at 15:29
qedqed
10.1k1269119
10.1k1269119
4
You don't specifically need to encode the input, it just wants a bytes-like object (e.g.b'something'
). It will return err and out as bytes also. If you want to avoid this, you can passuniversal_newlines=True
toPopen
. Then it will accept input as str and will return err/out as str also.
– Six
Jan 17 '16 at 15:00
2
But beware,universal_newlines=True
will also convert your newlines to match your system
– Nacht
Feb 25 '16 at 12:35
1
If you're using Python 3, see my answer for an even more convenient solution.
– Flimm
Dec 8 '16 at 10:05
add a comment |
4
You don't specifically need to encode the input, it just wants a bytes-like object (e.g.b'something'
). It will return err and out as bytes also. If you want to avoid this, you can passuniversal_newlines=True
toPopen
. Then it will accept input as str and will return err/out as str also.
– Six
Jan 17 '16 at 15:00
2
But beware,universal_newlines=True
will also convert your newlines to match your system
– Nacht
Feb 25 '16 at 12:35
1
If you're using Python 3, see my answer for an even more convenient solution.
– Flimm
Dec 8 '16 at 10:05
4
4
You don't specifically need to encode the input, it just wants a bytes-like object (e.g.
b'something'
). It will return err and out as bytes also. If you want to avoid this, you can pass universal_newlines=True
to Popen
. Then it will accept input as str and will return err/out as str also.– Six
Jan 17 '16 at 15:00
You don't specifically need to encode the input, it just wants a bytes-like object (e.g.
b'something'
). It will return err and out as bytes also. If you want to avoid this, you can pass universal_newlines=True
to Popen
. Then it will accept input as str and will return err/out as str also.– Six
Jan 17 '16 at 15:00
2
2
But beware,
universal_newlines=True
will also convert your newlines to match your system– Nacht
Feb 25 '16 at 12:35
But beware,
universal_newlines=True
will also convert your newlines to match your system– Nacht
Feb 25 '16 at 12:35
1
1
If you're using Python 3, see my answer for an even more convenient solution.
– Flimm
Dec 8 '16 at 10:05
If you're using Python 3, see my answer for an even more convenient solution.
– Flimm
Dec 8 '16 at 10:05
add a comment |
"Apparently a cStringIO.StringIO object doesn't quack close enough to a file duck to suit subprocess.Popen"
:-)
I'm afraid not. The pipe is a low-level OS concept, so it absolutely requires a file object that is represented by an OS-level file descriptor. Your workaround is the right one.
add a comment |
"Apparently a cStringIO.StringIO object doesn't quack close enough to a file duck to suit subprocess.Popen"
:-)
I'm afraid not. The pipe is a low-level OS concept, so it absolutely requires a file object that is represented by an OS-level file descriptor. Your workaround is the right one.
add a comment |
"Apparently a cStringIO.StringIO object doesn't quack close enough to a file duck to suit subprocess.Popen"
:-)
I'm afraid not. The pipe is a low-level OS concept, so it absolutely requires a file object that is represented by an OS-level file descriptor. Your workaround is the right one.
"Apparently a cStringIO.StringIO object doesn't quack close enough to a file duck to suit subprocess.Popen"
:-)
I'm afraid not. The pipe is a low-level OS concept, so it absolutely requires a file object that is represented by an OS-level file descriptor. Your workaround is the right one.
answered Oct 2 '08 at 18:33
Dan LenskiDan Lenski
50.8k95695
50.8k95695
add a comment |
add a comment |
from subprocess import Popen, PIPE
from tempfile import SpooledTemporaryFile as tempfile
f = tempfile()
f.write('onentwonthreenfournfivensixn')
f.seek(0)
print Popen(['/bin/grep','f'],stdout=PIPE,stdin=f).stdout.read()
f.close()
3
fyi, tempfile.SpooledTemporaryFile.__doc__ says: Temporary file wrapper, specialized to switch from StringIO to a real file when it exceeds a certain size or when a fileno is needed.
– Doug F
Aug 9 '13 at 20:18
add a comment |
from subprocess import Popen, PIPE
from tempfile import SpooledTemporaryFile as tempfile
f = tempfile()
f.write('onentwonthreenfournfivensixn')
f.seek(0)
print Popen(['/bin/grep','f'],stdout=PIPE,stdin=f).stdout.read()
f.close()
3
fyi, tempfile.SpooledTemporaryFile.__doc__ says: Temporary file wrapper, specialized to switch from StringIO to a real file when it exceeds a certain size or when a fileno is needed.
– Doug F
Aug 9 '13 at 20:18
add a comment |
from subprocess import Popen, PIPE
from tempfile import SpooledTemporaryFile as tempfile
f = tempfile()
f.write('onentwonthreenfournfivensixn')
f.seek(0)
print Popen(['/bin/grep','f'],stdout=PIPE,stdin=f).stdout.read()
f.close()
from subprocess import Popen, PIPE
from tempfile import SpooledTemporaryFile as tempfile
f = tempfile()
f.write('onentwonthreenfournfivensixn')
f.seek(0)
print Popen(['/bin/grep','f'],stdout=PIPE,stdin=f).stdout.read()
f.close()
answered Apr 13 '12 at 3:36
Michael WaddellMichael Waddell
8912
8912
3
fyi, tempfile.SpooledTemporaryFile.__doc__ says: Temporary file wrapper, specialized to switch from StringIO to a real file when it exceeds a certain size or when a fileno is needed.
– Doug F
Aug 9 '13 at 20:18
add a comment |
3
fyi, tempfile.SpooledTemporaryFile.__doc__ says: Temporary file wrapper, specialized to switch from StringIO to a real file when it exceeds a certain size or when a fileno is needed.
– Doug F
Aug 9 '13 at 20:18
3
3
fyi, tempfile.SpooledTemporaryFile.__doc__ says: Temporary file wrapper, specialized to switch from StringIO to a real file when it exceeds a certain size or when a fileno is needed.
– Doug F
Aug 9 '13 at 20:18
fyi, tempfile.SpooledTemporaryFile.__doc__ says: Temporary file wrapper, specialized to switch from StringIO to a real file when it exceeds a certain size or when a fileno is needed.
– Doug F
Aug 9 '13 at 20:18
add a comment |
"""
Ex: Dialog (2-way) with a Popen()
"""
p = subprocess.Popen('Your Command Here',
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
stdin=PIPE,
shell=True,
bufsize=0)
p.stdin.write('STARTn')
out = p.stdout.readline()
while out:
line = out
line = line.rstrip("n")
if "WHATEVER1" in line:
pr = 1
p.stdin.write('DO 1n')
out = p.stdout.readline()
continue
if "WHATEVER2" in line:
pr = 2
p.stdin.write('DO 2n')
out = p.stdout.readline()
continue
"""
..........
"""
out = p.stdout.readline()
p.wait()
4
Becauseshell=True
is so commonly used for no good reason, and this is a popular question, let me point out that there are a lot of situations wherePopen(['cmd', 'with', 'args'])
is decidedly better thanPopen('cmd with args', shell=True)
and having the shell break the command and arguments into tokens, but not otherwise providing anything useful, while adding a significant amount of complexity and thus also attack surface.
– tripleee
Oct 29 '14 at 13:43
add a comment |
"""
Ex: Dialog (2-way) with a Popen()
"""
p = subprocess.Popen('Your Command Here',
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
stdin=PIPE,
shell=True,
bufsize=0)
p.stdin.write('STARTn')
out = p.stdout.readline()
while out:
line = out
line = line.rstrip("n")
if "WHATEVER1" in line:
pr = 1
p.stdin.write('DO 1n')
out = p.stdout.readline()
continue
if "WHATEVER2" in line:
pr = 2
p.stdin.write('DO 2n')
out = p.stdout.readline()
continue
"""
..........
"""
out = p.stdout.readline()
p.wait()
4
Becauseshell=True
is so commonly used for no good reason, and this is a popular question, let me point out that there are a lot of situations wherePopen(['cmd', 'with', 'args'])
is decidedly better thanPopen('cmd with args', shell=True)
and having the shell break the command and arguments into tokens, but not otherwise providing anything useful, while adding a significant amount of complexity and thus also attack surface.
– tripleee
Oct 29 '14 at 13:43
add a comment |
"""
Ex: Dialog (2-way) with a Popen()
"""
p = subprocess.Popen('Your Command Here',
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
stdin=PIPE,
shell=True,
bufsize=0)
p.stdin.write('STARTn')
out = p.stdout.readline()
while out:
line = out
line = line.rstrip("n")
if "WHATEVER1" in line:
pr = 1
p.stdin.write('DO 1n')
out = p.stdout.readline()
continue
if "WHATEVER2" in line:
pr = 2
p.stdin.write('DO 2n')
out = p.stdout.readline()
continue
"""
..........
"""
out = p.stdout.readline()
p.wait()
"""
Ex: Dialog (2-way) with a Popen()
"""
p = subprocess.Popen('Your Command Here',
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
stdin=PIPE,
shell=True,
bufsize=0)
p.stdin.write('STARTn')
out = p.stdout.readline()
while out:
line = out
line = line.rstrip("n")
if "WHATEVER1" in line:
pr = 1
p.stdin.write('DO 1n')
out = p.stdout.readline()
continue
if "WHATEVER2" in line:
pr = 2
p.stdin.write('DO 2n')
out = p.stdout.readline()
continue
"""
..........
"""
out = p.stdout.readline()
p.wait()
edited Oct 8 '14 at 16:15
Arne Babenhauserheide
1,9722220
1,9722220
answered Jun 14 '13 at 13:20
Lucien HercaudLucien Hercaud
7911
7911
4
Becauseshell=True
is so commonly used for no good reason, and this is a popular question, let me point out that there are a lot of situations wherePopen(['cmd', 'with', 'args'])
is decidedly better thanPopen('cmd with args', shell=True)
and having the shell break the command and arguments into tokens, but not otherwise providing anything useful, while adding a significant amount of complexity and thus also attack surface.
– tripleee
Oct 29 '14 at 13:43
add a comment |
4
Becauseshell=True
is so commonly used for no good reason, and this is a popular question, let me point out that there are a lot of situations wherePopen(['cmd', 'with', 'args'])
is decidedly better thanPopen('cmd with args', shell=True)
and having the shell break the command and arguments into tokens, but not otherwise providing anything useful, while adding a significant amount of complexity and thus also attack surface.
– tripleee
Oct 29 '14 at 13:43
4
4
Because
shell=True
is so commonly used for no good reason, and this is a popular question, let me point out that there are a lot of situations where Popen(['cmd', 'with', 'args'])
is decidedly better than Popen('cmd with args', shell=True)
and having the shell break the command and arguments into tokens, but not otherwise providing anything useful, while adding a significant amount of complexity and thus also attack surface.– tripleee
Oct 29 '14 at 13:43
Because
shell=True
is so commonly used for no good reason, and this is a popular question, let me point out that there are a lot of situations where Popen(['cmd', 'with', 'args'])
is decidedly better than Popen('cmd with args', shell=True)
and having the shell break the command and arguments into tokens, but not otherwise providing anything useful, while adding a significant amount of complexity and thus also attack surface.– tripleee
Oct 29 '14 at 13:43
add a comment |
Beware that Popen.communicate(input=s)
may give you trouble ifs
is too big, because apparently the parent process will buffer it before forking the child subprocess, meaning it needs "twice as much" used memory at that point (at least according to the "under the hood" explanation and linked documentation found here). In my particular case,s
was a generator that was first fully expanded and only then written tostdin
so the parent process was huge right before the child was spawned,
and no memory was left to fork it:
File "/opt/local/stow/python-2.7.2/lib/python2.7/subprocess.py", line 1130, in _execute_child
self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory
add a comment |
Beware that Popen.communicate(input=s)
may give you trouble ifs
is too big, because apparently the parent process will buffer it before forking the child subprocess, meaning it needs "twice as much" used memory at that point (at least according to the "under the hood" explanation and linked documentation found here). In my particular case,s
was a generator that was first fully expanded and only then written tostdin
so the parent process was huge right before the child was spawned,
and no memory was left to fork it:
File "/opt/local/stow/python-2.7.2/lib/python2.7/subprocess.py", line 1130, in _execute_child
self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory
add a comment |
Beware that Popen.communicate(input=s)
may give you trouble ifs
is too big, because apparently the parent process will buffer it before forking the child subprocess, meaning it needs "twice as much" used memory at that point (at least according to the "under the hood" explanation and linked documentation found here). In my particular case,s
was a generator that was first fully expanded and only then written tostdin
so the parent process was huge right before the child was spawned,
and no memory was left to fork it:
File "/opt/local/stow/python-2.7.2/lib/python2.7/subprocess.py", line 1130, in _execute_child
self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory
Beware that Popen.communicate(input=s)
may give you trouble ifs
is too big, because apparently the parent process will buffer it before forking the child subprocess, meaning it needs "twice as much" used memory at that point (at least according to the "under the hood" explanation and linked documentation found here). In my particular case,s
was a generator that was first fully expanded and only then written tostdin
so the parent process was huge right before the child was spawned,
and no memory was left to fork it:
File "/opt/local/stow/python-2.7.2/lib/python2.7/subprocess.py", line 1130, in _execute_child
self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory
edited May 23 '17 at 11:33
Community♦
11
11
answered May 19 '14 at 14:56
Lord Henry WottonLord Henry Wotton
1,044810
1,044810
add a comment |
add a comment |
p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
p.stdin.write('onen')
time.sleep(0.5)
p.stdin.write('twon')
time.sleep(0.5)
p.stdin.write('threen')
time.sleep(0.5)
testresult = p.communicate()[0]
time.sleep(0.5)
print(testresult)
add a comment |
p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
p.stdin.write('onen')
time.sleep(0.5)
p.stdin.write('twon')
time.sleep(0.5)
p.stdin.write('threen')
time.sleep(0.5)
testresult = p.communicate()[0]
time.sleep(0.5)
print(testresult)
add a comment |
p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
p.stdin.write('onen')
time.sleep(0.5)
p.stdin.write('twon')
time.sleep(0.5)
p.stdin.write('threen')
time.sleep(0.5)
testresult = p.communicate()[0]
time.sleep(0.5)
print(testresult)
p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
p.stdin.write('onen')
time.sleep(0.5)
p.stdin.write('twon')
time.sleep(0.5)
p.stdin.write('threen')
time.sleep(0.5)
testresult = p.communicate()[0]
time.sleep(0.5)
print(testresult)
edited Jun 27 '12 at 14:01
dusan
7,53822753
7,53822753
answered Apr 9 '09 at 4:39
gedwarp
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f163542%2fpython-how-do-i-pass-a-string-into-subprocess-popen-using-the-stdin-argument%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
3
Instead of disputing my answer with this being deleted, I'm adding it as a comment... Recommended reading: Doug Hellmann's Python Module of the Week blog post on subprocess.
– Daryl Spitzer
Jun 18 '13 at 22:43
3
the blog post contains multiple errors e.g., the very first code example:
call(['ls', '-1'], shell=True)
is incorrect. I recommend to read common questions from subprocess' tag description instead. In particular, Why subprocess.Popen doesn't work when args is sequence? explains whycall(['ls', '-1'], shell=True)
is wrong. I remember leaving comments under the blog post but I don't see them now for some reason.– jfs
Mar 17 '16 at 14:24