In Python, the subprocess module allows you to execute linux/unix commands directly from Python. The official Python documentation is very useful would you need to go further. Yet it may be a bit scary for newbies. Syntax may also change from one Python version to the other.
Here is a solution to use subprocess to encode piped commands with Python 3.7. For example imagine you want to encode this command that contains a pipe and a redirection:
zcat file1.dat.gz file2.dat.gz | pigz > out.gz
Here is python example on how to implement this unix command using Python subprocess. The first call to the subprocess encodes the pipe. The second command encodes the redirection:
p1 = subprocess.Popen(["zcat", "file1.dat.gz", "file2.dat.gz"], stdout=subprocess.PIPE) fout = open('out.gz', 'wb') p2 = subprocess.run(['pigz'], stdin=p1.stdout, stdout=fout)
Note that the first subprocess uses the Popen method while the second call uses the run method. If you use the run() method for the pipe itself,
p1 = subprocess.run(['zcat', 'file1.dat.gz', 'file2.dat.gz'], stdout=subprocess.PIPE) fout = open('out.gz', 'wb') p2 = subprocess.run(['pigz'], stdin=p1.stdout, stdout=fout)
you would get this kind of error:
AttributeError: 'bytes' object has no attribute 'fileno'
One Response to How to use subprocess with PIPEs