• mtphr
  • New Member
  • Offline
  • Registered: December 13, 2011
  • Posts: 4

Topic: Reading subprocess output as a File

Hi everyone, I can't find a solution to (what seems to be) a simple problem.

The output of a program is a ppm image file. I tell the program to write the ppm to stdout. I redirect the stdout to a variable in Python with the subprocess module. Everything here is fine.

But then I want to handle it via another library (called netpbm)
The problem is, the library takes input as filename, but it is the exactly same thing in the variable. So isn't there a way to point the variable as a file???

Maybe an example would be better :

import subprocess, netpbm
# the "-c" tells dcraw to write to stdout, NOT to file.ppm
p = subprocess.Popen(["dcraw","-c","file.cr2"],stdout=subprocess.PIPE).communicate()[0]
# Here in the varible p we have the file.
# Normally, "$ dcraw -c file.cr2" will printout the bytes of the ppm file.
# Now i want to open it ("p") like a regular file.
p_arr = netpbm.asarray(p)
# This last line of course doesn't work, cuz the argument passed to .asarray has to be a file!

Please don't say that it has anything to do with the netpbm library I'm using. My problem is, I want the "p" variable to act as a file somehow, for any other function too, of course without writing it to an actual file.

Hope I made myself clear.

Thumbs up

  • Registered: December 21, 2011
  • Posts: 32

Re: Reading subprocess output as a File

mtphr wrote:

The problem is, the library takes input as filename

open help(netpbm)
maybe it has a special function
anyway you cannot pass file contents instead of filename

Thumbs up

  • mtphr
  • New Member
  • Offline
  • Registered: December 13, 2011
  • Posts: 4

Re: Reading subprocess output as a File

struct.h wrote:

anyway you cannot pass file contents instead of filename

i know i can't pass file contents instead of filename.
what i'm trying to do is pass the file contents as a file pointer.

a simple file I/O reads the file "into the memory" and does the processing there.
What I want is to do the processing with "what's already on the memory" to avoid an extra I/O.

which like i told before, has nothing to do with netpbm since the same goes for all I/O processes.
what I'm looking for is a generic workaround actually.

if the contents of a ppm file is in the memory (in a variable say ppm_var) and if I can write it to a file (ppmWrite("myfile.ppm")) and then access it via a function (ppmOpen("myfile.ppm")) isn't there a way to access it through ppm_var instead of doing the extra ppmWrite. This is what I meant. Instead of pointing the FILE ON THE HDD, i want to point the MEMORY ADDRESS.

Thank you for your responses.

Last edited by mtphr (December 21, 2011 12:50)

Thumbs up

  • Registered: December 21, 2011
  • Posts: 32

Re: Reading subprocess output as a File

mtphr wrote:

i know i can't pass file contents instead of filename.
what i'm trying to do is pass the file contents as a file pointer.

ok, a small example:

>>> def func(filename):
...     with open(filename, encoding='utf-8') as f:
...         print(next(f))
... 
>>> func('.bashrc')
# .bashrc

>>>

the function expects a file name
how do you pass a file pointer instead of filename ? (you cannot change the function)

Thumbs up

  • mtphr
  • New Member
  • Offline
  • Registered: December 13, 2011
  • Posts: 4

Re: Reading subprocess output as a File

Yes! that is exactly my problem...
So, maybe i've approached it in a wrong way, let's put it this way...

I have a string that consists of multiple lines;

MyString = "Line number 1\nThe second line...\nAaaand Here comes the third line..."

If this text was in a file,

f = open("myfile.txt","w")
f.write(MyString)

Then I could easily use the .readline function;

f = open("myfile.txt","r")
f.readline() # Which would return "Line number 1"

Now what I want, is to get a way of using the .readline function, without writing the MyString to myfile.txt
What I mean is that, if the MyString is already in memory, why would I have to write it to a file (I/O process) and then read it back (2nd I/O process) to get in into a memory address again to handle it via a function. (.readline() function)

The example ofcourse doesn't make sense with a stupid string variable but the idea is the same. "I want to handle something allready in the memory with a function that loads a file into the memory."

Thumbs up

  • Registered: December 21, 2011
  • Posts: 32

Re: Reading subprocess output as a File

>>> import io
>>> b = b'abcdefg'
>>> b
b'abcdefg'
>>> bs = io.BytesIO(b)
>>> bs
<_io.BytesIO object at 0xb73894dc>
>>> dir(bs)
['__class__', '__delattr__', '__doc__', '__enter__', '__eq__', '__exit__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__lt__', '__ne__', '__new__', '__next__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '_checkClosed', '_checkReadable', '_checkSeekable', '_checkWritable', 'close', 'closed', 'detach', 'fileno', 'flush', 'getvalue', 'isatty', 'read', 'read1', 'readable', 'readinto', 'readline', 'readlines', 'seek', 'seekable', 'tell', 'truncate', 'writable', 'write', 'writelines']
>>> bs.read(2)
b'ab'
>>> bs.read(2)
b'cd'
>>> bs.read(2)
b'ef'
>>> bs.read(2)
b'g'
>>> bs.read(2)
b''
>>> bs.seek(0)
0
>>> bs.read(2)
b'ab'
>>>
>>> b = b'''abcdefg
... 
... 
... defghjijkl
... 
... 
... mnopq
... '''
>>> bs = io.BytesIO(b)
>>> next(bs)
b'abcdefg\n'
>>> next(bs)
b'\n'
>>> next(bs)
b'\n'
>>> next(bs)
b'defghjijkl\n'
>>> next(bs)
b'\n'
>>> next(bs)
b'\n'
>>> next(bs)
b'mnopq\n'
>>> next(bs)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>>
>>> b
b'abcdefg\n\n\ndefghjijkl\n\n\nmnopq\n'
>>> bs.seek(9)
9
>>> bs.readline()
b'\n'
>>> bs.readline()
b'defghjijkl\n'
>>> bs.readline()
b'\n'
>>>

Last edited by struct.h (December 23, 2011 02:36)

Thumbs up