<html>
<head>
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 10pt;
font-family:Tahoma
}
--></style>
</head>
<body class='hmmessage'>
I have never tried it but you can save a file in process A and open it from another process, that's what I do. The H5py or pytables forum/list should be able to give you a better answer.<br><br>-mab<br>"The world goin' one way, people another!" - Poot<br><br><br><br>> Date: Mon, 9 May 2011 10:26:23 -0400<br>> Subject: Re: [PyQt] pyqt4 amd multiprocessing<br>> From: jmrbcu@gmail.com<br>> To: mabxsi@hotmail.com<br>> CC: sam.carcagno@gmail.com; pyqt@riverbankcomputing.com<br>> <br>> a dummy question, is possible, simple and fast to send a big array of<br>> integer, floats, etc, lets say, a 512x512x1500 array of unsigned int<br>> using hdf5 from one process to another?<br>> <br>> <br>> <br>> On Mon, May 9, 2011 at 10:16 AM, Belzile Marc-André <mabxsi@hotmail.com> wrote:<br>> > Indeed, QProcess is meant to run external python scripts, which allows<br>> > python to run one script per available core.<br>> > e.g.<br>> > p = QProcess('myscript.py "data"' )<br>> > p.start()<br>> > Look at the following files for a more an in-depth example:<br>> > http://code.google.com/p/ice-cache-explorer/source/browse/trunk/process_pool.py<br>> > http://code.google.com/p/ice-cache-explorer/source/browse/trunk/loader_process.py<br>> > http://code.google.com/p/ice-cache-explorer/source/browse/trunk/iceloader.py<br>> ><br>> > I'm using the H5py module for dealing with HDF5 files, it has nice support<br>> > for numpy and easy to use. You can also use pytable which is also a popular<br>> > choice.<br>> > -mab<br>> > "The world goin' one way, people another!" - Poot<br>> ><br>> ><br>> ><br>> >> Date: Mon, 9 May 2011 09:46:43 -0400<br>> >> Subject: Re: [PyQt] pyqt4 amd multiprocessing<br>> >> From: jmrbcu@gmail.com<br>> >> To: mabxsi@hotmail.com<br>> >> CC: sam.carcagno@gmail.com; pyqt@riverbankcomputing.com<br>> >><br>> >> sound interesting but, how do you use QProcess, what I have read about<br>> >> it is that this class is for executing external applications (yes,<br>> >> processes), how to use QProcess in a pool. Could you tell me more<br>> >> about HDF5, for example, how do you use it with python. My dataset<br>> >> could be very big (or what I think is big), for example, a serie of<br>> >> 1500 dicom images so I think that HDF5 could be useful to me.<br>> >><br>> >> On Mon, May 9, 2011 at 9:24 AM, Belzile Marc-André <mabxsi@hotmail.com><br>> >> wrote:<br>> >> > As an alternative to python multiprocessing module, I'm using QProcess<br>> >> > for<br>> >> > loading and exporting files with my own pool manager. It's pretty darn<br>> >> > fast<br>> >> > and works well, my data sets are managed with HDF5 which takes care of<br>> >> > handling huge data sets pretty easily. If you want to pass data back to<br>> >> > the<br>> >> > UI you can use QSharedMemory to give the GUI access to your data.<br>> >> > Check out my project for a QProcess example.<br>> >> > http://code.google.com/p/ice-cache-explorer/<br>> >> ><br>> >> > -mab<br>> >> > "The world goin' one way, people another!" - Poot<br>> >> ><br>> >> ><br>> >> ><br>> >> >> Date: Mon, 9 May 2011 09:10:20 -0400<br>> >> >> From: jmrbcu@gmail.com<br>> >> >> To: sam.carcagno@gmail.com<br>> >> >> CC: pyqt@riverbankcomputing.com<br>> >> >> Subject: Re: [PyQt] pyqt4 amd multiprocessing<br>> >> >><br>> >> >> I am very interested in the answer because I have plan to use<br>> >> >> multiprocessing module to do my hard work (because of the python GIL),<br>> >> >> my future design is something like Samuele say, a pool of process<br>> >> >> ready to do some hard job (using multiprocessing) and the GUI with<br>> >> >> pyqt4 and vtk, data pass back and forth between the gui and the worker<br>> >> >> processes (data will be vtk datasets)<br>> >> >><br>> >> >> On Mon, May 9, 2011 at 6:26 AM, Samuele Carcagno<br>> >> >> <sam.carcagno@gmail.com><br>> >> >> wrote:<br>> >> >> > Hi,<br>> >> >> ><br>> >> >> > I would like to use the multiprocessing module to create a set of<br>> >> >> > sounds<br>> >> >> > (as numpy arrays) in parallel.<br>> >> >> > The parameters of the sounds (duration, volume, etc..) are controlled<br>> >> >> > by<br>> >> >> > a pyqt4 GUI, and the sounds<br>> >> >> > need to be passed back to the GUI for further processing and<br>> >> >> > presentation. The sounds are generated<br>> >> >> > in a separate module from the GUI. I'm attaching a minimal example<br>> >> >> > with<br>> >> >> > a file that creates the gui and<br>> >> >> > a second file that contains the functions for generating the sounds.<br>> >> >> ><br>> >> >> > I've had mixed success so far, it seems to work on Kubuntu Natty -<br>> >> >> > Python 2.7.1+ - Qt 4.7.2 - PyQt 4.8.3<br>> >> >> > but sometimes it fails on Kubuntu Lucid - Python 2.6.5 - Qt 4.6.2 -<br>> >> >> > PyQt 4.7.2 with the following error:<br>> >> >> ><br>> >> >> > Traceback (most recent call last):<br>> >> >> > File "test_gui.py", line 20, in onClickButton1<br>> >> >> > self.doTrial()<br>> >> >> > File "test_gui.py", line 32, in doTrial<br>> >> >> > x = complexTone(F0, lowHarm, highHarm, level, duration, ramp,<br>> >> >> > channel, fs, maxLevel)<br>> >> >> > File "/media/ntfsShared/mptest/simple_test/stim.py", line 45, in<br>> >> >> > complexTone<br>> >> >> > pool.join()<br>> >> >> > File "/usr/lib/python2.6/multiprocessing/pool.py", line 342, in join<br>> >> >> > p.join()<br>> >> >> > File "/usr/lib/python2.6/multiprocessing/process.py", line 119, in<br>> >> >> > join<br>> >> >> > res = self._popen.wait(timeout)<br>> >> >> > File "/usr/lib/python2.6/multiprocessing/forking.py", line 117, in<br>> >> >> > wait<br>> >> >> > return self.poll(0)<br>> >> >> > File "/usr/lib/python2.6/multiprocessing/forking.py", line 106, in<br>> >> >> > poll<br>> >> >> > pid, sts = os.waitpid(self.pid, flag)<br>> >> >> > OSError: [Errno 4] Interrupted system call<br>> >> >> ><br>> >> >> > Am I doing something fundamentally unsafe by using multiprocessing in<br>> >> >> > this way with a pyqt4 GUI?<br>> >> >> > What is the best approach to use multiprocessing together with pyqt4?<br>> >> >> ><br>> >> >> > Thanks for any suggestions!<br>> >> >> ><br>> >> >> > Sam<br>> >> >> ><br>> >> >> > ###---------GUI----------File 1<br>> >> >> > from PyQt4 import QtGui<br>> >> >> > from PyQt4 import QtCore<br>> >> >> > import sys<br>> >> >> > from stim import* #import module for stimulus generation<br>> >> >> ><br>> >> >> ><br>> >> >> > class Example(QtGui.QWidget):<br>> >> >> ><br>> >> >> > def __init__(self):<br>> >> >> > super(Example, self).__init__()<br>> >> >> > self.setGeometry(300, 300, 250, 150)<br>> >> >> > self.setWindowTitle('Multiprocessing Test')<br>> >> >> ><br>> >> >> > self.button1 = QtGui.QPushButton('Button 1', self)<br>> >> >> > QtCore.QObject.connect(self.button1,<br>> >> >> > QtCore.SIGNAL('clicked()'),<br>> >> >> > self.onClickButton1)<br>> >> >> ><br>> >> >> > def onClickButton1(self):<br>> >> >> > self.doTrial()<br>> >> >> ><br>> >> >> > def doTrial(self):<br>> >> >> > F0 = 200<br>> >> >> > lowHarm=1<br>> >> >> > highHarm = 20<br>> >> >> > level = 50<br>> >> >> > duration = 180<br>> >> >> > ramp = 10<br>> >> >> > channel = "Both"<br>> >> >> > fs = 44100<br>> >> >> > maxLevel = 100.0<br>> >> >> > x = complexTone(F0, lowHarm, highHarm, level, duration, ramp,<br>> >> >> > channel, fs, maxLevel)<br>> >> >> ><br>> >> >> ><br>> >> >> > if __name__ == '__main__':<br>> >> >> > app = QtGui.QApplication(sys.argv)<br>> >> >> > ex = Example()<br>> >> >> > ex.show()<br>> >> >> > app.exec_()<br>> >> >> > #----------END OF FILE 1<br>> >> >> ><br>> >> >> ><br>> >> >> > ###-SOUND GENERATION-- File stim.py<br>> >> >> > from numpy import*<br>> >> >> > import multiprocessing<br>> >> >> ><br>> >> >> > def pureTone(frequency, phase, level, duration, ramp, channel, fs,<br>> >> >> > maxLevel):<br>> >> >> ><br>> >> >> > amp = 10**((level - maxLevel) / 20.)<br>> >> >> > duration = duration / 1000. #convert from ms to sec<br>> >> >> > ramp = ramp / 1000.<br>> >> >> ><br>> >> >> > nSamples = int(round(duration * fs))<br>> >> >> > nRamp = int(round(ramp * fs))<br>> >> >> > nTot = nSamples + (nRamp * 2)<br>> >> >> ><br>> >> >> > timeAll = arange(0., nTot) / fs<br>> >> >> > timeRamp = arange(0., nRamp)<br>> >> >> ><br>> >> >> > snd = zeros((nTot, 2))<br>> >> >> ><br>> >> >> > if channel == "Right":<br>> >> >> > snd[0:nRamp, 1] = amp * ((1-cos(pi * timeRamp/nRamp))/2) *<br>> >> >> > sin(2*pi*frequency * timeAll[0:nRamp] + phase)<br>> >> >> > snd[nRamp:nRamp+nSamples, 1] = amp* sin(2*pi*frequency *<br>> >> >> > timeAll[nRamp:nRamp+nSamples] + phase)<br>> >> >> > snd[nRamp+nSamples:len(timeAll), 1] = amp * ((1+cos(pi *<br>> >> >> > timeRamp/nRamp))/2) * sin(2*pi*frequency *<br>> >> >> > timeAll[nRamp+nSamples:len(timeAll)] + phase)<br>> >> >> > elif channel == "Left":<br>> >> >> > snd[0:nRamp, 0] = amp * ((1-cos(pi * timeRamp/nRamp))/2) *<br>> >> >> > sin(2*pi*frequency * timeAll[0:nRamp] + phase)<br>> >> >> > snd[nRamp:nRamp+nSamples, 0] = amp* sin(2*pi*frequency *<br>> >> >> > timeAll[nRamp:nRamp+nSamples] + phase)<br>> >> >> > snd[nRamp+nSamples:len(timeAll), 0] = amp * ((1+cos(pi *<br>> >> >> > timeRamp/nRamp))/2) * sin(2*pi*frequency *<br>> >> >> > timeAll[nRamp+nSamples:len(timeAll)] + phase)<br>> >> >> > elif channel == "Both":<br>> >> >> > snd[0:nRamp, 0] = amp * ((1-cos(pi * timeRamp/nRamp))/2) *<br>> >> >> > sin(2*pi*frequency * timeAll[0:nRamp] + phase)<br>> >> >> > snd[nRamp:nRamp+nSamples, 0] = amp* sin(2*pi*frequency *<br>> >> >> > timeAll[nRamp:nRamp+nSamples] + phase)<br>> >> >> > snd[nRamp+nSamples:len(timeAll), 0] = amp * ((1+cos(pi *<br>> >> >> > timeRamp/nRamp))/2) * sin(2*pi*frequency *<br>> >> >> > timeAll[nRamp+nSamples:len(timeAll)] + phase)<br>> >> >> > snd[:, 1] = snd[:, 0]<br>> >> >> ><br>> >> >> > return snd<br>> >> >> ><br>> >> >> ><br>> >> >> > def complexTone(F0, lowHarm, highHarm, level, duration, ramp,<br>> >> >> > channel,<br>> >> >> > fs, maxLevel):<br>> >> >> > pool = multiprocessing.Pool()<br>> >> >> > tn = []<br>> >> >> ><br>> >> >> > for i in range(lowHarm, highHarm+1):<br>> >> >> > res = pool.apply_async(pureTone, (F0*i, 0, level, duration,<br>> >> >> > ramp,<br>> >> >> > channel, fs, maxLevel,), callback=tn.append)<br>> >> >> ><br>> >> >> > pool.close()<br>> >> >> > pool.join()<br>> >> >> ><br>> >> >> > for i in range(len(tn)):<br>> >> >> > if i == 0:<br>> >> >> > snd = tn[i]<br>> >> >> > else:<br>> >> >> > snd = snd + tn[i]<br>> >> >> ><br>> >> >> > return snd<br>> >> >> ><br>> >> >> > #----------END OF FILE 2<br>> >> >> > _______________________________________________<br>> >> >> > PyQt mailing list PyQt@riverbankcomputing.com<br>> >> >> > http://www.riverbankcomputing.com/mailman/listinfo/pyqt<br>> >> >> ><br>> >> >><br>> >> >><br>> >> >><br>> >> >> --<br>> >> >> Lic. José M. Rodriguez Bacallao<br>> >> >> Centro de Biofisica Medica<br>> >> >> -----------------------------------------------------------------<br>> >> >> Todos somos muy ignorantes, lo que ocurre es que no todos ignoramos lo<br>> >> >> mismo.<br>> >> >><br>> >> >> Recuerda: El arca de Noe fue construida por aficionados, el titanic<br>> >> >> por profesionales<br>> >> >> -----------------------------------------------------------------<br>> >> >> _______________________________________________<br>> >> >> PyQt mailing list PyQt@riverbankcomputing.com<br>> >> >> http://www.riverbankcomputing.com/mailman/listinfo/pyqt<br>> >> ><br>> >><br>> >><br>> >><br>> >> --<br>> >> Lic. José M. Rodriguez Bacallao<br>> >> Centro de Biofisica Medica<br>> >> -----------------------------------------------------------------<br>> >> Todos somos muy ignorantes, lo que ocurre es que no todos ignoramos lo<br>> >> mismo.<br>> >><br>> >> Recuerda: El arca de Noe fue construida por aficionados, el titanic<br>> >> por profesionales<br>> >> -----------------------------------------------------------------<br>> ><br>> <br>> <br>> <br>> -- <br>> Lic. José M. Rodriguez Bacallao<br>> Centro de Biofisica Medica<br>> -----------------------------------------------------------------<br>> Todos somos muy ignorantes, lo que ocurre es que no todos ignoramos lo mismo.<br>> <br>> Recuerda: El arca de Noe fue construida por aficionados, el titanic<br>> por profesionales<br>> -----------------------------------------------------------------<br> </body>
</html>