<div dir="ltr"><div>Hi,</div><div><br></div><div>Sorry for the double post, but it was suggested I repost the full bug report not just link to the Qt forum.</div><div><div class="gmail-content">
<p>I have a <a href="https://github.com/argosopentech/argos-translate" rel="nofollow">PyQt application with a memory leak in it</a>.
It's an application for doing translations and it uses QThreads and
signal/slots to do the translation on a separate thread and then signal a
QTextEdit with the result when the translation finishes. The
application works but it leaks memory with every translation. <br></p><p><small class="gmail-pull-left"><strong></strong><span class="gmail-visible-xs-inline-block gmail-visible-sm-inline-block gmail-visible-md-inline-block gmail-visible-lg-inline-block"><a class="gmail-permalink" href="https://forum.qt.io/post/624628"><span class="gmail-timeago" title="28 Oct 2020, 21:34" datetime="2020-10-29T02:34:32.206Z"></span></a> </span></small>I've been able to narrow the problem down to occurring when I create
a CTranslate Translator inside of a PyQt QThread that is created from a
QWidget. If I remove the CTranslate Translator and do something else
that allocates a large amount of memory there is no leak. If I create
the CTranslate Translator from the QWidget with no QThread there is also
no leak. If I run the QThread outside of a QWidget there is no leak.
The leak only happens with the combo of all three.
</p><div class="gmail-content"><p>My best guess is that there is some bug/me misusing in the
combination of Python/Qt/CTranslate memory management. Python uses
automatic reference counting memory management, while Qt in native C++
use C++ parent based memory management. On top of that CTranslate uses
C++ extensions to Python so it seems like there are a lot of places
where the problem could be appearing.</p>
<p>I made an <a href="https://gist.github.com/argosopentech/326008095f2cd726d9567e171fcf3842" rel="nofollow">example script demonstrating the leak</a>. To run it you need a CTranslate model and need to provide a path to it in the script. <a href="https://drive.google.com/drive/folders/11wxM3Ze7NCgOk_tdtRjwet10DmtvFu3i" rel="nofollow">Here's a Google Drive link</a> where you can download a package for my project that if extracted (its just a renamed .zip archive) has a CTranslate model at <code>/model</code>. When this script runs it leaks ~5GB of memory.</p><p><br></p><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC1" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line"><span class="gmail-pl-k">from</span> <span class="gmail-pl-v">PyQt5</span>.<span class="gmail-pl-v">QtWidgets</span> <span class="gmail-pl-k">import</span> <span class="gmail-pl-v">QMainWindow</span>, <span class="gmail-pl-v">QApplication</span></td>
</tr>
<tr>
</tr></tbody></table><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC2" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line"><span class="gmail-pl-k">from</span> <span class="gmail-pl-v">PyQt5</span>.<span class="gmail-pl-v">QtCore</span> <span class="gmail-pl-k">import</span> <span class="gmail-pl-v">QThread</span></td>
</tr>
<tr>
</tr></tbody></table><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC3" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line"><span class="gmail-pl-k">import</span> <span class="gmail-pl-s1">ctranslate2</span></td>
</tr>
<tr>
</tr></tbody></table><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC4" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line">
<br></td>
</tr>
<tr>
</tr></tbody></table><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC5" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line"><span class="gmail-pl-k">class</span> <span class="gmail-pl-v">WorkerThread</span>(<span class="gmail-pl-v">QThread</span>):</td>
</tr>
<tr>
</tr></tbody></table><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC6" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line"> <span class="gmail-pl-k">def</span> <span class="gmail-pl-en">__del__</span>(<span class="gmail-pl-s1">self</span>):</td>
</tr>
<tr>
</tr></tbody></table><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC7" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line"> <span class="gmail-pl-s1">self</span>.<span class="gmail-pl-en">wait</span>()</td>
</tr>
<tr>
</tr></tbody></table><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC8" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line">
<br></td>
</tr>
<tr>
</tr></tbody></table><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC9" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line"> <span class="gmail-pl-k">def</span> <span class="gmail-pl-en">run</span>(<span class="gmail-pl-s1">self</span>):</td>
</tr>
<tr>
</tr></tbody></table><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC10" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line"> <span class="gmail-pl-s1">translator</span> <span class="gmail-pl-c1">=</span> <span class="gmail-pl-s1">ctranslate2</span>.<span class="gmail-pl-v">Translator</span>(<span class="gmail-pl-s">'/path/to/ctranslate/model'</span>)</td>
</tr>
<tr>
</tr></tbody></table><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC11" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line">
<br></td>
</tr>
<tr>
</tr></tbody></table><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC12" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line"><span class="gmail-pl-k">class</span> <span class="gmail-pl-v">GUIWindow</span>(<span class="gmail-pl-v">QMainWindow</span>):</td>
</tr>
<tr>
</tr></tbody></table><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC13" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line"> <span class="gmail-pl-k">def</span> <span class="gmail-pl-en">translate</span>(<span class="gmail-pl-s1">self</span>):</td>
</tr>
<tr>
</tr></tbody></table><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC14" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line"> <span class="gmail-pl-s1">new_worker_thread</span> <span class="gmail-pl-c1">=</span> <span class="gmail-pl-v">WorkerThread</span>()</td>
</tr>
<tr>
</tr></tbody></table><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC15" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line"> <span class="gmail-pl-s1">new_worker_thread</span>.<span class="gmail-pl-en">start</span>()</td>
</tr>
<tr>
</tr></tbody></table><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC16" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line">
<br></td>
</tr>
<tr>
</tr></tbody></table><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC17" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line"><span class="gmail-pl-s1">app</span> <span class="gmail-pl-c1">=</span> <span class="gmail-pl-v">QApplication</span>([])</td>
</tr>
<tr>
</tr></tbody></table><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC18" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line"><span class="gmail-pl-s1">main_window</span> <span class="gmail-pl-c1">=</span> <span class="gmail-pl-v">GUIWindow</span>()</td>
</tr>
<tr>
</tr></tbody></table><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC19" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line"><span class="gmail-pl-s1">main_window</span>.<span class="gmail-pl-en">show</span>()</td>
</tr>
<tr>
</tr></tbody></table><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC20" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line">
<br></td>
</tr>
<tr>
</tr></tbody></table><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC21" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line"><span class="gmail-pl-k">for</span> <span class="gmail-pl-s1">i</span> <span class="gmail-pl-c1">in</span> <span class="gmail-pl-en">range</span>(<span class="gmail-pl-c1">120</span>):</td>
</tr>
<tr>
</tr></tbody></table><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC22" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line"> <span class="gmail-pl-en">print</span>(<span class="gmail-pl-s1">i</span>)</td>
</tr>
<tr>
</tr></tbody></table><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC23" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line"> <span class="gmail-pl-s1">main_window</span>.<span class="gmail-pl-en">translate</span>()</td>
</tr>
<tr>
</tr></tbody></table><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container"><tbody><tr><td id="gmail-file-memory_leak-py-LC24" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line">
<br></td>
</tr>
<tr>
</tr></tbody></table><span class="gmail-pl-s1">app</span>.<span class="gmail-pl-en">exec_</span>()<p><br></p></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Oct 29, 2020 at 8:15 AM Admin Argos Open Technologies, LLC <<a href="mailto:admin@argosopentech.com">admin@argosopentech.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi,</div><div><br></div><div>I'm having a memory leak when I run CTranslate from a QThread from a QWidget. <br></div><div><br></div><div>I also posted this issue on the Qt Forum: <a href="https://forum.qt.io/topic/120341/how-to-debug-a-hard-to-find-memory-leak-in-pyqt" target="_blank">https://forum.qt.io/topic/120341/how-to-debug-a-hard-to-find-memory-leak-in-pyqt</a></div><div><br></div><div>Thanks for any help,</div><div><br></div><div>P.J.<br></div></div>
</blockquote></div></div>