[PyQt] Extracting QSqlQuery results in PyQt4

[mir amicitas]

I took your suggestion and tried out sqlite3 and also apsw.  Using either of
these modules the unpacking time is reduced to 1-2 seconds.

Furthermore just cycling through the results takes 1-1.5 seconds, meaning
that adding the results to a list is much quicker than using QSqlQuery (
less than 1 second ).

The query time is basically unaffected.

So it looks like the major problem I was having is in how Qt (or PyQT)
handles the query extraction (I suspect that part of it has to do with Qt
converting everything into QVariants).

With sqlite3 or apsw things are much more tolerable, though if any one has
ideas on how to make the extraction even faster I would love to here them.
What I am doing for now is to use:

 results = [row for row in self.cursor]

I think though you are right that if I want my app to feel fast I need to do
the query/extraction in the background and return stuff in chunks. I'll have
to see how well the gui handles that sort of thing.


-- amicitas



On Wed, Oct 8, 2008 at 1:20 PM, Aaron Digulla <digulla at hepe.com> wrote:

>
>
> It seems that loading 10'000 records out of the sqlite DB just takes a
> couple of seconds. sqlite isn't very fast when you have a lot of data
> (and yeah, 10'000 isn't that much ...).
>
> You must find a way to split the operation into smaller chunks. Also try
> the python sqlite3 module to get a feeling if Python is slow or if it's
> Qt or PyQt.
>
> In my app, I'm running sqlite operations in a background thread so they
> won't block the user from working. It takes a bit of effort but it was
> much more simple than I expected.
>
> Regards,
>
> --
> Aaron "Optimizer" Digulla a.k.a. Philmann Dark
> "It's not the universe that's limited, it's our imagination.
> Follow me and I'll show you something beyond the limits."
> http://darkviews.blogspot.com/          http://www.pdark.de/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.riverbankcomputing.com/pipermail/pyqt/attachments/20081009/7f4abe1b/attachment.html