SQLite improving performance with pre-sort

43 pointsposted 4 days ago
by tosh
(andersmurphy.com)

4 Comments

gandreani

3 hours ago

It's not clear in the code how you're batching the writes. Could the performance improvement be mostly explained by batches?

JSR_FDED

13 minutes ago

Looks like he’s doing 1M INSERTs per transaction.

In his 2nd example he’s doing the same thing, except he first sorts the data for each of those 1M INSERTs, leading to a 2-3x speedup.

Loranubi

2 hours ago

Depending on the size of the blob you might not want to use `WITHOUT ROWID`. I had a similar table with uuid keys and blobs of 40kb size on average and `WITHOUT ROWID` really tanked performance.

linolevan

3 hours ago

Neat optimization, I've never seen that before. Thanks for sharing.