Commit Graph

8182 Commits

Author SHA1 Message Date
PThorpe92
ae6f60b603 initial pass at upsert, integrate upsert into insert translation 2025-08-29 20:58:43 -04:00
PThorpe92
efd15721b1 initial pass at upsert, add upsert.rs 2025-08-29 20:58:43 -04:00
PThorpe92
1c4d1a2f28 Add upsert module to core/translate 2025-08-29 20:58:43 -04:00
Preston Thorpe
4a0655f2b8 Merge 'Update COMPAT.md to remove CREATE INDEX default disabled' from Preston Thorpe
Closes #2854
2025-08-29 20:58:30 -04:00
PThorpe92
8257496411 Update COMPAT.md to remove CREATE INDEX default disabled 2025-08-29 20:44:09 -04:00
Preston Thorpe
0899711439 Merge 'core/translate: remove unneessary agg clones' from Pere Diaz Bou
```
Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...
                        time:   [3.9978 µs 4.0085 µs 4.0193 µs]
                        change: [-5.0734% -4.6271% -4.1644%] (p = 0.00 < 0.05)
```

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2846
2025-08-29 11:40:53 -04:00
Pere Diaz Bou
167459389b core/translate: remove unneessary agg clones 2025-08-29 16:23:44 +02:00
Preston Thorpe
dc34097581 Merge 'core/vdbe: Micro-optimize "zero_or_null" opcode' from Pekka Enberg
It's a hot instruction for TPC-H, for example, so worth optimizing.
Reduces op_zero_or_null() from 5.6% to 2.4% in CPU flamegraph for TCP-H
Q1.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2842
2025-08-29 10:23:31 -04:00
Preston Thorpe
eb0f2b7029 Merge 'translate: with_capacity insns' from Pere Diaz Bou
Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2831
2025-08-29 10:23:09 -04:00
Pekka Enberg
9fc5947fa6 core/vdbe: Micro-optimize "zero_or_null" opcode
It's a hot instruction for TPC-H, for example, so worth optimizing.
Reduces op_zero_or_null() from 5.6% to 2.4% in CPU flamegraph for TCP-H
Q1.
2025-08-29 14:38:50 +03:00
Pekka Enberg
13e62ce435 Merge 'core: Initial pass on synchronous pragma' from Pekka Enberg
This adds support for "OFF" and "FULL" (default) synchronous modes. As
future work, we need to add NORMAL and EXTRA as well because
applications expect them.

Closes #2833
2025-08-29 07:27:12 +03:00
Pekka Enberg
70ae983685 Merge 'perf: avoid constructing PageType in helper methods' from Jussi Saurio
I think the `try_into()` is probably expensive. we can use the fact that
both interior page types are `<=5` to our advantage for a little perf
boost in these hot path function calls
Based on #2836 (just to see aggregate perf difference in benchmarks)

Closes #2837
2025-08-29 07:27:02 +03:00
Pekka Enberg
96778cfb0a Merge 'refactor/perf: remove BTreePageInner' from Jussi Saurio
it wasn't used for anything. no more `page.get().get().id`.
i did this simply to clean up the code (we haven't needed BTreePageInner
in probably at least 8 months or more?), but it also:
- makes practically every tpc-h query faster due to removing indirection
and refcell runtime borrowing every time the btree stack is accessed etc
- makes every SELECT * FROM users bench query faster
- drops the count() benchmark runtime by like half

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2836
2025-08-29 07:26:56 +03:00
Jussi Saurio
ae0ac189fa perf: avoid constructing PageType for helper methods 2025-08-28 22:56:44 +03:00
Jussi Saurio
ce860b7ec9 clippy 2025-08-28 21:48:29 +03:00
Jussi Saurio
9aae3fa859 refactor: remove BTreePageInner
it wasn't used for anything. no more `page.get().get().id`.
2025-08-28 21:44:54 +03:00
Pekka Enberg
3952dbb445 Merge 'Fix sorter column deduplication' from Piotr Rżysko
Previously, the added test case failed because the last result column
was missing - a nonexistent column in the sorter was referenced.

Closes #2824
2025-08-28 18:29:44 +03:00
Pekka Enberg
2ea4354afe Merge 'Improve integrity check' from Nikita Sivukhin
- check free list trunk and pages
- use shared hash map to check for duplicate references for pages
- properly check overflow pages

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2816
2025-08-28 16:06:15 +03:00
Pekka Enberg
44ed4d562f core: Initial pass on synchronous pragma
This adds support for "OFF" and "FULL" (default) synchronous modes. As
future work, we need to add NORMAL and EXTRA as well because
applications expect them.
2025-08-28 16:02:41 +03:00
Pekka Enberg
878147b931 Merge 'translate/insert: Improve string format performance' from Pere Diaz Bou
Rust's `fmt!` is slow af for the simplest of cases, let's just create
strings with a known size and skip all the fmt stuff.

Closes #2832
2025-08-28 14:36:09 +03:00
Pekka Enberg
a5322d2ab7 Merge 'core/schema: get_dependent_materialized_views_unnormalized' from Pere Diaz Bou
If we get a table name for in memory structure, it's safe to assume it's
already normalized.

Closes #2830
2025-08-28 14:34:06 +03:00
Pekka Enberg
eba90e1c5e Merge 'core/util: emit literal, cow instead of replace' from Pere Diaz Bou
Closes #2828
2025-08-28 14:33:04 +03:00
Pekka Enberg
44e619be1a Merge 'core/translate: sanize_string fast path improvement' from Pere Diaz Bou
Closes #2827
2025-08-28 14:32:43 +03:00
Pere Diaz Bou
964422375e translate/insert: string fmt perf improvmenets 2025-08-28 13:22:54 +02:00
Pere Diaz Bou
c7230f4ab0 translate: with_capacity insns 2025-08-28 13:13:19 +02:00
Pere Diaz Bou
48e5ad7a55 core/schema: get_dependent_materialized_views_unnormalized
If we get a table name for in memory structure, it's safe to assume it's
already normalized.
2025-08-28 13:11:40 +02:00
Pere Diaz Bou
84c5c4e581 core/util: emit literal, cow instead of replace 2025-08-28 12:59:55 +02:00
Pere Diaz Bou
082f18c073 core/translate: sanize_string fast path improvement 2025-08-28 12:57:28 +02:00
Piotr Rzysko
c383d9f16e Remove outdated comment in order_by.rs
The removed comment no longer matches the current code. The
OrderByRemapping struct and the surrounding comments are sufficient to
explain deduplication and remapping.
2025-08-28 09:49:55 +02:00
Piotr Rzysko
e33c2e0f0b Fix sorter column deduplication
Previously, the added test case failed because the last result column
was missing - a nonexistent column in the sorter was referenced.
2025-08-28 09:49:55 +02:00
Pekka Enberg
b4eba8b456 Merge 'Fix planner alias and table name handling' from
closes: #2817
```
turso> select * from simple_view as "v" where "v"."x" = 1;
┌───┐
│ x │
├───┤
│ 1 │
└───┘
turso> select "s"."x" from simple as "s" where "s"."x" = 1;
┌───┐
│ x │
├───┤
│ 1 │
└───
```

Reviewed-by: Preston Thorpe <preston@turso.tech>
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2818
2025-08-28 07:39:47 +03:00
Pekka Enberg
591914a63b Merge 'Introduce and propagate IOContext as required' from Avinash Sajjanshetty
I added `IOContext` to `DatbaseStorage` IO trait and this struct will
carry the necessary ctx required for encryption (or checksums.). This
lets us set the encryption at outside and let the IO layer handle it
properly

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2812
2025-08-28 07:38:52 +03:00
Pekka Enberg
20260c9dc0 Merge 'core/io: Switch Unix I/O to use libc::pwrite()' from Pekka Enberg
We use libc elsewhere for fault injection reasons, so let's do this
call-site too.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2811
2025-08-28 07:38:42 +03:00
Pekka Enberg
376dbc9261 Merge 'Update README.md for Go documentation' from Preston Thorpe
Closes #2819
2025-08-28 07:38:20 +03:00
PThorpe92
1526448089 Update README.md for Go documentation 2025-08-27 22:23:19 -04:00
themixednuts
79a9f4743e fix: planner alias and table name 2025-08-27 18:13:03 -05:00
Nikita Sivukhin
1c0efcfbff fix clippy 2025-08-27 23:22:21 +04:00
Nikita Sivukhin
09d4590ece fix compilation 2025-08-27 23:19:26 +04:00
Nikita Sivukhin
ae705445bf improve integrity check
- check free list trunk and pages
- use shared hash map to check for duplicate references for pages
- properly check overflow pages
2025-08-27 23:14:21 +04:00
Avinash Sajjanshetty
2c0842ff52 Set and propagate IOContext as required 2025-08-27 22:05:01 +05:30
Avinash Sajjanshetty
9e663c7f46 Add IOContext to carry encryption/checksum ctx 2025-08-27 21:33:05 +05:30
Pekka Enberg
e6ae7f990d Merge 'improve sync engine' from Nikita Sivukhin
This PR makes plenty of changes to the sync engine among which the main
ones are:
1. Now, we have only single database file which we use to directly
execute `pull` and `push` requests (so we don't have `draft` / `synced`
databases)
2. Last-write-win strategy were fixed a little bit - because before this
PR insert-insert conflict wasn't resolved automatically
3. Now sync engine can apply arbitrary `transform` function to the
logical changes
4. Sync-engine-aware checkpoint was implemented. Now, database created
by sync-engine has explicit `checkpoint()` method which under the hood
will use additional file to save frames needed for revert operation
during pull
5. Pull operation were separated into 2 phases internally: wait for
changes & apply changes
    * The problem is that pull operation itself (e.g. apply) right now
require exclusive lock to the sync engine and if user wants to pull &
push independently this will be problematic (as pull will lock the db
and push will never succeed)
6. Added support for V1 pull protocol
7. Exposed simple `stats()` method which return amount of pending cdc
operations and current wal size

Closes #2810
2025-08-27 18:08:21 +03:00
Pekka Enberg
bf7f80a937 core/io: Switch Unix I/O to use libc::pwrite()
We use libc elsewhere for fault injection reasons, so let's do this
call-site too.
2025-08-27 17:56:23 +03:00
Nikita Sivukhin
b67f14c785 fix clippy 2025-08-27 15:57:38 +04:00
Nikita Sivukhin
6e124d927e fix clippy 2025-08-27 15:51:29 +04:00
Nikita Sivukhin
009aa479bf improve sync engine 2025-08-27 15:30:00 +04:00
Pekka Enberg
30c5473151 Merge 'Add some docs on encryption' from Avinash Sajjanshetty
Reviewed-by: Preston Thorpe <preston@turso.tech>
Reviewed-by: bit-aloo (@Shourya742)

Closes #2805
2025-08-27 13:27:00 +03:00
Pekka Enberg
73ef26125b Merge 'Remove Go bindings' from Preston Thorpe
Go bindings have moved to their own repo
https://github.com/tursodatabase/turso-go
Otherwise it is a pain in the ass to have a Go module as a subdirectory
of another repository. and now, they can get the TLC that they needed,
as I found out that it never worked for windows, and the embedding
wasn't working properly in some cases. We want them to be plug and play.

Closes #2807
2025-08-27 13:26:45 +03:00
Pekka Enberg
eced1fe7db Merge 'core/storage: Micro-optimize Pager::commit_dirty_pages()' from Pekka Enberg
There's no need to call io.now() unless debug tracing is on. Let's
micro-optimize commit_dirty_pages() to avoid the unnecessary call.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2808
2025-08-27 13:26:22 +03:00
Pekka Enberg
2921033b28 core/storage: Micro-optimize Pager::commit_dirty_pages()
There's no need to call io.now() unless debug tracing is on. Let's
micro-optimize commit_dirty_pages() to avoid the unnecessary call.
2025-08-27 11:12:43 +03:00