Commit Graph

9205 Commits

Author SHA1 Message Date
TcMits
17c91c1fe2 resolve conflicts 2025-09-22 16:03:52 +07:00
Pekka Enberg
37866e74e5 Merge 'core/io: Ensure callbacks are invoked once' from Pedro Muniz
Add a `Once` object to uphold this property. We cannot use the OnceLock
to dictate this, because if we set the OnceLock before actually calling
the callback, there is a moment in time where we will have an incorrect
transient state. This change ensures we atomically call the callback and
then set the OnceLock
Should fix #3217
Closes #3217

Closes #3237
2025-09-22 11:44:39 +03:00
Pekka Enberg
26f90257a6 Turso 0.2.0-pre.6 2025-09-22 11:44:21 +03:00
Pekka Enberg
979e2542af Merge branch 'main' of https://github.com/tursodatabase/limbo 2025-09-22 11:44:07 +03:00
Pekka Enberg
f4b0fb17f7 Turso 0.2.0-pre.5 2025-09-22 11:34:22 +03:00
Pekka Enberg
0144ea8059 Merge 'Support UNION queries in DBSP-based Materialized Views' from Glauber Costa
UNION queries, while useful on their own, are a cornerstone of recursive
CTEs.
This PR implements:
* the merge operator, required to merge both sides of a union query.
* the circuitry necessary to issue the Merge operator.
* extraction of tables mentioned in union and CTE expressions, so we can
correctly populate tables that contain them.

Closes #3234
2025-09-22 11:33:19 +03:00
Pekka Enberg
19849c7895 Merge 'DBSP: Return a parse error for a non-equality join' from Glauber Costa
We currently don't handle non equality, but end up just returning a
bogus result. Let's parse error.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3232
2025-09-22 11:32:43 +03:00
Pekka Enberg
390181c024 Merge 'Upgrade dist to 0.30.0' from Pekka Enberg
Closes #3241
2025-09-22 11:30:51 +03:00
Pekka Enberg
e5a3512f79 Upgrade dist to 0.30.0 2025-09-22 10:19:46 +03:00
Pekka Enberg
751261566e Fix Antithesis and simulator Dockerfiles 2025-09-22 10:02:01 +03:00
Pekka Enberg
fbad158213 Turso 0.2.0-pre.4 2025-09-22 09:15:55 +03:00
Pekka Enberg
18c41b0f5a Merge 'sqlite3: Fix compatibility test error by canonicalizing path' from Samuel Marks
Use canonical path to fix temp path on macOS ; rename to resolve binding
connascence. Fixes:
```
---- tests::test_sqlite3_db_filename stdout ----

thread 'tests::test_sqlite3_db_filename' (62061) panicked at sqlite3/tests/compat/mod.rs:1251:13:
assertion `left == right` failed
  left: "/private/var/folders/w5/21g61wls7ksd3z8cxrj41s1c0000gn/T/.tmpNcCUO2.db"
 right: "/var/folders/w5/21g61wls7ksd3z8cxrj41s1c0000gn/T/.tmpNcCUO2.db"
```

Closes #3227
2025-09-22 09:15:00 +03:00
Pekka Enberg
46297acb31 Merge 'Sync improvements' from Nikita Sivukhin
This PR improves sync and database bindings for browser
List of changes:
- For node and browser database now run on main thread and only IO work
offloaded to the worker (web worker in browser)
- Simple locks are implemented for database access externally in order
to guard access to the same connection (when request is executed async -
main thread can try to start another request concurrently)
- parking_lot in the Wal replaced by spin-wait (by invoking
`parking_lot.try_read/try_write`) for WASM target because browser can't
park main thread
- js sync reworked in order to support few engine options
(`longPollTimeoutMs`) and introduce external locking which properly
guards concurrent access of sync methods

Closes #3218
2025-09-22 09:14:30 +03:00
Pekka Enberg
6280cfc59d Merge branch 'main' into sync-improvements 2025-09-22 07:35:39 +03:00
Pekka Enberg
54176d4548 Merge 'Add encryption throughput test' from Avinash Sajjanshetty
```
cargo run --release -- -b 100 -i 100000 --read-ratio 75

cargo run --release -- -b 100 -i 100000 --read-ratio 75 --encryption
```
```
Total time: 629.51s

Transaction throughput: 158.85 txns/sec
Operation throughput: 15885.37 ops/sec
  - Read operations: 7500165 (7500165 found, 0 not found)
  - Read throughput: 11914.29 reads/sec
  - Write operations: 2499835
  - Write throughput: 3971.08 writes/sec
```
vs
```
Total time: 665.05s

Transaction throughput: 150.36 txns/sec
Operation throughput: 15036.47 ops/sec
  - Read operations: 7500165 (7500165 found, 0 not found)
  - Read throughput: 11277.60 reads/sec
  - Write operations: 2499835
  - Write throughput: 3758.87 writes/sec
```

Closes #3230
2025-09-22 07:34:29 +03:00
Pekka Enberg
dae72a7418 Merge 'Remove some unnecessary unsafe impls' from Pedro Muniz
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3236
2025-09-22 07:33:50 +03:00
Glauber Costa
2627ad44de support union statements in the DBSP circuit compiler 2025-09-21 21:00:27 -03:00
Glauber Costa
b419db489a Implement the DBSP merge operator
The Merge operator is a stateless operator that merges two deltas.
There are two modes: Distinct, where we merge together values that
are the same, and All, where we preserve all values. We use the rowid of
the hashable row to guarantee that: In Distinct mode, the rowid is set
to 0 in both sides. If they values are the same, they will hash to the
same thing. For All, the rowids are different.

The merge operator is used for the UNION statement, which is a
cornerstone of Recursive CTEs.
2025-09-21 21:00:27 -03:00
Glauber Costa
9f54f60d45 make sure that complex select statements are captured by MV populate
The population code extracts table information from the select statement
so it can populate the materialized view. But the code, as written
today, is naive. It doesn't capture table information correctly if there
is more than one select statement (such in the case of a union query).
2025-09-21 21:00:27 -03:00
pedrocarlo
ffeb26b24a only ever call callbacks once 2025-09-21 14:36:18 -03:00
pedrocarlo
e5dfc942b1 remove some unnecessary unsafe impls 2025-09-21 13:29:59 -03:00
Glauber Costa
13260349b0 Return a parse error for a non-equality join
We currently don't handle non equality, but end up just returning a
bogus result. Let's parse error.
2025-09-20 20:35:10 -03:00
Preston Thorpe
25754cb0e4 Merge 'Pragma busy timeout' from Nikita Sivukhin
Expose busy_timeout added in #3067 as a `PRAGMA busy_timeout=<ms
duration>`
https://www.sqlite.org/pragma.html#pragma_busy_timeout

Reviewed-by: Avinash Sajjanshetty (@avinassh)
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #3224
2025-09-20 14:36:29 -04:00
Avinash Sajjanshetty
0597ea722a Add encryption throughput test 2025-09-20 21:56:58 +05:30
Samuel Marks
d2cd48d9e4 [sqlite3/tests/compat/mod.rs] Use canonical path to fix temp path on macOS ; rename to resolve binding connascence 2025-09-19 20:40:11 -05:00
Nikita Sivukhin
57e52077be add link to the docs 2025-09-19 16:48:43 +04:00
Nikita Sivukhin
c63c820bb7 add busy_timeout pragma 2025-09-19 16:48:12 +04:00
Preston Thorpe
6b273af7e9 Merge 'translate/optimize: centralize AST/expr traversal' from Preston Thorpe
Previously we were rewriting/traversing the AST in a couple different
places, each of these added kinda ad-hoc as we needed them. This
attempts to do the binding of column references as well as the rewriting
of anonymous `Expr::Variable` -> `__param_N` that we use to maintain the
order of bound variables, also normalizes the Qualified Name's.
Also we previously weren't accepting Variable (or at least they wouldn't
work) in places like `LIMIT ? OFFSET ?`, which this PR adds.
I kinda want to keep refactoring translation a bit, and try to break
plan building up into more easy-to-digest chunks.. but I will resist the
urge right now as it's definitely not high priority pre-beta

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3210
2025-09-19 08:03:39 -04:00
Preston Thorpe
20493441e0 Merge 'prevent alter table with materialized views' from Glauber Costa
I don't want to even think about the complexity involved in making sure
that materialized views are still sane after the base table(s) are
altered.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3223
2025-09-19 08:01:58 -04:00
Pekka Enberg
9dda5a6263 Merge 'bugfix: clear reserved space for a reused page' from Avinash Sajjanshetty
fixes #3184

Closes #3198
2025-09-19 14:16:24 +03:00
Glauber Costa
8300d0390e prevent alter table with materialized views
I don't want to even think about the complexity involved in making sure
that materialized views are still sane after the base table(s) are
altered.
2025-09-19 05:59:46 -05:00
Pekka Enberg
2307cf03be Merge 'core/mvcc: Wrap LogicalLog in RwLock' from Pekka Enberg
Closes #3215
2025-09-19 13:40:40 +03:00
Pekka Enberg
69d4745aa0 Merge 'Support JOINs in DBSP materialized views' from Glauber Costa
This PR introduces the final major operator: the JOIN operator.
Many things need to be fixed before we can properly support them, and we
handle those. In particular, JOINs always generate qualified column
statements, but we were not handling them correctly at all in the
operators. Not a problem for linear circuits, but fatal for JOINs.
The operator.rs file also becomes incredibly complex with not one, but
two stateful operator. So it is now broken apart.

Closes #3207
2025-09-19 13:40:21 +03:00
Jussi Saurio
c5ebb0ef63 Merge 'mvcc: remove unused code related to is_logical_log()' from Jussi Saurio
is always logical log

Closes #3220
2025-09-19 13:31:34 +03:00
Nikita Sivukhin
c24e5219d2 remove log file 2025-09-19 14:24:42 +04:00
Nikita Sivukhin
cfc8728774 fix clippy 2025-09-19 14:15:04 +04:00
Jussi Saurio
6a07d56742 Merge 'Put the unused variable behind a flag as intended' from Avinash Sajjanshetty
I keep getting these warnings and it slightly annoys me:
```rust
warning: unused variable: `overflow_cell_count_before`
    --> core/storage/btree.rs:3634:29
     |
3634 |                         let overflow_cell_count_before = parent_contents.overflow_cells.len();
     |                             ^^^^^^^^^^^^^^^^^^^^^^^^^^ help: if this is intentional, prefix it with an underscore: `_overflow_cell_count_before`
     |
     = note: `#[warn(unused_variables)]` on by default

warning: unused variable: `overflow_cell_count_after`
    --> core/storage/btree.rs:3641:29
     |
3641 |                         let overflow_cell_count_after = parent_contents.overflow_cells.len();
     |                             ^^^^^^^^^^^^^^^^^^^^^^^^^ help: if this is intentional, prefix it with an underscore: `_overflow_cell_count_after`

```

Reviewed-by: Nikita Sivukhin (@sivukhin)

Closes #3219
2025-09-19 12:58:57 +03:00
Jussi Saurio
ba7ae50eff mvcc: remove unused code related to is_logical_log()
is always logical log
2025-09-19 12:55:27 +03:00
Avinash Sajjanshetty
d5295fb45c Put the unused variable behind a flag as intended 2025-09-19 14:55:02 +05:30
Nikita Sivukhin
7049f3ddae fix clock implementation for OPFS IO 2025-09-19 13:21:00 +04:00
Nikita Sivukhin
bc2dbe9025 fix bug 2025-09-19 13:21:00 +04:00
Nikita Sivukhin
b106220743 main thread in browser can't execute parking - so we use parking lot in spin-lock style for that target 2025-09-19 13:21:00 +04:00
PThorpe92
e1ed12b284 rm claude comment 2025-09-19 05:20:20 -04:00
Nikita Sivukhin
1d3c823c7b wip 2025-09-19 13:19:30 +04:00
Nikita Sivukhin
66de28d84b wip 2025-09-19 13:19:30 +04:00
Nikita Sivukhin
1185298670 fix replay generator 2025-09-19 13:19:30 +04:00
Nikita Sivukhin
e68b642f4f set longPollTimeoutMs from js bindings 2025-09-19 13:19:30 +04:00
Nikita Sivukhin
83303b8c5b properly guard access to the sync engine with locks 2025-09-19 13:19:30 +04:00
Nikita Sivukhin
160119b12e propagate long poll timeout 2025-09-19 13:19:30 +04:00
Nikita Sivukhin
b0f60a29ba guard meta with mutex and make all methods to accept shared reference and offload locking of sync engine methods to the external wrapper 2025-09-19 13:19:30 +04:00