Commit Graph

10455 Commits

Author SHA1 Message Date
Jussi Saurio
7ac18a6952 Merge 'Remove some traces in super hot paths in btree' from Preston Thorpe
Particularly we were tracing `ImmutableRecord` / `BTreeKey` which would
then trace the bytes of records. These are super super hot paths and I
think we can probably remove even more to under debug assertions so we
dont eat those atomics/branches all the time.
This PR also introduces the `tracing_release` feature, which turns all
`trace!` and `debug!` macro invocations to noops at compile time, and
makes that feature available for all bindings.
it also removes the unused `lru` dependency, and cleans up the makefile
a bit

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2995
2025-09-11 13:33:25 +03:00
Pekka Enberg
ae3c1fc2a6 Turso 0.2.0-pre.1 2025-09-11 11:44:42 +03:00
Pekka Enberg
5ae4756760 Merge 'Sync package opfs' from Nikita Sivukhin
This PR adds OPFS support to the sync package and also restructure it in
the similar fashion as in #2799
The structure for sync packages looks like this:
1. `@tursodatabase/sync-common` - package with common abstract TS code
shared between node and browser
2. `@tursodatabase/sync` - native package for node
3. `@tursodatabase/sync-browser` - browser package
Also, additional package `@tursodatabase/database-browser-common` was
extracted to share some common OPFS related code between
`@tursodatabase/database-browser` and `@tursodatabase/sync-browser`
packages.
Also, this PR moves JS bindings for sync directly to the
`bindings/javascript` folder because this allows to use all 7 packages
within same workspace.

Closes #3014
2025-09-11 11:44:26 +03:00
TcMits
a7373c9a97 update some helper function 2025-09-11 15:12:38 +07:00
Nikita Sivukhin
020b3f61f3 add CI step for debugging in future 2025-09-11 11:41:28 +04:00
Pekka Enberg
772af0d692 Merge 'Ensure that Connection::query() checks whether its schema is up to date' from Jussi Saurio
Closes #2997
Fixes issue #2997 where connection 2 cannot see tables created by
another connection 1, because `Connection::query()` was not checking
whether its copy of the schema was stale.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3008
2025-09-11 10:10:39 +03:00
Jussi Saurio
f17997fc5d Extract methods for populating indices/views from schema 2025-09-11 09:51:46 +03:00
Jussi Saurio
07944e23b5 Extract common logic for handling sqlite_schema rows 2025-09-11 09:45:40 +03:00
Pekka Enberg
89d1c9a421 whopper: Switch to mmap2 crate to fix Windows build 2025-09-11 08:35:18 +03:00
Pekka Enberg
c5ca259abc whopper: Run cargo clippy --fix 2025-09-11 08:35:18 +03:00
Pekka Enberg
ae920c435d whopper: Ragnarök mode with cosmic rays 2025-09-11 08:35:18 +03:00
Pekka Enberg
a9694c87b1 whopper: A new DST with concurrency
This is a new deterministic simulator for Turso that focuses on finding
concurrency bugs.

You can run whopper with:

```console
penberg@vonneumann turso % SEED=1234 ./whopper/bin/run
   Compiling turso_whopper v0.1.5-pre.3 (/Users/penberg/src/tursodatabase/turso/whopper)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 2.22s
seed = 1234
       .             I/U/D/C
       .             44/19/11/2
       .             68/33/21/2
       |             104/51/29/3
       |             121/69/41/3
      ╱|╲            150/84/51/3
     ╱╲|╱╲           184/97/59/3
    ╱╲╱|╲╱╲          199/105/64/4
   ╱╲╱╲|╱╲╱╲         206/115/69/5
  ╱╲╱╲╱|╲╱╲╱╲        234/138/82/6
 ╱╲╱╲╱╲|╱╲╱╲╱╲       269/164/91/7
```
2025-09-11 08:35:18 +03:00
Pekka Enberg
74c14efdfa sql_generation: Add support for DROP INDEX 2025-09-11 08:35:18 +03:00
Pekka Enberg
b572366a2b core/vbe: Demote op_transaction() logging to debug 2025-09-11 08:35:18 +03:00
Pekka Enberg
ca51a60b3c core/storage: Demote restart_log() logging to debug 2025-09-11 08:35:18 +03:00
Levy A.
4070e05cd2 fix: math function parameter conversion 2025-09-10 20:49:30 -03:00
Nikita Sivukhin
47592f976d fix workflow file 2025-09-11 00:19:48 +04:00
Nikita Sivukhin
d54c271890 add explicit dependency 2025-09-11 00:17:58 +04:00
Nikita Sivukhin
53967cb8fb do not test sync - because it needs server for now 2025-09-11 00:10:54 +04:00
Nikita Sivukhin
1a4d334e47 fix clippy and remove old action yaml 2025-09-11 00:03:01 +04:00
Nikita Sivukhin
fde376f36c fix CI one more time 2025-09-10 23:56:59 +04:00
Jussi Saurio
5f410fd568 Add missing maybe_update_schema() calls
Connection::query() was not properly checking whether it needs to
refresh its schema.
2025-09-10 22:44:26 +03:00
Nikita Sivukhin
1eec8f85f4 fix CI 2025-09-10 23:40:49 +04:00
Nikita Sivukhin
9e9d5f56de yarn install 2025-09-10 23:03:17 +04:00
Nikita Sivukhin
aef0a6c83b fix CI 2025-09-10 23:02:11 +04:00
Nikita Sivukhin
0aa9d59438 rebuild 2025-09-10 22:48:37 +04:00
Preston Thorpe
a93fe6be52 Merge 'refactor cli: readline will write to input_buf' from Lâm Hoàng Phúc
`readline` will directly write to `input_buf` instead of returning
`String`

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2977
2025-09-10 14:39:38 -04:00
Nikita Sivukhin
11e582c058 update update-script 2025-09-10 22:35:58 +04:00
Nikita Sivukhin
d55026f84f opfs for sync in one commit! 2025-09-10 22:35:57 +04:00
TcMits
0ef5712d64 Merge branch 'main' into cli-1 2025-09-11 00:15:27 +07:00
TcMits
5caf9a2640 make it more safe + clippy 2025-09-11 00:14:38 +07:00
Pekka Enberg
656b0af1be Document DEFERRED and IMMEDIATE transaction modes 2025-09-10 19:39:58 +03:00
Pekka Enberg
8ab8b31cb1 Merge 'serverless: Add Connection.reconnect() method' from Mayank
Closes #2595

Closes #2990
2025-09-10 18:36:24 +03:00
Pekka Enberg
ce5e67a483 Merge 'Return parse error for unsupported exprs' from Jussi Saurio
Turns a bunch of `todo!()` crashes into parse errors

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3007
2025-09-10 17:56:22 +03:00
Pekka Enberg
4efa0a57fd Merge 'translate: return parse error for unsupported join types' from Jussi Saurio
We were silently ignoring these before

Closes #3006
2025-09-10 17:14:33 +03:00
PThorpe92
ba1ed72ed8 Add tracing_release feature for benchmarks to compile tracing macros to noops 2025-09-10 09:56:12 -04:00
PThorpe92
f117b2c966 Remove unused lru dependency 2025-09-10 09:55:04 -04:00
PThorpe92
b93ad749a9 Remove some traces in super hot paths in btree 2025-09-10 09:54:32 -04:00
Pekka Enberg
5adb586282 s/2024/2025/ 2025-09-10 16:39:11 +03:00
Pekka Enberg
bb3fbb7962 Merge 'check freelist count in integrity check' from Jussi Saurio
Closes #3003
2025-09-10 16:15:39 +03:00
Jussi Saurio
84ecef2718 Return parse error for unsupported exprs 2025-09-10 16:10:21 +03:00
Jussi Saurio
d7ce781a2a Merge 'Enable the use of indexes in DELETE statements' from Jussi Saurio
Closes #1714
This PR enables the use of an index as the iteration cursor for a point
or range deletion operation. Main changes:
- Use `Delete` opcode for the index that is iterating the rows - avoids
unnecessary seeking on that index, since it's already positioned
correctly
- Fix delete balancing; details below:
### current state
- a deletion may cause a btree rebalancing operation
- to get the cursor back to the right place after a rebalancing, we must
remember what the deleted key was and seek to it
- right now we are using `SeekOp::LT` to move to one slot BEFORE the
deleted key, so that if we delete rows in a loop, the following `Next()`
call will put us back into the right place
### problem
- When we delete multiple rows, we always iterate forwards. Using
`SeekOp::LT` implies backwards iteration, but it works OK for table
btrees since the cursor never remains on an internal node, because table
internal cells do not have payloads. However: this behavior is
problematic for indexes because we can effectively end up skipping
visiting a page entirely. Honestly: despite spending some debugging the
_old_ code, I still don't remember what exactly causes this to happen.
:) It's one of the `iter_dir` specific behaviors in `indexbtree_move_to`
or `get_prev_record()`, but I'm too tired to spend more time figuring it
out. I had the reason in my head before going on vacation, but it was
evicted from the cache it seems...
### solution
use `SeekOp::GE { eq_only: true }` instead and make the next call to
`Next()` a no-op instead. This has the same effect as SeekOp::LT +
next(), but without introducing bugs due to `LT` being implied backwards
iteration.

Reviewed-by: Nikita Sivukhin (@sivukhin)

Closes #2981
2025-09-10 16:00:54 +03:00
Jussi Saurio
eb2710438c translate: return parse error for unsupported join types 2025-09-10 15:46:52 +03:00
Jussi Saurio
752627a213 Merge 'Add scripts that help debug bugs from simulator' from Jussi Saurio
1. Add script that cleans simulator logs into just the SQL statements
2. Add script that bisects a set of SQL statements to find the minimal
prefix set of statements that fails SQLite integrity check

Closes #2999
2025-09-10 15:44:54 +03:00
Jussi Saurio
e3594d0ae0 make the comment for skip_advance more accurate 2025-09-10 15:38:57 +03:00
Jussi Saurio
32c4f5ce81 Assert that skip_advance is not set in the middle of a seek 2025-09-10 15:38:57 +03:00
Jussi Saurio
4eb61a9527 Add gaps to update/delete fuzz test where clauses 2025-09-10 15:38:57 +03:00
TcMits
284ade3497 add query mode 2025-09-10 19:21:21 +07:00
Pekka Enberg
860627942a Merge 'core: Rename IO::run_once() to IO::step()' from Pekka Enberg
The `run_once()` name is just a historical accident. Furthermore, it now
started to appear elsewhere as well, so let's just call it IO::step() as
we should have from the beginning.

Closes #3001
2025-09-10 15:11:28 +03:00
Jussi Saurio
8b7c0334d4 RUFFFffff 2025-09-10 15:00:08 +03:00