Commit Graph

6600 Commits

Author SHA1 Message Date
Levy A.
1f57ab02cf feat: instrument WindowsIO functions 2025-07-27 20:39:49 -03:00
Levy A.
c95c6b67ee fix: thread-safe WindowsFile 2025-07-27 20:39:49 -03:00
Pekka Enberg
a2a31a520f Remove CreateTable from COMPAT.md
There is no such opcode.
2025-07-27 09:12:31 +03:00
Pekka Enberg
6bf6cc28e4 Merge 'Implement the Returning statement for inserts and updates' from Glauber Costa
They are very similar. DELETE is very different, so that one we'll do it
later.

Closes #2276
2025-07-27 09:11:16 +03:00
Pekka Enberg
86c97fca6d Merge 'Fix sum() to follow the SQLite semantics' from FamHaggs
### Follow SUM [spec](https://sqlite.org/lang_aggfunc.html)
This PR updates the `SUM` aggregation logic to follow the
[Kahan–Babushka–Neumaier summation
algorithm](https://en.wikipedia.org/wiki/Kahan_summation_algorithm),
consistent with SQLite’s implementation. It improves the numerical
stability of floating-point summation.This fixes issue #2252 . I added a
fuzz test to ensure the compatibility of the implementations
I also fixed the return types for `SUM` to match SQLite’s documented
behavior. This was previously discussed in
[#2182](https://github.com/tursodatabase/turso/pull/2182), but part of
the logic was later unintentionally overwritten by
[#2265](https://github.com/tursodatabase/turso/pull/2265).
I introduced two helper functions, `apply_kbn_step` and
`apply_kbn_step_int`, in `vbde/execute.rs` to handle floating-point and
integer accumulation respectively. However, I’m new to this codebase and
would welcome constructive feedback on whether there’s a better place
for these helpers.

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #2270
2025-07-27 09:08:34 +03:00
Pekka Enberg
ab39ea54c7 Merge 'Fix error handling when binding column references while translating the UPDATE statement' from Iaroslav Zeigerman
Closes #1968

Reviewed-by: bit-aloo (@Shourya742)

Closes #2273
2025-07-27 09:05:17 +03:00
Pekka Enberg
6d88c6851b Merge 'io_uring: use Arc pointer for user data of entries' from Preston Thorpe
trying to pull bite sized adjustments out of other open PR's

Closes #2281
2025-07-27 09:04:35 +03:00
Pekka Enberg
b41f337616 Merge 'Stop checkpointing the entire WAL after every write when wal frame size > threshold' from Preston Thorpe
I know I have #2179 and now #2278 open both working on checkpointing,
but I discovered during async IO benchmarks that once we hit
`should_checkpoint` (there are greater than 1000 frames in the WAL), we
will trigger checkpoint on cache flush every single write, and since we
don't update `nbackfills` in the case that everything was backfilled,
every checkpoint will backfill every single frame to the db file 💀
At least this can be reviewed+merged faster and easier.
This also fixes the return value of `pragma wal_checkpoint;` to match
sqlite.
Sqlite returns the # of possible frames (`shared.max_frame -
shared.nbackfills`, and then the number of frames we successfully
checkpointed `ongoing_checkpoint.max_frame -
ongoing_checkpoint.min_frame + 1`)
If `pragma wal_checkpoint;` is called again when there are no pages to
backfill, then it returns the value from the previous checkpoint.
<img width="781" height="293" alt="image" src="https://github.com/user-
attachments/assets/9aa78a6c-aa18-444d-82bd-398d684fed75" />

Closes #2280
2025-07-27 09:04:11 +03:00
PThorpe92
e6737d923d Return correct value for pragma checkpoint 2025-07-26 23:09:40 -04:00
PThorpe92
fb611390c0 Update test to use realistic expectations for should_checkpoint in cacheflush 2025-07-26 23:03:51 -04:00
PThorpe92
7c027fed8c Keep should_checkpoint logic for now until greater checkpointing is fixed 2025-07-26 23:03:51 -04:00
PThorpe92
6644036be4 Stop checkpointing after every write when wal frame size > threshold 2025-07-26 23:03:47 -04:00
PThorpe92
60c37eb036 Merge 'implement the pragma encoding' from Glauber Costa
Do not allow setting it. That ship has sailed around 2005.

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #2282
2025-07-26 21:31:27 -04:00
PThorpe92
22df76d37d Merge 'compat police' from Glauber Costa
All those opcodes seem implemented.

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #2284
2025-07-26 21:28:28 -04:00
Glauber Costa
421ab44cf2 compat police
All those opcodes seem implemented.
2025-07-26 20:06:21 -05:00
PThorpe92
3f2414aceb Merge 'remove upsert statement from COMPAT.md' from Glauber Costa
There's no such statement in SQLite.

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #2283
2025-07-26 20:44:43 -04:00
Glauber Costa
f0b1abd567 remove upsert statement from COMPAT.md
There's no such statement in SQLite.
2025-07-26 19:39:32 -05:00
Glauber Costa
b8ee38868d implement the pragma encoding
Do not allow setting it. That ship has sailed around 2005.
2025-07-26 19:37:39 -05:00
PThorpe92
735026b502 Use Arc pointer for user data and save indirection when processing sqe/cqes 2025-07-26 16:35:40 -04:00
Pekka Enberg
d19bd0f24a Merge 'Update limbo -> turso in manual.md' from stano
Closes #2279
2025-07-26 18:56:34 +03:00
Glauber Costa
5d8d08d1b6 Implement the Returning statement for inserts and updates
They are very similar. DELETE is very different, so that one we'll
do it later.
2025-07-26 09:01:09 -05:00
stano
4ceea1a473 Update limbo -> turso in manual.md 2025-07-26 15:58:22 +03:00
Iaroslav Zeigerman
6f63327320 fix overlooked tests 2025-07-26 04:51:44 -07:00
Iaroslav Zeigerman
f13b9105b9 Fix error handling when binding column references while translating the UPDATE statement 2025-07-26 04:51:42 -07:00
Pekka Enberg
cc5d4dc3ba Merge 'support doubly qualified identifiers' from Glauber Costa
Closes #2271
2025-07-26 11:31:42 +03:00
Pekka Enberg
2ae5e7bd76 Merge 'mark detach as supported' from Glauber Costa
Honestly, I didn't even notice we had a compat entry for it because it
seems so trivial. But it was done as part of ATTACH.

Closes #2272
2025-07-26 11:31:07 +03:00
Glauber Costa
02e57c81f7 mark detach as supported
Honestly, I didn't even notice we had a compat entry for it because it
seems so trivial. But it was done as part of ATTACH.
2025-07-25 15:35:13 -05:00
Glauber Costa
b5927dcfd5 support doubly qualified identifiers 2025-07-25 14:52:45 -05:00
FHaggs
ef88b9914a Fix clippy warnings 2025-07-25 15:41:49 -03:00
Pekka Enberg
273c2923f0 Merge 'Fix get_column_name() when a column name doesn't exist' from meteorgan
we would encounter a panic in `display.rs` when a column name does not
exist.
```
turso> select * from (select 1);

thread 'main' panicked at core/translate/display.rs:252:50:
called `Option::unwrap()` on a `None` value
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
after this PR:
```
turso> select * from (select 1);
2025-07-25T15:52:24.241294Z DEBUG ThreadId(01) run_cmd:translate:optimize_plan: turso_core::translate::optimizer: 51: plan_sql="SELECT subquery_0.0 FROM (SELECT 1 FROM) AS subquery_0"
2025-07-25T15:52:24.242036Z DEBUG ThreadId(01) step:begin_read_tx:begin_read_tx: turso_core::storage::wal: 552: begin_read_tx(min_frame=1, max_frame=0, lock=0, max_frame_in_wal=0)
2025-07-25T15:52:24.242349Z DEBUG ThreadId(01) step:commit_txn:end_read_tx:end_read_tx: turso_core::storage::wal: 566: end_read_tx(lock=0)
┌───┐
│ . │
├───┤
│ 1 │
└───┘
```

Closes #2269
2025-07-25 21:38:09 +03:00
FHaggs
ab8040aa89 Add fuzz test for float sums 2025-07-25 15:26:43 -03:00
FHaggs
54edfa09d5 Replicate the sqlite Kahan-Babaska-Neumaier algorithm 2025-07-25 15:25:29 -03:00
Pekka Enberg
c7996652fe Merge 'bindings/javascript: Switch to napi v3' from Diego Reis
Switch to napi [v3](https://napi.rs/blog/announce-v3).
With the exception of `Statement.iterate()`,  the behavior is preserved.
I had to temporarily remove it because the trait `Generator` doesn't
supports the new lifetime scoped values, I already brought this issue in
napi's discord server and it should be fixed soon.

Closes #2262
2025-07-25 20:19:37 +03:00
Diego Reis
220a548f57 workflow: Fix macOS universal binary build 2025-07-25 14:01:39 -03:00
FHaggs
f0ffff3c8e Modify AggContext to support the kahan algorithm 2025-07-25 13:25:25 -03:00
FHaggs
d5049a46c2 Add kahan sum logic 2025-07-25 13:24:19 -03:00
meteorgan
b5a18d7dc9 fix get_column_name() when column name doesn't exist 2025-07-25 23:49:31 +08:00
Pekka Enberg
e0e3c52535 Merge 'Simplify sum() aggregation logic' from bit-aloo
This refactors AggContext::Sum by removing the extra bool flag and
simplifying type handling during aggregation:

Closes #2265
2025-07-25 17:57:58 +03:00
Diego Reis
0f95cf7751 bind/js: Switch to napi v3 2025-07-25 11:45:57 -03:00
Pekka Enberg
308f20a5cf turso-serverless: Fix wrong error import
```
  Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/Users/penberg/src/tursodatabase/turso/packages/turso-serverless/dist/error' imported from /Users/penberg/src/tursodatabase/turso/packages/turso-serverless/dist/protocol.js

  Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/Users/penberg/src/tursodatabase/turso/packages/turso-serverless/dist/error' imported from /Users/penberg/src/tursodatabase/turso/packages/turso-serverless/dist/protocol.js
      at finalizeResolution (node:internal/modules/esm/resolve:275:11)
      at moduleResolve (node:internal/modules/esm/resolve:860:10)
      at defaultResolve (node:internal/modules/esm/resolve:984:11)
      at ModuleLoader.defaultResolve (node:internal/modules/esm/loader:780:12)
      at #cachedDefaultResolve (node:internal/modules/esm/loader:704:25)
      at ModuleLoader.resolve (node:internal/modules/esm/loader:687:38)
      at ModuleLoader.getModuleJobForImport (node:internal/modules/esm/loader:305:38)
      at ModuleJob._link (node:internal/modules/esm/module_job:137:49)

  ✘ integration-tests/serverless.test.mjs exited with a non-zero exit code: 1
```
2025-07-25 17:11:13 +03:00
Pekka Enberg
b1c0ba4da0 stress: Allow multi-threading but warn about it 2025-07-25 16:53:02 +03:00
Pere Diaz Bou
805bcfe633 Merge 'Ignore WAL frames after bad checksum' from Pere Diaz Bou
SQLite basically ignores bad frames instead of panicking, let's try to
do the same.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1956
2025-07-25 15:31:12 +02:00
bit-aloo
4f8027990d detach the sum and total logic from using has_non_numeric flag 2025-07-25 17:59:19 +05:30
bit-aloo
f389c31ac9 remove bool from sum variant in AggContext 2025-07-25 17:55:53 +05:30
Pekka Enberg
76cf3444c6 Turso 0.1.3-pre.5 2025-07-25 13:24:33 +03:00
Pekka Enberg
c6c0db19e9 Merge 'Fix schema reparse logic' from Nikita Sivukhin
`maybe_reparse_schema` function introduced in the #2246 was incorrect as
it didn't update `schema_version` for internal schema representation and
basically updated only schema for connection which called
`maybe_reparse_schema`.
This PR fixes this issue by reading schema and cookie value within a
single transaction and updating both schema content and its version for
internal representation.

Reviewed-by: Pedro Muniz (@pedrocarlo)

Closes #2259
2025-07-25 13:24:23 +03:00
Nikita Sivukhin
020d567e78 fix clippy 2025-07-25 13:55:37 +04:00
Pekka Enberg
669b231714 Merge 'parser: Distinguish quoted identifiers and unify Id into Name enum' from bit-aloo
Closes: #1947
This PR replaces the `Name(pub String)` struct with a `Name` enum that
explicitly models how the name appeared in the source either as an
unquoted identifier (`Ident`) or a quoted string (`Quoted`).
In the process, the separate `Id` wrapper type has been coalesced into
the `Name` enum, simplifying the AST and reducing duplication in
identifier handling logic.
While this increases the size of some AST nodes (notably
`yyStackEntry`).
cc: @levydsa

Reviewed-by: Levy A. (@levydsa)
Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #2251
2025-07-25 12:08:54 +03:00
Pekka Enberg
c6d4a5c3ed Rename DatabaseIndexer to DatabaseCatalog
Avoid confusion with actual database indexes.
2025-07-25 10:36:33 +03:00
Pekka Enberg
97a2d36ae2 Merge 'Support ATTACH (read only)' from Glauber Costa
Support for attaching databases. The main difference from SQLite is that
we support an arbitrary number of attached databases, and we are not
bound to just 100ish.
We for now only support read-only databases. We open them as read-only,
but also, to keep things simple, we don't patch any of the insert
machinery to resolve foreign tables.  So if an insert is tried on an
attached database, it will just fail with a "no such table" error - this
is perfect for now.
The code in core/translate/attach.rs is written by Claude, who also
played a key part in the boilerplate for stuff like the .databases
command and extending the pragma database_list, and also aided me in the
test cases.

Closes #2235
2025-07-25 10:33:36 +03:00