Commit Graph

720 Commits

Author SHA1 Message Date
Jussi Saurio
ae5470f1d0 use default directly 2025-07-28 11:01:26 +03:00
Jussi Saurio
12d8b266a1 Define some helper traits to reduce duplication 2025-07-28 11:01:26 +03:00
Jussi Saurio
7eb52c65d3 Add missing program counter increment 2025-07-28 11:01:26 +03:00
Jussi Saurio
b14124ad3b VDBE/op_column: avoid first cloning text/blob and then copying it again
instead use RefValue to refer to record payload directly and then copy
to register as necessary
2025-07-28 11:01:26 +03:00
Pekka Enberg
6bf6cc28e4 Merge 'Implement the Returning statement for inserts and updates' from Glauber Costa
They are very similar. DELETE is very different, so that one we'll do it
later.

Closes #2276
2025-07-27 09:11:16 +03:00
Glauber Costa
5d8d08d1b6 Implement the Returning statement for inserts and updates
They are very similar. DELETE is very different, so that one we'll
do it later.
2025-07-26 09:01:09 -05:00
FHaggs
54edfa09d5 Replicate the sqlite Kahan-Babaska-Neumaier algorithm 2025-07-25 15:25:29 -03:00
FHaggs
d5049a46c2 Add kahan sum logic 2025-07-25 13:24:19 -03:00
Pekka Enberg
e0e3c52535 Merge 'Simplify sum() aggregation logic' from bit-aloo
This refactors AggContext::Sum by removing the extra bool flag and
simplifying type handling during aggregation:

Closes #2265
2025-07-25 17:57:58 +03:00
bit-aloo
4f8027990d detach the sum and total logic from using has_non_numeric flag 2025-07-25 17:59:19 +05:30
Pekka Enberg
c6c0db19e9 Merge 'Fix schema reparse logic' from Nikita Sivukhin
`maybe_reparse_schema` function introduced in the #2246 was incorrect as
it didn't update `schema_version` for internal schema representation and
basically updated only schema for connection which called
`maybe_reparse_schema`.
This PR fixes this issue by reading schema and cookie value within a
single transaction and updating both schema content and its version for
internal representation.

Reviewed-by: Pedro Muniz (@pedrocarlo)

Closes #2259
2025-07-25 13:24:23 +03:00
Pekka Enberg
669b231714 Merge 'parser: Distinguish quoted identifiers and unify Id into Name enum' from bit-aloo
Closes: #1947
This PR replaces the `Name(pub String)` struct with a `Name` enum that
explicitly models how the name appeared in the source either as an
unquoted identifier (`Ident`) or a quoted string (`Quoted`).
In the process, the separate `Id` wrapper type has been coalesced into
the `Name` enum, simplifying the AST and reducing duplication in
identifier handling logic.
While this increases the size of some AST nodes (notably
`yyStackEntry`).
cc: @levydsa

Reviewed-by: Levy A. (@levydsa)
Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #2251
2025-07-25 12:08:54 +03:00
Glauber Costa
988b16f962 Support ATTACH (read only)
Support for attaching databases. The main difference from SQLite is that
we support an arbitrary number of attached databases, and we are not
bound to just 100ish.

We for now only support read-only databases. We open them as read-only,
but also, to keep things simple, we don't patch any of the insert
machinery to resolve foreign tables.  So if an insert is tried on an
attached database, it will just fail with a "no such table" error - this
is perfect for now.

The code in core/translate/attach.rs is written by Claude, who also
played a key part in the boilerplate for stuff like the .databases
command and extending the pragma database_list, and also aided me in
the test cases.
2025-07-24 19:19:48 -05:00
Nikita Sivukhin
8b0e5b151e simplify parse_schema_rows signature 2025-07-24 22:43:31 +04:00
meteorgan
c48a5ef538 we don't need read_tx return IOResult anymore 2025-07-24 23:19:33 +08:00
meteorgan
08f1803a6b end read tx in op_transaction when write transaction return io 2025-07-24 23:18:29 +08:00
meteorgan
7ef50e0690 fix page_count pragma 2025-07-24 23:18:26 +08:00
bit-aloo
9a54ef214e parser: Distinguish quoted identifiers and unify Id into Name enum
This commit replaces the `Name(pub String)` struct with a `Name` enum that
explicitly models how the name appeared in the source either as an
unquoted identifier (`Ident`) or a quoted string (`Quoted`).

In the process, the separate `Id` wrapper type has been coalesced into the
`Name` enum, simplifying the AST and reducing duplication in identifier
handling logic.

While this increases the size of some AST nodes (notably `yyStackEntry`),
it improves correctness and makes source structure more explicit for
later phases.
2025-07-24 14:40:19 +05:30
Jussi Saurio
92a10f94d8 Merge 'Bail early for read-only virtual tables' from Preston Thorpe
This PR adds a const associated value on the VTabModule trait,
`READONLY` defaulted to `true`, so we can bail early when a write
operation is done on an invalid vtable.
This prevents extensions from having to implement `insert`,`update`,
`delete` just to return `Error::ReadOnly`, and prevents us from having
to step through `VUpdate` just to error out.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2247
2025-07-24 10:12:07 +03:00
Jussi Saurio
52b4c22be9 Merge 'fix: SUM returns correct float for mixed numeric/non-numeric types & return value on empty set' from Axel Tobieson Rova
# Fix SUM aggregate function for mixed types
Fixes #2133
The SUM aggregate function was returning incorrect results when
processing tables with mixed numeric and non-numeric values. According
to SQLite documentation:
> "If any input to sum() is neither an integer nor a NULL, then sum()
returns a floating point value"
[*](https://sqlite.org/lang_aggfunc.html)
Now both SQLite and Turso yield the same output of 44.0.
--
I modified `Sum` to increment only for numeric values, skipping non-
numeric values. However, if we have mixed numeric values or non-numeric
values, we return a float output. Added a flag to keep track of it.
as pointed out by @FHaggs , If there are no non-NULL input rows then
sum() returns NULL but total() returns 0.0. I decided to include it in
this PR as well. Empty was such a natural test case.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2182
2025-07-24 10:08:01 +03:00
PThorpe92
0871a8c7f3 Bail early when we detect a readonly virtual table 2025-07-23 16:57:30 -04:00
PThorpe92
a13fc3515e Fix cargo fmt warning 2025-07-22 21:47:15 -04:00
Glauber Costa
a10d8d7f94 silence clippy errors with features disabled
When compiling with features disabled, there are lots of clippy
warnings. This PR silences them.

For the utils file, I am using a bit of a hammer and just allowing
unused stuff in the whole file. Due to the box of utilities nature of
this file, it'll always be the case that things will be unused depending
on the feature-set.
2025-07-22 20:37:45 -05:00
PThorpe92
9c785ea0ea Merge 'make readonly a property of the database' from Glauber Costa
There's no such thing as a read-only connection.
In a normal connection, you can have many attached databases. Some r/o,
some r/w.
To properly fix that, we also need to fix the OpenWrite opcode. Right
now we are passing a name, which is the name of the table. That
parameter is not used anywhere. That is also not what the SQLite opcode
specifies. Same as OpenRead, the p3 register should be the database
index.
With that change, we can - for now - pass the index 0, which is all we
support anyway, and then use that to test if we are r/o.

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #2232
2025-07-22 21:12:08 -04:00
Levy A.
203239ff30 refactor: safer db_state 2025-07-22 17:20:29 -03:00
Axel
9d05344258 Fix Sum() return value if there are no non-NULL input rows
Add simple fuzz test for total and sum.
2025-07-22 17:38:09 +02:00
Glauber Costa
57a1113460 make readonly a property of the database
There's no such thing as a read-only connection.
In a normal connection, you can have many attached databases. Some
r/o, some r/w.

To properly fix that, we also need to fix the OpenWrite opcode. Right
now we are passing a name, which is the name of the table. That
parameter is not used anywhere. That is also not what the SQLite opcode
specifies. Same as OpenRead, the p3 register should be the database
index.

With that change, we can - for now - pass the index 0, which is all
we support anyway, and then use that to test if we are r/o.
2025-07-22 09:41:32 -05:00
PThorpe92
f7ba8efdbd Switch back to std::mutex because it was an unnecessary change 2025-07-21 19:20:17 -04:00
PThorpe92
411c4f059a Load compile time extensions on the initial db startup instead of once per conn 2025-07-21 19:09:31 -04:00
Glauber Costa
65312baee6 fix opcodes missing a database register
Two of the opcodes we implement (OpenRead and Transaction) should have
an opcode specifying the database to use, but they don't.

Add it, and for now always use 0 (the main database).
2025-07-20 12:27:26 -05:00
Axel
73389abf09 fix: SUM returns float for mixed numeric/non-numeric types 2025-07-20 07:24:41 +02:00
Glauber Costa
6506b3147d implement pragma application_id
Just for completeness, because it is easy.
2025-07-19 20:44:06 -05:00
Jussi Saurio
55151a8061 Fix cases where Insn::Insert needs to seek to ensure correct insertion 2025-07-18 13:48:23 +03:00
Jussi Saurio
9ee6988fc5 VDBE: NewRowid needs to call next() in case op_not_exists() is not called afterwards 2025-07-18 13:48:23 +03:00
Jussi Saurio
4f0ef663e2 btree: add target cell tracking for EQ seeks 2025-07-18 13:48:23 +03:00
Pekka Enberg
02f4bc39b3 Merge 'Reanimate MVCC' from Pekka Enberg
Bit-rot happened. Bring MVCC back from the dead.

Closes #2136
2025-07-18 11:22:49 +03:00
Iaroslav Zeigerman
a88b828268 Fix clippy 2025-07-18 07:28:36 +02:00
Iaroslav Zeigerman
fd042ac4c8 Use IOResult insteaed of CursorResult 2025-07-18 07:28:36 +02:00
Iaroslav Zeigerman
78f3bf3475 Core: Introduce external sorting 2025-07-18 07:28:36 +02:00
Jussi Saurio
1b52b5c764 Merge 'chore: update rust to version 1.88.0' from Nils Koch
This PR updates to version Rust 1.88.0 ([Release
notes](https://releases.rs/docs/1.88.0/)) and fixes all the clippy
errors that come with the new Rust version.
This is possible in the latest Rust version:
```rust
if let Some(foo) = bar && foo.is_cool() {
  ...
}
```
There are three complications in the migration (so far):
- A BUNCH of Clippy warnings (mostly fixed in
https://github.com/tursodatabase/limbo/pull/1827)
- Windows cross compilation failed; linking `advapi32` on windows fixes
it
  - Since Rust 1.87.0, advapi32 is not linked by default anymore
([Release notes](https://github.com/rust-
lang/rust/blob/master/RELEASES.md#compatibility-notes-1),
[PR](https://github.com/rust-lang/rust/pull/138233))
- Rust is more strict with FFIs and aligning pointers now. CI checks
failed with error below
  - Fixed in https://github.com/tursodatabase/turso/pull/2064
```
thread 'main' panicked at
core/ext/vtab_xconnect.rs:64:25:
misaligned pointer dereference: address must be
a multiple of 0x8 but is 0x7ffd9d901554
```

Closes #1807
2025-07-17 23:35:33 +03:00
Pekka Enberg
2b1ee907a9 core/vdbe: Fix op_new_rowid() with MVCC 2025-07-17 14:13:22 +03:00
Pekka Enberg
8e338d3e7a core/vdbe: Fix SetCookie when MVCC is enabled 2025-07-17 14:13:22 +03:00
Pekka Enberg
1fc6126157 core/storage: Allocate page1 lazily for MVCC transactions 2025-07-17 14:13:22 +03:00
Jussi Saurio
5a2efa3077 Merge 'refactor/btree&vdbe: fold index key info (sort order, collations) into a single struct' from Jussi Saurio
These are nearly always used together in some form, so it makes sense to
colocate them, and it also makes many code paths simpler, as we don't
separately pass `collations` and `key_sort_order` around
As a side effect, as the bitfield-based `IndexKeySortOrder` is removed,
we now remove the arbitrary 64 column restriction for indexes, see e.g.
this sim failure which fails to 64+ index columns (not sure why it uses
an index if they are disabled):
https://github.com/tursodatabase/turso/actions/runs/16339391964/job/4615
8045158

Closes #2131
2025-07-17 11:55:56 +03:00
Jussi Saurio
e8199cb26c btree/vdbe: fold index key info (sort order, collations) into a single struct
These are nearly always used together in some form, so it makes sense to colocate
them, and it also makes many code paths simpler.
2025-07-17 10:58:43 +03:00
Nils Koch
8dc066503e chore: fix clippy errors 2025-07-16 19:34:42 +01:00
Levy A.
8e8f1682df add with_schema_mut
removes all repeated `Arc::make_mut`
2025-07-16 13:54:39 -03:00
Levy A.
4c77d771ff only copy schema on writes 2025-07-16 13:54:36 -03:00
Nikita Sivukhin
41482915f6 make unixepoch to return i64 2025-07-16 14:02:56 +04:00
Jussi Saurio
f482424d77 Merge 'small refactor: rename "amount" to "extra_amount"' from Nikita Sivukhin
Small refactoring to reduce confusion (I was caught in this trap and set
`amount` to one in CDC branch during development)
Also, this PR slightly fix broken `concat_ws` emit logic.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2100
2025-07-16 06:51:35 +03:00