Commit Graph

10788 Commits

Author SHA1 Message Date
Pere Diaz Bou
724bc94f96 core/mvcc/cursor: rewind with btree 2025-11-10 16:48:13 +01:00
Pere Diaz Bou
a7614267af core/mvcc/cursor: next with btree 2025-11-10 16:48:13 +01:00
Pere Diaz Bou
38f6d20def core/mvcc/cursor: CursorPosition::Loaded include if points to btree 2025-11-10 16:48:13 +01:00
Jussi Saurio
a47ac6cb96 Commit changes to workspace Cargo.lock 2025-11-10 11:58:09 +02:00
Jussi Saurio
d0da6b5d16 Merge 'Fix seek not applying correct affinity to seek expr' from Pedro Muniz
Depends on #3923 .
To have similar semantics to how `op_compare` works, we need to apply an
affinity to the values referenced in the `SeekKey` that is used for
seeking. This means keeping some affinity metadata for the `WhereTerms`
in the optimization phase, then before seeking, we emit an affinity
conversion.  Had to dig deep in the sqlite code to understand this
better.
Unfortunately, we cannot have just one compare function to rule them all
here, as we have a specialized/optimized compare code to handle records
that have not yet been deserialized.
Closes #3707

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3925
2025-11-10 11:28:29 +02:00
Jussi Saurio
b024fdb17d Merge 'core: update aegis' from Daeho Ro
It seems that the build on macos arm is failing with `aegis` v0.9.0.
So, here I update `aegis`.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3561
2025-11-10 11:27:01 +02:00
pedrocarlo
176fa283bf add some ai generated queries to test for affinity related queries 2025-11-10 11:15:54 +02:00
pedrocarlo
32535ef4ed only emit affinity check on index seek + check if affinity is necessary at all 2025-11-10 11:15:54 +02:00
pedrocarlo
27e234f949 add affinity of the expr in the seek key, and emit affinity instruction before seeking 2025-11-10 11:15:54 +02:00
Pekka Enberg
e929c252b4 Merge 'bindings/java: implement stream binding methods (int, InputStream, int) in JDBC4PreparedStatement' from Orange banana
## Purpose
* Implement `setAsciiStream(int, InputStream, int)`,
`setUnicodeStream(int, InputStream, int)`, and `setBinaryStream(int,
InputStream, int)` methods in JDBC4PreparedStatemen
## Changes
* `setAsciiStream(int, InputStream, int)`: Reads ASCII bytes, converts
to `String` using `US_ASCII` and binds with `bindText()`.
* `setUnicodeStream(int, InputStream, int)`: Reads bytes as `UTF-8`
encoded text and binds with `bindText()`.
* `setBinaryStream(int, InputStream, int)`: Reads raw bytes and binds
with `bindBlob()`.
* Added consistent error handling and validation
  * null stream - `bindNull()`
  * Negative length - throws `SQLException`
  * Empty stream  - Empty String or Empty Array
  * I/O errors - throw `SQLException`
* Ensures consistency between `setXxxStream` and `getXxxStream` methods,
so data written and read use the same encoding.
## Related Issue
* #615

Reviewed-by: Kim Seon Woo (@seonWKim)

Closes #3917
2025-11-10 11:07:08 +02:00
Pekka Enberg
d872237ca8 Merge 'workflows: Add GITHUB_TOKEN to all Nyrkiö steps' from Henrik Ingo
Previously we didn't use GITHUB_TOKEN for anything. But now that PR
meta-data must be fetched with a extra GitHub API call, then PRs at
least will always nedd GITHUB_TOKEN.

Closes #3918
2025-11-10 09:03:38 +02:00
Pekka Enberg
b74ddf30f9 Merge 'extensions/vtabs: implement remaining opcodes' from Preston Thorpe
The only real benefit right now here is the ability to rename virtual
tables.
Then this now properly calls `VBegin` at the start of a vtab write
transaction, despite none of our extensions needing or implementing
transactions at this point.
```console
explain insert into t values ('key','value');
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     10    0                    0   Start at 10
1     VOpen              0     0     0                    0    t
2     VBegin             0     0     0                    0   
3     Null               0     1     0                    0   r[1]=NULL
4     Null               0     3     0                    0   r[3]=NULL
5     String8            0     4     0     key            0   r[4]='key'
6     String8            0     5     0     value          0   r[5]='value'
7     VUpdate            0     5     1                    0   args=r[1..5]
8     Close              0     0     0                    0   
9     Halt               0     0     0                    0   
10    Transaction        0     2     1                    0   iDb=0 tx_mode=Write
11    Goto               0     1     0                    0   
Exiting Turso SQL Shell.
```

Closes #3930
2025-11-10 09:03:07 +02:00
Pekka Enberg
7891be96fd Merge 'Refactor affinity conversions for reusability' from Pedro Muniz
Depends on #3920
Moves some code around so it is easier to reuse and less cluttered in
`execute.rs`, and changes how `compare` works. Instead of mutating some
register, we now just return the possible `ValueRef` representation of
that affinity. This allows other parts of the codebase to reuse this
logic without needing to have an owned `Value` or a `&mut Register`

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3923
2025-11-10 09:02:22 +02:00
Pekka Enberg
2be515247f Merge 'Create AsValueRef trait to allow us to be agnostic over ownership of Value or ValueRef' from Pedro Muniz
Depends on #3919
Also change `op_compare` to reuse the same compare_immutable logic
First step to finish #2304

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3920
2025-11-10 09:01:59 +02:00
Pekka Enberg
4bb0edac5e Merge 'Move value functions to separate file' from Pedro Muniz
Makes it easier to visualize what is related to Value and what is
related to opcodes. This will also facilitate in my next PR to
generalize certain function over `Value` and `ValueRef` as listed in
#2304

Closes #3919
2025-11-10 09:01:29 +02:00
Preston Thorpe
49f9b74c56 Merge 'Avoid heavy macro' from Nikita Sivukhin
Rewrite `parse_modifier` function because its current version lead to
enormous amount of generated LLVM code which significantly increase
compilation time
```sh
$> cargo llvm-lines
  Lines                  Copies               Function name
  -----                  ------               -------------
  1322611                29544                (TOTAL)
   278720 (21.1%, 21.1%)     1 (0.0%,  0.0%)  turso_core::functions::datetime::parse_modifier
```
Before:
```sh
$> cargo check
warning: `turso_core` (lib) generated 2 warnings
    Finished `dev` profile [unoptimized] target(s) in 5.61s
```
After:
```sh
$> cargo check
warning: `turso_core` (lib) generated 2 warnings
    Finished `dev` profile [unoptimized] target(s) in 2.24s
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3929
2025-11-09 11:37:06 -05:00
PThorpe92
5c207618a7 Fix extensions py test 2025-11-09 11:35:57 -05:00
PThorpe92
b443b09516 Remove VRollback and VCommit as they are unused opcodes in sqlite 2025-11-09 11:27:09 -05:00
PThorpe92
94b6d254a9 Fix comment on vtab_txn_states 2025-11-09 11:08:52 -05:00
PThorpe92
993c9d34b4 Rollback vtab txns when when err code is present in Halt 2025-11-09 11:07:43 -05:00
PThorpe92
30de27b8c6 Add test in extensions.py for virtual table renaming 2025-11-09 11:07:43 -05:00
PThorpe92
8bef159582 adjust testing extension to add some test assertions for new features 2025-11-09 11:07:43 -05:00
PThorpe92
f35ccfba17 Add support for renaming virtual tables 2025-11-09 11:07:42 -05:00
PThorpe92
40f7ddb28e Add VBegin, VRename, VRollback and VCommit opcodes to COMPAT.md 2025-11-09 11:07:42 -05:00
PThorpe92
e09d9eb720 Add VBegin, VRename, VRollback and VCommit opcodes 2025-11-09 11:07:42 -05:00
Nikita Sivukhin
6e0e4e20ab add comment 2025-11-09 16:04:16 +04:00
Nikita Sivukhin
ec91829835 rewrite parse_modifier function because its current version lead to enormous amount of generated LLVM code which significantly increase compilation time
- cargo llvm-lines
  Lines                  Copies               Function name
  -----                  ------               -------------
  1322611                29544                (TOTAL)
   278720 (21.1%, 21.1%)     1 (0.0%,  0.0%)  turso_core::functions::datetime::parse_modifier

- before:
$> cargo check
warning: `turso_core` (lib) generated 2 warnings
    Finished `dev` profile [unoptimized] target(s) in 5.61s

- after:
$> cargo check
warning: `turso_core` (lib) generated 2 warnings
    Finished `dev` profile [unoptimized] target(s) in 2.24s
2025-11-09 15:15:45 +04:00
Nikita Sivukhin
39daf6df37 ask agent to rewrite compile-heavy function 2025-11-09 15:15:45 +04:00
Pekka Enberg
23be2b2787 Merge 'Stop blob json parsing at null terminator' from Duy Dang
Close #3912

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #3915
2025-11-09 10:03:56 +02:00
Preston Thorpe
43119c1a1e Merge 'Throw an error when adding generated columns via an alter table' from Rohith Suresh
Fixes #3882 and #3887
```bash
turso> create table t(a);
turso> alter table t add column b as (null);
  × Parse error: Alter table does not support adding generated columns
```
```bash
turso> create table t(a); create table s(a);  alter table s add column b references t(a);
turso> select * from sqlite_master;
┌───────┬──────┬──────────┬──────────┬────────────────────────────────────────────────────────┐
│ type  │ name │ tbl_name │ rootpage │ sql                                                    │
├───────┼──────┼──────────┼──────────┼────────────────────────────────────────────────────────┤
│ table │ t    │ t        │        2 │ CREATE TABLE t (a)                                     │
├───────┼──────┼──────────┼──────────┼────────────────────────────────────────────────────────┤
│ table │ s    │ s        │        3 │ CREATE TABLE s (a, b, FOREIGN KEY (b) REFERENCES t(a)) │
└───────┴──────┴──────────┴──────────┴────────────────────────────────────────────────────────┘
turso> 
```

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3884
2025-11-08 09:42:31 -05:00
RS2007
3a562f734c feat: alter table disallow generated columns, support foreign keys for alter table 2025-11-08 13:45:17 +05:30
Pekka Enberg
36c3489c53 github: Make Nyrkiö comment less
We're getting tons of false positives in PRs so let's make Nyrkiö less
verbose.
2025-11-08 10:03:22 +02:00
Pekka Enberg
20364ea2c0 Merge 'core/translate: Remove unused ParamState' from Preston Thorpe
`ParamState` was a leftover from when we were doing some hacky rewriting
of anonymous Parameters/`Expr::Variable`s internally before it was moved
to the parser.

Closes #3926
2025-11-08 10:01:31 +02:00
PThorpe92
dd2e3e8e16 Fix clippy warning 2025-11-07 20:04:57 -05:00
PThorpe92
a012e98bfa core/translate remove unused ParamState and some minor refactoring 2025-11-07 19:18:10 -05:00
pedrocarlo
9007340e99 change convert function to accept 1 value 2025-11-07 12:47:39 -03:00
pedrocarlo
9f350f7fd9 change Text variant in ValueRef to hold a TextRef that can automatically convert to &str avoiding string allocations everywhere 2025-11-07 12:47:39 -03:00
pedrocarlo
5cfc898049 clippy 2025-11-07 12:47:39 -03:00
pedrocarlo
af05d9ba10 move more affinity logic to separate file and avoid more clones 2025-11-07 12:47:39 -03:00
pedrocarlo
61036d5f51 move affinity handling to separate file 2025-11-07 12:47:39 -03:00
pedrocarlo
99c596d340 separate part of comparison logic for reuse later with seek operations 2025-11-07 12:47:39 -03:00
pedrocarlo
ce3527df40 chnage RecordCompare::compare to use an iterator 2025-11-07 12:47:39 -03:00
pedrocarlo
e5e97a5b0a for op_compare reuse compare_immutable 2025-11-07 12:44:57 -03:00
pedrocarlo
b1f4554420 change compare_immutable signature to accept an iterator of AsValueRef 2025-11-07 12:21:59 -03:00
pedrocarlo
9c2324cbd8 move some more functions to be scoped under Value 2025-11-07 12:10:27 -03:00
pedrocarlo
52968b7913 create AsValueRef trait 2025-11-07 12:10:27 -03:00
pedrocarlo
44cab91722 move Value functions to separate file 2025-11-07 12:10:27 -03:00
Pekka Enberg
a6593d109e Merge 'Toy index improvements' from Nikita Sivukhin
This PR implements more sophisticated algorithm in the toy vector sparse
index: now we enumerate components based on the frequency (in order to
check unpopular "features" first) and also estimate length threshold
which can give us better results compared with current top-k set.
Also, this PR adds optional `delta` parameter which can enable
approximate search which will return results with score not more than
`delta` away from the optimal.
In order to implement this index method - index code were slightly
adjusted in order to allow to store some non-key payload in the index
rows. So, now index can hold N columns where first K <= N columns will
be used as identity (before that K always was equal to N).

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3862
2025-11-07 08:29:47 +02:00
Preston Thorpe
4e8b4c96d3 Merge 'use dyn DatabaseStorage instead of DatabaseFile' from Nikita Sivukhin
Partial sync for sync engine will need to implement its own version of
`DatabaseStorage` which willl load database pages on demand

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3922
2025-11-06 15:11:13 -05:00
Preston Thorpe
29c5271c44 Merge 'Prevent DROP TABLE when table is referenced by foreign keys' from Joao Faria
## Related issue
- closes #3885
## Description
Add a check to reject dropping a table when PRAGMA foreign_keys=ON and
the table is referenced by foreign keys

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3913
2025-11-06 15:10:05 -05:00