aljaz/turso
1
0
Fork 0
mirror of https://github.com/aljazceru/turso.git synced 2026-01-03 16:34:19 +01:00
Code Issues Packages Projects Releases Wiki Activity
4,725 Commits 1 Branch 0 Tags
ef28906be3cbf2cbc09ac4ed47bacca2047cf1c6
Commit Graph

4725 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
PThorpe92
ef28906be3 Update extensions README with example for xConnect 2025-05-24 17:10:26 -04:00
PThorpe92
cf163f2dc0 Prevent double free in ext connection 2025-05-24 16:49:52 -04:00
PThorpe92
1cacbf1f0d Close statements in extension tests, and use mut pointers for stmt 2025-05-24 16:45:25 -04:00
PThorpe92
d63f9d8cff Make sure all resources are cleaned up properly in xconnect 2025-05-24 16:38:33 -04:00
PThorpe92
a4ed464ec4 Add some traces for errors in xconnect 2025-05-24 15:44:06 -04:00
PThorpe92
0decafbbc1 Use transparent struct in public api wrapper for vtab connect 2025-05-24 15:32:14 -04:00
PThorpe92
2e4343402e Add null checks to prevent double frees in vtab connections 2025-05-24 15:20:09 -04:00
PThorpe92
999205f896 Add documentation for connection api 2025-05-24 15:19:37 -04:00
PThorpe92
c5d364064a Add python tests for xConnect behavior and testing extension 2025-05-24 14:49:59 -04:00
PThorpe92
687edefcdf Add option to py tests to create temporary db with clone of testing.db 2025-05-24 14:49:59 -04:00
PThorpe92
faa12987b4 Add test case to table stats extension 2025-05-24 14:49:59 -04:00
PThorpe92
4142d813c0 Change method name to bind_at to better reflect args in ext Statement 2025-05-24 14:49:58 -04:00
PThorpe92
a2f8b2dfea Fix add check for invalid argv index for vtab constraints in main loop 2025-05-24 14:49:58 -04:00
PThorpe92
58e1d5a4f8 Add additional test vtable extension for querying core 2025-05-24 14:49:58 -04:00
PThorpe92
d11ef6b9c5 Add execute method to xConnect db interface for vtables 2025-05-24 14:49:58 -04:00
PThorpe92
c2ec6caae1 Finish integrating xConnect into vtable open api 2025-05-24 14:49:58 -04:00
PThorpe92
cbd7245677 Update Vtable open method to accept core db connection 2025-05-24 14:49:58 -04:00
PThorpe92
2c784070f1 Impl Default for ext Value 2025-05-24 14:49:58 -04:00
PThorpe92
f61ccc78e8 Add from_ffi_ptr method to create OwnedValue from Ext type without taking ownership 2025-05-24 14:49:58 -04:00
PThorpe92
d51614a4fd Create extern functions to support vtab xConnect in core/ext 2025-05-24 14:49:57 -04:00
Jussi Saurio
02e7726249 Merge 'UNION ALL' from Jussi Saurio 
Adds support for `UNION ALL` and introduces `Plan::CompoundSelect` so
that it can be extended to support `UNION/EXCEPT/INTERSECT` as well
```sql
do_execsql_test_on_specific_db {:memory:} select-union-all-1 {
  CREATE TABLE t1(x INTEGER);
  CREATE TABLE t2(x INTEGER);
  CREATE TABLE t3(x INTEGER);

  INSERT INTO t1 VALUES(1),(2),(3);
  INSERT INTO t2 VALUES(4),(5),(6);
  INSERT INTO t3 VALUES(7),(8),(9);

  SELECT x FROM t1
  UNION ALL
  SELECT x FROM t2
  UNION ALL
  SELECT x FROM t3;
} {1
2
3
4
5
6
7
8
9}

do_execsql_test_on_specific_db {:memory:} select-union-all-with-filters {
  CREATE TABLE t4(x INTEGER);
  CREATE TABLE t5(x INTEGER);
  CREATE TABLE t6(x INTEGER);

  INSERT INTO t4 VALUES(1),(2),(3),(4);
  INSERT INTO t5 VALUES(5),(6),(7),(8);
  INSERT INTO t6 VALUES(9),(10),(11),(12);

  SELECT x FROM t4 WHERE x > 2
  UNION ALL
  SELECT x FROM t5 WHERE x < 7
  UNION ALL
  SELECT x FROM t6 WHERE x = 10;
} {3
4
5
6
10}
```
Supports LIMIT. Currently does not support `WITH()`, `OFFSET` or `ORDER
BY` and explicitly returns a parse error if those are present.

Closes #1541
 2025-05-24 13:41:57 +03:00
Jussi Saurio
8ed5334ca7 tests/fuzz: add compound_select_fuzz() 2025-05-24 13:12:41 +03:00
Jussi Saurio
f6443ae742 Support LIMIT with UNION ALL 2025-05-24 13:12:41 +03:00
Jussi Saurio
08bda9cc58 UNION ALL 2025-05-24 13:12:41 +03:00
Jussi Saurio
0b2c3298aa Merge 'refactor: introduce walk_expr() and walk_expr_mut() to reduce repetitive pattern matching' from Jussi Saurio 
We do a lot of
```rust
match expr {
  ...
}
```
just to find some specific case like `Expr::Column` deep inside an
expression tree. This PR introduces new helpers `walk_expr()` and
`walk_expr_mut()` that handle the tree-walking part, and the business
logic functions where this tree traversal is used can focus on the exact
enum variants of `Expr` that they are interested in.

Closes #1564
 2025-05-23 22:00:20 +03:00
Jussi Saurio
70433e100d Merge 'btree: fix infinite looping in backwards iteration of btree table' from Jussi Saurio 
Closes #1562
Existing "fuzz test" (not really fuzz, but kinda) didn't catch this due
to `LIMIT 3` clause

Closes #1563
 2025-05-23 21:46:16 +03:00
Jussi Saurio
2e095e6d03 Merge 'Add some comments for values statement' from meteorgan 
follow up: #1549
simultaneously, address a warning in CI as the package `sqlite3-parser`
has been renamed to `limbo_sqlite3_parser`.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1565
 2025-05-23 17:30:30 +03:00
meteorgan
3bf0ce7fb3 Add some comments for values statement 2025-05-23 22:11:34 +08:00
Jussi Saurio
2ea7321e63 Merge 'fix bindings/wasm wal file creation by implementing generate_random_number' from 오웬 
In binding/wasm `generate_random_number`'s implementation depends on an
external function `Math_random` which does not exist
A related error can be observed when viewing `limbo-opfs-test.html`
without an existing database. (Note: it will execute successfully if the
wal file already exists, so you may wish view the html in a
private/incognito session to ensure an existing file from a previous run
is not used)
![Screenshot 2025-05-22 at 22 48 52](https://github.com/user-
attachments/assets/e315ae74-29d5-4bdd-84b2-4a9de2b4675a)
This error occurs upon wal file creation as it uses
`generate_random_number` for `salt_1` and `slat_2` in the header
https://github.com/tursodatabase/limbo/blob/main/core/storage/wal.rs#L76
4-L773)
This PR uses the same implementation as other platforms
`core/io/unix.rs`
https://github.com/tursodatabase/limbo/blob/main/core/io/unix.rs#L259-
L263
`core/io/windows.rs`
https://github.com/tursodatabase/limbo/blob/main/core/io/windows.rs#L40-
L44
The library `getrandom` [supports wasm](https://github.com/rust-
random/getrandom?tab=readme-ov-file#webassembly-support) and uses
[Crypto.getRandomValues](https://www.w3.org/TR/WebCryptoAPI/#Crypto-
method-getRandomValues) for the implementation in wasm.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1556
 2025-05-23 17:02:05 +03:00
Jussi Saurio
1a937462b3 Merge 'core/pragma: Add support for update user_version' from Diego Reis 
It also changes the type from u32 to i32 since
sqlite supports negative values

Closes #1559
 2025-05-23 17:00:55 +03:00
Jussi Saurio
c18c6a00fa refactor: use walk_expr() in resolving vtab constraints 2025-05-23 16:28:56 +03:00
Jussi Saurio
fbfd2b2c38 refactor: use walk_expr_mut() in rewrite_expr() 2025-05-23 16:27:28 +03:00
Jussi Saurio
362347c474 refactor: use walk_expr() in determine_where_to_eval_expr() 2025-05-23 16:27:28 +03:00
Jussi Saurio
9ec84e3905 refactor: use walk_expr() in table_mask_from_expr() 2025-05-23 16:27:28 +03:00
Jussi Saurio
3835a29f47 refactor: use walk_expr() in resolve_aggregates() 2025-05-23 16:27:23 +03:00
Jussi Saurio
2ab5c5f6a9 refactor: use walk_expr_mut() in bind_column_references() 2025-05-23 15:56:49 +03:00
Jussi Saurio
40a4d162bc Introduce walker expressions for ast::Expr 2025-05-23 15:56:27 +03:00
Diego Reis
2d6405b3e9 core/pragma: Remove unnecessary clone in user_version and cache_size 2025-05-23 08:43:07 -03:00
Jussi Saurio
517c795f15 Add another test 2025-05-23 14:33:55 +03:00
Jussi Saurio
cbb3efab82 Fuzz: modify rowid_seek_fuzz so that it catches this regression in the future 2025-05-23 14:28:25 +03:00
Jussi Saurio
cbb56a182e Fix bug: backwards iteration of table btree hangs 2025-05-23 14:23:18 +03:00
Jussi Saurio
597020bc0c Merge 'Support values statement and values in select' from meteorgan 
Close: #866
**limbo output**:
```
limbo> explain values(1, 2);
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     5     0                    0   Start at 5
1     Integer            1     1     0                    0   r[1]=1
2     Integer            2     2     0                    0   r[2]=2
3     ResultRow          1     2     0                    0   output=r[1..2]
4     Halt               0     0     0                    0
5     Goto               0     1     0                    0

limbo> explain values(1, 2), (3, 4);
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     16    0                    0   Start at 16
1     InitCoroutine      1     9     2                    0
2     Integer            1     2     0                    0   r[2]=1
3     Integer            2     3     0                    0   r[3]=2
4     Yield              1     0     0                    0
5     Integer            3     2     0                    0   r[2]=3
6     Integer            4     3     0                    0   r[3]=4
7     Yield              1     0     0                    0
8     EndCoroutine       1     0     0                    0
9     InitCoroutine      1     0     2                    0
10    Yield              1     15    0                    0
11    Copy               2     4     0                    0   r[4]=r[2]
12    Copy               3     5     0                    0   r[5]=r[3]
13    ResultRow          4     2     0                    0   output=r[4..5]
14    Goto               0     10    0                    0
15    Halt               0     0     0                    0
16    Goto               0     1     0                    0

limbo> explain select * from (values(1, 2), (3, 4));
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     16    0                    0   Start at 16
1     InitCoroutine      1     9     2                    0
2     Integer            1     2     0                    0   r[2]=1
3     Integer            2     3     0                    0   r[3]=2
4     Yield              1     0     0                    0
5     Integer            3     2     0                    0   r[2]=3
6     Integer            4     3     0                    0   r[3]=4
7     Yield              1     0     0                    0
8     EndCoroutine       1     0     0                    0
9     InitCoroutine      1     0     2                    0
10    Yield              1     15    0                    0
11    Copy               2     4     0                    0   r[4]=r[2]
12    Copy               3     5     0                    0   r[5]=r[3]
13    ResultRow          4     2     0                    0   output=r[4..5]
14    Goto               0     10    0                    0
15    Halt               0     0     0                    0
16    Transaction        0     0     0                    0   write=false
17    Goto               0     1     0                    0
```
**sqlite output**:
```
sqlite> explain values(1, 2);
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     5     0                    0   Start at 5
1     Integer        1     1     0                    0   r[1]=1
2     Integer        2     2     0                    0   r[2]=2
3     ResultRow      1     2     0                    0   output=r[1..2]
4     Halt           0     0     0                    0
5     Goto           0     1     0                    0
sqlite> explain values(1, 2), (3, 4);
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     16    0                    0   Start at 16
1     InitCoroutine  1     9     2                    0
2     Integer        1     4     0                    0   r[4]=1
3     Integer        2     5     0                    0   r[5]=2
4     Yield          1     0     0                    0
5     Integer        3     4     0                    0   r[4]=3
6     Integer        4     5     0                    0   r[5]=4
7     Yield          1     0     0                    0
8     EndCoroutine   1     0     0                    0
9     InitCoroutine  1     0     2                    0
10      Yield          1     15    0                    0   next row of 2-ROW VALUES CLAUSE
11      Copy           4     8     0                    2   r[8]=r[4]
12      Copy           5     9     0                    2   r[9]=r[5]
13      ResultRow      8     2     0                    0   output=r[8..9]
14    Goto           0     10    0                    0
15    Halt           0     0     0                    0
16    Goto           0     1     0                    0
sqlite>  explain select * from (values(1, 2), (3, 4));
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     16    0                    0   Start at 16
1     InitCoroutine  1     9     2                    0
2     Integer        1     4     0                    0   r[4]=1
3     Integer        2     5     0                    0   r[5]=2
4     Yield          1     0     0                    0
5     Integer        3     4     0                    0   r[4]=3
6     Integer        4     5     0                    0   r[5]=4
7     Yield          1     0     0                    0
8     EndCoroutine   1     0     0                    0
9     InitCoroutine  1     0     2                    0
10      Yield          1     15    0                    0   next row of 2-ROW VALUES CLAUSE
11      Copy           4     8     0                    2   r[8]=r[4]
12      Copy           5     9     0                    2   r[9]=r[5]
13      ResultRow      8     2     0                    0   output=r[8..9]
14    Goto           0     10    0                    0
15    Halt           0     0     0                    0
16    Goto           0     1     0                    0
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1549
 2025-05-23 13:56:31 +03:00
meteorgan
01c8a4ca63 simpify values when it's subquery 2025-05-23 17:45:56 +08:00
Jussi Saurio
128a406f8c TableReference: fix stale comment 2025-05-23 10:06:24 +03:00
Jussi Saurio
89a727022a Merge 'Drop Table OpCodes Use Ephemeral Table As Scratch Table' from Zaid Humayun 
In SQLite, when [AUTOVACUUM](https://www.sqlite.org/lang_vacuum.html) is
enabled (in SQLite, the `SQLITE_OMIT_AUTOVACUUM` flag is used), and a
table is dropped, if the root page of the table is destroyed and *it is
not the last page*, then the last root page is transferred here to avoid
empty gaps.
At this point, the old root page value is copied into a register and
SQLite will copy over all rows using the old root page from the schema
table into an ephemeral table. Later, it copies over the rows from the
ephemeral table back into the schema table with the new root page number
to "patch" the row.
This PR introduces the same semantics to Limbo. It doesn't perform the
AUTOVACUUM functionality of moving the last root page into the empty
page currently, so it no-ops that by returning 0. This triggers an
`IfNot` instruction to skip the inserts into the ephemeral table.
I plan to address the auto vacuuming in a future PR. But this brings us
a lot closer to matching SQLite semantics.
![Screenshot 2025-05-21 at 8 18 12 PM](https://github.com/user-
attachments/assets/049125ad-4448-4c14-b647-8ed990aaadaf)

Closes #1548
 2025-05-23 09:46:30 +03:00
Zaid Humayun
bc5d93c18a Addresses comment https://github.com/tursodatabase/limbo/pull/1548#discussion_r2103333264 by @jussisaurio 
this commit uses more descriptive names for registers
 2025-05-23 10:19:08 +05:30
Diego Reis
bfe4f5acef pragma/user_version: Add test and update docs 2025-05-22 20:42:08 -03:00
Diego Reis
2f8042da22 core/pragma: Add support for update user_version 
It also changes the type from u32 to i32 since
sqlite supports negative values
 2025-05-22 20:38:27 -03:00
Zaid Humayun
4312d371fb addresses comment https://github.com/tursodatabase/limbo/pull/1548#discussion_r2102606810 by @jussisaurio 
this commit changes the btree_destroy() signature to return an Option<usize>. This more closely resembles Rust semantics instead of passing a pointer to a usize.
However, I'm unsure if I'm handling the cursor result correctly
 2025-05-23 00:46:05 +05:30
Zaid Humayun
780d0cb5a8 addresses https://github.com/tursodatabase/limbo/pull/1548#discussion_r2102599487, https://github.com/tursodatabase/limbo/pull/1548#discussion_r2102601189 & https://github.com/tursodatabase/limbo/pull/1548#discussion_r2102602774 by @jussisaurio 
this commit addresses comments regarding using decsriptive variable names for the loops and loop labels. Also, adds documentation for instructions that cause jumps in both loops
 2025-05-23 00:31:19 +05:30