Commit Graph

10900 Commits

Author SHA1 Message Date
Pere Diaz Bou
430bd457e6 core/mvcc: fix tests with delete 2025-11-19 17:18:44 +01:00
Pere Diaz Bou
bf1afb56cf core/mvcc: test with delete after checkpoint 2025-11-19 16:56:32 +01:00
Pere Diaz Bou
b4c11705f3 core/mvcc: few suggestions from pr 2025-11-19 16:44:24 +01:00
Pere Diaz Bou
ca30756dfd core/mvcc/cursor: implement prev and last 2025-11-18 19:51:27 +01:00
Pere Diaz Bou
b38e69b515 core/mvcc: add get_row_id_for_table_in_direction(forward/backwards) 2025-11-18 19:51:27 +01:00
Pere Diaz Bou
b19762a812 core/mvcc/cursor: get_new_position_from_mvcc_and_btree backwards and last fix 2025-11-18 19:51:27 +01:00
Pere Diaz Bou
73d9f0016c core/mvcc: test order by desc with mvcc 2025-11-18 19:51:27 +01:00
Pere Diaz Bou
72bf195f4b Merge 'core/mvcc/cursor: rowid don't seek first rowid' from Pere Diaz Bou
rowid should only try to use the current's position. So if we are not
pointing to a `Loaded` row, then it should return None
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Change `rowid()` to return `None` unless cursor is on a `Loaded` row,
removing the implicit seek from `BeforeFirst`.
>
> - **Core MVCC Cursor (`core/mvcc/cursor.rs`)**:
>   - Adjust `rowid()` behavior: remove implicit first-row seek when
`BeforeFirst`; return `None` unless position is `Loaded`.
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
8848775a71. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3977
2025-11-18 19:51:19 +01:00
Preston Thorpe
e61234d522 Merge 'translate/insert: Implement INSERT OR REPLACE' from Preston Thorpe
This PR implements support for `INSERT OR REPLACE INTO t`.
For `OR IGNORE`, we currently rewrite this internally to an `ON CONFLICT
DO NOTHING`, and I was hopeful we could do this with OR REPLACE, however
it seems SQLite actually deletes the row and then proceeds to insert, so
we could not simply rewrite this to an `ON CONFLICT DO UPDATE SET
col=excluded.col`, as this would result in differing rowid's when
compared to SQLite.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3972
2025-11-17 18:50:15 -05:00
PThorpe92
56f35ad4cd cargo fmt 2025-11-17 12:22:55 -05:00
PThorpe92
c3185d0b8c Properly handle foreign keys for INSERT OR REPLACE 2025-11-17 12:19:33 -05:00
Pere Diaz Bou
8848775a71 core/mvcc/cursor: rowid don't seek first rowid
rowid should only try to use the current's position. So if we are not
pointing to a `Loaded` row, then it should return None
2025-11-17 16:17:52 +01:00
PThorpe92
8cd33f3ec9 Add comment for or replace behavior require seek in translate/insert 2025-11-17 08:41:22 -05:00
PThorpe92
0ce5f81008 Cleanup translate/insert fix clippy warnings 2025-11-17 08:23:16 -05:00
PThorpe92
f8e78b73a8 Fix handling of partial indexes when deleting rows in ON REPLACE for insert 2025-11-17 08:23:16 -05:00
PThorpe92
0bc56d3f28 Sprinkle some OR REPLACE into INSERT statements in fuzzing 2025-11-17 08:23:16 -05:00
PThorpe92
b83921d838 Add TCL tests for insert or replace handling 2025-11-17 08:23:16 -05:00
PThorpe92
634af4d6f6 Handle NOT NULL behavior for INSERT OR REPLACE 2025-11-17 08:23:15 -05:00
PThorpe92
5bff10c56e Implement INSERT OR REPLACE translation/emission 2025-11-17 08:23:10 -05:00
Jussi Saurio
693eaeb851 Merge 'Add ColDef struct to make schema::Column creation more ergonomic' from Preston Thorpe
RE: #3970
That Column::new having 14 boolean arguments was not great.
Also this removes the unneeded `parent_cols: Vec<String>` from
`ResolvedFkRef`

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3973
2025-11-17 09:17:56 +02:00
Jussi Saurio
3b0d4f3214 Merge 'Enable nested self-inserts in simulator' from Mikaël Francoeur
This PR follows https://github.com/tursodatabase/turso/pull/3625, and
enables self-inserts with nested subqueries with arbitrary levels of
nesting, of the form:
```sql
INSERT INTO x SELECT * FROM (SELECT * FROM x WHERE TRUE) WHERE TRUE;
```
This is limited, compared to enabling INSERTs with arbitrary SELECTs
like Jussi [initially suggested](https://github.com/tursodatabase/turso/
pull/3625#issuecomment-3397069821), but there are some preexisting
issues in the simulator that need to be solved before arbitrary SELECTs
can be enabled. Already, this PR includes a fix for a preexisting issue
(JOINs weren't computed correctly), but there are also some other
issues, for which I left FIXME's:
* the shadow model doesn't handle type coercion correctly, so INSERT
statements across affinities will fail the
`AllTablesHaveExpectedContent` property
* `SelectInner::arbitrary_sized` can generate SELECT statements with
fewer columns than requested
* `pick_unique` can hang under certain conditions
In addition, there is likely another preexisting issue with the shadow
model, because during development, when I tried to support arbitrary
SELECT queries, I got multiple simulator failures where the shadow model
ended up with an incorrect number of rows.
-----
### Future Work
In order to implement arbitrary SELECTs in `INSERT INTO ... SELECT`
statements, the issues above will need to be addressed. The biggest
issue is with type affinity, and I see 2 solutions:
1. apply type affinity rules in the simulator
2. build some tolerance into the `AllTablesHaveExpectedContent` property
The first solution seems like the best one, but it's far from trivial.
I've started working on it, but I don't know how much longer it will
take. For this reason, I'm opening this PR with just the limited query
generation.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3933
2025-11-17 08:34:18 +02:00
Preston Thorpe
29f30c5c35 Merge 'allow null for unique columns and don't validate fkeys on parent' from Pavan Nambi
closes #3966
closes #3965
check constraint pr depends on this.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3969
2025-11-16 19:59:59 -05:00
PThorpe92
ced5aec2c7 Add ColDef arguments to Column::new in collate.rs 2025-11-16 19:36:11 -05:00
PThorpe92
ca783ccdff Add ColDef struct to make schema::Column creation more ergonomic 2025-11-16 19:10:42 -05:00
Preston Thorpe
22810f1186 Merge 'correct order in column creation in join tests' from Pavan Nambi
see
https://github.com/tursodatabase/turso/pull/3905#discussion_r2531910624
order was misplaced during refactor
ideally it would be nice if we change this to something like
```
name:xxx
rowid:xxx
pkey:xx
```
instead of relying on inlay hints. but or now just changing order back.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3970
2025-11-16 19:09:59 -05:00
Pavan-Nambi
134f898952 correct order in column creation tests 2025-11-16 18:36:10 +05:30
Pavan-Nambi
3de37d3f64 dont validate fkey on parent add tests
correct msitake and add null test issue

add fkey test
2025-11-16 09:41:57 +05:30
Pavan-Nambi
8edea305f6 allow null for unique columns 2025-11-16 08:37:57 +05:30
Mikaël Francoeur
d309e6ddb3 generate self-inserts with nested subqueries 2025-11-14 09:16:17 -05:00
Mikaël Francoeur
23d6080531 make FromClause recursive 2025-11-14 09:16:17 -05:00
Mikaël Francoeur
156693ce95 enable self-inserts 2025-11-14 09:16:17 -05:00
Mikaël Francoeur
8ea038c00b fix join 2025-11-14 09:16:17 -05:00
Jussi Saurio
ec149865f1 Merge 'Support DELETE ... RETURNING' from Jussi Saurio
Closes #3948
2025-11-14 13:53:57 +02:00
Jussi Saurio
3f970d5d95 Merge 'Return parse error if NULLS LAST used in ORDER BY' from Jussi Saurio
we don't support this yet, so let's not silently ignore it either

Closes #3945
2025-11-14 13:36:34 +02:00
Jussi Saurio
46440ad58d Merge 'don't overwrite col mappings' from Pavan Nambi
closes https://github.com/tursodatabase/turso/issues/3951

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3956
2025-11-14 13:36:07 +02:00
Jussi Saurio
b9b9a5ecab AI-generated tests for DELETE RETURNING 2025-11-14 13:35:24 +02:00
Jussi Saurio
272dadc4bc Support DELETE ... RETURNING
I didn't end up having to use the RowSet instructions for this after
all. Maybe there's some edge cases where it's required -- sqlite uses
ephemeral tables or rowsets in all RETURNING handling, but I haven't
found a need to do that yet.
2025-11-14 13:35:24 +02:00
Jussi Saurio
fca32737e7 Merge 'Refactor RETURNING to support arbitrary expressions' from Jussi Saurio
Note: contains 1000 lines of TCL tests generated by cursor :] runtime
changes are smaller and this actually deletes code in aggregate.
---
Main change is to support arbitrary expressions in RETURNING instead of
a specialcased subset, but also e.g. disallow returning TABLE.* which is
illegal syntax in SQLite.
Main idea is we add the columns of the target table (the table affected
by INSERT/UPDATE/DELETE) into `expr_to_reg_cache`) and then just
translate the RETURNING expressions as normal

Closes #3942
2025-11-14 13:34:53 +02:00
Pekka Enberg
a1e1e41ec2 Merge 'bindings/java: implement JDBC4 InputStream binding methods (ASCII/Binary, no-length and long overloads)' from Orange banana
## Purpose
* Implement JDBC4 stream binding methods in JDBC4PreparedStatement for
the following overloads:
  * `setAsciiStream(int, InputStream, long)`, `setAsciiStream(int,
InputStream)`
  * `setBinaryStream(int, InputStream, long)`, `setBinaryStream(int,
InputStream)`
## Changes
### In `(int, InputStream, long)` methods
* Added a shared helper method `requireLengthIsPositiveInt(long length)`
to validate stream length.
  * Validates `length` to fit SQLite’s 32-bit limit for compatibility
with SQLite/Turso engines.
* After validation passes, delegates to the `(int, InputStream, int)`
overload for actual binding logic.
### In `(int, InputeStream)` methods - no length
* Added a shared helper method `readBytes(InputStream x)`.
  * Reads the first byte before allocating the buffer to avoid
unnecessary memory allocation for empty streams.
* After reading, directly binds the data using` bindText()` (for ASCII)
or `bindBlob()` (for binary).
  * Avoids re-wrapping the stream or reallocation since the byte array
is already available.
## Related Issue
* #615

Reviewed-by: Kim Seon Woo (@seonWKim)

Closes #3937
2025-11-14 12:19:51 +02:00
Pekka Enberg
dae9e7990e Merge 'Create README.md for Turso Python bindings' from
Added README.md for Turso Database Python bindings with installation
instructions, features, and usage examples.
Suggestion: Errors when using python 3.14 but works for 3.13.9 so maybe
add which python versions are supported.

Closes #3955
2025-11-14 12:18:51 +02:00
Pavan-Nambi
8d2b06e6bf remove stupid files,clippy and tcl-syntax 2025-11-14 08:24:01 +05:30
Pavan-Nambi
eaa8edb6f7 don't overwrite col mappings 2025-11-14 07:41:41 +05:30
gigagrug
b7bdbefe8b Create README.md for Turso Database Python bindings
Added README.md for Turso Database Python bindings with installation instructions, features, and usage examples.
2025-11-13 20:19:50 -05:00
Pekka Enberg
82d7a6ff27 Merge 'Nyrkiö: Set all comment-on to false' from Henrik Ingo
Closes #3943
2025-11-13 16:38:48 +02:00
Pekka Enberg
1f79fbc22c Merge 'Partial sync basic' from Nikita Sivukhin
This PR implements basic support for partial sync. Right now the scope
is limited to only `:memory:` IO and later will be properly expanded to
the file based IO later.
The main addition is `PartialDatabaseStorage` which make request to the
remote server for missing local pages on demand.
The main change is that now tursodatabase JS bindings accept optional
"external" IO event loop which in case of sync will drive `ProtocolIo`
internal work associated with remote page fetching tasks.

Closes #3931
2025-11-13 16:38:04 +02:00
Jussi Saurio
950184a2c6 Return parse error if NULLS LAST used in ORDER BY
we don't support this yet, so let's not silently ignore it either
2025-11-13 10:47:17 +02:00
Jussi Saurio
c92ec33e83 Add a truckload of AI-generated TCL tests for RETURNING 2025-11-13 10:32:38 +02:00
Jussi Saurio
eee7fa5f95 Refactor RETURNING to support arbitrary expressions
Currently RETURNING was a bit of a hack since it had a special
translate_expr_for_returning() function that only supported a subset
of expressions.

Instead, we can store the columns of the target table of the INSERT/UPDATE/DELETE
we are RETURNING from in `Resolver::expr_to_reg_cache` and make those columns point
to the registers that hold the OLD/NEW column values (depending on the operation).
2025-11-13 10:32:38 +02:00
Jussi Saurio
50fbd9a3a2 Store owned strings in InsertEmitCtx for borrow-checker reasons 2025-11-13 09:35:09 +02:00
Jussi Saurio
34978d0fde Store Cow<&Expr> in expr_to_reg_cache
We will be storing owned expressions in it for RETURNING in a later commit.
2025-11-13 09:32:37 +02:00