This PR implements support for `INSERT OR REPLACE INTO t`.
For `OR IGNORE`, we currently rewrite this internally to an `ON CONFLICT
DO NOTHING`, and I was hopeful we could do this with OR REPLACE, however
it seems SQLite actually deletes the row and then proceeds to insert, so
we could not simply rewrite this to an `ON CONFLICT DO UPDATE SET
col=excluded.col`, as this would result in differing rowid's when
compared to SQLite.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#3972
RE: #3970
That Column::new having 14 boolean arguments was not great.
Also this removes the unneeded `parent_cols: Vec<String>` from
`ResolvedFkRef`
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#3973
This PR follows https://github.com/tursodatabase/turso/pull/3625, and
enables self-inserts with nested subqueries with arbitrary levels of
nesting, of the form:
```sql
INSERT INTO x SELECT * FROM (SELECT * FROM x WHERE TRUE) WHERE TRUE;
```
This is limited, compared to enabling INSERTs with arbitrary SELECTs
like Jussi [initially suggested](https://github.com/tursodatabase/turso/
pull/3625#issuecomment-3397069821), but there are some preexisting
issues in the simulator that need to be solved before arbitrary SELECTs
can be enabled. Already, this PR includes a fix for a preexisting issue
(JOINs weren't computed correctly), but there are also some other
issues, for which I left FIXME's:
* the shadow model doesn't handle type coercion correctly, so INSERT
statements across affinities will fail the
`AllTablesHaveExpectedContent` property
* `SelectInner::arbitrary_sized` can generate SELECT statements with
fewer columns than requested
* `pick_unique` can hang under certain conditions
In addition, there is likely another preexisting issue with the shadow
model, because during development, when I tried to support arbitrary
SELECT queries, I got multiple simulator failures where the shadow model
ended up with an incorrect number of rows.
-----
### Future Work
In order to implement arbitrary SELECTs in `INSERT INTO ... SELECT`
statements, the issues above will need to be addressed. The biggest
issue is with type affinity, and I see 2 solutions:
1. apply type affinity rules in the simulator
2. build some tolerance into the `AllTablesHaveExpectedContent` property
The first solution seems like the best one, but it's far from trivial.
I've started working on it, but I don't know how much longer it will
take. For this reason, I'm opening this PR with just the limited query
generation.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#3933
I didn't end up having to use the RowSet instructions for this after
all. Maybe there's some edge cases where it's required -- sqlite uses
ephemeral tables or rowsets in all RETURNING handling, but I haven't
found a need to do that yet.
Note: contains 1000 lines of TCL tests generated by cursor :] runtime
changes are smaller and this actually deletes code in aggregate.
---
Main change is to support arbitrary expressions in RETURNING instead of
a specialcased subset, but also e.g. disallow returning TABLE.* which is
illegal syntax in SQLite.
Main idea is we add the columns of the target table (the table affected
by INSERT/UPDATE/DELETE) into `expr_to_reg_cache`) and then just
translate the RETURNING expressions as normal
Closes#3942
## Purpose
* Implement JDBC4 stream binding methods in JDBC4PreparedStatement for
the following overloads:
* `setAsciiStream(int, InputStream, long)`, `setAsciiStream(int,
InputStream)`
* `setBinaryStream(int, InputStream, long)`, `setBinaryStream(int,
InputStream)`
## Changes
### In `(int, InputStream, long)` methods
* Added a shared helper method `requireLengthIsPositiveInt(long length)`
to validate stream length.
* Validates `length` to fit SQLite’s 32-bit limit for compatibility
with SQLite/Turso engines.
* After validation passes, delegates to the `(int, InputStream, int)`
overload for actual binding logic.
### In `(int, InputeStream)` methods - no length
* Added a shared helper method `readBytes(InputStream x)`.
* Reads the first byte before allocating the buffer to avoid
unnecessary memory allocation for empty streams.
* After reading, directly binds the data using` bindText()` (for ASCII)
or `bindBlob()` (for binary).
* Avoids re-wrapping the stream or reallocation since the byte array
is already available.
## Related Issue
* #615
Reviewed-by: Kim Seon Woo (@seonWKim)
Closes#3937
Added README.md for Turso Database Python bindings with installation
instructions, features, and usage examples.
Suggestion: Errors when using python 3.14 but works for 3.13.9 so maybe
add which python versions are supported.
Closes#3955
This PR implements basic support for partial sync. Right now the scope
is limited to only `:memory:` IO and later will be properly expanded to
the file based IO later.
The main addition is `PartialDatabaseStorage` which make request to the
remote server for missing local pages on demand.
The main change is that now tursodatabase JS bindings accept optional
"external" IO event loop which in case of sync will drive `ProtocolIo`
internal work associated with remote page fetching tasks.
Closes#3931
Currently RETURNING was a bit of a hack since it had a special
translate_expr_for_returning() function that only supported a subset
of expressions.
Instead, we can store the columns of the target table of the INSERT/UPDATE/DELETE
we are RETURNING from in `Resolver::expr_to_reg_cache` and make those columns point
to the registers that hold the OLD/NEW column values (depending on the operation).
# Fix: Clean up DBSP state table when dropping materialized views
## Problem
When dropping a materialized view, the internal DBSP state table (e.g.,
`__turso_internal_dbsp_state_v1_view_name`) and its automatic primary
key index were not being properly cleaned up. This caused two issues:
1. **Persistent schema entries**: The DBSP table and index entries
remained in `sqlite_schema` after dropping the view
2. **In-memory schema inconsistency**: The DBSP table remained in the
in-memory schema's `tables` HashMap, causing "table already exists"
errors when trying to recreate a materialized view with the same name
## Root Cause
The issue had two parts:
1. **Missing sqlite_schema cleanup**: The `translate_drop_view` function
deleted the view entry from `sqlite_schema` but didn't delete the
associated DBSP state table and index entries
2. **Missing in-memory schema cleanup**: The `remove_view` function
removed the materialized view from the in-memory schema but didn't
remove the DBSP state table and its indexes
## Solution
### Changes in `core/translate/view.rs`
- Added a second pass loop in `translate_drop_view` to scan
`sqlite_schema` and delete DBSP table and index entries
- The loop checks for entries matching the DBSP table name pattern
(`__turso_internal_dbsp_state_v{version}_{view_name}`) and the automatic
index name pattern (`sqlite_autoindex___turso_internal_dbsp_state_v{vers
ion}_{view_name}_1`)
- Registers for comparison values are allocated outside the loop for
efficiency
- Column registers are reused across loop iterations
### Changes in `core/schema.rs`
- Updated `remove_view` to also remove the DBSP state table and its
indexes from the in-memory schema's `tables` HashMap and `indexes`
collection
- This ensures consistency between the persistent schema
(`sqlite_schema`) and the in-memory schema
### Tests Added
Added two new test cases in `testing/materialized_views.test`:
1. **`matview-drop-cleans-up-dbsp-table`**: Explicitly verifies that
after dropping a materialized view:
- The view entry is removed from `sqlite_schema`
- The DBSP state table entry is removed from `sqlite_schema`
- The DBSP state index entry is removed from `sqlite_schema`
2. **`matview-recreate-after-drop`**: Verifies that a materialized view
can be successfully recreated after being dropped, which implicitly
tests that all underlying resources (including DBSP tables) are properly
cleaned up
## Testing
- All existing materialized view tests pass
- New tests specifically verify the cleanup behavior
- Manual testing confirms that materialized views can be dropped and
recreated without errors
## Related
This fix ensures that materialized views can be safely dropped and
recreated, resolving issues where the DBSP state table would persist and
cause conflicts.
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#3928
