Commit Graph

7774 Commits

Author SHA1 Message Date
Pere Diaz Bou
ffaf8580e0 mvcc/core: simple interactive transaction tests for mvcc 2025-08-20 12:22:31 +02:00
Pere Diaz Bou
3927aa037c core/mvcc: start first rowid at 1 2025-08-20 11:22:51 +02:00
Pere Diaz Bou
94cd504d4c core/mvcc: support schema_did change on commit_txn
This not only changes schema_did_change on commit_txn for mvcc, but also
extracts the connection transaction state from non mvcc transactions to
mvcc too.
2025-08-18 15:52:10 +02:00
Pere Diaz Bou
f8c110e664 core/mvcc: find latest valid version for row for every operation
Previously we only checked first version of row...
2025-08-18 13:54:10 +02:00
Pekka Enberg
33ddae1877 Fix JavaScript API reference 2025-08-18 14:00:46 +03:00
Jussi Saurio
3ee2be6750 Merge 'Refactor: use regular save/restore context mechanism for delete balancing' from Jussi Saurio
This is the first in a series of PRs whose end goal is to close the
#2004 issue, but I don't want to make that PR too bloated.
- Removes special `DeleteSavepoint` and uses the existing cursor
restoration mechanism.
- This required some restructuring of `DeleteState` to avoid cloning it,
i.e. some negotations with the borrow checker.
- CursorContext now takes a SeekOp as well to allow retaining the
behavior that we use LT for seeking after a delete-induced rebalancing.
This behavior will probably be removed as part of fixing #2004, but here
I am trying to preserve the current semantics.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2638
2025-08-18 13:38:48 +03:00
Jussi Saurio
9abc63d853 Add a bit of abstraction for creating EQ cursorcontexts 2025-08-18 13:13:02 +03:00
Pekka Enberg
3a72b478d2 Turso 0.1.4-pre.9 2025-08-18 12:55:45 +03:00
Jussi Saurio
3eb89982ba Remove obsolete FIXME 2025-08-18 12:08:40 +03:00
Jussi Saurio
50fd7ec58b Refactor: use regular save/restore context mechanism for delete balancing
- Removes special `DeleteSavepoint` and uses the existing cursor restoration
  mechanism.
- This required some restructuring of `DeleteState` to avoid cloning it, i.e.
  some negotations with the borrow checker.
- CursorContext now takes a SeekOp as well to allow retaining the behavior
  that we use LT for seeking after a delete-induced rebalancing. This behavior
  will probably be removed as part of fixing #2004, but here I am trying to
  preserve the current semantics.
2025-08-18 11:58:00 +03:00
Pekka Enberg
568737ec88 Merge 'Add list databases and open database commands to the MCP server' from Glauber Costa
When playing with our MCP server with Claude Code, I realized that
having the agent pre-configured to a specific path is just not very
useful.
I am now adding two new commands, to open a database and to list the
current database. Those make our server more useful.

Closes #2628
2025-08-18 08:30:34 +03:00
Jussi Saurio
b69709d9bc Merge 'fix/sim: prevent sim from trying to create an existing table or index' from Jussi Saurio
We recently merged a change that panics the sim on parse errors, because
not doing so has masked many scenarios where the sim unintentionally
creates incorrect sql and we just ignore it.
The aforementioned change also surfaced that the sim can try to create a
table that already exists, which now makes us panic and fail the sim. So
in this PR, we make sure that doesn't happen.
We already have Property::DoubleCreateFailure to assert that the same
table cannot be created twice, so this should not hide any bugs.

Closes #2634
2025-08-17 18:36:43 +03:00
Jussi Saurio
6c17fa2a5e fix/sim: prevent sim from trying to create an existing table or index
We recently merged a change that panics the sim on parse errors, because
not doing so has masked many scenarios where the sim unintentionally
creates incorrect sql and we just ignore it.

We already have Property::DoubleCreateFailure to assert that the same table
cannot be created twice, so this should not hide any bugs.
2025-08-17 18:13:05 +03:00
Jussi Saurio
6e8822792a Merge 'Improve handling of inserts with column names' from Wallys Ferreira
Closes #2495
Return error on inserts with qualified expressions to emulate same
functionality as SQLite.

Closes #2592
2025-08-17 14:41:48 +03:00
Jussi Saurio
5fc967c41d Merge 'Fail simulator on parse errors' from Jussi Saurio
Closes #2618 (sufficiently for now, IMO)
Prevents simulator from creating malformed queries which we don't notice
(ref e.g. #2609 , #2611, #2616)

Closes #2624
2025-08-17 14:40:38 +03:00
Jussi Saurio
c21d9871db Merge 'fix pragma table_info for views' from Glauber Costa
We were not generating table_info for views. This PR fixes it.  We were
so far storing columns as strings with just their names - since this is
all we needed - but we will move now to store Columns. We need to
convert the names to Column anyway for table_info to work.

Closes #2625
2025-08-17 14:40:21 +03:00
Jussi Saurio
cf9851ea9b Merge 'emit SetCookie when creating a view' from Glauber Costa
SetCookie is necessary to invalidate prepared statements in the
connection after DDL expressions.

Closes #2632
2025-08-17 14:39:58 +03:00
Jussi Saurio
c4f530d8f5 Merge 'unify halts' from Glauber Costa
We have halt and op_halt, doing essentially the same thing.
This PR unifies them. There is a minor difference between them now in
the way halt() handles auto-commit. My current understanding of the code
is that what we have in halt *is a bug*, which is already one bad
consequence of the duplication.

Closes #2631
2025-08-17 14:39:30 +03:00
Jussi Saurio
e89226a081 Merge 'Update stale in memory wal header after restarting log' from Preston Thorpe
Currently during `Restart|Truncate` modes we properly restart the WAL
header in the `WalFileShared`, but keep a stale one on the current
`WalFile` instance.

Closes #2627
2025-08-17 14:37:48 +03:00
Glauber Costa
9f5d3dbf87 setcookie 2025-08-16 21:37:31 -05:00
Glauber Costa
8c32f96748 emit SetCookie when creating a view
SetCookie is necessary to invalidate prepared statements in the
connection after DDL expressions.
2025-08-16 21:06:49 -05:00
Glauber Costa
270245b4d3 unify halts
We have halt and op_halt, doing essentially the same thing.

This PR unifies them. There is a minor difference between them now in
the way halt() handles auto-commit. My current understanding of the code
is that what we have in halt *is a bug*, which is already one bad
consequence of the duplication.
2025-08-16 16:52:53 -05:00
Preston Thorpe
7f8e181cda Merge 'Add documentation and rename functions' from Mikaël Francoeur
This PR adds documentation and renames some functions. Among other
things, I renamed everything that was still called `owned_value` to
either `db_value` or `value` (after
https://github.com/tursodatabase/turso/pull/1488).

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2619
2025-08-16 17:45:33 -04:00
PThorpe92
605b3e233f Move in memory reset logic to before truncation in wal restart 2025-08-16 17:42:22 -04:00
PThorpe92
eae68f3669 Update stale in memory wal header after restarting log 2025-08-16 17:36:48 -04:00
Preston Thorpe
53c68bccab Merge 'Add io_yield macros to reduce boilerplate' from Preston Thorpe
```rust
io_yield_one!(c);

// instead of

return Ok(IOResult::IO(IOCompletions::Single(c))));
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2630
2025-08-16 17:12:45 -04:00
PThorpe92
2c526c4c37 Add io_yield_x macros to reduce boilerplate 2025-08-16 16:14:00 -04:00
Glauber Costa
841296a39d add new commands to the MCP server
When playing with our MCP server with Claude Code, I realized that
having the agent pre-configured to a specific path is just not very
useful.

I am now adding two new commands, to open a database and to list
the current database. Those make our server more useful.
2025-08-16 11:00:28 -05:00
Glauber Costa
03eeabef18 fix pragma table_info for views
We were not generating table_info for views. This PR fixes it.  We were
so far storing columns as strings with just their names - since this is
all we needed - but we will move now to store Columns. We need to
convert the names to Column anyway for table_info to work.
2025-08-16 08:03:57 -05:00
Jussi Saurio
0a3ebf8914 Fail simulator on parse errors 2025-08-16 10:27:10 +03:00
Pekka Enberg
5a167517e8 Merge 'Add --keep-files flag to allow for inspection of files for successful simulator runs ' from Preston Thorpe
Also fixes an issue where since the old Database wasn't `Drop`d, it
caused an issue with the static Registry ™️ that had broken
`--doublecheck` mode.

Closes #2623
2025-08-16 09:53:41 +03:00
Pekka Enberg
39fe7a0a32 Merge 'core/printf: support for the %i operand' from Luiz Gustavo
This continues the work for `printf` support from from tthe issue
https://github.com/tursodatabase/turso/issues/885.
This PR adds support for `%i` operands in the printf function. An
example of a command that works now and didn't before:
```sql
SELECT PRINTF("Decimals: %d %i", 200, 300);
```
This is Just an initial very simple contribution to get my feet wet and
learn the basics about the project and how contributions here work. I
intend to do something more meaningful in next contributions. Probably
will try to finish support for missing `printf` functionality also.

Closes #2615
2025-08-16 09:53:10 +03:00
Pekka Enberg
63c22c79cf Merge 'sync-engine: Use SQL over HTTP instead of WAL push' from Nikita Sivukhin
This PR switches from usage of WAL push to SQL over http calls.
As we have sequence of logical changes from CDC - we can just send SQL
statements to the remote. This will give us benefits of more smooth
concurrent usage of sync - because WAL push is very unfriendly in case
of concurrent usages.

Closes #2617
2025-08-16 09:51:15 +03:00
Preston Thorpe
cfad078087 Merge 'Add parser to Dockerfiles for cargo chef' from Jussi Saurio
antithesis and limbo-sim push fail because this is missing

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2622
2025-08-15 18:02:25 -04:00
Preston Thorpe
97526ee4d1 Merge 'Add framework for testing extensions in TCL' from Piotr Rżysko
There is a distinction between tests that verify extension-specific
behavior and those that verify interactions between the database engine
and extensions. Previously, both types of tests were kept in
`extensions.py`. With this new framework, we can extract the latter type
of tests from `extensions.py` into TCL. This cleans up `extensions.py`
and enables compatibility testing with SQLite at no extra cost.
I’m currently working on supporting outer joins involving TVFs and
planning to add more tests that exercise the database’s handling of
virtual tables, so I decided to do this refactoring first.
In the future, we may consider moving extension-specific tests to TCL as
well, especially those that have counterparts in SQLite or sqlean.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2556
2025-08-15 17:58:21 -04:00
PThorpe92
2810b5d8a2 Add --keep-files flag and force Drop of database in simulator to prevent issues with the Registry 2025-08-15 17:54:40 -04:00
Mikaël Francoeur
2ee0132afe rename functions 2025-08-15 17:08:53 -04:00
Jussi Saurio
e4fb02b6e6 Add parser to Dockerfiles for cargo chef 2025-08-15 23:53:02 +03:00
Jussi Saurio
7a025c7c51 Merge 'Fix two issues in simulator' from Jussi Saurio
## 1. select equal number of columns per compound subselect in simulator
#2609 fixed compound selects incorrectly, introducing another bug: now
the sim can SELECT a different number of columns per subselect, which is
illegal. this PR forces the sim to select the same number of cols per
subselect.
Depends on #2614 , otherwise the sim constantly fails whenever it
decides to do a compound select with DISTINCT.
Closes #2611
## 2. introduce TableContext for the simulator to properly generate
predicates for Joins
The simulator has a construct called `JoinTable` which is really a
`JoinContext` that includes all the columns from the so-far joined
tables, so that the next table to be joined can refer to columns from
any of those tables. @pedrocarlo 's commit fixes an issue where
`JoinTable` would incorrectly generate predicates with empty table
names. Related: #2618

Closes #2616
2025-08-15 23:49:29 +03:00
pedrocarlo
2bc6edc3d4 introduce TableContext for the simulator to properly generate predicates for Joins 2025-08-15 23:44:00 +03:00
Jussi Saurio
3760d44c13 sim: force compound selects to have the same number of result columns 2025-08-15 23:43:38 +03:00
Jussi Saurio
c03517da40 Merge 'Fix distinct order by' from Jussi Saurio
Closes #2612
Two commits:
## Simplify ORDER BY sorter column remapping
In case an ORDER BY column exactly matches a result column in the
SELECT,
the insertion of the result column into the ORDER BY sorter can be
skipped
because it's already necessarily inserted as a sorting column.
For this reason we have a mapping to know what index a given result
column
has in the order by sorter.
This commit makes that mapping much simpler.
## Fix DISTINCT with ORDER BY
We had a bug where we were checking for duplicates in the DISTINCT
index based on both the result column count plus any ORDER BY columns
not present in the DISTINCT clause.
This is wrong, so fix it by only using the result columns for the
dedupe check.

Closes #2614
2025-08-15 23:43:29 +03:00
Piotr Rzysko
d1a91a63e6 Extract TVF-related tests to TCL
These tests verify interactions between the database engine and TVFs.
They happen to use generate_series, but they are not intended to test
the behavior of any specific extension. Tests that verify generate_series
specific behavior remain in extensions.py.
2025-08-15 21:06:30 +02:00
Piotr Rzysko
20ea079679 Add framework for testing extensions in TCL
There is a distinction between tests that verify extension-specific
behavior and tests that verify interactions between the database engine
and extensions. Previously, both types of tests were kept in extensions.py.
With this new framework, we can extract the latter type of tests from
extensions.py into TCL. This cleans up extensions.py and provides
compatibility testing with SQLite at no extra cost.

To demonstrate the framework’s usage, tests verifying the handling of
virtual tables were extracted to TCL.

In the future, we may consider moving extension-specific tests to TCL as
well, especially those that have counterparts in SQLite or sqlean.
2025-08-15 21:06:27 +02:00
Piotr Rzysko
116673c2e5 Unify how SQL is executed in TCL tests 2025-08-15 21:06:17 +02:00
Mikaël Francoeur
4a986080d2 add documentation 2025-08-15 14:59:37 -04:00
Nikita Sivukhin
b9a23f3067 fix clippy 2025-08-15 17:01:40 +04:00
Nikita Sivukhin
fef4e7e0e6 move tests from tursodb repo to turso-server repo 2025-08-15 16:59:53 +04:00
Nikita Sivukhin
979e7da633 use sql-over-http instead of WAL push 2025-08-15 16:59:43 +04:00
Jussi Saurio
d2cfe06aa5 Fix DISTINCT with ORDER BY
We had a bug where we were checking for duplicates in the DISTINCT
index based on both the result column count plus any ORDER BY columns
not present in the DISTINCT clause.

This is wrong, so fix it by only using the result columns for the
dedupe check.
2025-08-15 15:49:55 +03:00