Commit Graph

7711 Commits

Author SHA1 Message Date
Pekka Enberg
bbc88e8a16 Merge 'Convert SQLite parser in Rust by hand ' from Lâm Hoàng Phúc
working on #2337

Closes #2381
2025-08-15 14:51:14 +03:00
TcMits
e90e154f51 fmt 2025-08-15 17:09:30 +07:00
TcMits
145300877b merge main 2025-08-15 17:07:21 +07:00
TcMits
4d91f19ab2 rebase 2025-08-15 17:05:28 +07:00
TcMits
fb5203ce45 make eat_assert faster 2025-08-15 16:45:20 +07:00
TcMits
9cec83f20b remove unused deps 2025-08-15 16:45:20 +07:00
TcMits
949bc97ced clippy again 2025-08-15 16:45:18 +07:00
TcMits
22f53d1fe6 clippy again 2025-08-15 16:45:18 +07:00
TcMits
1cafdc1f8e fmt 2025-08-15 16:45:18 +07:00
TcMits
f0bd4cca69 clippy 2025-08-15 16:45:18 +07:00
TcMits
3f19d2aa1a deps 2025-08-15 16:45:18 +07:00
TcMits
bbd96d263c finish REINDEX 2025-08-15 16:45:08 +07:00
Jussi Saurio
a1401e43e6 Merge 'sim: add Property::TableHasExpectedContent' from Jussi Saurio
We are relying too much on the currently disabled `FaultyQuery` and
`FsyncNoWait` to assert the basic invariant of "the rows we have
inserted/updated/deleted in the database are still there", so adding
that as a separate `Property` here.

Closes #2610
2025-08-15 11:20:36 +03:00
Jussi Saurio
96072509f5 sim: add Property::TableHasExpectedContent 2025-08-15 11:16:31 +03:00
Jussi Saurio
4c76191fec fix/sim: fix incorrect implementation of compound select generation
Problem:

sim was generating compound selects like this:

- pick a random `table`
- create a random single SELECT
- create `n` random compound SELECTs (UNION ALL etc)

assign a WHERE clause that always has a condition based on `table`

This can result in e.g.

```
SELECT a FROM foo WHERE bar.x = 'baz'
```

Solution:

Don't base the WHERE clause on a table that might not be referenced
in the query.

Again, the only reason this wasn't caught before was because `FaultyQuery`
and `FsyncNoWait` are the only paths where this is actually tested with an
assertion, and those are both disabled
2025-08-15 10:48:42 +03:00
Pekka Enberg
e4f424f9e6 Merge 'Properly implement CLI command' from Preston Thorpe
Closes #2588
SQLite internally implements `.clone` by doing something like piping
`.dump` into a new connection to a database attached to the file you
want. This PR implements that by adding an `ApplyWriter` that implements
`std::fmt::Write`, and refactors our current `.dump` plumbing to work
with any `Write` interface, so we can `.dump` to stdout or `.dump` to a
new connection, therefore cloning the database.

Reviewed-by: Nikita Sivukhin (@sivukhin)

Closes #2590
2025-08-15 10:44:57 +03:00
Pekka Enberg
a39105f9ce Merge 'Ensure we fsync the db file in all paths that we checkpoint' from Preston Thorpe
closes #2607

Closes #2608
2025-08-15 10:43:47 +03:00
Pekka Enberg
6105dd0ae6 Merge 'docs: add Claude Code MCP integration guide' from Braden Wong
Add comprehensive documentation for connecting Turso databases to Claude
Code using the built-in MCP management commands.
This PR addresses the need for easier integration between Turso and
Claude Code by providing clear, step-by-step instructions for users who
want to query their databases using natural language.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2606
2025-08-15 08:50:31 +03:00
PThorpe92
9ccf79111a Support non-utf8 blobs in .clone command 2025-08-14 21:31:14 -04:00
PThorpe92
2b289157d0 Properly quote sequence value 2025-08-14 21:31:14 -04:00
PThorpe92
7abf071128 Conditionally emit pragma foreign_keys=off when not .cloning 2025-08-14 21:31:14 -04:00
PThorpe92
6d7b660dd4 Adjust test for .clone method 2025-08-14 21:31:14 -04:00
PThorpe92
4a612c1586 Properly implement .clone method safely 2025-08-14 21:31:13 -04:00
PThorpe92
7c021c01f8 Remove copy_db method from connection 2025-08-14 21:31:13 -04:00
PThorpe92
cc2fed3297 Remove copy_to API from file IO trait 2025-08-14 21:31:13 -04:00
PThorpe92
285dcdd2c1 Prevent potential corruption from copying db file without holding proper locks 2025-08-14 21:31:13 -04:00
PThorpe92
3c088dda59 Update callsites of copy_to Database impl 2025-08-14 21:31:13 -04:00
PThorpe92
55f09a01c4 Update copy_to method in file trait to separate source and destination IO 2025-08-14 21:31:13 -04:00
PThorpe92
2ad479f413 dont forget other instance where db file is expected size 2025-08-14 19:56:23 -04:00
PThorpe92
9017acd622 Ensure we fsync the db file in all paths that we checkpoint 2025-08-14 19:23:00 -04:00
Braden Wong
95d9e99b3d docs: add Claude Code MCP integration guide
Add comprehensive documentation for connecting Turso databases to Claude Code using the built-in MCP management commands.

The new guide includes:
- Quick 3-step setup process
- Visual command breakdown with annotations
- Multiple usage examples (local, absolute paths, project-scoped)
- MCP server management commands
- Example natural language queries users can make

This makes it much easier for developers to integrate their Turso databases with Claude Code without manually configuring JSON files or understanding the underlying MCP protocol details.
2025-08-14 13:20:02 -07:00
Preston Thorpe
5cea0f572e Merge 'Revive async io extension PR' from Preston Thorpe
bringing #1127 back to life, except better because this doesn't add
Tokio as a dependency for extension lib just for tests.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2418
2025-08-14 16:10:09 -04:00
Preston Thorpe
64c83237b9 apply copilot review doc comment
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-08-14 16:09:25 -04:00
Preston Thorpe
e9d23e7fea apply copilot suggestion
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-08-14 16:08:55 -04:00
Preston Thorpe
c9888375fd Merge 'sim: reduce frequency of compound selects and complex joins' from Jussi Saurio
we spend a shitload of time doing very complex selects in the simulator,
and i'm not sure there has been a lot of gain from them. using up a lot
of the runtime of a simulator run on these can mask other issues.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2605
2025-08-14 16:03:55 -04:00
Preston Thorpe
a425c72ddc Merge 'Add assertion for expected write amount in writev callback' from Preston Thorpe
as @avinassh pointed out, if we have an error in a `writev` call, it
could go unnoticed and the checkpoint could proceed as expected after
not writing the expected total amt.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2601
2025-08-14 14:05:08 -04:00
Jussi Saurio
165f2aed23 make big joins even less likely 2025-08-14 19:35:16 +03:00
Jussi Saurio
f5438d14de make big compound selects even less likely 2025-08-14 19:35:16 +03:00
Jussi Saurio
aed48bb665 sim: reduce frequency of compound selects and complex joins 2025-08-14 19:35:16 +03:00
Jussi Saurio
e85af59eaf Merge 'sim: add Property::ReadYourUpdatesBack' from Jussi Saurio
## Problem
We have been running the simulator for over a month without ever doing a
check like "did the rows we updated actually get updated" (#2602 ),
because the only properties that check those are - for whatever reason -
`FsyncNoWait` and `FaultyQuery`, and both of those have been disabled
for a long time.
Before we enable `FaultyQuery` again, let's make sure we don't have any
active bugs on `main` that don't depend on IO faults to happen. One of
these was an index-related corruption bug #2599 - already fixed - which
would've been caught by this test (or any similar assertion that just
checks what we updated in the DB, but we didn't have any)
## Solution
Add `Property::ReadYourUpdatesBack`, which essentially just does the
following:
```
UPDATE foo SET <arbitrary columns = arbitrary values> WHERE <arbitrary condition>
-- followed by
SELECT <the same columns> WHERE <the same condition>
```
And asserts that the values are the ones we just set

Reviewed-by: Pedro Muniz (@pedrocarlo)

Closes #2604
2025-08-14 19:18:09 +03:00
Jussi Saurio
1a11648974 sim: add Property::ReadYourUpdatesBack 2025-08-14 19:04:27 +03:00
Jussi Saurio
0b9d07db4c sim: restrict Update::arbitrary_from() to SET each column max once 2025-08-14 18:55:34 +03:00
PThorpe92
5a7c9325e2 Add assertion for expected write amount in writev callback 2025-08-14 10:53:16 -04:00
Jussi Saurio
649fcf9603 Merge 'Fix UPDATE: Do not use an index for iteration if that index is going to be updated' from Jussi Saurio
Closes #2598
Forbids using an index as the iteration cursor for an `UPDATE` statement
unless either of these is true:
1. The index is not affected by the `UPDATE` statement
2. The search condition on the index is guaranteed to return a maximum
of 1 row, so no iteration will happen
### Note
I have created a follow-up issue about handling this a bit better:
https://github.com/tursodatabase/turso/issues/2600

Closes #2599
2025-08-14 17:21:14 +03:00
Jussi Saurio
5da76c9125 Allow index in UPDATE for point queries (i.e. max 1 row affected) 2025-08-14 15:58:01 +03:00
Jussi Saurio
cd3b4bccd3 Fix UPDATE: Do not use an index for iteration if that index is going to be updated
Closes #2598
2025-08-14 15:35:00 +03:00
Jussi Saurio
f9bd047e4d Merge 'Fix non-4096 page sizes' from Jussi Saurio
Closes #2555
## Problem
The main problem we had with the current implementation of
`init_pager()` was that the WAL header was eagerly allocated and written
to disk with a page size, and a potential already-set page size on an
initialized database was not checked. Given this scenario:
- Initialized database with e.g. page size 512 but no WAL
- Tursodb connects to DB
It would not check the database file for the page size and instead would
initialize the WAL header with the default 4096 page size, plus
initialize the `BufferPool` similarly with the wrong size, and then
panic when reading pages from the DB, expecting to read `4096` instead
of `512`, as demonstrated in the reproduction of #2555.
## Fix
1. Add `Database::read_page_size_from_db_header()` method that can be
used in the above cases
2. Initialize the WAL header lazily during the first frame append, using
the existing `WalFile::ensure_header_if_needed()` method, removing the
need to eagerly pass `page_size` when constructing the in-memory
`WalFileShared` structure.
## Reader notes
This PR follows a fairly logical commit-by-commit structure so you'll
preferably want to read it that way.

Reviewed-by: Nikita Sivukhin (@sivukhin)

Closes #2569
2025-08-14 13:02:33 +03:00
Jussi Saurio
bd8c6f3c7c make PageSize more robust: only accept literal '1' value if it comes directly from db header 2025-08-14 12:40:58 +03:00
Jussi Saurio
c2e89f94f8 Change more page size panics to corrupt errors 2025-08-14 12:40:58 +03:00
Jussi Saurio
0c6d548402 integration test tweak 2025-08-14 12:40:58 +03:00