Commit Graph

9353 Commits

Author SHA1 Message Date
Jussi Saurio
b702af8ac0 mvcc: add truncate method to logical log 2025-09-25 08:12:13 +03:00
Jussi Saurio
8f33b31c3d mvcc: update global header on commit end 2025-09-25 08:12:13 +03:00
Jussi Saurio
7464d1c172 mvcc: remove tx from store when commit write set is empty
previously, this was leaking transaction IDs in the mv store.
2025-09-25 08:12:13 +03:00
Jussi Saurio
949e6dd728 mvcc: remove unused states and fields from CommitStateMachine
none of the pager commit related data and logic are used anymore,
so let's delete them.
2025-09-25 08:12:12 +03:00
Pekka Enberg
0a78ea87d5 Merge 'core: Wrap Connection::attached_databases with RwLock' from Pekka Enberg
Closes #3310
2025-09-25 07:46:48 +03:00
Pekka Enberg
11288619af Merge 'Add built-in manual pages for Turso' from Glauber Costa
In the hopes of doing a good job at teaching people what Turso can do, I
am adding built-in manual pages. When the CLI starts, it picks a feature
at random, and tells the user that the feature exists:
```
Turso v0.2.0-pre.8
Enter ".help" for usage hints.
Did you know that Turso supports Change Data Capture? Type .manual cdc to learn more.
This software is ALPHA, only use for development, testing, and experimentation.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
```
There is a lot we can do to make this feature world class:
- we can automatically compile examples during compile time like rust-
doc, to make sure examples used in the manuals always work
- we can implement scrolling and navigation
- we can document a lot more features
But for now, this is a start!

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3298
2025-09-25 07:46:40 +03:00
Pere Diaz Bou
74f0830920 Merge 'core/mvcc/logical-log: on disk format for logical log' from Pere Diaz Bou
This format is based on previous discussions:
1. Log header
```rust
/// Log's Header, this will be the 64 bytes in any logical log file.
/// Log header is 64 bytes at maximum, fields added must not exceed that size. If it doesn't exceed
/// it, any bytes missing will be padded with zeroes.
struct LogHeader {
    version: u8,
    salt: u64,
    encrypted: u8, // 0 is no
}
```
2. Transaction format:
* Transaction id
* Checksum u64
* Byte size of all rows combined
* Rows
* End marker (offset position after appending buffer)
3. Row format:
```rust
    /// Serialize a row_version into on disk format.
    /// Format of a "row" (maybe we could change the name because row is not general enough for
    /// future type of values):
    ///
    /// * table_id (root page) -> u64
    /// * row type -> u8
    ///
    /// (by row type)
    /// Delete:
    /// * Payload length -> u64
    /// * Rowid -> varint
    ///
    /// Insert:
    /// * Payload length -> u64
    /// * Data size -> varint
    /// * Rowid -> varint
    /// * Data -> [u8] (data size length)
    fn serialize(&self, buffer: &mut Vec<u8>, row_version: &RowVersion) {

```

Closes #3245
2025-09-25 00:39:03 +02:00
Pekka Enberg
456bd619b8 Merge 'Wrap more Connection fields with atomics' from Pekka Enberg
Closes #3307
2025-09-24 20:16:42 +03:00
Pekka Enberg
f8f23a2a15 core: Wrap Connection::attached_databases with RwLock 2025-09-24 20:15:58 +03:00
Pekka Enberg
03263a9977 core: Wrap Connection::data_sync_retry with AtomicBool 2025-09-24 19:30:31 +03:00
Pekka Enberg
60e9d1a1c4 core: Wrap Connection::is_nested_stmt in AtomicBool 2025-09-24 19:30:31 +03:00
Pekka Enberg
a50771fe38 core: Wrap Connection::query_only with AtomicBool 2025-09-24 19:23:13 +03:00
Pekka Enberg
465dba573d core: Wrap Connection::closed with AtomicBool 2025-09-24 19:15:13 +03:00
Pekka Enberg
84588c03a8 Merge 'core/mvcc: Wrap Transaction::database_header with RwLock' from Pekka Enberg
Closes #3296
2025-09-24 19:14:22 +03:00
Pekka Enberg
eaddf1030d Merge 'core: Wrap Connection::capture_data_changes in RwLock' from Pekka Enberg
Closes #3293
2025-09-24 19:14:15 +03:00
Glauber Costa
fbc3d0dbc3 Add built-in manual pages for Turso
In the hopes of doing a good job at teaching people what Turso can do,
I am adding built-in manual pages. When the CLI starts, it picks a
feature at random, and tells the user that the feature exists:

```
Turso v0.2.0-pre.8
Enter ".help" for usage hints.
Did you know that Turso supports Change Data Capture? Type .manual cdc to learn more.
This software is ALPHA, only use for development, testing, and experimentation.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
```

There is a lot we can do to make this feature world class:
- we can automatically compile examples during compile time like
  rust-doc, to make sure examples used in the manuals always work
- we can implement scrolling and navigation
- we can document a lot more features

But for now, this is a start!
2025-09-24 11:29:24 -03:00
Preston Thorpe
0511183abc Merge 'Fix program counter update in sequence test op' from Preston Thorpe
it's not emitted anywhere yet but still saw this bug

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3297
2025-09-24 10:01:01 -04:00
Pekka Enberg
3f11badf24 Merge 'antithesis-tests: Rename "utils.py" to "helper_utils.py"' from Pekka Enberg
Antithesis is now supposed to detect that this is not a runnable driver
and not complain about it.

Closes #3294
2025-09-24 15:31:24 +03:00
PThorpe92
47aa03997b fix pc issue in sequence test op 2025-09-24 08:26:34 -04:00
Pekka Enberg
b590b353eb core/mvcc: Wrap Transaction::database_header with RwLock 2025-09-24 15:19:00 +03:00
Pekka Enberg
7f7ad90244 antithesis-tests: Rename "utils.py" to "helper_utils.py"
Antithesis is now supposed to detect that this is not a runnable driver
and not complain about it.
2025-09-24 14:28:35 +03:00
Pekka Enberg
41d26d807b core: Wrap Connection::capture_data_changes in RwLock 2025-09-24 11:32:05 +03:00
Pekka Enberg
c894dcf438 Merge 'Make some Connection fields atomic' from Pekka Enberg
...the quest for Send continues.

Closes #3288
2025-09-24 11:27:57 +03:00
Pekka Enberg
d0e15f9ac0 Merge 'Fix INSERT INTO t DEFAULT VALUES' from Jussi Saurio
Closes #3279

Closes #3291
2025-09-24 11:09:27 +03:00
Pekka Enberg
30d00eff46 Merge 'Support referring to rowid as _rowid_ or oid' from Jussi Saurio
Closes #3283

Closes #3289
2025-09-24 11:08:32 +03:00
Jussi Saurio
5c82b72e5f fix INSERT INTO t DEFAULT VALUES 2025-09-24 09:54:43 +03:00
Jussi Saurio
726bc24e78 Support referring to rowid as _rowid_ or oid 2025-09-24 09:17:28 +03:00
Pekka Enberg
042a8dd031 core: Wrap Connection::wal_auto_checkpoint_disabled with AtomicBool 2025-09-24 09:12:46 +03:00
Pekka Enberg
aa95cb24ea core: Wrap Connection::page_size with AtomicU16 2025-09-24 09:12:46 +03:00
Pekka Enberg
60d3a837b7 core: Wrap Connection::cache_size with AtomicI32 2025-09-24 09:12:46 +03:00
Pekka Enberg
0b6f535f4a core: Wrap Connection fields with AtomicI64 2025-09-24 09:12:46 +03:00
Pekka Enberg
f72600431b Turso 0.2.0-pre.8 2025-09-24 09:08:34 +03:00
Jussi Saurio
73571d9c55 Merge 'Don't allow duplicate columns and get column type as passed ' from Pavan Nambi
fixes #3231
```zsh

❯ sqlite3
SQLite version 3.50.4 2025-07-30 19:33:53
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite>   CREATE TABLE t1 (a);
    ALTER TABLE t1 ADD COLUMN a;
Parse error: duplicate column name: a
sqlite>   ALTER TABLE t1 ADD COLUMN name varchar(255);
    SELECT sql FROM sqlite_schema WHERE name = 't1';
CREATE TABLE t1 (a, name varchar(255))
sqlite>
```
```zsh
turso>
turso>  CREATE TABLE t1 (a);
    ALTER TABLE t1 ADD COLUMN a;
  x Parse error: duplicate column name: a

turso>  ALTER TABLE t1 ADD COLUMN name varchar(255);
    SELECT sql FROM sqlite_schema WHERE name = 't1';
┌─────────────────────────────────────────┐
│ sql                                     │
├─────────────────────────────────────────┤
│ CREATE TABLE t1 (a, name varchar (255)) │
└─────────────────────────────────────────┘
turso>
```

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #3249
2025-09-24 09:04:08 +03:00
Jussi Saurio
c697638413 Merge 'Simulator Runtime generation' from Pedro Muniz
Closes #3253
Had to do some slight changes in how we shadow interactions and changed
some of the execution code to accommodate both runtime generation and
pre-run generation (pre-run generation is needed for instance when
running the shrunk plans). This behavior is abstracted by the
`InteractionPlanIterator` trait

Closes #3272
2025-09-24 09:03:19 +03:00
Jussi Saurio
75a989a215 Merge 'fix: CTE alias resolution in planner' from Mayank
Closes #3182

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3243
2025-09-24 09:01:45 +03:00
Pekka Enberg
a372e07bb8 Merge 'Use SQL over HTTP batch statements for sync push' from Nikita Sivukhin
Use SQL over HTTP batch statement in the push logic of sync-engine with
additional condition for `!auto_commit` for every step except from
initial `BEGIN` of txn.
This is needed to avoid non-transactional update if SQLite will decide
to close transaction due to some error and reset state to autocommit
mode.

Closes #3271
2025-09-24 09:01:26 +03:00
Pekka Enberg
89b9439b1d Merge 'JavaScript bindings browser tests' from Nikita Sivukhin
Closes #3266
2025-09-24 09:00:22 +03:00
Pekka Enberg
248b38b6a9 Merge 'core: Wrap Connection::transaction_state with RwLock' from Pekka Enberg
Closes #3269
2025-09-24 08:44:40 +03:00
Jussi Saurio
d5de088abe Merge 'translate: implement Sequence opcode and fix sort order' from Preston Thorpe
This PR implements the `Sequence` and `SequenceTest` opcodes, although
does not yet add plumbing to emit the latter.
SQLite has two distinct mechanisms that determine the final row order
with aggregates:
Traversal order of GROUP BY, and ORDER BY tiebreaking. When ORDER BY
contains only aggregate expressions and/or constants, SQLite has no
extra tiebreak key, but when ORDER BY mixes aggregate and non-aggregate
terms, SQLite adds an implicit, stable row `sequence` so “ties” respect
the input order.
This PR also fixes an issue with a query like the following:
```sql
SELECT u.first_name, COUNT(*) AS c
FROM users u
JOIN orders o ON o.user_id = u.id
GROUP BY u.first_name
ORDER BY c DESC;
```
Because ORDER BY has only an aggregate (COUNT(*) DESC) and no non-
aggregate terms, SQLite traverses the group key (u.first_name) in DESC
order in this case, so ties on c naturally appear with group keys in
descending order.
Previously tursodb would return the group key sorted in ASC order,
because it was used in all cases as the default

Closes #3287
2025-09-24 08:38:08 +03:00
PThorpe92
4cc02dbed1 Add regression test for sort order with aggregate term in orderby with groupby 2025-09-23 23:08:31 -04:00
PThorpe92
58625b1c6d Use expr.is_constant instead of matching for literal directly 2025-09-23 23:08:04 -04:00
PThorpe92
376d2bf7b1 Add plumbing to add sequence column to stabilize tiebreakers in order+group by 2025-09-23 22:35:59 -04:00
PThorpe92
5afebc5f74 Add Sequence and SequenceTest opcode to explain 2025-09-23 22:34:33 -04:00
PThorpe92
3c8216caab Add Sequence and SequenceTest opcodes to vdbe and sorter 2025-09-23 22:34:13 -04:00
PThorpe92
7dccff0bee Update COMPAT.md, add sequence and sequencetest opcodes 2025-09-23 22:32:22 -04:00
pedrocarlo
44c08c02d1 do not shadow interactions when generating the next plan, only after the interaction is ran 2025-09-23 21:57:06 -03:00
pedrocarlo
69d09bcf8c modify code to use runtime generation 2025-09-23 21:57:06 -03:00
pedrocarlo
5f83f3bdfe modify plan generation to generate interactions on demand 2025-09-23 21:57:06 -03:00
Nikita Sivukhin
57c3d783e1 fix clippy 2025-09-23 18:24:52 +04:00
Nikita Sivukhin
e89dac98f3 use hrana batch stmt with Not(IsAutocommit) condition for push operation 2025-09-23 16:33:40 +04:00