Commit Graph

4936 Commits

Author SHA1 Message Date
PThorpe92
5dd466941e Handle upsert even in inserting_multiple_rows case 2025-09-17 06:44:09 -04:00
PThorpe92
85eee42bf1 Support quoted qualified identifiers in UPSERT excluded.x clauses 2025-09-17 06:44:08 -04:00
PThorpe92
d2cd833b86 Rewrite exprs in set + where clause for UPSERT 2025-09-17 06:38:25 -04:00
Pekka Enberg
2b89dd4ea8 Merge 'core: Wrap Pager dirty_pages in RwLock' from Pekka Enberg
Make it Sync and Send.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3175
2025-09-17 13:34:47 +03:00
Pekka Enberg
58b0e8665d Merge 'core: Wrap MvCursor in Arc<RwLock<>>' from Pekka Enberg
Make it Send and Sync.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3174
2025-09-17 13:34:34 +03:00
Pekka Enberg
cb68c953e1 Merge 'core/incremental: Wrap ViewTransactionState in Arc' from Pekka Enberg
Make it Send.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3173
2025-09-17 13:07:54 +03:00
Pekka Enberg
a2f6ff819a Merge 'core/function: Wrap ExtFunc in Arc' from Pekka Enberg
Make it Send.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3172
2025-09-17 13:07:43 +03:00
Pekka Enberg
bede1229a9 core: Wrap Pager dirty_pages in RwLock
Make it Sync and Send.
2025-09-17 12:55:26 +03:00
Pekka Enberg
182565fe0c core: Wrap MvCursor in Arc<RwLock<>>
Make it Send and Sync.
2025-09-17 12:46:55 +03:00
Pekka Enberg
e6822d26ab Merge 'core/vtab: Mark VTabModuleImpl as Send and Sync' from Pekka Enberg
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3171
2025-09-17 12:25:12 +03:00
Pekka Enberg
c2b8bb0a2f core/incremental: Wrap ViewTransactionState in Arc
Make it Send.
2025-09-17 12:23:29 +03:00
Pekka Enberg
dcb92f0b50 Merge 'core/vtab: Use AtomicPtr for table_ptr' from Pekka Enberg
Make it Send.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3169
2025-09-17 12:13:18 +03:00
Jussi Saurio
9a2797963a Merge 'Remove LimboResult enum and InsnFunctionStepResult::Busy variant' from Jussi Saurio
We can just use `LimboError::Busy` for both of these.

Reviewed-by: Pekka Enberg <penberg@iki.fi>

Closes #3170
2025-09-17 12:06:54 +03:00
Pekka Enberg
d113c3ac6b core/function: Wrap ExtFunc in Arc
Make it Send.
2025-09-17 11:57:23 +03:00
Pekka Enberg
d7977c25a8 core/vtab: Mark VTabModuleImpl as Send and Sync 2025-09-17 11:48:42 +03:00
Pekka Enberg
6472a71ae7 Merge 'core: Wrap symbol table with RwLock' from Pekka Enberg
Make it Send.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3168
2025-09-17 11:47:40 +03:00
Jussi Saurio
e066e4f747 clippy 2025-09-17 11:25:50 +03:00
Jussi Saurio
b9ceacc356 Remove InsnFunctionStepResult::Busy
we don't need all these busy variants, let's just handle
LimboError::Busy
2025-09-17 11:22:49 +03:00
Pekka Enberg
e21959d80e core/vtab: Use AtomicPtr for table_ptr
Make it Send.
2025-09-17 11:22:40 +03:00
Pekka Enberg
602c247f87 Merge 'core/ext: Switch vtab_modules from Rc to Arc' from Pekka Enberg
Closes #3166
2025-09-17 11:19:27 +03:00
Pekka Enberg
d7158262ab Merge 'core/storage: Clean up unused import warning in encryption.rs' from Pekka Enberg
...happens when encryption feature is disabled.

Closes #3165
2025-09-17 11:19:20 +03:00
Jussi Saurio
dc103da2ed Remove LimboResult
this is only used for returning LimboResult::Busy, and we already
have LimboError::Busy, so it only adds confusion.

Moreover, the current busy handler was not handling LimboError::Busy,
because it's returned as an error, not as Ok. So this may fix the
"busy handler not working" issue in the perf thrpt benchmark.
2025-09-17 11:04:44 +03:00
Pekka Enberg
50653258cf core: Wrap symbol table with RwLock
Make it Send.
2025-09-17 10:58:32 +03:00
Pekka Enberg
06d869ea5e core/ext: Switch vtab_modules from Rc to Arc 2025-09-17 10:36:12 +03:00
Pekka Enberg
1e90572e7a core/storage: Clean up unused import warning in encryption.rs
...happens when encryption feature is disabled.
2025-09-17 10:22:36 +03:00
Pekka Enberg
17e9f05ea4 core: Convert Rc<Pager> to Arc<Pager> 2025-09-17 09:32:49 +03:00
Jussi Saurio
104b8dd083 Merge 'Encrypt page 1' from
This PR extends the existing encryption support to include the database
header page (page 1).

Reviewed-by: Avinash Sajjanshetty (@avinassh)

Closes #3040
2025-09-17 09:26:06 +03:00
Jussi Saurio
fad8d0c8b8 fix build 2025-09-17 08:45:13 +03:00
Jussi Saurio
cae234818b Merge 'Inital support for window functions' from Piotr Rżysko
This adds basic support for window functions. For now:
* Only existing aggregate functions can be used as window functions.
* Specialized window-specific functions (`rank`, `row_number`, etc.) are
not yet supported.
* Only the default frame definition is implemented:
`RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW EXCLUDE NO OTHERS`.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3079
2025-09-17 08:29:16 +03:00
rajajisai
e605aff31b Merge branch 'main' into enc-page-1 2025-09-16 10:06:00 -04:00
rajajisai
89caa868f9 Encryption support for database header page 2025-09-16 10:04:30 -04:00
Pekka Enberg
ae25a0f088 Merge 'Implement Min/Max aggregators' from Glauber Costa
We have not implemented them before because they require the raw
elements to be kept. It is easy to see why in the following example:
```
current_min = 3;
insert(2) => current_min = 2 // can be done without state
delete(2) => needs to look at the state to determine new min!
```
The aggregator state was a very simple key-value structure. To
accomodate for min/max, we will make it into a more complex table, where
we can encode a more complex structure.
The key insight is that we can use a primary key composed of:
```
1) storage_id
2) zset_id,
3) element
```
The storage_id and zset_id are our previous key, except they are now
exploded to support a larger range of storage_id. With more bits
available in the storage_id, we can encode information about which
column we are storing. For aggregations in multiple columns, we will
need to keep a different list of values for min/max!
The element is just the values of the columns.
Because this is a primary key, the data will be sorted in the btree. We
can then just do a prefix search in the first two components of the key
and easily find the min/max when needed.
This new format is also adequate for joins. Joins will just have a new
storage_id which encodes two "columns" (left side, right side).

Closes #3143
2025-09-16 16:19:59 +03:00
Jussi Saurio
d9e7b7f0e1 mvcc: starting a pager read tx can fail with busy 2025-09-16 15:19:49 +03:00
Jussi Saurio
e012768549 mvcc: dont allow CONCURRENT transaction to overwrite others changes
We start a pager read transaction at the beginning of the MV transaction, because
any reads we do from the database file and WAL must uphold snapshot isolation.
However, we must end and immediately restart the read transaction before committing.
This is because other transactions may have committed writes to the DB file or WAL,
and our pager must read in those changes when applying our writes; otherwise we would overwrite
the changes from the previous committed transactions.

Note that this would be incredibly unsafe in the regular transaction model, but in MVCC we trust
the MV-store to uphold the guarantee that no write-write conflicts happened.
2025-09-16 15:03:26 +03:00
Jussi Saurio
b4fba69fe2 mvcc: fix logic bug in CommitState::WriteRow iteration order
We must iterate the row versions in reverse order because the
versions are in order of oldest to newest, and we must commit
the newest version applied by the active transaction.
2025-09-16 12:56:17 +03:00
Jussi Saurio
139ce39a00 mvcc: fix logic bug in MvStore::insert_version_raw()
In insert_version_raw(), we correctly iterate the versions backwards
because we want to find the newest version that is still older than
the one we are inserting.

However, the order of `.enumerate()` and `.rev()` was wrong, so the
insertion position was calculated based on the position in the
_reversed_ iterator, not the original iterator.
2025-09-16 12:56:17 +03:00
Jussi Saurio
847e413c34 mvcc: assert that DeleteRowStateMachine must find the row it is deleting 2025-09-16 12:56:17 +03:00
Pekka Enberg
74331898a3 Merge 'Add quoted identifier test cases for ALTER TABLE' from Levy A.
Resolves #2093
There is a small incompatibility on how we quote the added column on the
final schema, but doesn't change any behavior.

Closes #2943
2025-09-16 11:46:12 +03:00
Pekka Enberg
3c62352bcb core/mvcc: Specify level for tracing
..otherwise we perform the tracing for every step() dropping write
throughput by 40%.
2025-09-16 09:51:08 +03:00
Pekka Enberg
950cb8a818 Merge 'Move common dependencies to workspace ' from Pedro Muniz
This removes 4 crates from the `cargo build` and tries to ensure that in
the future we avoid the same crates with different versions.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3141
2025-09-16 08:30:06 +03:00
Glauber Costa
6bee6bb785 implement min/max
We have not implemented them before because they require the raw
elements to be kept. It is easy to see why in the following example:

current_min = 3;
insert(2) => current_min = 2 // can be done without state
delete(2) => needs to look at the state to determine new min!

The aggregator state was a very simple key-value structure. To
accomodate for min/max, we will make it into a more complex table, where
we can encode a more complex structure.

The key insight is that we can use a primary key composed of:

1) storage_id
2) zset_id,
3) element

The storage_id and zset_id are our previous key, except they are now
exploded to support a larger range of storage_id. With more bits
available in the storage_id, we can encode information about which
column we are storing. For aggregations in multiple columns, we will
need to keep a different list of values for min/max!

The element is just the values of the columns.

Because this is a primary key, the data will be sorted in the btree.
We can then just do a prefix search in the first two components of
the key and easily find the min/max when needed.

This new format is also adequate for joins. Joins will just have
a new storage_id which encodes two "columns" (left side, right side).
2025-09-15 22:30:48 -05:00
Glauber Costa
3565e7978a Add an index to the dbsp internal table
And also change the schema of the main table. I have come to see the
current key-value schema as inadequate for non-aggregate operators.
Calculating Min/Max, for example, doesn't feat in this schema because
we have to be able to track existing values and index them.

Another alternative is to keep one table per operator type, but this
quickly leads to an explosion of tables.
2025-09-15 22:30:48 -05:00
pedrocarlo
3c91ae206b move as many dependencies as possible to workspace to avoid multiple versions of the same dependency 2025-09-15 17:19:36 -03:00
Jussi Saurio
d2d1d1bc61 fix re-entrancy issue in Pager::free_page
current logic can lead to a situation where:

- we call read_page(trunk_page_id)
- we assign trunk_page in the FreePageState state machine
- the page read fails and cache marks it as !locked && !loaded
- next call to Pager::free_page() asserts that the page is loaded and panics
2025-09-15 21:41:18 +03:00
pedrocarlo
7021386f86 move divider_cell_is_overflow_cell to debug assertions so it stops appearing in release builds 2025-09-15 11:11:28 -03:00
Jussi Saurio
32cd01a615 fix deadlock 2025-09-15 14:48:26 +03:00
Jussi Saurio
d493a72cc0 dont unwrap begin_tx 2025-09-15 14:48:26 +03:00
Pekka Enberg
247d4c06c6 Merge 'Fix MVCC update' from Jussi Saurio
Based on #3126
Closes #3029
Closes #3030
Closes #3065
Closes #3083
Closes #3084
Closes #3085
simple reason why mvcc update didn't work: it didn't try to update.

Closes #3127
2025-09-15 14:24:59 +03:00
Pekka Enberg
a5eac9b700 Merge 'avoid unnecessary cloning when formatting Txn for Display' from Avinash Sajjanshetty
Closes #3109
2025-09-15 14:24:32 +03:00
Pekka Enberg
244458199f Merge 'Various fixes to sync' from Nikita Sivukhin
This PR fixes incorrect path registration for sync in browser, add tests
and also expose revision string in the `stats()` method of synced
database

Closes #3124
2025-09-15 14:24:02 +03:00