Commit Graph

1267 Commits

Author SHA1 Message Date
Jussi Saurio
506bb5f67f Merge 'Direct schema mutation – add instruction' from Levy A.
Resolves #2378.
```
`ALTER TABLE _ RENAME TO _`/limbo_rename_table/
                        time:   [15.645 ms 15.741 ms 15.850 ms]
Found 12 outliers among 100 measurements (12.00%)
  8 (8.00%) high mild
  4 (4.00%) high severe
`ALTER TABLE _ RENAME TO _`/sqlite_rename_table/
                        time:   [34.728 ms 35.260 ms 35.955 ms]
Found 15 outliers among 100 measurements (15.00%)
  8 (8.00%) high mild
  7 (7.00%) high severe
  ```
<img width="1000" height="199" alt="image" src="https://github.com/user-
attachments/assets/ad943355-b57d-43d9-8a84-850461b8af41" />

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2399
2025-08-04 16:55:38 +03:00
Jussi Saurio
8a1723b3c8 fix/core/translate: ALTER TABLE DROP COLUMN: ensure schema cookie is updated even when target table is empty 2025-08-04 15:05:00 +03:00
Levy A.
1e177053cb feat: add RenameTable instruction
direct schema mutation, no reparsing
2025-08-01 21:11:25 -03:00
Diego Reis
7c70ac2c4a Fix #2390
Single quotes inside a string literal have to be doubled
2025-08-01 11:37:13 -03:00
Jussi Saurio
86b1232268 chore: enable indexes by default 2025-08-01 15:44:56 +03:00
Jussi Saurio
7259751eba Merge 'Support the OFFSET clause for Compound select' from meteorgan
Closes #2376
2025-08-01 10:18:13 +03:00
Jussi Saurio
77666b1eb5 Merge 'Fix parser error for repetition in row values' from Diego Reis
Closes #1948
This PR also adds pretty basic support for [row values in UPDATE stateme
nts](https://sqlite.org/rowvalue.html#row_values_in_update_statements),
but it only accepts expressions like:
```sql
UPDATE t SET (a, b) = (2 + 2, 'joe');
```
While SQLite accepts whole new statements, like:
```sql
UPDATE tab3 
   SET (a,b,c) = (SELECT x,y,z
                    FROM tab4
                   WHERE tab4.w=tab3.d)
 WHERE tab3.e BETWEEN 55 AND 66;
 ```
I noticed we don't explicitly have the concept of row values, maybe
doing some plumbing in that matter could solve it?
If there is a way to implement that with our current infrastructure
(a.k.a skill issue from my side) please comment here.

Closes #2355
2025-08-01 10:17:05 +03:00
meteorgan
6262ff4267 support offset for values 2025-08-01 00:46:46 +08:00
Glauber Costa
9d41fa4489 implement IN patterns for non-conditional SELECT queries
Extracts the core logic of IN from the conditional version, and uses the
conditional metadata to determine the jump. Then Uses the AddImm
operator we just added to force the integer conversion at the end (like
SQLite does).
2025-07-31 08:11:41 -05:00
meteorgan
a0f5554b08 support the OFFSET clause for Compound select 2025-07-31 17:43:54 +08:00
Jussi Saurio
f619556344 Merge 'Direct DatabaseHeader reads and writes – with_header and with_header_mut' from Levy A.
This PR introduces two methods to pager. Very much inspired by
`with_schema` and `with_schema_mut`. `Pager::with_header` and
`Pager::with_header_mut` will give to the closure a shared and unique
reference respectively that are transmuted references from the `PageRef`
buffer.
This PR also adds type-safe wrappers for `Version`, `PageSize`,
`CacheSize` and `TextEncoding`, as they have special in-memory
representations.
Writing the `DatabaseHeader` is just a single `memcpy` now.
```rs
pub fn write_database_header(&self, header: &DatabaseHeader) {
    let buf = self.as_ptr();
    buf[0..DatabaseHeader::SIZE].copy_from_slice(bytemuck::bytes_of(header));
}
```
`HeaderRef` and `HeaderRefMut` are used in the `with_header*` methods,
but also can be used on its own when there are multiple reads and writes
to the header, where putting everything in a closure would add too much
nesting.

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #2234
2025-07-31 10:02:47 +03:00
Diego Reis
ab01b4e8ca Refactor UPDATE .. SET row values logic and add some comments 2025-07-31 00:08:15 -03:00
Diego Reis
31c73f3c9a Add basic support for row values in UPDATE .. SET statements
e.g `.. SET (a, b) = (1, 2)` is equivalent to `.. SET a = 1, b = 2`.

Alongside, to repeated lhs values, `(a, a)`, the last rhs prevail; so
`.. SET (a, a) = (1, 2)` is equivalent to `.. SET a = 2`
2025-07-31 00:08:15 -03:00
Diego Reis
3834f441c4 Accept parsing SET statements with repeated names, like `.. SET (a, a) =
(1, 2)`
2025-07-31 00:08:12 -03:00
PThorpe92
07137c7aaf Merge 'Implement the Cast opcode' from Glauber Costa
Our compat matrix mentions a couple of opcodes: ToInt, ToBlob, etc.
Those opcodes do not exist.
Instead, there is a single Cast opcode, that takes the affinity as a
parameter.
Currently we just call a function when we need to cast. This PR fixes
the compat file, implements the cast opcode, and in at least one
instance, when explicitly using the CAST keyword, uses that opcode
instead of a function in the generated bytecode.

Reviewed-by: Preston Thorpe (@PThorpe92)
Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #2352
2025-07-30 22:32:09 -04:00
Glauber Costa
4bd1582e7d Implement the Cast opcode
Our compat matrix mentions a couple of opcodes: ToInt, ToBlob, etc.
Those opcodes do not exist.

Instead, there is a single Cast opcode, that takes the affinity as a
parameter.

Currently we just call a function when we need to cast. This PR fixes
the compat file, implements the cast opcode, and in at least one
instance, when explicitly using the CAST keyword, uses that opcode
instead of a function in the generated bytecode.
2025-07-30 20:44:54 -05:00
Levy A.
e35fdb8263 feat: zero-copy DatabaseHeader 2025-07-30 17:33:59 -03:00
Jussi Saurio
e128bd477e Merge 'Support VALUES clauses for compound select' from meteorgan
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2293
2025-07-30 21:34:40 +03:00
Jussi Saurio
7caef278a5 Merge 'Rewrite the WAL' from Preston Thorpe
closes #1893
Adds some fairly extensive tests but I'll continue to add some python
tests on top of the unit tests.
## Restart:
tested 
- open new DB
- create table and do a bunch of inserts
- `pragma wal_checkpoint(RESTART);`
- close db file
- re-open and verify we can read the wal/repopulate the frame cache
- verify min|max frame
tested 
- open same DB
- add more inserts
- `pragma wal_checkpoint(RESTART);`
- do _more_ inserts
- close
- re-open
- verify checksums/max_frame are valid
- verify row count
## Truncate
tested 
- open new db
- create table and add inserts
- `pragma wal_checkpoint(truncate);`
- close file
- verify WAL file is empty (32 bytes, header only)
- re-open file
- verify content/row count
tested 
- open db
- create table and insert many rows
- `pragma wal_checkpoint(truncate);`
- insert _more_ rows
- close db file
- verify WAL file is valid
- re-open file
- verify we can read entire file/repopulate the frame cache
<img width="541" height="315" alt="image" src="https://github.com/user-
attachments/assets/0470c795-5116-4866-b913-78c07b06b68c" />
```
# header
magic=0x377f0682
version=3007000
page_size=4096
seq=2
salt=ec475ff2-7ea94342
checksum=c9464aff-c571cc22
```

Closes #2179
2025-07-30 18:50:49 +03:00
PThorpe92
f4becd1296 Allow using !passive checkpoint methods in pragma wal_checkpoint 2025-07-30 14:08:33 +03:00
bit-aloo
a5dce2b50b add subvector execution flow 2025-07-30 09:51:08 +05:30
bit-aloo
e4d79a6516 add vec_concat execution flow 2025-07-30 06:07:03 +05:30
Pekka Enberg
94e2d3a004 Merge 'perf: fix logic error in is_simple_count()' from Jussi Saurio
```
Execute `SELECT count() FROM users`/limbo_execute_select_count
                        time:   [15.635 µs 15.676 µs 15.730 µs]
                        change: [-96.011% -95.991% -95.972%] (p = 0.00 < 0.05)
                        Performance has improved.
```
Performance has improved.
Closes #2313

Closes #2314
2025-07-29 09:14:48 +03:00
Jussi Saurio
574c15b5e4 perf: fix logic error in is_simple_count() 2025-07-29 09:11:54 +03:00
Diego Reis
bab10909c3 Disable extension loading for wasm
We should enable it later when wasm become more mature
2025-07-28 14:49:07 -03:00
Pere Diaz Bou
752a876f9a change every Rc to Arc in schema internals 2025-07-28 10:51:17 +02:00
Kristofer Lund
cbd5a26cf7 Adding Expr::Id as an allowed Expr when
creating an index.
2025-07-27 22:54:20 +02:00
meteorgan
aa69b279c3 support limit 2025-07-28 00:58:20 +08:00
meteorgan
ea660b947d support VALUES clauses for compound select 2025-07-27 19:13:23 +08:00
Pekka Enberg
6bf6cc28e4 Merge 'Implement the Returning statement for inserts and updates' from Glauber Costa
They are very similar. DELETE is very different, so that one we'll do it
later.

Closes #2276
2025-07-27 09:11:16 +03:00
Pekka Enberg
ab39ea54c7 Merge 'Fix error handling when binding column references while translating the UPDATE statement' from Iaroslav Zeigerman
Closes #1968

Reviewed-by: bit-aloo (@Shourya742)

Closes #2273
2025-07-27 09:05:17 +03:00
Glauber Costa
b8ee38868d implement the pragma encoding
Do not allow setting it. That ship has sailed around 2005.
2025-07-26 19:37:39 -05:00
Glauber Costa
5d8d08d1b6 Implement the Returning statement for inserts and updates
They are very similar. DELETE is very different, so that one we'll
do it later.
2025-07-26 09:01:09 -05:00
Iaroslav Zeigerman
6f63327320 fix overlooked tests 2025-07-26 04:51:44 -07:00
Iaroslav Zeigerman
f13b9105b9 Fix error handling when binding column references while translating the UPDATE statement 2025-07-26 04:51:42 -07:00
Pekka Enberg
cc5d4dc3ba Merge 'support doubly qualified identifiers' from Glauber Costa
Closes #2271
2025-07-26 11:31:42 +03:00
Glauber Costa
b5927dcfd5 support doubly qualified identifiers 2025-07-25 14:52:45 -05:00
meteorgan
b5a18d7dc9 fix get_column_name() when column name doesn't exist 2025-07-25 23:49:31 +08:00
Pekka Enberg
669b231714 Merge 'parser: Distinguish quoted identifiers and unify Id into Name enum' from bit-aloo
Closes: #1947
This PR replaces the `Name(pub String)` struct with a `Name` enum that
explicitly models how the name appeared in the source either as an
unquoted identifier (`Ident`) or a quoted string (`Quoted`).
In the process, the separate `Id` wrapper type has been coalesced into
the `Name` enum, simplifying the AST and reducing duplication in
identifier handling logic.
While this increases the size of some AST nodes (notably
`yyStackEntry`).
cc: @levydsa

Reviewed-by: Levy A. (@levydsa)
Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #2251
2025-07-25 12:08:54 +03:00
Glauber Costa
988b16f962 Support ATTACH (read only)
Support for attaching databases. The main difference from SQLite is that
we support an arbitrary number of attached databases, and we are not
bound to just 100ish.

We for now only support read-only databases. We open them as read-only,
but also, to keep things simple, we don't patch any of the insert
machinery to resolve foreign tables.  So if an insert is tried on an
attached database, it will just fail with a "no such table" error - this
is perfect for now.

The code in core/translate/attach.rs is written by Claude, who also
played a key part in the boilerplate for stuff like the .databases
command and extending the pragma database_list, and also aided me in
the test cases.
2025-07-24 19:19:48 -05:00
bit-aloo
3cb2db933d remove Id 2025-07-24 14:40:24 +05:30
bit-aloo
9a54ef214e parser: Distinguish quoted identifiers and unify Id into Name enum
This commit replaces the `Name(pub String)` struct with a `Name` enum that
explicitly models how the name appeared in the source either as an
unquoted identifier (`Ident`) or a quoted string (`Quoted`).

In the process, the separate `Id` wrapper type has been coalesced into the
`Name` enum, simplifying the AST and reducing duplication in identifier
handling logic.

While this increases the size of some AST nodes (notably `yyStackEntry`),
it improves correctness and makes source structure more explicit for
later phases.
2025-07-24 14:40:19 +05:30
Jussi Saurio
2d3c9001ee Merge 'emit SetCookie after DropTable' from Glauber Costa
The SetCookie opcode is used, among other things, to notify the
transaction of schema changes. We are not issuing it on DropTable.
Without it, the transaction thinks the schema hasn't changed, and does
not update the schema of the connection back to the database.
SQLite will, of course, issue it:
35    DropTable      0     0     0     foo            0
36    SetCookie      0     1     2                    0
Unfortunately I don't have a unit test that breaks with this, because
the one that is supposed to break is having, let's put it this way,
bigger problems.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2249
2025-07-24 10:12:16 +03:00
Jussi Saurio
92a10f94d8 Merge 'Bail early for read-only virtual tables' from Preston Thorpe
This PR adds a const associated value on the VTabModule trait,
`READONLY` defaulted to `true`, so we can bail early when a write
operation is done on an invalid vtable.
This prevents extensions from having to implement `insert`,`update`,
`delete` just to return `Error::ReadOnly`, and prevents us from having
to step through `VUpdate` just to error out.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2247
2025-07-24 10:12:07 +03:00
Glauber Costa
2a2468026c emit SetCookie after DropTable
The SetCookie opcode is used, among other things, to notify the
transaction of schema changes. We are not issuing it on DropTable.

Without it, the transaction thinks the schema hasn't changed, and does
not update the schema of the connection back to the database.

SQLite will, of course, issue it:

35    DropTable      0     0     0     foo            0
36    SetCookie      0     1     2                    0

Unfortunately I don't have a unit test that breaks with this, because
the one that is supposed to break is having, let's put it this way,
bigger problems.
2025-07-23 19:34:41 -05:00
PThorpe92
0871a8c7f3 Bail early when we detect a readonly virtual table 2025-07-23 16:57:30 -04:00
Nikita Sivukhin
001670c069 fix clippy 2025-07-23 16:00:24 +04:00
Nikita Sivukhin
a017baced7 small refactoring 2025-07-23 15:56:38 +04:00
Nikita Sivukhin
d3f3807ede fix cdc emit 2025-07-23 15:50:34 +04:00
Nikita Sivukhin
fd63128227 adjust behaviour in case when both rowid alias and rowid provided 2025-07-23 15:42:05 +04:00