Commit Graph

4261 Commits

Author SHA1 Message Date
Pekka Enberg
e7902541cd Merge 'Add time.Time support to Go driver parameter binding' from Jonathan Ness
### Problem
The Limbo Go driver currently throws an "unsupported type" error when
trying to bind `time.Time` values as query parameters. This requires
applications to manually convert datetime values to strings before
passing them to queries.
### Solution
Added `time.Time` support to the `buildArgs` function in `types.go`. The
implementation:
- Converts `time.Time` to RFC3339 format strings
- Uses the existing `textVal` type for storage
- Maintains compatibility with Limbo's datetime handling
### Example Usage
```go
// Previously failed with "unsupported type: time.Time"
now := time.Now()
db.Exec("INSERT INTO events (timestamp) VALUES (?)", now)

// Now works as expected
```
### Testing
I tested with various datetime operations:
- Parameter binding with time.Time
- Round-trip storage and retrieval
- Compatibility with existing datetime functions
Values are stored in standard ISO8601/RFC3339 format which I believe is
same as sqlite.

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #1442
2025-05-07 22:30:44 +03:00
Jussi Saurio
d592e8ec8a Merge 'Show explanation for the NewRowid opcode' from Anton Harniakou
After this commit explain will start showing a comment for NewRowid.
```
limbo> explain insert into t(id) values (1);
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     17    0                    0   Start at 17
...
6     NewRowid           0     1     0                    0   r[1]=rowid
...
18    Goto               0     1     0                    0
```

Closes #1445
2025-05-07 09:15:46 +03:00
Jussi Saurio
57b16d5b2b Merge 'Add notion of join ordering to plan' from Jussi Saurio
This PR is an enabler for our (Coming Soon ™️ ) join reordering
optimizer -- simply adds the notion of a join order to the current query
execution. This PR does not do any join ordering -- the join order is
always the same as expressed in the SQL query.

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #1439
2025-05-07 08:55:13 +03:00
Anton Harniakou
a971bce353 Show explanation for the NewRowid opcode 2025-05-04 14:14:19 +03:00
jnesss
061579e716 Add time.Time support to Go driver parameter binding 2025-05-03 14:09:25 -07:00
Jussi Saurio
46c915b13c Merge 'Add static feature to Cargo.toml to support extensions written inside core' from Pedro Muniz
Added the static feature from the extensions crate to core, as it is
needed to write extensions that should be defined and depend in code
from core. Added some docs to account for this.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1070
2025-05-03 19:26:06 +03:00
Jussi Saurio
4e05023bd3 Merge branch 'main' into ext-static-feature 2025-05-03 19:18:28 +03:00
Jussi Saurio
40c04c7074 Merge 'Adjust vtab schema creation to display the underlying columns' from Preston Thorpe
### The problem:
Sqlite displays the column names of the underlying vtab module when
displaying the `.schema`
![image](https://github.com/user-
attachments/assets/ca6aa1c9-0af7-4f34-a5c4-c8336fa23858)
Previously limbo omitted this, which makes it difficult for the user to
see what/how many columns the module's table has.
This matches sqlite's behavior by fetching the module's schema when the
schema entry is being inserted in translation.
![image](https://github.com/user-
attachments/assets/a56b8239-0f65-420b-a0b6-536ede117fba)

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1168
2025-05-03 19:17:25 +03:00
Jussi Saurio
c9eb56b54a Merge 'Read only mode' from Pedro Muniz
Closes #1413 . Basically, SQLite emits a check in a transaction to see
if it is attempting to write. If the db is in read only mode, it throws
an error, else the statement is executed. Mirroring how Rusqlite does
it, I modified the `OpenFlags` to use bitflags to better configure how
we open our VFS. This modification, will enable us to run tests against
the same database in parallel.

Closes #1433
2025-05-03 19:15:06 +03:00
Jussi Saurio
c2f30d796e Merge 'Test that DROP TABLE also deletes the related indices' from Anton Harniakou
These test were commented out, maybe they were written before CREATE
INDEX was added.

Closes #1437
2025-05-03 18:35:20 +03:00
Jussi Saurio
e57cea8de7 Merge 'reset statement before executing in rust binding' from Pedro Muniz
Closes #1426

Closes #1436
2025-05-03 18:34:44 +03:00
Jussi Saurio
7920161efc update Cargo.lock 2025-05-03 18:32:58 +03:00
Jussi Saurio
9ea958561b Merge 'Bump assorted dependencies' from Preston Thorpe
Closes #1425
2025-05-03 18:31:58 +03:00
Jussi Saurio
b86123a82e Merge 'Fix panic on async io due to reading locked page' from Preston Thorpe
closes  #1417
Man chasing this down was much much harder than it should have been.
We very frequently call `read_page` then push the return value onto the
page stack, or otherwise use it without it necessarily needing to not be
'in progress' of IO, so it was tricky to figure out where this was
happening and it had me thinking that it was something wrong with the
changes to `io_uring` on my branch.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1418
2025-05-03 18:30:29 +03:00
Jussi Saurio
5f91d30d94 Merge 'implement Clone for Arc<Mutex> types' from Pete Hayman
`Statement` and `Rows` both have a private Arc, implementing clone
avoids users needing to Arc<Mutex> it again.

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #1412
2025-05-03 18:30:00 +03:00
Jussi Saurio
fafeabd081 Merge 'Eliminate a superfluous read transaction when doing PRAGMA user_version' from Anton Harniakou
This PR removes an unnecessary read transaction.
Bytecode before this PR:
```
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     5     0                    0   Start at 5
1     Transaction        0     0     0                    0   write=false
2     ReadCookie         0     1     6                    0
3     ResultRow          1     1     0                    0   output=r[1]
4     Halt               0     0     0                    0
5     Transaction        0     0     0                    0   write=false
6     Goto               0     1     0                    0
```
Bytecode after this PR:
```limbo> explain PRAGMA user_version;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     4     0                    0   Start at 4
1     ReadCookie         0     1     6                    0
2     ResultRow          1     1     0                    0   output=r[1]
3     Halt               0     0     0                    0
4     Transaction        0     0     0                    0   write=false
5     Goto               0     1     0                    0
```

Closes #1431
2025-05-03 15:40:27 +03:00
Jussi Saurio
330fedbc2f Add notion of join ordering to plan + make determining where to eval expr dynamic always 2025-05-03 15:32:06 +03:00
Jussi Saurio
306e097950 Merge 'Fix bug: we cant remove order by terms from the head of the list' from Jussi Saurio
we had an incorrect optimization in `eliminate_orderby_like_groupby()`
where it could remove e.g. the first term of the ORDER BY if it matched
the first GROUP BY term and the result set was naturally ordered by that
term. this is invalid. see e.g.:
```sql
main branch - BAD: removes the `ORDER BY id` term because the results are naturally ordered by id.
However, this results in sorting the entire thing by last name only!

limbo> select id, last_name, count(1) from users GROUP BY 1,2 order by id, last_name desc limit 3;
┌──────┬───────────┬───────────┐
│ id   │ last_name │ count (1) │
├──────┼───────────┼───────────┤
│ 6235 │ Zuniga    │         1 │
├──────┼───────────┼───────────┤
│ 8043 │ Zuniga    │         1 │
├──────┼───────────┼───────────┤
│  944 │ Zimmerman │         1 │
└──────┴───────────┴───────────┘

after fix - GOOD:

limbo> select id, last_name, count(1) from users GROUP BY 1,2 order by id, last_name desc limit 3;
┌────┬───────────┬───────────┐
│ id │ last_name │ count (1) │
├────┼───────────┼───────────┤
│  1 │ Foster    │         1 │
├────┼───────────┼───────────┤
│  2 │ Salazar   │         1 │
├────┼───────────┼───────────┤
│  3 │ Perry     │         1 │
└────┴───────────┴───────────┘

I also refactored sorters to always use the ast `SortOrder` instead of boolean vectors, and use the `compare_immutable()` utility we use inside btrees too.

Closes #1365
2025-05-03 12:48:08 +03:00
Anton Harniakou
b6a5cbe626 Test that DROP TABLE also deletes the related indices 2025-05-03 12:41:19 +03:00
Anton Harniakou
3c0b7cad74 Eliminate a superfluous read transaction when doing PRAGMA user_version 2025-05-03 10:48:27 +03:00
Jussi Saurio
5689f0ef5e Merge 'update index on updated indexed columns' from Pere Diaz Bou
Previously columns that were indexed were updated only in the
BtreeTable, but not on Index table. This commit basically enables
updates on indexes too if they are needed.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1428
2025-05-03 10:41:13 +03:00
pedrocarlo
7cc190a12b reset statement before executing 2025-05-02 19:26:44 -03:00
pedrocarlo
2b3285d669 test opening in read only mode 2025-05-02 16:31:11 -03:00
pedrocarlo
0c22382f3c shared lock on file and throw ReadOnly error in transaction 2025-05-02 16:30:48 -03:00
PThorpe92
d4cf8367ba Wrap return_if_locked in balance non root in debug assertion cfg 2025-05-02 10:55:00 -04:00
PThorpe92
f025f7e91e Fix panic on async io due to reading locked page 2025-05-02 10:55:00 -04:00
Pere Diaz Bou
f15a17699b check indexes are not added twice in update plan 2025-05-01 12:38:34 +03:00
Pere Diaz Bou
c808863256 test update with index 2025-05-01 11:44:23 +03:00
Pere Diaz Bou
e503bb4641 run_query helper for test_write_path 2025-05-01 11:36:29 +03:00
Pere Diaz Bou
64a12ed887 update index on indexed columns
Previously columns that were indexed were updated only in the
BtreeTable, but not on Index table. This commit basically enables
updates on indexes too if they are needed.
2025-05-01 11:16:29 +03:00
Jussi Saurio
6096cfb3d8 Merge 'Add PRAGMA schema_version' from Anton Harniakou
This PR adds `PRAGMA schema_version` to get the value of the schema-
version integer at offset 40 in the database header.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1427
2025-05-01 10:50:02 +03:00
Jussi Saurio
a25f228ea7 Merge 'Fix setting default value for primary key on UPDATE' from Pere Diaz Bou
I noticed when updating a table with a primary key, it would sometimes
set primary key column to null. I believe the problem was due to
incorrect condition that was inconsistent with the comment above: "don't
emit null for pkey of virtual tables."
cc: @PThorpe92

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1422
2025-05-01 10:47:17 +03:00
Jussi Saurio
a525feb7ad Merge 'Fix: allow page_size=65536' from meteorgan
Since `page_size` in `DatabaseHeader` can be 1 representing 65526 bytes,
it can't be used it directly.  Additionally, we should use `u32` instead
of `u16` or `usize` in other contexts.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1411
2025-05-01 10:46:19 +03:00
Jussi Saurio
7643b7666c Merge 'Fix page_count pragma' from meteorgan
This issue was introduced in #819. However, I believe the solution is
suboptimal because `pragma page_count` can never return 1, which is
inconsistent with SQLite.
<img width="442" alt="image" src="https://github.com/user-
attachments/assets/c772eae7-3e9f-4687-a94a-230deb0eb034" />
To align with SQLite's behavior, we should allocate the first page when
the first schema object is created, rather than immediately after
creating database. And it's always preferable to return an accurate page
count.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1407
2025-05-01 10:36:36 +03:00
Pere Diaz Bou
1a2a383635 fix setting default value for primary key on UPDATE
I noticed when updating a table with a primary key, it would sometimes
set primary key column to null. A primary key can be nullified if it
isn't a rowid alias, meaning it isn't a INTEGER PRIMAR KEY.
2025-05-01 09:46:48 +03:00
Pete Hayman
56f0d25bb0 Merge branch 'tursodatabase:main' into main 2025-05-01 15:32:14 +10:00
Peter Hayman
8f366e98d5 add Row::column_count 2025-05-01 15:31:38 +10:00
Anton Harniakou
525b7fdbaa Add PRAGMA schema_version 2025-04-30 09:41:04 +03:00
PThorpe92
1e2be35e3b Add fs feature to rustix dependency 2025-04-29 23:07:28 -04:00
Pekka Enberg
f60fc26578 Merge 'Support literal-value current_time, current_date and current_timestamp' from meteorgan
I haven't found a way to automate these tests.
<p><img width="361" alt="image" src="https://github.com/user-
attachments/assets/a1563776-97e0-4aa5-844a-b9b23c5273e5" /></p>
<p><img width="279" alt="image" src="https://github.com/user-
attachments/assets/df036951-2649-4835-bffa-f25e6f59bb07" /></p>

Closes #1424
2025-04-29 21:51:33 +03:00
Pekka Enberg
3e6ac7c4a0 Merge 'Save history on exit' from Piotr Rżysko
Before this change, the history was only saved when the shell was
interrupted (e.g., Ctrl-C pressed twice). With this change, history is
now also saved when the `.exit` or `.quit` commands are used.
I attempted to add shell tests to cover the changes introduced in this
PR, but emulating a terminal/TTY that would work cross-platform seems to
require significant changes to `TestLimboShell` and `LimboShell`. For
example, the `pty` module [currently doesn't support
Windows](https://bugs.python.org/issue41663). I'm open to experimenting,
but I’m unsure if complicating these classes is worthwhile, as saving
history doesn't seem to be critical.
Additionally, it might be worth considering a refactor of the CLI so
that exit and cleanup operations are performed in one place.

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #1414
2025-04-29 21:50:41 +03:00
Pekka Enberg
f3a144638d Merge 'Fix broken fuzz target due to old name' from Levy A.
Closes #1416
2025-04-29 21:50:07 +03:00
Pekka Enberg
6409f347fd Merge 'Add state machine for op_idx_delete + DeleteState simplification' from Pere Diaz Bou
DeleteState had a bit too many unnecessary states so I removed them.
Usually we care about having a different state when I/O is triggered
requiring a state to be stored for later.
Furthermore, there was a bug with op_idx_delete where if balance is
triggered, op_idx_delete wouldn't be re-entrant. So a state machine was
added to prevent that from happening.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1421
2025-04-29 21:49:31 +03:00
PThorpe92
7b6452034b Bump lru dependency to 0.14.0 2025-04-29 10:44:26 -04:00
PThorpe92
7a3d949bd1 Bump mimalloc dependency to 0.1.46 2025-04-29 10:43:46 -04:00
PThorpe92
f581d1de3a Bump miette dependency to 7.6.0 2025-04-29 10:43:07 -04:00
PThorpe92
ba225ade0d Bump libc dependency to 0.2.172 2025-04-29 10:42:10 -04:00
PThorpe92
582ca68640 Bump rustix dependency to v1.0.5 2025-04-29 10:39:26 -04:00
PThorpe92
2785fd5d4a Bump polling crate dependency to 3.7.4 2025-04-29 10:38:46 -04:00
PThorpe92
be5ae7d0e3 Bump io_uring dependency to 0.7.5 2025-04-29 10:38:01 -04:00