Commit Graph

815 Commits

Author SHA1 Message Date
Diego Reis
f0f9ad4844 core: Get rid of maybe_init_database_file
Initialization now only occurs in the first write transaction
2025-06-24 14:41:50 -03:00
Diego Reis
ce73025202 core: Only actually allocate page1 on first write transaction 2025-06-24 14:41:50 -03:00
Jussi Saurio
cc2e14b11c Read page 1 from pager always, no separate db_header 2025-06-24 14:41:49 -03:00
Nils Koch
2827b86917 chore: fix clippy warnings 2025-06-23 19:52:13 +01:00
Pere Diaz Bou
404b5fad8a clippy 2025-06-23 18:19:44 +02:00
Pere Diaz Bou
b7fd4f1e18 update nchanges on op_insert 2025-06-23 18:11:58 +02:00
pedrocarlo
74beac5ea8 ephemeral table for update when rowid is being update 2025-06-20 16:28:10 -03:00
Jussi Saurio
c69047106c Merge 'Implement RowData opcode' from meteorgan
The `RowData` opcode is required to implement #1575.
I haven't found a ideal way to test this PR independently, but I
verified its functionality while working on #1575(to be committed soon),
and it performs effectively.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1756
2025-06-20 21:58:47 +03:00
Jussi Saurio
38f6b8c031 Merge 'Support indent for Goto opcode when executing explain' from meteorgan
it works as expected
```
limbo> explain insert into tb1 select * from tb2 union select * from tb3;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     29    0                    0   Start at 29
1     InitCoroutine      1     21    2                    0
2     OpenEphemeral      0     0     0                    0   cursor=0 is_table=false
3     OpenRead           1     3     0                    0   table=tb2, root=3
4     Rewind             1     9     0                    0   Rewind table tb2
5       Column           1     0     2                    0   r[2]=tb2.age
6       MakeRecord       2     1     3                    0   r[3]=mkrec(r[2..2]); for union_dedupe
7       IdxInsert        0     3     0                    0   key=r[3]
8     Next               1     5     0                    0
9     OpenRead           2     4     0                    0   table=tb3, root=4
10    Rewind             2     15    0                    0   Rewind table tb3
11      Column           2     0     2                    0   r[2]=tb3.age
12      MakeRecord       2     1     4                    0   r[4]=mkrec(r[2..2]); for union_dedupe
13      IdxInsert        0     4     0                    0   key=r[4]
14    Next               2     11    0                    0
15    Rewind             0     18    0                    0   Rewind  union_dedupe
16      Column           0     0     2                    0   r[2]=union_dedupe.age
17      Yield            1     0     0                    0
18    Next               0     16    0                    0
19    Close              0     0     0                    0
20    EndCoroutine       1     0     0                    0
21    OpenWrite          3     2     0                    0   root=2; tb1
22      Yield            1     28    0                    0
23      Copy             2     7     0                    0   r[7]=r[2]
24      NewRowid         3     6     0                    0   r[6]=rowid
25      MakeRecord       7     1     8                    0   r[8]=mkrec(r[7..7])
26      Insert           3     8     6     tb1            0   intkey=r[6] data=r[8]
27    Goto               0     22    0                    0
28    Halt               0     0     0                    0
29    Transaction        0     1     0                    0   write=true
30    Goto               0     1     0                    0
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1775
2025-06-20 21:57:40 +03:00
Luca Muscat
7cf77fb35b Fix fuzz issue #1763 by using the log2 & log10 functions where applicable
It is easy to chalk this fuzzer issue to erratic floating point
behaviour, but this is not the case here.

Currently, `exec_math_log` calculates log with arbitrary bases by using
the following formula: `log_a(b) ~= ln(b) / ln(a)`. This calculation is
an approximation with lots of its floating point precision lost to
dividing the results of natural logarithms.

By using the specialized versions of the log functions (`log2` &
`log10`), we can avoid this loss of precision.

SQLite also uses these specialized log functions when possible, so it
doesn't hurt to do the same thing when aiming for parity.
2025-06-20 10:52:13 +02:00
Pere Diaz Bou
65372994d4 op_transaction end_read_tx in case of begin_write_tx is busy
In case of starting a write txn, we always start by starting a read txn.
If we don't end_read_tx then it will never release locks.
2025-06-18 17:40:53 +02:00
meteorgan
5c7d1423e7 Support indent for Goto opcode when executing explain 2025-06-17 23:55:39 +08:00
Luca Muscat
a5ac1884c1 core: Stop treating the contents of a Value::Blob as a String
By encoding a Vec<u8> (vector of bytes), a lossy conversion from a
`Vec<u8>` to a `String` occurs. The lossy conversion leads to an
incorrect hex value to be displayed.

Avoid the lossy conversion and let the `hex` crate do its thing.
2025-06-16 21:10:17 +02:00
meteorgan
caa1801ec4 cargo fmt 2025-06-16 22:09:43 +08:00
meteorgan
4f742a3a0f Implement RowData opcode 2025-06-16 22:07:35 +08:00
Pekka Enberg
db4945eada Merge 'Fix update queries to set n_changes ' from Kim Seon Woo
- `Update` query doesn't update `n_changes`. Let's make it work
- Add `InsertFlags` to add meta information related to insert operations
- For update query, add `UPDATE` flag
- Currently, the update query executes `Insn::Delete` and `Insn::Insert`
internally, it increases `n_change` by 2. So, for the update query,
let's skip increasing `n_change` for the `Insn::Insert`
https://github.com/tursodatabase/limbo/issues/1681

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1683
2025-06-16 16:30:20 +03:00
Pekka Enberg
4496a0d08a core: Clean up integrity_check()
Suggested by Jussi.
2025-06-16 14:46:36 +03:00
Pekka Enberg
882c5ca168 Merge 'Simple integrity check on btree' from Pere Diaz Bou
This PR adds support for the instruction `IntegrityCk` which performs an
integrity check on the contents of a single table. Next PR I will try to
implement the rest of the integrity check where we would check indexes
containt correct amount of data and some more.
<img width="1151" alt="image" src="https://github.com/user-
attachments/assets/29d54148-55ba-480f-b972-e38587f0a483" />

Closes #1719
2025-06-16 13:46:26 +03:00
Pekka Enberg
90c1e3fc06 Switch Connection to use Arc instead of Rc
Connection needs to be Arc so that bindings can wrap it with `Mutex` for
multi-threading.
2025-06-16 10:43:19 +03:00
meteorgan
b910fd0012 drop unused code in op_column 2025-06-15 00:50:16 +08:00
Jussi Saurio
30e4511d62 Merge 'NOT NULL constraint' from Anton Harniakou
Implements basic support for `NOT NULL` contraint check.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1675
2025-06-13 14:25:24 +03:00
Anton Harniakou
25d0584b19 Fix typo 2025-06-13 11:57:39 +03:00
Jussi Saurio
3f90fad131 Merge 'Simulator Ast Generation + Simulator Unary Operator + Refactor to use limbo_core::Value in Simulator for massive code reuse' from Pedro Muniz
This PR is a Drop-In replacement to the Predicate defined in the
Simulator. Predicate is basically the same as our ast::Expr, but it
supports a small number of the SQL expression syntax. By creating a
NewType that wraps ast::Expr we can tap into our already mostly
correctly defined parser structs. This change will enable us to easily
add generation for more types of sql queries.
I also added an ArbitraryFrom impl for ast::Expr that can be used in a
freestyle way (for now) for differential testing.
This PR also aims to implement Unary Operator logic similar to the
Binary Operator logic we have for predicate. After this change we may
need to adjust the Logic for how some assertions are triggered.
<s>Sometimes the `Select-Select-Optimizer` property thinks that these
two queries should return the same thing:
```sql
SELECT (twinkling_winstanley.sensible_federations > x'66616e7461737469625e0f37879823db' AND twinkling_winstanley.sincere_niemeyer < -7428368947470022783) FROM twinkling_winstanley WHERE 1;

SELECT * FROM twinkling_winstanley WHERE twinkling_winstanley.sensible_federations > x'66616e7461737469625e0f37879823db' AND twinkling_winstanley.sincere_niemeyer < -7428368947470022783;
```
However after running the shrunk plan manually, the simulator was
incorrect in asserting that. Maybe this a bug a in the generation of
such a query? Not sure yet. </s>
<b>EDIT: The simulator was correctly catching a bug and I thought I was
the problem. The bug was in `exec_if` and I fixed it in this PR.</b>
I still need to expand the Unary Operator generation to other types of
predicates. For now, I just implemented it for `SimplePredicate` as I'm
trying to avoid to bloat even more this PR.
<b>EDIT: I decided to just have one PR open for all the changes I'm
making to make my life a bit easier and to avoid merge conflicts with my
own branches that I keep spawning for new code.</b>
PS: This should only be considered for merging after
https://github.com/tursodatabase/limbo/pull/1619 is merged. Then, I will
remove the draft status from this PR.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1674
2025-06-13 11:23:10 +03:00
Jussi Saurio
efed4bc07d Merge 'Complete ALTER TABLE implementation' from Levy A.
Resolves #895
- [x] `ALTER TABLE _ ADD _`
- [x] `ALTER TABLE _ DROP _`
- [x] `ALTER TABLE _ RENAME _ TO _`

Reviewed-by: Preston Thorpe (@PThorpe92)
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1527
2025-06-13 10:23:43 +03:00
Jussi Saurio
04c590f5a6 Add comment about re-entrancy of op_idx_insert() 2025-06-12 12:36:53 +03:00
Jussi Saurio
bdbd021bbb Fix large inserts to unique indexes hanging
We were incorrectly setting `moved_before` as `false` after checking
for unique constraint violation, but the reality is that after the
uniqueness check, we are already correctly positioned -- if no match
was found, the cursor is positioned at either:

a) no row, or
b) at the first entry that is greater than the key we are inserting.

This means we don't have to move anymore and can just insert.
2025-06-12 12:25:17 +03:00
pedrocarlo
8dbf09bb42 betters instrumentation for btree operations 2025-06-11 23:34:32 -03:00
Levy A.
43db84e6ea fix: broken rebase 2025-06-11 14:19:06 -03:00
Levy A.
01a680b69e feat(fuzz)+fix: add schema fuzz testing and fix some bugs 2025-06-11 14:19:06 -03:00
Levy A.
49a6ddad97 wip 2025-06-11 14:19:04 -03:00
Levy A.
41cb13aa74 fix: ignore non-constants 2025-06-11 14:18:41 -03:00
Levy A.
15e0cab8d8 refactor+fix: precompute default values from schema 2025-06-11 14:18:39 -03:00
Levy A.
7638b0dab7 fix: use default value on empty columns added via ALTER TABLE 2025-06-11 14:18:19 -03:00
Levy A.
6945c0c09e fix+refactor: incorrect label placement
also added a `cursor_loop` helper on `ProgramBuilder` to avoid making
this mistake in the future. this is zero-cost, and will be optimized to
the same thing (hopefully).
2025-06-11 14:17:36 -03:00
Levy A.
de2ac89ad2 feat: complete ALTER TABLE implementation 2025-06-11 14:17:36 -03:00
Pere Diaz Bou
3f3e91927f fmt 2025-06-11 19:02:23 +02:00
Pere Diaz Bou
d3c646378a Cell coverage checker
We check cells and freeblocks do not overlap and the fragmentation is
correct.
2025-06-11 16:50:30 +02:00
pedrocarlo
cf34e21ba9 fix for bug caught in simulator regarding exec_if 2025-06-11 11:32:45 -03:00
Pere Diaz Bou
9383ba207d introduce integrity_check pragma 2025-06-11 11:14:29 +02:00
Krishna Vishal
1c6a65ded4 Change seek op match from unit variants to struct variants. 2025-06-11 00:44:07 +05:30
krishvishal
c8da564aeb smol edit 2025-06-11 00:33:48 +05:30
krishvishal
5a1da026e6 Unify comparison function to reduce code duplication 2025-06-11 00:33:47 +05:30
krishvishal
5837f7329f clean up 2025-06-11 00:33:47 +05:30
krishvishal
d13abad4b1 Handle Blob type together with Text type in op_seek 2025-06-11 00:33:47 +05:30
krishvishal
f0dda1702f Fix AFFINITY_MASK value 2025-06-11 00:33:47 +05:30
krishvishal
faa9aedbae Add affinity based type coercion to comparison ops 2025-06-11 00:33:47 +05:30
krishvishal
7bd1589615 Added affinity inference and conversion for comparison ops.
Added affinity helper function for `CmpInsFlags`
2025-06-11 00:33:44 +05:30
krishvishal
3b2980c7c0 Fix op_seek to handle affinity coercion 2025-06-11 00:33:05 +05:30
krishvishal
e68293a1d1 Add affinity conversion to op_gt, op_le, op_lt, op_eq, op_ne 2025-06-11 00:33:05 +05:30
krishvishal
30ccbe46c7 Added apply_numeric_affinity function to handle string conversion
to integer. Exising functions' behavior is tailored to `CAST` ops.
SQLite has different behavior when it comes to handling string to
`integer` conversion in CAST vs predicate ops.
2025-06-11 00:33:05 +05:30