Commit Graph

2550 Commits

Author SHA1 Message Date
Jussi Saurio
d2b1be8af7 Merge 'optimizer: fix order by removal logic' from Jussi Saurio
1. `group_by_contains_all` was incorrect - it was not checking that all
order by columns are in group by; it was instead checking that all group
by columns are in order by, which is absolutely incorrect for the
intended purpose.
2. remove ORDER BY clause if GROUP BY clause can sort the rows in the
same way.
Test failures are not related

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1511
2025-05-19 11:29:17 +03:00
Jussi Saurio
b7b4f6a390 Merge 'Mark WHERE terms as consumed instead of deleting them' from Jussi Saurio
We've run into trouble in multiple places due to the fact that we delete
terms from the where clause (e.g. when a constant condition is removed,
or the term becomes part of an index seek key).
A simpler solution is to add a flag indicating that the term is consumed
(used), so that it is not translated in the main loop anymore when WHERE
clause terms are evaluated.
note: CI failures are unrelated

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1477
2025-05-19 11:28:09 +03:00
Jussi Saurio
092462fa74 fix build 2025-05-19 07:29:02 +03:00
Jussi Saurio
7c6a4410d2 Merge '(btree): Implement support for handling offset-based payload access with overflow support' from Krishna Vishal
This PR adds a new function `read_write_payload_with_offset` to support
reading and writing payload data at specific offsets, handling both
local content and overflow pages. This is a port of SQLite's
`accessPayload` function in `btree.c` and will be essential for
supporting incremental blob I/O in the coming PRs.
- Added a state machine called `PayloadOverflowWithOffset` to make the
procedure reentrant.
- Correctly processes both local payload data and payload stored in
overflow pages
Testing:
- Reading and writing to a column with no overflow pages.
- Reading and writing at an offset with overflow pages (spanning 10
pages)

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1476
2025-05-18 22:58:10 +03:00
Jussi Saurio
372850756d Merge 'Fix updating single value' from Pedro Muniz
Closes #1482. I needed to change the `key_exists_in_index` function
because it zips the values from the records it is comparing, but if one
of the records is empty or not of the same length, the `all` function
could return true incorrectly.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1514
2025-05-18 22:51:11 +03:00
pedrocarlo
fd51c0a970 invalidate records not necessary for fix 2025-05-18 16:43:25 -03:00
pedrocarlo
7f081c1ac9 remove transmute. Just iterate over columns. No need for unsafe 2025-05-18 12:32:49 -03:00
pedrocarlo
af1f9492ef fix updating single value 2025-05-17 19:43:24 -03:00
pedrocarlo
0e6ef1f478 removed some clone, simplified slightly logic + also inserted the column name of the current column we are iterating, not only the last column contained in PrimaryKeyDefinitionType::Simple 2025-05-17 15:32:58 -03:00
pedrocarlo
166dc2184e fix autoindex creation not detecting if column existed in created table declaration using transmute to avoid cloning 2025-05-17 12:58:00 -03:00
Jussi Saurio
93d88527c3 optimizer: remove order by if group by already sorts the result properly 2025-05-17 17:42:52 +03:00
Jussi Saurio
ce8b2722cf optimizer: fix incorrect logic in group_by_contains_all 2025-05-17 17:28:29 +03:00
Jussi Saurio
d584a1879b Mark WHERE terms as consumed instead of deleting them
We've run into trouble in multiple places due to the fact that
we delete terms from the where clause (e.g. when a constant condition
is removed, or the term becomes part of an index seek key).

A simpler solution is to add a flag indicating that the term is
consumed (used), so that it is not translated in the main loop
anymore when WHERE clause terms are evaluated.
2025-05-17 15:44:12 +03:00
Pekka Enberg
26e5496c19 Merge 'Fix autoindex of primary key marked as unique' from Pere Diaz Bou
Primary keys that are marked as unique constraint, do not need to have
separate indexes, one is enough. In the case primary key is integer,
therefore rowid alias, we still need an index to satisfy unique
constraint.

Closes #1494
2025-05-17 08:02:04 +03:00
Pekka Enberg
eb757436b2 Merge 'pager: bump default page cache size from 10 to 2000 pages' from Jussi Saurio
```
Execute `SELECT count() FROM users`/limbo_execute_select_count
                        time:   [12.867 µs 12.958 µs 13.104 µs]
                        change: [-91.233% -91.178% -91.120%] (p = 0.00 < 0.05)
                        Performance has improved.
```
also extracted it to a constant

Closes #1498
2025-05-16 16:32:33 +03:00
Jussi Saurio
416de9dd9c Extract page cache size constant and bump to 2k 2025-05-16 15:40:19 +03:00
Pere Diaz Bou
82e5597b00 long fuzz tests ci on btree changes
The idea is simple, if you modify the btree, we should verify fuzz tests
with long number of iterations to decrease the chance of a regression
2025-05-16 11:26:00 +02:00
Pere Diaz Bou
36dd5b9704 fmt 2025-05-16 09:06:44 +00:00
Pere Diaz Bou
45412a394f add another test with >1 indexes 2025-05-16 10:37:04 +02:00
Pere Diaz Bou
74328f2617 fix allocation of indices BTreeCreate registers
For some reason we always allocated one more index than required when we
had `total_indices>1`.
2025-05-16 10:37:04 +02:00
Pere Diaz Bou
ff524d037d fix autoindex of primary key marked as unique
Primary keys that are marked as unique constraint, do not need to have
separate indexes, one is enough. In the case primary key is integer,
therefore rowid alias, we still need an index to satisfy unique
constraint.
2025-05-16 10:37:04 +02:00
Jussi Saurio
b16044f34b pager: bump default page cache size from 10 to 1000 pages
```
Gnuplot not found, using plotters backend
Execute `SELECT count() FROM users`/limbo_execute_select_count
                        time:   [12.867 µs 12.958 µs 13.104 µs]
                        change: [-91.233% -91.178% -91.120%] (p = 0.00 < 0.05)
                        Performance has improved.
```
2025-05-16 09:23:42 +03:00
Pekka Enberg
a6270e8a6c Merge 'Add libsql_wal_frame_count() API' from Pekka Enberg
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1489
2025-05-15 12:01:45 +03:00
Pekka Enberg
2127de3422 cargo fmt 2025-05-15 12:00:55 +03:00
Pekka Enberg
524a523036 sqlite3: Add libsql_wal_frame_count() API 2025-05-15 11:43:44 +03:00
Pekka Enberg
9303262dfd OwnedValue -> Value 2025-05-15 11:43:23 +03:00
Pere Diaz Bou
ef65275bda Merge 'test page_free_array' from Pere Diaz Bou
Simply add a fuzz test to test free_array works as intended

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1480
2025-05-15 10:19:09 +02:00
Pekka Enberg
e3f71259d8 Rename OwnedValue -> Value
We have not had enough merge conflicts for a while so let's do a
tree-wide rename.
2025-05-15 09:59:46 +03:00
pedrocarlo
9fc9415b20 use Jussi's code to avoid cloning immutable record 2025-05-14 13:30:39 -03:00
pedrocarlo
72cc0fcdcb fixes and comments 2025-05-14 13:30:39 -03:00
pedrocarlo
b2615d7739 add CursorValidState and only save context in delete when rebalancing 2025-05-14 13:30:39 -03:00
pedrocarlo
814508981c fixing more rebase issues and cleaning up code. Save cursor context when calling delete for later use when needed 2025-05-14 13:30:39 -03:00
pedrocarlo
c69f503eac rebase adjustments 2025-05-14 13:30:39 -03:00
pedrocarlo
05f4ca28cc btree rewind and next fix. Keep track of rowids seen to avoid infinite loop 2025-05-14 13:30:39 -03:00
pedrocarlo
c146877344 add sqlite debug cli for nix. Fix cursor delete panic. Add tracing for cell indices in btree 2025-05-14 13:30:39 -03:00
pedrocarlo
6588004f80 fix incorrectly detecting if user provided row_id_alias to set clause 2025-05-14 13:30:39 -03:00
pedrocarlo
482634b598 adjust null opcode emission based in rowid_alias 2025-05-14 13:30:39 -03:00
pedrocarlo
758dfff2fe modified tests as we do not have rollback yet. Also correctly raise a contraint error on primary keys only 2025-05-14 13:30:39 -03:00
pedrocarlo
3aaf4206b7 altered constraint tests to create bad update statements. Tests caught a bug where I was copying the wrong values from the registers 2025-05-14 13:30:39 -03:00
pedrocarlo
cf7f60b8f5 changed from resolve_label to preassign_label 2025-05-14 13:30:39 -03:00
pedrocarlo
6457d7675a instruction emitted should be correct, but having an infinite loop bug 2025-05-14 13:30:39 -03:00
pedrocarlo
60a99851f8 emit NoConflict and Halt. Already detects unique constraints 2025-05-14 13:30:39 -03:00
pedrocarlo
5f2216cf8e modify explain for MakeRecord to show index name 2025-05-14 13:30:39 -03:00
pedrocarlo
9aebfa7b5d open cursors for write only once 2025-05-14 13:30:39 -03:00
pedrocarlo
5bae32fe3f modified OpenWrite to include index or table name in explain 2025-05-14 13:30:39 -03:00
pedrocarlo
e7fa023c26 Adding indexes to the update plan 2025-05-14 13:30:39 -03:00
pedrocarlo
4dc1431428 handling edge case when passing duplicate a multi-column unique index 2025-05-14 11:46:24 -03:00
pedrocarlo
ea15b1f617 extract primary key detection 2025-05-14 11:34:39 -03:00
pedrocarlo
b93156ee9d simplify unique sets creation 2025-05-14 11:34:39 -03:00
pedrocarlo
e75e20c9e9 fix incorrect matching in column name 2025-05-14 11:34:39 -03:00