Commit Graph

1512 Commits

Author SHA1 Message Date
Pekka Enberg
e559b0c4b3 Merge 'Expose types in Wal' from Jorge López Tello
We expose Wal trait as public, but there are three types in its
signature that are private.
Notice I chose to expose limbo_core::result instead of
limbo_core::result::LimboResult, since LimboResult is the only type in
that module.

Closes #939
2025-02-10 07:38:59 +02:00
Pekka Enberg
728e8ca4f1 Merge 'chore: remove unused dependencies' from meteorgan
remove unused dependencies

Closes #948
2025-02-09 17:36:18 +02:00
Pekka Enberg
9e65d7ddea Merge 'perf/prepare: box even more stuff to shrink the YYMINORTYPE enum further' from Jussi Saurio
the boxings will continue until morale improves
<img width="621" alt="Screenshot 2025-02-09 at 14 16 21"
src="https://github.com/user-
attachments/assets/887d218d-3db7-4a64-ad81-b7431adb23bb" />

Closes #947
2025-02-09 17:36:03 +02:00
meteorgan
a99d9a8988 chore: remove unused dependencies 2025-02-09 21:05:06 +08:00
Jussi Saurio
36a3cb1d5e sqlite3-parser: box AlterTable 2025-02-09 14:11:34 +02:00
Jussi Saurio
72a055e5fe sqlite3-parser: box Pragma 2025-02-09 13:10:52 +02:00
Jussi Saurio
23c4106433 sqlite3-parser: separate boxed Insert struct 2025-02-09 13:10:21 +02:00
Jussi Saurio
32887518ce sqlite3-parser: separate boxed Delete struct 2025-02-09 13:10:21 +02:00
Jussi Saurio
40a8dc14cd sqlite3-parser: separate boxed SelectInner struct 2025-02-09 12:54:30 +02:00
Jussi Saurio
358fda2ec7 sqlite3-parser: box the create table body 2025-02-09 12:42:53 +02:00
Pekka Enberg
e406a030e6 Merge 'Rework io_uring feature' from Jorge López Tello
This makes io_uring the default in CLI, but makes it non-default in
core. Before, if one built CLI without io_uring, core still built with
it as it was a default feature. To accommodate for the change, all
bindings have been updated to select the feature, except for WASM which
has a separate fs implementation.
This also adds some #[cfg] and #[allow] to silence unused-* warnings,
which I discovered when testing with different features disabled.

Closes #942
2025-02-09 08:46:29 +02:00
Pekka Enberg
3178e975b9 Merge 'perf/prepare: box many less frequently used AST nodes' from Jussi Saurio
`sqlite3-parser` spends a lot of time in `yy_reduce` assigning different
`enum YYMINORTYPE` members, and it suffers from bad performance because
the stack size of the enum is very large, and the size of individual
members varies wildly. This PR reduces the `YYMINORTYPE` stack size from
496 to 264 bytes and has the following effect:
Preparing statement:
```sql
Prepare `SELECT 1`/Limbo/SELECT 1
                        time:   [620.37 ns 621.08 ns 621.79 ns]
                        change: [-17.811% -17.582% -17.380%] (p = 0.00 < 0.05)
                        Performance has improved.

Prepare `SELECT * FROM users LIMIT 1`/Limbo/SELECT * FROM users LIMIT 1
                        time:   [1.2215 µs 1.2231 µs 1.2248 µs]
                        change: [-12.926% -12.627% -12.272%] (p = 0.00 < 0.05)
                        Performance has improved.

Benchmarking Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...: Collecting 100 samples in estimated 5.0152 s (
Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...
                        time:   [3.1056 µs 3.1096 µs 3.1138 µs]
                        change: [-13.279% -12.995% -12.712%] (p = 0.00 < 0.05)
                        Performance has improved.

```
Execute (mainly to check for regressions):
```sql
Execute `SELECT * FROM users LIMIT ?`/Limbo/1
                        time:   [402.19 ns 402.75 ns 403.36 ns]
                        change: [-3.4845% -2.4003% -1.6539%] (p = 0.00 < 0.05)
                        Performance has improved.

Execute `SELECT * FROM users LIMIT ?`/Limbo/10
                        time:   [2.7920 µs 2.7977 µs 2.8036 µs]
                        change: [-1.3135% -1.0123% -0.7132%] (p = 0.00 < 0.05)
                        Change within noise threshold.

Execute `SELECT * FROM users LIMIT ?`/Limbo/50
                        time:   [13.577 µs 13.633 µs 13.690 µs]
                        change: [-1.7709% -1.3575% -0.9563%] (p = 0.00 < 0.05)
                        Change within noise threshold.
```

Closes #936
2025-02-09 08:45:42 +02:00
Jorge López
83b158fb3a core: silence some unused warnings when building without default features 2025-02-09 01:13:12 +01:00
Jorge López
c07c08aa98 core: make io_uring feature non-default. All crates that depend on core select it by default. This enables us to build CLI without io_uring, which before this commit would still have used io_uring in core. 2025-02-09 01:12:27 +01:00
Jorge López
d7a6d48d8c core: make storage::wal::CheckPointMode public, because it is needed to implement trait Wal 2025-02-08 21:21:07 +01:00
Jorge López
9aedbf2d45 core: make storage:📟:PageRef public, because it is needed to implement trait Wal 2025-02-08 21:20:51 +01:00
Jorge López
8582a870fd core: make "result" module public, because it is needed to implement trait Wal, which is public 2025-02-08 21:12:56 +01:00
Jussi Saurio
781aa3b5d6 sqlite3-parser: box the having clause in GroupBy 2025-02-08 18:10:26 +02:00
Jussi Saurio
670dac5939 sqlite3-parser: box the where clause in Delete 2025-02-08 18:10:25 +02:00
Gustavo Sverzut Barbieri
cd2d817c10 github.com/penberg/limbo was moved to github.com/tursodatabase/limbo
Adjust all the references since they were broken
2025-02-08 10:21:49 -03:00
Pekka Enberg
71d2a9b6ab Merge 'Replace hashmap for io_uring pending ops with static array' from Preston Thorpe
Since we are already round robin'ing our `iovecs` in the same way, and
the map will never be larger than MAX_IOVECS, we can remove the cost of
hashing and keep o(1) lookups.

Closes #922
2025-02-08 11:15:54 +02:00
Pekka Enberg
b17610ab77 core/mvcc: Minor code cleanups
Make the source file readable from top to bottom by moving private
functions at the end of the struct implementation.
2025-02-08 10:55:13 +02:00
Jussi Saurio
791255fd8c MVCC: Add a few comments 2025-02-08 10:20:48 +02:00
Jussi Saurio
fab105c10c MVCC: fix write conflict handling 2025-02-08 10:10:09 +02:00
PThorpe92
575a524d04 Replace hashmap for io_uring pending ops with static array 2025-02-07 09:05:08 -05:00
Pekka Enberg
8c0c967ea2 Merge 'Implement json_quote' from Pedro Muniz
Hi! This is my first PR on the project, so I apologize if I did not
follow a convention from the project.
#127
This PR implements json_quote as specified in their source:
https://www.sqlite.org/json1.html#jquote. It follows the internal doc
guidelines for implementing functions. Most tests were added from sqlite
test suite for json_quote, while some others were added by me. Sqlite
test suite for json_quote depends on json_valid to test for correct
escape control characters, so that specific test at the moment cannot be
done the same way.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Reviewed-by: Sonny (@sonhmai)

Closes #763
2025-02-07 13:33:05 +02:00
Pekka Enberg
7169706809 Merge 'Add support for delete row' from Krishna Vishal
**Core delete tasks**:
- [x] Implement free page functionality
- [x] Clear overflow pages if any before deleting their cells using free
page.
- [x] Implement delete for leaf page
- [ ] Balance after delete properly
**Auxiliary tasks to make delete work properly**:
- [x] Implement block coalescing in `free_cell_range` to reduce
fragmentation.
- [x] Track page fragmentation in `free_cell_range`.
- [x] Update page offsets in `drop_cell` and update cell pointer array
after dropping a cell.
- [x] Add TCL tasks
Closes #455
--------
I will add support for balancing after delete once `balance_nonroot` is
extended. In the current state of `balance_nonroot` balancing won't work
after delete and corrupts page.
But delete itself is functional now.

Closes #785
2025-02-07 12:36:23 +02:00
Jussi Saurio
b37745ec14 Merge 'Add quickcheck tests for generate_series() and refine implementation' from Jussi Saurio
Adds `quickcheck` property based tests to the recently merged
`generate_series` implementation and fixes various issues in it:
- `0` for `step` should be treated as `1`
- missing `step` should be treated as `1`
- a string like `a` for `step` should be treated as `1`
- saturating overflow
- return an empty series for a variety of invalid inputs, instead of
erroring
For the future:
We should have e2e/fuzz tests comparing sqlite implementation to limbo
implementation

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #910
2025-02-07 10:40:25 +02:00
Pekka Enberg
7809df913a core/mvcc: Rename Database to MvStore 2025-02-07 07:40:33 +02:00
pedrocarlo
c3cad5dfdd corrected to use newly created as_str function to convert to string slice 2025-02-07 00:07:51 -03:00
pedrocarlo
8fe71309c0 cargo fmt 2025-02-06 23:46:00 -03:00
pedrocarlo
c8bb1fd353 unreachable to agg and record types, as it should not be possible to pass them to json_quote 2025-02-06 23:43:30 -03:00
pedrocarlo
b678375c69 increasing string capacity to reduce allocations 2025-02-06 23:37:30 -03:00
pedrocarlo
90ecaf40b5 removed unnecessary string allocations for escaped json value 2025-02-06 23:37:30 -03:00
pedrocarlo
26388cc802 fix: cargo fmt 2025-02-06 23:36:02 -03:00
pedrocarlo
303a687e65 rebase to main 2025-02-06 23:35:58 -03:00
Krishna Vishal
7cb7eb4e65 Merge branch 'main' into delete-btree-row 2025-02-07 01:15:21 +05:30
krishvishal
349fd51559 Get db_header from Pager instead of BTreeCursor.
Fix page_cache in btree tests to use `parking_lot::RwLock`
2025-02-07 00:11:06 +05:30
krishvishal
ea174e32f1 chore: cargo fmt 2025-02-06 23:39:12 +05:30
krishvishal
c80007d8a0 Add local constants for free_page offsets for clarity. 2025-02-06 23:39:12 +05:30
krishvishal
f836ad7561 Remove write to PAGE_HEADER_OFFSET_CELL_CONTENT_AREA in insert_into_cell. This causes corruption when you reuse cell spaces from free_list. 2025-02-06 23:39:12 +05:30
krishvishal
7d67895306 1. Added frangmentation handling to find_free_cell
2. Added asserts and corruption error to `allocate_cell_space`
2025-02-06 23:39:09 +05:30
krishvishal
8263bd0482 Update page offsets in drop_cell and update cell pointer array after dropping a cell. 2025-02-06 23:36:27 +05:30
krishvishal
73dae3003d Implement block coalescing in free_cell_range to reduce fragmentation and also added fragmentation tracking. 2025-02-06 23:36:26 +05:30
krishvishal
95da2244bb Implemented delete(). Now we are able to delete a row. It handles both interior page and leaf page cell differently. 2025-02-06 23:34:27 +05:30
krishvishal
f62ec61694 Added a unit-test to test the case where clear_overflow_pages function is called on cell with no overflow pages 2025-02-06 23:34:26 +05:30
krishvishal
8c763780a8 Added free list and free_page functionality in Pager module. Now we can add dropped pages to free list, so pages can be reused in allocate_page. 2025-02-06 23:34:12 +05:30
krishvishal
83b214a4e3 Added clear_over_pages it deletes all the overflow pages related to a cell. 2025-02-06 23:27:07 +05:30
Jussi Saurio
d5f58f5fea Add quickcheck tests for generate_series() and refine implementation 2025-02-06 18:36:21 +02:00
PThorpe92
ae88d51e6f Remove TableReferenceType enum to clean up planner 2025-02-06 09:15:39 -05:00