Commit Graph

1950 Commits

Author SHA1 Message Date
Pere Diaz Bou
bf37fd3314 wip 2025-03-29 22:02:49 +01:00
Pekka Enberg
4ee60348f2 Merge 'Fixes probably all floating point math issues and floating point display issues.' from Ihor Andrianov
Closes #1206
Closes #447
Closes #1117
Issues:
 1. Rust floating point math fucntions are non deterministic.
 2. SQLite have complex floating point display rules.
I changed rust functions to libm for math ops and implemented Display
trait for OwnedValue:Float. A lot of float formatting SQLite probably
inherits from C have to be handcrafted.

Closes #1208
2025-03-29 17:33:53 +02:00
Ihor Andrianov
09b55a9b41 fix nan to return null 2025-03-29 14:47:08 +02:00
Ihor Andrianov
8b9f34af71 fix tests and return nan as null 2025-03-29 14:46:11 +02:00
Ihor Andrianov
922945e819 all output should go through display 2025-03-29 12:46:06 +02:00
Ihor Andrianov
6e04709c4f add comprehensive float display trait 2025-03-29 12:44:34 +02:00
Ihor Andrianov
303f1b3749 replace std math functions with libm for compat 2025-03-29 12:16:14 +02:00
Pere Diaz Bou
c5b718ac32 fix review comments 2025-03-28 11:30:19 +01:00
Pere Diaz Bou
83d0c9a1b6 fix read overflow page procedure 2025-03-28 11:30:19 +01:00
Pere Diaz Bou
f51c20adf0 read overflow pages on demand 2025-03-28 11:12:43 +01:00
Pere Diaz Bou
dc8acf1a4a cell_get no allocations 2025-03-28 11:12:27 +01:00
Pekka Enberg
387b68fc06 Merge 'Expose 'Explain' to prepared statement to allow for alternate Writer ' from Preston Thorpe
### The problem:
I often need to copy the output of an `Explain` statement to my
clipboard. Currently this is not possible because it currently will only
write to stdout.
All other limbo output, I am able to run `.output file` in the CLI, then
enter my query and in another tmux pane I simply `cat file | xclip -in
-selection clipboard`.
### The solution:
Expose a `statement.explain()` method that returns the query explanation
as a string. If the user uses something like `execute` instead of
prepare, it will default to `stdout` as expected, but this allows the
user to access the query plan on the prepared statement and do with it
what they please.

Closes #1166
2025-03-28 09:55:58 +02:00
Pekka Enberg
9a646acead Merge 'Add BTree balancing after delete' from Krishna Vishal
This PR adds balancing after delete.
- [x] Remove linear search for cell in the page
- [x] Change the implementation to state machine approach
- [x] Handle cases when balancing is needed and not needed
- [x] Add unit test to verify that balancing after delete maintains
BTree integrity.
Fixes: https://github.com/tursodatabase/limbo/issues/1019
Closes: https://github.com/tursodatabase/limbo/issues/455

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1147
2025-03-28 07:32:15 +02:00
Pekka Enberg
41bdf25433 Merge 'Remove public unlock method from SpinLock to prevent unsafe aliasing' from Krishna Vishal
Currently we can create two guards and unlock one of them and still have
two mutable references to the same data.
By being able to unlock only via guard we prevent unsafe scenarios.
Currently, the test below will pass and shows that we can hold two
mutable references to the same data. After this fix, this would result
in a deadlock (have to remove `lock.unlock()`)
```rust
#[test]
fn two_mutable_reference() {
    let lock = SpinLock::new(42);
    let guard = lock.lock();
    lock.unlock();
    let guard2 = lock.lock();

    // two mutable references to same data
    *guard = 10;
    *guard2 = 20;

    assert_eq!(*guard, 20);
    assert_eq!(*guard2, 20);
}
```
Note: The javascript action failure is unrelated to this PR.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1194
2025-03-27 20:43:16 +02:00
Pere Diaz Bou
d01423df83 fix clippy 2025-03-27 17:54:32 +01:00
Pere Diaz Bou
9291f60722 Introduce Register struct
OwnedValue has become a powerhouse of madness, mainly because I decided
to do it like that when I first introduced AggContext. I decided it was
enough and I introduced a `Register` struct that contains `OwnedValue`,
`Record` and `Aggregation`, this way we don't use `OwnedValue` for
everything make everyone's life harder.

This is the next step towards making ImmutableRecords the default
because I want to remove unnecessary allocations. Right now we clone
OwnedValues when we generate a record more than needed.
2025-03-27 17:53:02 +01:00
krishvishal
dcd92954f4 Remove unlock method and move it to guard's drop.
This makes the interface safe and prevents unlocking while still holding the guard
2025-03-27 18:45:05 +05:30
Pekka Enberg
af6e9cd2c2 Merge 'Handle limit zero case in query plan emitter' from Preston Thorpe
closes #1190
Don't emit full query plan and open table if 0 `LIMIT`
```console
limbo> explain select * from t limit 0;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     2     0                    0   Start at 2
1     Halt               0     0     0                    0
2     Transaction        0     0     0                    0   write=false
3     Goto               0     1     0                    0
```
Surprisingly, sqlite will still emit the Open/Rewind/Column/etc, etc for
this case. Definitely feels super unnecessary.

Closes #1191
2025-03-27 08:59:54 +02:00
Pekka Enberg
ec742a8468 Merge 'Fix numeric conversion in SELECT -'e'' from Diego Reis
closes #1157

Closes #1167
2025-03-27 08:58:57 +02:00
Pekka Enberg
eb8866a106 Merge 'Reduce MVCC cursor memory consumption' from Ihor Andrianov
Fix memory hogging in MVCC scan cursor #1104
The current scan cursor loads all rowids at once, which blows up memory
on big tables.
Added BucketScanCursor that loads rowids in configurable batches -
better memory usage while keeping decent performance. It's a drop-in
replacement with the same interface.
Also included LazyScanCursor as an alternative that fetches one rowid at
a time, though it's less efficient due to log(n) skipmap lookups for
each row.
BucketScanCursor is the recommended approach for most use cases. WDYT?

Closes #1178
2025-03-27 08:56:35 +02:00
PThorpe92
a1ac0ca175 Handle limit zero case in query plan emitter 2025-03-26 15:33:05 -04:00
krishvishal
1b77b76d58 Add comments detailing Delete state machine flow and few delete steps. 2025-03-27 01:02:59 +05:30
krishvishal
fe7ff6f53d Add balancing states. Balancing test works. 2025-03-27 00:27:03 +05:30
Pekka Enberg
c1a0236dcc Merge 'Introduce immutable record' from Pere Diaz Bou
Currently we have a Record, which is a dumb vector of cloned values.
This is incredibly bad for performance as we do not want to clone
objects unless needed. Therefore, let's start by introducing this type
so that any record that has already been serialized will be returned
from btree in the format of a simple payload with reference to payload.

Closes #1176
2025-03-26 20:31:04 +02:00
krishvishal
3e3a0f56a1 Make delete re-entrant.
Setup DeleteState enum
2025-03-26 22:44:49 +05:30
Pere Diaz Bou
f07f10ac53 fix read empty blob/text 2025-03-26 18:11:44 +01:00
Pekka Enberg
114847415b Merge 'WAL frame checksum support' from Daniel Boll
closes #1151

Closes #1184
2025-03-26 17:48:52 +02:00
Daniel Boll
c2a2dfa67b Remove unused imports and handle WAL header read error
Refactor random number generation for WAL header salts
2025-03-26 11:31:29 -03:00
Pekka Enberg
fca643641c Merge 'Fix compute_shl negate with overflow' from Krishna Vishal
Fixes #1156
- Changed `compute_shl` implementation to handle negation with overflow.
- Added TCL tests.

Closes #1171
2025-03-26 16:08:59 +02:00
Pekka Enberg
3cb8cda746 Merge 'Unary + is a noop' from Levy A.
Fixes #1154.
Unary `+` is a noop in Sqlite: <https://sqlite.org/lang_expr.html#operat
ors_and_parse_affecting_attributes>.

Closes #1177
2025-03-26 15:59:45 +02:00
Pere Diaz Bou
63cf86ba36 fix comparison of records 2025-03-26 10:10:19 +01:00
krishvishal
16d77acac6 Add comments describing the test_delete_balancing 2025-03-26 12:29:19 +05:30
Daniel Boll
4ea3faf0f0 Remove unnecessary TODO comment in wal.rs 2025-03-25 21:46:17 -03:00
Daniel Boll
6d42d6d485 Remove commented-out code and update min_frame assignment 2025-03-25 21:44:18 -03:00
Daniel Boll
5fc9ccdc8c Update checkpoint result initialization and WAL frame handling
- Use `CheckpointResult::default()` instead of `CheckpointResult::new()`
- Correct WAL frame header salt and checksum handling
- Ensure frame ID is 1-based and adjust frame offset calculation
- Add `Default` implementation for `CheckpointResult`
- Use random values for WAL header salts
2025-03-25 21:38:12 -03:00
Pere Diaz Bou
8642d416c7 Introduce immutable record.
Currently we have a Record, which is a dumb vector of cloned values.
This is incredibly bad for performance as we do not want to clone
objects unless needed. Therefore, let's start by introducing this type
so that any record that has already been serialized will be returned
from btree in the format of a simple payload with reference to payload.
2025-03-25 17:35:41 +01:00
Pekka Enberg
79620946c1 Merge 'JSON cache' from Ihor Andrianov
SQLite uses a similar approach for operations where up to 4 JSON objects
are accessed multiple times in a single query.
`SELECT json_extact(a, 'some_path'), json_remove(a, 'some_path')
json_set(a, 'some_path', 'some_value') from t;`

Closes #1163
2025-03-25 18:11:33 +02:00
Ihor Andrianov
7c1d827d33 clippy 2025-03-25 17:13:31 +02:00
Ihor Andrianov
8bfacf3955 add lazy and bucket cursor 2025-03-25 16:55:29 +02:00
Levy A.
dd10fb13a7 fix: unary + is a noop 2025-03-25 11:43:19 -03:00
Pere Diaz Bou
004dc374b2 bump rusqlite to 0.34 2025-03-25 14:17:31 +01:00
Pekka Enberg
df6af6ed79 core: Rename FileStorage to DatabaseFile 2025-03-25 11:15:16 +02:00
krishvishal
785be8479f Fix a fuzzer failure and add tcl test covering the failure 2025-03-25 11:43:51 +05:30
krishvishal
b55dc586bd change compute_shl implementation to handle negation with overflow 2025-03-25 10:10:15 +05:30
krishvishal
6fe7e419e2 cleanup 2025-03-25 08:58:18 +05:30
krishvishal
ac29563efb Add test to verify balancing after maintains the following:
1. Delete is working properly.
2. Balancing after delete still maintains the validity of btree.
2025-03-25 08:55:13 +05:30
krishvishal
bf1718925e Delete with balancing now works!! 2025-03-25 08:38:18 +05:30
PThorpe92
e9420e7d2b Fix platform specific ffi c ptr types 2025-03-24 22:48:07 -04:00
Diego Reis
f499f756fb core/util: Fix invalid numeric parsing
To see details: https://github.com/tursodatabase/limbo/issues/1157
2025-03-24 20:21:09 -03:00
Diego Reis
5dba4999a7 core/util: Add unit tests for parse_numeric_str and fix whitespace handling 2025-03-24 19:56:13 -03:00