Commit Graph

5233 Commits

Author SHA1 Message Date
Pekka Enberg
6258dda5f1 core/json: Rename json_path.rs to path.rs 2025-03-31 09:40:01 +03:00
Pekka Enberg
4a91988755 core/json: Rename json_operations.rs to ops.rs 2025-03-31 09:33:30 +03:00
Pekka Enberg
4b77f52bac core/json: Rename json_cache.rs to cache.rs 2025-03-31 09:32:34 +03:00
Pekka Enberg
b8760c1cb0 Merge 'Jsonb support update' from Ihor Andrianov
1. Removed dependecy on serde, now all json functions use jsonb
implementation.
2. Add support for jsonb_patch.
3. Aggregation json functions like json_group_array, json_group_object.

Closes #1207
2025-03-31 09:29:28 +03:00
Jussi Saurio
42e25d23dd Fix IdxCmp insn comparisons 2025-03-30 23:01:41 +03:00
PThorpe92
211c9a0212 Remove From impl on iteration direction for sort order 2025-03-30 12:18:12 -04:00
PThorpe92
516e443a2b Fix use index cursor id in emitter and prevent reinsert pk on update 2025-03-30 12:15:25 -04:00
PThorpe92
43d3900777 Remove update limit tests from compat tcl tests 2025-03-30 12:15:25 -04:00
PThorpe92
7b25895940 Add update query tests with limit 2025-03-30 12:15:25 -04:00
PThorpe92
ff02d74afb Fix update queries when limit is 0 2025-03-30 12:15:25 -04:00
PThorpe92
a88ce2a4b7 Correct comment on update plan 2025-03-30 12:15:25 -04:00
PThorpe92
3f4636196c Fix error message for non btree table 2025-03-30 12:15:25 -04:00
PThorpe92
bbbd1df1ab Replace unwrap in update translation with parse error 2025-03-30 12:15:24 -04:00
PThorpe92
4ac0781991 Add update tcl tests for LIMIT clauses on update queries 2025-03-30 12:15:24 -04:00
PThorpe92
7486149643 Support LIMIT clause on update queries 2025-03-30 12:15:24 -04:00
PThorpe92
b7fca31ef6 Add comments and impl Copy on iterdir type 2025-03-30 12:15:24 -04:00
PThorpe92
3fe14f37a5 Create plan for Update queries 2025-03-30 12:15:24 -04:00
Ihor Andrianov
a234aa3647 remove vec cloning from json agg functions 2025-03-30 19:10:15 +03:00
Ihor Andrianov
40bb867d54 clippy 2025-03-30 19:01:16 +03:00
Ihor Andrianov
db5e364210 made json an optional module again 2025-03-30 19:01:03 +03:00
Ihor Andrianov
6c126dcd97 add jsonb_set 2025-03-30 18:58:40 +03:00
Ihor Andrianov
dba82b40e3 add compat tests 2025-03-30 18:58:40 +03:00
Ihor Andrianov
92a745ca49 fix tests 2025-03-30 18:58:39 +03:00
Ihor Andrianov
101dd51d7c add jsonb_group_object and array 2025-03-30 18:58:39 +03:00
Ihor Andrianov
35e1098314 update compat for json functions 2025-03-30 18:58:38 +03:00
Ihor Andrianov
c426c13763 make tests pass 2025-03-30 18:58:38 +03:00
Ihor Andrianov
568dc54b9e big cleanup 2025-03-30 18:58:33 +03:00
Ihor Andrianov
a983c979c6 jsonb_merge, json_group_array, json_group_object 2025-03-30 18:47:33 +03:00
Pekka Enberg
efd537dc20 Merge 'Allocation improvements with ImmutableRecord, OwnedRecord and read_record' from Pere Diaz Bou
This pr is huge again but I will try to introduce each improvement one
by one.
## Overview
### Remove Rc for Text and Blob.
In general copying is bad, that's why we hid it with `Rc`s. With the
introduction of `ImmutableRecord` we make it less relevant because now
we will copy only once anyways, no other place should copy it so we can
avoid using `Rc`. If we we were to copy it it most likely means where
are doing something wrong.
### Reuse `Text` and `Blob` OwnedValues.
Most of the queries spend time overwriting the same register over and
over. What about we don't allocate new `OwnedValue` and we just simply
reuse the `OwnedValue` and extend the internal buffer. That's what I did
and it worked quite nicely.
### Make `Register::Record` be `ImmutableRecord`
`ImmutableRecord` basically means "serialized record", that's why all
the data is contained in a single payload buffer. There is a list of
values to reference that payload to reduce time complexity of search --
there is an argument to make a record without this vec to reduce memory
footprint.  This improvement I don't think it had a direct impact on
performance but it is a simpler way to lay the memory without any
complicated reference counted pointers, and instead we use a contiguous
piece of memory.
### Make `ImmutableRecord` reusable in `BTreeCursor`.
`BTreeCursor` allocated and deallocated records when it needed a new
one. This is obviously a big waste because we could be reusing the
internal buffer to avoid allocations. `ImmutableRecord` proved to be
useful here because now, we will only store a single `ImmutableRecord`
in the cursor that we will never deallocate -- we will just reallocate
when needed and replace the current one with the next one on demand.
## Return `Row` as a reference of Registers.
A `ResultRow` bytecode takes care of gathering all the columns of a row
and returning them to the user. Previously we could create a new
`Record` struct with all the cloned values which proved to be wasteful.
SQLite is smart about this so we must be as well. Basically a row now is
a wrapper for `struct Row { *const Register, count: usize }`, and we
basically include some QOL methods to avoid using pointers directly.
I know pointers are unsafe. That's why this row will be invalidate on
the next step of the VM and this row should be not used outside there.
### Inlining go brrr
`read_varint` and `read_value` are called in a tight loop making it easy
to see overhead of the call stack. That's why I sprinkled some
`#[inline(always)]` and saw something like a 15% speed boost.
## read_record with custom `SmallVec<T>`
We tend to overuse vectors for everything, this is quite bad because it
requires heap memory allocations. We can avoid this with a simple
`SmallVec` that simply fallsback to a vec with more complex scenarios.
## Benchmarks!
```
### before
fun/limbo » cargo bench -- limbo_execute 2>&1 | grep -B 1 "time: " | tee out.log
Execute `SELECT 1`/limbo_execute_select_1
                        time:   [43.958 ns 44.056 ns 44.154 ns]
--
Execute `SELECT * FROM users LIMIT ?`/limbo_execute_select_rows/1
                        time:   [407.82 ns 408.57 ns 409.41 ns]
--
Execute `SELECT * FROM users LIMIT ?`/limbo_execute_select_rows/10
                        time:   [2.7335 µs 2.7386 µs 2.7443 µs]
--
Execute `SELECT * FROM users LIMIT ?`/limbo_execute_select_rows/50
                        time:   [13.451 µs 13.485 µs 13.520 µs]
--
Execute `SELECT * FROM users LIMIT ?`/limbo_execute_select_rows/100
                        time:   [26.967 µs 27.077 µs 27.201 µs]after:
```
### after
```
fun/limbo (more-register) » cargo bench -- limbo_execute 2>&1 | grep -B 1 "time: " | tee out.log                                                                                                                                                                                                                                                        130 ↵
Execute `SELECT 1`/limbo_execute_select_1
                        time:   [33.386 ns 33.440 ns 33.510 ns]
--
Execute `SELECT * FROM users LIMIT ?`/limbo_execute_select_rows/1
                        time:   [326.79 ns 327.37 ns 328.03 ns]
--
Execute `SELECT * FROM users LIMIT ?`/limbo_execute_select_rows/10
                        time:   [1.5817 µs 1.5849 µs 1.5889 µs]
--
Execute `SELECT * FROM users LIMIT ?`/limbo_execute_select_rows/50
                        time:   [7.3295 µs 7.3531 µs 7.3829 µs]
--
Execute `SELECT * FROM users LIMIT ?`/limbo_execute_select_rows/100
                        time:   [14.538 µs 14.570 µs 14.606 µs]
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1197
2025-03-30 13:17:16 +03:00
Anton Harniakou
9a61c75ea1 Let remainder (%) accept textual arguments; add more tests for core/vdbe/insn.rs
Tests for exec_add, exec_subtract, exec_multiply, exec_divide,
exec_remainder
2025-03-30 12:14:05 +03:00
Pere Diaz Bou
578bc9e3e6 extract constant min_header_size 2025-03-30 11:12:11 +02:00
Pere Diaz Bou
8d74f4b8ab remove unnecessary partial ord 2025-03-30 11:07:23 +02:00
Pere Diaz Bou
3899f8ca17 comment header size 2025-03-30 11:03:45 +02:00
Pere Diaz Bou
541b67bd2b rename get_lazy_immutable_record -> get_immutable_record_or_create 2025-03-30 11:00:59 +02:00
Pere Diaz Bou
6ccb2e16d1 safer api for ImmutableRecord recreation 2025-03-30 11:00:13 +02:00
Pere Diaz Bou
f2f6173670 assert capacity didn't change 2025-03-30 10:37:58 +02:00
Pere Diaz Bou
3ac1795c25 fix from_register serialization 2025-03-30 10:31:39 +02:00
pedrocarlo
14ed8c8c50 correct vfs command 2025-03-30 03:22:44 -03:00
pedrocarlo
a612ad193c accidently removed an Error string 2025-03-30 03:12:36 -03:00
pedrocarlo
d1ded8e069 removing accidental sqlite file in repo 2025-03-30 02:58:53 -03:00
pedrocarlo
c36dc61c92 adding listvfs to extra helpful message 2025-03-30 02:52:07 -03:00
pedrocarlo
62b866618b docs for cli 2025-03-30 02:41:48 -03:00
pedrocarlo
b7bbafd691 adjusting listvfs command 2025-03-30 01:52:16 -03:00
pedrocarlo
82db51b94f remove debug clap 2025-03-30 01:46:24 -03:00
pedrocarlo
fe25035c7c display name in clap is buggy 2025-03-30 01:46:24 -03:00
pedrocarlo
02c466cb1f start of refactor of repl to use clap 2025-03-30 01:44:58 -03:00
Pere Diaz Bou
587cdac2c1 ignore sequential write beause it takes too long 2025-03-29 22:26:29 +01:00
Pere Diaz Bou
37ddf0946f rever testing.db change 2025-03-29 22:09:53 +01:00
Pere Diaz Bou
a13b33fec9 clippy again 2025-03-29 22:07:43 +01:00
Pere Diaz Bou
d9f5cd870d clippy 2025-03-29 22:04:08 +01:00