Commit Graph

373 Commits

Author SHA1 Message Date
Zaid Humayun
dc2bb7cb9b DropTable: implementation complete
added helper methods to Schema to remove table and indices from in-memory structures
completed the implementation for DropTable using that
2025-02-19 21:46:26 +05:30
Zaid Humayun
40b08c559c vdbe: modified instruction DropTable
the instruction DropTable has been modified to correctly match SQLite semantics
2025-02-19 21:46:26 +05:30
Zaid Humayun
97d87955cc DROP TABLE: renamed BTreeCusor::btree_drop to BTreeCursor::btree_destroy
this more closely matches semantics
2025-02-19 21:46:26 +05:30
Zaid Humayun
713465c592 instruction: added destroy instruction
added required functionality for destroy. minor cleanup of print statements
2025-02-19 21:46:25 +05:30
Zaid Humayun
b8bebf3fa3 translate: updated the command to more closely match SQLite semantics
the command for drop table translation has been updated so that it more closely matches the semantics of SQLite's drop table command.

there are a few more things missing like ephemeral tables, destroy etc.
2025-02-19 21:46:25 +05:30
Zaid Humayun
76e2d98381 drop table: addresses issue https://github.com/tursodatabase/limbo/issues/894 which requires DROP TABLE to be implemented
this is the initial commit is for the implementation of DROP TABLE. It adds support for the DROP TABLE instruction and adds a DropBTree instruction. It also implements the btree_drop method in btree.rs which makes use of free_page method which will be implemented via PR https://github.com/tursodatabase/limbo/pull/785
2025-02-19 21:46:25 +05:30
Pekka Enberg
98010517d5 Merge 'Add affinity() function to Column' from Jussi Saurio
This is a first step to supporting [sqlite column
affinity](https://www.sqlite.org/datatype3.html) properly in limbo --
just adds an `affinity()` function to `Column` that can then be used
elsewhere

Closes #1030
2025-02-18 17:44:38 +02:00
Jussi Saurio
baf2aec3e9 Fix incorrect CAST text->numeric if valid prefix is 1 char long 2025-02-18 15:07:36 +02:00
Jussi Saurio
89e48a16db Add affinity() function to Column 2025-02-18 10:56:30 +02:00
Jussi Saurio
cbfd77849d Merge 'Fix substr' from Nikita Sivukhin
Align `substr` implementation with SQLite spec
(https://www.sqlite.org/lang_corefunc.html#substr):
> The substr(X,Y,Z) function returns a substring of input string X that
begins with the Y-th character and which is Z characters long. If Z is
omitted then substr(X,Y) returns all characters through the end of the
string X beginning with the Y-th. The left-most character of X is number
1. If Y is negative then the first character of the substring is found
by counting from the right rather than the left. If Z is negative then
the abs(Z) characters preceding the Y-th character are returned. If X is
a string then characters indices refer to actual UTF-8 characters. If X
is a BLOB then the indices refer to bytes.

Reviewed-by: Jussi Saurio (@jussisaurio)

Closes #1013
2025-02-15 18:16:57 +02:00
Jussi Saurio
e4541edb48 Fix IdxGt,IdxGe,IdxLt,IdxLe instructions
According to SQLite documentation, the way to use these instructions
is to compare the seek key to the index key as you would with the
Compare opcode. The compare opcode states:

"Compare two vectors of registers in reg(P1)..reg(P1+P3-1)
(call this vector "A") and in reg(P2)..reg(P2+P3-1) ("B")."

In other words, we should compare the same number of columns from each,
not compare the entire keys.

This fixes a few Clickbench queries returning incorrect results, and
so closes #1009

---

Future work: support index seek keys that use multiple columns. Our
index seek is many times slower than SQLite because we're not utilizing
all the possible columns -- instead we just use the first index column
to seek.
2025-02-15 12:44:56 +02:00
Nikita Sivukhin
91d723016d fix test 2025-02-15 13:29:14 +04:00
Nikita Sivukhin
b35dab5b6d fix substr implementation 2025-02-15 13:21:43 +04:00
[B
5214cf9859 Added IdxLE and IdxLT opcodes 2025-02-14 20:22:30 +01:00
Pekka Enberg
e3925c4531 core/vdbe: Fix Program::halt() to respect autocommit state 2025-02-14 12:11:37 +02:00
Pekka Enberg
ae3c6b7ec5 core/vdbe: Fix AutoCommit instruction to halt the VM
Pointed out by Jussi
2025-02-14 11:43:16 +02:00
Pekka Enberg
948585bb42 core/vdbe: Extract Program::halt() helper
We need this for AutoCommit opcode too.
2025-02-14 11:42:24 +02:00
Pekka Enberg
34b0c7c09a core/vdbe: AutoCommit instruction 2025-02-14 10:26:31 +02:00
Doug Anderson444
0aa5de6d9f rm log, switch all to tracing 2025-02-11 09:03:36 -04:00
Pekka Enberg
d221f158cc Merge 'Add read implementation of user_version pragma with ReadCookie opcode' from Jonathan Webb
Just a bare bones implementation of ReadCookie and support for the
user_version pragma

Closes #909
2025-02-10 12:12:15 +02:00
Pekka Enberg
604ca4085d Merge 'Make record values private' from Tiago
This is an attempt to move towards #881. I am not sure this is the
direction you want to take. In any case, I thought I would take a crack
at converting `values` from `Record` to private and see how bad it would
be.
In the end, as you can see, it is not so bad. I think performance-wise
it shouldn't be a bad hit with Rust's zero-cost abstraction. Also,
during the process I noticed a couple improvements that could be made
here and there but I honestly wanted to start with something small
enough that wouldn't be too hard to review.
Anyway, let me know if this is really how you would like to proceed.

Closes #962
2025-02-10 11:23:21 +02:00
Pekka Enberg
fcad8d125e Merge 'refactor: remove RC<String> requirement for build_text and new text' from Pedro Muniz
This PR aims to simplify text creation and to reduce allocation overhead
of creating a new OwnedValue::Text. Instead of creating Rc<String> every
time you need to create a text, you just pass the string slice and the
Rc<String> is created at the end. This change, at least on my machine,
has removed a lot of variability in the benchmarking performance, while
maintaining roughly the same performance.

Closes #961
2025-02-10 11:21:50 +02:00
Pekka Enberg
00a546e775 Merge 'Fix string funcs' from Nikita Sivukhin
Add fuzz test for string functions and fix 2 bugs in implementation of
`LTRIM/RTRIM/TRIM` and `QUOTE`:
- `QUOTE` needs to escape internal quote(`'`) symbols
- `LTRIM`/`RTRIM`/`TRIM` needs to check if they have additional argument

Closes #958
2025-02-10 11:17:51 +02:00
Pekka Enberg
55058cade3 Merge 'Fix cast' from Nikita Sivukhin
Fix codegen for `CAST` expression and also adjust implementation of
`CAST: Real -> Int` because `SQLite` use "closest integer between the
REAL value and zero" as a CAST result.

Closes #956
2025-02-10 10:53:56 +02:00
Tiago Ribeiro
295691d81b Update core/vdbe to use the new retrieval methods to access Record values. 2025-02-10 00:27:51 -07:00
Nikita Sivukhin
7cdad64a5f fix assertion in test_quote 2025-02-09 23:50:11 +04:00
Nikita Sivukhin
cd4bfac059 fix quote functions 2025-02-09 23:43:34 +04:00
Nikita Sivukhin
cc7267b0bd fix trim function pattern handling 2025-02-09 23:43:34 +04:00
Nikita Sivukhin
a37b74666f remove unnecessary floor() 2025-02-09 22:41:08 +04:00
Nikita Sivukhin
3c4d9a93af fix rounding of REAL to INTEGER
- SQLite rounds (x: f64) to the nearest number between x and 0 - so
  basically truncates the x
2025-02-09 22:32:54 +04:00
pedrocarlo
fe453ecfc5 remove RC<String> requirement for build_text and new text 2025-02-09 13:44:39 -03:00
Aarni Koskela
eaea02c567 Fix a handful of typos 2025-02-09 18:08:29 +02:00
Jonathan Webb
98e9d33478 Add read implementation of user_version pragma with ReadCookie opcode 2025-02-07 09:23:48 -05:00
Pekka Enberg
8c0c967ea2 Merge 'Implement json_quote' from Pedro Muniz
Hi! This is my first PR on the project, so I apologize if I did not
follow a convention from the project.
#127
This PR implements json_quote as specified in their source:
https://www.sqlite.org/json1.html#jquote. It follows the internal doc
guidelines for implementing functions. Most tests were added from sqlite
test suite for json_quote, while some others were added by me. Sqlite
test suite for json_quote depends on json_valid to test for correct
escape control characters, so that specific test at the moment cannot be
done the same way.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Reviewed-by: Sonny (@sonhmai)

Closes #763
2025-02-07 13:33:05 +02:00
pedrocarlo
303a687e65 rebase to main 2025-02-06 23:35:58 -03:00
Jussi Saurio
d5f58f5fea Add quickcheck tests for generate_series() and refine implementation 2025-02-06 18:36:21 +02:00
PThorpe92
661c74e338 Apply new planner structure to virtual table impl 2025-02-06 09:15:28 -05:00
Jussi Saurio
f5f77c0bd1 Initial virtual table implementation 2025-02-06 07:51:50 -05:00
Pekka Enberg
f4a574e6bc core: Move strftime to functions module 2025-02-06 13:53:36 +02:00
Pekka Enberg
ee8eabf167 core: Move datetime to functions module 2025-02-06 13:52:25 +02:00
Pekka Enberg
7513f859df core: Move printf to functions module 2025-02-06 13:50:05 +02:00
Pekka Enberg
f3902ef9b6 core: Rename OwnedRecord to Record
We only have one record type so let's call it `Record`.
2025-02-06 13:40:34 +02:00
Pekka Enberg
f9828e0e6f core: Parse UTF-8 strings lazily 2025-02-06 13:27:52 +02:00
Pekka Enberg
c210821100 core: Move result row to ProgramState
Move result row to `ProgramState` to mimic what SQLite does where `Vdbe`
struct has a `pResultRow` member. This makes it easier to deal with result
lifetime, but more importantly, eventually lazily parse values at the edges of
the API.
2025-02-06 11:52:26 +02:00
krishvishal
32080aba5d Make vector function accessible through Function op code. 2025-02-06 07:01:50 +05:30
Pekka Enberg
b5f5e40986 Merge 'prepare perf: dont eagerly allocate result column name strings' from Jussi Saurio
- Remove eagerly allocated `name` from `ResultSetColumn`
- `ResultSetColumn` can calculate `name()` on demand:
    - if it has an alias (`foo as bar`), use that
    - if it is a column reference, use that
    - otherwise return none, and callers can assign it a placeholder
name (like `column_1`)
- move the `plan.result_columns` and `plan.table_references` to
`Program` after preparing statement is done, so that column names can be
returned upon request
- make `name` in `Column` optional, not needed for pseudo tables and
sorters so avoids an extra string allocation
```sql
Prepare `SELECT 1`/Limbo/SELECT 1
                        time:   [756.80 ns 758.27 ns 760.04 ns]
                        change: [-3.3257% -3.0252% -2.7035%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  2 (2.00%) low severe
  3 (3.00%) low mild
  1 (1.00%) high mild
  2 (2.00%) high severe

Prepare `SELECT * FROM users LIMIT 1`/Limbo/SELECT * FROM users LIMIT 1
                        time:   [1.4646 µs 1.4669 µs 1.4696 µs]
                        change: [-6.4769% -6.2021% -5.9137%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) low severe
  3 (3.00%) low mild
  3 (3.00%) high severe

Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...`
                        time:   [3.7256 µs 3.7311 µs 3.7376 µs]
                        change: [-4.5195% -4.2192% -3.9309%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) low severe
  2 (2.00%) low mild
  2 (2.00%) high mild
```

Closes #898
2025-02-05 18:20:01 +02:00
Jussi Saurio
795576b2ec dont eagerly allocate result column name strings 2025-02-05 17:53:23 +02:00
Pekka Enberg
56d401fb67 Merge 'Implement json_set' from Marcus Nilsson
This PR adds support for `json_set`.
There are three helper functions added:
1. `json_path_from_owned_value`, this function turns an `OwnedValue`
into a `JsonPath`.
2. `find_or_create_target`, this function is similar to `find_target`
with the added bonus of creating the target if it doesn't exist. There
is a caveat with this function and that is that it will create
objects/arrays as it goes, meaning if you send `{}` into it and try
getting the path `$.some.nested.array[123].field`, it will return
`{"some":{"nested":array:[]}}` since creation of `some`, `nested` and
`array` will succeed, but accessing element `123` will fail.
3. `create_and_mutate_json_by_path`, this function is very similar to
`mutate_json_by_path` but calls `find_or_create_target` instead of
`find_target`

Related to #127

Closes #878
2025-02-05 14:15:02 +02:00
Pekka Enberg
9fdf54de2b Merge 'Small perf optimizations to statement preparation' from Jussi Saurio
```bash
Prepare `SELECT 1`/Limbo/SELECT 1
                        time:   [765.94 ns 768.26 ns 771.03 ns]
                        change: [-7.8340% -7.4887% -7.1406%] (p = 0.00 < 0.05)
                        Performance has improved.

Prepare `SELECT * FROM users LIMIT 1`/Limbo/SELECT * FROM users LIMIT 1
                        time:   [1.5673 µs 1.5699 µs 1.5731 µs]
                        change: [-10.810% -9.7122% -8.4951%] (p = 0.00 < 0.05)
                        Performance has improved.

Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...
                        time:   [4.1331 µs 4.1421 µs 4.1513 µs]
                        change: [-9.3157% -9.0255% -8.7372%] (p = 0.00 < 0.05)
                        Performance has improved.
```
flamegraph for prepare `SELECT 1`:
<img width="1718" alt="Screenshot 2025-02-03 at 10 34 14"
src="https://github.com/user-
attachments/assets/ba67fe2f-78b2-4796-9a09-837d8e79fe62" />

Closes #872
2025-02-05 10:46:57 +02:00
Pekka Enberg
0b0681c9f8 core/vdbe: Lazy cursor borrowing
This saves a few more nanoseconds:

```
Execute `SELECT 1`/Limbo
                        time:   [44.964 ns 45.064 ns 45.160 ns]
                        change: [-14.371% -13.724% -13.214%] (p = 0.00 < 0.05)
                        Performance has improved.
```
2025-02-05 09:47:17 +02:00