Commit Graph

458 Commits

Author SHA1 Message Date
Jussi Saurio
c19e4fc69c Merge 'Aggregation without group by produces incorrect results for scalars' from Ihor Andrianov
Closes #954
Before:
<img width="669" alt="Знімок екрана 2025-03-27 о 21 49 19"
src="https://github.com/user-
attachments/assets/d005e690-7dab-41e5-bc03-b574cade3965" />
After:
<img width="676" alt="Знімок екрана 2025-03-27 о 21 49 44"
src="https://github.com/user-
attachments/assets/1f4eb6bf-a238-496e-9fa4-32382799ef86" />
SQLite:
<img width="656" alt="Знімок екрана 2025-03-27 о 21 50 04"
src="https://github.com/user-
attachments/assets/3eca184e-6ea5-47c1-824f-51d11256a7af" />

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1198
2025-04-06 20:00:30 +03:00
Preston Thorpe
62e0a137dd Merge branch 'main' into update_offset 2025-04-06 12:34:23 -04:00
Pekka Enberg
2d3fd01f91 Merge 'Support Create Index' from Preston Thorpe
Closes #1193
```console
│limbo> explain create index idxp on products(price);
│addr  opcode             p1    p2    p3    p4             p5  comment
│----  -----------------  ----  ----  ----  -------------  --  -------
│0     Init               0     39    0                    0   Start at 39
│1     CreateBtree        0     1     2                    0   r[1]=root iDb=0 flags=2
│2     OpenWriteAsync     0     1     0                    0
│3     OpenWriteAwait     0     0     0                    0
│4     NewRowId           0     2     0                    0
│5     String8            0     3     0     index          0   r[3]='index'
│6     String8            0     4     0     idxp           0   r[4]='idxp'
│7     String8            0     5     0     products       0   r[5]='products'
│8     Copy               1     6     1                    0   r[6]=r[1]
│9     String8            0     7     0     CREATE INDEX idxp ON products (price)  0   r[7]='CREATE INDEX idxp ON products (price)'
│10    MakeRecord         3     5     8                    0   r[8]=mkrec(r[3..7])
│11    InsertAsync        0     8     2                    0
│12    InsertAwait        0     0     0                    0
│13    SorterOpen         3     1     0     k(1,B)         0   cursor=3
│14    OpenPseudo         4     9     2                    0   2 columns in r[9]
│15    OpenReadAsync      2     273   0                    0   table=products, root=273
│16    OpenReadAwait      0     0     0                    0
│17    RewindAsync        2     0     0                    0
│18    RewindAwait        2     25    0                    0   Rewind table products
│19      Column           2     2     10                   0   r[10]=products.price
│20      RowId            2     11    0                    0   r[11]=products.rowid
│21      MakeRecord       10    2     12                   0   r[12]=mkrec(r[10..11])
│22      SorterInsert     3     12    0     0              0   key=r[12]
│23    NextAsync          2     0     0                    0
│24    NextAwait          2     19    0                    0
│25    OpenWriteAsync     1     1     0                    0
│26    OpenWriteAwait     0     0     0                    0
│27    SorterSort         3     33    0                    0
│28      SorterData       3     13    4                    0   r[13]=data
│29      SeekEnd          1     0     0                    0
│30      IdxInsertAsync   1     13    0                    0   key=r[13]
│31      IdxInsertAwait   1     0     0                    0
│32    SorterNext         3     28    0                    0
│33    Close              3     0     0                    0
│34    Close              2     0     0                    0
│35    Close              1     0     0                    0
│36    ParseSchema        0     0     0     name = 'idxp' AND type = 'index'  0   name = 'idxp' AND type = 'index'
│37    Close              0     0     0                    0
│38    Halt               0     0     0                    0
│39    Transaction        0     1     0                    0   write=true
│40    Goto               0     1     0                    0
```
This will create the initial index btree and insert whatever relevant
records that need to be inserted, it doesn't handle the case of
inserting new index keys when normal records are created afterwards.
That will prob be added in next PR to keep this one concise.
Limbo will properly use the index in a subsequent query:
![image](https://github.com/user-
attachments/assets/eb41e985-4a70-49a5-8218-62c25e4d16c5)
Creating a unique index on a column that has 2 existing identical rows:
![image](https://github.com/user-
attachments/assets/ea46c720-5235-4451-81f0-25497ed9ee92)

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1199
2025-04-06 19:32:36 +03:00
PThorpe92
ff482fc753 Add test case for insert default values 2025-04-05 21:34:27 -04:00
PThorpe92
399994bf66 Fix ext tests start with no default schema 2025-04-05 11:19:40 -04:00
PThorpe92
c6c3f39959 Add test cases in python CLI tests for OFFSET on LIMIT clauses 2025-04-04 12:49:12 -04:00
Pekka Enberg
c3eaf47180 Merge 'Add support for default values in INSERT statements' from Diego Reis
While working on #494 I noticed that default values defined in schemas
weren't being applied.
Before:
![image](https://github.com/user-attachments/assets/af3a31be-
accd-47b4-848f-3820497e32ac)
Now:
![image](https://github.com/user-
attachments/assets/0ccb194c-4caa-481d-8f47-2405c3db06bb)

Closes #1249
2025-04-04 08:59:44 +03:00
Diego Reis
43daba9942 core/translate: Add support for default values in INSERT statements 2025-04-04 01:32:13 -03:00
PThorpe92
97c68f905a Move test back to original setup to ensure issue is solved 2025-04-03 20:22:47 -04:00
Ihor Andrianov
5632b15a44 add tests 2025-04-03 22:28:13 +03:00
PThorpe92
22fd3e9781 Fix cli tests 2025-04-03 15:23:49 -04:00
Pekka Enberg
6199c3994a Merge 'Create plan for Update queries' from Preston Thorpe
closes #1186, or at least works towards it by implementing an actual
Plan for update queries instead of translating everything inline. This
allows for actually using index's, rewriting const expressions, pushing
predicates instead of hardcoding a full scan in the translation.
### TODOs:
1.  `RETURNING` clause/result columns
2.  `OFFSET` clauses
3. on conflict
### LIMIT:
By supporting `LIMIT` directly in update queries, we'll have to put the
tests outside of the compatibility tests, maybe in the CLI tests.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1189
2025-04-01 17:33:32 +03:00
PThorpe92
43d3900777 Remove update limit tests from compat tcl tests 2025-03-30 12:15:25 -04:00
PThorpe92
7b25895940 Add update query tests with limit 2025-03-30 12:15:25 -04:00
PThorpe92
4ac0781991 Add update tcl tests for LIMIT clauses on update queries 2025-03-30 12:15:24 -04:00
Ihor Andrianov
dba82b40e3 add compat tests 2025-03-30 18:58:40 +03:00
Ihor Andrianov
c426c13763 make tests pass 2025-03-30 18:58:38 +03:00
Pekka Enberg
4ee60348f2 Merge 'Fixes probably all floating point math issues and floating point display issues.' from Ihor Andrianov
Closes #1206
Closes #447
Closes #1117
Issues:
 1. Rust floating point math fucntions are non deterministic.
 2. SQLite have complex floating point display rules.
I changed rust functions to libm for math ops and implemented Display
trait for OwnedValue:Float. A lot of float formatting SQLite probably
inherits from C have to be handcrafted.

Closes #1208
2025-03-29 17:33:53 +02:00
Ihor Andrianov
8b9f34af71 fix tests and return nan as null 2025-03-29 14:46:11 +02:00
Pere Diaz Bou
cb85ba8e82 fix extensions.py test 2025-03-28 11:58:03 +01:00
Pekka Enberg
ec742a8468 Merge 'Fix numeric conversion in SELECT -'e'' from Diego Reis
closes #1157

Closes #1167
2025-03-27 08:58:57 +02:00
Pekka Enberg
fca643641c Merge 'Fix compute_shl negate with overflow' from Krishna Vishal
Fixes #1156
- Changed `compute_shl` implementation to handle negation with overflow.
- Added TCL tests.

Closes #1171
2025-03-26 16:08:59 +02:00
Levy A.
5eae685fa8 add tests 2025-03-26 07:04:03 -03:00
krishvishal
1660ae5542 missed adding _ and a space. 2025-03-25 12:04:48 +05:30
krishvishal
785be8479f Fix a fuzzer failure and add tcl test covering the failure 2025-03-25 11:43:51 +05:30
krishvishal
f12e3a6993 For a few TCL tests more. 2025-03-25 10:28:48 +05:30
krishvishal
a8129d5e58 Add TCL tests for compute_shl 2025-03-25 10:26:08 +05:30
Diego Reis
f499f756fb core/util: Fix invalid numeric parsing
To see details: https://github.com/tursodatabase/limbo/issues/1157
2025-03-24 20:21:09 -03:00
Pekka Enberg
65bf33023c core: Fix Destroy opcode root page handling
The `p1` register points to the root page, not to a cursor.

Fixes #1136
2025-03-24 10:54:49 +02:00
Pekka Enberg
31bbc5144a Merge 'Initial pass at UPDATE support' from Preston Thorpe
This PR is to support `Update` queries. Follows sqlite behavior as much
as possible.
### limbo
```console
limbo> create table t (a,b,c);
limbo> explain update t set a = 1 where b = 2;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     18    0                    0   Start at 18
1     OpenWriteAsync     0     2     0                    0
2     OpenWriteAwait     0     0     0                    0
3     RewindAsync        0     0     0                    0
4     RewindAwait        0     17    0                    0   Rewind table t
5       Column           0     1     4                    0   r[4]=t.b
6       Ne               4     5     15                   0   if r[4]!=r[5] goto 15
7       RowId            0     6     0                    0   r[6]=t.rowid
8       IsNull           6     17    0                    0   if (r[6]==NULL) goto 17
9       Integer          1     1     0                    0   r[1]=1
10      Column           0     1     2                    0   r[2]=t.b
11      Column           0     2     3                    0   r[3]=t.c
12      MakeRecord       1     3     7                    0   r[7]=mkrec(r[1..3])
13      InsertAsync      0     7     6                    0
14      InsertAwait      0     0     0                    0
15    NextAsync          0     0     0                    0
16    NextAwait          0     5     0                    0
17    Halt               0     0     0                    0
18    Transaction        0     1     0                    0   write=true
19    Integer            2     5     0                    0   r[5]=2
20    Goto               0     1     0                    0
```
### sqlite
```console
sqlite> explain update t set a = 1 where b = 2;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     15    0                    0   Start at 15
1     Null           0     1     2                    0   r[1..2]=NULL
2     Noop           1     0     1                    0
3     OpenWrite      0     2     0     2              0   root=2 iDb=0; t
4     Rewind         0     14    0                    0
5       Column         0     1     5                    0   r[5]= cursor 0 column 1
6       Ne             6     13    5     BINARY-8       81  if r[5]!=r[6] goto 13
7       Rowid          0     2     0                    0   r[2]= rowid of 0
8       IsNull         2     14    0                    0   if r[2]==NULL goto 14
9       Integer        1     3     0                    0   r[3]=1
10      Column         0     1     4                    0   r[4]= cursor 0 column 1
11      MakeRecord     3     2     1                    0   r[1]=mkrec(r[3..4])
12      Insert         0     1     2     t              7   intkey=r[2] data=r[1]
13    Next           0     5     0                    1
14    Halt           0     0     0                    0
15    Transaction    0     1     1     0              1   usesStmtJournal=0
16    Integer        2     6     0                    0   r[6]=2
17    Goto           0     1     0                    0
```

Closes #1130
2025-03-24 09:19:22 +02:00
PThorpe92
2dec7b7255 Add more test cases for update 2025-03-23 20:12:10 -04:00
PThorpe92
4067c98848 Adjust update tests, remove unsupported syntax 2025-03-23 19:20:45 -04:00
PThorpe92
8f469f26b6 Add some additional tcl tests for update support 2025-03-23 17:08:15 -04:00
PThorpe92
dbfe94d677 Add initial tests for update support 2025-03-23 17:08:15 -04:00
l.gualtieri
a9ad5a56b9 fix bug #1155 2025-03-23 16:51:10 +01:00
Pekka Enberg
7832ae22df Enable pretty mode in shell by default
Fixes #929
2025-03-22 08:10:26 +02:00
Ihor Andrianov
bf649f98be add tests 2025-03-19 11:29:23 +02:00
Pekka Enberg
f9d7834874 Merge 'Jsonb extract' from Ihor Andrianov
Made a jsonb traversal by json path.
Changed some ordinary json functions to use jsonb under the hood, so now
behavior of our json module more like sqlite.
Found and fixed some bugs on the way.

Closes #1135
2025-03-17 18:25:28 +02:00
Diego Reis
250478fedf Implement deferred transactions
As explained in docs: https://sqlite.org/lang_transaction.html

"BEGIN DEFERRED statement merely sets a flag on the database connection that turns off the automatic commit that would normally occur when the last statement finishes."

The transaction upgrade (read -> write) is already handled by the VDBE
2025-03-17 10:01:00 -03:00
Diego Reis
590f90ad9a Fix AutoCommit handling of an ongoing halt checkpoint 2025-03-16 15:35:49 -03:00
Ihor Andrianov
a3a9376347 unblock some tests 2025-03-16 03:36:02 +02:00
Pekka Enberg
731fbaf3c7 Merge 'Jsonb implementation' from Ihor Andrianov
This PR implements a complete JSONB parser and serializer as current PR
draft looks stale.
Sorry for huge PR.
I've choose a recursive parsing approach because:
1. It's simpler to understand and maintain
2. It follows SQLite's implementation pattern, ensuring compatibility
3. It naturally maps to JSON's hierarchical structure
The implementation includes comprehensive test coverage for standard
JSON features and JSON5 extensions. All test cases pass successfully,
handling edge cases like nested structures, escape sequences, and
various number formats.
While the code is ready for review, I believe it would benefit from fuzz
testing in the future to identify any edge cases not covered by the
current tests.
Ready for review, proposals and feedback.

Closes #1114
2025-03-13 21:17:52 +02:00
PThorpe92
c638b64a59 Fix tests to use updated extension name 2025-03-12 21:55:50 -04:00
PThorpe92
35fc9df275 Rename and combine testing extension crate 2025-03-12 21:52:51 -04:00
Ihor Andrianov
39c2481ce3 add e2e tests 2025-03-12 15:34:58 +02:00
Pekka Enberg
b0636e4494 Merge 'Adds Drop Table' from Zaid Humayun
This PR adds support for `DROP TABLE` and addresses issue
https://github.com/tursodatabase/limbo/issues/894
It depends on https://github.com/tursodatabase/limbo/pull/785 being
merged in because it requires the implementation of `free_page`.
EDIT: The PR above has been merged.
It adds the following:
* an implementation for the `DropTable` AST instruction via a method
called `translate_drop_table`
* a couple of new instructions - `Destroy` and `DropTable`. The former
is to modify physical b-tree pages and the latter is to modify in-memory
structures like the schema hash table.
* `btree_destroy` on `BTreeCursor` to walk the tree of pages for this
table and place it in free list.
* state machine traversal for both `btree_destroy` and
`clear_overflow_pages` to ensure performant, correct code.
* unit & tcl tests
* modifies the `Null` instruction to follow SQLite semantics and accept
a second register. It will set all registers in this range to null. This
is required for `DROP TABLE`.
The screenshots below have a comparison of the bytecodes generated via
SQLite & Limbo.
Limbo has the same instruction set except for the subroutines which
involve opening an ephemeral table, copying over the triggers from the
`sqlite_schema` table and then re-inserting them back into the
`sqlite_schema` table.
This is because `OpenEphemeral` is still a WIP and is being tracked at
https://github.com/tursodatabase/limbo/pull/768
![Screenshot 2025-02-09 at 7 05 03 PM](https://github.com/user-
attachments/assets/1d597001-a60c-4a76-89fd-8b90881c77c9)
![Screenshot 2025-02-09 at 7 05 35 PM](https://github.com/user-
attachments/assets/ecfd2a7a-2edc-49cd-a8d1-7b4db8657444)

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #897
2025-03-06 18:27:41 +02:00
Pekka Enberg
2e4c18dca2 Merge 'Escape character is ignored in LIKE function' from lgualtieri75
Fixes #1051

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #1074
2025-03-04 10:23:09 +02:00
Pekka Enberg
01f121b567 Merge 'adding limbo.quit to cleanly quit the subprocess in python tests' from Pedro Muniz
Closes #1071
2025-03-04 10:21:49 +02:00
EmNudge
116350d139 Add ipaddr extension 2025-03-02 16:03:46 -05:00
l.gualtieri
6449c79e93 Escape character is ignored in LIKE function #1051 2025-03-01 18:32:09 +01:00