Commit Graph

274 Commits

Author SHA1 Message Date
Kacper Madej
19ae42dfa3 Implement json_array 2024-12-20 11:15:48 +01:00
Raul Ferrando
d74012bb59 fix pragma parsing potential errors 2024-12-18 17:26:16 +01:00
Raul Ferrando
a6d1a7cb56 pragma: inital approach to handle pragma statements
This change refactors how PRAGMA statements are handled, introducing a
more organized and extensible structure to simplify adding new PRAGMA
properties in the future.

Previously, only the `cache_size` PRAGMA was supported. With this
update, support for the `journal_mode` PRAGMA has been added.
2024-12-18 17:26:16 +01:00
jussisaurio
783ec65c77 Merge 'Expression equality checking, some optimizations' from Preston Thorpe
This PR mainly adds custom logic to check equality in ast expressions.
Not sure if this belongs in the `vendored` parser or not, let me know
and I'll bring it out. Also replaces `Vec` arguments with slice refs
where possible, as well as some clippy warnings in the same `emitter`
file.
I'll write some more tests tomorrow to make sure this is as thorough as
possible.
EDIT: failed test same issue referenced in #484. Marking as draft until
more tests + cases added

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #488
2024-12-17 22:09:20 +02:00
PThorpe92
66c44a021f Move ast expr equality check out of vendored, remove allocs and add cases 2024-12-16 19:39:06 -05:00
PThorpe92
25772ee1f3 Implement custom expression equality checking 2024-12-16 19:39:05 -05:00
PThorpe92
8d18263fd6 Replace vec args with slices where possible, clippy warnings 2024-12-16 19:39:05 -05:00
Lauri Virtanen
e69ee80fac Support log(X) and log(B,X) math functions 2024-12-17 00:14:25 +02:00
Lauri Virtanen
89d0289444 Support pi() function 2024-12-17 00:14:25 +02:00
Lauri Virtanen
5e426a7624 Support binary math functions 2024-12-16 22:29:05 +02:00
Lauri Virtanen
f69fdc1645 Support unary math functions 2024-12-16 19:31:20 +02:00
Lauri Virtanen
9720f63a55 Add types for mathematical functions 2024-12-16 19:31:20 +02:00
Pekka Enberg
61269e9c03 Merge 'Fix primary key handling' from Jussi Saurio
Closes #438
**Fixes:**
- Return the actual primary key instead of the rowid when the primary
key is not a rowid alias
sqlite:
```
limbo> create table foo (pk TEXT PRIMARY KEY, value);
sqlite> insert into foo values ('one', 'payload');
sqlite> insert into foo values ('two', 'payload');
sqlite> select * from foo;
one|payload
two|payload
```
limbo now:
```
limbo> create table foo (pk TEXT PRIMARY KEY, value);
limbo> insert into foo values ('one', 'payload');
limbo> insert into foo values ('two', 'payload');
limbo> select * from foo;
one|payload
two|payload
```
limbo before:
```
limbo> select * from foo;
1|payload
2|payload
```
Then, discovered two issues when running the whole TCL test suite
against a copy of `testing.db` that does not have rowid aliases in the
tables:
- Fix trying to convert a Scan into an IndexSearch when the associated
Expr refers to a different instance of the same table in a self-join
- Fix `scan_loop_body_labels` being pushed to in different parts of the
codegen for Scan/Search nodes resulting in incorrect label resolutions
**Additions:**
- Add a new db `testing/testing_norowidalias.db` that is a carbon copy
of `testing/testing.db` except the `id` columns of `users` and
`products` are not rowid aliases (i.e. they are `INT PRIMARY KEY`
instead of `INTEGER PRIMARY KEY`
- Run all TCL tests against both test databases with `do_execsql_test`
```
(testing/testing.db)                        Running test: coalesce-from-table
(testing/testing_norowidalias.db)           Running test: coalesce-from-table
(testing/testing.db)                        Running test: coalesce-from-table-column
(testing/testing_norowidalias.db)           Running test: coalesce-from-table-column
(testing/testing.db)                        Running test: coalesce-from-table-multiple-columns
(testing/testing_norowidalias.db)           Running test: coalesce-from-table-multiple-columns
(testing/testing.db)                        Running test: glob-fn
(testing/testing_norowidalias.db)           Running test: glob-fn
(testing/testing.db)                        Running test: where-glob
(testing/testing_norowidalias.db)           Running test: where-glob
(testing/testing.db)                        Running test: where-glob-question-mark
(testing/testing_norowidalias.db)           Running test: where-glob-question-mark
```
- Allow running tests against specific db file with
`do_execsql_test_on_specific_db $path`
- Wondering if I should add a new table with e.g. text primary key and
add some db-specific tests on that table...?

Closes #449
2024-12-16 18:49:19 +02:00
jussisaurio
419ccc327e Dont assume GROUP BY has aggregates 2024-12-16 00:30:38 +02:00
jussisaurio
351cb7b3f3 fix bug with incorrect ordering of scan loop body labels stack elements 2024-12-14 17:12:24 +02:00
jussisaurio
a04cf611e0 fix bug with making an index search node in a self join where expr refers to other instance of same table 2024-12-14 17:11:32 +02:00
jussisaurio
961e57df94 Fix returning rowid instead of PK when PK is not a rowid alias 2024-12-14 15:51:08 +02:00
jussisaurio
5e9e2dffe9 support TRUE and FALSE in predicates 2024-12-13 22:58:29 +02:00
Pekka Enberg
2b85d2a600 Merge 'Add implementation and tests for replace scalar function' from Alperen Keleş
Adds `replace` scalar function.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #446
2024-12-13 11:02:08 +02:00
JeanArhancet
8bf6572e9e feat: support unary positive 2024-12-13 02:07:34 +01:00
alpaylan
5742c51c5c improve error messages on replace function call 2024-12-12 14:14:04 -05:00
Alperen Keleş
841a4fe2f8 Merge branch 'tursodatabase:main' into main 2024-12-12 13:13:41 -05:00
Pekka Enberg
5796b418b9 Merge 'Add support for soundex() function' from flaneur
add [soundex](https://www.sqlite.org/lang_corefunc.html#soundex) scalar
function.
it seems that sqlite did not enable `soundex()` function by default
unless build it with `SQLITE_SOUNDEX`, while the sqlite in the ci
workflow did not enable it. this pr skipped the test over `soundex()`
temporarily in the `scalar-function.test` file.

Closes #453
2024-12-12 18:10:46 +02:00
Pekka Enberg
91764b85e6 Merge 'Add bitwise vdbe ops' from Preston Thorpe
Love the project, been following your blog posts for quite a while now.
I asked on Discord prior to submitting this, just because I didn't see a
specific issue for this feature... but if this PR is out of scope for
contributors, feel free to close it as I just had a good time hacking on
it.
This PR adds support for `BitAnd`, `BitOr`, and `BitNot` operators in
the vdbe, as well as unary expressions applied to aggregate functions;
which was needed in order to have `BitNot` support the same tests that
the other operators had.
*Also added unary negation of function calls, because since unary ops
were added, I figured adding support for the other existing unary
operator might be in scope, but lmk if not.
Let me know if there is any more tests or documentation to add/improve.

Closes #445
2024-12-12 17:35:12 +02:00
Li Yazhou
03288e5170 add impl about scalar function soundex with test 2024-12-12 21:48:05 +08:00
Alex Miller
c4d4569dc9 Merge remote-tracking branch 'upstream/main' into expr-iif 2024-12-11 20:13:54 -08:00
alpaylan
da28ed51ca add implementation and tests for replace scalar function 2024-12-11 16:23:13 -05:00
PThorpe92
16595f39f5 Add support for unary op negation of aggregates 2024-12-11 15:38:21 -05:00
PThorpe92
d5391dc716 Add vdbe bitwise operators: and, or, not 2024-12-11 11:06:22 -05:00
jussisaurio
eb9374aebf Merge 'Add support for CASE expressions.' from Alex Miller
There's two forms of case:
  CASE (WHEN [bool expr] THEN [value])+ (ELSE [value])? END
which checks a series of boolean conditions, and:
  CASE expr (WHEN [expr] THEN [value})+ (ELSE [value])? END
Which checks a series of equality conditions.
This implements support for both. Note that the ELSE is optional, and
will be equivalent to `ELSE null` if not specified.
sqlite3 gives the implementation as:
```
sqlite> explain select case a WHEN a THEN b WHEN c THEN d ELSE 0 END from casetest;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     16    0                    0   Start at 16
1     OpenRead       0     3     0     4              0   root=3 iDb=0; casetest
2     Rewind         0     15    0                    0
3       Column         0     0     2                    0   r[2]= cursor 0 column 0
4       Column         0     0     3                    0   r[3]= cursor 0 column 0
5       Ne             3     8     2     BINARY-8       83  if r[2]!=r[3] goto 8
6       Column         0     1     1                    0   r[1]= cursor 0 column 1
7       Goto           0     13    0                    0
8       Column         0     2     3                    0   r[3]= cursor 0 column 2
9       Ne             3     12    2     BINARY-8       83  if r[2]!=r[3] goto 12
10      Column         0     3     1                    0   r[1]= cursor 0 column 3
11      Goto           0     13    0                    0
12      Integer        0     1     0                    0   r[1]=0
13      ResultRow      1     1     0                    0   output=r[1]
14    Next           0     3     0                    1
15    Halt           0     0     0                    0
16    Transaction    0     0     2     0              1   usesStmtJournal=0
17    Goto           0     1     0                    0
```
and after this patch, limbo gives:
```
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     19    0                    0   Start at 19
1     OpenReadAsync      0     4     0                    0   table=casetest, root=4
2     OpenReadAwait      0     0     0                    0
3     RewindAsync        0     0     0                    0
4     RewindAwait        0     18    0                    0   Rewind table casetest
5       Column           0     0     2                    0   r[2]=casetest.a
6       Column           0     0     3                    0   r[3]=casetest.a
7       Ne               2     3     10                   0   if r[2]!=r[3] goto 10
8       Column           0     1     1                    0   r[1]=casetest.b
9       Goto             0     15    0                    0
10      Column           0     2     3                    0   r[3]=casetest.c
11      Ne               2     3     14                   0   if r[2]!=r[3] goto 14
12      Column           0     3     1                    0   r[1]=casetest.d
13      Goto             0     15    0                    0
14      Integer          0     1     0                    0   r[1]=0
15      ResultRow        1     1     0                    0   output=r[1]
16    NextAsync          0     0     0                    0
17    NextAwait          0     5     0                    0
18    Halt               0     0     0                    0
19    Transaction        0     0     0                    0
20    Goto               0     1     0                    0
```
And then as there's nowhere to annotate this new support in COMPAT.md, I
added a corresponding heading for SELECT expressions and what is/isn't
supported.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #425
2024-12-11 17:05:41 +02:00
Pekka Enberg
04f196113a Merge 'Add last_insert_rowid() function' from Krishna Vishal
- Changed `Cursor` trait to be able to get access to `root_page`
- SQLite only updates last_insert_rowid for non-schema inserts. So we
check if the `InsertAwait` is not for `root_page` before   updating
rowid
In SQLite it looks like this:
```
sqlite> EXPLAIN SELECT last_insert_rowid();
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     4     0                    0
1     Function       0     0     1     last_insert_rowid(0) 0
2     ResultRow      1     1     0                    0
3     Halt           0     0     0                    0
4     Goto           0     1     0                    0
```
In limbo it will look like this:
```
limbo> EXPLAIN SELECT last_insert_rowid();
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     4     0                    0   Start at 4
1     Function           0     2     1     last_insert_rowid  0   r[1]=func()
2     ResultRow          1     1     0                    0   output=r[1]
3     Halt               0     0     0                    0
4     Transaction        0     0     0                    0
5     Goto               0     1     0                    0
```

Closes #427
2024-12-11 10:44:34 +02:00
Pekka Enberg
ca272ba937 Merge 'Support JOIN USING and NATURAL JOIN' from Jussi Saurio
Closes #360
Closes #361

Closes #422
2024-12-11 09:17:51 +02:00
Alex Miller
88c862ce4d Comments, resolve label better, make tests more fun 2024-12-10 19:59:54 -08:00
Alex Miller
e85df1c895 resolve labels to current offset. make test clearer. 2024-12-10 19:36:54 -08:00
Kacper Kołodziej
d4bff2c93e add octet_length scalar function 2024-12-10 22:56:38 +01:00
krishvishal
134b5576ad Ran cargo fmt 2024-12-09 22:55:54 +05:30
jussisaurio
fe88d45e5e Add more comments to push_predicate/push_predicates 2024-12-09 17:50:29 +02:00
jussisaurio
840caed2f7 Fix bug with multiway joins that include the same table multiple times 2024-12-09 17:50:29 +02:00
jussisaurio
7924f9b64d consider all joined tables instead of just previous in natural/using 2024-12-09 17:50:29 +02:00
jussisaurio
81b6605453 support NATURAL JOIN 2024-12-09 17:50:29 +02:00
jussisaurio
bed932c186 Support join USING 2024-12-09 17:50:29 +02:00
Pekka Enberg
ba1f7cd16f Merge 'feat(core/translate): support HAVING' from Jussi Saurio
support the HAVING clause.
note that sqlite (and i think standard sql?) supports HAVING even
without GROUP BY, but `sqlite3-parser` doesn't.
also fixes some issues with the PartialOrd implementation of OwnedValue
and the implementations of `concat` and `round` which i discovered due
to my HAVING tcl tests failing

Closes #420
2024-12-09 17:30:40 +02:00
krishvishal
1e23af7d24 Added last_insert_rowid() function.
Need to fix its behavior. Problem is probably with `Cursor` implementation.
2024-12-09 17:41:28 +05:30
Alex Miller
f7bb7f8dee Fix typo and improve comment 2024-12-08 14:20:23 -08:00
Alex Miller
c2e3957d73 I misunderstood what a constant instruction was 2024-12-08 14:12:45 -08:00
Alex Miller
eb00226cfe Add support for CASE expressions.
There's two forms of case:
  CASE (WHEN [bool expr] THEN [value])+ (ELSE [value])? END
which checks a series of boolean conditions, and:
  CASE expr (WHEN [expr] THEN [value})+ (ELSE [value])? END
Which checks a series of equality conditions.

This implements support for both. Note that the ELSE is optional, and
will be equivalent to `ELSE null` if not specified.

sqlite3 gives the implementation as:

sqlite> explain select case a WHEN a THEN b WHEN c THEN d ELSE 0 END from casetest;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     16    0                    0   Start at 16
1     OpenRead       0     3     0     4              0   root=3 iDb=0; casetest
2     Rewind         0     15    0                    0
3       Column         0     0     2                    0   r[2]= cursor 0 column 0
4       Column         0     0     3                    0   r[3]= cursor 0 column 0
5       Ne             3     8     2     BINARY-8       83  if r[2]!=r[3] goto 8
6       Column         0     1     1                    0   r[1]= cursor 0 column 1
7       Goto           0     13    0                    0
8       Column         0     2     3                    0   r[3]= cursor 0 column 2
9       Ne             3     12    2     BINARY-8       83  if r[2]!=r[3] goto 12
10      Column         0     3     1                    0   r[1]= cursor 0 column 3
11      Goto           0     13    0                    0
12      Integer        0     1     0                    0   r[1]=0
13      ResultRow      1     1     0                    0   output=r[1]
14    Next           0     3     0                    1
15    Halt           0     0     0                    0
16    Transaction    0     0     2     0              1   usesStmtJournal=0
17    Goto           0     1     0                    0

and after this patch, limbo gives:

addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     18    0                    0   Start at 18
1     OpenReadAsync      0     4     0                    0   table=casetest, root=4
2     OpenReadAwait      0     0     0                    0
3     RewindAsync        0     0     0                    0
4     RewindAwait        0     17    0                    0   Rewind table casetest
5       Column           0     0     2                    0   r[2]=casetest.a
6       Column           0     0     3                    0   r[3]=casetest.a
7       Ne               2     3     10                   0   if r[2]!=r[3] goto 10
8       Column           0     1     1                    0   r[1]=casetest.b
9       Goto             0     14    0                    0
10      Column           0     2     3                    0   r[3]=casetest.c
11      Ne               2     3     14                   0   if r[2]!=r[3] goto 14
12      Column           0     3     1                    0   r[1]=casetest.d
13      Goto             0     14    0                    0
14      ResultRow        1     1     0                    0   output=r[1]
15    NextAsync          0     0     0                    0
16    NextAwait          0     5     0                    0
17    Halt               0     0     0                    0
18    Transaction        0     0     0                    0
19    Integer            0     1     0                    0   r[1]=0
20    Goto               0     1     0                    0

And then as there's nowhere to annotate this new support in COMPAT.md, I
added a corresponding heading for SELECT expressions and what is/isn't
supported.
2024-12-08 14:09:03 -08:00
Alex Miller
27ef95e0ab LLM recommended a better way to write args matching 2024-12-07 21:15:04 -08:00
Alex Miller
183ea8e362 Implement support for iif().
In sqlite, iif() looks like:

sqlite> create table iiftest(a int, b int, c int);
sqlite> explain select iif(a,b,c) from iiftest;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     11    0                    0   Start at 11
1     OpenRead       0     2     0     3              0   root=2 iDb=0; iiftest
2     Rewind         0     10    0                    0
3       Column         0     0     2                    0   r[2]= cursor 0 column 0
4       IfNot          2     7     1                    0
5       Column         0     1     1                    0   r[1]= cursor 0 column 1
6       Goto           0     8     0                    0
7       Column         0     2     1                    0   r[1]= cursor 0 column 2
8       ResultRow      1     1     0                    0   output=r[1]
9     Next           0     3     0                    1
10    Halt           0     0     0                    0
11    Transaction    0     0     1     0              1   usesStmtJournal=0
12    Goto           0     1     0                    0

And with this change, in limbo it looks like:

addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     14    0                    0   Start at 14
1     OpenReadAsync      0     2     0                    0   table=iiftest, root=2
2     OpenReadAwait      0     0     0                    0
3     RewindAsync        0     0     0                    0
4     RewindAwait        0     13    0                    0   Rewind table iiftest
5       Column           0     0     2                    0   r[2]=iiftest.a
6       IfNot            2     9     1                    0   if !r[2] goto 9
7       Column           0     1     1                    0   r[1]=iiftest.b
8       Goto             0     10    0                    0
9       Column           0     2     1                    0   r[1]=iiftest.c
10      ResultRow        1     1     0                    0   output=r[1]
11    NextAsync          0     0     0                    0
12    NextAwait          0     5     0                    0
13    Halt               0     0     0                    0
14    Transaction        0     0     0                    0
15    Goto               0     1     0                    0
2024-12-07 21:04:03 -08:00
jussisaurio
ca25a73c95 a little bit more explanation about left join handling 2024-11-30 20:54:22 +02:00
jussisaurio
83f8ea1b13 Fix bug with multiway joins and clean up left join implementation 2024-11-30 20:47:48 +02:00