Commit Graph

163 Commits

Author SHA1 Message Date
Glauber Costa
38def26704 Add expr_compiler
To be used in DBSP-based projections. This will compile an expression
to VDBE bytecode and execute it.

To do that we need to add a new type of Expression, which we call a
Register.

This is a way for us to pass parameters to a DBSP program which will be
not columns or literals, but inputs from the DBSP deltas.
2025-08-25 17:48:17 +03:00
Avinash Sajjanshetty
93774ffc3b Add PRAGMA key to set the encryption key
If set, set the key for the connection
2025-08-20 11:39:07 +05:30
pedrocarlo
d0c13f0104 remove IOError from Parser + store only ErrorKind in LimboError 2025-08-19 10:48:21 -03:00
Glauber Costa
770f86e490 move our dbsp-based views to materialized views
We will implement normal SQLite-style view-as-an-alias for
compatibility, and will call our incremental views materialized views.
2025-08-12 14:19:17 -05:00
Jussi Saurio
f598c86fa4 Merge 'Handle single, double and unquoted strings in values clause' from Mikaël Francoeur
I'm not sure how much this will clash with @TcMits's parser rewrite,
hopefully not too much. If it does and we eventually have to remove it,
at least we'll have two new regression tests.
Closes https://github.com/tursodatabase/turso/issues/2484

Closes #2499
2025-08-11 21:08:15 +03:00
Preston Thorpe
cb59877821 Merge 'Feat/pragma module list' from Lucas Forato
Implementation of the module_list pragma

Reviewed-by: Diego Reis (@el-yawd)
Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2430
2025-08-11 13:54:24 -04:00
Lucas Forato
27fd6aa13a chore: updated comment 2025-08-11 08:42:03 -03:00
Lucas Forato
ff157e10e5 feat: included ModuleList in PragmaName 2025-08-11 08:41:53 -03:00
bit-aloo
4dfa0d77c5 put freelist_count lexicographically in pragma name enum 2025-08-11 11:06:56 +05:30
bit-aloo
d2171e24a5 add pragma freelist_count 2025-08-11 10:03:46 +05:30
Mikaël Francoeur
2cf4e4fe96 handle single, double and unquoted strings in values clause 2025-08-08 09:03:38 -04:00
Pekka Enberg
0f9d0cf519 Merge branch 'main' into 2025-08-07-add-query-only-pragma 2025-08-08 07:41:38 +03:00
bit-aloo
6ffd4215a2 sort Pragma names in lexicographical order 2025-08-08 02:52:45 +05:30
bit-aloo
aaeec4d4f3 Implement PRAGMA query_only logic 2025-08-07 23:49:07 +05:30
Glauber Costa
f36974f086 implement the MaxPgCount opcode
It is used by the pragma max_page_count, which is also implemented.
2025-08-06 13:20:15 -05:00
Diego Reis
3834f441c4 Accept parsing SET statements with repeated names, like `.. SET (a, a) =
(1, 2)`
2025-07-31 00:08:12 -03:00
Glauber Costa
b8ee38868d implement the pragma encoding
Do not allow setting it. That ship has sailed around 2005.
2025-07-26 19:37:39 -05:00
meteorgan
b5a18d7dc9 fix get_column_name() when column name doesn't exist 2025-07-25 23:49:31 +08:00
bit-aloo
3cb2db933d remove Id 2025-07-24 14:40:24 +05:30
bit-aloo
9a54ef214e parser: Distinguish quoted identifiers and unify Id into Name enum
This commit replaces the `Name(pub String)` struct with a `Name` enum that
explicitly models how the name appeared in the source either as an
unquoted identifier (`Ident`) or a quoted string (`Quoted`).

In the process, the separate `Id` wrapper type has been coalesced into the
`Name` enum, simplifying the AST and reducing duplication in identifier
handling logic.

While this increases the size of some AST nodes (notably `yyStackEntry`),
it improves correctness and makes source structure more explicit for
later phases.
2025-07-24 14:40:19 +05:30
Jussi Saurio
022f679fab chore: make every CREATE TABLE stmt in entire repo have 1 space after tbl name
`BTreeTable::to_sql` makes us incompatible with SQLite by losing e.g. the original whitespace provided during the CREATE TABLE command.

For now let's fix our tests by regex-replacing every CREATE TABLE in
the entire repo to have exactly 1 space after the table name in the
CREATE TABLE statement.
2025-07-22 11:35:21 +03:00
Glauber Costa
0545049d59 Implement pragma database_list
And also the CLI option .databases, which is just manipulating that.

This is one step in the road to attach.
2025-07-21 08:49:35 -05:00
Glauber Costa
6506b3147d implement pragma application_id
Just for completeness, because it is easy.
2025-07-19 20:44:06 -05:00
Levy A.
89911ee8d1 remove to_sql_string from simulator 2025-07-16 12:34:10 -03:00
Levy A.
714225b9f0 remove ToSqlString trait 2025-07-16 12:16:34 -03:00
Levy A.
6fe2505425 add more ToTokens impls 2025-07-16 12:16:31 -03:00
Levy A.
e81c7b07fb modify tests for new formatter 2025-07-16 12:16:31 -03:00
Levy A.
373a4a26c4 fix: comma function 2025-07-16 12:16:28 -03:00
Levy A.
9ff9c3fdc2 feat: add context to ToTokens 2025-07-16 12:12:15 -03:00
alpaylan
0b3fb2ecdd Merge branch 'main' of https://github.com/tursodatabase/limbo 2025-07-13 11:10:52 -04:00
Nils Koch
1a91966c7e fix clippy errors for rust 1.88.0 (manual fix) 2025-07-12 18:58:55 +03:00
alpaylan
c3d904b9ba remove automatic parenthesis insertion from binary and unary expressions 2025-07-11 11:08:21 -04:00
alpaylan
65fe60ba57 fix the merge conflicts 2025-07-11 02:04:14 -04:00
alpaylan
8f46bbc77e Merge branch 'main' of https://github.com/tursodatabase/limbo 2025-07-11 01:33:50 -04:00
Pekka Enberg
27705ce00a Merge 'parser: replace KEYWORDS with matching' from Lâm Hoàng Phúc
before:
```sh
sqlparser-rs parsing benchmark/sqlparser::select
                        time:   [693.20 ns 693.96 ns 694.73 ns]
                        change: [+7.4382% +7.6384% +7.8250%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) low severe
  1 (1.00%) low mild
  1 (1.00%) high mild
sqlparser-rs parsing benchmark/sqlparser::with_select
                        time:   [2.5734 µs 2.5763 µs 2.5796 µs]
                        change: [+16.583% +16.809% +17.024%] (p = 0.00 < 0.05)
                        Performance has regressed.
sqlparser-rs parsing benchmark/keyword_token
                        time:   [3.1919 µs 3.1983 µs 3.2047 µs]
                        change: [+944.74% +948.97% +952.91%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) low mild
```
after:
```sh
sqlparser-rs parsing benchmark/sqlparser::select
                        time:   [637.09 ns 638.50 ns 640.15 ns]
                        change: [-1.8412% -1.5494% -1.2424%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  1 (1.00%) low severe
  3 (3.00%) low mild
  3 (3.00%) high mild
  1 (1.00%) high severe
sqlparser-rs parsing benchmark/sqlparser::with_select
                        time:   [2.1896 µs 2.1919 µs 2.1942 µs]
                        change: [-0.6894% -0.3923% -0.1517%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) low severe
sqlparser-rs parsing benchmark/keyword_token
                        time:   [298.99 ns 299.82 ns 300.72 ns]
                        change: [-1.4726% -1.0148% -0.5702%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) low mild
  6 (6.00%) high mild
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1939
2025-07-08 10:21:03 +03:00
alpaylan
b0cf2ba92c add paranthesis to the unary expression printer 2025-07-06 14:58:29 -04:00
alpaylan
0bce68b38d wip: add joins to the select 2025-07-06 14:46:38 -04:00
Nikita Sivukhin
1ee475f04a rename pragma to unsable_capture_data_changes_conn 2025-07-06 22:32:42 +04:00
Nikita Sivukhin
6a6276878c fix test 2025-07-06 21:16:58 +04:00
Nikita Sivukhin
04f2efeaa4 small renames 2025-07-06 21:16:57 +04:00
Nikita Sivukhin
234dda322f handle change_capture pragma 2025-07-06 21:16:25 +04:00
TcMits
954b58d8ac generate MAX_KEYWORD_LEN, MIN_KEYWORD_LEN 2025-07-04 17:11:26 +07:00
TcMits
122f5c3f42 parser: replace KEYWORDS with matching 2025-07-03 20:43:33 +07:00
Pekka Enberg
725c3e4ddc Rename limbo_sqlite3_parser crate to turso_sqlite3_parser 2025-06-29 12:34:46 +03:00
Pekka Enberg
53ba3ff926 Rename limbo_core crate to turso_core 2025-06-29 09:59:17 +03:00
Nils Koch
2827b86917 chore: fix clippy warnings 2025-06-23 19:52:13 +01:00
Pekka Enberg
882c5ca168 Merge 'Simple integrity check on btree' from Pere Diaz Bou
This PR adds support for the instruction `IntegrityCk` which performs an
integrity check on the contents of a single table. Next PR I will try to
implement the rest of the integrity check where we would check indexes
containt correct amount of data and some more.
<img width="1151" alt="image" src="https://github.com/user-
attachments/assets/29d54148-55ba-480f-b972-e38587f0a483" />

Closes #1719
2025-06-16 13:46:26 +03:00
Jussi Saurio
3f90fad131 Merge 'Simulator Ast Generation + Simulator Unary Operator + Refactor to use limbo_core::Value in Simulator for massive code reuse' from Pedro Muniz
This PR is a Drop-In replacement to the Predicate defined in the
Simulator. Predicate is basically the same as our ast::Expr, but it
supports a small number of the SQL expression syntax. By creating a
NewType that wraps ast::Expr we can tap into our already mostly
correctly defined parser structs. This change will enable us to easily
add generation for more types of sql queries.
I also added an ArbitraryFrom impl for ast::Expr that can be used in a
freestyle way (for now) for differential testing.
This PR also aims to implement Unary Operator logic similar to the
Binary Operator logic we have for predicate. After this change we may
need to adjust the Logic for how some assertions are triggered.
<s>Sometimes the `Select-Select-Optimizer` property thinks that these
two queries should return the same thing:
```sql
SELECT (twinkling_winstanley.sensible_federations > x'66616e7461737469625e0f37879823db' AND twinkling_winstanley.sincere_niemeyer < -7428368947470022783) FROM twinkling_winstanley WHERE 1;

SELECT * FROM twinkling_winstanley WHERE twinkling_winstanley.sensible_federations > x'66616e7461737469625e0f37879823db' AND twinkling_winstanley.sincere_niemeyer < -7428368947470022783;
```
However after running the shrunk plan manually, the simulator was
incorrect in asserting that. Maybe this a bug a in the generation of
such a query? Not sure yet. </s>
<b>EDIT: The simulator was correctly catching a bug and I thought I was
the problem. The bug was in `exec_if` and I fixed it in this PR.</b>
I still need to expand the Unary Operator generation to other types of
predicates. For now, I just implemented it for `SimplePredicate` as I'm
trying to avoid to bloat even more this PR.
<b>EDIT: I decided to just have one PR open for all the changes I'm
making to make my life a bit easier and to avoid merge conflicts with my
own branches that I keep spawning for new code.</b>
PS: This should only be considered for merging after
https://github.com/tursodatabase/limbo/pull/1619 is merged. Then, I will
remove the draft status from this PR.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1674
2025-06-13 11:23:10 +03:00
Levy A.
b88cb99ff0 fix warnings and some refactoring 2025-06-11 14:19:06 -03:00
Levy A.
49a6ddad97 wip 2025-06-11 14:19:04 -03:00