Commit Graph

145 Commits

Author SHA1 Message Date
bit-aloo
3cb2db933d remove Id 2025-07-24 14:40:24 +05:30
bit-aloo
9a54ef214e parser: Distinguish quoted identifiers and unify Id into Name enum
This commit replaces the `Name(pub String)` struct with a `Name` enum that
explicitly models how the name appeared in the source either as an
unquoted identifier (`Ident`) or a quoted string (`Quoted`).

In the process, the separate `Id` wrapper type has been coalesced into the
`Name` enum, simplifying the AST and reducing duplication in identifier
handling logic.

While this increases the size of some AST nodes (notably `yyStackEntry`),
it improves correctness and makes source structure more explicit for
later phases.
2025-07-24 14:40:19 +05:30
Jussi Saurio
022f679fab chore: make every CREATE TABLE stmt in entire repo have 1 space after tbl name
`BTreeTable::to_sql` makes us incompatible with SQLite by losing e.g. the original whitespace provided during the CREATE TABLE command.

For now let's fix our tests by regex-replacing every CREATE TABLE in
the entire repo to have exactly 1 space after the table name in the
CREATE TABLE statement.
2025-07-22 11:35:21 +03:00
Glauber Costa
0545049d59 Implement pragma database_list
And also the CLI option .databases, which is just manipulating that.

This is one step in the road to attach.
2025-07-21 08:49:35 -05:00
Glauber Costa
6506b3147d implement pragma application_id
Just for completeness, because it is easy.
2025-07-19 20:44:06 -05:00
Levy A.
89911ee8d1 remove to_sql_string from simulator 2025-07-16 12:34:10 -03:00
Levy A.
714225b9f0 remove ToSqlString trait 2025-07-16 12:16:34 -03:00
Levy A.
6fe2505425 add more ToTokens impls 2025-07-16 12:16:31 -03:00
Levy A.
e81c7b07fb modify tests for new formatter 2025-07-16 12:16:31 -03:00
Levy A.
373a4a26c4 fix: comma function 2025-07-16 12:16:28 -03:00
Levy A.
9ff9c3fdc2 feat: add context to ToTokens 2025-07-16 12:12:15 -03:00
alpaylan
0b3fb2ecdd Merge branch 'main' of https://github.com/tursodatabase/limbo 2025-07-13 11:10:52 -04:00
Nils Koch
1a91966c7e fix clippy errors for rust 1.88.0 (manual fix) 2025-07-12 18:58:55 +03:00
alpaylan
c3d904b9ba remove automatic parenthesis insertion from binary and unary expressions 2025-07-11 11:08:21 -04:00
alpaylan
65fe60ba57 fix the merge conflicts 2025-07-11 02:04:14 -04:00
alpaylan
8f46bbc77e Merge branch 'main' of https://github.com/tursodatabase/limbo 2025-07-11 01:33:50 -04:00
Pekka Enberg
27705ce00a Merge 'parser: replace KEYWORDS with matching' from Lâm Hoàng Phúc
before:
```sh
sqlparser-rs parsing benchmark/sqlparser::select
                        time:   [693.20 ns 693.96 ns 694.73 ns]
                        change: [+7.4382% +7.6384% +7.8250%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) low severe
  1 (1.00%) low mild
  1 (1.00%) high mild
sqlparser-rs parsing benchmark/sqlparser::with_select
                        time:   [2.5734 µs 2.5763 µs 2.5796 µs]
                        change: [+16.583% +16.809% +17.024%] (p = 0.00 < 0.05)
                        Performance has regressed.
sqlparser-rs parsing benchmark/keyword_token
                        time:   [3.1919 µs 3.1983 µs 3.2047 µs]
                        change: [+944.74% +948.97% +952.91%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) low mild
```
after:
```sh
sqlparser-rs parsing benchmark/sqlparser::select
                        time:   [637.09 ns 638.50 ns 640.15 ns]
                        change: [-1.8412% -1.5494% -1.2424%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  1 (1.00%) low severe
  3 (3.00%) low mild
  3 (3.00%) high mild
  1 (1.00%) high severe
sqlparser-rs parsing benchmark/sqlparser::with_select
                        time:   [2.1896 µs 2.1919 µs 2.1942 µs]
                        change: [-0.6894% -0.3923% -0.1517%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) low severe
sqlparser-rs parsing benchmark/keyword_token
                        time:   [298.99 ns 299.82 ns 300.72 ns]
                        change: [-1.4726% -1.0148% -0.5702%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) low mild
  6 (6.00%) high mild
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1939
2025-07-08 10:21:03 +03:00
alpaylan
b0cf2ba92c add paranthesis to the unary expression printer 2025-07-06 14:58:29 -04:00
alpaylan
0bce68b38d wip: add joins to the select 2025-07-06 14:46:38 -04:00
Nikita Sivukhin
1ee475f04a rename pragma to unsable_capture_data_changes_conn 2025-07-06 22:32:42 +04:00
Nikita Sivukhin
6a6276878c fix test 2025-07-06 21:16:58 +04:00
Nikita Sivukhin
04f2efeaa4 small renames 2025-07-06 21:16:57 +04:00
Nikita Sivukhin
234dda322f handle change_capture pragma 2025-07-06 21:16:25 +04:00
TcMits
954b58d8ac generate MAX_KEYWORD_LEN, MIN_KEYWORD_LEN 2025-07-04 17:11:26 +07:00
TcMits
122f5c3f42 parser: replace KEYWORDS with matching 2025-07-03 20:43:33 +07:00
Pekka Enberg
725c3e4ddc Rename limbo_sqlite3_parser crate to turso_sqlite3_parser 2025-06-29 12:34:46 +03:00
Pekka Enberg
53ba3ff926 Rename limbo_core crate to turso_core 2025-06-29 09:59:17 +03:00
Nils Koch
2827b86917 chore: fix clippy warnings 2025-06-23 19:52:13 +01:00
Pekka Enberg
882c5ca168 Merge 'Simple integrity check on btree' from Pere Diaz Bou
This PR adds support for the instruction `IntegrityCk` which performs an
integrity check on the contents of a single table. Next PR I will try to
implement the rest of the integrity check where we would check indexes
containt correct amount of data and some more.
<img width="1151" alt="image" src="https://github.com/user-
attachments/assets/29d54148-55ba-480f-b972-e38587f0a483" />

Closes #1719
2025-06-16 13:46:26 +03:00
Jussi Saurio
3f90fad131 Merge 'Simulator Ast Generation + Simulator Unary Operator + Refactor to use limbo_core::Value in Simulator for massive code reuse' from Pedro Muniz
This PR is a Drop-In replacement to the Predicate defined in the
Simulator. Predicate is basically the same as our ast::Expr, but it
supports a small number of the SQL expression syntax. By creating a
NewType that wraps ast::Expr we can tap into our already mostly
correctly defined parser structs. This change will enable us to easily
add generation for more types of sql queries.
I also added an ArbitraryFrom impl for ast::Expr that can be used in a
freestyle way (for now) for differential testing.
This PR also aims to implement Unary Operator logic similar to the
Binary Operator logic we have for predicate. After this change we may
need to adjust the Logic for how some assertions are triggered.
<s>Sometimes the `Select-Select-Optimizer` property thinks that these
two queries should return the same thing:
```sql
SELECT (twinkling_winstanley.sensible_federations > x'66616e7461737469625e0f37879823db' AND twinkling_winstanley.sincere_niemeyer < -7428368947470022783) FROM twinkling_winstanley WHERE 1;

SELECT * FROM twinkling_winstanley WHERE twinkling_winstanley.sensible_federations > x'66616e7461737469625e0f37879823db' AND twinkling_winstanley.sincere_niemeyer < -7428368947470022783;
```
However after running the shrunk plan manually, the simulator was
incorrect in asserting that. Maybe this a bug a in the generation of
such a query? Not sure yet. </s>
<b>EDIT: The simulator was correctly catching a bug and I thought I was
the problem. The bug was in `exec_if` and I fixed it in this PR.</b>
I still need to expand the Unary Operator generation to other types of
predicates. For now, I just implemented it for `SimplePredicate` as I'm
trying to avoid to bloat even more this PR.
<b>EDIT: I decided to just have one PR open for all the changes I'm
making to make my life a bit easier and to avoid merge conflicts with my
own branches that I keep spawning for new code.</b>
PS: This should only be considered for merging after
https://github.com/tursodatabase/limbo/pull/1619 is merged. Then, I will
remove the draft status from this PR.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1674
2025-06-13 11:23:10 +03:00
Levy A.
b88cb99ff0 fix warnings and some refactoring 2025-06-11 14:19:06 -03:00
Levy A.
49a6ddad97 wip 2025-06-11 14:19:04 -03:00
Levy A.
fa621115b5 fix: broken test 2025-06-11 14:17:36 -03:00
Levy A.
f92e000277 fix: remove unused variable 2025-06-11 14:17:36 -03:00
Levy A.
91f981a8b1 fix(compat): sqlite firt checks if old_table exists
something like `ALTER TABLE a RENAME TO a` should fail with
`no such table: a` if `a` doesn't exists.
2025-06-11 14:17:36 -03:00
Pere Diaz Bou
c2541c0e52 fix order pragma integrity check 2025-06-11 18:59:25 +02:00
pedrocarlo
470093ca03 fix generation for string literal 2025-06-11 11:32:17 -03:00
pedrocarlo
f535ff1398 add optional serde implementations for parser ast 2025-06-11 11:32:17 -03:00
pedrocarlo
acb8a47911 sanitize string for ast::Literal 2025-06-11 11:31:47 -03:00
Pere Diaz Bou
9383ba207d introduce integrity_check pragma 2025-06-11 11:14:29 +02:00
Krishna Vishal
0d5cbc4f1d Add affinity check as a function as ast::Operator impl 2025-06-11 00:33:48 +05:30
pedrocarlo
36f60e4dd1 Fix rowid to_sql_string printing 2025-06-10 10:48:05 -03:00
Pere Diaz Bou
8cd7c7e82e Merge 'fix: make keyword_token safe by validating UTF-8 input' from ankit
This PR fixes an unsound usage of unsafe {
str::from_utf8_unchecked(word) } in the public function keyword_token in
mod.rs.
The function now uses std::str::from_utf8(word).ok()? to safely handle
invalid UTF-8, eliminating the unsoundness.
No logic or API changes.
Code compiles and tests pass (where possible).
Closes: https://github.com/tursodatabase/libsql/issues/1859

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1677
2025-06-09 16:07:37 +02:00
Jussi Saurio
eec7c0529c Merge 'Beginnings of AUTOVACUUM' from Zaid Humayun
This PR adds the beginnings of
[AUTOVACUUM](https://www.sqlite.org/lang_vacuum.html) to Limbo. It adds
a feature flag called `omit_autovacuum` which is analogous to
`SQLITE_OMIT_AUTOVACUUM`. It is off by default, same as SQLite.
It introduces the concept of [pointer map pages](https://www.sqlite.org/
fileformat.html#pointer_map_or_ptrmap_pages) which are reverse index
pages used to map pages to their parents. This is used to swap pages
(when a table is deleted for instance) to keep root pages clustered at
the beginning of the file. It's also used while creating a table to
ensure that root pages are clustered at the beginning (although, this
isn't completely implemented yet)
Finally, it also adds a couple of missing instructions like `Int64` that
are required for `PRAGMA` commands related to `auto_vacuum` settings
<img width="1512" alt="Screenshot 2025-05-28 at 8 47 51 PM"
src="https://github.com/user-
attachments/assets/d52eb74f-5b79-4d52-9401-1bdc2dcc304d" />

Closes #1600
2025-06-09 08:20:24 +03:00
Zaid Humayun
5827a33517 Beginnings of AUTOVACUUM
This commit introduces AUTOVACUUM to Limbo. It introduces the concept of ptrmap pages and also adds some additional instructions that are required to make AUTOVACUUM PRAGMA work
2025-06-06 23:14:22 +05:30
ankit
4c3c72b666 fix: make keyword_token safe by validating UTF-8 input 2025-06-06 16:25:49 +05:30
pedrocarlo
22d1a1eaa8 fix blob printing 2025-06-04 12:06:43 -03:00
pedrocarlo
ebee9516ba clippy 2025-06-04 12:06:43 -03:00
pedrocarlo
5f379fe2d6 when no context is needed use Display Impl 2025-06-04 12:06:43 -03:00
pedrocarlo
f90bebbfbc small fix and remove dbg 2025-06-04 12:06:43 -03:00