Commit Graph

15 Commits

Author SHA1 Message Date
TcMits
954b58d8ac generate MAX_KEYWORD_LEN, MIN_KEYWORD_LEN 2025-07-04 17:11:26 +07:00
Pekka Enberg
725c3e4ddc Rename limbo_sqlite3_parser crate to turso_sqlite3_parser 2025-06-29 12:34:46 +03:00
pedrocarlo
946b59f4ee even better BadNumber 2025-04-11 11:01:35 -03:00
pedrocarlo
a2ca9e5a46 better BadNumber 2025-04-11 10:09:00 -03:00
pedrocarlo
4d1ecd2d50 better MalformedHexInteger 2025-04-11 09:44:02 -03:00
Pekka Enberg
ac54c35f92 Switch to workspace dependencies
...makes it easier to specify a version, which is needed for `cargo publish`.
2025-02-12 17:28:04 +02:00
krishvishal
2faaa9f719 Add fix for paser not stopping at first error encounter.
Fix is to track the state of encountering error in the `Parser` struct and set it to true whereever/whenever
we encounter an error. And the beginning of the next() we return `Ok(None)` to signal to `FallibleIterator` that we should stop parsing.

Fixes https://github.com/tursodatabase/limbo/issues/865
2025-02-04 04:45:34 +05:30
Jussi Saurio
a26fc1619a Merge 'Fix select X'1'; causes limbo to go in infinite loop' from Krishna Vishal
Closes https://github.com/tursodatabase/limbo/issues/730.
Fixed `blog_literal` function to make it return `TK_ILLEGAL` token which
in turn now causes parser to stop and return `UnrecognizedTokenError`.
Added `TK_ILLEGAL` to `TokenType` Enum.
Behavior now:
```sql
SELECT X'1';  -- Odd number of hex digits -> TK_ILLEGAL -> unrecognized token error

  × unrecognized token at (1, 12)
   ╭────
 1 │ SELECT X'1';
   ·        ▲
   ·        ╰── here
   ╰────

SELECT X'AB'; -- Valid blob -> TK_BLOB -> parses successfully
�
SELECT X'G1'; -- Invalid hex digit -> TK_ILLEGAL -> unrecognized token error

  × unrecognized token at (1, 13)
   ╭────
 1 │ SELECT X'G1';
   ·        ▲
   ·        ╰── here
   ╰────

```

Reviewed-by: Jussi Saurio (@jussisaurio)

Closes #736
2025-01-21 11:32:10 +02:00
Krishna Vishal
b43c1544d4 Tokenizer::split() function returns TK_ILLEGAL instead of Error. 2025-01-20 19:37:15 +05:30
Krishna Vishal
04fd5a40d6 Finalize the parser in the case of Error while running queries. This resets the parser stack and prevents triggering the assertion and thereby panic.
Closes https://github.com/tursodatabase/limbo/issues/742
2025-01-20 16:10:35 +05:30
Krishna Vishal
ee88781c2a chore: cargo fmt 2025-01-19 06:23:19 +05:30
Krishna Vishal
5aee588078 Fixed blog_literal function to make it return UnrecognisedToken error.
Added `TK_ILLEGAL` to `TokenType` Enum
2025-01-19 06:11:45 +05:30
Levy A.
9b8722f38e refactor: more well rounded implementation
`?0` parameters are now handled by the parser.
2025-01-15 16:53:26 -03:00
Samyak S Sarnayak
c09a0bcbf3 Nicer parse errors using miette
I noticed that the parse errors were a bit hard to read - only the nearest token and the line/col offsets were printed.

I made a first attempt at improving the errors using [miette](https://github.com/zkat/miette).
- Added derive for `miette::Diagnostic` to both the parser's error type and LimboError.
- Added miette dependency to both sqlite3_parser and core. The `fancy` feature is only enabled for CLI.

Some future improvements that can be made further:
- Add spans to AST nodes so that errors can better point to the correct token. See upstream issue: https://github.com/gwenn/lemon-rs/issues/33
- Construct more errors with offset information. I noticed that most parser errors are constructed with `None` as the offset.

Comparisons.
Before:
```
❯ cargo run --package limbo --bin limbo database.db --output-mode pretty
...
limbo> selet * from a;
[2025-01-05T11:22:55Z ERROR sqlite3Parser] near "Token([115, 101, 108, 101, 116])": syntax error
Parse error: near "selet": syntax error at (1, 6)
```

After:
```
❯ cargo run --package limbo --bin limbo database.db --output-mode pretty
...
limbo> selet * from a;
[2025-01-05T12:25:52Z ERROR sqlite3Parser] near "Token([115, 101, 108, 101, 116])": syntax error

  × near "selet": syntax error at (1, 6)
   ╭────
 1 │ selet * from a
   ·     ▲
   ·     ╰── syntax error
   ╰────

```
2025-01-05 17:56:59 +05:30
jussisaurio
3cc9d9d79f vendor sqlite3-parser (lemon-rs) 2024-11-16 20:08:59 +02:00