aljaz/turso
1
0
Fork 0
mirror of https://github.com/aljazceru/turso.git synced 2026-01-06 09:44:21 +01:00
Code Issues Packages Projects Releases Wiki Activity
8,371 Commits 1 Branch 0 Tags
2addeb5a9f73c79485e5a66d28be87db75f3df65
Commit Graph

8371 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Pekka Enberg
2addeb5a9f Merge 'introduce eq/contains/starts_with/ends_with_ignore_ascii_case macros' from Lâm Hoàng Phúc 
depend on #2865
```sh
`ALTER TABLE _ RENAME TO _`/limbo_rename_table/
                        time:   [10.100 ms 10.191 ms 10.283 ms]
                        change: [-16.770% -15.559% -14.417%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild

`ALTER TABLE _ RENAME COLUMN _ TO _`/limbo_rename_column/
                        time:   [7.4829 ms 7.5492 ms 7.6128 ms]
                        change: [-19.397% -18.093% -16.789%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) low mild
  1 (1.00%) high mild

`ALTER TABLE _ ADD COLUMN _`/limbo_add_column/
                        time:   [5.3255 ms 5.3713 ms 5.4183 ms]
                        change: [-24.002% -22.612% -21.195%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 39 outliers among 100 measurements (39.00%)
  17 (17.00%) low severe
  1 (1.00%) low mild
  1 (1.00%) high mild
  20 (20.00%) high severe

`ALTER TABLE _ DROP COLUMN _`/limbo_drop_column/
                        time:   [5.8858 ms 5.9183 ms 5.9510 ms]
                        change: [-16.233% -14.679% -13.083%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 25 outliers among 100 measurements (25.00%)
  8 (8.00%) low severe
  11 (11.00%) low mild
  2 (2.00%) high mild
  4 (4.00%) high severe

Prepare `SELECT 1`/limbo_parse_query/SELECT 1
                        time:   [590.28 ns 591.31 ns 592.35 ns]
                        change: [-3.7810% -3.5059% -3.2444%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) low severe
  6 (6.00%) high mild

Prepare `SELECT * FROM users LIMIT 1`/limbo_parse_query/SELECT * FROM users LIMIT 1
                        time:   [1.2569 µs 1.2582 µs 1.2596 µs]
                        change: [-5.0125% -4.7516% -4.4933%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  3 (3.00%) low severe
  2 (2.00%) low mild
  1 (1.00%) high mild
  1 (1.00%) high severe

Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...
                        time:   [3.7180 µs 3.7227 µs 3.7274 µs]
                        change: [-3.0557% -2.7642% -2.4761%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
  2 (2.00%) low mild
  4 (4.00%) high mild

Execute `SELECT 1`/limbo_execute_select_1
                        time:   [27.455 ns 27.477 ns 27.499 ns]
                        change: [-2.9461% -2.7493% -2.5589%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild
  1 (1.00%) high severe

Execute `SELECT * FROM users LIMIT ?`/limbo_execute_select_rows/1
                        time:   [410.53 ns 411.05 ns 411.54 ns]
                        change: [-15.364% -15.133% -14.912%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  4 (4.00%) low mild
  1 (1.00%) high mild

Execute `SELECT * FROM users LIMIT ?`/limbo_execute_select_rows/10
                        time:   [2.1100 µs 2.1122 µs 2.1145 µs]
                        change: [-11.517% -11.065% -10.662%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) low severe
  2 (2.00%) low mild

Execute `SELECT * FROM users LIMIT ?`/limbo_execute_select_rows/50
                        time:   [9.5156 µs 9.5268 µs 9.5383 µs]
                        change: [-10.284% -10.086% -9.8833%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) low severe
  2 (2.00%) low mild

Execute `SELECT * FROM users LIMIT ?`/limbo_execute_select_rows/100
                        time:   [18.669 µs 18.698 µs 18.731 µs]
                        change: [-9.5949% -9.3407% -9.1140%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) low severe
  1 (1.00%) high mild

Execute `SELECT count() FROM users`/limbo_execute_select_count
                        time:   [7.1027 µs 7.1098 µs 7.1170 µs]
                        change: [-43.739% -43.596% -43.469%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  2 (2.00%) low mild
  5 (5.00%) high mild
  2 (2.00%) high severe

```

Closes #2866
 2025-09-02 18:35:14 +03:00
Pekka Enberg
adc6cb008a Merge 'introduce match_ignore_ascii_case macro' from Lâm Hoàng Phúc 
this PR converts generate code in `turso_parser`'s `build.rs` into macro
for reusability. `match_ignore_ascii_case` will generate trie-like tree
matching from normal match expression.
example:
```rust
    match_ignore_ascii_case!(match input {
        b"AB" => TokenType::TK_ABORT,
        b"AC" => TokenType::TK_ACTION,
        _ => TokenType::TK_ID,
    })
```
will generate:
```rust
    match input.get(0) {
        Some(b'A') | Some(b'a') => match input.get(1) {
            Some(b'B') | Some(b'b') => match input.get(2) {
                None => TokenType::TK_ABORT,
                _ => TokenType::TK_ID,
            },
            Some(b'C') | Some(b'c') => match input.get(2) {
                None => TokenType::TK_ACTION,
                _ => TokenType::TK_ID,
            },
            _ => TokenType::TK_ID,
        },
        _ => TokenType::TK_ID,
    }
```

Closes #2865
 2025-09-02 18:34:55 +03:00
Pekka Enberg
d77b76e75a Merge 'string sometimes used as identifier quoting' from Lâm Hoàng Phúc 
fix https://github.com/tursodatabase/turso/issues/2886#issuecomment-
3244885481

Closes #2894
 2025-09-02 18:34:43 +03:00
Pekka Enberg
36909d4551 Merge 'CURRENT_TIMESTAMP can fallback TK_ID' from Lâm Hoàng Phúc 
fix #2875

Closes #2893
 2025-09-02 18:34:19 +03:00
Pekka Enberg
52ef7dd675 Merge 'Fix memory leak in page cache during balancing' from Preston Thorpe 
Currently we have `Pager::update_dirty_loaded_page_in_cache` which does
exactly what you would expect, but `DumbLruPageCache::_insert` method
with `ignore_existing` set to true, totally ignores the previous entry
and leaks the memory.
I really want to get #2885 finished and through because of the perf, but
I ran into this when inspecting it for correctness changes

Closes #2892
 2025-09-02 18:32:56 +03:00
Pekka Enberg
4e5274e646 Merge 'core: Make strict schema support experimental' from Pekka Enberg 
It's not tested properly so let's mark it as experimental for now.
Fixes #2775

Closes #2891
 2025-09-02 18:31:31 +03:00
TcMits
635402fc6f string sometimes used as identifier quoting 2025-09-02 21:35:37 +07:00
PThorpe92
cfadc4f579 Fix memory leak in page cache during balancing 2025-09-02 10:35:04 -04:00
TcMits
d0cb3d0d08 CURRENT_TIMESTAMP can fallback TK_ID 2025-09-02 20:50:58 +07:00
Pekka Enberg
12cf4d2e72 core: Make strict schema support experimental 
It's not tested properly so let's mark it as experimental for now.

Fixes #2775
 2025-09-02 16:40:02 +03:00
TcMits
40adf3fcfd Merge branch 'perf-3' into perf-4 2025-09-02 18:47:05 +07:00
TcMits
53d239ad1e resolve conflict 2025-09-02 18:46:41 +07:00
TcMits
bc8b848aba docs 2025-09-02 18:43:12 +07:00
TcMits
bfff05faba merge main 2025-09-02 18:25:20 +07:00
Pekka Enberg
8f7e43b32b scripts/publish-crates.sh: Remove turso_sqlite3_parser package 2025-09-02 14:21:04 +03:00
Pekka Enberg
3f3125c4f4 Merge 'remove turso_sqlite3_parser from turso_parser' from Lâm Hoàng Phúc 
Closes #2888
 2025-09-02 14:20:39 +03:00
TcMits
06e14c8ace merge main 2025-09-02 18:17:37 +07:00
Pekka Enberg
8204fbc8ec simulator: Fix 64-bit offset build failures 
Fix brekage from first merging commit d959319b ("Merge 'Use u64 for file
offsets in I/O and calculate such offsets in u64' from Preston Thorpe")
and then commit 6591b66c ("Merge 'Simulate I/O in memory' from Pedro
Muniz"), which was unaware of the changes.
 2025-09-02 14:14:04 +03:00
TcMits
d298480e4a Merge branch 'main' into perf-3 2025-09-02 18:13:58 +07:00
TcMits
07feacbc76 remove turso_sqlite3_parser from turso_parser 2025-09-02 18:10:28 +07:00
Pekka Enberg
6591b66c3d Merge 'Simulate I/O in memory' from Pedro Muniz 
Revives the `MemorySim` PR and fixes a page cache issue where we could
have a unlocked and unloaded page in the page cache after a FaultyQuery.
The page would continue in the cache and could affect other queries as
the `page_cache` is at the `Connection` level.
Depends on #2785

Closes #2693
 2025-09-02 13:28:48 +03:00
Pekka Enberg
3ec6f37555 scripts: Add turso_sqlite3_parser back to publish-crates.sh 
Turns out turso_parser depends on it.
 2025-09-02 12:52:40 +03:00
Pekka Enberg
91a51c170e Fix up turso_parser version in Cargo.toml 2025-09-02 12:47:49 +03:00
Pekka Enberg
6c7936a016 Turso 0.1.5-pre.2 2025-09-02 12:45:41 +03:00
Pekka Enberg
adaf0bec6c scripts: Fix publish-crates.sh to publish new parser crate 2025-09-02 12:45:16 +03:00
Pekka Enberg
483cd92dab Drop removed Go bindings from Antithesis Dockerfile 2025-09-02 09:48:53 +03:00
Pekka Enberg
15d45e3f68 Merge 'Refactor encryption to manage authentication tag internally' from bit-aloo 
This PR updates the internal encryption framework to handle
authentication tags explicitly rather than relying on the underlying
cipher libraries to append/verify them automatically.
closes: #2850

Reviewed-by: Avinash Sajjanshetty (@avinassh)

Closes #2858
 2025-09-02 09:44:22 +03:00
Pekka Enberg
7189e98455 Merge 'Unify handling of grouped and ungrouped aggregations' from Piotr Rżysko 
The initial commits fix issues and plug gaps between ungrouped and
grouped aggregations.
The final commit consolidates the code that emits `AggStep` to prevent
future disparities between the two.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2867
 2025-09-02 09:11:40 +03:00
Pekka Enberg
0868af29df Merge 'core/printf: support for more basic substitution types' from Luiz Gustavo 
Some progress working on `printf` support.
(relevant issue https://github.com/tursodatabase/turso/issues/885)
Implementation of the basic substitution types cited in the `TODO`
comment on the beginning of the file (%i, %x, %X, %o, %e, %E, %c). There
are some others in the sqlite spec which I will implement in a future
PR.
I tried to pay attention to the specific behaviors from sqlite as much
as possible while testing this, but if there's something I missed please
tell me.
Also, I see this code needs to be reorganized already, I'm still
thinking on the best approach to do that without affecting the
ergonomics of new implementations, I'm still learning Rust so this is
not obvious for me right now. I'm open to suggestions about it.

Closes #2868
 2025-09-02 09:10:03 +03:00
Pekka Enberg
1f6ba4f822 Merge 'Fix sim-schema command ' from Pedro Muniz 
Create the directory to store the schema if it does not exists

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2882
 2025-09-02 09:08:57 +03:00
Pekka Enberg
87d3f74e6e Merge 'Evict page from cache if page is unlocked and unloaded' from Pedro Muniz 
Because we can abort a read_page completion, this means a page can be in
the cache but be unloaded and unlocked. However, if we do not evict that
page from the page cache, we will return an unloaded page later which
will trigger assertions later on. This is worsened by the fact that page
cache is not per `Statement`, so you can abort a completion in one
Statement, and trigger some error in the next one if we don't evict the
page in these circumstances.
Also, to propagate IO errors we need to return the Error from
IOCompletions on step.

Closes #2785
 2025-09-02 09:08:12 +03:00
Pekka Enberg
d959319b42 Merge 'Use u64 for file offsets in I/O and calculate such offsets in u64' from Preston Thorpe 
Using `usize` to compute file offsets caps us at ~16GB on 32-bit
systems. For example, with 4 KiB pages we can only address up to 1048576
pages; attempting the next page overflows a 32-bit usize and can wrap
the write offset, corrupting data. Switching our I/O APIs and offset
math to u64 avoids this overflow on 32-bit targets

Closes #2791
 2025-09-02 09:06:49 +03:00
Pekka Enberg
cfaba4ab10 Merge 'Implement libSQL's ALTER COLUMN extension' from Levy A. 
Implement `ALTER COLUMN` as described here:
https://github.com/tursodatabase/libsql/blob/main/libsql-
sqlite3/doc/libsql_extensions.md#altering-columns
- [x] Add `ALTER COLUMN` to parser
- [x] Implement `Insn::AlterColumn`
- [x] Add tests

Closes #2814
 2025-09-02 09:06:03 +03:00
Pekka Enberg
da81e43f9a Merge 'Document how to use CDC' from Pavan Nambi 
Fixes #2221

Reviewed-by: Nikita Sivukhin (@sivukhin)

Closes #2871
 2025-09-02 09:04:44 +03:00
Pekka Enberg
149942abe4 Merge 'Return sqlite_version() without being initialized' from Preston Thorpe 
needed to support opening in Gorm

Closes #2878
 2025-09-02 09:04:25 +03:00
Pekka Enberg
a21d580f80 Merge 'Upgrade Rust version in simulator build Dockerfile' from Preston Thorpe 
otherwise simulator profiles will not build as it uses the new if let
syntax

Closes #2880
 2025-09-02 09:04:03 +03:00
pedrocarlo
72d697e0eb fix sim-schema command by creating directory if it not exists 2025-09-01 21:05:25 -03:00
pedrocarlo
51a54d3c33 Fd should be part of Operation struct 2025-09-01 16:53:07 -03:00
PThorpe92
be2807b61f Upgrade rust version in simulator build dockerfile 2025-09-01 14:56:05 -04:00
PThorpe92
e9b50b63fb Return sqlite_version() without being initialized 2025-09-01 13:36:41 -04:00
Preston Thorpe
f3d252d665 Merge 'Parse booleans to integer literals in expressions' from Preston Thorpe 
Adds tests for edgecases for the insane cases where someone might want
to name a table `true` :)

Closes #2874
 2025-09-01 13:36:29 -04:00
pedrocarlo
1eb1171f55 do not fault on Fsync until we correctly define the expected behaviour in the simulator 2025-09-01 14:12:11 -03:00
pedrocarlo
c158db072b inject fault in the IO Operation in the MemorySim 2025-09-01 14:12:11 -03:00
pedrocarlo
be855a8059 IOCompletions: abort other remaining completions if previous one errors 2025-09-01 14:12:11 -03:00
pedrocarlo
bd8cfe40f1 impl remove_file for MemoryIO + skip completion if finished 2025-09-01 14:12:11 -03:00
pedrocarlo
cc3488bba0 print path in stats for memory IO 2025-09-01 14:12:11 -03:00
pedrocarlo
ff51965c3e fix shrinking to not include some properties that are present in faulty query, but that are not relevant to the error 2025-09-01 14:12:11 -03:00
pedrocarlo
fd4e74929a Cli option to enable memory IO 2025-09-01 14:12:11 -03:00
pedrocarlo
117451fba1 impl WriteV for MemorySim 2025-09-01 14:12:11 -03:00
PThorpe92
b76f9b7733 Use eq_ignore_ascii_case in place of allocating new string in parser 2025-09-01 12:32:33 -04:00