Commit Graph

5323 Commits

Author SHA1 Message Date
Pekka Enberg
581d143595 github: Disable TPC-H benchmarks from CI
With indexing disabled, TPC-H benchmarks take forever to run. Disable
them for now.
2025-06-18 08:43:54 +03:00
Pekka Enberg
aee8ee6d3a Merge ' index_experimental flag to enable index usages ' from Pere Diaz Bou
Currently indexes are the bulk of the problem with `UPDATE` and
`DELETE`, while we work on fixing those it makes sense to disable
indexing since they are not stable. We want to try to make everything
else stable before we continue with indexing.
There is a small hack in Tcl tests where we check `if {[info exists
::env(SQLITE_EXEC)] && $::env(SQLITE_EXEC) eq "scripts/limbo-
sqlite3-index-experimental"} {`
to ensure we run those tests only with the script index-experimental.

Closes #1758
2025-06-18 08:42:28 +03:00
Pere Diaz Bou
871eee109f test_limbo_cli use SQLITE_EXEC if possible 2025-06-17 21:26:32 +02:00
Pekka Enberg
4b208f819e Merge 'antithesis: Build Python package from sources' from Pekka Enberg
Fixes #1630

Closes #1768
2025-06-17 22:14:54 +03:00
Pekka Enberg
4cbea9fcbb Add Blacksmith to README.md
Limbo's CI is powered by Blacksmith!
2025-06-17 22:14:11 +03:00
Pere Diaz Bou
34592b172c run index tests with flags instead of ignore 2025-06-17 19:33:23 +02:00
Pere Diaz Bou
b86491c54f more integer pk in java tests 2025-06-17 19:33:23 +02:00
Pere Diaz Bou
de1f29dadf core_tester index fuzz tests with flag 2025-06-17 19:33:23 +02:00
Pere Diaz Bou
26eb63b09f use integer pk in java tests 2025-06-17 19:33:23 +02:00
Pere Diaz Bou
4514bd5681 use integer primary key on test_in_memory_fetchone_select_all_users 2025-06-17 19:33:23 +02:00
Pere Diaz Bou
814f68043d filter out sqlite3 executable too 2025-06-17 19:33:23 +02:00
Pere Diaz Bou
e90996783b clippy 2025-06-17 19:33:23 +02:00
Pere Diaz Bou
878e39eee0 skip python write tests without indexes 2025-06-17 19:33:23 +02:00
Pere Diaz Bou
032337745b disable more tests without index 2025-06-17 19:33:23 +02:00
Pere Diaz Bou
4b88d47aba disable last unique_insert tcl tests 2025-06-17 19:33:23 +02:00
Pere Diaz Bou
48ae6766d7 fix comp errors 2025-06-17 19:33:23 +02:00
Pere Diaz Bou
133827e9f2 fix delete test with primary key 2025-06-17 19:33:23 +02:00
Pere Diaz Bou
f91d2c5e99 fix disable in write cases 2025-06-17 19:33:23 +02:00
Pere Diaz Bou
b5f2f375b8 disable alter, delete, create index, insert and update for indexes 2025-06-17 19:33:23 +02:00
Pere Diaz Bou
63b37ea6f2 re-enable tests that work with indexes 2025-06-17 19:33:23 +02:00
Pere Diaz Bou
bcbce15d7b disable UNION deduplication 2025-06-17 19:33:23 +02:00
Pere Diaz Bou
dde93e8deb disable distinct without index_experimental
distinct uses indexes, therefore we need to disable them
2025-06-17 19:33:23 +02:00
Pere Diaz Bou
9ae4563bcd index_experimental flag to enable index usages
Currently indexes are the bulk of the problem with `UPDATE` and
`DELETE`, while we work on fixing those it makes sense to disable
indexing since they are not stable. We want to try to make everything
else stable before we continue with indexing.
2025-06-17 19:33:23 +02:00
Pekka Enberg
a69369a62d Merge 'overwrite sqlite3 in install_sqlite' from Pere Diaz Bou
It looks like its possible to have sqlite3 persisted between runs
somehow, let's make sure we overwrite it.
Context:
```
Prepare all required actions
Run ./.github/shared/install_sqlite
Run curl -o /tmp/sqlite.zip
https://sqlite.org/$YEAR/sqlite-tools-linux-x64-$SQLITE_VERSION.zip >
/dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time
Current
                                 Dload  Upload   Total   Spent    Left
Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--
0
  2 3292k    2 98304    0     0   326k      0  0:00:10 --:--:--  0:00:10
326k
100 3292k  100 3292k    0     0  6686k      0 --:--:-- --:--:-- --:--:--
6678k
replace /usr/local/bin/sqlite3? [y]es, [n]o, [A]ll, [N]one, [r]ename:
NULL
(EOF or read error, treating as "[N]one" ...)
Archive:  /tmp/sqlite.zip
Error: Process completed with exit code 1.
```

Closes #1766
2025-06-17 20:32:40 +03:00
Pekka Enberg
9b5f5e213a antithesis: Build Python package from sources
Fixes #1630
2025-06-17 20:28:00 +03:00
Pere Diaz Bou
11275076a1 overwrite sqlite3 in install_sqlite
It looks like its possible to have sqlite3 persisted between runs
somehow, let's make sure we overwrite it.

Context:
```
Prepare all required actions
Run ./.github/shared/install_sqlite
Run curl -o /tmp/sqlite.zip
https://sqlite.org/$YEAR/sqlite-tools-linux-x64-$SQLITE_VERSION.zip >
/dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time
Current
                                 Dload  Upload   Total   Spent    Left
Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--
0
  2 3292k    2 98304    0     0   326k      0  0:00:10 --:--:--  0:00:10
326k
100 3292k  100 3292k    0     0  6686k      0 --:--:-- --:--:-- --:--:--
6678k
replace /usr/local/bin/sqlite3? [y]es, [n]o, [A]ll, [N]one, [r]ename:
NULL
(EOF or read error, treating as "[N]one" ...)
Archive:  /tmp/sqlite.zip
Error: Process completed with exit code 1.
```
2025-06-17 19:26:44 +02:00
Pekka Enberg
c944f81c69 Merge 'Implement pragma wal_checkpoint(<MODE>)' from Pedro Muniz
Only supported mode is PASSIVE

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1765
2025-06-17 18:29:05 +03:00
pedrocarlo
20115c1e74 return parse error when calling unimplemented pragma checkpoint modes 2025-06-17 11:42:20 -03:00
Pekka Enberg
7893ed7803 Update README.md 2025-06-17 13:55:23 +03:00
Pekka Enberg
9f4de5c4c5 Improve Limbo manual 2025-06-17 13:39:53 +03:00
Pekka Enberg
6aece7af2f Merge 'Fix incorrect lossy conversion of Value::Blob to a utf-8 String' from Luca Muscat
Hey 👋
I admire the work that you guys are doing here, great job so far!
`Value::Blob::encode_hex` incorrectly does a lossy conversion of the
blob's underlying byte vector into a utf8 String, incorrectly displaying
the underlying hex in the process.
This PR fixes issue #1751

Closes #1759
2025-06-17 11:36:55 +03:00
Luca Muscat
a5ac1884c1 core: Stop treating the contents of a Value::Blob as a String
By encoding a Vec<u8> (vector of bytes), a lossy conversion from a
`Vec<u8>` to a `String` occurs. The lossy conversion leads to an
incorrect hex value to be displayed.

Avoid the lossy conversion and let the `hex` crate do its thing.
2025-06-16 21:10:17 +02:00
Pekka Enberg
15daa9e81a Merge 'stress: Run integrity check for every iteration' from Pekka Enberg
Depends on https://github.com/tursodatabase/limbo/pull/1754

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1755
2025-06-16 16:31:00 +03:00
Pekka Enberg
db4945eada Merge 'Fix update queries to set n_changes ' from Kim Seon Woo
- `Update` query doesn't update `n_changes`. Let's make it work
- Add `InsertFlags` to add meta information related to insert operations
- For update query, add `UPDATE` flag
- Currently, the update query executes `Insn::Delete` and `Insn::Insert`
internally, it increases `n_change` by 2. So, for the update query,
let's skip increasing `n_change` for the `Insn::Insert`
https://github.com/tursodatabase/limbo/issues/1681

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1683
2025-06-16 16:30:20 +03:00
Pekka Enberg
d148f66670 stress: Make sure we run integrity check after every step
Spotted by Jussi.
2025-06-16 16:18:52 +03:00
Pekka Enberg
08bb4665b6 stress: Run integrity check for every iteration 2025-06-16 15:08:49 +03:00
Pekka Enberg
23b2d59871 Merge 'bindings/rust: Fix Rows::next() I/O dispatcher handling' from Pekka Enberg
The `next()` function needs to be a loop to make sure we actually return
rows.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1754
2025-06-16 15:08:34 +03:00
Pekka Enberg
f594a9db5a Merge 'core: Clean up integrity_check()' from Pekka Enberg
Suggested by Jussi.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1753
2025-06-16 15:08:19 +03:00
Pekka Enberg
4496a0d08a core: Clean up integrity_check()
Suggested by Jussi.
2025-06-16 14:46:36 +03:00
Pekka Enberg
47e08d34bf bindings/rust: Fix Rows::next() I/O dispatcher handling
The `next()` function needs to be a loop to make sure we actually return rows.
2025-06-16 14:28:08 +03:00
Pekka Enberg
882c5ca168 Merge 'Simple integrity check on btree' from Pere Diaz Bou
This PR adds support for the instruction `IntegrityCk` which performs an
integrity check on the contents of a single table. Next PR I will try to
implement the rest of the integrity check where we would check indexes
containt correct amount of data and some more.
<img width="1151" alt="image" src="https://github.com/user-
attachments/assets/29d54148-55ba-480f-b972-e38587f0a483" />

Closes #1719
2025-06-16 13:46:26 +03:00
Pekka Enberg
8c100f03fb Merge 'Make SQLite bindings thread-safe' from Pekka Enberg
Wrap the `sqlite3` object members with a mutex like SQLite does to make
it thread-safe. Note that as future work we should make the threading
model configurable as per SQLite.

Closes #1748
2025-06-16 11:48:00 +03:00
Pekka Enberg
96c47667ea Merge 'Add abbreviated alias for .quit and .exit' from Krishna Vishal
Accidentally found that you can quit SQLite CLI just by typing `.q`,
`.qu` and `.qui` instead of full `.quit`.
IMO this will be an improvement to the DX.
I've also found bunch of other dot commands that work with abbreviated
aliases in SQLite. If this PR is okay then I will also add them here.

Closes #1720
2025-06-16 11:11:17 +03:00
Pekka Enberg
5c35810f78 sqlite3: Make Clippy happy 2025-06-16 11:07:39 +03:00
Pekka Enberg
4d6b5c2b88 Merge 'cli: fix panic of queries with less than 7 chars' from Nils Koch
Hey everyone! I think this project is really really cool and I am here
to answer @jussisaurio
[call](https://x.com/jussisaur/status/1930290889496129562).
I started tinkering around with limbo and ran into this panic. Any input
with less than 7 characters (`"explain".len()`) will trigger it. In my
case it was `show;`. The issue was introduced a week ago in #1678.
PS: As I [said on X](https://x.com/nilskch/status/1930362930605723818),
I would love to help you with Limbo. I have a lot of experience with
DSLs and LSPs, and I have recently become interested in databases. I
don't have much experience with databases yet, but I am happy to learn.
I see that you have a lot of stuff going on with DSLs at the moment, so
I would be productive straight away in that area. Happy to chat if you
need help :)

Closes #1730
2025-06-16 11:06:02 +03:00
Pekka Enberg
dd001f9444 Merge 'Return parse error instead of corrupt error for no such table' from Pedro Muniz
This is more in line with SQLite:
```sql
sqlite> insert into t values (randomblob(1024*1024 * 6));
Parse error: no such table: t
```

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #1744
2025-06-16 11:04:50 +03:00
Pekka Enberg
5b47c9932d sqlite3: Make sqlite3 thread safe with a mutex
Make the sqlite3 object thread-safe with a mutex, like SQLite does.
2025-06-16 11:03:21 +03:00
Pekka Enberg
5fb50245ed sqlite3: Clean up sqlite3_wal_checkpoint() function 2025-06-16 11:02:44 +03:00
Pekka Enberg
50fdf7105d sqlite3: Clean up sqlite3_extended_errcode() function 2025-06-16 11:02:44 +03:00
Pekka Enberg
6901b7bcd6 sqlite3: Clean up sqlite3_errmsg() function 2025-06-16 11:02:44 +03:00