Commit Graph

5716 Commits

Author SHA1 Message Date
pedrocarlo
56d87cb916 move disable behavior to connection instead of checkpoint 2025-07-03 12:05:53 -03:00
pedrocarlo
3861584edc add test for wal checkpointing correctly 2025-07-03 12:04:17 -03:00
pedrocarlo
44b8275b26 add test + implement sqlite3_column_int64 for test 2025-07-03 12:04:17 -03:00
pedrocarlo
db005c81a0 add option to disable wal checkpoint 2025-07-03 12:04:17 -03:00
Pekka Enberg
9e92325bad antithesis: Make experiments run for longer
...increases probability of Antithesis finding interesting things.
2025-07-03 15:58:49 +03:00
Pekka Enberg
90e035b6b0 Merge 'Rollback schema support' from Pere Diaz Bou
Fixes #1890
Once rollback was implement we quickly saw that it lacked support for
schema changes so we had to re-estructure things a bit.
## Example of failure:
```bash
turso> begin;
turso> create table t(x);
turso> rollback;
turso> pragma integrity_check;
thread 'main' panicked at core/storage/sqlite3_ondisk.rs:386:36:
called `Result::unwrap()` on an `Err` value: Corrupt("Invalid page type: 83")
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
This happened because it thought table `t` existed because we didn't
rollback that schema.
## Changes:
* The most important change: now every connection has a private copy of
schema. On write txn commit we update a global schema shared between
connections in order for new connections to get updated version from
there. In case of rollback, we simply change connection's schema to
previous version. This change allowed us to remove locks for schema
private copy and keeping schema changes locally in case of concurrency.
 Sqlite does things differently, they lazily parse schema in case of
outdated schema, this many schema changes to trigger reading schema from
db file which is slow. If we are able to keep local copy in memory, even
when if we add multiprocessing, it will speed up schema reloading by a
bunch.
* `schema_cookie` is now update for every schema change
* `Insn::ParseSchema` had a nasty bug where it would commit all the
changes made in a query that changed a schema, we fixed that by setting
`auto_commit` to `false` before parsing schema, and setting it back to
previous value once schema is parsed.

Closes #1928
2025-07-03 14:18:00 +03:00
Pekka Enberg
1b3317e6c8 Merge 'core/storage: Switch to turso_assert in btree.rs' from Pekka Enberg
Let's help out Antithesis to find interesting bugs.

Closes #1937
2025-07-03 13:58:34 +03:00
Pere Diaz Bou
5eca507867 fix type null spacing 2025-07-03 12:53:15 +02:00
Pere Diaz Bou
2cfd209e56 clippy 2025-07-03 12:40:08 +02:00
Pere Diaz Bou
151debcb63 fix to_sql btreetable 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
6b16950488 fmt 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
06685ac191 fix formatting create table in rollback.test 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
470fb8d23b rollabck translate remove querymode 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
cde7202981 Revert "Merge 'core: Disable ROLLBACK statement' from Pekka Enberg"
This reverts commit 8a13e4b02f, reversing
changes made to cc935f97cc.
2025-07-03 12:36:48 +02:00
Pere Diaz Bou
a6954ee9aa index test 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
f37893eb8f set cookie for index operations 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
ba988685cf set cookie create virtual table 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
5d856499c4 move update schema global on commit and not on rollback txn 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
2414502268 parse schema set auto_commit false in nested query 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
d8658264d9 alter set cookie 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
c1e87d7f24 more tests 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
9590768914 few more schema change rollback tests 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
c799396c3d rollback schema in connection 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
5b733663ab update schema in case it's outdated 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
65a7fe13cf remove lock from private schema copy 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
abf1699dd2 set scheam version and update shared schema in txn 2025-07-03 12:36:48 +02:00
Pekka Enberg
fa442ecd6e core/storage: Switch to turso_assert in btree.rs
Let's help out Antithesis to find interesting bugs.
2025-07-03 13:25:13 +03:00
Pekka Enberg
c76625eb64 Merge 'fix: buffer pool is not thread safe problem' from KaguraMilet
This PR will close #1446. Buffer pool implementation should be thread
safe after this PR.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1910
2025-07-03 13:12:21 +03:00
Pekka Enberg
471d26a632 Merge 'Fix index update when INTEGER PRIMARY KEY (rowid alias)' from Adrian-Ryan Acala
When an `UPDATE` statement modifies a table's `INTEGER PRIMARY KEY`
(which acts as a `rowid` alias) alongside other indexed columns, the
index entries were incorrectly retaining the old `rowid`. This led to
stale index references, causing subsequent queries to return incorrect
results.
This change ensures that when the `rowid` alias is part of the `SET`
clause in an `UPDATE` statement, the new `rowid` value is used for
generating and updating index records. This guarantees that all index
entries correctly point to the updated row, resolving the data
inconsistency.
Fixes #1897

Closes #1916
2025-07-03 13:10:53 +03:00
Pekka Enberg
ca990e8fd1 Revert "Merge 'Rust binding improvements' from Pedro Muniz"
This reverts commit bd60cd214c, reversing
changes made to 74e48a3a8f because it
makes limbo_stress hang.
2025-07-03 12:28:10 +03:00
Pekka Enberg
4e5de4b03f Merge 'Fix Python test import naming' from Pedro Muniz
When we renamed the Python imports, we forgot to rename the
`test_limbo_cli` to `test_turso_cli`. Also adds a vfs-bench-compile
github ci to compile and test that the bench-vfs scripts works

Closes #1933
2025-07-03 11:23:43 +03:00
KaguraMilet
f339e9c1ad fix integrity check error 2025-07-03 13:47:30 +08:00
KaguraMilet
562dd389db Merge branch 'tursodatabase:main' into buffer 2025-07-03 13:46:37 +08:00
pedrocarlo
c5bed21dff fix python test import naming 2025-07-03 02:15:08 -03:00
Pekka Enberg
3bd5d4c732 core: Drop debugging code 2025-07-02 19:55:38 +03:00
Pekka Enberg
e8af5f1022 Merge 'from_uri was not passing mvcc and indexes flag to database creation for memory path' from Pedro Muniz
Closes #1932
2025-07-02 19:55:27 +03:00
pedrocarlo
191f732088 from_uri was not passing mvcc and indexes flag to database creation for memory path 2025-07-02 13:46:49 -03:00
Pekka Enberg
36b550cca4 Merge 'Fix boxed memory leaks' from Ihor Andrianov
We should recreate original box to drop it properly
Also made a fast path for hashing. When key div by 2. It should decrease
cpu cycles on hot path by x10 approximately
This thing is tricky, made a long running test that verify bug, put
#[ignore] on it to not slow down CI

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #1873
2025-07-02 19:42:54 +03:00
Pekka Enberg
df257d3048 Merge 'Turso, not Limbo, in pyproject.toml' from Simon Willison
https://pypi.org/project/pyturso/0.1.1/ is still showing the old name:
![CleanShot 2025-07-01 at 07 30 11@2x](https://github.com/user-
attachments/assets/a80441ec-507c-4ff7-a698-3cb88625c2cc)

Closes #1912
2025-07-02 19:33:05 +03:00
Pekka Enberg
acc64820f2 Merge 'Rename Limbo -> Turso in python tests' from Preston Thorpe
Closes #1909
2025-07-02 19:32:56 +03:00
Pekka Enberg
9a27583dd7 Merge 'bindings/javascript: Formatting and typos' from Mikaël Francoeur
This is a follow-up PR to
https://github.com/tursodatabase/turso/pull/1907#discussion_r2175959782.
I had my formatter set to `prettier` instead of `tsserver`, which seems
to be what the team is using.
So I:
* removed the `prettier-ignore` comments that I added in the other PR;
* formatted js files using `tsserver`;
* formatted md files using `prettier` (it just makes the tables nicer);
* fixed some typos;
* added some formatting info to `CONTRIBUTING.md`.
-----
as part of https://github.com/tursodatabase/turso/issues/1900

Closes #1914
2025-07-02 19:32:30 +03:00
Pekka Enberg
eef118569f Merge 'clarify discord situation' from Glauber Costa
There is confusion (one could say, discord) between discords.

Closes #1929
2025-07-02 18:39:34 +03:00
Pekka Enberg
cbf234da6c Merge 'add a README for the rust bindings' from Glauber Costa
Closes #1931
2025-07-02 18:35:39 +03:00
Pekka Enberg
17e8aff188 Merge 'add a basic readme for the typescript binding' from Glauber Costa
Closes #1930
2025-07-02 18:35:27 +03:00
Glauber Costa
bff5a440dc add a README for the rust bindings 2025-07-02 10:18:23 -05:00
Glauber Costa
cdb1edec4e add a basic readme for the typescript binding 2025-07-02 10:03:53 -05:00
PThorpe92
0d80e3a21b Fix naming and ruff format check 2025-07-02 11:02:32 -04:00
PThorpe92
297cbbf726 Rename Limbo -> Turso in python tests 2025-07-02 10:57:46 -04:00
Glauber Costa
aae30b85a6 clarify discord situation
There is confusion (one could say, discord) between discords.
2025-07-02 08:49:05 -05:00
Pekka Enberg
5de904be47 Merge 'automatically select terminal colors for pretty mode' from Glauber Costa
I just tried turso and couldn't read the last column. Turns out I guess
Pekka's taste is not the best, at least not for everybody.
Auto-detect if terminal is light or dark mode and select colors
accordingly.

Closes #1922
2025-07-02 15:06:37 +03:00