aljaz/turso
1
0
Fork 0
mirror of https://github.com/aljazceru/turso.git synced 2025-12-30 06:24:21 +01:00
Code Issues Packages Projects Releases Wiki Activity
3,474 Commits 1 Branch 0 Tags
a9099cd6a53323f3bf76f7d76abd34fe6d97ff4b
Commit Graph

1888 Commits

Author SHA1 Message Date
Pekka Enberg
a9099cd6a5 Merge 'Schema translation cleanups' from Pekka Enberg 
Closes #1161
 2025-03-24 11:09:08 +02:00
Pekka Enberg
65bf33023c core: Fix Destroy opcode root page handling 
The `p1` register points to the root page, not to a cursor.

Fixes #1136
 2025-03-24 10:54:49 +02:00
Pekka Enberg
0ec7dbc44e core: Move translate_create_table() to schema module 2025-03-24 10:44:41 +02:00
Pekka Enberg
0727f4aca6 core: Move temporary table handling to translate_create_table() 2025-03-24 10:38:55 +02:00
Pekka Enberg
7d4ac13926 core: Move translate_drop_table() to schema module 2025-03-24 10:37:02 +02:00
Pekka Enberg
31bbc5144a Merge 'Initial pass at UPDATE support' from Preston Thorpe 
This PR is to support `Update` queries. Follows sqlite behavior as much
as possible.
### limbo
```console
limbo> create table t (a,b,c);
limbo> explain update t set a = 1 where b = 2;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     18    0                    0   Start at 18
1     OpenWriteAsync     0     2     0                    0
2     OpenWriteAwait     0     0     0                    0
3     RewindAsync        0     0     0                    0
4     RewindAwait        0     17    0                    0   Rewind table t
5       Column           0     1     4                    0   r[4]=t.b
6       Ne               4     5     15                   0   if r[4]!=r[5] goto 15
7       RowId            0     6     0                    0   r[6]=t.rowid
8       IsNull           6     17    0                    0   if (r[6]==NULL) goto 17
9       Integer          1     1     0                    0   r[1]=1
10      Column           0     1     2                    0   r[2]=t.b
11      Column           0     2     3                    0   r[3]=t.c
12      MakeRecord       1     3     7                    0   r[7]=mkrec(r[1..3])
13      InsertAsync      0     7     6                    0
14      InsertAwait      0     0     0                    0
15    NextAsync          0     0     0                    0
16    NextAwait          0     5     0                    0
17    Halt               0     0     0                    0
18    Transaction        0     1     0                    0   write=true
19    Integer            2     5     0                    0   r[5]=2
20    Goto               0     1     0                    0
```
### sqlite
```console
sqlite> explain update t set a = 1 where b = 2;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     15    0                    0   Start at 15
1     Null           0     1     2                    0   r[1..2]=NULL
2     Noop           1     0     1                    0
3     OpenWrite      0     2     0     2              0   root=2 iDb=0; t
4     Rewind         0     14    0                    0
5       Column         0     1     5                    0   r[5]= cursor 0 column 1
6       Ne             6     13    5     BINARY-8       81  if r[5]!=r[6] goto 13
7       Rowid          0     2     0                    0   r[2]= rowid of 0
8       IsNull         2     14    0                    0   if r[2]==NULL goto 14
9       Integer        1     3     0                    0   r[3]=1
10      Column         0     1     4                    0   r[4]= cursor 0 column 1
11      MakeRecord     3     2     1                    0   r[1]=mkrec(r[3..4])
12      Insert         0     1     2     t              7   intkey=r[2] data=r[1]
13    Next           0     5     0                    1
14    Halt           0     0     0                    0
15    Transaction    0     1     1     0              1   usesStmtJournal=0
16    Integer        2     6     0                    0   r[6]=2
17    Goto           0     1     0                    0
```

Closes #1130
 2025-03-24 09:19:22 +02:00
Pekka Enberg
e8c0a6e728 Merge 'Various JSON improvements' from Ihor Andrianov 
Added jsonb_object, jsonb_array, json_insert, jsonb_insert.
MongoDB is sweating now.

Closes #1160
 2025-03-24 09:17:40 +02:00
PThorpe92
a0188e5163 Use bind_col_refs to rewrite the Id expressions 2025-03-23 22:18:41 -04:00
PThorpe92
1202653e76 Use normal conditional translation for update where clause 2025-03-23 19:20:14 -04:00
PThorpe92
3597b32e4b Resolve ambiguous columns in expr translator 2025-03-23 19:19:35 -04:00
PThorpe92
8455f612bd Possibly translate both sides of expr in update 2025-03-23 17:08:15 -04:00
PThorpe92
c83cc6dff2 Small nits/clippy errors in vdbe 2025-03-23 17:08:15 -04:00
PThorpe92
676ddd4fb6 Add logic to handle overwrite cell if insert to same rowid to support update 2025-03-23 17:08:14 -04:00
PThorpe92
ef878a2e20 Begin update implementation, add translation 2025-03-23 17:08:14 -04:00
Ihor Andrianov
b7be728d64 fix copypaste typo 2025-03-23 21:15:29 +02:00
Ihor Andrianov
479dd9c35a clippy 2025-03-23 21:11:35 +02:00
Ihor Andrianov
2cab36bfc3 add json_replace, jsonb_replace 2025-03-23 20:52:03 +02:00
Ihor Andrianov
c4549ad2cd split json traversal and mutation operation logic 2025-03-23 20:37:12 +02:00
l.gualtieri
a9ad5a56b9 fix bug #1155 2025-03-23 16:51:10 +01:00
Pekka Enberg
52ccc36061 Merge 'Impl Copy on some types in the pager to prevent explicit clones' from Preston Thorpe 
Tried to keep this as small and focused as possible, just a few that I
ran into while debugging the page cache

Closes #1107
 2025-03-21 18:40:14 +02:00
Pekka Enberg
f039ffe5c5 Merge 'Add commit() and placeholding insert to Python binding' from Diego Reis 
It is a Minimal Valuable Close (MVC) to #494 and #1146 , so probably
there are some corner cases to handle, if so, just point it out and I'll
be happy to fix it :)

Closes #1148
 2025-03-21 12:10:34 +02:00
Diego Reis
16b9325830 ext/python: Basic support for placeholding insert 2025-03-20 17:10:12 -03:00
Diego Reis
2481d73d70 ext/python: Partially implements commit() 
It was based on https://docs.python.org/3/library/sqlite3.html but some more work is needed specially in LEGACY_TRANSACTION_CONTROL and isolation levels.

See: https://docs.python.org/3/library/sqlite3.html#sqlite3.Connection.autocommit
 2025-03-20 17:09:55 -03:00
Ihor Andrianov
252583cb1e add jsonb_obj jsonb_array 2025-03-20 15:22:52 +02:00
Daniel Boll
fc50609491 Fix io_uring WAL write corruption by ensuring buffer lifetime 
Ensure the Arc<RefCell<Buffer>> in UringFile::pwrite remains alive until
the io_uring write completes by referencing it in the completion callback.
This prevents WAL file corruption where the correct buffer data was
overwritten with stale memory (e.g., 00 18 27 xx instead of 37 7f 06 82).

Validation:
- Tested with limbo -v io_uring and WAL operations.
- Verified with xxd and wal-browser.

Signed-off-by: Daniel Boll <danielboll.academico@gmail.com>
 2025-03-19 23:55:43 -03:00
Pekka Enberg
4142f4f4cb Merge 'Organize extension library and feature gate VFS' from Preston Thorpe 
I keep having 3+ PR's in at the same time and always deal with crazy
conflicts because everything in the `ext` library is together in one
file.
This PR moves each category of extension into its own file, and
separates the `vfs` functionality in Core into the `ext/dynamic` module,
so that it can be more easily separated from wasm (or non feature =
"fs") targets to prevent build issues.
The only semantic changes made in this PR is the feature gating of vfs,
the rest is simply organizing and cleaning up imports.
Was unsure if `vfs` should be a feature on the `core` side too, or to
just enable it with the `fs` feature which seemed reasonable, as that
was already the current behavior. But let me know if we want it entirely
behind it's own feature.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1124
 2025-03-19 19:08:13 +02:00
Pere Diaz Bou
13d518d086 fragment bench functions 2025-03-19 17:55:24 +01:00
Pekka Enberg
d4db5eb4c1 Merge 'Various JSON and JSONB function improvements' from Ihor Andrianov 
Added jsonb_remove, jsonb_replace, json_replace.
Updated json_remove to use jsonb under the hood.
Fixed json function big numbers serialization.
Add tests for new functions.

Closes #1140
 2025-03-19 17:21:44 +02:00
PThorpe92
57d4aa7216 Reorganize ext library and feature gate vfs to more easily prevent wasm build issues 2025-03-19 10:17:11 -04:00
Ihor Andrianov
ca0be110ec fix big number parsing for json functions 2025-03-19 12:53:34 +02:00
Ihor Andrianov
62916f3a97 cargo clippy 2025-03-19 11:59:20 +02:00
Ihor Andrianov
32ea972151 make tests pass 2025-03-19 11:29:46 +02:00
Ihor Andrianov
b5e86a9e36 remove and replace functions defenitions 2025-03-18 21:43:48 +02:00
Ihor Andrianov
779e2c9e97 high order functions for remove and replace 2025-03-18 21:43:14 +02:00
Ihor Andrianov
403d0e8c21 implemented remove and replace in jsonb struct 2025-03-18 21:42:27 +02:00
Pekka Enberg
76887af59e Merge 'Fix read completion error in VFS' from Preston Thorpe 
Closes #1126
 2025-03-18 20:18:48 +02:00
Pekka Enberg
a2c6831f30 Merge 'Implement FastLock for DatabaseHeader' from Pere Diaz Bou 
The motivation behind implementing our own lock is to not depend on any
dependency as we should moving forward. This is a experiment for now as
a single test obviously is not enough but I believe this is the right
direction to on.
## benchmark tldr;
Execute benchmarks have a performance improvement around [1.78%, 7.5%]
which seems like it went okay as it was expected from removing
`pthread_mutex` calls.
## benchmarks before
```
Prepare `SELECT 1`/Limbo/SELECT 1
                        time:   [575.63 ns 577.33 ns 580.07 ns]
                        change: [-1.3304% -0.8881% -0.4675%] (p = 0.00 < 0.05)
                        Change within noise threshold.

Prepare `SELECT * FROM users LIMIT 1`/Limbo/SELECT * FROM users LIMIT 1
                        time:   [1.2070 µs 1.2114 µs 1.2166 µs]
                        change: [-0.8670% -0.4084% -0.0252%] (p = 0.06 > 0.05)
                        No change in performance detected.

Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...
                        time:   [2.9845 µs 2.9895 µs 2.9951 µs]
                        change: [-3.0470% -2.6038% -2.1301%] (p = 0.00 < 0.05)
                        Performance has improved.
Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou... #2
                        time:   [1.6015 µs 1.6084 µs 1.6157 µs]
                        change: [-0.0676% +0.3850% +0.8704%] (p = 0.11 > 0.05)
                        No change in performance detected.
Execute `SELECT * FROM users LIMIT ?`/Limbo/1
                        time:   [442.46 ns 446.72 ns 454.13 ns]
                        change: [+3.9744% +4.5337% +5.3357%] (p = 0.00 < 0.05)
                        Performance has regressed.
Execute `SELECT * FROM users LIMIT ?`/Limbo/10
                        time:   [3.1722 µs 3.1850 µs 3.1980 µs]
                        change: [+7.1994% +7.7452% +8.2856%] (p = 0.00 < 0.05)
                        Performance has regressed.
Execute `SELECT * FROM users LIMIT ?`/Limbo/50
                        time:   [14.976 µs 15.024 µs 15.078 µs]
                        change: [+5.7879% +6.2419% +6.7139%] (p = 0.00 < 0.05)
                        Performance has regressed.
Execute `SELECT * FROM users LIMIT ?`/Limbo/100
                        time:   [29.834 µs 29.925 µs 30.024 µs]
                        change: [+4.6519% +5.0384% +5.4491%] (p = 0.00 < 0.05)
                        Performance has regressed.
Execute `SELECT 1`/Limbo
                        time:   [45.135 ns 45.439 ns 45.763 ns]
                        change: [-0.4703% -0.0496% +0.3622%] (p = 0.81 > 0.05)
                        No change in performance detected.
```
## benchmarks after
```
Prepare `SELECT 1`/Limbo/SELECT 1
                        time:   [585.61 ns 590.92 ns 596.49 ns]
                        change: [+0.5902% +1.1505% +1.7012%] (p = 0.00 < 0.05)
                        Change within noise threshold.

Prepare `SELECT * FROM users LIMIT 1`/Limbo/SELECT * FROM users LIMIT 1
                        time:   [1.2061 µs 1.2090 µs 1.2119 µs]
                        change: [-0.2364% +0.0977% +0.4252%] (p = 0.57 > 0.05)
                        No change in performance detected.

Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...
                        time:   [2.9854 µs 2.9893 µs 2.9936 µs]
                        change: [-0.5752% -0.2529% +0.0167%] (p = 0.09 > 0.05)
                        No change in performance detected.
Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou... #2
                        time:   [1.5853 µs 1.5983 µs 1.6108 µs]
                        change: [-2.3810% -1.7986% -1.2748%] (p = 0.00 < 0.05)
                        Performance has improved.
Execute `SELECT * FROM users LIMIT ?`/Limbo/1
                        time:   [429.84 ns 431.34 ns 433.07 ns]
                        change: [-2.7721% -1.8504% -0.8738%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Execute `SELECT * FROM users LIMIT ?`/Limbo/10
                        time:   [2.9184 µs 2.9254 µs 2.9323 µs]
                        change: [-8.2377% -7.7816% -7.3373%] (p = 0.00 < 0.05)
                        Performance has improved.
Execute `SELECT * FROM users LIMIT ?`/Limbo/50
                        time:   [14.190 µs 14.229 µs 14.271 µs]
                        change: [-6.2034% -5.7858% -5.3552%] (p = 0.00 < 0.05)
                        Performance has improved.
Execute `SELECT * FROM users LIMIT ?`/Limbo/100
                        time:   [28.734 µs 28.856 µs 28.979 µs]
                        change: [-4.3640% -3.9462% -3.5492%] (p = 0.00 < 0.05)
                        Performance has improved.

Execute `SELECT 1`/Limbo
                        time:   [43.144 ns 43.237 ns 43.326 ns]
                        change: [-4.9417% -4.5554% -4.2030%] (p = 0.00 < 0.05)
                        Performance has improved.
```

Closes #1120
 2025-03-18 13:44:16 +02:00
Pekka Enberg
f9d7834874 Merge 'Jsonb extract' from Ihor Andrianov 
Made a jsonb traversal by json path.
Changed some ordinary json functions to use jsonb under the hood, so now
behavior of our json module more like sqlite.
Found and fixed some bugs on the way.

Closes #1135
 2025-03-17 18:25:28 +02:00
Diego Reis
16396c57c7 Removes unnecessary clone 2025-03-17 10:06:14 -03:00
Diego Reis
250478fedf Implement deferred transactions 
As explained in docs: https://sqlite.org/lang_transaction.html

"BEGIN DEFERRED statement merely sets a flag on the database connection that turns off the automatic commit that would normally occur when the last statement finishes."

The transaction upgrade (read -> write) is already handled by the VDBE
 2025-03-17 10:01:00 -03:00
Diego Reis
2314e7f906 Improve explain output for Transaction bytecode. 
It isn't SQLite compliant but it helps a lot, specially when the user doesn't know what each register means.
 2025-03-17 09:50:22 -03:00
Pere Diaz Bou
829888f886 hint spin loop 2025-03-17 10:23:58 +01:00
Pere Diaz Bou
00ab3d1c0c Fix ordering and implement Deref 2025-03-17 10:22:42 +01:00
Pere Diaz Bou
cbc46ce2fe use compare_exchange 2025-03-17 10:21:34 +01:00
Pere Diaz Bou
20f5ade95e Experiment with a custom Lock for database header 2025-03-17 10:21:34 +01:00
Preston Thorpe
a2aca1f747 Merge branch 'tursodatabase:main' into main 2025-03-16 21:47:20 -04:00
Diego Reis
590f90ad9a Fix AutoCommit handling of an ongoing halt checkpoint 2025-03-16 15:35:49 -03:00
Ihor Andrianov
328ebfcd9e clippy 2025-03-16 15:31:06 +02:00
Ihor Andrianov
23d7d82b6c add jsonb_extract function 2025-03-16 15:14:29 +02:00
Ihor Andrianov
a878738a1e Fix jsonb bug with unclosed string and two commas in a row 2025-03-16 03:27:40 +02:00