Commit Graph

3392 Commits

Author SHA1 Message Date
Ihor Andrianov
f42b62f43c update compat for json functions 2025-03-19 13:00:55 +02:00
Ihor Andrianov
ca0be110ec fix big number parsing for json functions 2025-03-19 12:53:34 +02:00
Ihor Andrianov
62916f3a97 cargo clippy 2025-03-19 11:59:20 +02:00
Ihor Andrianov
32ea972151 make tests pass 2025-03-19 11:29:46 +02:00
Ihor Andrianov
bf649f98be add tests 2025-03-19 11:29:23 +02:00
Ihor Andrianov
b5e86a9e36 remove and replace functions defenitions 2025-03-18 21:43:48 +02:00
Ihor Andrianov
779e2c9e97 high order functions for remove and replace 2025-03-18 21:43:14 +02:00
Ihor Andrianov
403d0e8c21 implemented remove and replace in jsonb struct 2025-03-18 21:42:27 +02:00
Pekka Enberg
f9d7834874 Merge 'Jsonb extract' from Ihor Andrianov
Made a jsonb traversal by json path.
Changed some ordinary json functions to use jsonb under the hood, so now
behavior of our json module more like sqlite.
Found and fixed some bugs on the way.

Closes #1135
2025-03-17 18:25:28 +02:00
Ihor Andrianov
2fb18b4177 update compat 2025-03-17 16:20:43 +02:00
Pekka Enberg
7fc61b7b3d sqlite3: Add trace logging to sqlite3_exec() 2025-03-17 16:17:18 +02:00
Pekka Enberg
f82b1169a9 Fix Mobibench instructions to use WAL mode 2025-03-17 16:12:25 +02:00
Pekka Enberg
9435bd2f48 Merge 'Implement BEGIN DEFERRED' from Diego Reis
As explained in [docs](https://sqlite.org/lang_transaction.html):
> "DEFERRED means that the transaction does not actually start until the
database is first accessed. Internally, the BEGIN DEFERRED statement
merely sets a flag on the database connection that turns off the
automatic commit that would normally occur when the last statement
finishes. This causes the transaction that is automatically started to
persist until an explicit COMMIT or ROLLBACK or until a rollback is
provoked by an error or an ON CONFLICT ROLLBACK clause. If the first
statement after BEGIN DEFERRED is a SELECT, then a read transaction is
started. Subsequent write statements will upgrade the transaction to a
write transaction if possible, or return SQLITE_BUSY. If the first
statement after BEGIN DEFERRED is a write statement, then a write
transaction is started. "
The transaction upgrade `read -> write` is already handled by the VDBE
in `Transaction`.
closes #1001

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1133
2025-03-17 16:02:28 +02:00
Diego Reis
16396c57c7 Removes unnecessary clone 2025-03-17 10:06:14 -03:00
Diego Reis
250478fedf Implement deferred transactions
As explained in docs: https://sqlite.org/lang_transaction.html

"BEGIN DEFERRED statement merely sets a flag on the database connection that turns off the automatic commit that would normally occur when the last statement finishes."

The transaction upgrade (read -> write) is already handled by the VDBE
2025-03-17 10:01:00 -03:00
Diego Reis
2314e7f906 Improve explain output for Transaction bytecode.
It isn't SQLite compliant but it helps a lot, specially when the user doesn't know what each register means.
2025-03-17 09:50:22 -03:00
Pekka Enberg
51237b92bc Merge 'Fix AutoCommit for IO halting' from Diego Reis
IDK if I'm being naive here but it was the easiest way that I found to
handle it.
closes #1004

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1129
2025-03-17 11:56:00 +02:00
Pekka Enberg
fd8ea0179d Merge 'bindings/python: Update PyO3 dependency to 0.24.0' from Pekka Enberg
Closes #1131
2025-03-17 11:16:36 +02:00
Pekka Enberg
a81ed4a523 bindings/python: Update PyO3 dependency to 0.24.0 2025-03-17 10:58:33 +02:00
Diego Reis
590f90ad9a Fix AutoCommit handling of an ongoing halt checkpoint 2025-03-16 15:35:49 -03:00
Ihor Andrianov
328ebfcd9e clippy 2025-03-16 15:31:06 +02:00
Ihor Andrianov
23d7d82b6c add jsonb_extract function 2025-03-16 15:14:29 +02:00
Ihor Andrianov
a3a9376347 unblock some tests 2025-03-16 03:36:02 +02:00
Ihor Andrianov
a878738a1e Fix jsonb bug with unclosed string and two commas in a row 2025-03-16 03:27:40 +02:00
Ihor Andrianov
0b22fbd566 Add jsonb to json_valid 2025-03-16 03:26:08 +02:00
Ihor Andrianov
dc6342c0de Add jsonb impl to existing json functions where possible 2025-03-16 02:08:49 +02:00
Ihor Andrianov
e327707ac6 Fix json_path broken condition 2025-03-15 22:01:48 +02:00
Ihor Andrianov
353422a25c Add follow json path in jsonb 2025-03-15 22:00:12 +02:00
Pekka Enberg
9efcacc601 Merge 'Add jsonb bench and rewrite jsonb parser to use slice and cursor' from Ihor Andrianov
Added bench to see how far i am from SQLite. If ~~Cloude~~ I did benches
correct moving from iterator to slice and cursor gives crazy performance
bump where we are almost on par with SLQLite when we are not talking
about huge jsons, but i suspect SQLite do some trickery there.

Closes #1125
2025-03-15 08:49:49 +02:00
Ihor Andrianov
b366045cb7 Rewrite parser to use slice and cursor instead of iterator 2025-03-14 19:47:18 +02:00
Ihor Andrianov
37c2655b6f Add jsonb bench 2025-03-14 19:46:29 +02:00
Pekka Enberg
731fbaf3c7 Merge 'Jsonb implementation' from Ihor Andrianov
This PR implements a complete JSONB parser and serializer as current PR
draft looks stale.
Sorry for huge PR.
I've choose a recursive parsing approach because:
1. It's simpler to understand and maintain
2. It follows SQLite's implementation pattern, ensuring compatibility
3. It naturally maps to JSON's hierarchical structure
The implementation includes comprehensive test coverage for standard
JSON features and JSON5 extensions. All test cases pass successfully,
handling edge cases like nested structures, escape sequences, and
various number formats.
While the code is ready for review, I believe it would benefit from fuzz
testing in the future to identify any edge cases not covered by the
current tests.
Ready for review, proposals and feedback.

Closes #1114
2025-03-13 21:17:52 +02:00
Pekka Enberg
b1419ab434 Merge 'Initial pass on Virtual FileSystem extension module' from Preston Thorpe
This PR implements a `VFS` module for our extension library, allowing
extensions to be written that can introduce different I/O back-ends.
EDIT: there is an included plain external/sync vfs example for testing,
as mentioned in #996 they can be combined after they are both merged and
we can keep 1 extension crate just for testing that features can be
added to, without making a new extension just to test stuff.
This PR also adds the `.vfslist` dot command, and replaces the `--io`
CLI argument with `--vfs` to match sqlite.
In order to support building vfs modules at compile time, and to then
support opening a brand new db file using a staticly built-in extension
module, a new method was created `open_with_vfs` that will load any vfs
modules before a `Database` is created, and uses that `IO` to create the
initial file, and returns it: `Result<(Arc<dyn IO>, Arc<Database>)>`. in
keeping with the API of core.
When #1039 is merged, the vfs module can be specified in a query
parameter.

Closes #960
2025-03-13 21:16:52 +02:00
Pekka Enberg
9f641f17c6 Disable some B-Tree fuzzers
Fuzz testing is great for finding bugs, but until we fix the bugs,
failing CI runs out of the blue for unrelated PRs is not very
productive. Hopefully we can enable this soon again, but until then,
let's not fail the test suite all the time randomly.
2025-03-13 10:30:34 +02:00
PThorpe92
c638b64a59 Fix tests to use updated extension name 2025-03-12 21:55:50 -04:00
PThorpe92
64d8575ee8 Hide add_builtin_vfs_extensions from non fs feature 2025-03-12 21:52:52 -04:00
PThorpe92
c1f5537d39 Fix feature flagging static vfs modules 2025-03-12 21:52:52 -04:00
PThorpe92
b306cd416d Add debug logging to testing vfs extension 2025-03-12 21:52:52 -04:00
PThorpe92
216a8e7848 Update getrandom dependency in ext api crate 2025-03-12 21:52:52 -04:00
PThorpe92
f8455a6a3b feature flag register static vfs to fs feature 2025-03-12 21:52:51 -04:00
PThorpe92
2cc72ed9ab Feature flag vfs for fs feature/prevent wasm 2025-03-12 21:52:51 -04:00
PThorpe92
8e2c9367c0 add missing method to add builtin vfs to ext api 2025-03-12 21:52:51 -04:00
PThorpe92
89a08b7611 Add vfslist command and setup CLI with new db open api 2025-03-12 21:52:51 -04:00
PThorpe92
6cb9091dc8 Remove unused macro method 2025-03-12 21:52:51 -04:00
PThorpe92
35fc9df275 Rename and combine testing extension crate 2025-03-12 21:52:51 -04:00
PThorpe92
18537ed43e Add documentation/example to extensions/core README.md 2025-03-12 21:52:51 -04:00
PThorpe92
68eca4feed Add demo vfs module to vtab kvstore 2025-03-12 21:52:51 -04:00
PThorpe92
44f6054657 Impl copy + clone for io openflags 2025-03-12 21:52:51 -04:00
PThorpe92
8d3c44cf00 Impl IO trait for VfsMod type 2025-03-12 21:52:50 -04:00
PThorpe92
7d18b6b8b0 Create global vfs module registry 2025-03-12 21:52:50 -04:00