Start of syntax highlighting and hinting. Still need to figure out how
to sublime-syntax works to produce good highlights.
Edit:
Personally, I believe there are more interesting syntax highlighting
possibilities with `reedline` crate, but currently, we cannot use it as
the our DB `Connection` would have to be `Send`. This PR is an
introduction and quality of life changes for the users of the CLI and
for us developers, as we now won't have to look at black and white text
only. I want to have a config file to personalize the color pallets,
that will be made in a following PR.
Closes#1101
For example the commmand `cargo run --package limbo_cli --bin limbo
database.db` will generate a `database.db` file, and it's unlikely we'll
ever need to track such files **if they are placed inside the root of
the project**.
Closes#1150
It is a Minimal Valuable Close (MVC) to #494 and #1146 , so probably
there are some corner cases to handle, if so, just point it out and I'll
be happy to fix it :)
Closes#1148
For example the commmand `cargo run --package limbo_cli --bin limbo database.db`
will generate a `database.db` file, and it's unlikely we'll ever need to
track such files if they are placed inside the root of the project.
### Problem
When using the `io_uring` backend, WAL file writes were corrupted: the
submitted buffer data (e.g., WAL header magic `37 7f 06 82`) was correct
in logs, but the file on disk showed incorrect data (e.g., `00 18 27
xx`). This occurred because the `Arc<RefCell<Buffer>>` was dropped
before the asynchronous `io_uring` write completed, allowing the kernel
to write stale or freed memory.
### Root Cause
In `UringFile::pwrite`, the `buffer` was passed to `io_uring` via an
`iovec`, but the `Arc<RefCell<Buffer>>` wasn’t guaranteed to live until
the write finished. Unlike synchronous `UnixIO`, where the buffer
persists during the `pwrite` call, `io_uring`’s async nature exposed
this lifetime issue.
### Fix
Modified `UringFile::pwrite` to hold a reference to the `buffer` in the
completion callback by calling `buffer.borrow()`. This ensures the
`Buffer` remains alive until `io_uring` completes the write, preventing
memory corruption.
### Changes
- Updated `core/io/io_uring.rs`:
- Added `WriteCompletion` import.
- Wrapped the original `Completion` in a new `WriteCompletion` closure
that references the `buffer`, extending its lifetime until the write
completes.
### Validation
- Tested with `limbo -v io_uring`:
- `.open limbo.db`
- `CREATE TABLE users (id INT PRIMARY KEY, username TEXT);`
- `INSERT INTO users VALUES (1, 'alice');`
- `INSERT INTO users VALUES (2, 'bob');`
- `SELECT * FROM users;`
- Verified WAL file with `xxd -l 16 limbo.db-wal`:
- Before: `0018 2734 ...`
- After: `377f 0682 ...` (correct WAL magic).
- `wal-browser limbo.db-wal` confirms the header is written correctly,
**though frame checksums still need separate fixing** (tracked in a
follow-up issue).
### Follow-Up
- Frame checksum mismatches persist in `wal-browser` output (e.g.,
`00000000-00000000 != 14d64367-7b77a5a0`). This is a separate issue in
`begin_write_wal_frame` or WAL frame initialization, to be addressed in
a subsequent PR.
Closes: #1137Closes#1143
Ensure the Arc<RefCell<Buffer>> in UringFile::pwrite remains alive until
the io_uring write completes by referencing it in the completion callback.
This prevents WAL file corruption where the correct buffer data was
overwritten with stale memory (e.g., 00 18 27 xx instead of 37 7f 06 82).
Validation:
- Tested with limbo -v io_uring and WAL operations.
- Verified with xxd and wal-browser.
Signed-off-by: Daniel Boll <danielboll.academico@gmail.com>
I keep having 3+ PR's in at the same time and always deal with crazy
conflicts because everything in the `ext` library is together in one
file.
This PR moves each category of extension into its own file, and
separates the `vfs` functionality in Core into the `ext/dynamic` module,
so that it can be more easily separated from wasm (or non feature =
"fs") targets to prevent build issues.
The only semantic changes made in this PR is the feature gating of vfs,
the rest is simply organizing and cleaning up imports.
Was unsure if `vfs` should be a feature on the `core` side too, or to
just enable it with the `fs` feature which seemed reasonable, as that
was already the current behavior. But let me know if we want it entirely
behind it's own feature.
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#1124
Added jsonb_remove, jsonb_replace, json_replace.
Updated json_remove to use jsonb under the hood.
Fixed json function big numbers serialization.
Add tests for new functions.
Closes#1140
Made a jsonb traversal by json path.
Changed some ordinary json functions to use jsonb under the hood, so now
behavior of our json module more like sqlite.
Found and fixed some bugs on the way.
Closes#1135
As explained in [docs](https://sqlite.org/lang_transaction.html):
> "DEFERRED means that the transaction does not actually start until the
database is first accessed. Internally, the BEGIN DEFERRED statement
merely sets a flag on the database connection that turns off the
automatic commit that would normally occur when the last statement
finishes. This causes the transaction that is automatically started to
persist until an explicit COMMIT or ROLLBACK or until a rollback is
provoked by an error or an ON CONFLICT ROLLBACK clause. If the first
statement after BEGIN DEFERRED is a SELECT, then a read transaction is
started. Subsequent write statements will upgrade the transaction to a
write transaction if possible, or return SQLITE_BUSY. If the first
statement after BEGIN DEFERRED is a write statement, then a write
transaction is started. "
The transaction upgrade `read -> write` is already handled by the VDBE
in `Transaction`.
closes#1001
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#1133
As explained in docs: https://sqlite.org/lang_transaction.html
"BEGIN DEFERRED statement merely sets a flag on the database connection that turns off the automatic commit that would normally occur when the last statement finishes."
The transaction upgrade (read -> write) is already handled by the VDBE
IDK if I'm being naive here but it was the easiest way that I found to
handle it.
closes#1004
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#1129
