Start of syntax highlighting and hinting. Still need to figure out how
to sublime-syntax works to produce good highlights.
Edit:
Personally, I believe there are more interesting syntax highlighting
possibilities with `reedline` crate, but currently, we cannot use it as
the our DB `Connection` would have to be `Send`. This PR is an
introduction and quality of life changes for the users of the CLI and
for us developers, as we now won't have to look at black and white text
only. I want to have a config file to personalize the color pallets,
that will be made in a following PR.
Closes#1101
I don't know when and why we dropped log::* in favor of tracing but when it was done, it made relevant logs not appear any more while debugging so... I added test_log::test which helps by automatically adding info logs from trace package.
Relatively simple one, although I notice we don't have a lot of testing
here. The extensions have all their tests in the python cli extension
tests. Do we want to keep it that way or motivate inline rust tests for
these modules?
Closes#1081
This adds a "limbo_stress" tool for stress testing Limbo in
non-deterministic way together with support code to run the tests under
Antithesis (which makes them deterministic). The stress tester does not
really do anything useful yet, this is just a step to make sure we can
run tests under Antithesis.
This PR started out as one to improve the API of extensions but I ended
up building on top of this quite a bit and it just kept going. Sorry
this one is so large but there wasn't really a good stopping point, as
it kept leaving stuff in broken states.
**VCreate**: Support for `CREATE VIRTUAL TABLE t USING vtab_module`
**VUpdate**: Support for `INSERT` and `DELETE` methods on virtual
tables.
Sqlite uses `xUpdate` function with the `VUpdate` opcode to handle all
insert/update/delete functionality in virtual tables..
have to just document that:
```
if args[0] == NULL: INSERT args[1] the values in args[2..]
if args[1] == NULL: DELETE args[0]
if args[0] != NULL && len(args) > 2: Update values=args[2..] rowid=args[0]
```
I know I asked @jussisaurio on discord about this already, but it just
sucked so bad that I added some internal translation so we could expose
a [nice API](https://github.com/tursodatabase/limbo/pull/996/files#diff-
3e8f8a660b11786745b48b528222d11671e9f19fa00a032a4eefb5412e8200d1R54) and
handle the logic ourselves while keeping with sqlite's opcodes.
I'll change it back if I have to, I just thought it was genuinely awful
to have to rely on comments to explain all that to extension authors.
The included extension is not meant to be a legitimately useful one, it
is there for testing purposes. I did something similar in #960 using a
test extension, so I figure when they are both merged, I will go back
and combine them into one since you can do many kinds at once, and that
way it will reduce the amount of crates and therefore compile time.
1. Remaining opcodes.
2. `UPDATE` (when we support the syntax)
3. `xConnect` - expose API for a DB connection to a vtab so it can
perform arbitrary queries.
Closes#996
This PR reworks the unix I/O backend, removing runtime reference
counting/borrow checking and optimizing away the hashmap in favor of a
static array, with an unlikely fallback vec.
The only reason the fallback vec is there is because unlike the
`io_uring` module, we cannot simply index into the array with the fd as
the OS could theoretically give us a fd up to I believe 1024 so keeping
an array of that size for a few elements is unnecessary.
Closes#940
In order [experimentally
compile](https://github.com/DougAnderson444/wit-limbo) `limbo_core` to a
[wasm component](https://component-model.bytecodealliance.org/), limbo
needed to have no reliance on `js`, `js-sys`, `wasm-bindgen`, et al.
(for those who aren't familiar, there are many `wasm` runtimes and not
all of them play nice with `wasm-bindgen`)
This PR simply cleans up the dependencies, and puts them behind optional
flags and whatnot in order to enable this. Both `log` and `tracing` were
being used, so I reduced this only to `tracing`.
End result is limbo can be used like this:
https://github.com/DougAnderson444/wit-limbo
We can open a discussion on the possibilities that running limbo as a
wasm component can offer, including potentially using composable
components to implement the sqlite runtime extensions, as well as giving
us a clean interface for PlatformIO operations -- define them once,
implement many ways on various platforms. I'm new to limbo, but it looks
like current extension are Rust based deps and features flags, whereas
sqlite is runtime, right? What if limbo was runtime extensible too?
The WIT interface is largely sync (though I believe wasmtime has an
async feature), but in my limited exposure to limbo so far a lot of the
wasm seems sync already anyway. Again, topic for further discussion.
Suffice to say, aligning these deps in this way paves the road for
further experiments and possibilities.
Related: https://github.com/neilg63/julian_day_converter/pull/2
Related: https://github.com/tursodatabase/limbo/issues/950
Closes: https://github.com/tursodatabase/limbo/issues/950Closes#983
chrono has default features that are incompatible with some targets, such as non-js WebAssembly.
Removing the unused defaults allows limbo to compile to these targets
This PR introduce simple fuzz test for BTree insertion algorithm and
fixes few bugs found by fuzzer
- BTree algorithm returned early although there were overflow pages on
stack and more rebalances were needed
- BTree balancing algorithm worked under assumption that single page
will be enough for rebalance - although this is not always true (if page
were tightly packed with relatively big cells, insertion of new very big
cell can require 3 split pages to distribute the content between them)
- `overflow_cells` wasn't cleared properly during rebalancing
- insertions of dividers to the parent node were implemented incorrectly
- `defragment_page` didn't reset
`PAGE_HEADER_OFFSET_FRAGMENTED_BYTES_COUNT` field which can lead to
suboptimal usage of pages
Closes#951
