Lots of cleanup still left to do. Draft PR for adding support for OPFS
for WASM build (add support for limbo in browser).
Overall explanation of the architecture: this follows the sqlite wasm
architecture for OPFS.
main <> (limbo-worker.js) limbo (VFS - opfs.js) <> opfs-sync-proxy
The main thread loads limbo-worker.js which bootstraps the opfs.js and
opfs-sync-proxy.js and then launches the limbo-wasm.js.
At that point it can be used with worker.postmessage and
worker.onmessage interactions from the main thread.
The VFS provided by opfs.js provides a sync API by offloading async
operations to opfs-sync-proxy.js. This is done through SharedArrayBuffer
and Atomic.wait() to make the actual async operations appear synchronous
for limbo.
resolves#531Closes#594
#514
Introduces a new feature for lazy evaluation in the
`Statement.raw().iterate()` method and includes related changes in both
the test and implementation files. The most important changes include
adding a test case for lazy evaluation, creating a `RowIterator` struct,
and modifying the `iterate` method to use this new struct.
Everything seems to works fine, but suggestions on code improvement and
test use cases are welcoming.
Closes#527
The name "row result" is confusing because it really *is* a result from
a step() call. The only difference is how a row is represented as we
return from VDBE or from a statement.
Therefore, rename RowResult to StepResult.
This adds an interrupt() method to Statement that allows apps to
interrupt a running statement. Please note that this is different from
`sqlite3_interrupt()` which interrupts all ongoing operations in a
database. Although we want to support that too, per statement interrupt
is much more useful to apps.
Since page cache is now shared by default, we need to cache pages by
page number and something else. I chose to go with max_frame of
connection, because this connection will have a max_frame set until from
the start of a transaction until the end of it.
With key pairs of (pgno, max_frame) we make sure each connection is
caching based on the snapshot it is at as two different connections
might have the same pageno being using but a different frame. If both
have same max_frame then they will share same page.
Since we expect to ensure thread safety between multiple threads in the
future, we extract what is important to be shared between multiple
connections with regards to WAL.
This is WIP so I just put whatever feels like important behind a RwLock
but expect this to change to Atomics in the future as needed. Maybe even
these locks might disappear because they will be better served with
transaction locks.
We don't expect any return value from the underlying `fsync()` so fix
the function signature accordingly.
Fixes the following obscure error:
```
TypeError: Cannot convert undefined to a BigInt
at wasm://wasm/00942492:wasm-function[1501]:0x1c4dc1
at wasm://wasm/00942492:wasm-function[694]:0x189586
at wasm://wasm/00942492:wasm-function[50]:0x143d7
at wasm://wasm/00942492:wasm-function[60]:0x3f91a
at new Database (/Users/penberg/src/penberg/limbo/bindings/wasm/pkg/limbo_wasm.js:162:26)
at file:///Users/penberg/src/penberg/limbo/bindings/wasm/perf/perf-limbo.js:5:12
at ModuleJob.run (node:internal/modules/esm/module_job:195:25)
at async ModuleLoader.import (node:internal/modules/esm/loader:336:24)
at async loadESM (node:internal/process/esm_loader:34:7)
at async handleMainPromise (node:internal/modules/run_main:106:12)
```
Removed MemPage from the code in favor of an array to encode the stack
of the cursor. This is both simpler and better in terms of memory
access.
O_DIRECT was removed from WAL file as it introduces alignment
constraints that are too hard to follow in regular appends. Maybe in the
future?
Closes#378
This adds a basic VFS interface to use Node filesystem API from Rust
code. Not a lot, but parses the SQLite database file header and
completes database open without errors.
