Commit Graph

1115 Commits

Author SHA1 Message Date
Pekka Enberg
e818bdb3ef Merge 'open file in append mode' from Pere Diaz Bou
Closes #409
2024-11-18 19:15:05 +02:00
Pere Diaz Bou
2cc89f712e open file in append mode 2024-11-18 18:06:01 +01:00
Pekka Enberg
91702abbe0 perf/latency: Update Cargo.lock
Update Cargo.lock to get the latest jsonb version, which fixes security
issue with fast-float.
2024-11-18 18:46:46 +02:00
Pekka Enberg
a83e62f6f4 Limbo 0.0.5 2024-11-18 14:57:00 +02:00
Pere Diaz Bou
9c4ca28bd9 Merge 'CREATE TABLE support' from Pekka Enberg
Closes #383
2024-11-18 13:40:20 +01:00
Pere Diaz Bou
80f60852df remove wrong comment 2024-11-18 10:41:20 +01:00
Pere Diaz Bou
9f72655e30 tree_create -> btree_create 2024-11-18 10:31:47 +01:00
Pekka Enberg
5efc218e6e Merge 'support subtract in translate_expr() (not in condition expressions yet)' from Jussi Saurio
closes #402

Closes #403
2024-11-18 09:28:23 +02:00
Pekka Enberg
78e723c61f Merge 'support parenthesized(single expr) in translate_expr()' from Jussi Saurio
fix #397 (for the most part)

Closes #401
2024-11-18 09:28:14 +02:00
jussisaurio
491bdd3bfc support subtract in translate_expr() (not in condition expressions yet) 2024-11-17 18:47:16 +02:00
jussisaurio
9a4864bc6a support parenthesized(single expr) in translate_expr() 2024-11-17 18:36:30 +02:00
Pekka Enberg
f8391ea40c Update CHANGELOG 2024-11-17 17:14:57 +02:00
Pekka Enberg
0a45a04be6 Merge 'Fix COUNT(*) panic' from Jussi Saurio
Closes #399

Closes #400
2024-11-17 17:14:29 +02:00
jussisaurio
ccdcf302ca quick fix for #399 2024-11-17 17:06:09 +02:00
jussisaurio
de801f1e25 Merge 'vendor sqlite3-parser (lemon-rs)' from Jussi Saurio
- vendor sqlite3-parser (lemon-rs) to have full control over the parser
- update limbo-core in perf/latency/limbo

Reviewed-by: Pekka Enberg <penberg@iki.fi>

Closes #396
2024-11-16 20:14:11 +02:00
jussisaurio
3cc9d9d79f vendor sqlite3-parser (lemon-rs) 2024-11-16 20:08:59 +02:00
Pekka Enberg
8efeb16b82 Merge 'bindings/wasm: "SELECT 1" benchmark' from Pekka Enberg
This adds a benchmark for "SELECT 1" that essentially just evaluates the
overhead of calling into the library. Looks like WebAssembly adds 2.5x
overhead compared to native code...
```
penberg@vonneumann perf % node perf-limbo.js
cpu: Apple M1
runtime: node v18.19.0 (arm64-darwin)

benchmark      time (avg)             (min … max)       p75       p99      p999
------------------------------------------------- -----------------------------
• Statement
------------------------------------------------- -----------------------------
SELECT 1      771 ns/iter     (624 ns … 2'183 µs)    750 ns  1'085 ns  1'750 ns

summary for Statement
  SELECT 1
penberg@vonneumann perf % node perf-better-sqlite3.js
cpu: Apple M1
runtime: node v18.19.0 (arm64-darwin)

benchmark      time (avg)             (min … max)       p75       p99      p999
------------------------------------------------- -----------------------------
• Statement
------------------------------------------------- -----------------------------
SELECT 1      302 ns/iter       (166 ns … 779 µs)    292 ns    375 ns  1'000 ns

summary for Statement
  SELECT 1
```

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #394
2024-11-16 19:00:39 +02:00
Pekka Enberg
9473668196 core: Bump jsonb dependency
The `fast-float` library has soundness issues, which why `jsonb`
switched to `fast-float2`:

https://github.com/databendlabs/jsonb/issues/68
2024-11-16 18:58:13 +02:00
Pere Diaz Bou
569e98cb32 fmt 2024-11-16 17:08:22 +01:00
Pere Diaz Bou
d1dee484df properly format table and "if not exists" support 2024-11-16 17:02:53 +01:00
Pere Diaz Bou
090615b289 create btree table + parse schema 2024-11-16 16:24:28 +01:00
Pere Diaz Bou
66e20df309 fix some offset usages in btree 2024-11-16 16:23:06 +01:00
Pekka Enberg
fc5e42bb94 CREATE TABLE support 2024-11-16 13:07:10 +01:00
Pekka Enberg
422dac50f2 Merge 'core: Make mockall a dev dependency' from Pekka Enberg
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #392
2024-11-16 13:24:47 +02:00
Pekka Enberg
85b70e381d core: Make mockall a dev dependency 2024-11-16 13:18:35 +02:00
Pekka Enberg
3442287902 Merge 'bindings/wasm: Fix VFS.sync() function signature' from Pekka Enberg
We don't expect any return value from the underlying `fsync()` so fix
the function signature accordingly.
Fixes the following obscure error:
```
TypeError: Cannot convert undefined to a BigInt
    at wasm://wasm/00942492:wasm-function[1501]:0x1c4dc1
    at wasm://wasm/00942492:wasm-function[694]:0x189586
    at wasm://wasm/00942492:wasm-function[50]:0x143d7
    at wasm://wasm/00942492:wasm-function[60]:0x3f91a
    at new Database (/Users/penberg/src/penberg/limbo/bindings/wasm/pkg/limbo_wasm.js:162:26)
    at file:///Users/penberg/src/penberg/limbo/bindings/wasm/perf/perf-limbo.js:5:12
    at ModuleJob.run (node:internal/modules/esm/module_job:195:25)
    at async ModuleLoader.import (node:internal/modules/esm/loader:336:24)
    at async loadESM (node:internal/process/esm_loader:34:7)
    at async handleMainPromise (node:internal/modules/run_main:106:12)
```

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #395
2024-11-16 13:10:51 +02:00
Pekka Enberg
60cc8c0347 bindings/wasm: "SELECT 1" benchmark
This adds a benchmark for "SELECT 1" that essentially just evaluates the
overhead of calling into the library. Looks like WebAssembly adds 2.5x
overhead compared to native code...

```
penberg@vonneumann perf % node perf-limbo.js
cpu: Apple M1
runtime: node v18.19.0 (arm64-darwin)

benchmark      time (avg)             (min … max)       p75       p99      p999
------------------------------------------------- -----------------------------
• Statement
------------------------------------------------- -----------------------------
SELECT 1      771 ns/iter     (624 ns … 2'183 µs)    750 ns  1'085 ns  1'750 ns

summary for Statement
  SELECT 1
penberg@vonneumann perf % node perf-better-sqlite3.js
cpu: Apple M1
runtime: node v18.19.0 (arm64-darwin)

benchmark      time (avg)             (min … max)       p75       p99      p999
------------------------------------------------- -----------------------------
• Statement
------------------------------------------------- -----------------------------
SELECT 1      302 ns/iter       (166 ns … 779 µs)    292 ns    375 ns  1'000 ns

summary for Statement
  SELECT 1
```
2024-11-16 13:10:23 +02:00
Pekka Enberg
82ec900b07 bindings/wasm: Fix VFS.sync() function signature
We don't expect any return value from the underlying `fsync()` so fix
the function signature accordingly.

Fixes the following obscure error:

```
TypeError: Cannot convert undefined to a BigInt
    at wasm://wasm/00942492:wasm-function[1501]:0x1c4dc1
    at wasm://wasm/00942492:wasm-function[694]:0x189586
    at wasm://wasm/00942492:wasm-function[50]:0x143d7
    at wasm://wasm/00942492:wasm-function[60]:0x3f91a
    at new Database (/Users/penberg/src/penberg/limbo/bindings/wasm/pkg/limbo_wasm.js:162:26)
    at file:///Users/penberg/src/penberg/limbo/bindings/wasm/perf/perf-limbo.js:5:12
    at ModuleJob.run (node:internal/modules/esm/module_job:195:25)
    at async ModuleLoader.import (node:internal/modules/esm/loader:336:24)
    at async loadESM (node:internal/process/esm_loader:34:7)
    at async handleMainPromise (node:internal/modules/run_main:106:12)
```
2024-11-16 13:07:32 +02:00
Pere Diaz Bou
0e149ae7dd Merge 'core: Reduce library text size some more' from Pekka Enberg
Apply some more tricks from to reduce text size by 1 MB:
https://github.com/johnthagen/min-sized-rust
Before:
  penberg@vonneumann sqlite3 % ls -lh
../target/release/liblimbo_sqlite3.a
  -rw-r--r--  1 penberg  staff   6,7M 16 Mar 09:52
../target/release/liblimbo_sqlite3.a
After:
  penberg@vonneumann sqlite3 % ls -lh
../target/release/liblimbo_sqlite3.a
  -rw-r--r--  1 penberg  staff   5,6M 16 Mar 09:50
../target/release/liblimbo_sqlite3.a

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #391
2024-11-16 11:34:16 +01:00
Pekka Enberg
2eb299907f Merge 'core: Make JSON support configurable' from Pekka Enberg
This adds a `json` feature flag, which allows users to disable JSON
support if needed.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #390
2024-11-16 12:14:06 +02:00
Pekka Enberg
8afad9f79b core: Reduce library text size some more
Apply some more tricks from to reduce text size by 1 MB:

https://github.com/johnthagen/min-sized-rust

Before:

  penberg@vonneumann sqlite3 % ls -lh ../target/release/liblimbo_sqlite3.a
  -rw-r--r--  1 penberg  staff   6,7M 16 Mar 09:52 ../target/release/liblimbo_sqlite3.a

After:

  penberg@vonneumann sqlite3 % ls -lh ../target/release/liblimbo_sqlite3.a
  -rw-r--r--  1 penberg  staff   5,6M 16 Mar 09:50 ../target/release/liblimbo_sqlite3.a
2024-11-16 09:53:24 +02:00
Pekka Enberg
cd5db55cf2 core: Make JSON support configurable
This adds a `json` feature flag, which allows users to disable JSON
support if needed.
2024-11-16 09:49:09 +02:00
Pekka Enberg
17c7e8d346 sqlite3: Disable env_logger default features
Let's keep the dependencies as light-weight as possible.
2024-11-16 09:47:12 +02:00
Pekka Enberg
fe75ed275e Merge 'Bump PyO3 dependency' from Pekka Enberg
Github's bot reports a use-after-free issue so let's bump up the
version:
https://github.com/penberg/limbo/security/dependabot/4

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #388
2024-11-16 09:02:10 +02:00
Pekka Enberg
0e2291a57e Merge 'build: Enable LTO for release builds' from Pekka Enberg
This reduces `liblimbo_sqlite3.a` binary size from 25 MB to 10 MB:
Before:
  $ -lh ../target/release/liblimbo_sqlite3.a
  -rw-r--r--  1 penberg  staff    25M 15 Mar 18:36
../target/release/liblimbo_sqlite3.a
After:
  $ penberg@vonneumann sqlite3 % ls -lh
../target/release/liblimbo_sqlite3.a
  -rw-r--r--  1 penberg  staff   9,9M 15 Mar 18:59
../target/release/liblimbo_sqlite3.a

Closes #389
2024-11-16 09:01:59 +02:00
Pekka Enberg
c3da75d260 build: Enable LTO for release builds
This reduces `liblimbo_sqlite3.a` binary size from 25 MB to 10 MB:

Before:

  $ -lh ../target/release/liblimbo_sqlite3.a
  -rw-r--r--  1 penberg  staff    25M 15 Mar 18:36 ../target/release/liblimbo_sqlite3.a

After:

  $ penberg@vonneumann sqlite3 % ls -lh ../target/release/liblimbo_sqlite3.a
  -rw-r--r--  1 penberg  staff   9,9M 15 Mar 18:59 ../target/release/liblimbo_sqlite3.a
2024-11-15 19:06:17 +02:00
Pekka Enberg
851ea65530 Bump PyO3 dependency
Github's bot reports a use-after-free issue so let's bump up the
version:

https://github.com/penberg/limbo/security/dependabot/4
2024-11-15 16:14:34 +02:00
Pekka Enberg
9d96b22b4a Merge 'core: create databases from limbo' from Pere Diaz Bou
Behold, the database emerges from within.

Reviewed-by: Pekka Enberg <penberg@iki.fi>

Closes #387
2024-11-15 15:55:48 +02:00
Pekka Enberg
44ececce7d github: Fix Pekka's email address 2024-11-15 15:55:33 +02:00
Pekka Enberg
5e1184d559 Update COMPAT.md 2024-11-15 13:49:15 +02:00
Pere Diaz Bou
6cd0f03643 core: create databases from limbo 2024-11-15 12:09:07 +01:00
Pere Diaz Bou
87c80b2a07 use core wal file for wasm 2024-11-14 17:54:20 +01:00
Pere Diaz Bou
4a55e0b26d Merge 'core: drop mutex on contents' from Pere Diaz Bou
There is no need to have mutexes on buffers, we will introduce mutexes
if we want later on a file level to introduce serializability.
This pr also includes a fix in the cacheflush procedure where a wait of
inflight writes weren't waited.

Reviewed-by: Pekka Enberg <penberg@ki.fi>

Closes #384
2024-11-13 19:26:50 +01:00
Pere Diaz Bou
a4b473ba59 fmt 2024-11-13 19:17:49 +01:00
Pere Diaz Bou
c46bd63b5a core: drop mutex on contents
There is no need to have mutexes on buffers, we will introduce mutexes
if we want later on a file level to introduce serializability.
2024-11-13 19:13:15 +01:00
Pere Diaz Bou
71ac1eacb7 Merge 'btree: cursor with lineal stack structure' from Pere Diaz Bou
Removed MemPage from the code in favor of an array to encode the stack
of the cursor. This is both simpler and better in terms of memory
access.
O_DIRECT was removed from WAL file as it introduces alignment
constraints that are too hard to follow in regular appends. Maybe in the
future?

Closes #378
2024-11-13 18:23:31 +01:00
Pere Diaz Bou
2a787aedb5 core: extract page stack and document a bit 2024-11-13 17:13:30 +01:00
Pere Diaz Bou
23923e4175 ignore wal checkpoint test 2024-11-13 14:09:38 +00:00
Pere Diaz Bou
9d7a3e8e60 ignore sequential test 2024-11-13 14:04:28 +00:00
Pere Diaz Bou
dfdd8083e2 remove ignore from wal tests 2024-11-13 13:58:05 +00:00