Commit Graph

726 Commits

Author SHA1 Message Date
gandeevanr
a9cb8157b5 initial pass at implementing NewRowId 2024-08-07 09:04:09 -07:00
Pekka Enberg
0cb45d4ebb Merge 'core: add datetime modifiers helpers.' from Sonny
Adding the foundational helpers for integrating with supporting datetime Functions modifiers later.
part of https://github.com/penberg/limbo/issues/158.

Integrating with date time functions in `vdbe` `Program.step` will come in other PRs later.
reference to sqlite eaa560f3fc/src/date.c (L723)

Closes #277
2024-08-07 16:20:53 +03:00
sonhmai
c104e50a66 core: add datetime modifiers helpers.
Building the helpers for integrating with supporting datetime Functions modifiers later.
2024-08-07 13:43:51 +07:00
Pekka Enberg
d4d7042373 Merge 'Use the correct integer PK column idx as the row-id alias' from GV
This pull request addresses [issue #256](https://github.com/penberg/limbo/issues/256).

However, it currently breaks insertions into tables that lack a row-id
alias due to the NewRowId not being implemented yet. I have created an
[issue](https://github.com/penberg/limbo/issues/275) to track this
problem and will submit a separate pull request to resolve it soon.

Closes #274
2024-08-06 08:11:10 +03:00
Pekka Enberg
7b06150df6 Update CHANGELOG.md and COMPAT.md 2024-08-06 08:06:43 +03:00
Pekka Enberg
5b7d112e74 Merge 'implementation of json function json(X)' from Jean Arhancet
Add the `json` function `json(X)` (related to the issue https://github.com/penberg/limbo/issues/127)

Closes #230
2024-08-06 08:05:33 +03:00
JeanArhancet
15ab7955d3 add licences 2024-08-05 21:34:30 +02:00
gandeevanr
2b86f89d8d use the correct integer PK column idx as the row-id alias 2024-08-04 18:53:54 -07:00
JeanArhancet
4050a3ebe2 refactor: impl json5 direclty 2024-08-04 12:47:08 +02:00
JeanArhancet
4d0a25c1b4 test: add json tcl test 2024-08-04 10:56:52 +02:00
JeanArhancet
8dbced6a39 refactor: delete Insn Json 2024-08-04 10:56:51 +02:00
JeanArhancet
552090cb29 feat: add json support 2024-08-04 10:54:27 +02:00
Pekka Enberg
a765828206 Merge 'First pass on WebAssembly I/O' from Pekka Enberg
Just some basic plumbing to call Node filesystem API from Rust.

Closes #270
2024-08-04 11:45:19 +03:00
Pekka Enberg
933bf89bb9 wasm: VFS interface to use Node filesystem API
This adds a basic VFS interface to use Node filesystem API from Rust
code. Not a lot, but parses the SQLite database file header and
completes database open without errors.
2024-08-04 11:12:21 +03:00
Pekka Enberg
7df979cd3a wasm: Enable console.error() panic hook
Makes debugging a little bit easier...
2024-08-04 09:45:27 +03:00
Pekka Enberg
e5034acb9e Merge 'fix: ctrl-c stop current query execution' from Jean Arhancet
Fix #260 by ensuring `Ctrl-C` interrupts the query without terminating
the process. Implement the SQLite strategy where pressing `Ctrl-C` twice
exits the shell.

Closes #269
2024-08-04 08:13:39 +03:00
JeanArhancet
38a1b85094 fix: ctrl-c stop current query execution 2024-08-03 22:58:52 +02:00
Pekka Enberg
31308f3699 core: Switch SQLite to use locking_mode EXCLUSIVE
Let's switch the SQLite benchmarks to use similar file locking protocol
as we do. Improves SQLite's performance by 2x, but Limbo is still
faster.

Before:

rusqlite/Prepare statement: 'SELECT * FROM users LIMIT 1'
                        time:   [2.1027 µs 2.1239 µs 2.1563 µs]
                        thrpt:  [463.75 Kelem/s 470.83 Kelem/s 475.57 Kelem/s]

After:

rusqlite/Execute prepared statement: 'SELECT * FROM users LIMIT 1'
                        time:   [940.07 ns 944.16 ns 949.39 ns]
                        thrpt:  [1.0533 Melem/s 1.0591 Melem/s 1.0638 Melem/s]
2024-08-03 21:13:50 +03:00
Pekka Enberg
89079d1ccd Merge 'core: fix clippy' from Sonny
Closes #268
2024-08-03 17:53:51 +03:00
sonhmai
0e7bd95e4e core: fix clippy 2024-08-03 20:14:26 +07:00
Pekka Enberg
e890a1d64f Merge 'More WAL work' from Pekka Enberg
Turn the WAL into a trait and open + parse WAL header.

Closes #267
2024-08-03 14:10:36 +03:00
Pekka Enberg
a290b2f102 core: Open WAL file and parse header 2024-08-03 12:48:16 +03:00
Pekka Enberg
ed4116e7c2 core: Introduce Wal trait
We're going to need it for WebAssembly anyway, which does not have
standard filesystem support.
2024-08-03 12:34:10 +03:00
Pekka Enberg
d9b1cac27b testing: Add test database + WAL file 2024-08-03 12:16:34 +03:00
Pekka Enberg
7937c165fe Merge 'Storage layer cleanups' from Pekka Enberg
Closes #265
2024-08-03 11:00:33 +03:00
Pekka Enberg
18883b5a7e core: Document top-level storage module 2024-08-03 10:44:19 +03:00
Pekka Enberg
090a577dd5 core: Move DatabaseStorage to storage/database.rs 2024-08-03 10:41:10 +03:00
Pekka Enberg
83650a797a core: Document DatabaseStorage purpose 2024-08-03 10:37:41 +03:00
Pekka Enberg
3f7c788e5b core: Rename DatabaseStorage methods
Let's call them read_page() and write_page().
2024-08-03 10:35:14 +03:00
Pekka Enberg
8a54e31803 core: Rename PageIO to DatabaseStorage 2024-08-03 10:33:52 +03:00
Pekka Enberg
4349b946e5 core: Eliminate PageSource wrapper
The PageSource wrapper is useless. Let's inline it and use PageIO
directly.
2024-08-03 10:27:20 +03:00
Pekka Enberg
90308defb2 Merge 'Fix serial type decoding for BLOB and TEXT' from Lauri Virtanen
Replace `>` with `>=` to match the [SQLite documentation](https://www.sqlite.org/fileformat.html#record_format).

Serial type value `12` is for 0-size BLOB, and `13` is for 0-size TEXT.

Closes #264
2024-08-02 22:25:10 +03:00
Lauri Virtanen
145af04c7e Fix serial type decoding for BLOB and TEXT
Replace `>` with `>=` to match the SQLite documentation. Serial type
value `12` is for 0-size BLOB, and `13` is for 0-size TEXT.
2024-08-02 21:50:50 +03:00
Pekka Enberg
34ed11e3a4 Merge 'Function cleanups' from Pekka Enberg
Closes #262
2024-08-02 17:38:53 +03:00
Pekka Enberg
1e9384accc Merge 'update cargo-dist and enable PS installer/github attestations' from ashley williams
saw you were building a windows bin but not a ps installer and thought i would turn it off. also turned on github attestations because why not.

`cargo dist plan` is basically the same as before but now also produces the PS installer.
```
announcing v0.0.3
  limbo 0.0.3
    source.tar.gz
      [checksum] source.tar.gz.sha256
    limbo-installer.sh
    limbo-installer.ps1
    limbo-aarch64-apple-darwin-update
    limbo-aarch64-apple-darwin.tar.xz
      [bin] limbo
      [misc] CHANGELOG.md, LICENSE.md, README.md
      [checksum] limbo-aarch64-apple-darwin.tar.xz.sha256
    limbo-x86_64-apple-darwin-update
    limbo-x86_64-apple-darwin.tar.xz
      [bin] limbo
      [misc] CHANGELOG.md, LICENSE.md, README.md
      [checksum] limbo-x86_64-apple-darwin.tar.xz.sha256
    limbo-x86_64-pc-windows-msvc-update
    limbo-x86_64-pc-windows-msvc.zip
      [bin] limbo.exe
      [misc] CHANGELOG.md, LICENSE.md, README.md
      [checksum] limbo-x86_64-pc-windows-msvc.zip.sha256
    limbo-x86_64-unknown-linux-gnu-update
    limbo-x86_64-unknown-linux-gnu.tar.xz
      [bin] limbo
      [misc] CHANGELOG.md, LICENSE.md, README.md
      [checksum] limbo-x86_64-unknown-linux-gnu.tar.xz.sha256
  ```

Closes #263
2024-08-02 17:36:13 +03:00
Pekka Enberg
0affdada2a core: Move datetime.rs to vdbe/
The file contains SQL functions invoked by the VDBE so let's move the
file there.
2024-08-02 17:34:10 +03:00
Pekka Enberg
9c479734be core: Rename to exec_time()
Follow the same naming convention as other SQL functions.
2024-08-02 17:34:10 +03:00
Pekka Enberg
465dfa3cb5 core: Rename to exec_date()
Follow the same naming convention as other SQL functions.
2024-08-02 17:34:10 +03:00
Ashley Williams
c9c96f01f8 feat(dist): enable github attestations 2024-08-02 09:30:45 -05:00
Ashley Williams
88964e691a feat(dist): update and add PS installer 2024-08-02 09:25:03 -05:00
Pekka Enberg
e926ade455 Merge 'Date and time code cleanups' from Pekka Enberg
Closes #261
2024-08-02 16:21:28 +03:00
Pekka Enberg
69eb851120 core/datetime: Move get_max_datetime_exclusive() to bottom
Move the function so that it's callers are above the function definition
for smoother flow when reading the code.
2024-08-02 16:15:22 +03:00
Pekka Enberg
6ffb03216f core/datetime: Simplify error handling 2024-08-02 16:15:22 +03:00
Pekka Enberg
3412b65b8a core/datetime: Remove TimeUnit
...it's not used for anything much.
2024-08-02 16:04:09 +03:00
Pekka Enberg
763bf17c9e core/datetime: Use "cfg(test)" annotation for tests 2024-08-02 16:03:53 +03:00
Pekka Enberg
f8492c85ae core/datetime: Remove trace calls
We should trace in high-level code like VDBE interpreter loop, but not
in error handling path of specific SQL functions.
2024-08-02 16:03:46 +03:00
Pekka Enberg
707d1a0705 Merge 'Update Flake and add Darwin link flags' from Ethan Niser
This pull request:

 - updates the nix flake to include tcl 8.6 addressing #243

 - updates the nix flake to add the darwin "core foundation" lib

 - updates the make flake to link the core foundation lib on darwin

I was previously unable to run `make test-sqlite3` with this error:
https://gist.github.com/ethanniser/5b96888637da6ea9e9d56dcc3af9b573

But now it is able to run.

Closes #259
2024-08-02 08:12:14 +03:00
Ethan Niser
d47d494865 update flake to include tcl latest (addresses #243) and add core foundation lib and link flags for darwin 2024-08-01 18:00:46 -07:00
Pekka Enberg
e18fc511bb scripts/merge-pr.py: Use temporal file for commit message
...fixes problems like eating perfectly valid Markdown and stuff from
pull request descriptions.
2024-08-01 21:07:39 +03:00
Pekka Enberg
64738d6348 Merge 'Add ability to annotate instructions with comments' from Jussi Saurio
Similar to what SQLite does.

```
limbo> EXPLAIN SELECT u.age, p.name FROM users u LEFT JOIN products p ON u.first_name = p.name;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     25    0                    0   Start at 25
1     OpenReadAsync      0     2     0                    0   table=u, root=2
2     OpenReadAwait      0     0     0                    0
3     OpenReadAsync      1     3     0                    0   table=p, root=3
4     OpenReadAwait      0     0     0                    0
5     RewindAsync        0     0     0                    0
6     RewindAwait        0     -5    0                    0   Rewind table u
7       Integer          0     1     0                    0   r[1]=0; init LEFT JOIN match flag
8       RewindAsync      1     0     0                    0
9       RewindAwait      1     -11   0                    0   Rewind table p
10        Column         0     1     2                    0   r[2]=u.first_name
11        Column         1     1     3                    0   r[3]=p.name
12        Ne             2     3     17                   0   if r[2]!=r[3] goto 17
13        Integer        1     1     0                    0   r[1]=1; record LEFT JOIN hit
14        Column         0     9     4                    0   r[4]=u.age
15        Column         1     1     5                    0   r[5]=p.name
16        ResultRow      4     2     0                    0   output=r[4..5]
17      NextAsync        1     0     0                    0
18      NextAwait        1     9     0                    0
19      IfPos            1     22    0                    0   r[1]>0 -> r[1]-=0, goto 22
20      NullRow          1     0     0                    0   Set cursor 1 to a (pseudo) NULL row
21      Goto             0     13    0                    0
22    NextAsync          0     0     0                    0
23    NextAwait          0     6     0                    0
24    Halt               0     0     0                    0
25    Transaction        0     0     0                    0
26    Goto               0     1     0                    0
```

Closes #258
2024-08-01 21:07:31 +03:00