Commit Graph

1851 Commits

Author SHA1 Message Date
Pekka Enberg
9db5061d64 Merge 'Simulator improvements' from Pekka Enberg
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #683
2025-01-14 18:37:31 +02:00
Pekka Enberg
053bc0b8cf Add Jussi to .github.json 2025-01-14 18:37:26 +02:00
Pekka Enberg
0c7ebd4df5 simulator: Enable info-level logging by default 2025-01-14 17:54:39 +02:00
Pekka Enberg
30a380cab1 simulator: Move more logging under trace level 2025-01-14 17:54:29 +02:00
Pekka Enberg
d355ce785c core/storage: Remove debug printout 2025-01-14 17:54:17 +02:00
Pekka Enberg
3c6c6041ff simulator: Log query errors with debug level
...it is totally fine for a SQL query to fail as part of the simulation.
For example, if we attempt to create a table that already exists, the
expectation is that the query fails. No need to spam the logs.
2025-01-14 17:41:07 +02:00
Pekka Enberg
e1f5fa875e simulator: Make simulator runs longer by default 2025-01-14 17:37:53 +02:00
Pekka Enberg
14ec057a34 simulator: Make stats printout prettier
```
op           calls   faults
--------- -------- --------
pread            3        0
pwrite           1        0
sync             0        0
--------- -------- --------
total            4        0
```
2025-01-14 17:32:31 +02:00
Pekka Enberg
5b4c7ec7f5 simulator: Rename stats in SimulatorFile 2025-01-14 17:24:14 +02:00
Pekka Enberg
1df1f7afc5 Add scripts/run-sim helper
...to run the simulator in a loop with different seeds.
2025-01-14 17:19:58 +02:00
Pekka Enberg
9c9a6e5821 Merge 'Fix unsafe calls to mark_last_insn_constant()' from Jussi Saurio
Bug found by @alpaylan and described here: https://github.com/tursodatab
ase/limbo/issues/662#issuecomment-2589756954
The reason is that we were, as a bytecode optimization, marking
instructions as constant in places where it was not safe to do so. It is
ONLY safe to mark an instruction as constant if the register allocated
for the result of that instruction is allocated during the translation
of that expression.
In the case of e.g. `Unary(Minus, Number)` we were doing the following:
```
limbo> CREATE TABLE likable_fire (captivating_insolacion INTEGER,mirthful_shihab REAL);
limbo> INSERT INTO likable_fire VALUES (8358895602713329453, -7435384732.72567), (4233751081339504981, -6653311696.714637);
limbo> select * from likable_fire;
8358895602713329453|-6653311696.714637
4233751081339504981|-6653311696.714637
limbo> explain INSERT INTO likable_fire VALUES (8358895602713329453, -7435384732.72567), (4233751081339504981, -6653311696.714637);
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     16    0                    0   Start at 16
1     InitCoroutine      5     7     2                    0
2     Integer            1783463725  2     0                    0   r[2]=8358895602713329453
3     Yield              5     15    0                    0
4     Integer            1454033237  2     0                    0   r[2]=4233751081339504981
5     Yield              5     15    0                    0
6     EndCoroutine       5     0     0                    0
7     OpenWriteAsync     0     2     0                    0
8     OpenWriteAwait     0     0     0                    0
9     Yield              5     15    0                    0
10    NewRowId           0     1     0                    0
11    MakeRecord         2     2     4                    0   r[4]=mkrec(r[2..3])
12    InsertAsync        0     4     1                    0
13    InsertAwait        0     0     0                    0
14    Goto               0     9     0                    0
15    Halt               0     0     0                    0
16    Transaction        0     1     0                    0

<!-- Reg 3 evaluated in a loop, but marked as "constant" twice, resulting in the first value being overwritten! -->
<!-- The reason mark_last_insn_constant() breaks this is because different values are being evaluated into the -->
<!-- Same register in a loop -->
17    Real               0     3     0     -7435384732.72567  0   r[3]=-7435384732.72567
18    Real               0     3     0     -6653311696.714637  0   r[3]=-6653311696.714637

19    Goto               0     1     0                    0
```
This PR removes `.mark_last_insn_constant()` from the affected places.
We should think about a small abstraction to prevent this going forward

Closes #679
2025-01-14 17:15:38 +02:00
Jussi Saurio
3cbb2d2d7c Add regression test for multi insert with unary operator 2025-01-14 16:22:16 +02:00
Jussi Saurio
fcfee24c50 Remove mark_last_insn_constant() from places where it is not safe to do so 2025-01-14 16:06:49 +02:00
Pekka Enberg
55e79a72c1 Merge 'Initial pass on loadable rust extensions' from Preston Thorpe
This PR adds the start of an implementation of an extension library for
`limbo` so users can write extensions in rust, that can be loaded at
runtime with the `.load` cli command.
The existing "ExtensionFunc"  `uuid`, has been replaced with the first
complete limbo extension in `extensions/uuid`
![image](https://github.com/user-
attachments/assets/63aa06cc-9390-4277-ba09-3a9be5524535)
![image](https://github.com/user-
attachments/assets/75899748-5e26-406a-84ee-26063383afeb)
There is still considerable work to do on this, as this only implements
scalar functions, but this PR is already plenty big enough.
Design + implementation comments or suggestions would be appreciated
👍
I tried out using `abi_stable`, so that trait objects and other goodies
could be used across FFI bounds, but to be honest I didn't find it too
much better than this. I personally haven't done a whole lot with FFI,
or anything at all linking dynamically in Rust, so if there is something
I seem to be missing here, please let me know.
I added some tests, similar to how shell-tests are setup.
If anyone can test this on other platforms, that would be helpful as
well as I am limited to x86_64 linux here

Closes #658
2025-01-14 15:36:56 +02:00
PThorpe92
9c208dc866 Add tests for first extension 2025-01-14 07:27:35 -05:00
PThorpe92
e4ce6402eb Remove previous uuid implementation 2025-01-14 07:20:50 -05:00
PThorpe92
3099e5c9ba Improve api, standardize conversions between types, finish extension 2025-01-14 07:20:50 -05:00
PThorpe92
6e05258d36 Add safety comments and clean up extension types 2025-01-14 07:20:50 -05:00
PThorpe92
98eff6cf7a Enable passing arguments to external functions 2025-01-14 07:20:50 -05:00
PThorpe92
852817c9ff Have args macro in extension take a range 2025-01-14 07:20:50 -05:00
PThorpe92
c565fba195 Adjust types in extension API 2025-01-14 07:20:49 -05:00
PThorpe92
3412a3d4c2 Rough design for extension api/draft extension 2025-01-14 07:20:48 -05:00
PThorpe92
0a10d893d9 Sketch out runtime extension loading 2025-01-14 07:18:07 -05:00
Pekka Enberg
bfbaa80bdc Merge 'Change bindings/java to support java 8' from Kim Seon Woo
## Purpose of this PR
Change java bindings support from jdk version 17 to 8.
## Changes
- Mostly related to dependency version updates
- Replace java 8 non supported methods
## Reference
https://github.com/tursodatabase/limbo/issues/615

Closes #678
2025-01-14 14:17:39 +02:00
Pekka Enberg
537aea590f Merge 'Clean up license management ' from Kim Seon Woo
## Purpose of this PR
- Set up on how to manage licenses in limbo project
## Changes
- Move all licenses under `licenses` directory in the root path
- Add `NOTICE.md`
- Update `CONTRIBUTING.md` to explain how to add licenses to project
## Reference
https://github.com/tursodatabase/limbo/issues/615

Closes #677
2025-01-14 14:15:53 +02:00
Pekka Enberg
e6d389e89c Merge 'v2 of Consolidate libc implementations' from Jorge López Tello
This adds the missing parts of #668 which was just making the locks non-
blocking so they fail right away on conflict. Rustix was not very clear
on that, and the doc for `fcntl` didn't mention it, it was in the docs
for `flock`.
I have checked that this doesn't hang the test (🤦‍♂️).

Closes #676
2025-01-14 14:14:56 +02:00
김선우
b3883d03d6 Apply necessary changes for java 8 2025-01-14 20:14:32 +09:00
김선우
eacd7b7945 Change bindings/java to support java 8 2025-01-14 20:08:13 +09:00
김선우
e687ae3bdf Update README.md 2025-01-14 19:51:50 +09:00
김선우
8e4f6e7d30 Split serde-license.md to serde-apache-license.md and serde-mit-license.md 2025-01-14 19:44:13 +09:00
김선우
8ed2d14ca4 Reposition licenses in core package 2025-01-14 19:42:09 +09:00
김선우
66c9832bec Create top level licenses directory 2025-01-14 19:33:29 +09:00
Jorge López
a16282ea62 core: remove nix as a dependency 2025-01-14 11:06:13 +01:00
Jorge López
55e06b0c72 core/io: make file locks non-blocking so they fail right away 2025-01-14 11:01:11 +01:00
Pekka Enberg
b6ae8990e3 Limbo 0.0.12 2025-01-14 11:39:17 +02:00
Pekka Enberg
0461fd4c41 Update CHANGELOG 2025-01-14 11:38:26 +02:00
Pekka Enberg
a2506528d8 Merge '[bindings/java] Add support for Java nullability checks ' from Kim Seon Woo
## Purpose of this PR
- Add Nullability checks during compile time
## Changes
- Add `com.user.nullaway:nullaway` dependency for checking null issues
during compile time
  - Related dependencie(`com.google.errorprone`) and plugin
`net.ltgt.errorprone` is added
- Because added dependencies has Apache 2.0 License, added NOTICE.md
under the root directory.
  - Individual license for each dependency is placed under `licenses`
directory
## Note
- This PR has to be merged after
https://github.com/tursodatabase/limbo/pull/646
## Reference
- [Issue](https://github.com/tursodatabase/limbo/issues/615)

Closes #648
2025-01-14 11:25:48 +02:00
김선우
a48cb8bbfc Merge branch 'main' into java-nullability 2025-01-14 17:49:36 +09:00
Pekka Enberg
d4f55d4a1b Merge 'Change VALID_URL_PREFIX to use sqlite instead' from Kim Seon Woo
To maximize compatibility between limbo and sqlite, update
`VALID_URL_PREFIX`
## Reference
- [Issue](https://github.com/tursodatabase/limbo/issues/615)

Closes #674
2025-01-14 10:48:53 +02:00
Pekka Enberg
cd04bf796a Update README.md 2025-01-14 10:34:00 +02:00
Jussi Saurio
d3c2868fcb Merge 'refactor: json functions vdbe code should be consistent with scalar functions' from Jorge Hermo
While working to include new json functions (related to #127) I noticed
that the program step code for json functions https://github.com/tursoda
tabase/limbo/blob/0dceb02ec04241b3772332853bcbfb9eb559adb9/core/vdbe/mod
.rs#L1346 was a bit different from scalar functions's code, where the
match to the inner function is nested https://github.com/tursodatabase/l
imbo/blob/0dceb02ec04241b3772332853bcbfb9eb559adb9/core/vdbe/mod.rs#L142
4
I added the same nesting to the json functions so it is more consistent

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #663
2025-01-14 10:30:33 +02:00
Pekka Enberg
5c9505e8f7 Revert "core/io/io_uring: replace nix and libc calls with their rustix counterparts."
This reverts commit b146f5d4cb because it
causes tests to hang.
2025-01-14 10:25:23 +02:00
Pekka Enberg
d223c72d03 Revert "core: Previous commits didn't actually remove nix as dependency, so do that here"
This reverts commit cca3846f95, we need to
bring it back unfortunately.
2025-01-14 10:25:11 +02:00
김선우
04cd655574 Change VALID_URL_PREFIX to use sqlite instead 2025-01-14 17:19:33 +09:00
김선우
f604e227b1 Update java.yaml workflow to use java 17 2025-01-14 17:12:50 +09:00
김선우
f03b6bffde Update license path 2025-01-14 17:12:03 +09:00
김선우
5c9990f41a Merge branch 'main' into java-nullability 2025-01-14 17:09:45 +09:00
Pekka Enberg
945a91dee5 Merge 'core: Consolidate libc implementations' from Jorge López Tello
This shaves off `nix` as a dependency in core, which was only being used
in the io_uring backend for the `fcntl` advisory record locking. Since a
previous PR made `rustix` a dep not only for Mac but for any Unix, and
`rustix` also has `fcntl`, `nix` becomes redundant.
Furthermore, it reduces `libc` usage across core. The only remaining
uses are:
```rust
io_uring::opcode::Readv::new(fd, iovec as *const iovec as *const libc::iovec, 1)
io_uring::opcode::Writev::new(fd, iovec as *const iovec as *const libc::iovec, 1)
```
These two are a little ugly, but sadly the `io_uring` crate requires
both opcodes to take a `libc::iovec` while it doesn't export the type.
See tokio-rs/io-uring#259 for a request to use `std::io::IoSlice` or to
export the type directly.
Apart from those two, there are no other usages of libc, so if those are
resolved, we could also drop the dependency on libc.

Closes #668
2025-01-14 10:07:20 +02:00
Pekka Enberg
2186af6c89 Bump "build-native" timeout to 10 minutes 2025-01-14 10:07:00 +02:00
Pekka Enberg
a3b9cf823a Merge 'Initial pass on Rust bindings' from Pekka Enberg
This implements libSQL compatible Rust API on top of Limbo's core. The
purpose of this is to allow libraries and apps that build on libSQL to
use Limbo.

Closes #597
2025-01-14 09:19:50 +02:00