Commit Graph

2182 Commits

Author SHA1 Message Date
Jussi Saurio
c950b177aa Add test for property: vector -> text -> vector produces an equal vector 2025-01-28 16:09:48 +02:00
Jussi Saurio
e01555467f Add quickcheck property tests for vector extension 2025-01-28 15:53:11 +02:00
Pekka Enberg
a9d2ef3878 Update COMPAT.md 2025-01-28 14:26:09 +02:00
Pekka Enberg
b9c89e79c2 testing: Add few TCL tests for vector extensions 2025-01-28 14:24:09 +02:00
Pekka Enberg
ee05ad172b core: Bundle vector extension by default 2025-01-28 14:24:09 +02:00
Pekka Enberg
9462426685 Vector extension functions
This patch adds some libSQL vector extension functions such as
`vector()` and `vector_distance_cos()`, which can be used for exact
nearest neighbor search as follows:

```
limbo> SELECT embedding, vector_distance_cos(embedding, '[9, 9, 9]')
   ...> FROM movies ORDER BY vector_distance_cos(embedding, '[9, 9, 9]');
[4, 5, 6]|0.013072490692138672
[1, 2, 3]|0.07417994737625122
```
2025-01-28 14:24:09 +02:00
Pekka Enberg
0eedea0bdb Merge 'Fix panic when double quoted strings are used for column names.' from Krishna Vishal
Now:
```sql
limbo> create table t (a,b,c); insert into t (a,b,c) values ("hello", 234, 432);
thread 'main' panicked at core/translate/expr.rs:1621:29:
internal error: entered unreachable code: Id should be resolved to a Column before translation
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
After fix:
```sql
limbo> create table t (a,b,c); insert into t (a,b,c) values ("hello", 234, 432);

  × Parse error: no such column: "hello" - should this be a string literal in single-quotes?

limbo>
```
Closes #800

Reviewed-by: Diego Reis (@diegoreis42)
Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #801
2025-01-28 14:20:58 +02:00
Pekka Enberg
d164ad1035 Merge 'Improve docs' from Sonny
Improving docs
1. adding journaling modes in compat
2. reformat the tables in compat
3. adding ref resources for DST

Closes #804
2025-01-28 14:19:18 +02:00
Pekka Enberg
3baca19f75 Merge 'bindings/java: Implement close() for LimboStatement and LimboResultSet ' from Kim Seon Woo
## Purpose of this PR
- Implement `close()` method for `LimboStatement`(+`JDBC4Statement`) and
`LimboResultSet`(+ `JDBC4ResultSet`)
## Changes
- Add `consumeAll` method in `LimboResultSet`
- Implement `close()` methods
  - Because  `JDBC4Statement` has longer lifecycle in compared to
`LimboStatement`, we manage different `close` fields(`LimboStatement` is
created when first `execute` method is called on `JDBC4Statemenet`)
## Reference
- [Issue](https://github.com/tursodatabase/limbo/issues/615)

Closes #799
2025-01-28 14:18:01 +02:00
Pekka Enberg
810ad77951 Merge 'bindings/java: Add README.md for bindings/java' from Kim Seon Woo
## Purpose of thie PR
- Add README.md for bindings/java
## Reference
- https://github.com/tursodatabase/limbo/issues/615

Closes #802
2025-01-28 14:17:47 +02:00
sonhmai
f59379e0c9 fix wrong replacement in compat 2025-01-28 11:35:49 +07:00
sonhmai
d4e5f83932 add references for dst 2025-01-28 11:32:17 +07:00
sonhmai
fc4192362d add journaling modes to compat 2025-01-28 11:13:37 +07:00
김선우
55a7cd230e Add README.md for bindings/java 2025-01-28 10:48:44 +09:00
김선우
b17511b559 Fix test to use lighter query 2025-01-28 09:25:57 +09:00
김선우
a82c459ed0 Implement close on LimboStatement and JDBC4Statement independently 2025-01-28 09:20:07 +09:00
Krishna Vishal
61d60cf4b7 Fix panic when double quoted strings are used for column names in SQL statements. Double quoted strings are not supported in SQLite. Fixes https://github.com/tursodatabase/limbo/issues/800 2025-01-27 23:15:04 +05:30
Pekka Enberg
ad8e05b9a1 Merge 'bindings/java: Implement LimboConnection close() ' from Kim Seon Woo
## Purpose of thie PR
- Implement `close()` method for `LimboConnection` and `JDBC4Connection`
## Changes
- Instead of closing the database when closing connections, close the
connection instead
## References
- https://github.com/tursodatabase/limbo/issues/615

Closes #797
2025-01-27 14:51:04 +02:00
김선우
eeed305b07 Nit 2025-01-27 20:40:43 +09:00
김선우
c18418bed0 Nit 2025-01-27 20:35:11 +09:00
김선우
e48d7aa763 Add consumeAll method in LimboResultSet and let JDBC4Statement to use it 2025-01-27 20:31:37 +09:00
김선우
4dd2d1c64a Implement close() on LimboStatement 2025-01-27 20:20:10 +09:00
김선우
6252947232 Fix the logic on determining whether connection is closed 2025-01-27 19:49:10 +09:00
김선우
1125e51a59 Add test for close() 2025-01-27 19:19:34 +09:00
김선우
dae15ef0e3 Implement limbo close() 2025-01-27 19:15:02 +09:00
Pekka Enberg
ef6a1be335 Merge 'Enable static linking for 'built-in' extensions' from Preston Thorpe
This PR introduces the ability to build and link with an extension
library, enabling features like `uuid` to not have to be shipped as
independent libraries and loaded at runtime.
To build and link with an extension, you simply add it as a dependency
with the `static` feature, and call register_extension_static. in this
case, we feature flag that with `uuid`
```rust
    #[cfg(feature = "uuid")]
    pub fn register_uuid(&self) -> Result<(), String> {
        let ext_api = Box::new(self.build_limbo_ext());
        if unsafe { !limbo_uuid::register_extension_static(&ext_api).is_ok() } {
            return Err("Failed to register uuid extension".to_string());
        }
        Ok(())
    }
```
So fortunately wasm targets are no longer excluded from extensions, only
loading them at runtime for now

Closes #737
2025-01-27 09:49:48 +02:00
Pekka Enberg
e8600fa2a1 Merge branch 'main' into static 2025-01-27 09:49:34 +02:00
Pekka Enberg
5a1e1f690a Merge 'bindings/java: Log driver loading error' from Pekka Enberg
Reviewed-by: Kim Seon Woo (@seonWKim)

Closes #791
2025-01-27 09:09:03 +02:00
Pekka Enberg
6f93f290e1 bindings/java: Log driver loading error 2025-01-26 21:07:18 +02:00
Pekka Enberg
9b9d5f9b4c Merge 'bindings/go: Rename to Limbo' from Pekka Enberg
...we'll likely call this Turso eventually, but right now, let's keep
the code consistent.

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #790
2025-01-26 21:04:43 +02:00
Pekka Enberg
0918fc40d4 bindings/go: Rename to Limbo
...we'll likely call this Turso eventually, but right now, let's keep
the code consistent.
2025-01-26 20:58:10 +02:00
Pekka Enberg
4f1548d27b Merge 'bindings/java: Apply spotless formatting in bindings/java project ' from Kim Seon Woo
## Changes
- Add lint related commands in Makefile
- Apply spotless formatting
## Related
- [Issue](https://github.com/tursodatabase/limbo/issues/615)

Closes #783
2025-01-26 20:48:42 +02:00
Pekka Enberg
089e4e66eb Merge 'Implemented json_valid function' from Harin
Closes #789
2025-01-26 20:48:16 +02:00
Pekka Enberg
76e76ba404 Merge 'core: Remove database header from BTreeCursor' from Pekka Enberg
It's already in the pager so use it from there to reduce the size of the
`BTreeCursor` struct.

Closes #788
2025-01-26 20:48:03 +02:00
Harin
0903b9b019 Implemented JSON valid function 2025-01-26 23:35:47 +05:30
Pekka Enberg
983875c443 core: Remove database header from BTreeCursor
It's already in the pager so use it from there to reduce the size of the
`BTreeCursor` struct.
2025-01-26 16:48:12 +02:00
Pekka Enberg
359576d910 Merge 'core: Kill Rows wrapper struct' from Pekka Enberg
It's just an useless wrapper, kill it.

Closes #786
2025-01-26 16:40:51 +02:00
Pekka Enberg
7967cc5efc core: Kill Rows wrapper struct
It's just an useless wrapper, kill it.
2025-01-26 16:27:19 +02:00
Pekka Enberg
9f3b2dcc92 Merge 'Program state optimizations' from Jussi Saurio
Current micro-optimizations:
- store single data structure for cursors in programstate, not one for
every cursor type
- use vec for cursors instead of btreemap
- use knowledge of cursor amount to pre-allocate cursor vec capacity
- don't use resize in ::new() and ::reset(), instead populate directly
and reset by iterating over each elem (somehow this is faster)
- use bitfield for ended_coroutine instead of hashmap

Closes #782
2025-01-26 16:05:04 +02:00
Jussi Saurio
faa6d0c69d use vec for cursors, not map 2025-01-26 14:04:05 +02:00
Jussi Saurio
cdafc9033e ProgramState::registers - no resize 2025-01-26 13:56:09 +02:00
Jussi Saurio
b687cf66eb use bitfield for ended_coroutine 2025-01-26 13:12:47 +02:00
김선우
2212cc2a09 Apply lints 2025-01-26 20:04:57 +09:00
김선우
8b6e761496 Add lint commands in Makefile 2025-01-26 20:04:30 +09:00
Jussi Saurio
9e32ce6c77 Add Cursor enum and store a single BTreeMap of cursors in ProgramState 2025-01-26 12:57:15 +02:00
Pekka Enberg
609839d79a Merge 'core: Fix Statement::reset()' from Pekka Enberg
The first rule of writing fast programs: don't use dynamic memory
allocation!
Brings `SELECT 1` micro-benchmark back to 70 ns, which is a bit closer
to what we had before.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #781
2025-01-26 11:56:44 +02:00
Pekka Enberg
8942c38bda core: Fix Statement::reset()
The first rule of writing fast programs: don't use dynamic memory
allocation!

Brings back some performance for the `SELECT 1` micro-benchmark,
although we're still not where we need to be.
2025-01-26 11:47:28 +02:00
Pekka Enberg
17e7e03423 Revert "cargo: Disable LTO.."
This reverts commit 4943217045 to get 5%
of performance back.

Before:

```
limbo/Execute prepared statement: 'SELECT 1'
                        time:   [111.45 ns 111.52 ns 111.61 ns]
                        thrpt:  [8.9594 Melem/s 8.9667 Melem/s 8.9727 Melem/s]
Found 8 outliers among 100 measurements (8.00%)
  2 (2.00%) low mild
  2 (2.00%) high mild
  4 (4.00%) high severe
```

```
limbo/Execute prepared statement: 'SELECT 1'
                        time:   [106.22 ns 106.48 ns 106.85 ns]
                        thrpt:  [9.3587 Melem/s 9.3911 Melem/s 9.4142 Melem/s]
                 change:
                        time:   [-4.8335% -4.6750% -4.5222%] (p = 0.00 < 0.05)
                        thrpt:  [+4.7364% +4.9043% +5.0790%]
                        Performance has improved.
```
2025-01-26 10:31:35 +02:00
Pekka Enberg
0d0906dce4 Merge 'simulator: implement --load and --watch flags' from Alperen Keleş
The current status of the PR is halfway. The new framing of simulation
runner where `setup_simulation` is separated from `run_simulation`
allows for injecting custom plans easily. The PR is currently missing
the functionality to update the `SimulatorEnv` ad hoc from the plan, as
the environment tables were typically created during the planning phase.
The next steps will be to implement a function `fn
mk_env(InteractionPlan, SimulatorEnv) -> SimulatorEnv`, add `--load`
flag to the CLI for loading a serialized plan file, making a
corresponding environment and running the simulation.
We can optionally combine this with a `--save` option, in which we keep
a seed-vault as part of limbo simulator, corresponding each seed with
its generated plan and save the time to regenerate existing seeds by
just loading them into memory. I am curious to hear thoughts on this?
Would the maintainers be open to adding such a seed-vault? Do you think
the saved time would be worth the complexity of the approach?

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #720
2025-01-26 08:52:58 +02:00
Pekka Enberg
6c80b1dc78 Merge 'bindings/go: Begin implementation of Go database/sql driver' from Preston Thorpe
This WIP driver uses the [purego](github.com/ebitengine/purego) library,
that supports cross platform `Dlopen`/`Dlsym` and not a whole lot else.
I really didn't want to use CGO, have very little experience with WASM
and I heard nothing but good things about this library. It's very easy
to use and stable especially when you consider the use case here of 3
functions.
![image](https://github.com/user-
attachments/assets/ae28c8f2-1d11-4d25-b999-22af8bd65a92)
NOTE: The WIP state that this PR is in right at this moment, is not able
to run these simple queries. This screengrab was taken from a couple
days ago when I wrote up a quick demo to load the library, call a simple
query and had it println! the result to make sure everything was working
properly.
I am opening this so kind of like the Java bindings, I can incrementally
work on this. I didn't want to submit a massive PR, try to keep them at
~1k lines max. The state of what's in this PR is highly subject and
likely to change.
I will update when they are at a working state where they can be tested
out and make sure they work across platforms.

Closes #776
2025-01-26 08:51:27 +02:00