Commit Graph

2134 Commits

Author SHA1 Message Date
Jussi Saurio
b687cf66eb use bitfield for ended_coroutine 2025-01-26 13:12:47 +02:00
Jussi Saurio
9e32ce6c77 Add Cursor enum and store a single BTreeMap of cursors in ProgramState 2025-01-26 12:57:15 +02:00
Pekka Enberg
609839d79a Merge 'core: Fix Statement::reset()' from Pekka Enberg
The first rule of writing fast programs: don't use dynamic memory
allocation!
Brings `SELECT 1` micro-benchmark back to 70 ns, which is a bit closer
to what we had before.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #781
2025-01-26 11:56:44 +02:00
Pekka Enberg
8942c38bda core: Fix Statement::reset()
The first rule of writing fast programs: don't use dynamic memory
allocation!

Brings back some performance for the `SELECT 1` micro-benchmark,
although we're still not where we need to be.
2025-01-26 11:47:28 +02:00
Pekka Enberg
17e7e03423 Revert "cargo: Disable LTO.."
This reverts commit 4943217045 to get 5%
of performance back.

Before:

```
limbo/Execute prepared statement: 'SELECT 1'
                        time:   [111.45 ns 111.52 ns 111.61 ns]
                        thrpt:  [8.9594 Melem/s 8.9667 Melem/s 8.9727 Melem/s]
Found 8 outliers among 100 measurements (8.00%)
  2 (2.00%) low mild
  2 (2.00%) high mild
  4 (4.00%) high severe
```

```
limbo/Execute prepared statement: 'SELECT 1'
                        time:   [106.22 ns 106.48 ns 106.85 ns]
                        thrpt:  [9.3587 Melem/s 9.3911 Melem/s 9.4142 Melem/s]
                 change:
                        time:   [-4.8335% -4.6750% -4.5222%] (p = 0.00 < 0.05)
                        thrpt:  [+4.7364% +4.9043% +5.0790%]
                        Performance has improved.
```
2025-01-26 10:31:35 +02:00
Pekka Enberg
0d0906dce4 Merge 'simulator: implement --load and --watch flags' from Alperen Keleş
The current status of the PR is halfway. The new framing of simulation
runner where `setup_simulation` is separated from `run_simulation`
allows for injecting custom plans easily. The PR is currently missing
the functionality to update the `SimulatorEnv` ad hoc from the plan, as
the environment tables were typically created during the planning phase.
The next steps will be to implement a function `fn
mk_env(InteractionPlan, SimulatorEnv) -> SimulatorEnv`, add `--load`
flag to the CLI for loading a serialized plan file, making a
corresponding environment and running the simulation.
We can optionally combine this with a `--save` option, in which we keep
a seed-vault as part of limbo simulator, corresponding each seed with
its generated plan and save the time to regenerate existing seeds by
just loading them into memory. I am curious to hear thoughts on this?
Would the maintainers be open to adding such a seed-vault? Do you think
the saved time would be worth the complexity of the approach?

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #720
2025-01-26 08:52:58 +02:00
Pekka Enberg
6c80b1dc78 Merge 'bindings/go: Begin implementation of Go database/sql driver' from Preston Thorpe
This WIP driver uses the [purego](github.com/ebitengine/purego) library,
that supports cross platform `Dlopen`/`Dlsym` and not a whole lot else.
I really didn't want to use CGO, have very little experience with WASM
and I heard nothing but good things about this library. It's very easy
to use and stable especially when you consider the use case here of 3
functions.
![image](https://github.com/user-
attachments/assets/ae28c8f2-1d11-4d25-b999-22af8bd65a92)
NOTE: The WIP state that this PR is in right at this moment, is not able
to run these simple queries. This screengrab was taken from a couple
days ago when I wrote up a quick demo to load the library, call a simple
query and had it println! the result to make sure everything was working
properly.
I am opening this so kind of like the Java bindings, I can incrementally
work on this. I didn't want to submit a massive PR, try to keep them at
~1k lines max. The state of what's in this PR is highly subject and
likely to change.
I will update when they are at a working state where they can be tested
out and make sure they work across platforms.

Closes #776
2025-01-26 08:51:27 +02:00
Pekka Enberg
538b3ef823 Merge 'Implement strftime function' from Pedro Muniz
This PR implements the strftime scalar function. It uses underneath the
hood chrono's strftime function and the already existent modifier logic.
The caveat of using chrono's strftime implementation is that it supports
some additional syntax that is not supported in sqlite, such as
precision modifiers (%.3f). If this is a deal breaker for this function
to be implemented at the moment, I will then write a strftime
implementation from scratch.

Closes #778
2025-01-26 08:47:44 +02:00
Pekka Enberg
ef466cbdf7 Merge 'bindings/java: Add auto formatting for java bindings project ' from Kim Seon Woo
## Purpose of this PR
- Add bindings/java project level formatting style
## Changes
- Add spotless as the formatting tool
- USe Makefile to run lint on CI
## Reference
- [Corresponding PR](https://github.com/tursodatabase/limbo/pull/761)
should be merged first
- [Related Issue](https://github.com/tursodatabase/limbo/issues/615)

Closes #773
2025-01-26 08:47:00 +02:00
PThorpe92
32c985f9a8 Progress on Go bindings, add prepare + query statement 2025-01-25 23:01:46 -05:00
Pedro Muniz
9d858f5cb3 Merge branch 'tursodatabase:main' into feature/strftime 2025-01-25 16:23:32 -03:00
pedrocarlo
a316ab51ac feature: implement strftime function 2025-01-25 16:22:53 -03:00
PThorpe92
4be1f9c3cc Begin work on Go bindings (purego) 2025-01-25 11:37:12 -05:00
Pekka Enberg
aded7d3896 Merge 'Implement Or and And bytecodes' from Diego Reis
I think it is mostly correct, not so sure how to handle `BLOB`. One
thing that caught my attention is that sqlite seems to have a
optimization for trivial cases, saving some bytecodes, for instance:
![image](https://github.com/user-
attachments/assets/78b78a0d-5ab6-4a9e-aeac-fa97f1fc5c25)
I'm looking that right now.

Closes #777
2025-01-25 08:42:03 +02:00
Diego Reis
7902d5f200 Fix Text handling of And bytecode 2025-01-25 03:27:11 -03:00
김선우
7d81c32261 Add spotless support for bindings/java formatting 2025-01-25 15:25:13 +09:00
Pekka Enberg
12dd64a599 Merge 'chore: fix typos' from Sonny
while reading the code, I saw some typos so just fixing them.

Closes #770
2025-01-25 08:13:12 +02:00
Pekka Enberg
3c0d727c01 Merge 'Support returning column names from prepared statement' from Preston Thorpe
Closes #764
Go driver needs support for fetching the column names after `prepare`

Reviewed-by: Sonny (@sonhmai)

Closes #766
2025-01-25 08:12:21 +02:00
Pekka Enberg
27c9d06742 Merge 'Add Nyrkiö change point detection to 'cargo bench' workflow' from Henrik Ingo
This adds a separate push-only.yml workflow. For now pull request API
wasn't integrated yet, so shouldn't run on PRs.
disable cargo color
Fixes https://github.com/nyrkio/nyrkio/issues/304

Closes #762
2025-01-25 08:11:48 +02:00
Pekka Enberg
dde164f06a Merge 'Correct discord link' from Pedro Muniz
One of the discord links in the readme was pointing to a Chinese
discord. Both links are now pointing to Limbo's discord

Reviewed-by: Preston Thorpe (@PThorpe92)
Reviewed-by: Kim Seon Woo (@seonWKim)
Reviewed-by: Sonny (@sonhmai)

Closes #767
2025-01-25 08:11:31 +02:00
Pekka Enberg
d1b1ee2c96 Merge '[bindings/java] Fix failing tests ' from Kim Seon Woo
## The purpose of this PR
- Do not create a new `io` and instead use `io` which is used when
creating `LimboDatabase`.
- Run the loop of `step` function on the rust side(better for
performance as we don't have to switch between java <-> rust)
## Changes
- Java
  - handle invalid cases
  - remove @Disabled annotation from tests
- Rust
  - add loops in the step function
## Reference
- https://github.com/tursodatabase/limbo/issues/615

Closes #761
2025-01-25 08:08:34 +02:00
Diego Reis
e7d95399e3 Add Or bytecode
Take the logical OR of the values in register P1 and P2 and store the answer in register P3. If either P1 or P2 is nonzero (true) then the result is 1 (true) even if the other input is NULL. A NULL and false or two NULLs give a NULL output.
2025-01-25 02:54:14 -03:00
Diego Reis
aff454b5f6 Implement And bytecode
Take the logical AND of the values in registers P1 and P2 and write the result into register P3. If either P1 or P2 is 0 (false) then the result is 0 even if the other input is NULL. A NULL and true or two NULLs give a NULL output.
2025-01-25 02:12:50 -03:00
김선우
f10b41c5b5 Pass io to statement from db 2025-01-24 15:43:44 +09:00
김선우
53586b9d00 Break the loop when step() returns Err 2025-01-24 15:06:10 +09:00
김선우
0481e69217 Handle Err case from connection.io 2025-01-24 14:07:52 +09:00
김선우
36dff168b3 Execute io.run_once when receiving StepResult::IO 2025-01-24 13:52:54 +09:00
김선우
d05ffce613 Apply fmt 2025-01-24 13:52:54 +09:00
김선우
82e9fe0219 Handle invalid step results 2025-01-24 13:52:54 +09:00
김선우
f7a8d1b428 Change Java_org_github_tursodatabase_core_LimboStatement_step to run in loop to handle StepResult::IO 2025-01-24 13:52:54 +09:00
PThorpe92
545990f806 Support returning column names from prepared statement 2025-01-23 11:02:31 -05:00
sonhmai
fcd893284b chore: fix typos 2025-01-23 11:25:01 +07:00
pedrocarlo
667a3f594e change foreign discord link 2025-01-22 15:02:25 -03:00
Jussi Saurio
a26fc1619a Merge 'Fix select X'1'; causes limbo to go in infinite loop' from Krishna Vishal
Closes https://github.com/tursodatabase/limbo/issues/730.
Fixed `blog_literal` function to make it return `TK_ILLEGAL` token which
in turn now causes parser to stop and return `UnrecognizedTokenError`.
Added `TK_ILLEGAL` to `TokenType` Enum.
Behavior now:
```sql
SELECT X'1';  -- Odd number of hex digits -> TK_ILLEGAL -> unrecognized token error

  × unrecognized token at (1, 12)
   ╭────
 1 │ SELECT X'1';
   ·        ▲
   ·        ╰── here
   ╰────

SELECT X'AB'; -- Valid blob -> TK_BLOB -> parses successfully
�
SELECT X'G1'; -- Invalid hex digit -> TK_ILLEGAL -> unrecognized token error

  × unrecognized token at (1, 13)
   ╭────
 1 │ SELECT X'G1';
   ·        ▲
   ·        ╰── here
   ╰────

```

Reviewed-by: Jussi Saurio (@jussisaurio)

Closes #736
2025-01-21 11:32:10 +02:00
Jussi Saurio
5b00e4143e Merge 'Centralize Rust integration and regression tests' from Sonny
## What?
- centralized Rust integration and regression tests
- no new tests added
- no tests removed
- only refactored tests into modules and common utils
## Why?
- @penberg and I have a discussion [here](https://github.com/tursodataba
se/limbo/pull/694#discussion_r1921949665) about centralizing the
integration/ regression tests so that they are not scattered around.
- this is a PR to do that and some refactor of the existing tests to
make the structure easier to navigate + add new tests in the future.

Reviewed-by: Jussi Saurio (@jussisaurio)
Reviewed-by: Preston Thorpe (@PThorpe92)
Reviewed-by: Pere Diaz Bou (@pereman2)

Closes #753
2025-01-21 11:31:05 +02:00
sonhmai
a090fb927a centralize Rust integration and regression tests 2025-01-21 15:41:09 +07:00
Pekka Enberg
c282b23e6b Merge 'Implement Concat opcode' from Harin
This adds the Concat opcode to the VDBE.

Closes #758
2025-01-21 07:45:16 +02:00
Pekka Enberg
db3fa2514c Merge 'add PRAGMA statements in COMPAT doc' from Sonny
What? adding PRAGMA statements in COMPAT doc.
Why?
- each pragma is quite independent of each other.
- make it clear for contribution.
- maybe lower the bar of contributing.

Closes #760
2025-01-21 07:43:26 +02:00
Harin
ab3a15e489 Code refactor 2025-01-21 10:02:19 +05:30
sonhmai
dd436d3fc5 add PRAGMA statements in COMPAT doc 2025-01-21 11:27:22 +07:00
Henrik Ingo
c354219675 Add Nyrkiö change point detection to 'cargo bench' workflow
This adds a separate push-only.yml workflow. For now pull request
API wasn't integrated yet, so shouldn't run on PRs.

disable cargo color
2025-01-21 03:03:27 +02:00
Harin
da53cc3821 Added Concat Opcode 2025-01-21 00:29:23 +05:30
Pekka Enberg
c27427d644 Merge 'translate_condition_expr(): fix cases where 1. we jump on false and 2. either operand is NULL' from Jussi Saurio
Change explanation is in the code comment for `Insn::Eq`:
```
        /// Jump if either of the operands is null. Used for "jump when false" logic.
        /// Eg. "SELECT * FROM users WHERE id = NULL" becomes:
        /// <JUMP TO NEXT ROW IF id != NULL>
        /// Without the jump_if_null flag it would not jump because the logical comparison "id != NULL" is never true.
        /// This flag indicates that if either is null we should still jump.
        jump_if_null: bool,
```
Closes #754
Excerpt from SQLite bytecode documentation for e.g. `Lt`:
> If the SQLITE_JUMPIFNULL bit of P5 is set and either reg(P1) or
reg(P3) is NULL then the take the jump. If the SQLITE_JUMPIFNULL bit is
clear then fall through if either operand is NULL.
I didn't want to start putting these flags into a bitmask so I just
added a separate boolean. Probably for sqlite `EXPLAIN` compatibility we
should, IF we want to be exactly compatible (which we aren't anyway atm)

Closes #755
2025-01-20 19:40:08 +02:00
Pekka Enberg
2cea85c13b github: Disable failing Nyrkiö workflow
We can revert this patch once the following issue is fixed:

https://github.com/nyrkio/nyrkio/issues/304
2025-01-20 19:09:17 +02:00
Jussi Saurio
ce15ad7d32 Simplify added tests with foreach 2025-01-20 17:30:04 +02:00
Jussi Saurio
2cd9118be6 Fix jump_if_true to be a bool literal in places where it was used as a register number 2025-01-20 17:13:34 +02:00
Jussi Saurio
f88a4d6ac6 Add jump_if_null to cmp insns to account for either operand being NULL 2025-01-20 16:54:39 +02:00
Pekka Enberg
2e5cccee90 Merge 'Fix parser panic when duplicate column names are given to CREATE TABLE' from Krishna Vishal
Currently, we are using the same `Parser` instance across multiple
queries (9cc9577c91).  But during this
error the parser is not finalized, so parser stack is not reset, thereby
triggering the assertion at https://github.com/tursodatabase/limbo/blob/
15f7928551435458e2991dde3a76ce71e8a32057/vendored/sqlite3-
parser/third_party/lemon/lempar.rs#L663
This PR fixes the panic by calling `sqlite3ParserFinalize()` in the case
of error.
Closes https://github.com/tursodatabase/limbo/issues/742
-----------
`git bisect` FTW

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #752
2025-01-20 16:36:37 +02:00
Krishna Vishal
b43c1544d4 Tokenizer::split() function returns TK_ILLEGAL instead of Error. 2025-01-20 19:37:15 +05:30
Krishna Vishal
04fd5a40d6 Finalize the parser in the case of Error while running queries. This resets the parser stack and prevents triggering the assertion and thereby panic.
Closes https://github.com/tursodatabase/limbo/issues/742
2025-01-20 16:10:35 +05:30