Commit Graph

2984 Commits

Author SHA1 Message Date
Nikita Sivukhin
db7544fe7a Merge branch 'tursodatabase:main' into fix-and-predicate 2025-02-15 22:57:56 +04:00
Nikita Sivukhin
becc58565d run IO in fuzz tests 2025-02-15 22:48:01 +04:00
Jussi Saurio
aea8b416bc Merge 'mvcc: comments and small cleanup' from Jussi Saurio
Closes #930
2025-02-15 18:18:01 +02:00
Jussi Saurio
01f51a4c20 Merge 'Fix math binary' from Nikita Sivukhin
Fix codegen for binary functions and add fuzz test for math functions
(we need to compile `rusqlite` with `-DSQLITE_ENABLE_MATH_FUNCTIONS` in
order to bundle sqlite with math functions compiled)

Reviewed-by: Jussi Saurio (@jussisaurio)

Closes #1015
2025-02-15 18:17:08 +02:00
Jussi Saurio
cbfd77849d Merge 'Fix substr' from Nikita Sivukhin
Align `substr` implementation with SQLite spec
(https://www.sqlite.org/lang_corefunc.html#substr):
> The substr(X,Y,Z) function returns a substring of input string X that
begins with the Y-th character and which is Z characters long. If Z is
omitted then substr(X,Y) returns all characters through the end of the
string X beginning with the Y-th. The left-most character of X is number
1. If Y is negative then the first character of the substring is found
by counting from the right rather than the left. If Z is negative then
the abs(Z) characters preceding the Y-th character are returned. If X is
a string then characters indices refer to actual UTF-8 characters. If X
is a BLOB then the indices refer to bytes.

Reviewed-by: Jussi Saurio (@jussisaurio)

Closes #1013
2025-02-15 18:16:57 +02:00
Pekka Enberg
d1282dc413 Merge 'Fix IdxGt, IdxGe, IdxLt, and IdxLe instructions' from Jussi Saurio
According to SQLite documentation, the way to use these instructions is
to compare the seek key to the index key as you would with the Compare
opcode. The compare opcode states:
> "Compare two vectors of registers in reg(P1)..reg(P1+P3-1) (call this
vector "A") and in reg(P2)..reg(P2+P3-1) ("B")."
In other words, we should compare the same number of columns from each,
not compare the entire keys.
This fixes a few Clickbench queries returning incorrect results, and so
closes #1009
---
Future work: support index seek keys that use multiple columns. Our
index seek is many times slower than SQLite because we're not utilizing
all the possible columns -- instead we just use the first index column
to seek and then make the rest of the comparisons as normal loop
condition expressions.
Note that IdxLE and IdxLT are currently "dead" opcodes since they were
recently introduced in #1010 but not yet utilized, as we lack support
for descending indexes

Closes #1016
2025-02-15 17:07:06 +02:00
Nikita Sivukhin
3233d60bea trying to debug hungs in CI 2025-02-15 18:03:29 +04:00
Nikita Sivukhin
ee8b03528d fix codegen for and predicate - as jump_if_condition_is_true can be overwritten higher in the stack 2025-02-15 16:27:27 +04:00
Nikita Sivukhin
e7c501a3be add simple fuzz test with table data 2025-02-15 16:27:11 +04:00
Nikita Sivukhin
10d0dc218b add comment about config.toml env var 2025-02-15 15:05:36 +04:00
Nikita Sivukhin
7fa6ff9bb3 reset rust caches in the CI 2025-02-15 14:57:59 +04:00
Nikita Sivukhin
32ec8f64d5 move .config to the top level in order to have rusqlite dep built identically across multiple projects 2025-02-15 14:54:19 +04:00
Jussi Saurio
e4541edb48 Fix IdxGt,IdxGe,IdxLt,IdxLe instructions
According to SQLite documentation, the way to use these instructions
is to compare the seek key to the index key as you would with the
Compare opcode. The compare opcode states:

"Compare two vectors of registers in reg(P1)..reg(P1+P3-1)
(call this vector "A") and in reg(P2)..reg(P2+P3-1) ("B")."

In other words, we should compare the same number of columns from each,
not compare the entire keys.

This fixes a few Clickbench queries returning incorrect results, and
so closes #1009

---

Future work: support index seek keys that use multiple columns. Our
index seek is many times slower than SQLite because we're not utilizing
all the possible columns -- instead we just use the first index column
to seek.
2025-02-15 12:44:56 +02:00
Nikita Sivukhin
ac263fa5a8 reset rust caches in the CI 2025-02-15 14:42:03 +04:00
Nikita Sivukhin
e25660833b fix codegen for binary functions 2025-02-15 14:33:57 +04:00
Nikita Sivukhin
610d41ae9e add fuzz test for math functions 2025-02-15 14:33:37 +04:00
Pekka Enberg
f220f9a948 Merge 'Add modified clickbench script to repo + CI' from Jussi Saurio
Closes #1014
2025-02-15 12:05:15 +02:00
Jussi Saurio
e8c0717f5b Add modified clickbench script to repo + CI 2025-02-15 11:48:06 +02:00
Nikita Sivukhin
91d723016d fix test 2025-02-15 13:29:14 +04:00
Nikita Sivukhin
2ee5382689 add substr cases in TCL tests 2025-02-15 13:25:49 +04:00
Nikita Sivukhin
b35dab5b6d fix substr implementation 2025-02-15 13:21:43 +04:00
Nikita Sivukhin
aa3cafba51 add fuzz test for replace and substr 2025-02-15 13:21:40 +04:00
Pekka Enberg
30f700174c Merge 'Added IdxLE and IdxLT opcodes' from Omolola Olamide
I added the two opcodes as an initial step. They are pretty easy to
implement since we already have the counterparts i.e., IdxGE and IdxGT
Is there a design reason behind their omission @penberg @PThorpe92?
I noticed the same for SeekLE and SeekLT.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1010
2025-02-15 11:10:17 +02:00
Pekka Enberg
7e173291d5 Merge 'cleanup shell tests and cli' from Clyde K.
Refactored CLI, cleaned up duplicate code.
@PThorpe92

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #941
2025-02-15 11:08:58 +02:00
Pekka Enberg
20e37e485a Merge 'Implement the legacy_file_format pragma' from Glauber Costa
easy implementation, sqlite claims it is a noop now
"This pragma no longer functions. It has become a no-op. The
capabilities formerly provided by PRAGMA legacy_file_format are now
available using the SQLITE_DBCONFIG_LEGACY_FILE_FORMAT option to the
sqlite3_db_config() C-language interface."

Closes #1007
2025-02-15 11:03:05 +02:00
Pekka Enberg
c53897cb6c Merge 'Support numeric column references in GROUP BY' from Jussi Saurio
We already supported this for ORDER BY but not GROUP BY - again noticed
this when running against some clickbench queries

Closes #1008
2025-02-15 11:02:28 +02:00
Pekka Enberg
71b15ae4f1 Merge 'cli: Basic dump support' from Glauber Costa
Basic dump support
This is a basic support for the very useful .dump command. It doesn't
yet implement any of the dump options sqlite has, and it doesn't add
some of the logic for things like indexes, since we don't have them.

Closes #1011
2025-02-15 10:49:56 +02:00
Pekka Enberg
a853e6f538 Merge 'select: fix bug with referring to a mixed-case alias' from Jussi Saurio
was running some clickbench queries against limbo and noticed this one

Closes #1006
2025-02-15 10:49:17 +02:00
Glauber Costa
82ceaebe01 Basic dump support
This is a basic support for the very useful .dump command.
It doesn't yet implement any of the dump options sqlite has,
and it doesn't add some of the logic for things like indexes,
since we don't have them.
2025-02-14 15:54:46 -05:00
[B
e52f218136 fix format 2025-02-14 21:10:51 +01:00
[B
5214cf9859 Added IdxLE and IdxLT opcodes 2025-02-14 20:22:30 +01:00
Jussi Saurio
eca196a54b Support numeric column references in GROUP BY 2025-02-14 16:57:45 +02:00
Glauber Costa
fbe439f6c2 Implement the legacy_file_format pragma
easy implementation, sqlite claims it is a noop now

"This pragma no longer functions. It has become a no-op. The capabilities
formerly provided by PRAGMA legacy_file_format are now available using
the SQLITE_DBCONFIG_LEGACY_FILE_FORMAT option to the sqlite3_db_config()
C-language interface."
2025-02-14 09:50:29 -05:00
Jussi Saurio
a7300a4e0c select: fix bug with referring to a mixed-case alias 2025-02-14 16:40:58 +02:00
Pekka Enberg
d9f4558255 build: Clean up top-level Cargo.toml 2025-02-14 12:43:17 +02:00
Pekka Enberg
a302828376 core/transalte: Fix invalid sqlite3_parser import 2025-02-14 12:27:57 +02:00
Pekka Enberg
43574528c9 Merge 'Switch to workspace dependencies' from Pekka Enberg
...makes it easier to specify a version, which is needed for `cargo
publish`.

Closes #991
2025-02-14 12:21:24 +02:00
Pekka Enberg
64cdfd829e Merge 'core/translate: BEGIN EXCLUSIVE support' from Pekka Enberg
After reading the fine print, SQLite documentation explains that `BEGIN
IMMEDIATE` and `BEGIN EXCLUSIVE` are the same thing in WAL mode:
https://www.sqlite.org/lang_transaction.html
As that's the only mode we support, let's just add code generation for
`BEGIN EXCLUSIVE`.
Fixes #1002

Closes #1003
2025-02-14 12:20:22 +02:00
Pekka Enberg
20e881ae0d testing: Disable transaction tests for now 2025-02-14 12:19:58 +02:00
Pekka Enberg
e3925c4531 core/vdbe: Fix Program::halt() to respect autocommit state 2025-02-14 12:11:37 +02:00
Pekka Enberg
b949ef5360 Update COMPAT.md 2025-02-14 11:58:54 +02:00
Pekka Enberg
76bdbb54ef core/translate: BEGIN EXCLUSIVE support
After reading the fine print, SQLite documentation explains that `BEGIN
IMMEDIATE` and `BEGIN EXCLUSIVE` are the same thing in WAL mode:

https://www.sqlite.org/lang_transaction.html

As that's the only mode we support, let's just add code generation for
`BEGIN EXCLUSIVE`.

Fixes #1002
2025-02-14 11:52:18 +02:00
Pekka Enberg
567a2e9a9f Merge 'BEGIN IMMEDIATE + COMMIT support' from Pekka Enberg
Closes #1000
2025-02-14 11:48:38 +02:00
Pekka Enberg
ae3c6b7ec5 core/vdbe: Fix AutoCommit instruction to halt the VM
Pointed out by Jussi
2025-02-14 11:43:16 +02:00
Pekka Enberg
948585bb42 core/vdbe: Extract Program::halt() helper
We need this for AutoCommit opcode too.
2025-02-14 11:42:24 +02:00
Pekka Enberg
4b7270de11 Merge 'fix some typo' from meteorgan
Closes #999
2025-02-14 10:37:40 +02:00
Pekka Enberg
a8cadf205c Merge 'doc: Use limbo_cli as package instead for running limbo cli' from Takayuki Maeda
I found invalid option while reading CONTRIBUTING.md and trying some
commands in it.
error:
```
$ cargo run --package limbo --bin limbo database.db
error: no bin target named `limbo`.

$ rustc --version
rustc 1.83.0 (90b35a623 2024-11-26)
```
ref: https://github.com/tursodatabase/limbo/blob/291637cc7120303fd7337c3
42cf5dbc9363faa85/cli/Cargo.toml#L17

Closes #995
2025-02-14 10:35:42 +02:00
Pekka Enberg
076331d8cf testing: Basic BEGIN + END test 2025-02-14 10:26:55 +02:00
Pekka Enberg
5626ca450f core/translate: COMMIT support
```
limbo> EXPLAIN COMMIT;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     3     0                    0   Start at 3
1     AutoCommit         1     0     0                    0   auto_commit=true, rollback=false
2     Halt               0     0     0                    0
3     Goto               0     1     0                    0
```
2025-02-14 10:26:55 +02:00
Pekka Enberg
9fff9f6081 core/translate: BEGIN IMMEDIATE support
Emit the following code sequence for `BEGIN IMMEDIATE`:

```
limbo> EXPLAIN BEGIN IMMEDIATE;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     4     0                    0   Start at 4
1     Transaction        0     1     0                    0
2     AutoCommit         0     0     0                    0   auto_commit=false, rollback=false
3     Halt               0     0     0                    0
4     Goto               0     1     0                    0
```

Please note that SQLite emits *two* transaction instructions -- one for
main database and one for temporary tables. However, since we don't
support the latter, we only emit one transaction instruction.
2025-02-14 10:26:31 +02:00