Commit Graph

5376 Commits

Author SHA1 Message Date
pedrocarlo
b3351dc709 tests + adjustment to halt error message 2025-06-20 16:29:10 -03:00
pedrocarlo
9048ad398b modify loop functions to accomodate for ephemeral tables 2025-06-20 16:29:10 -03:00
pedrocarlo
eda9d20a0b tests 2025-06-20 16:28:10 -03:00
pedrocarlo
74beac5ea8 ephemeral table for update when rowid is being update 2025-06-20 16:28:10 -03:00
Jussi Saurio
edc6eb9a36 Merge 'Use UV more in python related scripts and actions' from Pedro Muniz
This PR initializes an UV project in `antithesis_tests` so that we can
have an easier time to track dependencies and build pylimbo
automatically for our environment. Consequently, making it easier to
create new antithesis tests in the future with better IDE support.
Also modified our Github actions to check python linting with Ruff, and
removed unnecessary Python jobs. With that, I applied the Ruff fixes
which is the cause of the many file changes.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1782
2025-06-20 22:04:56 +03:00
Jussi Saurio
f396528d53 Merge 'Fix DELETE not emitting constant WhereTerms' from Pedro Muniz
Fixes DELETE not emitting conditional jumps at all if the associated
WhereTerm is a constant, e.g.
```sql
limbo> create table t(x);
limbo> explain DELETE FROM t WHERE 5-5;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     7     0                    0   Start at 7
1     OpenWrite          0     2     0                    0   root=2; t
2     Rewind             0     6     0                    0   Rewind table t
3       RowId            0     1     0                    0   r[1]=t.rowid
4       Delete           0     0     0                    0
5     Next               0     3     0                    0
6     Halt               0     0     0                    0
7     Transaction        0     1     0                    0   write=true
8     Goto               0     1     0                    0
```
I was adding more stuff to the simulator in a Branch of mine, and I
caught this error with delete. Upstreaming the fix here. As we do with
Update, I added the translation step for the `WhereTerms` of the query.
Edit: Closes #1732. Closes #1733. Closes #1734. Closes #1735. Closes
#1736. Closes #1738. Closes #1739. Closes #1740.
Edit: Also pushes constant where term translation to `init_loop` for
Update and Select as well.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1746
2025-06-20 22:00:32 +03:00
pedrocarlo
8b8f43334e remove python makefile as it is obsolete with UV 2025-06-20 15:59:03 -03:00
pedrocarlo
80ccca8827 ruff lint fix 2025-06-20 15:59:03 -03:00
pedrocarlo
50c8b2ca2e extract ruff lint rules to workspace 2025-06-20 15:59:03 -03:00
pedrocarlo
46135e43ce use uv in github actions 2025-06-20 15:59:03 -03:00
pedrocarlo
b21b6db448 adjust project config to use UV 2025-06-20 15:59:03 -03:00
Jussi Saurio
c69047106c Merge 'Implement RowData opcode' from meteorgan
The `RowData` opcode is required to implement #1575.
I haven't found a ideal way to test this PR independently, but I
verified its functionality while working on #1575(to be committed soon),
and it performs effectively.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1756
2025-06-20 21:58:47 +03:00
Jussi Saurio
38f6b8c031 Merge 'Support indent for Goto opcode when executing explain' from meteorgan
it works as expected
```
limbo> explain insert into tb1 select * from tb2 union select * from tb3;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     29    0                    0   Start at 29
1     InitCoroutine      1     21    2                    0
2     OpenEphemeral      0     0     0                    0   cursor=0 is_table=false
3     OpenRead           1     3     0                    0   table=tb2, root=3
4     Rewind             1     9     0                    0   Rewind table tb2
5       Column           1     0     2                    0   r[2]=tb2.age
6       MakeRecord       2     1     3                    0   r[3]=mkrec(r[2..2]); for union_dedupe
7       IdxInsert        0     3     0                    0   key=r[3]
8     Next               1     5     0                    0
9     OpenRead           2     4     0                    0   table=tb3, root=4
10    Rewind             2     15    0                    0   Rewind table tb3
11      Column           2     0     2                    0   r[2]=tb3.age
12      MakeRecord       2     1     4                    0   r[4]=mkrec(r[2..2]); for union_dedupe
13      IdxInsert        0     4     0                    0   key=r[4]
14    Next               2     11    0                    0
15    Rewind             0     18    0                    0   Rewind  union_dedupe
16      Column           0     0     2                    0   r[2]=union_dedupe.age
17      Yield            1     0     0                    0
18    Next               0     16    0                    0
19    Close              0     0     0                    0
20    EndCoroutine       1     0     0                    0
21    OpenWrite          3     2     0                    0   root=2; tb1
22      Yield            1     28    0                    0
23      Copy             2     7     0                    0   r[7]=r[2]
24      NewRowid         3     6     0                    0   r[6]=rowid
25      MakeRecord       7     1     8                    0   r[8]=mkrec(r[7..7])
26      Insert           3     8     6     tb1            0   intkey=r[6] data=r[8]
27    Goto               0     22    0                    0
28    Halt               0     0     0                    0
29    Transaction        0     1     0                    0   write=true
30    Goto               0     1     0                    0
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1775
2025-06-20 21:57:40 +03:00
Jussi Saurio
020f083b98 Merge 'Fix handling of non-aggregate expressions' from Piotr Rżysko
This PR has two parts:
1. The first commit refactors how information about which registers
should be populated in the aggregation loop is calculated and
propagated. This simplification revealed a bug, which is addressed as
part of the same commit (see the included test).
2. The second commit fixes incorrect behavior for queries where complex
expressions include both aggregate and non-aggregate components. For
example, the following query previously produced incorrect results:
```sql
SELECT
  CASE WHEN c0 != 'x' THEN group_concat(c1, ',') ELSE 'x' END
FROM t0
GROUP BY c0;
```
In such cases, non-aggregate columns like `c0` were not available during
the result construction for each group, leading to incorrect evaluation.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1780
2025-06-20 21:56:35 +03:00
Pere Diaz Bou
ca3a482d48 Merge 'Fix fuzz issue #1763 by using the log2 & log10 functions where applicable' from Luca Muscat
It is easy to chalk this fuzzer issue to erratic floating point
behaviour, but this is not the case here.
Currently, `exec_math_log` calculates log with arbitrary bases by using
the following formula: `log_a(b) ~= ln(b) / ln(a)`. This calculation is
an approximation with lots of its floating point precision lost to
dividing the results of natural logarithms.
By using the specialized versions of the log functions (`log2` &
`log10`), we can avoid this loss of precision.
SQLite also uses these specialized log functions when possible, so it
doesn't hurt to do the same thing when aiming for parity.
This PR fixes #1763

Reviewed-by: Diego Reis (@el-yawd)
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1786
2025-06-20 17:58:54 +02:00
Luca Muscat
7cf77fb35b Fix fuzz issue #1763 by using the log2 & log10 functions where applicable
It is easy to chalk this fuzzer issue to erratic floating point
behaviour, but this is not the case here.

Currently, `exec_math_log` calculates log with arbitrary bases by using
the following formula: `log_a(b) ~= ln(b) / ln(a)`. This calculation is
an approximation with lots of its floating point precision lost to
dividing the results of natural logarithms.

By using the specialized versions of the log functions (`log2` &
`log10`), we can avoid this loss of precision.

SQLite also uses these specialized log functions when possible, so it
doesn't hurt to do the same thing when aiming for parity.
2025-06-20 10:52:13 +02:00
Pekka Enberg
62c8a97c10 Merge 'Copy instrumented image and symbols in Dockerfile.antithesis' from eric-dinh-antithesis
Copies the binary from the antithesis build instead of release
Copies symbol files from the binary to the `/symbols` directory
Resolves the `Symbols were uploaded` and `Software was instrumented`
properties in the Antithesis triage reports

Closes #1783
2025-06-20 09:23:45 +03:00
Piotr Rzysko
64a0333119 Fix missing column references in non-aggregate expressions
Previously, queries like:
```
SELECT
    CASE WHEN c0 != 'x' THEN group_concat(c1, ',') ELSE 'x' END
FROM t0
GROUP BY c0;
```

would return incorrect results because c0 was not copied during the
aggregation loop into a register accessible to the logic processing the
grouped results (e.g., the CASE WHEN expression in this example).

The same issue applied to expressions in the HAVING and ORDER BY clauses.
2025-06-20 06:19:16 +02:00
Piotr Rzysko
08c1767ba7 Collect non-aggregate columns in one place
Previously, the logic for collecting non-aggregate columns was duplicated
across multiple locations and implemented inconsistently. This caused a
bug that was revealed by the refactoring in this commit (see the added
test).
2025-06-20 06:17:14 +02:00
eric-dinh-antithesis
dbdea044c2 copy instrumented image and symbols in Dockerfile.antithesis 2025-06-19 17:08:25 -04:00
pedrocarlo
fcff306f98 emit constant where terms in init_loop 2025-06-19 13:50:38 -03:00
pedrocarlo
b1706ae849 change location where WhereTerms are emitted 2025-06-19 13:50:38 -03:00
pedrocarlo
86ea224069 add delete where basic functionality 2025-06-19 13:50:38 -03:00
Pekka Enberg
74fc789719 antithesis: Add integritycheck parallel driver 2025-06-19 14:40:38 +03:00
Pekka Enberg
7684dc98d0 Merge ' op_transaction end_read_tx in case of begin_write_tx is busy ' from Pere Diaz Bou
In case of starting a write txn, we always start by starting a read txn.
If we don't end_read_tx then it will never release locks.
Furthermore, we add some concurrent tests that catched this issue.
The idea is quite simple: write with 4 concurrent writers and once all
are finsihed, check the count of rows written is correct.

Closes #1777
2025-06-19 14:27:59 +03:00
Pekka Enberg
b2ee030846 github: Antithesis workflow fixes 2025-06-19 14:25:17 +03:00
Pekka Enberg
032a523d1d github: Add nightly Antithesis run 2025-06-19 14:22:24 +03:00
Pekka Enberg
b33b4edc5f scripts: Use ANTITHESIS_REGISTRY_KEY for Antithesis login
We can now use that as Github actions secret too.
2025-06-19 14:22:24 +03:00
Pekka Enberg
68d4389ee9 antithesis: Remove PRIMARY KEY and UNIQUE constraints
...we don't have indexing enabled by default so drop them.
2025-06-19 13:48:12 +03:00
Pekka Enberg
e28b58bb6a Merge 'antithesis-tests: Make test drivers robust when database is locked' from Pekka Enberg
If multiple processes access the same database file concurrently, Limbo
fails with:
```
LockingError("Failed locking file. File is locked by another process
```
Make test driver robust against that.

Closes #1769
2025-06-19 13:13:01 +03:00
Pere Diaz Bou
e681e32198 fix test_limbo_open_read_only 2025-06-19 10:56:01 +02:00
Pekka Enberg
e59c5185dd Limbo 0.0.22 2025-06-19 10:40:29 +03:00
Pekka Enberg
782ee09279 Update CHANGELOG.md 2025-06-19 10:39:01 +03:00
Pekka Enberg
419ddf0cfa antithesis-tests: Make test drivers robust when database is locked
If multiple processes access the same database file concurrently, Limbo fails with:

```
LockingError("Failed locking file. File is locked by another process
```

Make test driver robust against that.
2025-06-19 10:14:18 +03:00
Pekka Enberg
627e7daa65 Limbo 0.0.22-pre.5 2025-06-19 09:49:24 +03:00
Pekka Enberg
e878c0a871 Merge 'cli: Remove extra binary' from Pekka Enberg
The `cargo-dist` tool attempts to find it but because we never build it,
packaging fails. Remove the extra binary. Probably better to work
towards making experimental indexes a runtime flag instead.

Closes #1779
2025-06-19 09:46:53 +03:00
Pekka Enberg
37566d29cc cli: Remove extra binary
The `cargo-dist` tool attempts to find it but because we never build it,
packaging fails. Remove the extra binary. Probably better to work
towards making experimental indexes a runtime flag instead.
2025-06-19 09:41:01 +03:00
Pekka Enberg
ea2b15a4de stress: Disable multi-threaded testing
Depends on https://github.com/tursodatabase/limbo/issues/1552
2025-06-19 09:31:43 +03:00
Pekka Enberg
b07821374f Add Antithesis to README.md 2025-06-18 21:26:23 +03:00
Pere Diaz Bou
65372994d4 op_transaction end_read_tx in case of begin_write_tx is busy
In case of starting a write txn, we always start by starting a read txn.
If we don't end_read_tx then it will never release locks.
2025-06-18 17:40:53 +02:00
Pere Diaz Bou
10d02525d6 introduce concurrent write test
The idea is quite simple: write with 4 concurrent writers and once all
are finsihed, check the count of rows written is correct.
2025-06-18 17:40:53 +02:00
Pekka Enberg
27a164bd04 Limbo 0.0.22-pre.4 2025-06-18 14:58:54 +03:00
Pekka Enberg
14da7defbf cli: Exclude experimental binary from dist package 2025-06-18 14:58:48 +03:00
Pere Diaz Bou
f264cdadb1 Merge 'disable constraints on stress test' from Pere Diaz Bou
without indexes constraints are useless
Fixes #1774

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1776
2025-06-18 13:24:14 +02:00
Pekka Enberg
111d9ad79f Limbo 0.0.22-pre.3 2025-06-18 14:14:00 +03:00
Pekka Enberg
a108f146dd github: Disable compat tests with indexes on
...failing currently.
2025-06-18 13:52:14 +03:00
Pere Diaz Bou
9aecc1c903 disable constraints on stress test
without indexes constraints are useless
2025-06-18 12:45:04 +02:00
Pekka Enberg
cf78479850 Merge 'github: Node 18 is EOL' from Pekka Enberg
Closes #1772
2025-06-18 13:38:50 +03:00
Pekka Enberg
55b660210c Limbo 0.0.22-pre.2 2025-06-18 08:45:39 +03:00
Pekka Enberg
581d143595 github: Disable TPC-H benchmarks from CI
With indexing disabled, TPC-H benchmarks take forever to run. Disable
them for now.
2025-06-18 08:43:54 +03:00