Commit Graph

4905 Commits

Author SHA1 Message Date
Jussi Saurio
4fc04c8db3 sim/aws: fix sim timeout handling
code was trying to consume the same readablestream twice: once in
the timeouter's catch block, and a second time in the body of the
try block before postGithubIssue() could run.
2025-06-04 09:08:59 +03:00
Pekka Enberg
025714ce94 Merge 'bindings/javascript: Add proper exec() method and raw() mode' from Diego Reis
Now we can execute a sequence of statements from exec, it's similar to
`execute_batch()` from rusqlite
EDIT: It also add support for raw mode and do a small clean up to push
if statements outside loops

Closes #1620
2025-06-04 08:13:36 +03:00
Pekka Enberg
e16b12b2f0 s/modern/next/g 2025-06-03 16:36:41 +03:00
Pekka Enberg
e3e0208360 Merge 'Set maximum open simulator-created issues' from Jussi Saurio
Closes #1653
2025-06-03 15:40:52 +03:00
Jussi Saurio
3caf10d42f Set maximum open simulator-created issues 2025-06-03 15:39:00 +03:00
Jussi Saurio
a8106a0bbd Merge 'Fix cursors not being opened for indexes in DELETE' from Jussi Saurio
Closes #1648
Closes #1628

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1652
2025-06-03 15:37:44 +03:00
Jussi Saurio
31b37332d5 all index cursors must be opened when DELETE does an index seek too 2025-06-03 15:18:45 +03:00
Jussi Saurio
06626f72eb Fix cursors not being opened for indexes in DELETE 2025-06-03 14:45:01 +03:00
Jussi Saurio
c488c32d43 Merge 'Make cursor seek reentrant' from Pedro Muniz
Closes #1628.  Every function that calls `process_overflow_read` needs
to be reentrant. I did not change it here, but it would include
`get_prev_record` and `get_next_record`. Maybe `tablebtree_move_to` did
not need to use the state machine, but I included it as a safeguard.
Edit: Closes #1625 . When I implemented `restore_context`, I forgot to
add a `return_if_io` after calling it in `next` 🤦‍♂️
Edit: Closes #1617 . Just tested it and it also solves this bug.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1636
2025-06-03 14:24:40 +03:00
Pekka Enberg
6ffbc63ab4 Merge 'Add simulator-docker-runner for running limbo-sim in a loop on AWS' from Jussi Saurio
Closes #1645
2025-06-03 14:20:46 +03:00
Jussi Saurio
ea301de726 Merge 'Pass input string to translate function' from Pedro Muniz
In preparation for `CREATE VIEW`, we need to have the original sql query
that was used to create the view. I'm using the scanner's offset to
slice into the original input, trimming the newlines, and passing it to
the translate function.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1621
2025-06-02 17:43:11 +03:00
Jussi Saurio
5f586b7b24 Merge 'Small tracing enhancement' from Pedro Muniz
Instrument trace_insn to debug print its the stack pc and instruction.
Also, disable rustyline logs for the CLI as it is too noisy to work
with.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1635
2025-06-02 17:41:52 +03:00
Jussi Saurio
20555bce0a Merge 'Adjust write cursors for delete to avoid opening more than once. ' from Pedro Muniz
We were opening Write cursors inside of the rewind loop, which meant
that we were opening the same cursor multiple times per iteration. Also,
in this case, we need to open all indexes before the loop starts, and
avoid opening the same cursor again. While I was trying to debug this, I
also added instrumentation to some functions, so that I could track
where the instructions were being emitted. This helped me a lot. The
`#[instrument]` macro creates a span for the particular function, which
enables us to examine the path our functions are taking when any log is
emitted. Lastly, I fixed an explain bug, where it was unwrapping a None
`CursorKey` when opening the `sqlite_schema` table.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1640
2025-06-02 17:41:19 +03:00
Jussi Saurio
d06bb70514 Add simulator-docker-runner for running limbo-sim in a loop on AWS 2025-06-02 15:51:29 +03:00
Pekka Enberg
e6cfeb9552 Merge 'simulator: add option to disable BugBase' from Jussi Saurio
we will soon run the sim on AWS where we don't need bugbase

Closes #1644
2025-06-02 12:45:15 +03:00
Jussi Saurio
51e3a04668 simulator: add option to disable BugBase 2025-06-02 12:14:35 +03:00
pedrocarlo
9b5f5f6053 do not move_to if we are already inserting in correct place 2025-06-02 02:55:32 -03:00
pedrocarlo
9dc6638313 cleaner approach for opening indexes 2025-06-02 01:13:14 -03:00
pedrocarlo
c2942a5819 small fixes 2025-06-01 12:11:03 -03:00
pedrocarlo
39434fd20f return_io when restoring context 2025-06-01 03:07:16 -03:00
pedrocarlo
2ddbb7eeed adjust move_to and seek functions to make them truly reentrant + adding return_if_locked_maybe_load in some places so that we read loaded pages 2025-06-01 01:01:35 -03:00
pedrocarlo
d688cfd547 make find_cell and process_overflow_page reentrant 2025-05-31 23:31:59 -03:00
pedrocarlo
e97227ccb9 added delete operations to blobs in memory and write tests 2025-05-31 14:47:49 -03:00
pedrocarlo
dae58be071 make move_to reentrant 2025-05-31 02:56:30 -03:00
pedrocarlo
c9c73f2497 fix explain panicking on None CursorKey 2025-05-31 01:19:26 -03:00
pedrocarlo
bc563266b3 add instrumentation to more functions for debugging + adjust how cursors are opened 2025-05-30 20:35:50 -03:00
Diego Reis
0f685c5b9a bind/js: Add support for raw() Statements 2025-05-30 15:44:34 -03:00
pedrocarlo
33480540f1 make cursor seek reentrant 2025-05-30 13:25:46 -03:00
pedrocarlo
0757109676 instrument trace_insn 2025-05-30 11:33:22 -03:00
pedrocarlo
08f415370c remove rustyline traces from cli 2025-05-30 11:33:22 -03:00
pedrocarlo
b73200de86 pass input string to translate function 2025-05-30 11:20:36 -03:00
Pere Diaz Bou
09e6bb5438 Merge 'Convert u64 rowid to i64' from Pere Diaz Bou
Rowids can be negative, therefore let's swap to i64
Fixes: #1631

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1632
2025-05-30 14:46:07 +02:00
Pere Diaz Bou
d4f1b8e068 update i64::MAX comment 2025-05-30 14:02:05 +02:00
Pere Diaz Bou
da4190a23e Convert u64 rowid to i64
Rowids can be negative, therefore let's swap to i64
2025-05-30 13:07:31 +02:00
Pekka Enberg
0fbff54b90 Merge 'simulator: switch to tracing, run io.run_once and add update queries' from Pere Diaz Bou
* Without tracing crate we cannot log anything that happens in
limbo_core
* IO never ran in step loop inside simulator.
* Added update queries (which currently loop forever for some reason I'm
debugging).

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1627
2025-05-30 12:52:34 +03:00
Pekka Enberg
aa637d5ae1 antithesis: Build workload image with antithesis profile
...need for stuff like instrumentation to work.
2025-05-30 12:19:12 +03:00
Pere Diaz Bou
8ee7228c43 simulator: log to file too
most of the time debugging the log of a simulator run is spent reading
from a file for convinience. Let's just log to a file too.
2025-05-30 11:10:17 +02:00
Pere Diaz Bou
83db9be818 log seed creating plan 2025-05-30 11:04:19 +02:00
Pere Diaz Bou
1ca67d0092 remove dumb log 2025-05-30 11:03:09 +02:00
Pere Diaz Bou
d3495238c4 set default level info and log seed at startup 2025-05-30 11:02:05 +02:00
Pere Diaz Bou
8ec7e616b4 simulator: switch to tracing, run io.run_once and add update queries
* Without tracing crate we cannot log anything that happens in
limbo_core
* IO never ran in step loop inside simulator.
* Added update queries (which currently loop forever for some reason I'm
debugging).
2025-05-30 10:57:13 +02:00
Pekka Enberg
48e0441fae Update README.md 2025-05-30 09:45:41 +03:00
Jussi Saurio
819a6138d0 Merge 'Fix: aggregate regs must be initialized as NULL at the start' from Jussi Saurio
Again found when fuzzing nested where clause subqueries:
Aggregate registers need to be NULLed at the start because the same
registers might be reused on another invocation of a subquery, and if
they are not NULLed, the 2nd invocation of the same subquery will have
values left over from the first invocation.

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #1614
2025-05-30 09:39:37 +03:00
Jussi Saurio
5632a6046e Merge 'Fix: allow DeferredSeek on more than one cursor per program' from Jussi Saurio
Found while fuzzing nested subqueries. Since subqueries result in nested
plans, it quickly revealed that there can be multiple `DeferredSeek`
instructions issued for different cursors, but our `ProgramState` only
supported one at a time.

Closes #1610
2025-05-30 09:39:23 +03:00
Jussi Saurio
0d91971639 Merge 'Use tempfile in constraint test' from Jussi Saurio
Closes #1618
2025-05-30 09:38:29 +03:00
Diego Reis
1367b453e9 bind/js: Add proper exec() method 2025-05-29 16:00:07 -03:00
Jussi Saurio
d52a9a635c fix: use tempfile as db path in constraint.py 2025-05-29 21:03:10 +03:00
Jussi Saurio
d18ab34193 Fix: dont pollute testing.db in insert tests 2025-05-29 20:51:29 +03:00
Jussi Saurio
482eb4aa9a Merge 'Refactor: make clear distinction between 'joined tables' and 'tables referenced from outer query scopes'' from Jussi Saurio
**Beef:** we need to distinguish between references to tables in the
current query scope (CTEs, FROM clause) and references to tables from
outer query scopes (inside subqueries, or inside CTEs that refer to
previous CTEs). We don't want to consider 'tables from outside' in the
join order of a subquery, but we want the subquery to be able to
_reference_ those tables in e.g. its WHERE clause.
This PR -- or at least some sort of equivalent of it -- is a requirement
for #1595.
---
This PR replaces the `Vec<TableReference>` we use with new data
structures:
- TableReferences struct, which holds both:
     - joined_tables, and
     - outer_query_refs
- JoinedTable:
     - this is just a rename of the previous TableReference struct
- OuterQueryReference
     - this is to distinguish from JoinedTable those cases where
       e.g. a subquery refers to an outer query's table, or a CTE
       refers to a previous CTE.
Both JoinedTable and OuterQueryReference can be referred to by
expressions,
but only JoinedTables are considered for join ordering optimization and
so
forth.
These data structures are then used everywhere, which resulted in a lot
of changes.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1580
2025-05-29 20:45:33 +03:00
Jussi Saurio
f8257df77b Fix: aggregate regs must be initialized as NULL at the start 2025-05-29 18:44:53 +03:00