mirror of
https://github.com/aljazceru/turso.git
synced 2025-12-27 13:04:20 +01:00
e3a4400329
# Multi column indexes + index seek refactor ## PR reader guide I would say mostly you should just focus on the content of `optimizer.rs` and `plan.rs` because the rest is just small type changes, or in the case of `main_loop.rs`, a bunch of logic was just moved out of there and rewritten. ## New feature - multi column index seeks This PR adds support for utilizing multi-column indexes properly, i.e. using as many columns in the seek key as possible. Previously, we only used max one column per index. I've modified the existing compound index seek fuzz test to use this functionality. ## Refactoring of index seek related logic This PR moves a lot of index seek related logic out of `main_loop.rs` into `optimizer.rs` and `plan.rs` and introduces a bunch of helper structures to model finding and using an index to perform a seek + scan. ## Examples Here are some examples of multi-column seeks: ### Example table setup: ```sql sqlite> CREATE TABLE t(a,b,c,d,e); sqlite> CREATE INDEX abc ON t (a,b,c); -- create 10000 rows with random values between 0-9 for all columns sqlite >INSERT INTO t SELECT ABS(RANDOM() % 10),ABS(RANDOM() % 10),ABS(RANDOM() % 10),ABS(RANDOM() % 10),ABS(RANDOM() % 10) FROM generate_series(1,10000,1); ``` ### Example bytecode plans, results and timings vs main branch: ```sql limbo> EXPLAIN SELECT * FROM t WHERE a = 5 and b = 6 and c = 7; addr opcode p1 p2 p3 p4 p5 comment ---- ----------------- ---- ---- ---- ------------- -- ------- 0 Init 0 20 0 0 Start at 20 1 OpenReadAsync 0 2 0 0 table=t, root=2 2 OpenReadAwait 0 0 0 0 3 OpenReadAsync 1 3 0 0 table=abc, root=3 4 OpenReadAwait 0 0 0 0 5 Integer 5 6 0 0 r[6]=5 6 Integer 6 7 0 0 r[7]=6 7 Integer 7 8 0 0 r[8]=7 8 SeekGE 1 19 6 0 key=[6..8] 9 IdxGT 1 19 6 0 key=[6..8] 10 DeferredSeek 1 0 0 0 11 Column 0 0 1 0 r[1]=t.a 12 Column 0 1 2 0 r[2]=t.b 13 Column 0 2 3 0 r[3]=t.c 14 Column 0 3 4 0 r[4]=t.d 15 Column 0 4 5 0 r[5]=t.e 16 ResultRow 1 5 0 0 output=r[1..5] 17 NextAsync 1 0 0 0 18 NextAwait 1 9 0 0 19 Halt 0 0 0 0 20 Transaction 0 0 0 0 write=false 21 Goto 0 1 0 0 limbo> SELECT * FROM t WHERE a = 5 and b = 6 and c = 7; 5|6|7|9|9 5|6|7|4|7 5|6|7|3|2 5|6|7|3|7 5|6|7|5|2 5|6|7|5|3 5|6|7|9|7 runtime (debug build, this branch): total: 2 ms (this includes parsing/coloring of cli app) runtime (debud build, main branch): total: 67 ms (this includes parsing/coloring of cli app) ``` ```sql limbo> EXPLAIN SELECT * FROM t WHERE a = 5 and b = 6 and c < 7; addr opcode p1 p2 p3 p4 p5 comment ---- ----------------- ---- ---- ---- ------------- -- ------- 0 Init 0 21 0 0 Start at 21 1 OpenReadAsync 0 2 0 0 table=t, root=2 2 OpenReadAwait 0 0 0 0 3 OpenReadAsync 1 3 0 0 table=abc, root=3 4 OpenReadAwait 0 0 0 0 5 Integer 5 6 0 0 r[6]=5 6 Integer 6 7 0 0 r[7]=6 7 Null 0 8 0 0 r[8]=NULL 8 SeekGT 1 20 6 0 key=[6..8] 9 Integer 7 8 0 0 r[8]=7 10 IdxGE 1 20 6 0 key=[6..8] 11 DeferredSeek 1 0 0 0 12 Column 0 0 1 0 r[1]=t.a 13 Column 0 1 2 0 r[2]=t.b 14 Column 0 2 3 0 r[3]=t.c 15 Column 0 3 4 0 r[4]=t.d 16 Column 0 4 5 0 r[5]=t.e 17 ResultRow 1 5 0 0 output=r[1..5] 18 NextAsync 1 0 0 0 19 NextAwait 1 10 0 0 20 Halt 0 0 0 0 21 Transaction 0 0 0 0 write=false 22 Goto 0 1 0 0 limbo> SELECT * FROM t WHERE a = 5 and b = 6 and c < 7; 5|6|0|0|3 5|6|0|5|1 5|6|0|3|1 5|6|0|6|3 5|6|0|8|1 5|6|0|2|7 5|6|0|9|9 5|6|0|5|3 5|6|0|4|2 5|6|0|4|2 5|6|0|0|2 5|6|0|7|2 5|6|1|8|5 5|6|1|7|5 5|6|1|7|2 5|6|1|1|2 5|6|1|6|5 5|6|1|1|5 5|6|1|5|7 5|6|1|1|9 5|6|1|4|3 5|6|1|1|2 5|6|1|2|2 5|6|1|4|4 5|6|1|9|6 5|6|1|2|5 5|6|1|2|4 5|6|1|7|1 5|6|2|0|9 5|6|2|6|9 5|6|2|4|5 5|6|2|9|3 5|6|2|5|2 5|6|2|9|0 5|6|2|7|1 5|6|3|6|5 5|6|3|8|5 5|6|3|5|4 5|6|3|5|2 5|6|3|1|1 5|6|3|2|0 5|6|3|9|3 5|6|3|6|9 5|6|3|7|6 5|6|3|3|5 5|6|3|0|8 5|6|3|6|4 5|6|4|1|1 5|6|4|9|8 5|6|4|3|7 5|6|4|1|3 5|6|4|8|9 5|6|4|9|7 5|6|4|7|9 5|6|4|8|8 5|6|4|3|1 5|6|4|2|6 5|6|4|5|7 5|6|4|2|6 5|6|4|4|3 5|6|5|2|4 5|6|5|6|7 5|6|5|3|8 5|6|5|7|8 5|6|5|9|6 5|6|5|2|7 5|6|5|1|7 5|6|5|0|6 5|6|6|2|4 5|6|6|9|4 5|6|6|4|9 5|6|6|5|6 5|6|6|2|2 5|6|6|0|6 runtime (debug build, this branch): total: 9 ms (this includes parsing/coloring of cli app) runtime (debug build, main branch): total: 71 ms (this includes parsing/coloring of cli app) ``` ```sql limbo> EXPLAIN SELECT * FROM t WHERE a = 5 and b = 6 and c < 7 ORDER BY a desc, b desc, c desc; addr opcode p1 p2 p3 p4 p5 comment ---- ----------------- ---- ---- ---- ------------- -- ------- 0 Init 0 20 0 0 Start at 20 1 OpenReadAsync 0 2 0 0 table=t, root=2 2 OpenReadAwait 0 0 0 0 3 OpenReadAsync 1 3 0 0 table=abc, root=3 4 OpenReadAwait 0 0 0 0 5 Integer 5 6 0 0 r[6]=5 6 Integer 6 7 0 0 r[7]=6 7 Integer 7 8 0 0 r[8]=7 8 SeekLT 1 19 6 0 key=[6..8] 9 IdxLT 1 19 6 0 key=[6..7] 10 DeferredSeek 1 0 0 0 11 Column 0 0 1 0 r[1]=t.a 12 Column 0 1 2 0 r[2]=t.b 13 Column 0 2 3 0 r[3]=t.c 14 Column 0 3 4 0 r[4]=t.d 15 Column 0 4 5 0 r[5]=t.e 16 ResultRow 1 5 0 0 output=r[1..5] 17 PrevAsync 1 0 0 0 18 PrevAwait 1 0 0 0 19 Halt 0 0 0 0 20 Transaction 0 0 0 0 write=false 21 Goto 0 1 0 0 limbo> SELECT * FROM t WHERE a = 5 and b = 6 and c < 7 ORDER BY a desc, b desc, c desc; 5|6|6|0|6 5|6|6|2|2 5|6|6|5|6 5|6|6|4|9 5|6|6|9|4 5|6|6|2|4 5|6|5|0|6 5|6|5|1|7 5|6|5|2|7 5|6|5|9|6 5|6|5|7|8 5|6|5|3|8 5|6|5|6|7 5|6|5|2|4 5|6|4|4|3 5|6|4|2|6 5|6|4|5|7 5|6|4|2|6 5|6|4|3|1 5|6|4|8|8 5|6|4|7|9 5|6|4|9|7 5|6|4|8|9 5|6|4|1|3 5|6|4|3|7 5|6|4|9|8 5|6|4|1|1 5|6|3|6|4 5|6|3|0|8 5|6|3|3|5 5|6|3|7|6 5|6|3|6|9 5|6|3|9|3 5|6|3|2|0 5|6|3|1|1 5|6|3|5|2 5|6|3|5|4 5|6|3|8|5 5|6|3|6|5 5|6|2|7|1 5|6|2|9|0 5|6|2|5|2 5|6|2|9|3 5|6|2|4|5 5|6|2|6|9 5|6|2|0|9 5|6|1|7|1 5|6|1|2|4 5|6|1|2|5 5|6|1|9|6 5|6|1|4|4 5|6|1|2|2 5|6|1|1|2 5|6|1|4|3 5|6|1|1|9 5|6|1|5|7 5|6|1|1|5 5|6|1|6|5 5|6|1|1|2 5|6|1|7|2 5|6|1|7|5 5|6|1|8|5 5|6|0|7|2 5|6|0|0|2 5|6|0|4|2 5|6|0|4|2 5|6|0|5|3 5|6|0|9|9 5|6|0|2|7 5|6|0|8|1 5|6|0|6|3 5|6|0|3|1 5|6|0|5|1 5|6|0|0|3 runtime (debug build, this branch): total: 9 ms (this includes parsing/coloring of cli app) runtime (debug build, main branch): total: 71 ms (this includes parsing/coloring of cli app) ``` Closes #1288