Commit Graph

5832 Commits

Author SHA1 Message Date
Pekka Enberg
9285d8b83b Merge 'Fix: OP_NewRowId to generate semi random rowid when largest rowid is i64::MAX' from Krishna Vishal
- `OP_NewRowId` now generates new rowid semi randomly when the largest
rowid in the table is `i64::MAX`.
- Introduced new `LimboError` variant `DatabaseFull` to signify that
database might be full (SQLite behaves this way returning
`SQLITE_FULL`).
Now:
```SQL
turso> CREATE TABLE q(x INTEGER PRIMARY KEY, y);
turso> INSERT INTO q VALUES (9223372036854775807, 1);
turso> INSERT INTO q(y) VALUES (2);
turso> INSERT INTO q(y) VALUES (3);
turso> SELECT * FROM q;
┌─────────────────────┬───┐
│ x                   │ y │
├─────────────────────┼───┤
│ 1841427626667347484 │ 2 │
├─────────────────────┼───┤
│ 4000338366725695791 │ 3 │
├─────────────────────┼───┤
│ 9223372036854775807 │ 1 │
└─────────────────────┴───┘
```
Fixes: https://github.com/tursodatabase/turso/issues/1977

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1985
2025-07-14 11:56:09 +03:00
Pekka Enberg
0b544717a1 Merge 'do not check rowid alias for null' from Nikita Sivukhin
Simple PR to check minor issue that `INTEGER PRIMARY KEY NOT NULL` (`NOT
NULL` is redundant here obviously) will prevent user to insert anything
to the table as rowid-alias column always set to null by `turso-db`

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2063
2025-07-14 11:55:06 +03:00
Pere Diaz Bou
3a34f21434 io/windows: pread return bytes read 2025-07-14 10:44:56 +02:00
Pere Diaz Bou
340391538a io: change comment for assert 2025-07-14 10:36:06 +02:00
Pere Diaz Bou
88ff218810 io: assert small I/O
Let's assert **for now** that we do not read/write less bytes than
expected. This should be fixed to retrigger several reads/writes if we
couldn't read/write enough but for now let's assert.
2025-07-14 10:19:41 +02:00
Nikita Sivukhin
0457567714 more clippy fixes 2025-07-14 12:09:39 +04:00
Krishna Vishal
12f9743443 Remove unused imports 2025-07-14 13:13:54 +05:30
Krishna Vishal
ab0cb06755 split seek and getting rowid as two separate states 2025-07-14 13:11:41 +05:30
Krishna Vishal
3e880c34d6 Make op_new_rowid re-entrant
Introduce `OpNewRowidState` state machine

remove `get_new_rowid` from vdbe/mod.rs
2025-07-14 13:11:40 +05:30
Krishna Vishal
98ca275b33 Add a way to semi randomly generate rowid when the max rowid reaches
`i64::MAX`. We do this by attempting to generate random values smaller
than `i64::MAX` for 100 times and returns `DatabaseFull` error on
failure

- Introduced `DatabaseFull` error variant

Fixes: https://github.com/tursodatabase/turso/issues/1977
2025-07-14 13:09:34 +05:30
Nikita Sivukhin
b330c6b70e fix clippy 2025-07-14 11:38:08 +04:00
Nikita Sivukhin
e94ebbad04 remove unwanted changes 2025-07-14 11:27:51 +04:00
Nikita Sivukhin
cc04f11bd6 remove clone 2025-07-14 11:27:51 +04:00
Nikita Sivukhin
f61d733dd3 make new functions dependend on "json" Cargo feature 2025-07-14 11:26:51 +04:00
Nikita Sivukhin
c9e7271eaf properly pass subtype 2025-07-14 11:20:49 +04:00
Nikita Sivukhin
bf25a0e3f1 fix clippy 2025-07-14 11:20:16 +04:00
Nikita Sivukhin
81cd04dd65 add bin_record_json_object and table_columns_json_array functions 2025-07-14 11:19:45 +04:00
Nikita Sivukhin
eed89993f9 fix clippy 2025-07-14 11:17:32 +04:00
Nikita Sivukhin
5409812610 properly implement generation of before/after records for new modes 2025-07-14 11:17:32 +04:00
Nikita Sivukhin
9e04102a94 add basic cdc tests for new modes 2025-07-14 11:17:31 +04:00
Nikita Sivukhin
fabb00f385 fix test 2025-07-14 11:16:06 +04:00
Nikita Sivukhin
b258c10c9a generate before/after row values in modification statements 2025-07-14 11:16:06 +04:00
Nikita Sivukhin
9129991b62 add id,before,after,full modes 2025-07-14 11:16:06 +04:00
Pekka Enberg
8f8d582b4a Merge 'Ignore double quotes around table names' from Zaid Humayun
This PR normalizes the table name identifier by removing quotes to
improve SQLite compatibility. Fixes
https://github.com/tursodatabase/turso/issues/1964
<img width="545" height="175" alt="Screenshot 2025-07-13 at 5 12 23 PM"
src="https://github.com/user-
attachments/assets/10952fc7-9ade-4c97-a427-385ff2dc3b44" />
<img width="384" height="110" alt="Screenshot 2025-07-13 at 5 12 32 PM"
src="https://github.com/user-
attachments/assets/5d87e0fe-72a4-4472-abc3-24c0d4cc8add" />

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2069
2025-07-14 10:04:59 +03:00
Piotr Rzysko
30ae6538ee Treat table-valued functions as tables
With this change, the following two queries are considered equivalent:
```sql
SELECT value FROM generate_series(5, 50);
SELECT value FROM generate_series WHERE start = 5 AND stop = 50;
```
Arguments passed in parentheses to the virtual table name are now
matched to hidden columns.

Column references are still not supported as table-valued function
arguments. The only difference is that previously, a query like:
```sql
SELECT one.value, series.value
FROM (SELECT 1 AS value) one, generate_series(one.value, 3) series;
```
would cause a panic. Now, it returns a proper error message instead.

Adding support for column references is more nuanced for two main
reasons:
- We need to ensure that in joins where a TVF depends on other tables,
those other tables are processed first. For example, in:
```sql
SELECT one.value, series.value
FROM generate_series(one.value, 3) series, (SELECT 1 AS value) one;
```
the one table must be processed by the top-level loop, and series must
be nested.
- For outer joins involving TVFs, the arguments must be treated as ON
predicates, not WHERE predicates.
2025-07-14 07:16:53 +02:00
Piotr Rzysko
9102f4a2f4 Extract table parsing into separate method
This is in preparation for reusing the method when parsing `TableCall`s.
2025-07-14 07:16:53 +02:00
Piotr Rzysko
631d62e9f8 Implement best_index for table-valued functions
In SQLite, the field equivalent to `constraint_usage` (`aConstraintUsage`
from `sqlite3_index_info`) is used to request arguments that are later
passed to the `xFilter` method. In Limbo, this behavior applies to
virtual tables, but not to table-valued functions. Currently, TVFs have
dedicated handling that passes all function arguments to the filter
method and doesn't use information provided in the `constraint_usage`
field.

This commit is a step toward unifying the handling of virtual tables and
TVFs.
2025-07-14 07:16:53 +02:00
Piotr Rzysko
44b1b1852a Fix referencing virtual table predicates
We need to enumerate first and filter afterward — not the other way
around — because we later use the indexes produced by `enumerate` to
access the original `predicates` slice.
2025-07-14 07:16:53 +02:00
Piotr Rzysko
319cdbe3af Don't use search for virtual tables
Previously, the test queries added in this commit would fail with:
thread 'main' panicked at core/schema.rs:129:34:
not implemented
stack backtrace:
   0: rust_begin_unwind
             at /rustc/90b35a6239c3d8bdabc530a6a0816f7ff89a0aaf/library/std/src/panicking.rs:665:5
   1: core::panicking::panic_fmt
             at /rustc/90b35a6239c3d8bdabc530a6a0816f7ff89a0aaf/library/core/src/panicking.rs:74:14
   2: core::panicking::panic
             at /rustc/90b35a6239c3d8bdabc530a6a0816f7ff89a0aaf/library/core/src/panicking.rs:148:5
   3: limbo_core::schema::Table::get_root_page
             at ./core/schema.rs:129:34
   4: limbo_core::translate::main_loop::init_loop
             at ./core/translate/main_loop.rs:260:44
   5: limbo_core::translate::emitter::emit_query
             at ./core/translate/emitter.rs:568:5
   6: limbo_core::translate::emitter::emit_program_for_select
             at ./core/translate/emitter.rs:496:5
   7: limbo_core::translate::emitter::emit_program
             at ./core/translate/emitter.rs:187:31
   8: limbo_core::translate::select::translate_select
             at ./core/translate/select.rs:82:5
   9: limbo_core::translate::translate_inner
             at ./core/translate/mod.rs:241:13
  10: limbo_core::translate::translate
             at ./core/translate/mod.rs:95:17
  11: limbo_core::Connection::run_cmd
             at ./core/lib.rs:416:31
  12: <limbo_core::QueryRunner as core::iter::traits::iterator::Iterator>::next
             at ./core/lib.rs:916:22
  13: limbo::app::Limbo::run_query
             at ./cli/app.rs:442:27
  14: limbo::app::Limbo::handle_input_line
             at ./cli/app.rs:544:13
  15: limbo::main
             at ./cli/main.rs:51:31
  16: core::ops::function::FnOnce::call_once
2025-07-14 07:16:53 +02:00
Piotr Rzysko
000d70f1f3 Propagate info about hidden columns 2025-07-14 07:16:53 +02:00
Krishna Vishal
370d437491 Add docs for get_tie_breaker_from_idx_comp_op 2025-07-14 03:28:55 +05:30
Krishna Vishal
4c5383b0b3 chore: clippy 2025-07-14 03:28:55 +05:30
Krishna Vishal
e27b9c7e0f Address review comments 2025-07-14 03:28:55 +05:30
Krishna Vishal
a79fe458db Fix merge conflicts and adapt schema.rs to use RecordCursor 2025-07-14 03:28:55 +05:30
Krishna Vishal
ea4a4708ea - Address some review comments
- Add docs for `RecordCursor`
2025-07-14 03:28:55 +05:30
Krishna Vishal
b1f27cad94 chore: fix clippy 2025-07-14 03:28:55 +05:30
Krishna Vishal
d3368a28bc fix merge conflicts 2025-07-14 03:28:55 +05:30
Krishna Vishal
894bdca05f refactor: change ImmutableRecord payload to Value for compatibility 2025-07-14 03:28:54 +05:30
Krishna Vishal
d78185bd62 Remove mistakenly pushed file 2025-07-14 03:28:54 +05:30
Krishna Vishal
9de3cf0c60 Remove redundant checks 2025-07-14 03:28:54 +05:30
Krishna Vishal
235e798561 Return corrupt errors. 2025-07-14 03:28:54 +05:30
Krishna Vishal
9393aba0bd Add docs for RecordCompare 2025-07-14 03:28:54 +05:30
Krishna Vishal
e7e5f28c0a chore: Clippy chill 2025-07-14 03:28:54 +05:30
Krishna Vishal
f3b169bf30 Fix empty blob test failure. 2025-07-14 03:28:54 +05:30
Krishna Vishal
9b315d1d7e Manually inline the record deserialization code for performance.
This is done because the compiler is refusing to inline even after
adding inline hint.
- Get refvalues from directly from registers without using
`make_record`
2025-07-14 03:28:54 +05:30
Krishna Vishal
35ed279644 Clean up indexbtree_move_to 2025-07-14 03:28:54 +05:30
Krishna Vishal
dca4e669f7 Reduce allocations in compare_records_generic 2025-07-14 03:28:54 +05:30
Krishna Vishal
f0e8e5871b Replace compare_immutable with compare_records_generic 2025-07-14 03:28:54 +05:30
Krishna Vishal
860de412d9 Add num_columns to BTreeCursor so we can initialize
`Vecs` inside `RecordCursor` to their appropriate to reduce
allocations.
2025-07-14 03:28:54 +05:30
Krishna Vishal
ef147181c2 Working version of the incremental column and "optimal" record
compare functions. Now we optimize them
2025-07-14 03:28:54 +05:30