Commit Graph

843 Commits

Author SHA1 Message Date
Pere Diaz Bou
2414502268 parse schema set auto_commit false in nested query 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
d8658264d9 alter set cookie 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
c799396c3d rollback schema in connection 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
65a7fe13cf remove lock from private schema copy 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
abf1699dd2 set scheam version and update shared schema in txn 2025-07-03 12:36:48 +02:00
KaguraMilet
562dd389db Merge branch 'tursodatabase:main' into buffer 2025-07-03 13:46:37 +08:00
Nikita Sivukhin
c9c5ef4e25 remote query_mode from ProgramBuilderOpts and from function arguments
- mode never changes and ProgramBuilder already created with proper mode set correctly
2025-07-02 13:24:12 +04:00
Pekka Enberg
e718cfef6b Merge 'Add opening new connection from a sqlite compatible URI, read-only connections' from Preston Thorpe
@penberg reminded me that this never got integrated :)

Closes #1908
2025-07-02 08:05:36 +03:00
KaguraMilet
aca08238d8 fix buffer pool is not thread safe problem 2025-07-01 16:06:55 +08:00
PThorpe92
2e967b778b Prevent writes to db file in the VDBE if we are in readonly mode 2025-06-30 22:04:56 -04:00
Levy A.
ffd6844b5b refactor: remove PseudoTable from Table
the only reason for `PseudoTable` to exist, is to provide column
information for `PseudoCursor` creation. this should not be part of the
schema.
2025-06-30 14:31:58 -03:00
Levy A.
afc55b27f0 refactor: remove unnecessary column definitions for PseudoTable
the only information that matters in the amount of column
2025-06-30 13:54:29 -03:00
Pekka Enberg
725c3e4ddc Rename limbo_sqlite3_parser crate to turso_sqlite3_parser 2025-06-29 12:34:46 +03:00
Pekka Enberg
eec994386b Rename limbo_macros to turso_macros 2025-06-29 12:00:17 +03:00
Pekka Enberg
572c722390 Merge 'write page1 on database initialization' from Pere Diaz Bou
Page 1 must be initialized and written as soon as possible without
marking page as dirty.
OpenEphemeral now requires a state machine to accomodate new
begin_write_tx semantics.

Closes #1839
2025-06-26 20:43:40 +03:00
Pere Diaz Bou
4d80b8237d write page1 on database initialization
Page 1 must be initialized and written as soon as possible without
marking page as dirty.
2025-06-26 14:44:23 +02:00
Pekka Enberg
2fc5c0ce5c Switch to runtime flag for enabling indexes
Makes it easier to test the feature:

```
$ cargo run --  --experimental-indexes
Limbo v0.0.22
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
limbo> CREATE TABLE t(x);
limbo> CREATE INDEX t_idx ON t(x);
limbo> DROP INDEX t_idx;
```
2025-06-26 10:07:28 +03:00
Pekka Enberg
a48198ec60 Merge 'Rollback simple support' from Pere Diaz Bou
Support for simple interactive rollback like:
```sql
    create table t (x);
    insert into t values (1);
    begin;
    insert into t values (2);
    rollback;
    select * from t;
```
This PR also fixes some other issues I found while debugging:
* Checkpoint would never `clear_dirty` on pages in page cache.
* Auto commit for interactive transactions was not respected so any
`insert` after `begin` would flush frames regardless of `auto_commit`
state.
* `max_frame` on wal shared state was being updated after every
`append_frame` which was incorrect, as another transaction would be able
to use that new `max_frame` even tho the transaction could've rolled
back. Instead we update the private copy of `max_frame` and only update
it at the end.
Follow up for later are savepoints which require implementing a
subjournal to track savepoints and their modified pages.

Closes #1825
2025-06-25 20:02:09 +03:00
Pere Diaz Bou
22f9cd695d commit_txn track rollback case 2025-06-25 14:00:57 +02:00
Pere Diaz Bou
bdd2010df3 autocommit rollback 2025-06-25 14:00:57 +02:00
Pere Diaz Bou
d66c683a4c implement rollback translation 2025-06-25 13:45:32 +02:00
Jussi Saurio
27b3ecf599 core/db&pager: fix locking for initializing empty database
When `struct Database` is constructed, store `is_empty` as an
`Arc<AtomicBool>` - the value is true if:

1. DB size is zero
2. WAL has no frames

When `struct Pager` is constructed, this `Arc` is simply cloned.
When any connection runs a transaction it will first check `is_empty`,
and if the DB is empty, it will lock `init_lock` and then check `is_empty`
again, and if it's still true, it allocates page1 and stores `false` in
the `is_empty` `AtomicBool` and drops the lock.

---

Note that Limbo can currently have a zero DB and a WAL with frames,
as we have no special logic for folding page1 to the main DB file
during initialization.

Page 1 allocation currently happens on the first transaction (read or
write, due to having to support `select * from sqlite_schema` on an
empty DB; we should really check how SQLite actually does this.).
2025-06-25 14:45:21 +03:00
Jussi Saurio
f306547ff3 booleans are hard 2025-06-24 14:41:50 -03:00
Jussi Saurio
920e88a6a9 clippy 2025-06-24 14:41:50 -03:00
Jussi Saurio
133d498724 Implement a header_accessor module so that DatabaseHeader structs arent initialized on every access 2025-06-24 14:41:50 -03:00
Jussi Saurio
2dc9b1cf7f Fix: use either db_size>0 or wal max frame>0 to decide when to read schema 2025-06-24 14:41:50 -03:00
Diego Reis
af193ef76a Move first page allocation logic to begin_write_tx 2025-06-24 14:41:50 -03:00
Jussi Saurio
acafe71d9e Dont allocate fresh page1 if it's in the WAL 2025-06-24 14:41:50 -03:00
Diego Reis
f0f9ad4844 core: Get rid of maybe_init_database_file
Initialization now only occurs in the first write transaction
2025-06-24 14:41:50 -03:00
Diego Reis
ce73025202 core: Only actually allocate page1 on first write transaction 2025-06-24 14:41:50 -03:00
Jussi Saurio
cc2e14b11c Read page 1 from pager always, no separate db_header 2025-06-24 14:41:49 -03:00
Nils Koch
2827b86917 chore: fix clippy warnings 2025-06-23 19:52:13 +01:00
Pere Diaz Bou
404b5fad8a clippy 2025-06-23 18:19:44 +02:00
Pere Diaz Bou
b7fd4f1e18 update nchanges on op_insert 2025-06-23 18:11:58 +02:00
pedrocarlo
74beac5ea8 ephemeral table for update when rowid is being update 2025-06-20 16:28:10 -03:00
Jussi Saurio
c69047106c Merge 'Implement RowData opcode' from meteorgan
The `RowData` opcode is required to implement #1575.
I haven't found a ideal way to test this PR independently, but I
verified its functionality while working on #1575(to be committed soon),
and it performs effectively.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1756
2025-06-20 21:58:47 +03:00
Jussi Saurio
38f6b8c031 Merge 'Support indent for Goto opcode when executing explain' from meteorgan
it works as expected
```
limbo> explain insert into tb1 select * from tb2 union select * from tb3;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     29    0                    0   Start at 29
1     InitCoroutine      1     21    2                    0
2     OpenEphemeral      0     0     0                    0   cursor=0 is_table=false
3     OpenRead           1     3     0                    0   table=tb2, root=3
4     Rewind             1     9     0                    0   Rewind table tb2
5       Column           1     0     2                    0   r[2]=tb2.age
6       MakeRecord       2     1     3                    0   r[3]=mkrec(r[2..2]); for union_dedupe
7       IdxInsert        0     3     0                    0   key=r[3]
8     Next               1     5     0                    0
9     OpenRead           2     4     0                    0   table=tb3, root=4
10    Rewind             2     15    0                    0   Rewind table tb3
11      Column           2     0     2                    0   r[2]=tb3.age
12      MakeRecord       2     1     4                    0   r[4]=mkrec(r[2..2]); for union_dedupe
13      IdxInsert        0     4     0                    0   key=r[4]
14    Next               2     11    0                    0
15    Rewind             0     18    0                    0   Rewind  union_dedupe
16      Column           0     0     2                    0   r[2]=union_dedupe.age
17      Yield            1     0     0                    0
18    Next               0     16    0                    0
19    Close              0     0     0                    0
20    EndCoroutine       1     0     0                    0
21    OpenWrite          3     2     0                    0   root=2; tb1
22      Yield            1     28    0                    0
23      Copy             2     7     0                    0   r[7]=r[2]
24      NewRowid         3     6     0                    0   r[6]=rowid
25      MakeRecord       7     1     8                    0   r[8]=mkrec(r[7..7])
26      Insert           3     8     6     tb1            0   intkey=r[6] data=r[8]
27    Goto               0     22    0                    0
28    Halt               0     0     0                    0
29    Transaction        0     1     0                    0   write=true
30    Goto               0     1     0                    0
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1775
2025-06-20 21:57:40 +03:00
Luca Muscat
7cf77fb35b Fix fuzz issue #1763 by using the log2 & log10 functions where applicable
It is easy to chalk this fuzzer issue to erratic floating point
behaviour, but this is not the case here.

Currently, `exec_math_log` calculates log with arbitrary bases by using
the following formula: `log_a(b) ~= ln(b) / ln(a)`. This calculation is
an approximation with lots of its floating point precision lost to
dividing the results of natural logarithms.

By using the specialized versions of the log functions (`log2` &
`log10`), we can avoid this loss of precision.

SQLite also uses these specialized log functions when possible, so it
doesn't hurt to do the same thing when aiming for parity.
2025-06-20 10:52:13 +02:00
Pere Diaz Bou
65372994d4 op_transaction end_read_tx in case of begin_write_tx is busy
In case of starting a write txn, we always start by starting a read txn.
If we don't end_read_tx then it will never release locks.
2025-06-18 17:40:53 +02:00
meteorgan
5c7d1423e7 Support indent for Goto opcode when executing explain 2025-06-17 23:55:39 +08:00
Luca Muscat
a5ac1884c1 core: Stop treating the contents of a Value::Blob as a String
By encoding a Vec<u8> (vector of bytes), a lossy conversion from a
`Vec<u8>` to a `String` occurs. The lossy conversion leads to an
incorrect hex value to be displayed.

Avoid the lossy conversion and let the `hex` crate do its thing.
2025-06-16 21:10:17 +02:00
meteorgan
caa1801ec4 cargo fmt 2025-06-16 22:09:43 +08:00
meteorgan
4f742a3a0f Implement RowData opcode 2025-06-16 22:07:35 +08:00
Pekka Enberg
db4945eada Merge 'Fix update queries to set n_changes ' from Kim Seon Woo
- `Update` query doesn't update `n_changes`. Let's make it work
- Add `InsertFlags` to add meta information related to insert operations
- For update query, add `UPDATE` flag
- Currently, the update query executes `Insn::Delete` and `Insn::Insert`
internally, it increases `n_change` by 2. So, for the update query,
let's skip increasing `n_change` for the `Insn::Insert`
https://github.com/tursodatabase/limbo/issues/1681

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1683
2025-06-16 16:30:20 +03:00
Pekka Enberg
4496a0d08a core: Clean up integrity_check()
Suggested by Jussi.
2025-06-16 14:46:36 +03:00
Pekka Enberg
882c5ca168 Merge 'Simple integrity check on btree' from Pere Diaz Bou
This PR adds support for the instruction `IntegrityCk` which performs an
integrity check on the contents of a single table. Next PR I will try to
implement the rest of the integrity check where we would check indexes
containt correct amount of data and some more.
<img width="1151" alt="image" src="https://github.com/user-
attachments/assets/29d54148-55ba-480f-b972-e38587f0a483" />

Closes #1719
2025-06-16 13:46:26 +03:00
Pekka Enberg
90c1e3fc06 Switch Connection to use Arc instead of Rc
Connection needs to be Arc so that bindings can wrap it with `Mutex` for
multi-threading.
2025-06-16 10:43:19 +03:00
meteorgan
b910fd0012 drop unused code in op_column 2025-06-15 00:50:16 +08:00
Jussi Saurio
30e4511d62 Merge 'NOT NULL constraint' from Anton Harniakou
Implements basic support for `NOT NULL` contraint check.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1675
2025-06-13 14:25:24 +03:00
Anton Harniakou
25d0584b19 Fix typo 2025-06-13 11:57:39 +03:00