This PR is extracted from #1674. This PR creates the optional feature
flag `simulator` that exposes the numeric module and enables an optional
Serde derive on limbo's `Value` type. Additionally, it also namespaces
all of the exec_* functions that operate on a `Value` or indirectly
operate on a `Value` (e.g `exec_like` operates on a pattern string that
is derived from the `Value`). This is necessary so that instead of
reimplementing all of the expected behaviours of different operations
inside the simulator, we can just tap into the existing code we already
have. The next step for this will be wrap be to use `Value` inside the
simulator, which is something that I did in #1674.
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#1692
My guess as to why the sleep solves the problem is that when you ask
Limbo to quit it needs to checkpoint and with large blobs this probably
takes a while. What I think happens is:
- Now after the python code asks for the program to quit, it already
tries to sleep to allow the checkpointing to succeed
- It terminates the pipe
- Then kills it so that the locking of file is dropped sooner
Then later, we open another shell with SQLite in `test.test_compat()`
and try to run some queries in the database, but the file locking is
still in place. This is my guess on what is going on. I remember
@PThorpe92 had a similar problem a while back on something similar to
this.
```python
def quit(self) -> None:
self._write_to_pipe(".quit")
sleep(0.3)
self.pipe.terminate()
self.pipe.kill()
```
This PR also adds a `PRAGMA integrity_check` for the write tests
Reviewed-by: Preston Thorpe (@PThorpe92)
Closes#1694
* `write` was returning `true` even though it shouldn't because it
should return `false` in case it was already acquired.
* `read` theoritically can increase `nread` after another thread calls
`unlock` between first lock load and increase of `nread`. So it looks
something like this:
1. THREAD 1: read()
2. THREAD 2: get lock = `SHARED_LOCK`
3. THREAD 1: unlock -> lock is now `NO_LOCK`
4. THREAD 2: increase nread, return true.
This is obviously wrong because `nreads` will be ` > 0 ` but `lock`
is `NO_LOCK`
Closes#1676
This PR fixes an unsound usage of unsafe {
str::from_utf8_unchecked(word) } in the public function keyword_token in
mod.rs.
The function now uses std::str::from_utf8(word).ok()? to safely handle
invalid UTF-8, eliminating the unsoundness.
No logic or API changes.
Code compiles and tests pass (where possible).
Closes: https://github.com/tursodatabase/libsql/issues/1859
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#1677
## Purpose
Implement `getTables` which is used to extract metadata about the
database
## Changes
- Implement `JDBC4DatabaseMetaData's` `getTables` method
- Extract `JDBC4ResultSet` as field in `JDBC4PreparedStatement`
## Related Issue
https://github.com/tursodatabase/limbo/issues/615Closes#1687
we were skipping `MakeRecord` for the new values from the `SET` clause
if the index wasn't unique, effectively emitting NULLs. Although this
would actually already fail in the `IdxInsert` instruction because the
record register didn't actually contain a record.
this has been (I think) caught in at least 1. limbo stress 2.
antithesis, but I incorrectly assumed it was something more edge-casey
instead of broken like this in a fairly basic way
Closes#1689
Throughout the cleaning I discovered that the current pluck mode is
broken, so I took the lead and also fixed it.
EDIT: Address comments on #1620 by improving our documentation
Closes#1663
This PR adds the beginnings of
[AUTOVACUUM](https://www.sqlite.org/lang_vacuum.html) to Limbo. It adds
a feature flag called `omit_autovacuum` which is analogous to
`SQLITE_OMIT_AUTOVACUUM`. It is off by default, same as SQLite.
It introduces the concept of [pointer map pages](https://www.sqlite.org/
fileformat.html#pointer_map_or_ptrmap_pages) which are reverse index
pages used to map pages to their parents. This is used to swap pages
(when a table is deleted for instance) to keep root pages clustered at
the beginning of the file. It's also used while creating a table to
ensure that root pages are clustered at the beginning (although, this
isn't completely implemented yet)
Finally, it also adds a couple of missing instructions like `Int64` that
are required for `PRAGMA` commands related to `auto_vacuum` settings
<img width="1512" alt="Screenshot 2025-05-28 at 8 47 51 PM"
src="https://github.com/user-
attachments/assets/d52eb74f-5b79-4d52-9401-1bdc2dcc304d" />
Closes#1600
This PR implements the `ToSqlString` trait to most of the `ast` structs
and to the `SelectPlan`, `UpdatePlan`, `DeletePlan`,
`CompoundSelectPlan`.
Inside the files in the `to_sql_string` folder, I annotated many `TODOs`
with things that seem to diverge from SQLite syntax. The most egregious
by far was that Create Trigger statements do not use the standard
`delete`, `select`, `update`, and `insert` statements. The parser uses
different structs for those statements only in Create Trigger. E.g
`ast::TriggerCmdUpdate` instead of `ast::Update` and so on.
Also, as this iteration of reverse parsing is not particularly efficient
in the number of string allocations it does. I tested different methods
of achieving this by using `format!`, pushing directly to a `String`, or
just pushing to `Vec<String>` and joining all the string with a space
separator. I focused mainly on trying to get the syntax to print
correctly without major hurdles in understanding the code.
Lastly, I intend in the future to use this code in the simulator to
expand the its available syntax.
Closes#1619
This commit ensures that the metadata in the database header and the pointer map pages allocated are correctly persisted to the page cache. This was not being done earlier.
