Commit Graph

4570 Commits

Author SHA1 Message Date
pedrocarlo
0df6c87f07 Fixed Group By collation 2025-05-19 15:22:14 -03:00
pedrocarlo
bba9689674 Fixed matching bug for defining collation context to use 2025-05-19 15:22:14 -03:00
pedrocarlo
a818b6924c Removed repeated binary expression translation. Adjusted the set_collation to capture additional context of whether it was set by a Collate expression or not. Added some tests to prove those modifications were necessary. 2025-05-19 15:22:14 -03:00
pedrocarlo
f8854f180a Added collation to create table columns 2025-05-19 15:22:14 -03:00
pedrocarlo
d0a63429a6 Naive implementation of collate for queries. Not implemented for column constraints 2025-05-19 15:22:14 -03:00
pedrocarlo
b5b1010e7c set binary collation as default 2025-05-19 15:22:14 -03:00
pedrocarlo
510c70e919 Create CollationSeq enum and functions. Move strum to workspace dependency to avoid version mismatch with Parser 2025-05-19 15:22:14 -03:00
Pekka Enberg
4cf9305947 Merge 'bindings/javascript: Add Statement.iterate() method' from Diego Reis
I still didn't find a good way to implement variadic functions, we
should have some sort of wrapper in JS layer but it didn't work so well
for me so far. But once done it will be easily transferable to any
function.
It also should probably be async, but AFAIC napi doesn't have a straight
way to implement async iterators.

Closes #1515
2025-05-19 20:44:40 +03:00
Pekka Enberg
95ea92faca Merge 'Improve debug build validation speed' from Pere Diaz Bou
Various things to improve speed of long fuzz test execution time:
* remove unnecessary debug_validate_cell calls
* Add SortedVec for keys in fuzz tests
* Validate btree's depth in fuzz test every 1K inserts to not overload
test with validations. We add `VALIDATE_BTREE`  env variable to enable
validation on every insert in case it is needed.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1521
2025-05-19 20:42:48 +03:00
Pekka Enberg
5bd85774cf Merge 'Update README.md' from Yusheng Guo
A syntax error.

Closes #1522
2025-05-19 20:42:25 +03:00
Yusheng Guo
810beeea93 Update README.md
A syntax error.
2025-05-19 18:29:57 +08:00
Jussi Saurio
d2b1be8af7 Merge 'optimizer: fix order by removal logic' from Jussi Saurio
1. `group_by_contains_all` was incorrect - it was not checking that all
order by columns are in group by; it was instead checking that all group
by columns are in order by, which is absolutely incorrect for the
intended purpose.
2. remove ORDER BY clause if GROUP BY clause can sort the rows in the
same way.
Test failures are not related

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1511
2025-05-19 11:29:17 +03:00
Jussi Saurio
b7b4f6a390 Merge 'Mark WHERE terms as consumed instead of deleting them' from Jussi Saurio
We've run into trouble in multiple places due to the fact that we delete
terms from the where clause (e.g. when a constant condition is removed,
or the term becomes part of an index seek key).
A simpler solution is to add a flag indicating that the term is consumed
(used), so that it is not translated in the main loop anymore when WHERE
clause terms are evaluated.
note: CI failures are unrelated

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1477
2025-05-19 11:28:09 +03:00
Pere Diaz Bou
f2d0d61962 copilot nice suggestions :) 2025-05-19 09:59:28 +02:00
Pere Diaz Bou
5eab588115 improve debug build validation speed
Various things:
* remove unnecessary debug_validate_cell calls
* Add SortedVec for keys in fuzz tests
* Validate btree's depth in fuzz test every 1K inserts to not overload
test with validations. We add `VALIDATE_BTREE`  env variable to enable
validation on every insert in case it is needed.
2025-05-19 09:53:15 +02:00
Jussi Saurio
092462fa74 fix build 2025-05-19 07:29:02 +03:00
Jussi Saurio
7c6a4410d2 Merge '(btree): Implement support for handling offset-based payload access with overflow support' from Krishna Vishal
This PR adds a new function `read_write_payload_with_offset` to support
reading and writing payload data at specific offsets, handling both
local content and overflow pages. This is a port of SQLite's
`accessPayload` function in `btree.c` and will be essential for
supporting incremental blob I/O in the coming PRs.
- Added a state machine called `PayloadOverflowWithOffset` to make the
procedure reentrant.
- Correctly processes both local payload data and payload stored in
overflow pages
Testing:
- Reading and writing to a column with no overflow pages.
- Reading and writing at an offset with overflow pages (spanning 10
pages)

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1476
2025-05-18 22:58:10 +03:00
Jussi Saurio
3185aabd20 Merge 'Cli config 2' from Pedro Muniz
As there were many merge conflicts for the other PR, I rewrote the code
and condensed it here.
ORIGINAL PR TEXT: Provides the code to almost close
https://github.com/tursodatabase/limbo/issues/1251 . The JsonSchema is
derived, but I am still not sure how to automate the distribution to
SchemaStore for autocomplete. I added some docs for that want to see the
config file description. I still am not sure how to automate this
documentation. Maybe some macro magic?

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #1430
2025-05-18 22:56:22 +03:00
Jussi Saurio
372850756d Merge 'Fix updating single value' from Pedro Muniz
Closes #1482. I needed to change the `key_exists_in_index` function
because it zips the values from the records it is comparing, but if one
of the records is empty or not of the same length, the `all` function
could return true incorrectly.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1514
2025-05-18 22:51:11 +03:00
pedrocarlo
fd51c0a970 invalidate records not necessary for fix 2025-05-18 16:43:25 -03:00
Jussi Saurio
071940f9a7 Merge 'Autoindex fix' from Pedro Muniz
Closes #1508 . There were two small issues to fix:
1. We were not checking in the IndexMap of columns, if the unique column
name is declared in the composite declaration exists in the IndexMap.
This solved the first this statement `create table t4(a, unique(b));`.
2. The second thing was that we forgot to add the column_name to the
HashSet of columns.
```rust
Some(PrimaryKeyDefinitionType::Simple { column, .. }) => {
      let mut columns = HashSet::new();
      columns.insert(std::mem::take(column));
      // Have to also insert the current column_name we are iterating over in primary_key_column_results
      columns.insert(column_name.clone()); <-- Fix here
      primary_key_definition =
           Some(PrimaryKeyDefinitionType::Composite { columns });
      }
```
The rest of the modifications are just some small simplifications for
readability and avoiding some clones

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1512
2025-05-18 22:41:02 +03:00
pedrocarlo
c8b768f1ea add tests 2025-05-18 12:43:11 -03:00
pedrocarlo
7f081c1ac9 remove transmute. Just iterate over columns. No need for unsafe 2025-05-18 12:32:49 -03:00
Jussi Saurio
06c4a5dea9 Merge 'use temporary db in sqlite3 wal tests to fix later tests failing' from Preston Thorpe
This prevents the new wal checkpoint tests in `sqlite3/tests/compat`
from writing/creating `test` table to `testing/testing.db`, which is
queried in later tests which fail for having an extra table.
There is another issue with failing tests related to the new `count`
impl that I am in the process of fixing as well, but that will be a
separate PR.

Closes #1513
2025-05-18 09:48:23 +03:00
Diego Reis
bc88b7cb65 bind/js: Formatting 2025-05-18 00:51:49 -03:00
Diego Reis
9f6e242e42 bind/js: Partially implements iterate() method
The API still is sync and isn't variadic
2025-05-18 00:51:23 -03:00
pedrocarlo
af1f9492ef fix updating single value 2025-05-17 19:43:24 -03:00
PThorpe92
6d70e6d048 Add reset db to Makefile to create clean testing db between tests that perform writes 2025-05-17 16:23:17 -04:00
PThorpe92
45de41626c Adjust sqlite3 compat tests to use temp cloned database so further tests dont break 2025-05-17 16:11:59 -04:00
PThorpe92
f128887348 Add script to create clone of testing/testing.db to allow for writes in tests 2025-05-17 16:11:16 -04:00
pedrocarlo
b3b52f7f2f applying config in app 2025-05-17 15:51:43 -03:00
pedrocarlo
c73927729d config docs 2025-05-17 15:51:17 -03:00
pedrocarlo
a614114591 Config Definitions 2025-05-17 15:51:17 -03:00
pedrocarlo
0e6ef1f478 removed some clone, simplified slightly logic + also inserted the column name of the current column we are iterating, not only the last column contained in PrimaryKeyDefinitionType::Simple 2025-05-17 15:32:58 -03:00
pedrocarlo
166dc2184e fix autoindex creation not detecting if column existed in created table declaration using transmute to avoid cloning 2025-05-17 12:58:00 -03:00
Jussi Saurio
93d88527c3 optimizer: remove order by if group by already sorts the result properly 2025-05-17 17:42:52 +03:00
Jussi Saurio
ce8b2722cf optimizer: fix incorrect logic in group_by_contains_all 2025-05-17 17:28:29 +03:00
Jussi Saurio
0fa4ebaec2 Merge 'tpc-h: fix 'time' output in Ubuntu CI' from Jussi Saurio
Closes #1509
2025-05-17 17:12:25 +03:00
Jussi Saurio
53b135829a tpc-h: fix 'time' output in Ubuntu CI 2025-05-17 16:55:05 +03:00
Jussi Saurio
d584a1879b Mark WHERE terms as consumed instead of deleting them
We've run into trouble in multiple places due to the fact that
we delete terms from the where clause (e.g. when a constant condition
is removed, or the term becomes part of an index seek key).

A simpler solution is to add a flag indicating that the term is
consumed (used), so that it is not translated in the main loop
anymore when WHERE clause terms are evaluated.
2025-05-17 15:44:12 +03:00
Jussi Saurio
31bb26c8bb Merge 'fix labeler correct file name extension use .yml instead of .yaml' from Mohamed A. Salah
I saw that the labeler workflow trying to get `labeler.yml` file but i
see it's `.yaml` in the repo so i renamed it.
![image](https://github.com/user-
attachments/assets/698bf54c-aeb2-48f1-bade-95862968b706)

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1506
2025-05-17 15:32:59 +03:00
Mohamed A. Salah
9b52e42fe6 feat: revert the labler workflow 2025-05-17 13:55:42 +03:00
Mohamed A. Salah
c7871e9e27 checkout first 2025-05-17 13:32:58 +03:00
Mohamed A. Salah
fe183e8956 fix labeler correct file name extension use .yml instead of .yaml 2025-05-17 13:27:32 +03:00
Pekka Enberg
26e5496c19 Merge 'Fix autoindex of primary key marked as unique' from Pere Diaz Bou
Primary keys that are marked as unique constraint, do not need to have
separate indexes, one is enough. In the case primary key is integer,
therefore rowid alias, we still need an index to satisfy unique
constraint.

Closes #1494
2025-05-17 08:02:04 +03:00
Pekka Enberg
5209d0c14d Merge 'Add labeler workflow and reorganize macros' from Preston Thorpe
No semantic changes in this PR, the macros were organized because
`lib.rs` was getting out of hand anyhow, and because it's now easier to
determine if a change submitted in a PR is related to `extensions/core`
for labeling.
# NOTE:
this will not work until it is merged, because it needs write
permissions from the base or root of the repo
https://github.com/actions/labeler?tab=readme-ov-file#initial-set-up-of-
the-labeler-action

Closes #1503
2025-05-17 08:01:21 +03:00
PThorpe92
92185f7ba8 Add macros to list of PR labels 2025-05-16 21:43:26 -04:00
PThorpe92
a4dca99bfc Add stress path to anthithesis label workflow 2025-05-16 11:20:14 -04:00
PThorpe92
e86aec892e Reorganize macros to separate those needed for extension library for better labeling of PRs 2025-05-16 11:20:14 -04:00
PThorpe92
426c1c6155 Add configuration file for labeler workflow 2025-05-16 11:20:14 -04:00