Commit Graph

1302 Commits

Author SHA1 Message Date
Pere Diaz Bou
39a75147d4 Page cache by page_number and frame_number
Since page cache is now shared by default, we need to cache pages by
page number and something else. I chose to go with max_frame of
connection, because this connection will have a max_frame set until from
the start of a transaction until the end of it.

With key pairs of (pgno, max_frame) we make sure each connection is
caching based on the snapshot it is at as two different connections
might have the same pageno being using but a different frame. If both
have same max_frame then they will share same page.
2024-12-13 21:57:27 +01:00
Pekka Enberg
138b3a00e8 Merge 'use correct min/max frames' from Pere Diaz Bou
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #465
2024-12-13 18:48:26 +02:00
Pekka Enberg
f5212b0672 Merge 'loop writing header while initializing' from Pere Diaz Bou
if somehow this doesn't finish in a single loop, do more just in case

Closes #463
2024-12-13 18:48:04 +02:00
Pere Diaz Bou
1e5239a164 use correct min/max frames 2024-12-13 14:06:29 +01:00
Pere Diaz Bou
352b3540ee loop writing header while initializing
if somehow this doesn't finish in a single loop, do more just in case
2024-12-13 13:25:56 +01:00
Pere Diaz Bou
47d25292b2 Merge 'Start multithread support' from Pere Diaz Bou
Since we expect to ensure thread safety between multiple threads in the
future, we extract what is important to be shared between multiple
connections with regards to WAL/Pager.
This is WIP so I just put whatever feels like important behind a RwLock
but expect this to change to Atomics in the future as needed. Maybe even
these locks might disappear because they will be better served with
transaction locks.
This also includes addition of unsafe Sync/Send + UnsafeCell in Pages.

Closes #415
2024-12-13 13:22:17 +01:00
Pere Diaz Bou
3e59da439c fmt 2024-12-13 13:10:33 +01:00
Pere Diaz Bou
1a663a6ed7 cargo stuff rm 2024-12-13 13:09:13 +01:00
Pere Diaz Bou
b43e8e46f6 impl sync/send for cache 2024-12-13 13:09:13 +01:00
Pere Diaz Bou
97dd95abea core: change Rc<RefCell<Page>> to Arc<Page>
This includes an inner struct in Page wrapped with Unsafe cell to access
it. This is done intentionally because concurrency control of pages is
handled by pages and not by the page itself.
2024-12-13 13:09:13 +01:00
Pere Diaz Bou
c816186326 fmt 2024-12-13 13:09:13 +01:00
Pere Diaz Bou
3fda2d09b9 Extract multi threaded part from WalFile to WalFileShared
Since we expect to ensure thread safety between multiple threads in the
future, we extract what is important to be shared between multiple
connections with regards to WAL.

This is WIP so I just put whatever feels like important behind a RwLock
but expect this to change to Atomics in the future as needed. Maybe even
these locks might disappear because they will be better served with
transaction locks.
2024-12-13 13:09:13 +01:00
Pere Diaz Bou
a4297702bd extract page cache to be multi threaded 2024-12-13 13:09:13 +01:00
Pekka Enberg
da3765d061 Merge 'Document typical code flow with charts' from Kacper Madej
I'd love to contribute to Limbo, but wrapping my head around the whole
project is hard.
I decided to start by creating a diagram of a typical flow so that other
potential contributors might have an easier time :)
LMK if this is useful, I can create more of those when I dig deeper.

Closes #456
2024-12-13 11:49:02 +02:00
Pekka Enberg
2b85d2a600 Merge 'Add implementation and tests for replace scalar function' from Alperen Keleş
Adds `replace` scalar function.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #446
2024-12-13 11:02:08 +02:00
Pekka Enberg
fefcf1261a Merge 'Add support for unary positive' from Jean Arhancet
Add support for the unary positive operator

Closes #461
2024-12-13 11:01:40 +02:00
JeanArhancet
8bf6572e9e feat: support unary positive 2024-12-13 02:07:34 +01:00
alpaylan
4f395623ab fix the comment on replace panic in case of text cast failure 2024-12-12 15:03:35 -05:00
alpaylan
5742c51c5c improve error messages on replace function call 2024-12-12 14:14:04 -05:00
Alperen Keleş
841a4fe2f8 Merge branch 'tursodatabase:main' into main 2024-12-12 13:13:41 -05:00
Pekka Enberg
3023d228c7 Limbo 0.0.9 2024-12-12 18:49:05 +02:00
Pekka Enberg
afbf9eb9da Update CHANGELOG 2024-12-12 18:47:42 +02:00
Pekka Enberg
96547e924c Merge 'Add buffered cli input and match sqlite cli repl' from Preston Thorpe
This PR prevents input from being evaluated in the case of an incomplete
statement entered, allowing the user to continue the query on following
lines.
This matches the behavior of the sqlite CLI by alerting the user how
many parenthesis are currently open (up to 9) `(x1...`, or too many
closed `(!...` and only processes `.`commands when they are the only
input.
![image](https://github.com/user-
attachments/assets/c062814c-5499-41c1-8ccb-3c5b1dfa682c)
>sqlite
![image](https://github.com/user-
attachments/assets/754fa52b-f006-40ed-89e9-7b69f513bbec)
fixes #450

Closes #457
2024-12-12 18:47:08 +02:00
Kacper Madej
52b94f7181 Update diagram 2024-12-12 17:22:03 +01:00
Kacper Madej
ffede00531 Add link to sqlite docs 2024-12-12 17:11:51 +01:00
Pekka Enberg
5796b418b9 Merge 'Add support for soundex() function' from flaneur
add [soundex](https://www.sqlite.org/lang_corefunc.html#soundex) scalar
function.
it seems that sqlite did not enable `soundex()` function by default
unless build it with `SQLITE_SOUNDEX`, while the sqlite in the ci
workflow did not enable it. this pr skipped the test over `soundex()`
temporarily in the `scalar-function.test` file.

Closes #453
2024-12-12 18:10:46 +02:00
Kacper Madej
03c4f6f6ef Document typical flow 2024-12-12 17:06:29 +01:00
PThorpe92
4dc71bc9ad Add buffered cli input to allow for incomplete statements 2024-12-12 11:01:14 -05:00
Pekka Enberg
91764b85e6 Merge 'Add bitwise vdbe ops' from Preston Thorpe
Love the project, been following your blog posts for quite a while now.
I asked on Discord prior to submitting this, just because I didn't see a
specific issue for this feature... but if this PR is out of scope for
contributors, feel free to close it as I just had a good time hacking on
it.
This PR adds support for `BitAnd`, `BitOr`, and `BitNot` operators in
the vdbe, as well as unary expressions applied to aggregate functions;
which was needed in order to have `BitNot` support the same tests that
the other operators had.
*Also added unary negation of function calls, because since unary ops
were added, I figured adding support for the other existing unary
operator might be in scope, but lmk if not.
Let me know if there is any more tests or documentation to add/improve.

Closes #445
2024-12-12 17:35:12 +02:00
Pere Diaz Bou
e79e42b0b1 Merge 'Change octet_length to yes in COMPAT.md' from Krishna Vishal
- Its added in https://github.com/tursodatabase/limbo/pull/430

Closes #454
2024-12-12 16:22:17 +01:00
krishvishal
33f3eb02c8 Change octet_length to yes in COMPAT.md
- Its added in https://github.com/tursodatabase/limbo/pull/430
2024-12-12 19:46:01 +05:30
Li Yazhou
03288e5170 add impl about scalar function soundex with test 2024-12-12 21:48:05 +08:00
Pekka Enberg
0722037465 github: Fix stale workflow access token 2024-12-12 10:59:10 +02:00
Pekka Enberg
4e40d9ecee Update CHANGELOG 2024-12-12 10:47:11 +02:00
Pekka Enberg
58cd9d72b0 Merge 'Implement support for iif()' from Alex Miller
In sqlite, iif() looks like:
```
sqlite> create table iiftest(a int, b int, c int);
sqlite> explain select iif(a,b,c) from iiftest;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     11    0                    0   Start at 11
1     OpenRead       0     2     0     3              0   root=2 iDb=0; iiftest
2     Rewind         0     10    0                    0
3       Column         0     0     2                    0   r[2]= cursor 0 column 0
4       IfNot          2     7     1                    0
5       Column         0     1     1                    0   r[1]= cursor 0 column 1
6       Goto           0     8     0                    0
7       Column         0     2     1                    0   r[1]= cursor 0 column 2
8       ResultRow      1     1     0                    0   output=r[1]
9     Next           0     3     0                    1
10    Halt           0     0     0                    0
11    Transaction    0     0     1     0              1   usesStmtJournal=0
12    Goto           0     1     0                    0
```
And with this change, in limbo it looks like:
```
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     14    0                    0   Start at 14
1     OpenReadAsync      0     2     0                    0   table=iiftest, root=2
2     OpenReadAwait      0     0     0                    0
3     RewindAsync        0     0     0                    0
4     RewindAwait        0     13    0                    0   Rewind table iiftest
5       Column           0     0     2                    0   r[2]=iiftest.a
6       IfNot            2     9     1                    0   if !r[2] goto 9
7       Column           0     1     1                    0   r[1]=iiftest.b
8       Goto             0     10    0                    0
9       Column           0     2     1                    0   r[1]=iiftest.c
10      ResultRow        1     1     0                    0   output=r[1]
11    NextAsync          0     0     0                    0
12    NextAwait          0     5     0                    0
13    Halt               0     0     0                    0
14    Transaction        0     0     0                    0
15    Goto               0     1     0                    0
```
---
Rust commentary overly welcome, as idk what I'm doing in this language.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #424
2024-12-12 10:46:22 +02:00
Pekka Enberg
2e754e37f1 perf: Update Cargo.lock files
We already bumped up clap to fix up a Github dependabot complaint, but
let's also update the lock files...
2024-12-12 09:08:51 +02:00
Pekka Enberg
929ca22c9d Merge 'Enhance README.md' from Kim Seon Woo
# Purpose of the PR
Let's add instructions on how to build and run limbo for first-time
users:

Closes #452
2024-12-12 09:07:34 +02:00
김선우
8e6184c932 Enhance README.md 2024-12-12 16:05:23 +09:00
김선우
c083e59457 Enhance README.md 2024-12-12 13:47:14 +09:00
Alex Miller
c4d4569dc9 Merge remote-tracking branch 'upstream/main' into expr-iif 2024-12-11 20:13:54 -08:00
alpaylan
021456326e change 0.1+0.2 test into 0.3 as limbo does not yet support decimals 2024-12-11 16:32:06 -05:00
alpaylan
da28ed51ca add implementation and tests for replace scalar function 2024-12-11 16:23:13 -05:00
PThorpe92
16595f39f5 Add support for unary op negation of aggregates 2024-12-11 15:38:21 -05:00
PThorpe92
d5391dc716 Add vdbe bitwise operators: and, or, not 2024-12-11 11:06:22 -05:00
jussisaurio
dddf850111 Merge 'Update clap to 4.5' from Pekka Enberg
The Github dependabot complains about anstream, which comes through
`clap`:
https://github.com/tursodatabase/limbo/security/dependabot/8

Closes #444
2024-12-11 17:08:38 +02:00
jussisaurio
eb9374aebf Merge 'Add support for CASE expressions.' from Alex Miller
There's two forms of case:
  CASE (WHEN [bool expr] THEN [value])+ (ELSE [value])? END
which checks a series of boolean conditions, and:
  CASE expr (WHEN [expr] THEN [value})+ (ELSE [value])? END
Which checks a series of equality conditions.
This implements support for both. Note that the ELSE is optional, and
will be equivalent to `ELSE null` if not specified.
sqlite3 gives the implementation as:
```
sqlite> explain select case a WHEN a THEN b WHEN c THEN d ELSE 0 END from casetest;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     16    0                    0   Start at 16
1     OpenRead       0     3     0     4              0   root=3 iDb=0; casetest
2     Rewind         0     15    0                    0
3       Column         0     0     2                    0   r[2]= cursor 0 column 0
4       Column         0     0     3                    0   r[3]= cursor 0 column 0
5       Ne             3     8     2     BINARY-8       83  if r[2]!=r[3] goto 8
6       Column         0     1     1                    0   r[1]= cursor 0 column 1
7       Goto           0     13    0                    0
8       Column         0     2     3                    0   r[3]= cursor 0 column 2
9       Ne             3     12    2     BINARY-8       83  if r[2]!=r[3] goto 12
10      Column         0     3     1                    0   r[1]= cursor 0 column 3
11      Goto           0     13    0                    0
12      Integer        0     1     0                    0   r[1]=0
13      ResultRow      1     1     0                    0   output=r[1]
14    Next           0     3     0                    1
15    Halt           0     0     0                    0
16    Transaction    0     0     2     0              1   usesStmtJournal=0
17    Goto           0     1     0                    0
```
and after this patch, limbo gives:
```
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     19    0                    0   Start at 19
1     OpenReadAsync      0     4     0                    0   table=casetest, root=4
2     OpenReadAwait      0     0     0                    0
3     RewindAsync        0     0     0                    0
4     RewindAwait        0     18    0                    0   Rewind table casetest
5       Column           0     0     2                    0   r[2]=casetest.a
6       Column           0     0     3                    0   r[3]=casetest.a
7       Ne               2     3     10                   0   if r[2]!=r[3] goto 10
8       Column           0     1     1                    0   r[1]=casetest.b
9       Goto             0     15    0                    0
10      Column           0     2     3                    0   r[3]=casetest.c
11      Ne               2     3     14                   0   if r[2]!=r[3] goto 14
12      Column           0     3     1                    0   r[1]=casetest.d
13      Goto             0     15    0                    0
14      Integer          0     1     0                    0   r[1]=0
15      ResultRow        1     1     0                    0   output=r[1]
16    NextAsync          0     0     0                    0
17    NextAwait          0     5     0                    0
18    Halt               0     0     0                    0
19    Transaction        0     0     0                    0
20    Goto               0     1     0                    0
```
And then as there's nowhere to annotate this new support in COMPAT.md, I
added a corresponding heading for SELECT expressions and what is/isn't
supported.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #425
2024-12-11 17:05:41 +02:00
Pekka Enberg
617f95c7b6 Update clap to 4.5
The Github dependabot complains about anstream, which comes through `clap`:

https://github.com/tursodatabase/limbo/security/dependabot/8
2024-12-11 14:39:27 +02:00
Pekka Enberg
c03839cc6a Merge 'Upgrade pprof to 0.14' from Pekka Enberg
Github's dependabot complains that the current version has an unsoudness
issue so let's bump to a newer version:
https://github.com/tursodatabase/limbo/security/dependabot/10

Closes #440
2024-12-11 14:07:26 +02:00
Pekka Enberg
ab07c77036 Upgrade pprof to 0.14
Github's dependabot complains that the current version has an unsoudness
issue so let's bump to a newer version:

https://github.com/tursodatabase/limbo/security/dependabot/10
2024-12-11 11:21:09 +02:00
Pekka Enberg
b8aca48a0f Update CHANGELOG 2024-12-11 10:45:02 +02:00