Files
turso/core/translate/delete.rs
Glauber Costa 08b2e685d5 Persistence for DBSP-based materialized views
This fairly long commit implements persistence for materialized view.
It is hard to split because of all the interdependencies between components,
so it is a one big thing. This commit message will at least try to go into
details about the basic architecture.

Materialized Views as tables
============================

Materialized views are now a normal table - whereas before they were a virtual
table.  By making a materialized view a table, we can reuse all the
infrastructure for dealing with tables (cursors, etc).

One of the advantages of doing this is that we can create indexes on view
columns.  Later, we should also be able to write those views to separate files
with ATTACH write.

Materialized Views as Zsets
===========================

The contents of the table are a ZSet: rowid, values, weight. Readers will
notice that because of this, the usage of the ZSet data structure dwindles
throughout the codebase. The main difference between our materialized ZSet and
the standard DBSP ZSet, is that obviously ours is backed by a BTree, not a Hash
(since SQLite tables are BTrees)

Aggregator State
================

In DBSP, the aggregator nodes also have state. To store that state, there is a
second table.  The table holds all aggregators in the view, and there is one
table per view. That is __turso_internal_dbsp_state_{view_name}. The format of
that table is similar to a ZSet: rowid, serialized_values, weight. We serialize
the values because there will be many aggregators in the table. We can't rely
on a particular format for the values.

The Materialized View Cursor
============================

Reading from a Materialized View essentially means reading from the persisted
ZSet, and enhancing that with data that exists within the transaction.
Transaction data is ephemeral, so we do not materialize this anywhere: we have
a carefully crafted implementation of seek that takes care of merging weights
and stitching the two sets together.
2025-09-05 07:04:33 -05:00

144 lines
4.8 KiB
Rust

use crate::schema::Table;
use crate::translate::emitter::emit_program;
use crate::translate::optimizer::optimize_plan;
use crate::translate::plan::{DeletePlan, Operation, Plan};
use crate::translate::planner::{parse_limit, parse_where};
use crate::util::normalize_ident;
use crate::vdbe::builder::{ProgramBuilder, ProgramBuilderOpts, TableRefIdCounter};
use crate::{schema::Schema, Result, SymbolTable};
use std::sync::Arc;
use turso_parser::ast::{Expr, Limit, QualifiedName, ResultColumn};
use super::plan::{ColumnUsedMask, JoinedTable, TableReferences};
#[allow(clippy::too_many_arguments)]
pub fn translate_delete(
schema: &Schema,
tbl_name: &QualifiedName,
where_clause: Option<Box<Expr>>,
limit: Option<Limit>,
returning: Vec<ResultColumn>,
syms: &SymbolTable,
mut program: ProgramBuilder,
connection: &Arc<crate::Connection>,
) -> Result<ProgramBuilder> {
let tbl_name = normalize_ident(tbl_name.name.as_str());
// Check if this is a system table that should be protected from direct writes
if crate::schema::is_system_table(&tbl_name) {
crate::bail_parse_error!("table {} may not be modified", tbl_name);
}
if schema.table_has_indexes(&tbl_name) && !schema.indexes_enabled() {
// Let's disable altering a table with indices altogether instead of checking column by
// column to be extra safe.
crate::bail_parse_error!(
"DELETE for table with indexes is disabled. Omit the `--experimental-indexes=false` flag to enable this feature."
);
}
// FIXME: SQLite's delete using Returning is complex. It scans the table in read mode first, building
// the result set, and only after that it opens the table for writing and deletes the rows. It
// also uses a couple of instructions that we don't implement yet (i.e.: RowSetAdd, RowSetRead,
// RowSetTest). So for now I'll just defer it altogether.
if !returning.is_empty() {
crate::bail_parse_error!("RETURNING currently not implemented for DELETE statements.");
}
let result_columns = vec![];
let mut delete_plan = prepare_delete_plan(
schema,
tbl_name,
where_clause,
limit,
result_columns,
&mut program.table_reference_counter,
connection,
)?;
optimize_plan(&mut delete_plan, schema)?;
let Plan::Delete(ref delete) = delete_plan else {
panic!("delete_plan is not a DeletePlan");
};
let opts = ProgramBuilderOpts {
num_cursors: 1,
approx_num_insns: estimate_num_instructions(delete),
approx_num_labels: 0,
};
program.extend(&opts);
emit_program(&mut program, delete_plan, schema, syms, |_| {})?;
Ok(program)
}
pub fn prepare_delete_plan(
schema: &Schema,
tbl_name: String,
where_clause: Option<Box<Expr>>,
limit: Option<Limit>,
result_columns: Vec<super::plan::ResultSetColumn>,
table_ref_counter: &mut TableRefIdCounter,
connection: &Arc<crate::Connection>,
) -> Result<Plan> {
let table = match schema.get_table(&tbl_name) {
Some(table) => table,
None => crate::bail_parse_error!("no such table: {}", tbl_name),
};
// Check if this is a materialized view
if schema.is_materialized_view(&tbl_name) {
crate::bail_parse_error!("cannot modify materialized view {}", tbl_name);
}
let table = if let Some(table) = table.virtual_table() {
Table::Virtual(table.clone())
} else if let Some(table) = table.btree() {
Table::BTree(table.clone())
} else {
crate::bail_parse_error!("Table is neither a virtual table nor a btree table");
};
let indexes = schema.get_indices(table.get_name()).to_vec();
let joined_tables = vec![JoinedTable {
op: Operation::default_scan_for(&table),
table,
identifier: tbl_name,
internal_id: table_ref_counter.next(),
join_info: None,
col_used_mask: ColumnUsedMask::default(),
database_id: 0,
}];
let mut table_references = TableReferences::new(joined_tables, vec![]);
let mut where_predicates = vec![];
// Parse the WHERE clause
parse_where(
where_clause.as_deref(),
&mut table_references,
None,
&mut where_predicates,
connection,
)?;
// Parse the LIMIT/OFFSET clause
let (resolved_limit, resolved_offset) =
limit.map_or(Ok((None, None)), |mut l| parse_limit(&mut l, connection))?;
let plan = DeletePlan {
table_references,
result_columns,
where_clause: where_predicates,
order_by: vec![],
limit: resolved_limit,
offset: resolved_offset,
contains_constant_false_condition: false,
indexes,
};
Ok(Plan::Delete(plan))
}
fn estimate_num_instructions(plan: &DeletePlan) -> usize {
let base = 20;
base + plan.table_references.joined_tables().len() * 10
}