Files
turso/bindings/go
Pekka Enberg 2bd221e5db Merge 'Add embedded library support to Go adapter' from Jonathan Ness
This change enables the Go adapter to embed platform-specific libraries
and extract them at runtime, eliminating the need for users to set
LD_LIBRARY_PATH or other environment variables.
- Add embedded.go with core library extraction functionality
- Update limbo_unix.go and limbo_windows.go to use embedded libraries
- Add build_lib.sh script to generate platform-specific libraries
- Update README.md with documentation for the new feature
- Add .gitignore to prevent committing binary files
- Add test coverage for Vector operations (vector(), vector_extract(),
vector_distance_cos()) and sqlite core features
The implementation maintains backward compatibility with the traditional
library loading mechanism as a fallback. This approach is inspired by
projects like go-embed-python that use a similar technique for native
library distribution.
https://github.com/tursodatabase/limbo/issues/506

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #1434
2025-05-07 22:31:29 +03:00
..
2025-03-29 22:04:08 +01:00
2025-05-06 21:43:18 -07:00

Limbo driver for Go's database/sql library

NOTE: this is currently heavily W.I.P and is not yet in a usable state.

This driver uses the awesome purego library to call C (in this case Rust with C ABI) functions from Go without the use of CGO.

Embedded Library Support

This driver includes an embedded library feature that allows you to distribute a single binary without requiring users to set environment variables. The library for your platform is automatically embedded, extracted at runtime, and loaded dynamically.

Building from Source

To build with embedded library support, follow these steps:

# Clone the repository
git clone https://github.com/tursodatabase/limbo

# Navigate to the Go bindings directory
cd limbo/bindings/go

# Build the library (defaults to release build)
./build_lib.sh

# Alternatively, for faster builds during development:
./build_lib.sh debug

Build Options:

  • Release Build (default): ./build_lib.sh or ./build_lib.sh release

    • Optimized for performance and smaller binary size
    • Takes longer to compile and requires more system resources
    • Recommended for production use
  • Debug Build: ./build_lib.sh debug

    • Faster compilation times with less resource usage
    • Larger binary size and slower runtime performance
    • Recommended during development or if release build fails

If the embedded library cannot be found or extracted, the driver will fall back to the traditional method of finding the library in the system paths.

To use: (UNSTABLE testing or development purposes only)

Build the driver with the embedded library as described above, then simply import and use. No environment variables needed!

Option 2: Manual library setup

Linux | MacOS

All commands listed are relative to the bindings/go directory in the limbo repository

cargo build --package limbo-go

# Your LD_LIBRARY_PATH environment variable must include limbo's `target/debug` directory

export LD_LIBRARY_PATH="/path/to/limbo/target/debug:$LD_LIBRARY_PATH"

Windows

cargo build --package limbo-go

# You must add limbo's `target/debug` directory to your PATH
# or you could built + copy the .dll to a location in your PATH
# or just the CWD of your go module

cp path\to\limbo\target\debug\lib_limbo_go.dll .

go test

Temporarily you may have to clone the limbo repository and run:

go mod edit -replace github.com/tursodatabase/limbo=/path/to/limbo/bindings/go

import (
    "fmt"
    "database/sql"
    _"github.com/tursodatabase/limbo"
)

func main() {
	conn, err := sql.Open("sqlite3", ":memory:")
	if err != nil {
        fmt.Printf("Error: %v\n", err)
        os.Exit(1)
	}
    sql := "CREATE table go_limbo (foo INTEGER, bar TEXT)"
    _ = conn.Exec(sql)

    sql = "INSERT INTO go_limbo (foo, bar) values (?, ?)"
    stmt, _ := conn.Prepare(sql)
    defer stmt.Close()
    _  = stmt.Exec(42, "limbo")
    rows, _ := conn.Query("SELECT * from go_limbo")
    defer rows.Close()
    for rows.Next() {
        var a int
        var b string
		_ = rows.Scan(&a, &b)
        fmt.Printf("%d, %s", a, b)
    }
}

Implementation Notes

The embedded library feature was inspired by projects like go-embed-python, which uses a similar approach for embedding and distributing native libraries with Go applications.