alh477/streamdb-rs repository overview

⁠StreamDB (v2.0.0)

StreamDB is a lightweight, thread-safe embedded key-value database implemented in Rust. It utilizes a reverse trie data structure for suffix-based search performance, making it ideal for domain analysis, log parsing, and bioinformatics.

⁠Key Features

• Suffix Search Optimization: Built-in reverse trie for efficient lookup of keys sharing a common suffix.
• Thread-Safe Architecture: Uses interior mutability (RwLock and Mutex) for safe concurrent access from multiple threads.
• Structural Sharing: Powered by im::OrdMap for memory-efficient persistent trie updates.
• C FFI & AI-Ready: Standard C headers and shared libraries included for seamless integration with C, C++, and Python.
• Deterministic Environment: Built using Nix Flakes to ensure bit-for-bit reproducibility across all deployments.

⁠Architecture Overview

StreamDB consists of a core Rust library that manages a persistent Trie and multiple storage backends (In-Memory or File-based).

⁠Quick Start (Rust)

use streamdb::{StreamDb, Result};

fn main() -> Result<()> {
    let db = StreamDb.open_memory()?; // Open in-memory instance
    
    // Insert binary data
    db.insert(b"user:alice", b"Alice Smith")?;
    
    // Perform suffix search
    let results = db.suffix_search(b"alice")?;
    for result in results {
        println!("Found key: {:?}", result.key);
    }
    
    Ok(())
}

⁠🤖 AI & Scripting (Python)

The StreamDB AI-Ready Container provides a pre-configured Python environment with the database library pre-loaded at /lib/libstreamdb.so.

⁠Python Example (ctypes wrapper)

import ctypes

# Load the deterministic binary
lib = ctypes.CDLL("/lib/libstreamdb.so")

# Initialize In-Memory DB
db_handle = ctypes.c_void_p()
lib.streamdb_open_memory(ctypes.byref(db_handle))

# Insert a key
key = b"agent:task_01"
val = b"completed"
lib.streamdb_insert(db_handle, key, len(key), val, len(val), None)

lib.streamdb_close(db_handle)

⁠🐳 Docker & Reproducibility

StreamDB uses Nix Flakes to guarantee that the container image is identical regardless of the build machine.

⁠Build and Load

nix build .#docker
docker load < result

⁠Verify Installation

docker run --rm streamdb-ai-ready:latest ls -l /include/streamdb.h /lib/libstreamdb.so

The file timestamps will appear as Jan 1, 1970, confirming a deterministic Nix build.

⁠🛠 Configuration

StreamDB can be tuned via the Config struct:

⁠📊 Benchmarks

Theoretical results from AMD Ryzen 7 5800X equivalent:

• Insert (100B): ~1.2M ops/sec
• Get (Cached): ~3.5M ops/sec
• Suffix Search: ~120K ops/sec (200 matches)

⁠⚖️ License

This project is licensed under the GNU Lesser General Public License v2.1 (LGPL-2.1-or-later).

Copyright (C) 2025 DeMoD LLC.