# README

This directory contains the complete documentation for ChronDB, a chronological database based on Git's internal architecture.

## Documentation Structure

The documentation is organized as follows:

### Getting Started

* **Introduction**: Overview of ChronDB and its key features
* **Quick Start Guide**: Step-by-step guide to get up and running quickly
* **Core Concepts**: Fundamental concepts of ChronDB's data model
* **FAQ**: Answers to frequently asked questions

### Tutorials

* **Time Travel Guide**: Practical guide to ChronDB's version history features
* **Branching Guide**: How to effectively use branching in your applications

### Reference

* **Data Model**: Detailed explanation of ChronDB's document structure
* **Version Control**: Complete guide to history and versioning features
* **API Reference**: Comprehensive API documentation

### Connection Methods

* **Protocols Overview**: Summary of all connection protocols
* **REST API Examples**: Using ChronDB with HTTP clients
* **Redis Protocol Examples**: Using ChronDB with Redis clients
* **PostgreSQL Protocol Examples**: Using ChronDB with SQL clients
* **Clojure API**: Native API for JVM applications

### Operations

* **Configuration**: Setup and configuration options
* **Performance**: Tuning and optimization guidelines
* **Operations Guide**: Administration and monitoring

## Documentation Standards

This documentation follows these principles:

1. **User-focused**: Written from the user's perspective, addressing their needs
2. **Progressive disclosure**: Start with simple concepts before introducing complexity
3. **Practical examples**: Each concept is illustrated with concrete code examples
4. **Multiple interfaces**: Examples cover all supported connection protocols
5. **Visual aids**: Diagrams and illustrations clarify complex concepts

## Contributing to Documentation

Contributions to improve the documentation are welcome! Please consider:

1. Adding more practical examples for different use cases
2. Creating tutorials for specific applications
3. Improving explanations of complex concepts
4. Fixing errors or clarifying confusing sections
5. Adding diagrams or visual illustrations

Submit a pull request with your improvements.

## Building the Documentation

The documentation is built using [GitBook](https://www.gitbook.com/) and is available online at [chrondb.avelino.run](https://chrondb.avelino.run/).

When adding links between pages, remember:

1. Do not include the `.md` extension in links between pages
2. Use relative paths for links to other pages in the documentation
3. For example, use `[Data Model](data-model)` instead of `[Data Model](data-model.md)`
4. External links should still include their full URLs and extensions

This ensures compatibility with GitBook's rendering system.

## Community Resources

* **Documentation**: [chrondb.avelino.run](https://chrondb.avelino.run/)
* **Discord Community**: [Join our Discord](https://discord.com/channels/1099017682487087116/1353399752636497992)
* **Discussion Forum**: [GitHub Discussions](https://github.com/avelino/chrondb/discussions)


# Introduction

**Welcome to ChronDB!** A modern, Git-based database that remembers everything.

## What is ChronDB?

ChronDB is a chronological database built on Git's powerful architecture that gives your data complete version history. Think of it as "Git for your database" - every change is tracked, every version is accessible, and you can time-travel through your data's complete history.

ChronDB is licensed under the GNU Affero General Public License v3.0 (AGPLv3), ensuring improvements deployed as hosted services remain available to the community.

## Why Choose ChronDB?

* **Complete History** - Never lose data again. Every change is preserved in your database's timeline.
* **Time Travel** - Query your data as it existed at any point in time.
* **Branching & Merging** - Create isolated environments for testing, development, or experimentation.
* **Multiple Interfaces** - Connect through Clojure, REST API, Redis protocol, or even PostgreSQL protocol.
* **ACID Compliance** - Enjoy full transaction support with commit, rollback, and consistency guarantees.
* **Schema-less** - Store any JSON-compatible data without rigid structure requirements.
* **Lucene-Powered Search** - Execute full-text, composite, and geospatial queries using Lucene's production-grade indexing engine.

## How It Works

ChronDB leverages Git's internal structure as a storage engine:

```
Repository (database)
  └── Branches (schemas/environments)
       └── Directories (tables)
            └── Files (documents as JSON)
```

Each document is a file in the Git repository, and each change creates a commit in the history. This gives you all the power of Git for your database:

* See diffs between versions
* Browse commit history
* Create branches for development
* Merge changes between branches
* Tag important versions
* Hook into events

## Quick Example

Here's how simple it is to use ChronDB with its Clojure API:

```clojure
;; Create/connect to a database
(def db (chrondb/create-chrondb))

;; Create a user document
(chrondb/save db "user:1" {:name "Alice" :email "alice@example.com"})

;; Update the document
(chrondb/save db "user:1" {:name "Alice" :email "alice@example.com" :role "admin"})

;; Time travel: see how the document looked yesterday
(def yesterday (chrondb/get-at db "user:1" "2023-07-15T00:00:00Z"))

;; See all versions of the document
(def history (chrondb/history db "user:1"))
```

## Choose Your Interface

ChronDB speaks multiple protocols so you can connect with your favorite tools:

* **Native Clojure API** - Direct, powerful access from Clojure applications
* **REST API** - HTTP interface for any language or client
* **Redis Protocol** - Use Redis clients to connect directly
* **PostgreSQL Protocol** - Connect with SQL clients and tools

## Start Your Journey

New to ChronDB? Here's how to get started:

1. [Quick Start Guide](https://github.com/avelino/chrondb/blob/main/docs/quickstart/README.md) - Up and running in minutes
2. [Core Concepts](https://github.com/avelino/chrondb/blob/main/docs/data-model/README.md) - Understand ChronDB's data model
3. [Examples](https://github.com/avelino/chrondb/blob/main/docs/examples/README.md) - See ChronDB in action
4. [API Reference](https://github.com/avelino/chrondb/blob/main/docs/api/README.md) - Complete API documentation

## Ready to dive in?

→ [Get Started Now](https://github.com/avelino/chrondb/blob/main/docs/quickstart/README.md)

## Community Resources

* **Official Documentation**: [chrondb.avelino.run](https://chrondb.avelino.run/)
* **Join our Community**: [Discord](https://discord.com/channels/1099017682487087116/1353399752636497992)
* **Discussions & Ideas**: [GitHub Discussions](https://github.com/avelino/chrondb/discussions)


# Quick Start Guide

This quick start guide will have you up and running with ChronDB in minutes, with clear examples for each supported interface.

## Step 1: Installation

Choose the installation method that works best for you:

### 🐳 Using Docker (Simplest)

```bash
# Start ChronDB with all protocols enabled
docker run -d --name chrondb \
  -p 3000:3000 \  # REST API
  -p 6379:6379 \  # Redis protocol
  -p 5432:5432 \  # PostgreSQL protocol
  -v chrondb-data:/data \
  avelino/chrondb:latest
```

### 🧪 Using JAR File

```bash
# Download the latest release
curl -L -o chrondb.jar https://github.com/avelino/chrondb/releases/latest/download/chrondb.jar

# Run ChronDB
java -jar chrondb.jar
```

### 🔧 For Clojure Developers

```bash
# Clone the repository
git clone https://github.com/avelino/chrondb.git
cd chrondb

# Start the server
clojure -M:run
```

## Step 2: Verify Installation

Let's make sure everything is working correctly:

```bash
# Check the REST API
curl http://localhost:3000/api/v1/health

# Expected response:
# {"status":"ok","version":"0.1.0"}
```

## Step 3: Choose Your Interface

ChronDB offers multiple ways to connect. Let's try each one with a simple example:

### 📡 Using the REST API

The REST API is the most universal interface, accessible from any language with HTTP capabilities.

```bash
# Create your first document
curl -X POST http://localhost:3000/api/v1/documents/greeting:hello \
  -H "Content-Type: application/json" \
  -d '{"message": "Hello, ChronDB!", "created": "2023-06-01T12:00:00Z"}'

# Retrieve the document
curl http://localhost:3000/api/v1/documents/greeting:hello

# Update the document
curl -X POST http://localhost:3000/api/v1/documents/greeting:hello \
  -H "Content-Type: application/json" \
  -d '{"message": "Hello, ChronDB!", "created": "2023-06-01T12:00:00Z", "updated": "2023-06-02T15:30:00Z"}'

# View the document history
curl http://localhost:3000/api/v1/documents/greeting:hello/history
```

### 🔄 Using the Redis Protocol

If you're familiar with Redis, you'll feel right at home:

```bash
# Connect with redis-cli
redis-cli -p 6379

# Create a document
SET user:101 '{"name":"Alex","email":"alex@example.com"}'

# Retrieve the document
GET user:101

# Update the document
SET user:101 '{"name":"Alex","email":"alex@example.com","role":"editor"}'

# View document history (ChronDB-specific command)
CHRONDB.HISTORY user:101
```

### 🗄️ Using the PostgreSQL Protocol

Connect with your favorite SQL tools:

```bash
# Connect with psql
psql -h localhost -p 5432 -U chrondb -d chrondb -W
# Password is 'chrondb' by default

# Create a document as a row
INSERT INTO product (id, name, price, stock)
VALUES ('123', 'Ergonomic Chair', 299.99, 10);

# Query the document
SELECT * FROM product WHERE id = '123';

# Update the document
UPDATE product SET stock = 9, last_sold = '2023-06-15' WHERE id = '123';

# View document history
SELECT * FROM chrondb_history('product', '123');
```

### 🧩 Using the Clojure API

For Clojure applications, add ChronDB as a dependency:

```clojure
;; In deps.edn
{:deps {com.github.avelino/chrondb {:git/tag "v0.1.0"
                                     :git/sha "..."}}}
```

Then use it in your code:

```clojure
(require '[chrondb.core :as chrondb])

;; Create a database connection
(def db (chrondb/create-chrondb))

;; Create a document
(chrondb/save db "order:1001"
  {:customer "Pat Smith"
   :items [{:product "123" :quantity 1}]
   :total 299.99
   :status "pending"})

;; Retrieve the document
(def order (chrondb/get db "order:1001"))

;; Update the document
(chrondb/save db "order:1001"
  (assoc order :status "shipped"))

;; View document history
(def history (chrondb/history db "order:1001"))
```

## Step 4: Explore Time Travel Features

Now let's try ChronDB's most powerful feature - time travel:

### REST API Time Travel

```bash
# Get document at a specific point in time
curl http://localhost:3000/api/v1/documents/greeting:hello/at/2023-06-01T13:00:00Z

# Compare versions
curl http://localhost:3000/api/v1/documents/greeting:hello/diff \
  -d '{"from":"2023-06-01T12:00:00Z","to":"2023-06-02T16:00:00Z"}'
```

### Redis Protocol Time Travel

```bash
# Get document at a specific commit/timestamp
CHRONDB.GETAT user:101 "2023-06-10T15:00:00Z"

# Compare versions
CHRONDB.DIFF user:101 "2023-06-10T15:00:00Z" "2023-06-15T18:00:00Z"
```

### PostgreSQL Protocol Time Travel

```sql
-- Get document at a specific point in time
SELECT * FROM chrondb_at('product', '123', '2023-06-15T10:00:00Z');

-- Compare versions
SELECT * FROM chrondb_diff('product', '123',
  '2023-06-14T00:00:00Z', '2023-06-16T00:00:00Z');
```

### Clojure API Time Travel

```clojure
;; Get document at a specific point in time
(def old-order (chrondb/get-at db "order:1001" "2023-06-10T12:00:00Z"))

;; Compare versions
(def changes (chrondb/diff db "order:1001"
              "2023-06-10T12:00:00Z" "2023-06-11T12:00:00Z"))
```

## Step 5: Try Branching (Optional)

ChronDB's Git foundation enables powerful branching:

### REST API Branching

```bash
# Create a test branch
curl -X POST http://localhost:3000/api/v1/branches/test

# Switch to the test branch
curl -X PUT http://localhost:3000/api/v1/branches/test/checkout

# Make a change in the test branch
curl -X POST http://localhost:3000/api/v1/documents/greeting:hello \
  -H "Content-Type: application/json" \
  -d '{"message": "Hello from test branch!", "created": "2023-06-01T12:00:00Z"}'

# Compare with main branch
curl http://localhost:3000/api/v1/branches/main/documents/greeting:hello
curl http://localhost:3000/api/v1/branches/test/documents/greeting:hello

# Merge test branch to main
curl -X POST http://localhost:3000/api/v1/branches/test/merge/main
```

### Clojure API Branching

```clojure
;; Create a test branch
(def test-db (chrondb/create-branch db "test"))

;; Make changes in the test branch
(chrondb/save test-db "order:1001"
  (assoc (chrondb/get test-db "order:1001")
         :status "canceled"))

;; Compare changes
(println (chrondb/get db "order:1001"))  ;; Main branch
(println (chrondb/get test-db "order:1001"))  ;; Test branch

;; Merge test changes to main
(chrondb/merge-branch db "test" "main")
```

## Next Steps

Congratulations! You've taken your first steps with ChronDB. Here's where to go next:

* [Data Model](https://github.com/avelino/chrondb/blob/main/docs/data-model/README.md) - Learn more about ChronDB's document structure
* [Protocol References](https://github.com/avelino/chrondb/blob/main/docs/protocols/README.md) - Complete protocol documentation
* [Configuration](https://github.com/avelino/chrondb/blob/main/docs/configuration/README.md) - Customize your ChronDB instance
* [Examples](https://github.com/avelino/chrondb/blob/main/docs/examples/README.md) - More detailed usage examples

Need help? Check our [FAQ](https://github.com/avelino/chrondb/blob/main/docs/faq/README.md) or join our community:

* [Discord Community](https://discord.com/channels/1099017682487087116/1353399752636497992)
* [GitHub Discussions](https://github.com/avelino/chrondb/discussions)
* [Official Documentation](https://chrondb.avelino.run/)

**Happy time traveling with ChronDB!**


# Core Concepts

ChronDB uses a document-oriented data model built on top of Git's version control system. This document explains the core concepts of this model and how data is organized, stored, and versioned.

## Core Concepts

ChronDB organizes data using the following hierarchy:

```
Repository
  └── Collections
       └── Documents
            └── Fields
```

### Documents

The primary unit of storage in ChronDB is a **document**. Documents are:

* JSON-compatible objects (maps, arrays, strings, numbers, booleans, and null)
* Identified by a unique key
* Schema-less (fields can vary between documents)
* Automatically versioned

Example document:

```json
{
  "name": "John Doe",
  "email": "john@example.com",
  "age": 30,
  "roles": ["admin", "user"],
  "address": {
    "street": "123 Main St",
    "city": "Anytown",
    "zip": "12345"
  }
}
```

### Document Keys

Each document is identified by a unique key that follows the convention:

```
collection:identifier
```

For example:

* `user:1234`
* `product:macbook-pro`
* `order:ORD-2023-0001`

The part before the colon (`user`, `product`, `order`) implicitly defines the collection, while the part after the colon is the identifier within that collection.

### Collections

Collections are logical groupings of related documents. Collections in ChronDB:

* Are implicitly created when documents are saved with a collection prefix
* Do not require explicit schema definition
* Can contain heterogeneous documents (different field structures)
* Are represented as directories in the underlying Git storage

## Storage Model

### Git-Based Storage

ChronDB maps its document model to Git's storage system:

1. Each document is stored as a separate file in the Git repository
2. The document's path is derived from its key
3. The document's content is stored as JSON
4. Each change to a document creates a Git commit
5. Document history is tracked through Git's commit history

For example, a document with key `user:1234` might be stored as:

```
repository/
  └── user/
       └── 1234.json
```

### Versioning Model

Every change to a document in ChronDB is automatically versioned:

1. When a document is created or updated, a new commit is created
2. Each commit includes:
   * The document's new state
   * A timestamp
   * Optional commit message or metadata
   * Reference to previous versions

This versioning system enables:

* Point-in-time snapshots of any document
* Complete audit trail of changes
* The ability to revert to previous versions
* Branch-based development and testing

## Data Types

ChronDB supports the following data types:

| Type    | JSON Representation | Example                                 |
| ------- | ------------------- | --------------------------------------- |
| String  | String              | `"Hello World"`                         |
| Number  | Number              | `42`, `3.14159`                         |
| Boolean | Boolean             | `true`, `false`                         |
| Array   | JSON Array          | `[1, 2, 3]`, `["red", "green", "blue"]` |
| Object  | JSON Object         | `{"name": "John", "age": 30}`           |
| Null    | null                | `null`                                  |
| Date\*  | String (ISO 8601)   | `"2023-05-15T14:30:00Z"`                |

\* Dates are typically stored as ISO 8601 strings and parsed by client libraries.

## Schema Flexibility

ChronDB is schema-less by default, meaning:

* Documents in the same collection can have different fields
* Fields can be added or removed without affecting other documents
* No up-front schema definition is required

However, applications can implement optional schema validation:

```clojure
;; Example: Register a validation hook
(chrondb/register-hook db :pre-save
  (fn [doc]
    (when (and (= (namespace (:_id doc)) "user")
               (not (:email doc)))
      (throw (ex-info "User documents must have an email" {:doc doc})))
    doc))
```

## Branches and Isolation

ChronDB's Git foundation enables powerful branching capabilities:

* Multiple branches can exist simultaneously
* Each branch can have its own version of the documents
* Changes in one branch do not affect others
* Branches can be merged to combine changes

This enables several workflows:

1. **Development/Testing**: Make changes in a development branch before merging to production
2. **Multi-Tenancy**: Use branches to isolate data between tenants
3. **Scenarios/Simulations**: Create branches to model different scenarios
4. **Feature Flags**: Implement features in separate branches before enabling them

## Querying and Indexes

ChronDB provides two main approaches for retrieving documents:

### Direct Access

* Retrieve documents by their key
* Extremely fast, constant-time operation

```
GET user:1234  # Direct key lookup
```

### Search Queries

* Search across document contents using query expressions
* Based on Lucene's query syntax
* Supports boolean operations, wildcards, ranges, and more

```
SEARCH name:John AND age:[25 TO 35]  # Search with multiple criteria
```

Indexes are maintained for fields commonly used in search queries to improve performance.

## Mapping to Other Protocols

ChronDB's data model is designed to be accessible through multiple protocols with consistent semantics:

### PostgreSQL Protocol

* Collections map to tables
* Document keys map to primary keys
* Document fields map to columns

### Redis Protocol

* Document keys map directly to Redis keys
* Document values are stored as JSON strings
* Special commands provide history and versioning functionality

## Transactions

ChronDB supports multi-document transactions to ensure atomic operations across multiple documents:

```clojure
(with-transaction [db]
  (save db "account:123" {:balance 500})
  (save db "account:456" {:balance 1500})
  (save db "transfer:789" {:from "123", :to "456", :amount 500}))
```

All changes within a transaction are committed together or rolled back if any operation fails.

## Time-Travel Queries

The versioned nature of ChronDB enables powerful time-travel capabilities:

* Retrieve any document as it existed at a specific point in time
* Compare document versions across time
* Query the database as it existed at a specific time

```clojure
;; Get document as it was on January 1, 2023
(get-at db "user:1234" "2023-01-01T00:00:00Z")

;; Compare versions
(diff db "user:1234" "2023-01-01T00:00:00Z" "2023-06-01T00:00:00Z")
```

## Conclusion

ChronDB's document-oriented model built on Git offers a flexible, versioned approach to data storage. By combining the simplicity of document databases with the power of version control, ChronDB provides a unique solution for applications that require data versioning, history tracking, and branching capabilities.


# Frequently Asked Questions

## General Questions

### What is ChronDB?

ChronDB is a chronological database that uses Git's internal architecture to store data with complete version history. It's a key-value database where every change is tracked, allowing you to access any previous version of your data.

### How is ChronDB different from other databases?

Unlike traditional databases that overwrite data when updated, ChronDB preserves every version of your data. The key differences are:

* **Complete history** - Nothing is ever deleted; all changes are preserved
* **Time travel** - Query data as it existed at any point in time
* **Branching** - Create isolated environments for testing or development
* **Multiple interfaces** - Connect through Clojure API, REST, Redis, or PostgreSQL protocols

### Is ChronDB production-ready?

Yes, ChronDB is designed for production use. However, as with any database, consider your specific requirements around performance, scaling, and backup strategies.

### What kinds of applications is ChronDB best suited for?

ChronDB excels in applications where data history and audit trails are important:

* Financial systems requiring complete audit trails
* Healthcare applications with strict data governance
* Collaborative tools where change history matters
* Systems needing point-in-time recovery capabilities
* Applications benefiting from isolated testing environments

## Technical Questions

### Is ChronDB ACID compliant?

Yes, ChronDB provides ACID (Atomicity, Consistency, Isolation, Durability) compliance:

* **Atomicity**: All operations in a transaction either succeed or fail together
* **Consistency**: The database remains in a valid state after each transaction
* **Isolation**: Transactions are isolated from each other
* **Durability**: Once a transaction is committed, it remains saved

### How does ChronDB handle schemas?

ChronDB is schema-less by default. Documents can have any structure, and documents in the same collection can have different fields. However, you can implement schema validation through hooks if needed.

### What's the maximum document size?

Individual documents are stored as files in the underlying Git repository, so practical size limits are similar to those of Git. We recommend keeping documents under 10MB for optimal performance.

### Does ChronDB support indexes?

Yes, ChronDB uses Lucene to provide full-text search capabilities. Fields that are frequently searched can be indexed for better performance.

## Performance Questions

### Is ChronDB as fast as traditional databases?

For most operations, ChronDB performance is competitive with other document databases. However, there are some differences:

* **Read performance**: Reading current data is fast
* **Time travel queries**: Reading historical data may be slower depending on history depth
* **Write performance**: Writes involve creating Git commits, which can be slower than write-optimized databases
* **Search performance**: Indexed searches are optimized but may not match dedicated search engines

### How does ChronDB scale?

ChronDB is primarily designed for vertical scaling (running on more powerful hardware). For horizontal scaling, consider these approaches:

* Separate databases for different domains of your application
* Read replicas for distributing read queries
* Sharding based on document collections

### What's the performance impact of storing all history?

Storing complete history means storage requirements grow over time. The impact on query performance is minimal for current data but can affect performance for deep historical queries.

## Storage and Operations

### How much disk space does ChronDB require?

Storage requirements depend on:

* The size of your data
* How frequently data changes
* How long you retain history

As a rough estimate, expect 2-5x the size of your current data if you have frequent updates and retain all history.

### Can I limit the history retention period?

Yes, ChronDB supports history pruning through Git's garbage collection mechanism. You can configure retention policies to balance storage needs with history requirements.

### How do I back up a ChronDB database?

Since ChronDB uses Git for storage, you can:

1. Use Git's native mechanisms (push to a remote repository)
2. Create point-in-time snapshots with `chrondb/backup`
3. Use filesystem-level backup tools

### Can I run ChronDB in the cloud?

Yes, ChronDB can run in any environment that supports Java and Docker:

* AWS, Google Cloud, or Azure VMs
* Kubernetes clusters
* Docker-based environments like AWS ECS

## Working with ChronDB

### How do I connect to ChronDB?

ChronDB supports multiple connection protocols:

1. **Clojure API**: Direct integration for Clojure applications
2. **REST API**: HTTP interface for any language
3. **Redis Protocol**: Connect using Redis clients
4. **PostgreSQL Protocol**: Connect using SQL clients

### Can I use ChronDB from JavaScript/Python/other languages?

Yes! While the native API is Clojure-based, you can use ChronDB from any language:

* JavaScript/Node.js: Use the REST API or Redis client
* Python: Use the REST API, Redis client, or PostgreSQL client
* Java: Use the REST API or JVM interop with the Clojure API
* Any language: Use the REST API

### How do I query data in ChronDB?

For current data:

* Clojure API: `(chrondb/get db "key")`
* REST API: `GET /api/v1/documents/key`
* Redis Protocol: `GET key`
* PostgreSQL Protocol: `SELECT * FROM collection WHERE id = 'key'`

For historical data:

* Clojure API: `(chrondb/get-at db "key" "2023-01-01T00:00:00Z")`
* REST API: `GET /api/v1/documents/key/at/2023-01-01T00:00:00Z`
* Redis Protocol: `CHRONDB.GETAT key 2023-01-01T00:00:00Z`
* PostgreSQL Protocol: `SELECT * FROM chrondb_at('collection', 'id', '2023-01-01T00:00:00Z')`

### How do I handle schema migrations?

Since ChronDB is schema-less, you don't need formal migrations. Options include:

1. **Progressive Enhancement**: Add new fields without modifying existing documents
2. **Batch Updates**: Run a process to update all documents to a new format
3. **Version Fields**: Include a schema version in documents
4. **Read-Time Transformation**: Transform documents to the latest schema when read

## Troubleshooting

### I can't connect to ChronDB

Check these common issues:

1. Ensure ChronDB is running (`docker ps` if using Docker)
2. Verify port mapping and network settings
3. Check firewall rules
4. Confirm correct protocol and port (3000 for REST, 6379 for Redis, 5432 for PostgreSQL)

### Searches aren't returning expected results

If searches aren't working as expected:

1. Verify your query syntax matches the protocol you're using
2. Check that documents are properly indexed
3. Try simplifying your query to isolate the issue
4. Use exact key lookups to confirm the document exists

### How do I resolve merge conflicts?

When merging branches, conflicts may occur if the same document has been modified differently in both branches. ChronDB provides these options:

1. **Automatic resolution**: ChronDB will attempt to merge changes automatically
2. **Manual resolution**: Specify which version to keep or provide a merged version
3. **Pre-merge checks**: Check for potential conflicts before merging

## Community and Support

### Where can I get help with ChronDB?

Several support options are available:

* [GitHub Issues](https://github.com/avelino/chrondb/issues) for bug reports and feature requests
* [Discord Community](https://discord.com/channels/1099017682487087116/1353399752636497992) for discussions and questions
* [Official Documentation](https://chrondb.avelino.run/) for guides and reference
* [GitHub Discussions](https://github.com/avelino/chrondb/discussions) for longer form conversations and ideas
* [Stack Overflow](https://stackoverflow.com/questions/tagged/chrondb) with the `chrondb` tag

### How can I contribute to ChronDB?

We welcome contributions! Ways to contribute include:

* Reporting bugs and suggesting features
* Improving documentation
* Creating example applications
* Submitting pull requests
* Sharing your ChronDB experience

### Is commercial support available?

Contact the team at [support@example.com](/getting-started/faq) for information about commercial support options, including:

* Implementation consulting
* Custom feature development
* Production support agreements
* Training and workshops


# Time Travel Guide

One of ChronDB's most powerful features is **time travel** - the ability to view, query, and work with your data as it existed at any point in time. This tutorial will guide you through practical examples of how to use time travel in your applications.

## What is Time Travel?

In ChronDB, every change to your data is preserved. Nothing is ever truly deleted or overwritten. This means you can:

* View any document as it existed at a specific point in time
* Compare different versions of a document
* Restore previous versions
* Query your entire database as it existed in the past
* Analyze how your data has evolved

## Time Travel Use Cases

Time travel is invaluable for many scenarios:

* **Audit trails** - See who changed what and when
* **Debugging** - Understand when a data issue was introduced
* **Compliance** - Fulfill regulatory requirements for data history
* **Analytics** - Compare trends over time
* **Undo** - Recover from mistakes or data corruption

## Tutorial Setup

Before starting, make sure you have ChronDB running:

```bash
docker run -d --name chrondb \
  -p 3000:3000 -p 6379:6379 -p 5432:5432 \
  -v chrondb-data:/data \
  avelino/chrondb:latest
```

## Exercise 1: Creating Documents with History

Let's create a document and track its evolution:

### Using the REST API

```bash
# Create initial customer document
curl -X POST http://localhost:3000/api/v1/documents/customer:1001 \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Acme Corp",
    "contact": "John Doe",
    "email": "john@acme.com",
    "plan": "starter",
    "created": "2023-01-01T10:00:00Z"
  }'

# Note the timestamp for reference
TIMESTAMP1=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
echo "First version created at: $TIMESTAMP1"

# Wait a moment, then update the document
sleep 5

# Update the customer plan
curl -X POST http://localhost:3000/api/v1/documents/customer:1001 \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Acme Corp",
    "contact": "John Doe",
    "email": "john@acme.com",
    "plan": "professional",
    "created": "2023-01-01T10:00:00Z",
    "upgraded": "2023-03-15T14:30:00Z"
  }'

# Note the second timestamp
TIMESTAMP2=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
echo "Second version created at: $TIMESTAMP2"

# Wait again, then update once more
sleep 5

# Update contact information
curl -X POST http://localhost:3000/api/v1/documents/customer:1001 \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Acme Corp",
    "contact": "Jane Smith",
    "email": "jane@acme.com",
    "plan": "professional",
    "created": "2023-01-01T10:00:00Z",
    "upgraded": "2023-03-15T14:30:00Z",
    "contact_changed": "2023-05-20T09:15:00Z"
  }'

TIMESTAMP3=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
echo "Third version created at: $TIMESTAMP3"
```

## Exercise 2: Viewing Document History

Now that we have a document with multiple versions, let's explore its history:

### REST API

```bash
# Get the complete document history
curl http://localhost:3000/api/v1/documents/customer:1001/history
```

### Redis Protocol

```bash
# Connect to Redis interface
redis-cli -p 6379

# Get document history
CHRONDB.HISTORY customer:1001
```

### PostgreSQL Protocol

```bash
# Connect to PostgreSQL interface
psql -h localhost -p 5432 -U chrondb -d chrondb -W
# Password: chrondb

# View document history
SELECT * FROM chrondb_history('customer', '1001');
```

### Clojure API

```clojure
(require '[chrondb.core :as chrondb])

(def db (chrondb/create-chrondb))

;; Get document history
(def history (chrondb/history db "customer:1001"))

;; Print each version
(doseq [version history]
  (println "Timestamp:" (:timestamp version))
  (println "Data:" (:data version))
  (println "---"))
```

## Exercise 3: Time-Travel Queries

Now let's retrieve specific versions of our document:

### REST API

```bash
# Get the document as it was at the first timestamp
curl http://localhost:3000/api/v1/documents/customer:1001/at/$TIMESTAMP1

# Get the document as it was after the plan upgrade
curl http://localhost:3000/api/v1/documents/customer:1001/at/$TIMESTAMP2

# Get the current version
curl http://localhost:3000/api/v1/documents/customer:1001
```

### Redis Protocol

```bash
# In redis-cli
CHRONDB.GETAT customer:1001 "$TIMESTAMP1"
CHRONDB.GETAT customer:1001 "$TIMESTAMP2"
GET customer:1001  # Current version
```

### PostgreSQL Protocol

```sql
-- In psql
SELECT * FROM chrondb_at('customer', '1001', '$TIMESTAMP1');
SELECT * FROM chrondb_at('customer', '1001', '$TIMESTAMP2');
SELECT * FROM customer WHERE id = '1001';  -- Current version
```

### Clojure API

```clojure
;; Get specific versions
(def version1 (chrondb/get-at db "customer:1001" "$TIMESTAMP1"))
(def version2 (chrondb/get-at db "customer:1001" "$TIMESTAMP2"))
(def current (chrondb/get db "customer:1001"))

(println "Version 1:" version1)
(println "Version 2:" version2)
(println "Current:" current)
```

## Exercise 4: Comparing Versions

Let's examine what changed between versions:

### REST API

```bash
# Compare the first and second versions
curl -X POST http://localhost:3000/api/v1/documents/customer:1001/diff \
  -H "Content-Type: application/json" \
  -d '{
    "from": "'$TIMESTAMP1'",
    "to": "'$TIMESTAMP2'"
  }'
```

### Redis Protocol

```bash
# In redis-cli
CHRONDB.DIFF customer:1001 "$TIMESTAMP1" "$TIMESTAMP2"
```

### PostgreSQL Protocol

```sql
-- In psql
SELECT * FROM chrondb_diff('customer', '1001', '$TIMESTAMP1', '$TIMESTAMP2');
```

### Clojure API

```clojure
;; Compare versions
(def diff-1-2 (chrondb/diff db "customer:1001" "$TIMESTAMP1" "$TIMESTAMP2"))
(def diff-2-3 (chrondb/diff db "customer:1001" "$TIMESTAMP2" "$TIMESTAMP3"))

(println "Changes from v1 to v2:" diff-1-2)
(println "Changes from v2 to v3:" diff-2-3)
```

## Exercise 5: Reverting to a Previous Version

Sometimes you need to restore data to a previous state:

### Clojure API (most direct way)

```clojure
;; Get the version you want to restore
(def version-to-restore (chrondb/get-at db "customer:1001" "$TIMESTAMP1"))

;; Save it as the current version
(chrondb/save db "customer:1001" version-to-restore)

;; Verify
(println "Restored version:" (chrondb/get db "customer:1001"))
```

### REST API

```bash
# First, get the version you want to restore
old_version=$(curl -s http://localhost:3000/api/v1/documents/customer:1001/at/$TIMESTAMP1)

# Then save it as the current version
curl -X POST http://localhost:3000/api/v1/documents/customer:1001 \
  -H "Content-Type: application/json" \
  -d "$old_version"
```

## Exercise 6: Checking Who Made Changes (Audit Trail)

ChronDB tracks metadata about each change. Let's see who changed what:

### PostgreSQL Protocol (includes committer info)

```sql
-- In psql
SELECT commit_id, timestamp, committer, committer_email
FROM chrondb_history('customer', '1001');
```

### Clojure API with Metadata

```clojure
;; Get history with metadata
(def history-with-meta (chrondb/history db "customer:1001" {:include-metadata true}))

;; Show committer information for each version
(doseq [version history-with-meta]
  (println "Timestamp:" (:timestamp version))
  (println "Committer:" (:committer version))
  (println "Email:" (:committer-email version))
  (println "---"))
```

## Exercise 7: Branching and Time Travel

ChronDB's branching feature works seamlessly with time travel. Let's see how:

### Creating a Test Branch

```clojure
;; Create a test branch
(def test-db (chrondb/create-branch db "test"))

;; Make changes only in the test branch
(chrondb/save test-db "customer:1001"
  (assoc (chrondb/get test-db "customer:1001")
         :plan "enterprise"
         :notes "Considering enterprise upgrade"))

;; Compare the versions in different branches
(println "Main branch:" (chrondb/get db "customer:1001"))
(println "Test branch:" (chrondb/get test-db "customer:1001"))

;; Time travel in the test branch
(def test-history (chrondb/history test-db "customer:1001"))
(println "Test branch history:" test-history)
```

## Real-World Application: Feature Flagging with Time Travel

Let's implement a practical example: using ChronDB's time travel for feature flagging and rollbacks:

```clojure
;; Create a feature flags document
(chrondb/save db "system:feature-flags"
  {:dark-mode-enabled false
   :beta-features-enabled false
   :new-dashboard-enabled false
   :updated "2023-06-01T00:00:00Z"})

;; Record the initial timestamp
(def initial-timestamp "2023-06-01T00:00:00Z")

;; Later, enable some features
(chrondb/save db "system:feature-flags"
  {:dark-mode-enabled true
   :beta-features-enabled true
   :new-dashboard-enabled false
   :updated "2023-06-15T00:00:00Z"})

;; Record this timestamp
(def beta-timestamp "2023-06-15T00:00:00Z")

;; Later, enable all features
(chrondb/save db "system:feature-flags"
  {:dark-mode-enabled true
   :beta-features-enabled true
   :new-dashboard-enabled true
   :updated "2023-07-01T00:00:00Z"})

;; Oh no! The new dashboard has bugs, roll back to beta features only
(def beta-flags (chrondb/get-at db "system:feature-flags" beta-timestamp))
(chrondb/save db "system:feature-flags" beta-flags)

;; Verify we're back to the beta stage
(println "Current flags:" (chrondb/get db "system:feature-flags"))
```

## Conclusion

Time travel is one of ChronDB's most powerful features. By preserving the complete history of your data, ChronDB enables:

* Complete audit trails
* Point-in-time recovery
* Historical analysis
* Safe experimentation with branching
* Compliance with data retention requirements

In this tutorial, you've learned how to:

1. Create and update documents to build history
2. View the complete history of a document
3. Retrieve specific historical versions
4. Compare different versions
5. Revert to previous versions
6. Track who made changes
7. Use branching with time travel
8. Implement practical time travel patterns

For more advanced time travel techniques, check out the [Version Control](https://github.com/avelino/chrondb/blob/main/docs/version-control/README.md) documentation.

## Next Steps

* [Data Model](https://github.com/avelino/chrondb/blob/main/docs/data-model/README.md) - Understand ChronDB's document structure
* [Branching Guide](https://github.com/avelino/chrondb/blob/main/docs/tutorials/branching-guide/README.md) - Learn more about branching
* [Performance Considerations](https://github.com/avelino/chrondb/blob/main/docs/performance/README.md) - Optimize for time travel queries

## Community Resources

* **Documentation**: [chrondb.avelino.run](https://chrondb.avelino.run/)
* **Questions & Help**: [Discord Community](https://discord.com/channels/1099017682487087116/1353399752636497992)
* **Share Your Experience**: [GitHub Discussions](https://github.com/avelino/chrondb/discussions)


# Branching Guide

One of ChronDB's most powerful features is its **branching** capability, inherited from its Git foundation. This guide will teach you how to use branches effectively in your applications.

## What Are Branches?

In ChronDB, branches are isolated environments that contain their own version of the database. Changes in one branch don't affect others until you explicitly merge them. This enables:

* Developing new features without affecting production data
* Testing changes in isolation
* Creating "what-if" scenarios
* Supporting multiple concurrent users with different views
* Providing tenant isolation in multi-tenant applications

## Branching Use Cases

Let's explore practical applications of branching:

### 1. Development Workflows

Create isolated environments for development → testing → production:

```
main (production)
   ↑
  test
   ↑
   dev
```

### 2. Feature Branches

Develop and test each feature in isolation:

```
main
 ↑ ↑ ↑
 │ │ └─ feature-C
 │ └─── feature-B
 └───── feature-A
```

### 3. Tenant Isolation

Give each tenant their own branch for complete isolation:

```
template
 ↑ ↑ ↑
 │ │ └─ tenant-C
 │ └─── tenant-B
 └───── tenant-A
```

### 4. Scenario Analysis

Create "what-if" scenarios for business analysis:

```
current
 ↑ ↑ ↑
 │ │ └─ scenario-high-growth
 │ └─── scenario-medium-growth
 └───── scenario-recession
```

## Getting Started with Branching

Let's walk through some practical examples:

### Setup

Ensure you have ChronDB running:

```bash
docker run -d --name chrondb \
  -p 3000:3000 -p 6379:6379 -p 5432:5432 \
  -v chrondb-data:/data \
  avelino/chrondb:latest
```

## Exercise 1: Creating a Development Branch

Let's create a real-world development workflow with branches:

### Using the Clojure API

```clojure
(require '[chrondb.core :as chrondb])

;; Create a connection to the main branch
(def main-db (chrondb/create-chrondb))

;; Add some production data
(chrondb/save main-db "product:101"
  {:name "Deluxe Widget"
   :price 29.99
   :in_stock 100})

;; Create a development branch
(def dev-db (chrondb/create-branch main-db "dev"))

;; The dev branch starts with all the data from main
(println "Initial dev product:" (chrondb/get dev-db "product:101"))

;; Make changes in the dev branch
(chrondb/save dev-db "product:101"
  {:name "Deluxe Widget Pro"  ;; Name changed
   :price 39.99               ;; Price increased
   :in_stock 100
   :features ["Enhanced UI", "Faster performance"]})  ;; New field

;; Add a new product only in dev
(chrondb/save dev-db "product:102"
  {:name "Super Widget"
   :price 49.99
   :in_stock 50})

;; Compare main and dev branches
(println "Main branch product:" (chrondb/get main-db "product:101"))
(println "Dev branch product:" (chrondb/get dev-db "product:101"))

;; In main, product:102 doesn't exist yet
(println "Product:102 in main:" (chrondb/get main-db "product:102"))
(println "Product:102 in dev:" (chrondb/get dev-db "product:102"))
```

### Using the REST API

```bash
# Create a dev branch
curl -X POST http://localhost:3000/api/v1/branches/dev \
  -H "Content-Type: application/json" \
  -d '{"source": "main"}'

# Add a product to main
curl -X POST http://localhost:3000/api/v1/documents/product:201 \
  -H "Content-Type: application/json" \
  -d '{"name": "Economy Widget", "price": 19.99, "in_stock": 200}'

# Update the product in dev branch
curl -X POST http://localhost:3000/api/v1/branches/dev/documents/product:201 \
  -H "Content-Type: application/json" \
  -d '{"name": "Economy Widget Plus", "price": 24.99, "in_stock": 200, "features": ["Value option"]}'

# Compare the versions
curl http://localhost:3000/api/v1/documents/product:201
curl http://localhost:3000/api/v1/branches/dev/documents/product:201
```

## Exercise 2: Testing with Branches

Let's simulate a testing workflow:

```clojure
;; Create a test branch from dev
(def test-db (chrondb/create-branch dev-db "test"))

;; QA discovers an issue and fixes the price in test
(chrondb/save test-db "product:101"
  (-> (chrondb/get test-db "product:101")
      (assoc :price 34.99)  ;; Price adjusted after review
      (assoc :features ["Enhanced UI", "Faster performance", "Bug fixes"])))

;; All three branches now have different versions
(println "Main price:" (:price (chrondb/get main-db "product:101")))  ;; 29.99
(println "Dev price:" (:price (chrondb/get dev-db "product:101")))    ;; 39.99
(println "Test price:" (:price (chrondb/get test-db "product:101")))  ;; 34.99
```

## Exercise 3: Merging Branches

After testing, let's merge our changes to the main branch:

### Using Clojure API

```clojure
;; First, merge test changes back to dev
(chrondb/merge-branch dev-db "test" "dev")

;; Verify the test fixes are now in dev
(println "Dev price after merge:" (:price (chrondb/get dev-db "product:101")))  ;; Should be 34.99

;; Now, merge dev to main for release
(chrondb/merge-branch main-db "dev" "main")

;; Verify the changes are in main
(println "Main after release:")
(println "Product 101:" (chrondb/get main-db "product:101"))
(println "Product 102:" (chrondb/get main-db "product:102"))
```

### Using REST API

```bash
# Merge test branch into dev
curl -X POST http://localhost:3000/api/v1/branches/test/merge/dev

# Merge dev branch into main (production release)
curl -X POST http://localhost:3000/api/v1/branches/dev/merge/main

# Verify the results
curl http://localhost:3000/api/v1/documents/product:201
```

## Exercise 4: Working with Multiple Users

Branching can isolate changes between different users:

```clojure
;; Create user-specific branches
(def alice-db (chrondb/create-branch main-db "user-alice"))
(def bob-db (chrondb/create-branch main-db "user-bob"))

;; Each user makes their own changes
(chrondb/save alice-db "shared-doc:1" {:title "Project Plan" :content "Alice's version"})
(chrondb/save bob-db "shared-doc:1" {:title "Project Plan" :content "Bob's version"})

;; Each user sees their own changes
(println "Alice sees:" (chrondb/get alice-db "shared-doc:1"))
(println "Bob sees:" (chrondb/get bob-db "shared-doc:1"))

;; Later, alice's changes are chosen as authoritative
(chrondb/merge-branch main-db "user-alice" "main")
(println "Final version:" (chrondb/get main-db "shared-doc:1"))
```

## Exercise 5: "What-If" Analysis for Business Scenarios

```clojure
;; Current financial data
(chrondb/save main-db "finance:2023-forecast"
  {:revenue 1000000
   :expenses 750000
   :growth_rate 0.05})

;; Create scenario branches
(def optimistic-db (chrondb/create-branch main-db "scenario-optimistic"))
(def pessimistic-db (chrondb/create-branch main-db "scenario-pessimistic"))

;; Model different scenarios
(chrondb/save optimistic-db "finance:2023-forecast"
  {:revenue 1200000
   :expenses 800000
   :growth_rate 0.15})

(chrondb/save pessimistic-db "finance:2023-forecast"
  {:revenue 800000
   :expenses 700000
   :growth_rate -0.05})

;; Compare scenarios
(def current (chrondb/get main-db "finance:2023-forecast"))
(def optimistic (chrondb/get optimistic-db "finance:2023-forecast"))
(def pessimistic (chrondb/get pessimistic-db "finance:2023-forecast"))

(println "Current profit:" (- (:revenue current) (:expenses current)))
(println "Optimistic profit:" (- (:revenue optimistic) (:expenses optimistic)))
(println "Pessimistic profit:" (- (:revenue pessimistic) (:expenses pessimistic)))
```

## Advanced Branching Patterns

### Multi-Tenant Architecture

In a multi-tenant SaaS application:

```clojure
;; Create a template with base structure
(def template-db (chrondb/create-chrondb))
(chrondb/save template-db "system:settings" {:default_theme "light", :features {:basic true, :advanced false}})

;; Create tenant-specific branches
(def tenant1-db (chrondb/create-branch template-db "tenant-acme"))
(def tenant2-db (chrondb/create-branch template-db "tenant-globex"))

;; Customize per tenant
(chrondb/save tenant1-db "system:settings"
  (-> (chrondb/get tenant1-db "system:settings")
      (assoc :company_name "Acme Inc")
      (assoc-in [:features :advanced] true)))

(chrondb/save tenant2-db "system:settings"
  (-> (chrondb/get tenant2-db "system:settings")
      (assoc :company_name "Globex Corp")
      (assoc :custom_domain "globex.example.com")))

;; Each tenant gets their own isolated environment
(println "Tenant 1 settings:" (chrondb/get tenant1-db "system:settings"))
(println "Tenant 2 settings:" (chrondb/get tenant2-db "system:settings"))

;; Update the template (affects new tenants but not existing ones)
(chrondb/save template-db "system:settings"
  (-> (chrondb/get template-db "system:settings")
      (assoc :default_theme "dark")))
```

### Feature Flagging with Branches

Test new features in isolation before enabling for all users:

```clojure
;; Main production branch
(def main-db (chrondb/create-chrondb))

;; Branch for testing new UI
(def new-ui-db (chrondb/create-branch main-db "feature-new-ui"))

;; Add some users to both branches
(chrondb/save main-db "user:1" {:name "Alice", :email "alice@example.com"})
(chrondb/save main-db "user:2" {:name "Bob", :email "bob@example.com"})

;; In the feature branch, enable the new UI flag
(chrondb/save new-ui-db "system:flags" {:new_ui_enabled true})

;; Function to decide which branch to use based on user
(defn get-user-db [user-id]
  (if (= user-id "1")
    ;; Alice gets the new UI (beta tester)
    new-ui-db
    ;; Everyone else gets the main branch
    main-db))

;; Simulate serving requests
(defn handle-request [user-id]
  (let [db (get-user-db user-id)
        user (chrondb/get db (str "user:" user-id))
        flags (chrondb/get db "system:flags")]
    {:user user
     :ui_version (if (:new_ui_enabled flags) "new" "classic")}))

(println "Alice's experience:" (handle-request "1"))
(println "Bob's experience:" (handle-request "2"))

;; When ready to launch for everyone, merge the feature branch
(chrondb/merge-branch main-db "feature-new-ui" "main")
```

## Best Practices for Branching

1. **Keep a Clean Main Branch**
   * Main should always contain stable, production-ready data
   * Only merge thoroughly tested changes
2. **Use Descriptive Branch Names**
   * `feature-new-ui`, `bugfix-pricing`, `tenant-acme`
   * Include tracking numbers: `feature-123-new-login`
3. **Regular Merges**
   * Regularly merge from main into development branches to prevent divergence
   * The longer branches exist separately, the harder merging becomes
4. **Delete Stale Branches**
   * Remove branches after merging to keep your branch list clean
   * Consider archiving important historical branches
5. **Automate Branch Management**
   * Use CI/CD pipelines to automate testing and merging
   * Integrate with your development workflow

## Conclusion

Branching in ChronDB offers powerful capabilities for development workflows, testing, and multi-tenant applications. By understanding how to create, modify, and merge branches, you can leverage ChronDB's full potential to create isolated environments while maintaining the ability to consolidate changes when needed.

## Next Steps

* [Time Travel Guide](https://github.com/avelino/chrondb/blob/main/docs/tutorials/time-travel-guide/README.md) - Learn how to access historical versions
* [Multi-Tenant Patterns](https://github.com/avelino/chrondb/blob/main/docs/tutorials/multi-tenant-patterns/README.md) - Advanced patterns for SaaS applications
* [Performance Considerations](https://github.com/avelino/chrondb/blob/main/docs/performance/README.md) - Optimize for branching operations

## Community Resources

* **Documentation**: [chrondb.avelino.run](https://chrondb.avelino.run/)
* **Questions & Help**: [Discord Community](https://discord.com/channels/1099017682487087116/1353399752636497992)
* **Share Your Experience**: [GitHub Discussions](https://github.com/avelino/chrondb/discussions)


# Schema Validation

ChronDB is schemaless by default, but supports **optional JSON Schema validation** per namespace (table). This allows you to enforce data integrity where needed while keeping the flexibility of schemaless storage elsewhere.

## Overview

Validation schemas are:

* **Optional** - Only namespaces with defined schemas are validated
* **Per-namespace** - Each table/namespace can have its own schema
* **Version-controlled** - Schema changes are tracked in Git history
* **Mode-configurable** - Choose between strict rejection or warning-only logging

## Validation Modes

| Mode       | Behavior                                                           |
| ---------- | ------------------------------------------------------------------ |
| `strict`   | Rejects invalid documents with an error                            |
| `warning`  | Logs violations but allows the write (ideal for gradual migration) |
| `disabled` | No validation (default behavior)                                   |

## Creating a Validation Schema

### Via REST API

```bash
# Create schema for the "users" namespace
curl -X PUT http://localhost:3000/api/v1/schemas/validation/users \
  -H "Content-Type: application/json" \
  -d '{
    "mode": "strict",
    "schema": {
      "$schema": "http://json-schema.org/draft-07/schema#",
      "type": "object",
      "required": ["id", "email"],
      "properties": {
        "id": { "type": "string" },
        "email": { "type": "string", "format": "email" },
        "name": { "type": "string" },
        "age": { "type": "integer", "minimum": 0 }
      },
      "additionalProperties": false
    }
  }'
```

### Via Redis Protocol

```redis
SCHEMA.SET users '{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "required": ["id", "email"],
  "properties": {
    "id": { "type": "string" },
    "email": { "type": "string", "format": "email" }
  }
}' MODE strict
```

### Via SQL (PostgreSQL Protocol)

```sql
CREATE VALIDATION SCHEMA FOR users AS '{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "required": ["id", "email"],
  "properties": {
    "id": { "type": "string" },
    "email": { "type": "string", "format": "email" }
  }
}' MODE STRICT;
```

## Querying Schemas

### List All Schemas

**REST:**

```bash
curl http://localhost:3000/api/v1/schemas/validation
```

**Redis:**

```redis
SCHEMA.LIST
```

**SQL:**

```sql
SHOW VALIDATION SCHEMAS;
```

### Get Specific Schema

**REST:**

```bash
curl http://localhost:3000/api/v1/schemas/validation/users
```

**Redis:**

```redis
SCHEMA.GET users
```

**SQL:**

```sql
SHOW VALIDATION SCHEMA FOR users;
```

### View Schema History

**REST:**

```bash
curl http://localhost:3000/api/v1/schemas/validation/users/history
```

## Dry-Run Validation

Test if a document is valid before saving:

**REST:**

```bash
curl -X POST http://localhost:3000/api/v1/schemas/validation/users/validate \
  -H "Content-Type: application/json" \
  -d '{"id": "users:1", "name": "John"}'
```

Response (invalid document):

```json
{
  "valid": false,
  "errors": [
    {
      "path": "$.email",
      "message": "required property 'email' not found",
      "keyword": "required"
    }
  ]
}
```

**Redis:**

```redis
SCHEMA.VALIDATE users '{"id": "users:1", "name": "John"}'
```

## Updating a Schema

Simply send a new schema with PUT - version history is automatically maintained:

```bash
curl -X PUT http://localhost:3000/api/v1/schemas/validation/users \
  -H "Content-Type: application/json" \
  -d '{
    "mode": "strict",
    "schema": {
      "type": "object",
      "required": ["id", "email", "name"],
      "properties": { ... }
    }
  }'
```

## Changing the Mode

```bash
# Change from strict to warning
curl -X PUT http://localhost:3000/api/v1/schemas/validation/users \
  -H "Content-Type: application/json" \
  -d '{"mode": "warning", "schema": { ... }}'
```

## Removing Validation

**REST:**

```bash
curl -X DELETE http://localhost:3000/api/v1/schemas/validation/users
```

**Redis:**

```redis
SCHEMA.DEL users
```

**SQL:**

```sql
DROP VALIDATION SCHEMA FOR users;
```

## Validation Errors

When `mode: strict` and a document is invalid:

### REST API (HTTP 400)

```json
{
  "error": "VALIDATION_ERROR",
  "namespace": "users",
  "document_id": "users:1",
  "mode": "strict",
  "violations": [
    {"path": "$.email", "message": "required property 'email' not found", "keyword": "required"},
    {"path": "$.age", "message": "must be >= 0", "keyword": "minimum"}
  ]
}
```

### Redis Protocol

```
-ERR VALIDATION_ERROR users: required property 'email' not found at $.email
```

### PostgreSQL Protocol

```
ERROR: Validation failed for table 'users': $.email - required property 'email' not found
```

## Gradual Migration Strategy

When adding validation to existing data:

1. **Create schema in warning mode:**

   ```bash
   curl -X PUT .../schemas/validation/users -d '{"mode": "warning", "schema": {...}}'
   ```
2. **Monitor logs** to identify existing invalid documents
3. **Fix existing data** that violates the schema
4. **Switch to strict mode:**

   ```bash
   curl -X PUT .../schemas/validation/users -d '{"mode": "strict", "schema": {...}}'
   ```

## JSON Schema Reference

ChronDB supports JSON Schema Draft-07, 2019-09, and 2020-12. Common patterns:

```json
{
  "type": "object",
  "required": ["field1", "field2"],
  "properties": {
    "string_field": { "type": "string", "minLength": 1, "maxLength": 100 },
    "email_field": { "type": "string", "format": "email" },
    "number_field": { "type": "number", "minimum": 0, "maximum": 100 },
    "integer_field": { "type": "integer" },
    "boolean_field": { "type": "boolean" },
    "array_field": { "type": "array", "items": { "type": "string" } },
    "enum_field": { "enum": ["value1", "value2", "value3"] },
    "nullable_field": { "type": ["string", "null"] }
  },
  "additionalProperties": false
}
```

### Nested Objects

```json
{
  "type": "object",
  "properties": {
    "address": {
      "type": "object",
      "required": ["city"],
      "properties": {
        "street": { "type": "string" },
        "city": { "type": "string" },
        "zip": { "type": "string", "pattern": "^[0-9]{5}$" }
      }
    }
  }
}
```

### Array Validation

```json
{
  "type": "object",
  "properties": {
    "tags": {
      "type": "array",
      "items": { "type": "string" },
      "minItems": 1,
      "uniqueItems": true
    }
  }
}
```

For complete JSON Schema documentation, see: <https://json-schema.org/>

## Storage Details

Validation schemas are stored in Git at:

```
_schema/validation/{namespace}.json
```

Each schema file contains:

```json
{
  "namespace": "users",
  "version": 1,
  "mode": "strict",
  "schema": { ... },
  "created_at": "2024-01-15T10:00:00Z",
  "created_by": "admin"
}
```

Schema changes are versioned and can be time-traveled like any other data in ChronDB.


# Version Control Features

ChronDB uniquely combines a document database with Git's powerful version control system, providing capabilities that traditional databases lack. This document explores the version control features of ChronDB and how they can be leveraged in your applications.

## Version Control Overview

Unlike traditional databases that only store the current state of data, ChronDB automatically tracks the complete history of every document:

* **All changes are versioned**: Every document update creates a new revision
* **Full history preservation**: Historical states are never lost
* **Git foundation**: Built on Git's proven version control infrastructure
* **First-class version management**: Version control is a core feature, not an add-on

## Document History and Revisions

### Automatic Change Tracking

Every time a document is created, modified, or deleted, ChronDB:

1. Creates a new version of the document
2. Records metadata about the change (timestamp, author)
3. Preserves the document's previous state
4. Links the revisions in a chronological history

```clojure
;; Create initial document
(chrondb/save db "user:1" {:name "John" :email "john@example.com"})

;; Update the document
(chrondb/save db "user:1" {:name "John" :email "john.doe@example.com"})

;; Each operation creates a new version in the history
```

### Accessing Document History

ChronDB provides API methods to access the complete history of any document:

```clojure
;; Get the complete history of a document
(def history (chrondb/history db "user:1"))

;; Example history result
;; [
;;   {:timestamp "2023-05-10T14:30:00Z",
;;    :data {:name "John", :email "john@example.com"}}
;;   {:timestamp "2023-05-15T09:45:00Z",
;;    :data {:name "John", :email "john.doe@example.com"}}
;; ]
```

ChronDB also provides low-level access to Git-based document history, allowing you to retrieve detailed commit information and document content at each version:

```clojure
;; Get detailed document history with Git commit metadata
(def git-history (chrondb/get-document-history db "user:1"))

;; Example git-history result
;; [
;;   {:commit-id "8f7e6d5c4b3a2...",
;;    :commit-time #inst "2023-05-15T09:45:00Z",
;;    :commit-message "Save document",
;;    :committer-name "ChronDB",
;;    :committer-email "system@chrondb.com",
;;    :document {:name "John", :email "john.doe@example.com"}}
;;   {:commit-id "1a2b3c4d5e6f7...",
;;    :commit-time #inst "2023-05-10T14:30:00Z",
;;    :commit-message "Save document",
;;    :committer-name "ChronDB",
;;    :committer-email "system@chrondb.com",
;;    :document {:name "John", :email "john@example.com"}}
;; ]
```

### Retrieving Specific Document Versions

You can retrieve a document at a specific commit by providing the commit hash. Note that each retrieval operation generates a new commit to maintain chronological integrity:

```clojure
;; Get a document at a specific commit
(def old-version (chrondb/get-document-at-commit db-repository "user:1" "1a2b3c4d5e6f7..."))

;; Example result
;; {:name "John", :email "john@example.com"}
```

Through other protocols, document history is also accessible:

**REST API:**

```
GET /api/v1/documents/user:1/history
```

**PostgreSQL Protocol:**

```sql
SELECT * FROM chrondb_history('user', '1');
```

**Redis Protocol:**

```
CHRONDB.HISTORY user:1
```

### Point-in-Time Access

A key feature of ChronDB is the ability to access any document as it existed at a specific point in time:

```clojure
;; Get document as it was on May 10, 2023
(def old-version (chrondb/get-at db "user:1" "2023-05-10T14:30:00Z"))
```

This enables:

* Historical reporting
* Audit trails
* Compliance requirements
* Debugging and troubleshooting
* "Time machine" capabilities for applications

## Branching

ChronDB extends Git's branching model to document databases, allowing multiple parallel versions of your data to exist simultaneously.

### Creating and Using Branches

```clojure
;; Create a new branch
(def test-db (chrondb/create-branch db "test"))

;; Make changes in the test branch
(chrondb/save test-db "user:1" {:name "John (Test)" :email "john.test@example.com"})

;; Changes in test branch don't affect main branch
(println (chrondb/get db "user:1"))        ;; Original version
(println (chrondb/get test-db "user:1"))   ;; Modified version

;; Switch the current branch
(def current-db (chrondb/switch-branch db "test"))
```

### Branch Use Cases

Branches enable powerful workflows:

1. **Development and Testing**
   * Use branches to develop and test new features without affecting production data
   * Validate data migrations before applying to production
2. **What-If Analysis**
   * Create branches to model different business scenarios
   * Run simulations with alternate data sets
3. **Multi-Tenancy**
   * Use branches to isolate data for different tenants
   * Apply schema changes to specific tenants
4. **Feature Toggles**
   * Implement unreleased features in separate branches
   * Merge when ready for production
5. **Temporary Workspaces**
   * Create temporary branches for exploratory analysis
   * Discard when no longer needed

### Branch Management

ChronDB provides APIs for managing branches:

```clojure
;; List all branches
(chrondb/list-branches db)  ;; => ["main", "test", "dev"]

;; Check current branch
(chrondb/current-branch db)  ;; => "main"

;; Delete a branch
(chrondb/delete-branch db "test")
```

## Merging

ChronDB allows changes from one branch to be incorporated into another through merging.

### Merge Operations

```clojure
;; Merge changes from test branch into main
(def merged-db (chrondb/merge-branch db "test" "main"))
```

### Merge Strategies

ChronDB supports different merge strategies:

1. **Fast-forward merge**
   * Applied when the target branch hasn't changed since the source branch was created
   * Results in a linear history
2. **Recursive merge**
   * Used when both branches have diverged
   * Combines changes from both branches
3. **Ours strategy**
   * Prioritizes the target branch in conflict resolution
   * Useful when you want to override changes from the source branch
4. **Theirs strategy**
   * Prioritizes the source branch in conflict resolution
   * Useful when you want to adopt all changes from the source branch

```clojure
;; Merge with a specific strategy
(chrondb/merge-branch db "test" "main" {:strategy :ours})
```

## Conflict Resolution

When changes in different branches conflict, ChronDB provides tools for resolution:

### Conflict Detection

ChronDB automatically detects conflicts during merge operations:

```clojure
;; Attempt merge that may have conflicts
(try
  (chrondb/merge-branch db "branch1" "branch2")
  (catch Exception e
    (if (= (:type (ex-data e)) :merge-conflict)
      (handle-conflict (:conflicts (ex-data e)))
      (throw e))))
```

### Conflict Resolution Strategies

1. **Manual Resolution**
   * Identify conflicting fields
   * Choose the correct value for each conflict
   * Create a resolution commit
2. **Automatic Policy-Based Resolution**
   * Define policies for automatic conflict resolution
   * Examples: "newest wins", "field-specific rules"

```clojure
;; Register a conflict resolution policy
(chrondb/register-conflict-resolver db
  (fn [doc1 doc2]
    ;; Implement your resolution logic
    (merge doc1 doc2)))
```

## Comparing and Diffing

ChronDB provides powerful tools for comparing document versions:

```clojure
;; Compare two versions of a document
(def diff (chrondb/diff db "user:1"
                       "2023-05-10T14:30:00Z"
                       "2023-05-15T09:45:00Z"))

;; Example diff result
;; {
;;   :added {}
;;   :removed {}
;;   :changed {:email ["john@example.com" "john.doe@example.com"]}
;; }
```

This enables:

* Visualizing changes over time
* Understanding data evolution
* Debugging unexpected changes
* Audit and compliance reporting

## Tagging and Snapshots

ChronDB allows you to tag specific points in your database's history:

```clojure
;; Create a tag/snapshot
(chrondb/create-tag db "release-2023-q2")

;; List all tags
(chrondb/list-tags db)  ;; => ["release-2023-q1", "release-2023-q2"]

;; Get database state at a specific tag
(def q2-db (chrondb/checkout-tag db "release-2023-q2"))
```

Use cases for tagging:

* Mark significant application releases
* Create quarterly snapshots for reporting
* Tag before major data migrations
* Create recovery points

## Auditing and Compliance

ChronDB's version control features naturally support auditing and compliance requirements:

### Audit Trail

Every change in ChronDB is tracked with:

* What changed (the data diff)
* When it changed (timestamp)
* Who changed it (committer information)
* Why it changed (commit message)

```clojure
;; Save with audit information
(chrondb/save db "user:1"
             {:name "John", :email "john@example.com"}
             {:message "Updated email address"
              :author "admin@example.com"})
```

### Compliance Features

ChronDB helps meet regulatory requirements:

* **GDPR**: Track all changes to personal data
* **HIPAA**: Maintain comprehensive audit trails
* **SOX**: Ensure financial data integrity and history
* **21 CFR Part 11**: Support for electronic records in regulated industries

## Rollback and Reversion

ChronDB makes it easy to revert to previous states:

```clojure
;; Revert a document to a previous state
(chrondb/revert db "user:1" "2023-05-10T14:30:00Z")

;; Revert an entire collection
(chrondb/revert-collection db "user" "2023-05-10T14:30:00Z")

;; Revert the entire database
(chrondb/revert-all db "2023-05-10T14:30:00Z")
```

For git-based storage, you can also restore a document to a specific version by commit hash while preserving the complete history:

```clojure
;; Restore a document to a specific version by commit hash
(def restored-doc (chrondb/restore-document-version db "user:1" "1a2b3c4d5e6f7..."))

;; This creates a new commit that reverts the document to that version,
;; while preserving the complete history of changes
```

Unlike a traditional rollback that might discard history, this approach adds a new restoration commit that preserves the complete chronology of changes, ensuring full audit trails are maintained.

This enables:

* Recovering from errors
* Undoing problematic changes
* Testing with real data then reverting
* Maintaining a complete audit trail of all operations

## Performance Considerations

Version control adds powerful capabilities but requires understanding some performance implications:

1. **Storage Growth**
   * Historical versions increase storage requirements
   * Git's compression and deduplication help minimize impact
2. **Operation Overhead**
   * Each write includes versioning overhead
   * Reading the latest version remains efficient
   * Historical queries have additional cost
3. **Optimization Strategies**
   * Use `git gc` for repository optimization
   * Consider repository sharding for very large datasets
   * Use appropriate indexing for common queries

## Best Practices

### Commit Messages and Metadata

Add meaningful context to changes:

```clojure
(chrondb/save db "order:1234"
             updated-order
             {:message "Fixed incorrect shipping address"
              :author "customer-service@example.com"})
```

### Branch Organization

Establish branch naming conventions:

* `feature/...` for new features
* `fix/...` for bug fixes
* `customer/...` for customer-specific branches

### Backup and Migration

Even though ChronDB preserves history, regular backups are recommended:

```clojure
;; Create a backup
(chrondb/backup db "/path/to/backup/location")
```

## Conclusion

ChronDB's version control features provide unprecedented capabilities for tracking, managing, and leveraging data history. By combining the power of Git with a document database, ChronDB enables new workflows and solutions to problems that traditional databases can't easily address.

Whether you need audit trails for compliance, branches for development and testing, or time-travel for analysis, ChronDB's version control features offer a robust foundation for building temporally-aware applications.


# Architecture

## Git as a Database

Git is traditionally known as a version control system for code, but its internal architecture presents characteristics that make it suitable for chronological data storage.

### Git Internal Structure

Git stores data in four main types of objects:

1. **Blobs**: Store file contents
2. **Trees**: Represent directories and contain references to blobs and other trees
3. **Commits**: Capture the state of the repository at a specific point in time
4. **Tags**: Point to specific commits with friendly names

This structure creates a content-addressed database, where each object is identified by the SHA-1 hash of its content.

### Alignment with Database Concepts

In ChronDB, these concepts are mapped to database terminology:

* **Git Repository** → Database
* **Git Branch** → Schema
* **Directory** → Table/Collection
* **File** → Document/Record
* **Commit** → Transaction
* **Commit Hash** → Transaction ID
* **Tag** → Named Snapshot

## ChronDB Architecture

ChronDB is built in layers:

1. **Storage Layer**: Uses JGit to interact with Git's internal structure
2. **Indexing Layer**: Delegates to Apache Lucene for fast document search, supporting configurable secondary and composite indexes, geospatial shapes, and a planner-driven execution engine with result caching
3. **Access Layer**: Provides multiple interfaces (Clojure API, REST, Redis, PostgreSQL)
4. **Concurrency Layer**: Manages concurrent transactions and conflicts

### Data Flow

1. Write operations are converted to Git operations
2. Documents are serialized as JSON and stored as files
3. Each transaction results in a Git commit
4. Indices are updated to reflect changes
5. Reads can access any point in time using specific commits

### Transaction Metadata via Git Notes

Every commit recorded by ChronDB receives a Git note under the `chrondb` ref. The note payload is JSON and contains:

* A transaction identifier (`tx_id`) shared by all commits that belong to the same logical operation
* The origin (for example `rest`, `redis`, `sql`, `cli`)
* Optional user information and request correlation ids
* Semantic flags such as `bulk-load`, `rollback`, `migration`, or `automated-merge`
* Additional metadata supplied by the protocol handler (HTTP endpoint, Redis command, SQL table, etc.)

These notes provide an append-only audit trail without mutating commit messages or tracked files. Operators can inspect them using standard tooling:

```
git log --show-notes=chrondb
git notes --ref=chrondb show <commit>
```

Because they live outside the object graph, notes can be replicated, filtered, and queried independently of the document contents while preserving Git’s immutable history.

### Indexing Layer Details

The Lucene layer receives document mutations from the storage layer and updates the appropriate secondary indexes. It maintains statistics about term distributions and query plans so that complex requests—such as multi-field boolean filters or temporal slices—can be executed without scanning entire collections. When a query arrives, the planner determines the optimal combination of indexes, warms the cache when necessary, and streams results back to the access layer. Geospatial fields are stored in BKD trees, while full-text fields use analyzers that can be tuned per collection.

## Architecture Benefits

* **Immutability**: Data is never overwritten, only added
* **Traceability**: Complete history of changes
* **Reliability**: Leveraging Git's proven robustness
* **Flexibility**: Support for multiple protocols and interfaces


# Indexing and Search

ChronDB uses [Apache Lucene 9.8](https://lucene.apache.org/) for full-text and structured search. Every document saved through any protocol (REST, Redis, SQL, FFI) is automatically indexed. This guide covers how indexing works, available query types, and performance tuning.

## How Indexing Works

### Write Path

When a document is saved:

1. The document is stored in Git (immutable commit)
2. The document is indexed in Lucene with all fields
3. The Lucene NRT (Near-Real-Time) reader is refreshed

Reads see indexed documents immediately after the write returns.

### Field Type Detection

Lucene needs to know field types for storage and querying. ChronDB automatically detects types from document values:

| JSON Value Type | Lucene Field Type                       | Indexed As                         |
| --------------- | --------------------------------------- | ---------------------------------- |
| String          | `StringField` + `TextField`             | Exact match + full-text            |
| Integer/Long    | `LongPoint` + `NumericDocValuesField`   | Range queries + sorting            |
| Float/Double    | `DoublePoint` + `NumericDocValuesField` | Range queries + sorting            |
| Boolean         | `StringField`                           | Exact match (`"true"` / `"false"`) |
| Nested Object   | Flattened with dot notation             | `address.city` becomes a field     |

Every field is also stored as a `StoredField` so the original document can be retrieved from the index.

### Full-Text Analysis

Text fields are indexed twice:

* **Exact match**: stored as-is for term queries and filters
* **Analyzed**: processed through Lucene's `StandardAnalyzer` for full-text search (tokenized, lowercased, stop words removed)

Full-text search fields use the `_fts` suffix internally (e.g., `content` becomes `content_fts`).

***

## Query Types

ChronDB supports several query types through the AST query system. All query types are available across REST, Redis, and SQL protocols.

### Term Query

Exact match on a field value:

```clojure
;; Clojure AST
(ast/term "status" "active")
```

```sql
-- SQL equivalent
SELECT * FROM user WHERE status = 'active';
```

```bash
# REST search
curl "http://localhost:3000/api/v1/search?q=status:active"
```

### Wildcard Query

Pattern matching with `*` (multiple characters) and `?` (single character):

```clojure
(ast/wildcard "email" "*@example.com")
```

```sql
SELECT * FROM user WHERE email LIKE '%@example.com';
```

### Range Query

Numeric and string range comparisons:

```clojure
;; Numeric range
(ast/range "age" {:gte 18 :lt 65})

;; Open-ended range
(ast/range "salary" {:gte 50000})
```

```sql
SELECT * FROM employee WHERE age >= 18 AND age < 65;
SELECT * FROM employee WHERE salary >= 50000;
```

### Full-Text Search (FTS)

Search analyzed text fields using natural language:

```clojure
(ast/fts "content" "distributed database chronological")
```

```sql
SELECT * FROM articles WHERE FTS_MATCH(content, 'distributed database chronological');
```

FTS queries are processed through Lucene's QueryParser, which supports:

* Boolean operators: `AND`, `OR`, `NOT`
* Phrase search: `"exact phrase"`
* Field targeting: `title:database`
* Grouping: `(database OR store) AND distributed`

### Prefix Query

Match documents where a field starts with a given string:

```clojure
(ast/prefix "name" "Joh")
```

### Exists / Missing

Check for field presence or absence:

```clojure
(ast/exists "email")      ;; documents that have an email field
(ast/missing "deleted_at") ;; documents without a deleted_at field
```

```sql
SELECT * FROM user WHERE email IS NOT NULL;
SELECT * FROM user WHERE deleted_at IS NULL;
```

### Boolean Combinations

Combine queries with boolean logic:

```clojure
(ast/bool
  {:must [(ast/term "status" "active")
          (ast/range "age" {:gte 18})]
   :should [(ast/fts "bio" "engineer")]
   :must-not [(ast/term "role" "banned")]})
```

| Clause     | Behavior                                          |
| ---------- | ------------------------------------------------- |
| `must`     | All conditions must match (AND)                   |
| `should`   | At least one should match (OR)                    |
| `must-not` | None of these must match (NOT)                    |
| `filter`   | Like `must` but does not affect relevance scoring |

***

## AST Query System

For a comprehensive reference on building structured queries, including pagination, sorting, and cursor-based navigation, see [AST Queries](https://github.com/avelino/chrondb/blob/main/docs/ast-queries/README.md).

Quick example via REST:

```bash
# Structured query with sorting and pagination
curl -G http://localhost:3000/api/v1/search \
  --data-urlencode 'query={"clauses":[{"type":"term","field":"status","value":"active"}]}' \
  --data-urlencode 'sort=name:asc' \
  --data-urlencode 'limit=20'
```

***

## Sorting

Search results can be sorted by any indexed field:

```clojure
(ast/query [(ast/fts "content" "database")]
           {:sort [{:field "name" :direction :asc}
                   {:field "created_at" :direction :desc}]
            :limit 10})
```

```bash
# REST API
curl "http://localhost:3000/api/v1/search?q=database&sort=name:asc,created_at:desc"
```

Sort type is detected automatically from the field name:

* Fields containing `date`, `time`, `created`, `updated` → long (timestamp)
* Fields containing `age`, `count`, `size`, `price`, `score` → numeric
* Default → string (lexicographic)

***

## Background Reindexing

ChronDB maintains index consistency through automatic background reindexing:

### On Startup

When ChronDB starts, a background process walks all Git commits to ensure every document is present in the Lucene index. This handles:

* First startup after restoring from a Git backup
* Recovery after a crash that left the index incomplete
* Index rebuilds after deleting the index directory

### Periodic Maintenance

A scheduled task runs reindexing verification every hour (default). This is safe for production — it processes incremental batches and does not block reads or writes.

### Monitoring Reindexing

```bash
# Check reindexing status in logs
grep "reindex" chrondb.log

# Check index document count via metrics
curl -s http://localhost:3000/metrics | grep chrondb_index_documents
```

### Forcing a Full Reindex

To rebuild the index from scratch:

```bash
# 1. Stop ChronDB
# 2. Delete the index directory
rm -rf data/index/
# 3. Start ChronDB — reindexing happens automatically
```

***

## Performance Tuning

### NRT Reader Configuration

The Near-Real-Time reader pool controls how quickly new writes become visible in search results. Default settings work well for most workloads:

* **Batch size**: 100 documents before committing to the index
* **Commit interval**: Periodic flush to ensure durability
* **RAM buffer**: In-memory buffer before flushing segments to disk

### Query Performance Tips

1. **Use specific field queries over FTS** — `term("status", "active")` is faster than `fts("content", "status:active")`
2. **Use `filter` instead of `must`** — filter clauses skip scoring, which is faster when relevance ordering is not needed
3. **Limit results** — always set a `limit` to avoid loading all matches
4. **Prefer cursor-based pagination** — for deep pagination (offset > 1000), use the `after` cursor instead of `offset`
5. **Avoid leading wildcards** — `*@example.com` requires scanning all terms; `user@*` only scans terms starting with `user@`

### Index Storage

The Lucene index is stored alongside the Git data directory. On production systems:

* Use SSDs for the data directory
* Ensure the filesystem supports `mmap` (ext4, xfs, APFS)
* Monitor index size: it typically grows to 30-50% of the raw document data size


# Protocols Overview

ChronDB supports multiple access protocols, allowing integration with different clients and frameworks. All protocols share the same storage and index backend, so data written via one protocol is immediately available through any other.

## Health and Monitoring

These endpoints are available on the REST HTTP port (default 3000) and are protocol-independent.

| Method | Endpoint    | Description                                                    |
| ------ | ----------- | -------------------------------------------------------------- |
| GET    | `/health`   | Full health check (storage, index, WAL, disk, memory)          |
| GET    | `/healthz`  | Kubernetes liveness probe (is the process alive?)              |
| GET    | `/readyz`   | Kubernetes readiness probe (is the service accepting traffic?) |
| GET    | `/startupz` | Kubernetes startup probe (has initialization completed?)       |
| GET    | `/metrics`  | Prometheus-format metrics                                      |

### Health Check Response

```json
{
  "status": "healthy",
  "timestamp": "2025-01-15T10:30:00Z",
  "total-latency-ms": 12,
  "checks": [
    {"component": "storage", "status": "healthy", "latency_ms": 3},
    {"component": "index", "status": "healthy", "latency_ms": 2},
    {"component": "wal", "status": "healthy", "details": {"pending-entries": 0}},
    {"component": "disk", "status": "healthy", "details": {"free-gb": 45.2, "used-percent": 55.0}},
    {"component": "memory", "status": "healthy", "details": {"used-mb": 256, "max-mb": 1024}}
  ]
}
```

Status values: `healthy`, `degraded` (returns HTTP 200), `unhealthy` (returns HTTP 503).

### Prometheus Metrics

Available at `GET /metrics` in Prometheus text format:

| Metric                            | Type      | Description                      |
| --------------------------------- | --------- | -------------------------------- |
| `chrondb_write_latency_seconds`   | Histogram | Write operation latency          |
| `chrondb_read_latency_seconds`    | Histogram | Read operation latency           |
| `chrondb_query_latency_seconds`   | Histogram | Query execution latency          |
| `chrondb_documents_saved_total`   | Counter   | Total documents saved            |
| `chrondb_documents_deleted_total` | Counter   | Total documents deleted          |
| `chrondb_documents_read_total`    | Counter   | Total documents read             |
| `chrondb_queries_executed_total`  | Counter   | Total queries executed           |
| `chrondb_active_connections`      | Gauge     | Current active connections       |
| `chrondb_index_documents`         | Gauge     | Documents in the index           |
| `chrondb_wal_pending_entries`     | Gauge     | Pending WAL entries              |
| `chrondb_occ_conflicts_total`     | Counter   | Optimistic concurrency conflicts |
| `chrondb_occ_retries_total`       | Counter   | OCC retry attempts               |

***

## REST API

ChronDB provides a complete REST API for database operations.

### Configuration

```clojure
:servers {
  :rest {
    :enabled true
    :host "0.0.0.0"
    :port 3000
  }
}
```

### Transaction Metadata Headers

All write endpoints accept optional headers to enrich the Git commit metadata:

| Header             | Description                                                  |
| ------------------ | ------------------------------------------------------------ |
| `X-ChronDB-Origin` | Override the origin label (defaults to `rest`)               |
| `X-ChronDB-User`   | Associate the commit with a user id or service account       |
| `X-ChronDB-Flags`  | Comma-separated semantic flags (e.g., `bulk-load,migration`) |
| `X-Request-Id`     | Propagate request correlation id into commit metadata        |
| `X-Correlation-Id` | Propagate correlation id for distributed tracing             |

All query parameters support `?branch=name` to target a specific branch (defaults to `main`).

### Document Endpoints

| Method | Endpoint                   | Description                          |
| ------ | -------------------------- | ------------------------------------ |
| POST   | `/api/v1/save`             | Save document (id in body)           |
| POST   | `/api/v1/put`              | Save document (id in params or body) |
| GET    | `/api/v1/get/:id`          | Get document by id                   |
| GET    | `/api/v1/get/:id/history`  | Get document version history         |
| DELETE | `/api/v1/delete/:id`       | Delete document                      |
| GET    | `/api/v1/documents`        | Export all documents                 |
| POST   | `/api/v1/documents/import` | Bulk import documents                |

### Search Endpoints

| Method | Endpoint                     | Description                 |
| ------ | ---------------------------- | --------------------------- |
| GET    | `/api/v1/search?q=term`      | Full-text search            |
| GET    | `/api/v1/search?query={...}` | Structured AST query (JSON) |

Search parameters:

| Parameter | Type    | Description                                                                                                   |
| --------- | ------- | ------------------------------------------------------------------------------------------------------------- |
| `q`       | string  | Full-text search query string                                                                                 |
| `query`   | JSON    | Structured query (see [AST Queries](https://github.com/avelino/chrondb/blob/main/docs/ast-queries/README.md)) |
| `branch`  | string  | Target branch (default: `main`)                                                                               |
| `limit`   | integer | Max results to return                                                                                         |
| `offset`  | integer | Number of results to skip                                                                                     |
| `sort`    | string  | Sort specification: `field:asc,field2:desc`                                                                   |
| `after`   | string  | Base64-encoded cursor for cursor-based pagination                                                             |

### History Endpoints

| Method | Endpoint                 | Description                          |
| ------ | ------------------------ | ------------------------------------ |
| GET    | `/api/v1/history/:id`    | Document version history             |
| GET    | `/api/v1/history?id=key` | Document history (id as query param) |

### Schema Validation Endpoints

| Method | Endpoint                                         | Description                        |
| ------ | ------------------------------------------------ | ---------------------------------- |
| GET    | `/api/v1/schemas/validation`                     | List all validation schemas        |
| GET    | `/api/v1/schemas/validation/:namespace`          | Get schema for namespace           |
| PUT    | `/api/v1/schemas/validation/:namespace`          | Save/update schema                 |
| DELETE | `/api/v1/schemas/validation/:namespace`          | Delete schema                      |
| GET    | `/api/v1/schemas/validation/:namespace/history`  | Schema version history             |
| POST   | `/api/v1/schemas/validation/:namespace/validate` | Validate a document against schema |

### Backup and Restore Endpoints

| Method | Endpoint          | Description                                          |
| ------ | ----------------- | ---------------------------------------------------- |
| POST   | `/api/v1/backup`  | Create backup (JSON body with `output` and `format`) |
| POST   | `/api/v1/restore` | Restore from backup (multipart upload)               |
| POST   | `/api/v1/export`  | Export snapshot (JSON body with `output` and `refs`) |
| GET    | `/api/v1/export`  | Export all documents                                 |

### System Endpoints

| Method | Endpoint         | Description                  |
| ------ | ---------------- | ---------------------------- |
| GET    | `/api/v1/info`   | Database info and statistics |
| POST   | `/api/v1/init`   | Initialize database          |
| POST   | `/api/v1/verify` | Verify data integrity        |

### Example with curl

```bash
# Save a document
curl -X POST http://localhost:3000/api/v1/save \
  -H "Content-Type: application/json" \
  -H "X-ChronDB-User: alice@example.com" \
  -d '{"id":"user:1","name":"John Doe","email":"john@example.com"}'

# Get a document
curl http://localhost:3000/api/v1/get/user:1

# Get document on a specific branch
curl http://localhost:3000/api/v1/get/user:1?branch=staging

# Search documents
curl "http://localhost:3000/api/v1/search?q=name:John&limit=10&sort=name:asc"

# Get document history
curl http://localhost:3000/api/v1/get/user:1/history

# Delete a document
curl -X DELETE http://localhost:3000/api/v1/delete/user:1

# Create a backup
curl -X POST http://localhost:3000/api/v1/backup \
  -H "Content-Type: application/json" \
  -d '{"output":"/tmp/backup.tar.gz","format":"tar.gz"}'

# Health check
curl http://localhost:3000/health
```

***

## Redis Protocol

ChronDB implements a subset of the Redis protocol (RESP2), allowing standard Redis clients to connect directly.

### Configuration

```clojure
:servers {
  :redis {
    :enabled true
    :host "0.0.0.0"
    :port 6379
  }
}
```

### Supported Commands

#### String Commands

| Command  | Syntax                    | Description                          |
| -------- | ------------------------- | ------------------------------------ |
| `GET`    | `GET key`                 | Retrieve document by key             |
| `SET`    | `SET key value`           | Store document (JSON value)          |
| `SETEX`  | `SETEX key seconds value` | Store with TTL                       |
| `SETNX`  | `SETNX key value`         | Store only if key does not exist     |
| `DEL`    | `DEL key`                 | Delete document                      |
| `EXISTS` | `EXISTS key`              | Check if key exists (returns 1 or 0) |

#### Hash Commands

| Command   | Syntax                                  | Description                      |
| --------- | --------------------------------------- | -------------------------------- |
| `HSET`    | `HSET key field value`                  | Set a field in a hash document   |
| `HGET`    | `HGET key field`                        | Get a specific field from a hash |
| `HMSET`   | `HMSET key field1 val1 field2 val2 ...` | Set multiple fields              |
| `HMGET`   | `HMGET key field1 field2 ...`           | Get multiple fields              |
| `HGETALL` | `HGETALL key`                           | Get all fields and values        |

#### List Commands

| Command  | Syntax                        | Description                        |
| -------- | ----------------------------- | ---------------------------------- |
| `LPUSH`  | `LPUSH key value [value ...]` | Push elements to the head          |
| `RPUSH`  | `RPUSH key value [value ...]` | Push elements to the tail          |
| `LPOP`   | `LPOP key`                    | Remove and return the head element |
| `RPOP`   | `RPOP key`                    | Remove and return the tail element |
| `LRANGE` | `LRANGE key start stop`       | Get a range of elements            |
| `LLEN`   | `LLEN key`                    | Get the list length                |

#### Set Commands

| Command     | Syntax                         | Description                   |
| ----------- | ------------------------------ | ----------------------------- |
| `SADD`      | `SADD key member [member ...]` | Add members to a set          |
| `SMEMBERS`  | `SMEMBERS key`                 | Get all members of a set      |
| `SISMEMBER` | `SISMEMBER key member`         | Check if member exists in set |
| `SREM`      | `SREM key member`              | Remove a member from a set    |

#### Sorted Set Commands

| Command  | Syntax                               | Description               |
| -------- | ------------------------------------ | ------------------------- |
| `ZADD`   | `ZADD key score member`              | Add member with score     |
| `ZRANGE` | `ZRANGE key start stop [WITHSCORES]` | Get members by rank range |
| `ZRANK`  | `ZRANK key member`                   | Get the rank of a member  |
| `ZSCORE` | `ZSCORE key member`                  | Get the score of a member |
| `ZREM`   | `ZREM key member`                    | Remove a member           |

#### Search Commands

| Command     | Syntax                                       | Description                          |
| ----------- | -------------------------------------------- | ------------------------------------ |
| `SEARCH`    | `SEARCH query [LIMIT offset count]`          | Search documents using Lucene syntax |
| `FT.SEARCH` | `FT.SEARCH index query [LIMIT offset count]` | RediSearch-compatible search         |

#### Schema Validation Commands

| Command           | Syntax                                    | Description                          |
| ----------------- | ----------------------------------------- | ------------------------------------ |
| `SCHEMA.SET`      | `SCHEMA.SET namespace schema-json`        | Define a validation schema           |
| `SCHEMA.GET`      | `SCHEMA.GET namespace`                    | Retrieve a schema                    |
| `SCHEMA.DEL`      | `SCHEMA.DEL namespace`                    | Delete a schema                      |
| `SCHEMA.LIST`     | `SCHEMA.LIST`                             | List all schemas                     |
| `SCHEMA.VALIDATE` | `SCHEMA.VALIDATE namespace document-json` | Validate a document against a schema |

#### Server Commands

| Command   | Syntax           | Description                                        |
| --------- | ---------------- | -------------------------------------------------- |
| `PING`    | `PING [message]` | Test connectivity (returns PONG or echoes message) |
| `ECHO`    | `ECHO message`   | Echo the given string                              |
| `COMMAND` | `COMMAND`        | List available commands                            |
| `INFO`    | `INFO`           | Server information                                 |

### Example with redis-cli

```bash
# Connect to ChronDB
redis-cli -h localhost -p 6379

# String operations
SET user:1 '{"name":"John Doe","email":"john@example.com"}'
GET user:1

# Hash operations
HSET user:2 name "Jane Doe"
HSET user:2 email "jane@example.com"
HGETALL user:2

# List operations
LPUSH notifications:user:1 '{"type":"welcome","text":"Hello!"}'
LRANGE notifications:user:1 0 -1

# Set operations
SADD tags:user:1 "admin" "editor"
SMEMBERS tags:user:1

# Sorted set operations
ZADD leaderboard 100 "player:1"
ZADD leaderboard 200 "player:2"
ZRANGE leaderboard 0 -1 WITHSCORES

# Search
SEARCH "name:John"
FT.SEARCH idx "name:John" LIMIT 0 10

# Schema validation
SCHEMA.SET user '{"type":"object","required":["name","email"]}'
SCHEMA.VALIDATE user '{"name":"John","email":"john@example.com"}'
```

***

## PostgreSQL Protocol

ChronDB implements a subset of the PostgreSQL wire protocol, allowing connection with standard SQL clients (`psql`, JDBC drivers, etc.).

### Configuration

```clojure
:servers {
  :postgresql {
    :enabled true
    :host "0.0.0.0"
    :port 5432
    :username "chrondb"
    :password "chrondb"
  }
}
```

### Data Model

Documents are mapped to virtual tables based on their key prefix:

* The prefix before `:` becomes the table name
* Document fields become columns
* The `id` column contains the key suffix

Example: key `user:1` with `{"name":"John","email":"john@example.com"}` is queryable as:

```sql
SELECT * FROM user WHERE id = '1';
```

### SQL Features

#### DML (Data Manipulation)

| Statement | Description                              |
| --------- | ---------------------------------------- |
| `SELECT`  | Query documents with full clause support |
| `INSERT`  | Create documents                         |
| `UPDATE`  | Modify documents                         |
| `DELETE`  | Remove documents                         |

#### SELECT Clause Support

| Clause       | Example                                              | Description        |
| ------------ | ---------------------------------------------------- | ------------------ |
| `WHERE`      | `WHERE age > 18 AND status = 'active'`               | Filter results     |
| `GROUP BY`   | `GROUP BY department`                                | Group results      |
| `ORDER BY`   | `ORDER BY name ASC, age DESC`                        | Sort results       |
| `LIMIT`      | `LIMIT 10`                                           | Limit result count |
| `INNER JOIN` | `... INNER JOIN orders ON users.id = orders.user_id` | Inner join         |
| `LEFT JOIN`  | `... LEFT JOIN orders ON users.id = orders.user_id`  | Left outer join    |

#### WHERE Operators

| Operator                 | Example                                 |
| ------------------------ | --------------------------------------- |
| `=`, `!=`, `<>`          | `WHERE status = 'active'`               |
| `>`, `<`, `>=`, `<=`     | `WHERE age >= 18`                       |
| `LIKE`                   | `WHERE name LIKE 'John%'`               |
| `BETWEEN`                | `WHERE age BETWEEN 18 AND 65`           |
| `IS NULL`, `IS NOT NULL` | `WHERE email IS NOT NULL`               |
| `IN`, `NOT IN`           | `WHERE status IN ('active', 'pending')` |

#### Aggregate Functions

| Function      | Description   |
| ------------- | ------------- |
| `COUNT(*)`    | Count rows    |
| `SUM(column)` | Sum values    |
| `AVG(column)` | Average value |
| `MIN(column)` | Minimum value |
| `MAX(column)` | Maximum value |

#### Full-Text Search in SQL

```sql
-- Using FTS_MATCH function
SELECT * FROM articles WHERE FTS_MATCH(content, 'database chronological');

-- Using PostgreSQL-style tsquery
SELECT * FROM articles WHERE content @@ to_tsquery('database & chronological');
```

#### DDL and Metadata

| Statement                                    | Description                          |
| -------------------------------------------- | ------------------------------------ |
| `CREATE TABLE name (columns...)`             | Create table with column definitions |
| `DROP TABLE [IF EXISTS] name`                | Drop a table                         |
| `SHOW TABLES`                                | List all tables                      |
| `SHOW SCHEMAS` / `SHOW DATABASES`            | List schemas                         |
| `DESCRIBE table` / `SHOW COLUMNS FROM table` | Show table structure                 |

#### Validation Schema Management

```sql
CREATE VALIDATION SCHEMA FOR namespace AS '{"type":"object","required":["name"]}';
DROP VALIDATION SCHEMA FOR namespace;
SHOW VALIDATION SCHEMAS;
```

### ChronDB Temporal Functions

These functions expose ChronDB's time-travel capabilities through SQL:

#### `chrondb_history(table, id)`

Returns the complete modification history of a document.

| Column      | Description                             |
| ----------- | --------------------------------------- |
| `commit_id` | Git commit hash identifying the version |
| `timestamp` | When the change was made                |
| `committer` | Who made the change                     |
| `data`      | Document content at that version        |

```sql
SELECT * FROM chrondb_history('user', '1');
SELECT commit_id, timestamp FROM chrondb_history('user', '1');
```

#### `chrondb_at(table, id, commit_hash)`

Returns the document exactly as it was at a specific commit.

```sql
SELECT * FROM chrondb_at('user', '1', 'abc123def456');
```

#### `chrondb_diff(table, id, commit1, commit2)`

Compares two versions and returns the differences.

| Column    | Description                                    |
| --------- | ---------------------------------------------- |
| `id`      | Document id                                    |
| `commit1` | First commit hash                              |
| `commit2` | Second commit hash                             |
| `added`   | Fields added between versions (JSON)           |
| `removed` | Fields removed between versions (JSON)         |
| `changed` | Fields modified with old and new values (JSON) |

```sql
SELECT * FROM chrondb_diff('user', '1', 'abc123', 'def456');
SELECT changed FROM chrondb_diff('user', '1', 'abc123', 'def456');
```

#### Branch Management Functions

| Function                                   | Description         |
| ------------------------------------------ | ------------------- |
| `chrondb_branch_list()`                    | List all branches   |
| `chrondb_branch_create('name')`            | Create a new branch |
| `chrondb_branch_checkout('name')`          | Switch to a branch  |
| `chrondb_branch_merge('source', 'target')` | Merge branches      |

### Example with psql

```bash
# Connect to ChronDB
psql -h localhost -p 5432 -U chrondb

# Create a document
INSERT INTO user (id, name, email) VALUES ('1', 'John Doe', 'john@example.com');

# Query documents
SELECT * FROM user WHERE id = '1';
SELECT name, email FROM user WHERE name LIKE 'John%';

# Aggregation
SELECT department, COUNT(*), AVG(salary) FROM employee GROUP BY department ORDER BY COUNT(*) DESC;

# Join tables
SELECT u.name, o.total
FROM user u
INNER JOIN orders o ON u.id = o.user_id
WHERE o.total > 100;

# Update a document
UPDATE user SET email = 'john.doe@example.com' WHERE id = '1';

# Delete a document
DELETE FROM user WHERE id = '1';

# Time-travel queries
SELECT * FROM chrondb_history('user', '1');
SELECT * FROM chrondb_at('user', '1', 'abc123def456');
SELECT * FROM chrondb_diff('user', '1', 'abc123', 'def456');

# Branch management
SELECT * FROM chrondb_branch_list();
SELECT * FROM chrondb_branch_create('staging');

# Full-text search
SELECT * FROM articles WHERE FTS_MATCH(content, 'database distributed');

# Show metadata
SHOW TABLES;
DESCRIBE user;
```

***

## Protocol Comparison

| Feature              | REST                  | Redis               | PostgreSQL                          |
| -------------------- | --------------------- | ------------------- | ----------------------------------- |
| Document CRUD        | Full                  | Full                | Full (via SQL)                      |
| Search / Queries     | AST + FTS             | Lucene syntax       | SQL WHERE + FTS                     |
| Version History      | `/get/:id/history`    | Not available       | `chrondb_history()`                 |
| Time Travel          | Not available         | Not available       | `chrondb_at()`, `chrondb_diff()`    |
| Branch Support       | `?branch=name`        | Not available       | Branch functions                    |
| Schema Validation    | REST endpoints        | `SCHEMA.*` commands | `CREATE VALIDATION SCHEMA`          |
| Backup / Restore     | REST endpoints        | Not available       | Not available                       |
| Aggregation          | Not available         | Not available       | `COUNT`, `SUM`, `AVG`, `MIN`, `MAX` |
| JOINs                | Not available         | Not available       | `INNER JOIN`, `LEFT JOIN`           |
| Monitoring           | `/health`, `/metrics` | `INFO`              | Not available                       |
| Transaction Metadata | HTTP headers          | Not available       | Not available                       |


# Clojure API

This document provides detailed information about the ChronDB API.

## Core API (Clojure/Java)

### Creating a Database

```clojure
(require '[chrondb.core :as chrondb])

;; Create with default configuration
(def db (chrondb/create-chrondb))

;; Create with custom configuration
(def db (chrondb/create-chrondb config))
```

### Document Operations

#### Save Document

```clojure
(chrondb/save db "key" value)
```

Parameters:

* `db`: ChronDB instance
* `key`: String identifier for the document
* `value`: Document data (will be stored as JSON)

Returns: The saved document

#### Get Document

```clojure
(chrondb/get db "key")
```

Parameters:

* `db`: ChronDB instance
* `key`: String identifier for the document

Returns: The document if found, nil otherwise

#### Delete Document

```clojure
(chrondb/delete db "key")
```

Parameters:

* `db`: ChronDB instance
* `key`: String identifier for the document

Returns: true if document was deleted, false otherwise

### Search Operations

#### Search Documents

```clojure
(chrondb/search db query)
(chrondb/search db query {:limit 10 :offset 0})
```

Parameters:

* `db`: ChronDB instance
* `query`: Lucene query string
* `options`: Optional map with:
  * `:limit`: Maximum number of results (default: 10)
  * `:offset`: Number of results to skip (default: 0)

Returns: Sequence of matching documents

### Version Control Operations

#### Get Document History

```clojure
(chrondb/history db "key")
```

Parameters:

* `db`: ChronDB instance
* `key`: String identifier for the document

Returns: Sequence of document versions with timestamps

#### Get Document at Point in Time

```clojure
(chrondb/get-at db "key" timestamp)
```

Parameters:

* `db`: ChronDB instance
* `key`: String identifier for the document
* `timestamp`: ISO-8601 timestamp string or java.time.Instant

Returns: The document as it existed at the specified time

#### Compare Document Versions

```clojure
(chrondb/diff db "key" timestamp1 timestamp2)
```

Parameters:

* `db`: ChronDB instance
* `key`: String identifier for the document
* `timestamp1`: First timestamp
* `timestamp2`: Second timestamp

Returns: Differences between versions

### Branch Operations

#### Create Branch

```clojure
(chrondb/create-branch db "branch-name")
```

Parameters:

* `db`: ChronDB instance
* `branch-name`: Name of the new branch

Returns: Updated ChronDB instance

#### Switch Branch

```clojure
(chrondb/switch-branch db "branch-name")
```

Parameters:

* `db`: ChronDB instance
* `branch-name`: Name of the branch to switch to

Returns: Updated ChronDB instance

#### Merge Branches

```clojure
(chrondb/merge-branch db "source-branch" "target-branch")
```

Parameters:

* `db`: ChronDB instance
* `source-branch`: Branch to merge from
* `target-branch`: Branch to merge into

Returns: Updated ChronDB instance

### Transaction Operations

```clojure
(chrondb/with-transaction [db]
  (chrondb/save db "key1" value1)
  (chrondb/save db "key2" value2))
```

All operations within the transaction block are atomic:

* Either all succeed or all fail
* Changes are only visible after successful commit
* Automatic rollback on failure

#### Transaction Metadata and Git Notes

ChronDB attaches a structured Git note to every commit. The payload includes the transaction id (`tx_id`), origin, optional user identifier, flags, and request metadata. You can enrich this payload by providing HTTP headers on REST requests:

| Header                              | Description                                                                   |
| ----------------------------------- | ----------------------------------------------------------------------------- |
| `X-ChronDB-Origin`                  | Overrides the origin label stored in the note (defaults to `rest`).           |
| `X-ChronDB-User`                    | Associates commits with an authenticated user id or service account.          |
| `X-ChronDB-Flags`                   | Comma-separated list of semantic flags (for example: `bulk-load, migration`). |
| `X-Request-Id` / `X-Correlation-Id` | Propagate request correlation identifiers into commit metadata.               |

All ChronDB front-ends (REST, Redis, SQL) reuse the same transaction core, so commits that belong to the same logical operation share the same `tx_id`. Special operations automatically set semantic flags. For instance, document imports mark the transaction with `bulk-load`, and restore flows add `rollback`.

Inspect commit notes with standard Git tooling:

```
git log --show-notes=chrondb
git notes --ref=chrondb show <commit-hash>
```

You can parse the JSON payload to correlate commits with external systems or audit trails.

### Event Hooks

#### Register Hook

```clojure
(chrondb/register-hook db :pre-save
  (fn [doc]
    ;; Hook logic here
    doc))
```

Available hook points:

* `:pre-save`: Before saving a document
* `:post-save`: After saving a document
* `:pre-delete`: Before deleting a document
* `:post-delete`: After deleting a document

### Utility Functions

#### Health Check

```clojure
(chrondb/health-check db)
```

Returns: Map with health status information

#### Backup / Restore

```clojure
(chrondb.backup.core/create-full-backup storage {:output-path "backups/full.tar.gz"})
(chrondb.backup.core/create-incremental-backup storage {:output-path "backups/incr.bundle"
                                                       :base-commit "<commit>"
                                                       :format :bundle})
```

```clojure
(chrondb.backup.core/restore-backup storage {:input-path "backups/full.tar.gz"})
```

```clojure
(chrondb.backup.core/export-snapshot storage {:output "backups/main.bundle"
                                              :refs ["refs/heads/main"]})
```

```clojure
(chrondb.backup.core/import-snapshot storage {:input "backups/main.bundle"})
```

Scheduled backup example:

```clojure
(chrondb.backup.core/schedule-backup storage {:mode :bundle
                                              :format :bundle
                                              :interval-minutes 60
                                              :output-dir "backups/"})
```

#### Statistics

```clojure
(chrondb/stats db)
```

Returns: Map with database statistics

## Error Handling

All API functions may throw the following exceptions:

* `ChronDBException`: Base exception class
* `StorageException`: Storage-related errors
* `IndexException`: Index-related errors
* `ConfigurationException`: Configuration errors
* `ValidationException`: Data validation errors

Example error handling:

```clojure
(try
  (chrondb/save db "key" value)
  (catch chrondb.exceptions.StorageException e
    (log/error "Storage error:" (.getMessage e)))
  (catch chrondb.exceptions.ValidationException e
    (log/error "Validation error:" (.getMessage e))))

```


# REST API Examples

This document provides examples for using the ChronDB REST API with curl and JavaScript.

## REST API Overview

ChronDB provides a RESTful API that allows you to interact with the database over HTTP. This makes it easy to integrate with any programming language or environment that can make HTTP requests.

The REST API server can be configured in the `config.edn` file:

```clojure
:servers {
  :rest {
    :enabled true
    :host "0.0.0.0"
    :port 3000
  }
}
```

## Examples with curl

### Document Operations

#### Create or Update a Document

```bash
# Create a new document
curl -X POST http://localhost:3000/api/v1/documents/user:1 \
  -H "Content-Type: application/json" \
  -d '{
    "name": "John Doe",
    "email": "john@example.com",
    "age": 30,
    "roles": ["admin", "user"]
  }'

# Response
# {"status":"success","key":"user:1"}
```

#### Get a Document

```bash
# Retrieve a document by key
curl http://localhost:3000/api/v1/documents/user:1

# Response
# {"name":"John Doe","email":"john@example.com","age":30,"roles":["admin","user"]}
```

#### Delete a Document

```bash
# Delete a document by key
curl -X DELETE http://localhost:3000/api/v1/documents/user:1

# Response
# {"status":"success","key":"user:1"}
```

### Search Operations

```bash
# Search for documents by field
curl http://localhost:3000/api/v1/search?q=name:John

# Search with multiple criteria
curl http://localhost:3000/api/v1/search?q=name:John%20AND%20age:%5B25%20TO%2035%5D

# Search with pagination
curl http://localhost:3000/api/v1/search?q=roles:admin&limit=5&offset=0
```

### Version Control Operations

#### Get Document History

```bash
# Get the history of a document
curl http://localhost:3000/api/v1/documents/user:1/history

# Response
# [
#   {
#     "timestamp": "2023-10-15T14:30:45Z",
#     "data": {"name":"John Doe","email":"john@example.com","age":30,"roles":["admin","user"]}
#   },
#   {
#     "timestamp": "2023-10-10T09:15:22Z",
#     "data": {"name":"John Doe","email":"john@example.com","roles":["user"]}
#   }
# ]
```

#### Get Document at a Point in Time

```bash
# Get a document as it was at a specific time
curl http://localhost:3000/api/v1/documents/user:1/at/2023-10-10T09:15:22Z

# Response
# {"name":"John Doe","email":"john@example.com","roles":["user"]}
```

#### Compare Document Versions

```bash
# Compare two versions of a document
curl http://localhost:3000/api/v1/documents/user:1/diff?t1=2023-10-10T09:15:22Z&t2=2023-10-15T14:30:45Z

# Response
# {
#   "added": {"age":30,"roles":["admin"]},
#   "removed": {"roles":["user"]},
#   "changed": {}
# }
```

### Branch Operations

#### List Branches

```bash
# List all branches
curl http://localhost:3000/api/v1/branches

# Response
# ["main","dev","test"]
```

#### Create a Branch

```bash
# Create a new branch
curl -X POST http://localhost:3000/api/v1/branches/feature-login

# Response
# {"status":"success","branch":"feature-login"}
```

#### Switch Branch

```bash
# Switch to a different branch
curl -X PUT http://localhost:3000/api/v1/branches/feature-login/checkout

# Response
# {"status":"success","branch":"feature-login"}
```

#### Merge Branches

```bash
# Merge one branch into another
curl -X POST http://localhost:3000/api/v1/branches/feature-login/merge/main

# Response
# {"status":"success","source":"feature-login","target":"main"}
```

## Examples with JavaScript

### Setting Up

```javascript
// Utility function for API requests
async function chronDBRequest(endpoint, method = 'GET', body = null) {
  const options = {
    method,
    headers: {
      'Content-Type': 'application/json',
      'Accept': 'application/json'
    }
  };

  if (body) {
    options.body = JSON.stringify(body);
  }

  const response = await fetch(`http://localhost:3000/api/v1/${endpoint}`, options);
  return response.json();
}
```

### Document Operations

```javascript
// Create or update a document
async function saveDocument(key, data) {
  return chronDBRequest(`documents/${key}`, 'POST', data);
}

// Get a document
async function getDocument(key) {
  return chronDBRequest(`documents/${key}`);
}

// Delete a document
async function deleteDocument(key) {
  return chronDBRequest(`documents/${key}`, 'DELETE');
}

// Usage examples
async function documentExamples() {
  // Create a user
  const user = {
    name: "Jane Smith",
    email: "jane@example.com",
    age: 28,
    roles: ["editor"]
  };

  await saveDocument('user:2', user);

  // Retrieve the user
  const savedUser = await getDocument('user:2');
  console.log('Retrieved user:', savedUser);

  // Update the user
  savedUser.roles.push('reviewer');
  await saveDocument('user:2', savedUser);

  // Delete the user
  await deleteDocument('user:2');
}
```

### Search Operations

```javascript
// Search for documents
async function searchDocuments(query, limit = 10, offset = 0) {
  const params = new URLSearchParams({
    q: query,
    limit,
    offset
  });

  return chronDBRequest(`search?${params.toString()}`);
}

// Usage examples
async function searchExamples() {
  // Simple search
  const admins = await searchDocuments('roles:admin');
  console.log('Admins:', admins);

  // Complex search
  const activeUsers = await searchDocuments('lastLogin:[NOW-7DAYS TO NOW]');
  console.log('Recently active users:', activeUsers);

  // Pagination
  const page1 = await searchDocuments('type:article', 5, 0);
  const page2 = await searchDocuments('type:article', 5, 5);
  console.log('Articles page 1:', page1);
  console.log('Articles page 2:', page2);
}
```

### Version Control Operations

```javascript
// Get document history
async function getDocumentHistory(key) {
  return chronDBRequest(`documents/${key}/history`);
}

// Get document at a point in time
async function getDocumentAt(key, timestamp) {
  return chronDBRequest(`documents/${key}/at/${encodeURIComponent(timestamp)}`);
}

// Compare document versions
async function compareDocuments(key, timestamp1, timestamp2) {
  const params = new URLSearchParams({
    t1: timestamp1,
    t2: timestamp2
  });

  return chronDBRequest(`documents/${key}/diff?${params.toString()}`);
}

// Usage examples
async function versionControlExamples() {
  // Get document history
  const history = await getDocumentHistory('user:1');
  console.log('User history:', history);

  // Get version from a week ago
  const oneWeekAgo = new Date();
  oneWeekAgo.setDate(oneWeekAgo.getDate() - 7);
  const oldVersion = await getDocumentAt('user:1', oneWeekAgo.toISOString());
  console.log('User from one week ago:', oldVersion);

  // Compare current version with week-old version
  const diff = await compareDocuments('user:1', oneWeekAgo.toISOString(), new Date().toISOString());
  console.log('Changes in the last week:', diff);
}
```

### Branch Operations

```javascript
// List all branches
async function listBranches() {
  return chronDBRequest('branches');
}

// Create a new branch
async function createBranch(name) {
  return chronDBRequest(`branches/${name}`, 'POST');
}

// Switch to a branch
async function switchBranch(name) {
  return chronDBRequest(`branches/${name}/checkout`, 'PUT');
}

// Merge branches
async function mergeBranches(source, target) {
  return chronDBRequest(`branches/${source}/merge/${target}`, 'POST');
}

// Usage examples
async function branchExamples() {
  // List all branches
  const branches = await listBranches();
  console.log('Available branches:', branches);

  // Create a feature branch
  await createBranch('feature-newui');

  // Switch to the feature branch
  await switchBranch('feature-newui');

  // Make changes in the feature branch
  await saveDocument('ui:config', { theme: 'dark', layout: 'responsive' });

  // Merge back to main
  await mergeBranches('feature-newui', 'main');
}
```

### Complete Example: User Management System

```javascript
// User management module
const UserAPI = {
  baseUrl: 'http://localhost:3000/api/v1',

  async request(endpoint, method = 'GET', body = null) {
    const options = {
      method,
      headers: {
        'Content-Type': 'application/json',
        'Accept': 'application/json'
      }
    };

    if (body) {
      options.body = JSON.stringify(body);
    }

    const response = await fetch(`${this.baseUrl}/${endpoint}`, options);
    if (!response.ok) {
      throw new Error(`API request failed: ${response.statusText}`);
    }
    return response.json();
  },

  // User operations
  async createUser(userData) {
    const userId = `user:${Date.now()}`;
    await this.request(`documents/${userId}`, 'POST', {
      ...userData,
      createdAt: new Date().toISOString(),
      updatedAt: new Date().toISOString()
    });
    return userId;
  },

  async updateUser(userId, updates) {
    const user = await this.request(`documents/${userId}`);
    const updatedUser = {
      ...user,
      ...updates,
      updatedAt: new Date().toISOString()
    };
    return this.request(`documents/${userId}`, 'POST', updatedUser);
  },

  async getUser(userId) {
    return this.request(`documents/${userId}`);
  },

  async deleteUser(userId) {
    return this.request(`documents/${userId}`, 'DELETE');
  },

  async searchUsers(query) {
    return this.request(`search?q=${encodeURIComponent(query)}`);
  },

  async getUserHistory(userId) {
    return this.request(`documents/${userId}/history`);
  }
};

// Usage
async function userManagementExample() {
  try {
    // Create a user
    const userId = await UserAPI.createUser({
      name: "Alice Johnson",
      email: "alice@example.com",
      role: "developer",
      skills: ["JavaScript", "React", "Node.js"]
    });
    console.log(`Created user with ID: ${userId}`);

    // Get the user
    const user = await UserAPI.getUser(userId);
    console.log("Retrieved user:", user);

    // Update the user
    await UserAPI.updateUser(userId, {
      role: "senior developer",
      skills: [...user.skills, "TypeScript"]
    });
    console.log("Updated user skills and role");

    // Get user history
    const history = await UserAPI.getUserHistory(userId);
    console.log("User modification history:", history);

    // Search for developers
    const developers = await UserAPI.searchUsers("role:*developer*");
    console.log("Found developers:", developers);

  } catch (error) {
    console.error("Error in user management:", error);
  }
}

// Run the example
userManagementExample();
```

## Schema Validation Operations

ChronDB supports optional JSON Schema validation per namespace. See the [Schema Validation](/key-concepts/validation) documentation for full details.

### Managing Validation Schemas

#### Create a Validation Schema

```bash
# Create schema with strict mode for the "users" namespace
curl -X PUT http://localhost:3000/api/v1/schemas/validation/users \
  -H "Content-Type: application/json" \
  -d '{
    "mode": "strict",
    "schema": {
      "$schema": "http://json-schema.org/draft-07/schema#",
      "type": "object",
      "required": ["id", "email"],
      "properties": {
        "id": { "type": "string" },
        "email": { "type": "string", "format": "email" },
        "name": { "type": "string" },
        "age": { "type": "integer", "minimum": 0 }
      },
      "additionalProperties": false
    }
  }'

# Response
# {"namespace":"users","version":1,"mode":"strict","schema":{...}}
```

#### List All Validation Schemas

```bash
curl http://localhost:3000/api/v1/schemas/validation

# Response
# [{"namespace":"users","version":1,"mode":"strict"},{"namespace":"products","version":2,"mode":"warning"}]
```

#### Get a Specific Schema

```bash
curl http://localhost:3000/api/v1/schemas/validation/users

# Response
# {"namespace":"users","version":1,"mode":"strict","schema":{...},"created_at":"2024-01-15T10:00:00Z"}
```

#### Get Schema History

```bash
curl http://localhost:3000/api/v1/schemas/validation/users/history

# Response
# [{"commit-id":"abc123","timestamp":"2024-01-15T10:00:00Z","message":"Updated validation schema for users"}]
```

#### Delete a Validation Schema

```bash
curl -X DELETE http://localhost:3000/api/v1/schemas/validation/users

# Response
# {"status":"success","namespace":"users"}
```

### Dry-Run Validation

Test if a document is valid before saving:

```bash
# Valid document
curl -X POST http://localhost:3000/api/v1/schemas/validation/users/validate \
  -H "Content-Type: application/json" \
  -d '{"id": "users:1", "email": "john@example.com", "name": "John"}'

# Response (valid)
# {"valid":true,"errors":[]}

# Invalid document (missing required email)
curl -X POST http://localhost:3000/api/v1/schemas/validation/users/validate \
  -H "Content-Type: application/json" \
  -d '{"id": "users:1", "name": "John"}'

# Response (invalid)
# {"valid":false,"errors":[{"path":"$.email","message":"required property 'email' not found","keyword":"required"}]}
```

### Validation Error Response

When saving a document that fails validation in `strict` mode:

```bash
curl -X POST http://localhost:3000/api/v1/documents/users:1 \
  -H "Content-Type: application/json" \
  -d '{"name": "John", "age": -5}'

# Response (HTTP 400)
# {
#   "error": "VALIDATION_ERROR",
#   "namespace": "users",
#   "document_id": "users:1",
#   "mode": "strict",
#   "violations": [
#     {"path": "$.email", "message": "required property 'email' not found", "keyword": "required"},
#     {"path": "$.age", "message": "must be >= 0", "keyword": "minimum"}
#   ]
# }
```

### JavaScript Validation Examples

```javascript
// Schema management functions
const SchemaAPI = {
  baseUrl: 'http://localhost:3000/api/v1',

  async request(endpoint, method = 'GET', body = null) {
    const options = {
      method,
      headers: {
        'Content-Type': 'application/json',
        'Accept': 'application/json'
      }
    };
    if (body) {
      options.body = JSON.stringify(body);
    }
    const response = await fetch(`${this.baseUrl}/${endpoint}`, options);
    return response.json();
  },

  // Create or update a validation schema
  async setSchema(namespace, schema, mode = 'strict') {
    return this.request(`schemas/validation/${namespace}`, 'PUT', { mode, schema });
  },

  // Get a validation schema
  async getSchema(namespace) {
    return this.request(`schemas/validation/${namespace}`);
  },

  // List all schemas
  async listSchemas() {
    return this.request('schemas/validation');
  },

  // Delete a schema
  async deleteSchema(namespace) {
    return this.request(`schemas/validation/${namespace}`, 'DELETE');
  },

  // Validate a document without saving
  async validateDocument(namespace, document) {
    return this.request(`schemas/validation/${namespace}/validate`, 'POST', document);
  },

  // Get schema history
  async getSchemaHistory(namespace) {
    return this.request(`schemas/validation/${namespace}/history`);
  }
};

// Usage example
async function schemaValidationExample() {
  // Create a schema for users
  const userSchema = {
    type: 'object',
    required: ['id', 'email'],
    properties: {
      id: { type: 'string' },
      email: { type: 'string', format: 'email' },
      name: { type: 'string' },
      age: { type: 'integer', minimum: 0 }
    },
    additionalProperties: false
  };

  await SchemaAPI.setSchema('users', userSchema, 'strict');
  console.log('Schema created');

  // Validate a document before saving
  const validDoc = { id: 'users:1', email: 'john@example.com', name: 'John' };
  const validResult = await SchemaAPI.validateDocument('users', validDoc);
  console.log('Valid document:', validResult.valid); // true

  const invalidDoc = { id: 'users:2', name: 'Jane' }; // missing email
  const invalidResult = await SchemaAPI.validateDocument('users', invalidDoc);
  console.log('Invalid document:', invalidResult.valid); // false
  console.log('Errors:', invalidResult.errors);
}

schemaValidationExample();
```


# Redis Protocol Examples

This document provides examples for using the ChronDB Redis protocol interface with redis-cli and JavaScript.

## Redis Protocol Overview

ChronDB implements a subset of the Redis protocol, allowing you to connect using standard Redis clients. This makes it easy to integrate with existing applications that already use Redis or to leverage the simplicity of the Redis command structure.

The Redis protocol server can be configured in the `config.edn` file:

```clojure
:servers {
  :redis {
    :enabled true
    :host "0.0.0.0"
    :port 6379
  }
}
```

## Standard Redis Commands

ChronDB supports the following standard Redis commands:

* `GET` - Retrieve a document
* `SET` - Create or update a document
* `DEL` - Delete a document
* `EXISTS` - Check if a document exists
* `KEYS` - List keys matching a pattern
* `HGET` - Get a specific field from a document
* `HSET` - Set a specific field in a document
* `HDEL` - Remove a specific field from a document
* `HGETALL` - Get all fields from a document

## ChronDB-Specific Commands

In addition to standard Redis commands, ChronDB provides special commands:

* `CHRONDB.HISTORY` - Get document history
* `CHRONDB.GETAT` - Get document at a point in time
* `CHRONDB.DIFF` - Compare document versions
* `CHRONDB.BRANCH.LIST` - List branches
* `CHRONDB.BRANCH.CREATE` - Create a new branch
* `CHRONDB.BRANCH.CHECKOUT` - Switch to a branch
* `CHRONDB.BRANCH.MERGE` - Merge branches

### Schema Validation Commands

* `SCHEMA.SET namespace schema [MODE strict|warning]` - Create or update a validation schema
* `SCHEMA.GET namespace` - Get a validation schema
* `SCHEMA.DEL namespace` - Delete a validation schema
* `SCHEMA.LIST` - List all validation schemas
* `SCHEMA.VALIDATE namespace document` - Validate a document without saving

## Examples with redis-cli

### Connecting to ChronDB

```bash
# Connect to ChronDB using redis-cli
redis-cli -h localhost -p 6379
```

### Document Operations

```bash
# Create a document
SET user:1 '{"name":"John Doe","email":"john@example.com","age":30}'

# Get a document
GET user:1
# Output: {"name":"John Doe","email":"john@example.com","age":30}

# Check if a document exists
EXISTS user:1
# Output: (integer) 1

# Delete a document
DEL user:1
# Output: (integer) 1

# Create a document with expiration (ChronDB supports TTL)
SET user:1 '{"name":"John Doe"}' EX 3600  # Expires in 1 hour
```

### Working with Document Fields

```bash
# Set a document with multiple fields
HSET product:1 name "Laptop" price 999.99 category "Electronics" stock 50

# Get a specific field
HGET product:1 price
# Output: "999.99"

# Get all fields
HGETALL product:1
# Output:
# 1) "name"
# 2) "Laptop"
# 3) "price"
# 4) "999.99"
# 5) "category"
# 6) "Electronics"
# 7) "stock"
# 8) "50"

# Delete a field
HDEL product:1 stock
```

### Searching for Documents

```bash
# Find all user keys
KEYS user:*
# Output:
# 1) "user:1"
# 2) "user:2"
# 3) "user:3"

# Find keys with wildcard
KEYS *:laptop*

# Full-text search using AST (new in Lucene migration)
SEARCH Software
# Output: Array of JSON documents matching "Software" in content field

# Search with limit
SEARCH Software LIMIT 10

# Search with limit and offset
SEARCH Software LIMIT 10 OFFSET 5

# Search with sorting
SEARCH Software SORT age:asc
SEARCH Software SORT age:desc

# Search on specific branch
SEARCH Software BRANCH main

# Combined options
SEARCH Software LIMIT 10 OFFSET 0 SORT name:asc BRANCH main

# Using FT.SEARCH alias (compatible with RediSearch)
FT.SEARCH Software LIMIT 10
```

### Version Control Commands

```bash
# Get document history
CHRONDB.HISTORY user:1
# Output: Array of document versions with timestamps

# Get document at a specific point in time
CHRONDB.GETAT user:1 "2023-10-15T14:30:00Z"
# Output: Document as it existed at that timestamp

# Compare document versions
CHRONDB.DIFF user:1 "2023-10-10T09:15:00Z" "2023-10-15T14:30:00Z"
# Output: Differences between versions
```

### Branch Operations

```bash
# List all branches
CHRONDB.BRANCH.LIST
# Output:
# 1) "main"
# 2) "dev"
# 3) "test"

# Create a new branch
CHRONDB.BRANCH.CREATE feature-login
# Output: OK

# Switch to a branch
CHRONDB.BRANCH.CHECKOUT feature-login
# Output: OK

# Merge branches
CHRONDB.BRANCH.MERGE feature-login main
# Output: OK
```

## Examples with JavaScript

### Setting Up

The following examples use the standard [redis](https://www.npmjs.com/package/redis) package for Node.js.

```javascript
// Install the redis package:
// npm install redis

const redis = require('redis');

// Create a Redis client connected to ChronDB
const client = redis.createClient({
  url: 'redis://localhost:6379'
});

// Connect to the server
async function connect() {
  await client.connect();
  console.log('Connected to ChronDB via Redis protocol');
}

// Handle connection errors
client.on('error', (err) => {
  console.error('Redis Client Error:', err);
});
```

### Document Operations

```javascript
// Basic CRUD operations
async function documentOperations() {
  try {
    // Create or update a document
    await client.set('user:1', JSON.stringify({
      name: 'Jane Smith',
      email: 'jane@example.com',
      age: 28,
      roles: ['editor']
    }));
    console.log('Document created');

    // Get a document
    const userJson = await client.get('user:1');
    const user = JSON.parse(userJson);
    console.log('Retrieved user:', user);

    // Update a document
    user.roles.push('reviewer');
    await client.set('user:1', JSON.stringify(user));
    console.log('Document updated');

    // Check if a document exists
    const exists = await client.exists('user:1');
    console.log('Document exists:', exists === 1);

    // Delete a document
    await client.del('user:1');
    console.log('Document deleted');

  } catch (err) {
    console.error('Error in document operations:', err);
  }
}
```

### Working with Document Fields (Hash Operations)

```javascript
// Using Redis hash operations for field-level access
async function fieldOperations() {
  try {
    // Create document with fields
    await client.hSet('product:1', {
      name: 'Smartphone',
      price: '799.99',
      category: 'Electronics',
      stock: '100',
      specs: JSON.stringify({
        screen: '6.5 inch',
        ram: '8GB',
        storage: '128GB'
      })
    });
    console.log('Product created with fields');

    // Get a specific field
    const price = await client.hGet('product:1', 'price');
    console.log('Product price:', price);

    // Get all fields
    const product = await client.hGetAll('product:1');
    console.log('Complete product:', product);

    // Parse JSON fields if needed
    if (product.specs) {
      product.specs = JSON.parse(product.specs);
    }

    // Update specific fields
    await client.hSet('product:1', 'stock', '95');
    await client.hSet('product:1', 'price', '749.99');
    console.log('Product fields updated');

    // Delete a field
    await client.hDel('product:1', 'specs');
    console.log('Specs field removed');

  } catch (err) {
    console.error('Error in field operations:', err);
  }
}
```

### Searching for Documents

```javascript
// Finding documents with key patterns
async function searchOperations() {
  try {
    // Generate some test data
    await client.set('user:101', JSON.stringify({ name: 'Alice', dept: 'Engineering' }));
    await client.set('user:102', JSON.stringify({ name: 'Bob', dept: 'Marketing' }));
    await client.set('user:103', JSON.stringify({ name: 'Charlie', dept: 'Engineering' }));

    // Find all user keys
    const userKeys = await client.keys('user:*');
    console.log('All user keys:', userKeys);

    // Get multiple documents at once
    if (userKeys.length > 0) {
      const usersData = await client.mGet(userKeys);
      const users = usersData.map(data => JSON.parse(data));
      console.log('All users:', users);
    }

    // Find specific keys
    const engineeringUserKeys = await Promise.all((await client.keys('user:*')).map(async key => {
      const userData = await client.get(key);
      const user = JSON.parse(userData);
      return user.dept === 'Engineering' ? key : null;
    })).then(keys => keys.filter(Boolean));

    console.log('Engineering users:', engineeringUserKeys);

  } catch (err) {
    console.error('Error in search operations:', err);
  }
}
```

### Version Control Operations

```javascript
// Using ChronDB-specific commands for version control
async function versionControlOperations() {
  try {
    // Create a document with initial version
    await client.set('doc:1', JSON.stringify({ title: 'Initial Document', content: 'Draft content' }));
    console.log('Document created');

    // Wait a moment to ensure different timestamps
    await new Promise(resolve => setTimeout(resolve, 1000));

    // Update the document
    await client.set('doc:1', JSON.stringify({
      title: 'Updated Document',
      content: 'Revised content',
      lastModified: new Date().toISOString()
    }));
    console.log('Document updated');

    // Get document history
    const history = await client.sendCommand(['CHRONDB.HISTORY', 'doc:1']);
    console.log('Document history:', history);

    // Get document at a specific time (using the first version's timestamp)
    if (history && history.length > 0) {
      const firstVersion = JSON.parse(history[0]);
      const oldDoc = await client.sendCommand([
        'CHRONDB.GETAT',
        'doc:1',
        firstVersion.timestamp
      ]);
      console.log('Original version:', oldDoc);

      // Compare versions
      if (history.length > 1) {
        const secondVersion = JSON.parse(history[1]);
        const diff = await client.sendCommand([
          'CHRONDB.DIFF',
          'doc:1',
          firstVersion.timestamp,
          secondVersion.timestamp
        ]);
        console.log('Changes between versions:', diff);
      }
    }

  } catch (err) {
    console.error('Error in version control operations:', err);
  }
}
```

### Branch Operations

```javascript
// Working with branches
async function branchOperations() {
  try {
    // List all branches
    const branches = await client.sendCommand(['CHRONDB.BRANCH.LIST']);
    console.log('Available branches:', branches);

    // Create a new branch
    await client.sendCommand(['CHRONDB.BRANCH.CREATE', 'feature-search']);
    console.log('Created feature-search branch');

    // Switch to the new branch
    await client.sendCommand(['CHRONDB.BRANCH.CHECKOUT', 'feature-search']);
    console.log('Switched to feature-search branch');

    // Create a document in the feature branch
    await client.set('search:config', JSON.stringify({
      indexFields: ['title', 'content', 'tags'],
      caseSensitive: false,
      maxResults: 100
    }));
    console.log('Created document in feature branch');

    // Switch back to main
    await client.sendCommand(['CHRONDB.BRANCH.CHECKOUT', 'main']);

    // Verify document doesn't exist in main
    const exists = await client.exists('search:config');
    console.log('Document exists in main branch:', exists === 1);

    // Merge feature branch into main
    await client.sendCommand(['CHRONDB.BRANCH.MERGE', 'feature-search', 'main']);
    console.log('Merged feature-search into main');

    // Verify document now exists in main
    const existsAfterMerge = await client.exists('search:config');
    console.log('Document exists in main after merge:', existsAfterMerge === 1);

  } catch (err) {
    console.error('Error in branch operations:', err);
  }
}
```

## Schema Validation Examples

ChronDB supports optional JSON Schema validation per namespace. See the [Schema Validation](/key-concepts/validation) documentation for full details.

### Validation with redis-cli

```bash
# Create a validation schema with strict mode
SCHEMA.SET users '{"type":"object","required":["id","email"],"properties":{"id":{"type":"string"},"email":{"type":"string","format":"email"},"name":{"type":"string"},"age":{"type":"integer","minimum":0}}}' MODE strict
# Output: OK

# Get a validation schema
SCHEMA.GET users
# Output: {"namespace":"users","version":1,"mode":"strict","schema":{...}}

# List all validation schemas
SCHEMA.LIST
# Output:
# 1) "users"
# 2) "products"

# Validate a document without saving (valid)
SCHEMA.VALIDATE users '{"id":"users:1","email":"john@example.com","name":"John"}'
# Output: {"valid":true,"errors":[]}

# Validate a document without saving (invalid - missing email)
SCHEMA.VALIDATE users '{"id":"users:1","name":"John"}'
# Output: {"valid":false,"errors":[{"path":"$.email","message":"required property 'email' not found","keyword":"required"}]}

# Try to save an invalid document (will fail in strict mode)
SET users:1 '{"id":"users:1","name":"John"}'
# Output: -ERR VALIDATION_ERROR users: required property 'email' not found at $.email

# Save a valid document
SET users:1 '{"id":"users:1","email":"john@example.com","name":"John"}'
# Output: OK

# Delete a validation schema
SCHEMA.DEL users
# Output: (integer) 1
```

### Validation with JavaScript

```javascript
// Schema validation operations using ChronDB via Redis protocol
async function schemaValidationOperations(client) {
  try {
    // Create a validation schema
    const schema = {
      type: 'object',
      required: ['id', 'email'],
      properties: {
        id: { type: 'string' },
        email: { type: 'string', format: 'email' },
        name: { type: 'string' },
        age: { type: 'integer', minimum: 0 }
      }
    };

    await client.sendCommand([
      'SCHEMA.SET',
      'users',
      JSON.stringify(schema),
      'MODE',
      'strict'
    ]);
    console.log('Schema created');

    // List all schemas
    const schemas = await client.sendCommand(['SCHEMA.LIST']);
    console.log('Available schemas:', schemas);

    // Get a specific schema
    const userSchema = await client.sendCommand(['SCHEMA.GET', 'users']);
    console.log('User schema:', JSON.parse(userSchema));

    // Validate a document before saving
    const validDoc = { id: 'users:1', email: 'john@example.com', name: 'John' };
    const validResult = await client.sendCommand([
      'SCHEMA.VALIDATE',
      'users',
      JSON.stringify(validDoc)
    ]);
    console.log('Validation result (valid):', JSON.parse(validResult));

    // Try validating an invalid document
    const invalidDoc = { id: 'users:2', name: 'Jane' }; // missing email
    const invalidResult = await client.sendCommand([
      'SCHEMA.VALIDATE',
      'users',
      JSON.stringify(invalidDoc)
    ]);
    console.log('Validation result (invalid):', JSON.parse(invalidResult));

    // Save a valid document
    await client.set('users:1', JSON.stringify(validDoc));
    console.log('Valid document saved');

    // Try to save an invalid document (will throw error in strict mode)
    try {
      await client.set('users:2', JSON.stringify(invalidDoc));
    } catch (err) {
      console.log('Expected validation error:', err.message);
    }

    // Delete the schema
    const deleted = await client.sendCommand(['SCHEMA.DEL', 'users']);
    console.log('Schema deleted:', deleted === 1);

  } catch (err) {
    console.error('Error in schema validation operations:', err);
  }
}
```

### Complete Example: Inventory Tracking System

```javascript
const redis = require('redis');

// Inventory Management System using ChronDB via Redis protocol
class InventorySystem {
  constructor() {
    this.client = redis.createClient({
      url: 'redis://localhost:6379'
    });
    this.client.on('error', err => console.error('Redis Client Error:', err));
  }

  async connect() {
    await this.client.connect();
    console.log('Connected to ChronDB');
    return this;
  }

  async disconnect() {
    await this.client.quit();
    console.log('Disconnected from ChronDB');
  }

  // Product management
  async addProduct(id, details) {
    const productKey = `product:${id}`;
    const productData = {
      ...details,
      createdAt: new Date().toISOString(),
      lastModified: new Date().toISOString()
    };

    await this.client.set(productKey, JSON.stringify(productData));
    return productData;
  }

  async getProduct(id) {
    const productKey = `product:${id}`;
    const data = await this.client.get(productKey);
    return data ? JSON.parse(data) : null;
  }

  async updateProduct(id, updates) {
    const productKey = `product:${id}`;
    const currentData = await this.getProduct(id);

    if (!currentData) {
      throw new Error(`Product ${id} not found`);
    }

    const updatedData = {
      ...currentData,
      ...updates,
      lastModified: new Date().toISOString()
    };

    await this.client.set(productKey, JSON.stringify(updatedData));
    return updatedData;
  }

  async deleteProduct(id) {
    const productKey = `product:${id}`;
    return await this.client.del(productKey);
  }

  // Inventory operations
  async adjustStock(id, adjustment) {
    const product = await this.getProduct(id);

    if (!product) {
      throw new Error(`Product ${id} not found`);
    }

    const currentStock = product.stock || 0;
    const newStock = currentStock + adjustment;

    if (newStock < 0) {
      throw new Error(`Insufficient stock for product ${id}`);
    }

    return this.updateProduct(id, { stock: newStock });
  }

  // Version control features
  async getProductHistory(id) {
    const productKey = `product:${id}`;
    return await this.client.sendCommand(['CHRONDB.HISTORY', productKey]);
  }

  async getProductAtTime(id, timestamp) {
    const productKey = `product:${id}`;
    const data = await this.client.sendCommand(['CHRONDB.GETAT', productKey, timestamp]);
    return data ? JSON.parse(data) : null;
  }

  // Searching inventory
  async findProducts(pattern) {
    const keys = await this.client.keys(pattern);
    if (keys.length === 0) return [];

    const productsData = await this.client.mGet(keys);
    return productsData.map(data => JSON.parse(data));
  }

  // Bulk operations with transactions
  async bulkUpdateStock(updates) {
    const multi = this.client.multi();

    for (const [id, adjustment] of Object.entries(updates)) {
      const productKey = `product:${id}`;
      const productData = await this.client.get(productKey);

      if (!productData) {
        throw new Error(`Product ${id} not found`);
      }

      const product = JSON.parse(productData);
      const currentStock = product.stock || 0;
      const newStock = currentStock + adjustment;

      if (newStock < 0) {
        throw new Error(`Insufficient stock for product ${id}`);
      }

      product.stock = newStock;
      product.lastModified = new Date().toISOString();

      multi.set(productKey, JSON.stringify(product));
    }

    return await multi.exec();
  }
}

// Usage example
async function runInventoryExample() {
  const inventory = await new InventorySystem().connect();

  try {
    // Add products
    await inventory.addProduct('1001', {
      name: 'Ergonomic Chair',
      category: 'Furniture',
      price: 299.99,
      stock: 20
    });

    await inventory.addProduct('1002', {
      name: 'Standing Desk',
      category: 'Furniture',
      price: 499.99,
      stock: 15
    });

    await inventory.addProduct('2001', {
      name: 'Wireless Keyboard',
      category: 'Electronics',
      price: 79.99,
      stock: 50
    });

    // Get a product
    const chair = await inventory.getProduct('1001');
    console.log('Product details:', chair);

    // Update a product
    await inventory.updateProduct('1001', { price: 279.99, featured: true });
    console.log('Product updated');

    // Record sales (reduce stock)
    await inventory.adjustStock('1001', -5);
    await inventory.adjustStock('2001', -10);
    console.log('Stock adjusted after sales');

    // Get product history
    const history = await inventory.getProductHistory('1001');
    console.log('Chair price history:', history);

    // Find all furniture products
    const furniture = await inventory.findProducts('product:*');
    const furnitureItems = furniture.filter(item => item.category === 'Furniture');
    console.log('Furniture products:', furnitureItems);

    // Bulk restock
    await inventory.bulkUpdateStock({
      '1001': 10,
      '1002': 5,
      '2001': 25
    });
    console.log('Bulk restock completed');

    // Check current stock levels
    const currentStock = await Promise.all([
      inventory.getProduct('1001'),
      inventory.getProduct('1002'),
      inventory.getProduct('2001')
    ]);

    console.log('Current stock levels:', currentStock.map(p => ({ id: p.id, name: p.name, stock: p.stock })));

  } catch (err) {
    console.error('Inventory system error:', err);
  } finally {
    await inventory.disconnect();
  }
}

// Run the example
runInventoryExample();
```


# PostgreSQL Protocol Examples

This document provides examples for using the ChronDB PostgreSQL protocol interface with psql and JavaScript clients.

## PostgreSQL Protocol Overview

ChronDB implements a subset of the PostgreSQL protocol, allowing you to connect using standard PostgreSQL clients and drivers. This makes it easy to integrate with existing applications that already use PostgreSQL or to leverage the SQL query capability.

The PostgreSQL protocol server can be configured in the `config.edn` file:

```clojure
:servers {
  :postgresql {
    :enabled true
    :host "0.0.0.0"
    :port 5432
    :username "chrondb"
    :password "chrondb"
  }
}
```

## Data Model Mapping

ChronDB maps its document structure to SQL tables based on document keys:

* Document keys with the format `collection:id` are mapped to tables
* The part before the colon becomes the table name
* The part after the colon becomes the row's ID
* Document fields become columns in the table

For example:

* A document with key `user:1` becomes a row in the `user` table with `id='1'`
* A document with key `product:xyz` becomes a row in the `product` table with `id='xyz'`

## Supported SQL Features

ChronDB supports the following SQL operations:

### DML (Data Manipulation Language)

* `SELECT` - Query documents
* `INSERT` - Create documents
* `UPDATE` - Update documents
* `DELETE` - Delete documents

### DDL (Data Definition Language)

**Note:** ChronDB is schemaless by default. `CREATE TABLE` is optional - you can insert data directly without defining a schema first.

* `CREATE TABLE` - Create table with explicit schema (optional)
* `CREATE TABLE IF NOT EXISTS` - Create table only if it doesn't exist
* `DROP TABLE` - Delete table schema
* `DROP TABLE IF EXISTS` - Delete table only if it exists
* `SHOW TABLES` - List all tables (both schemaless and schema-defined)
* `SHOW SCHEMAS` - List all schemas (Git branches)
* `DESCRIBE table` / `SHOW COLUMNS FROM table` - Show table structure (infers from data if no schema)

### Validation Schema DDL

ChronDB supports optional JSON Schema validation per namespace:

* `CREATE VALIDATION SCHEMA FOR namespace AS 'json-schema' MODE strict|warning` - Create validation schema
* `DROP VALIDATION SCHEMA FOR namespace` - Delete validation schema
* `SHOW VALIDATION SCHEMA FOR namespace` - Get validation schema
* `SHOW VALIDATION SCHEMAS` - List all validation schemas

## Special Functions

ChronDB provides special SQL functions to access version control features:

* `chrondb_history(table_name, id)` - Get document history
* `chrondb_at(table_name, id, timestamp)` - Get document at a point in time
* `chrondb_diff(table_name, id, t1, t2)` - Compare document versions
* `chrondb_branch_list()` - List branches
* `chrondb_branch_create(name)` - Create a new branch
* `chrondb_branch_checkout(name)` - Switch to a branch
* `chrondb_branch_merge(source, target)` - Merge branches

## Examples with psql

### Connecting to ChronDB

```bash
# Connect to ChronDB using psql
psql -h localhost -p 5432 -U chrondb -d chrondb
```

### Document Operations

#### Creating Documents (INSERT)

```sql
-- Create a user
INSERT INTO user (id, name, email, age, roles)
VALUES ('1', 'John Doe', 'john@example.com', 30, '["admin", "user"]');

-- Create another user
INSERT INTO user (id, name, email, age, roles)
VALUES ('2', 'Jane Smith', 'jane@example.com', 28, '["editor"]');

-- Create a product
INSERT INTO product (id, name, price, category, in_stock)
VALUES ('laptop1', 'Premium Laptop', 1299.99, 'Electronics', true);
```

#### Querying Documents (SELECT)

```sql
-- Get all users
SELECT * FROM user;

-- Get a specific user
SELECT * FROM user WHERE id = '1';

-- Get users with specific attributes
SELECT * FROM user WHERE age > 25;

-- Get specific fields
SELECT name, email FROM user;

-- Count documents
SELECT COUNT(*) FROM user;
```

#### Updating Documents (UPDATE)

```sql
-- Update a user
UPDATE user
SET email = 'john.doe@example.com', roles = '["admin", "user", "reviewer"]'
WHERE id = '1';

-- Update with JSON operations
UPDATE product
SET price = 1199.99, tags = '["sale", "featured"]'
WHERE id = 'laptop1';
```

#### Deleting Documents (DELETE)

```sql
-- Delete a specific user
DELETE FROM user WHERE id = '2';

-- Delete all products in a category
DELETE FROM product WHERE category = 'Discontinued';
```

### Version Control Operations

```sql
-- Get document history
SELECT * FROM chrondb_history('user', '1');

-- Get document at a specific point in time
SELECT * FROM chrondb_at('user', '1', '2023-10-10T09:15:00Z');

-- Compare document versions
SELECT * FROM chrondb_diff('user', '1',
                         '2023-10-10T09:15:00Z',
                         '2023-10-15T14:30:00Z');
```

### DDL Operations

```sql
-- Create a table with explicit schema
CREATE TABLE users (
    id TEXT PRIMARY KEY,
    name TEXT NOT NULL,
    email TEXT UNIQUE,
    age INTEGER,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

-- Create table only if it doesn't exist
CREATE TABLE IF NOT EXISTS products (
    id TEXT PRIMARY KEY,
    name TEXT NOT NULL,
    price INTEGER DEFAULT 0
);

-- List all tables
SHOW TABLES;

-- List tables from a specific schema (branch)
SHOW TABLES FROM feature_branch;

-- List all schemas (branches)
SHOW SCHEMAS;

-- Describe table structure
DESCRIBE users;
-- or
SHOW COLUMNS FROM users;

-- Drop a table
DROP TABLE users;

-- Drop table only if it exists
DROP TABLE IF EXISTS old_table;
```

### Schema Validation Operations

ChronDB supports optional JSON Schema validation per namespace. See the [Schema Validation](/key-concepts/validation) documentation for full details.

```sql
-- Create a validation schema with strict mode
CREATE VALIDATION SCHEMA FOR users AS '{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "required": ["id", "email"],
  "properties": {
    "id": { "type": "string" },
    "email": { "type": "string", "format": "email" },
    "name": { "type": "string" },
    "age": { "type": "integer", "minimum": 0 }
  },
  "additionalProperties": false
}' MODE STRICT;

-- Create a validation schema with warning mode (logs violations but allows writes)
CREATE VALIDATION SCHEMA FOR products AS '{
  "type": "object",
  "required": ["id", "name"],
  "properties": {
    "id": { "type": "string" },
    "name": { "type": "string" },
    "price": { "type": "number", "minimum": 0 }
  }
}' MODE WARNING;

-- List all validation schemas
SHOW VALIDATION SCHEMAS;

-- Get a specific validation schema
SHOW VALIDATION SCHEMA FOR users;

-- Drop a validation schema
DROP VALIDATION SCHEMA FOR users;
```

#### Validation Errors

When inserting invalid data with a strict schema:

```sql
-- This will fail because email is required
INSERT INTO users (id, name) VALUES ('1', 'John');
-- ERROR: Validation failed for table 'users': $.email - required property 'email' not found

-- This will fail because age must be >= 0
INSERT INTO users (id, email, name, age) VALUES ('1', 'john@example.com', 'John', -5);
-- ERROR: Validation failed for table 'users': $.age - must be >= 0

-- Valid insert
INSERT INTO users (id, email, name, age) VALUES ('1', 'john@example.com', 'John', 30);
-- INSERT 0 1
```

### Branch Operations

```sql
-- List all branches
SELECT * FROM chrondb_branch_list();

-- Create a new branch
SELECT * FROM chrondb_branch_create('feature-reporting');

-- Switch to a branch
SELECT * FROM chrondb_branch_checkout('feature-reporting');

-- Merge branches
SELECT * FROM chrondb_branch_merge('feature-reporting', 'main');

-- Create table in a specific branch/schema
CREATE TABLE feature_branch.new_feature_data (id TEXT);
```

### Advanced Queries

```sql
-- Join documents from different collections
SELECT u.name, u.email, o.total, o.status
FROM user u
JOIN order o ON o.user_id = u.id
WHERE o.status = 'pending';

-- Aggregation
SELECT category, COUNT(*) as product_count, AVG(price) as avg_price
FROM product
GROUP BY category
ORDER BY product_count DESC;

-- Filtering with JSON
SELECT * FROM user
WHERE roles::jsonb ? 'admin';
```

## Full-Text Search with to\_tsquery

ChronDB supports PostgreSQL-style full-text search using the familiar `@@` operator and `to_tsquery` function syntax. This allows you to perform efficient text searches across your documents with a PostgreSQL-compatible syntax.

### FTS Syntax

```sql
-- Basic full-text search using to_tsquery
SELECT * FROM products
WHERE name @@ to_tsquery('laptop');

-- Search for multiple words
SELECT * FROM articles
WHERE content @@ to_tsquery('database AND performance');

-- Search with wildcards
SELECT * FROM documents
WHERE description @@ to_tsquery('cloud*');
```

### How It Works

When you use the `field @@ to_tsquery('term')` syntax:

1. ChronDB parses the query and recognizes it as a full-text search condition
2. The search term is normalized (lowercase, accent removal)
3. If the term is short (less than 4 characters), wildcards are automatically added
4. The query is then passed to the underlying index (Lucene) for efficient search
5. Results are filtered to only include matches from the specified collection/table

### FTS Field Optimization

For better full-text search performance, you can create specialized FTS fields with the `_fts` suffix:

```sql
-- Insert a document with a dedicated FTS field
INSERT INTO articles (id, title, content, content_fts)
VALUES ('article:1', 'Introduction to Databases', 'Long article text...', 'optimized searchable content');

-- Search using the dedicated FTS field
SELECT * FROM articles
WHERE content_fts @@ to_tsquery('database');
```

When an indexed field ends with `_fts`, ChronDB will use it specifically for full-text search operations.

### Comparison with FTS\_MATCH

ChronDB also supports the `FTS_MATCH` function for backward compatibility:

```sql
-- Traditional FTS_MATCH syntax
SELECT * FROM products
WHERE FTS_MATCH(name, 'laptop');

-- Equivalent to_tsquery syntax
SELECT * FROM products
WHERE name @@ to_tsquery('laptop');
```

The `to_tsquery` approach is recommended as it:

* Follows standard PostgreSQL syntax
* Provides better compatibility with existing PostgreSQL tools
* Supports the same normalization and text processing features

### Index Implementation Details

Full-text search operations are powered by:

* Lucene index for efficient text search (when enabled in configuration)
* Automatic text normalization and accent handling
* Wildcard prefix matching for better search results
* Fallback to basic string matching with MemoryIndex implementation

For optimal performance:

* Enable the Lucene index in your configuration
* Use dedicated FTS fields with the `_fts` suffix for frequently searched content
* Utilize more specific search terms to reduce result sets

## Examples with JavaScript (node-postgres)

The following examples use the [node-postgres](https://node-postgres.com/) package, a popular PostgreSQL client for Node.js.

### Setting Up

```javascript
// Install the pg package:
// npm install pg

const { Pool } = require('pg');

// Create a connection pool
const pool = new Pool({
  host: 'localhost',
  port: 5432,
  user: 'chrondb',
  password: 'chrondb',
  database: 'chrondb'
});

// Helper function for queries
async function query(text, params) {
  const client = await pool.connect();
  try {
    const result = await client.query(text, params);
    return result.rows;
  } finally {
    client.release();
  }
}
```

### Document Operations

```javascript
// Create a document (INSERT)
async function createUser(id, userData) {
  const { name, email, age, roles } = userData;

  const text = `
    INSERT INTO user (id, name, email, age, roles)
    VALUES ($1, $2, $3, $4, $5)
    RETURNING *
  `;

  // Convert JavaScript array to JSON string
  const rolesJson = JSON.stringify(roles);
  const values = [id, name, email, age, rolesJson];

  const rows = await query(text, values);
  return rows[0];
}

// Get a document (SELECT)
async function getUser(id) {
  const text = 'SELECT * FROM user WHERE id = $1';
  const values = [id];

  const rows = await query(text, values);

  if (rows.length === 0) {
    return null;
  }

  // Parse JSON string fields back to JavaScript objects
  const user = rows[0];
  if (user.roles) {
    user.roles = JSON.parse(user.roles);
  }

  return user;
}

// Update a document (UPDATE)
async function updateUser(id, updates) {
  // First get the current document
  const user = await getUser(id);
  if (!user) {
    throw new Error(`User ${id} not found`);
  }

  // Prepare the update fields
  const fields = [];
  const values = [];
  let index = 1;

  for (const [key, value] of Object.entries(updates)) {
    let formattedValue = value;

    // Convert arrays or objects to JSON strings
    if (typeof value === 'object' && value !== null) {
      formattedValue = JSON.stringify(value);
    }

    fields.push(`${key} = $${index}`);
    values.push(formattedValue);
    index++;
  }

  // Add the timestamp and id
  fields.push(`updated_at = $${index}`);
  values.push(new Date().toISOString());
  index++;
  values.push(id);

  const text = `
    UPDATE user
    SET ${fields.join(', ')}
    WHERE id = $${index}
    RETURNING *
  `;

  const rows = await query(text, values);

  // Parse JSON fields
  const updatedUser = rows[0];
  if (updatedUser.roles) {
    updatedUser.roles = JSON.parse(updatedUser.roles);
  }

  return updatedUser;
}

// Delete a document (DELETE)
async function deleteUser(id) {
  const text = 'DELETE FROM user WHERE id = $1 RETURNING id';
  const values = [id];

  const rows = await query(text, values);
  return rows.length > 0;
}

// Search documents
async function searchUsers(conditions = {}, options = {}) {
  const { limit = 10, offset = 0, orderBy = 'id', order = 'ASC' } = options;

  const clauses = [];
  const values = [];
  let index = 1;

  // Build WHERE clauses
  for (const [key, value] of Object.entries(conditions)) {
    clauses.push(`${key} = $${index}`);
    values.push(value);
    index++;
  }

  const whereClause = clauses.length > 0 ? `WHERE ${clauses.join(' AND ')}` : '';

  const text = `
    SELECT * FROM user
    ${whereClause}
    ORDER BY ${orderBy} ${order}
    LIMIT $${index} OFFSET $${index + 1}
  `;

  values.push(limit, offset);

  const rows = await query(text, values);

  // Parse JSON fields
  return rows.map(user => {
    if (user.roles) {
      user.roles = JSON.parse(user.roles);
    }
    return user;
  });
}
```

### Version Control Operations

```javascript
// Get document history
async function getUserHistory(id) {
  const text = 'SELECT * FROM chrondb_history($1, $2)';
  const values = ['user', id];

  const rows = await query(text, values);

  // Parse JSON data in each version
  return rows.map(version => {
    return {
      timestamp: version.timestamp,
      data: JSON.parse(version.data)
    };
  });
}

// Get document at a point in time
async function getUserAtTime(id, timestamp) {
  const text = 'SELECT * FROM chrondb_at($1, $2, $3)';
  const values = ['user', id, timestamp];

  const rows = await query(text, values);

  if (rows.length === 0) {
    return null;
  }

  // Parse JSON fields
  const user = rows[0];
  if (user.roles) {
    user.roles = JSON.parse(user.roles);
  }

  return user;
}

// Compare document versions
async function compareUserVersions(id, timestamp1, timestamp2) {
  const text = 'SELECT * FROM chrondb_diff($1, $2, $3, $4)';
  const values = ['user', id, timestamp1, timestamp2];

  const rows = await query(text, values);

  if (rows.length === 0) {
    return null;
  }

  const diff = rows[0];

  // Parse JSON diff data
  return {
    added: JSON.parse(diff.added || '{}'),
    removed: JSON.parse(diff.removed || '{}'),
    changed: JSON.parse(diff.changed || '{}')
  };
}
```

### Branch Operations

```javascript
// List branches
async function listBranches() {
  const text = 'SELECT * FROM chrondb_branch_list()';
  const rows = await query(text);
  return rows.map(row => row.branch_name);
}

// Create a branch
async function createBranch(name) {
  const text = `SELECT * FROM chrondb_branch_create('${name}')`;
  const rows = await query(text);
  return rows[0].success === 'true';
}

// Switch to a branch
async function switchBranch(name) {
  const text = `SELECT * FROM chrondb_branch_checkout('${name}')`;
  const rows = await query(text);
  return rows[0].success === 'true';
}

// Merge branches
async function mergeBranches(source, target) {
  const text = `SELECT * FROM chrondb_branch_merge('${source}', '${target}')`;
  const rows = await query(text);
  return rows[0].success === 'true';
}
```

### Complete Example: Customer Order System

```javascript
const { Pool } = require('pg');
const pool = new Pool({
  host: 'localhost',
  port: 5432,
  user: 'chrondb',
  password: 'chrondb',
  database: 'chrondb'
});

// OrderSystem class
class OrderSystem {
  constructor() {
    this.pool = pool;
  }

  async query(text, params) {
    const client = await this.pool.connect();
    try {
      const result = await client.query(text, params);
      return result.rows;
    } finally {
      client.release();
    }
  }

  // Initialize the database with tables
  async initialize() {
    const tables = [
      `CREATE TABLE IF NOT EXISTS customer (
         id TEXT PRIMARY KEY,
         name TEXT,
         email TEXT,
         address TEXT,
         created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
       )`,
      `CREATE TABLE IF NOT EXISTS product (
         id TEXT PRIMARY KEY,
         name TEXT,
         price NUMERIC,
         description TEXT,
         stock INTEGER,
         created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
       )`,
      `CREATE TABLE IF NOT EXISTS order_item (
         id TEXT PRIMARY KEY,
         order_id TEXT,
         product_id TEXT,
         quantity INTEGER,
         price NUMERIC,
         created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
       )`,
      `CREATE TABLE IF NOT EXISTS customer_order (
         id TEXT PRIMARY KEY,
         customer_id TEXT,
         total NUMERIC,
         status TEXT,
         created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
         updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
       )`
    ];

    for (const table of tables) {
      await this.query(table);
    }

    console.log('Database initialized');
  }

  // Customer methods
  async createCustomer(id, data) {
    const { name, email, address } = data;

    const text = `
      INSERT INTO customer (id, name, email, address)
      VALUES ($1, $2, $3, $4)
      RETURNING *
    `;

    const values = [id, name, email, address];
    const rows = await this.query(text, values);
    return rows[0];
  }

  async getCustomer(id) {
    const text = 'SELECT * FROM customer WHERE id = $1';
    const rows = await this.query(text, [id]);
    return rows[0] || null;
  }

  async updateCustomer(id, updates) {
    const fields = [];
    const values = [];
    let index = 1;

    for (const [key, value] of Object.entries(updates)) {
      fields.push(`${key} = $${index}`);
      values.push(value);
      index++;
    }

    values.push(id);

    const text = `
      UPDATE customer
      SET ${fields.join(', ')}
      WHERE id = $${index}
      RETURNING *
    `;

    const rows = await this.query(text, values);
    return rows[0] || null;
  }

  // Product methods
  async createProduct(id, data) {
    const { name, price, description, stock } = data;

    const text = `
      INSERT INTO product (id, name, price, description, stock)
      VALUES ($1, $2, $3, $4, $5)
      RETURNING *
    `;

    const values = [id, name, price, description, stock];
    const rows = await this.query(text, values);
    return rows[0];
  }

  async getProduct(id) {
    const text = 'SELECT * FROM product WHERE id = $1';
    const rows = await this.query(text, [id]);
    return rows[0] || null;
  }

  async updateProductStock(id, adjustment) {
    const text = `
      UPDATE product
      SET stock = stock + $1
      WHERE id = $2
      RETURNING *
    `;

    const values = [adjustment, id];
    const rows = await this.query(text, values);
    return rows[0] || null;
  }

  // Order methods
  async createOrder(data) {
    const { id, customer_id, items } = data;

    // Start transaction
    const client = await this.pool.connect();

    try {
      await client.query('BEGIN');

      // Calculate total
      let total = 0;
      for (const item of items) {
        const product = await this.getProduct(item.product_id);
        if (!product) {
          throw new Error(`Product ${item.product_id} not found`);
        }

        if (product.stock < item.quantity) {
          throw new Error(`Insufficient stock for product ${item.product_id}`);
        }

        total += product.price * item.quantity;

        // Update stock
        await client.query(
          'UPDATE product SET stock = stock - $1 WHERE id = $2',
          [item.quantity, item.product_id]
        );

        // Create order item
        const orderItemId = `item:${id}-${item.product_id}`;
        await client.query(
          `INSERT INTO order_item (id, order_id, product_id, quantity, price)
           VALUES ($1, $2, $3, $4, $5)`,
          [orderItemId, id, item.product_id, item.quantity, product.price]
        );
      }

      // Create the order
      const orderResult = await client.query(
        `INSERT INTO customer_order (id, customer_id, total, status)
         VALUES ($1, $2, $3, $4)
         RETURNING *`,
        [id, customer_id, total, 'pending']
      );

      await client.query('COMMIT');
      return orderResult.rows[0];

    } catch (e) {
      await client.query('ROLLBACK');
      throw e;
    } finally {
      client.release();
    }
  }

  async getOrder(id) {
    const orderText = 'SELECT * FROM customer_order WHERE id = $1';
    const orderRows = await this.query(orderText, [id]);

    if (orderRows.length === 0) {
      return null;
    }

    const order = orderRows[0];

    // Get order items
    const itemsText = 'SELECT * FROM order_item WHERE order_id = $1';
    const items = await this.query(itemsText, [id]);

    return {
      ...order,
      items
    };
  }

  async updateOrderStatus(id, status) {
    const text = `
      UPDATE customer_order
      SET status = $1, updated_at = CURRENT_TIMESTAMP
      WHERE id = $2
      RETURNING *
    `;

    const values = [status, id];
    const rows = await this.query(text, values);
    return rows[0] || null;
  }

  // Reporting methods
  async getCustomerOrders(customerId) {
    const text = `
      SELECT co.*, c.name as customer_name, c.email
      FROM customer_order co
      JOIN customer c ON co.customer_id = c.id
      WHERE co.customer_id = $1
      ORDER BY co.created_at DESC
    `;

    return await this.query(text, [customerId]);
  }

  async getOrderHistory(orderId) {
    const text = 'SELECT * FROM chrondb_history($1, $2)';
    const rows = await this.query(text, ['customer_order', orderId]);

    return rows.map(version => ({
      timestamp: version.timestamp,
      data: JSON.parse(version.data)
    }));
  }

  async getOrderStatistics() {
    const text = `
      SELECT
        status,
        COUNT(*) as count,
        SUM(total) as total_value,
        MIN(total) as min_value,
        MAX(total) as max_value,
        AVG(total) as avg_value
      FROM customer_order
      GROUP BY status
    `;

    return await this.query(text);
  }
}

// Usage example
async function runOrderSystemExample() {
  const orderSystem = new OrderSystem();

  try {
    // Initialize database
    await orderSystem.initialize();

    // Create customers
    await orderSystem.createCustomer('cust1', {
      name: 'Alice Johnson',
      email: 'alice@example.com',
      address: '123 Main St, Anytown, CA'
    });

    await orderSystem.createCustomer('cust2', {
      name: 'Bob Smith',
      email: 'bob@example.com',
      address: '456 Oak St, Somewhere, NY'
    });

    // Create products
    await orderSystem.createProduct('prod1', {
      name: 'Mechanical Keyboard',
      price: 129.99,
      description: 'Mechanical keyboard with RGB lighting',
      stock: 50
    });

    await orderSystem.createProduct('prod2', {
      name: 'Wireless Mouse',
      price: 49.99,
      description: 'Ergonomic wireless mouse',
      stock: 100
    });

    await orderSystem.createProduct('prod3', {
      name: 'Monitor Stand',
      price: 79.99,
      description: 'Adjustable monitor stand',
      stock: 30
    });

    // Create an order
    const order = await orderSystem.createOrder({
      id: 'order1',
      customer_id: 'cust1',
      items: [
        { product_id: 'prod1', quantity: 1 },
        { product_id: 'prod2', quantity: 2 }
      ]
    });

    console.log('Created order:', order);

    // Get order details
    const orderDetails = await orderSystem.getOrder('order1');
    console.log('Order details:', orderDetails);

    // Update order status
    await orderSystem.updateOrderStatus('order1', 'shipped');
    console.log('Updated order status');

    // Create another order
    await orderSystem.createOrder({
      id: 'order2',
      customer_id: 'cust2',
      items: [
        { product_id: 'prod3', quantity: 1 },
        { product_id: 'prod1', quantity: 1 }
      ]
    });

    // Get order history
    const history = await orderSystem.getOrderHistory('order1');
    console.log('Order history:', history);

    // Get customer orders
    const customerOrders = await orderSystem.getCustomerOrders('cust1');
    console.log('Customer orders:', customerOrders);

    // Get order statistics
    const statistics = await orderSystem.getOrderStatistics();
    console.log('Order statistics:', statistics);

  } catch (err) {
    console.error('Error:', err);
  } finally {
    await pool.end();
  }
}

// Run the example
runOrderSystemExample();
```

## Schema Validation with JavaScript

ChronDB supports optional JSON Schema validation per namespace. See the [Schema Validation](/key-concepts/validation) documentation for full details.

```javascript
const { Pool } = require('pg');

const pool = new Pool({
  host: 'localhost',
  port: 5432,
  user: 'chrondb',
  password: 'chrondb',
  database: 'chrondb'
});

// Schema validation operations
class SchemaValidation {
  constructor() {
    this.pool = pool;
  }

  async query(text, params) {
    const client = await this.pool.connect();
    try {
      const result = await client.query(text, params);
      return result.rows;
    } finally {
      client.release();
    }
  }

  // Create a validation schema
  async createSchema(namespace, schema, mode = 'STRICT') {
    const schemaJson = JSON.stringify(schema);
    const text = `CREATE VALIDATION SCHEMA FOR ${namespace} AS '${schemaJson}' MODE ${mode}`;
    return await this.query(text);
  }

  // Get a validation schema
  async getSchema(namespace) {
    const text = `SHOW VALIDATION SCHEMA FOR ${namespace}`;
    return await this.query(text);
  }

  // List all validation schemas
  async listSchemas() {
    const text = 'SHOW VALIDATION SCHEMAS';
    return await this.query(text);
  }

  // Drop a validation schema
  async dropSchema(namespace) {
    const text = `DROP VALIDATION SCHEMA FOR ${namespace}`;
    return await this.query(text);
  }
}

// Usage example
async function schemaValidationExample() {
  const validation = new SchemaValidation();

  try {
    // Create a validation schema for users
    const userSchema = {
      '$schema': 'http://json-schema.org/draft-07/schema#',
      type: 'object',
      required: ['id', 'email'],
      properties: {
        id: { type: 'string' },
        email: { type: 'string', format: 'email' },
        name: { type: 'string' },
        age: { type: 'integer', minimum: 0 }
      },
      additionalProperties: false
    };

    await validation.createSchema('users', userSchema, 'STRICT');
    console.log('Validation schema created');

    // List all schemas
    const schemas = await validation.listSchemas();
    console.log('Validation schemas:', schemas);

    // Get the specific schema
    const schema = await validation.getSchema('users');
    console.log('User schema:', schema);

    // Try to insert valid data
    await validation.query(
      "INSERT INTO users (id, email, name, age) VALUES ($1, $2, $3, $4)",
      ['1', 'john@example.com', 'John', 30]
    );
    console.log('Valid document inserted');

    // Try to insert invalid data (will fail)
    try {
      await validation.query(
        "INSERT INTO users (id, name) VALUES ($1, $2)",
        ['2', 'Jane']
      );
    } catch (err) {
      console.log('Expected validation error:', err.message);
    }

    // Drop the schema
    await validation.dropSchema('users');
    console.log('Validation schema dropped');

  } catch (err) {
    console.error('Error:', err);
  } finally {
    await pool.end();
  }
}

schemaValidationExample();
```


# AST Query System

ChronDB uses an Abstract Syntax Tree (AST) to represent queries uniformly across all protocols (REST, Redis, SQL, CLI). This enables consistent query behavior and advanced features like pagination, sorting, and filtering.

## Overview

The AST query system provides:

* **Unified query representation** across all protocols
* **Advanced pagination** with cursor-based navigation
* **Flexible sorting** by any document field
* **Type-aware queries** for numeric, temporal, and geospatial data
* **Composable query building** with helper functions

## AST Structure

### Basic Query

```clojure
(require '[chrondb.query.ast :as ast])

;; Simple FTS query
(ast/query [(ast/fts "content" "search term")]
           {:limit 10
            :offset 0
            :branch "main"})
```

### Query Components

#### FTS Clause

Full-text search clause:

```clojure
(ast/fts "content" "search term")
(ast/fts "title" "Lucene" {:analyzer :fts})
```

#### Sort Descriptors

```clojure
(ast/sort-by "age" :asc)
(ast/sort-by "name" :desc)
(ast/sort-by "score" :desc)
```

#### Range Queries

```clojure
;; Numeric ranges
(ast/range-long "age" 18 65)
(ast/range-double "price" 10.0 100.0)

;; Date ranges
(ast/range-long "timestamp" start-ts end-ts)
```

#### Query Options

```clojure
(ast/query [clauses]
           {:limit 10           ; Max results
            :offset 0            ; Skip results
            :after cursor-map    ; searchAfter cursor for pagination
            :sort [sort-desc]    ; Sort descriptors
            :branch "main"})     ; Branch name
```

## Usage Examples

### REST API

```bash
# Basic search
GET /api/v1/search?q=Software

# With pagination
GET /api/v1/search?q=Software&limit=10&offset=0

# With sorting
GET /api/v1/search?q=Software&sort=age:asc,name:desc

# With cursor-based pagination
GET /api/v1/search?q=Software&limit=10&after=<base64-cursor>

# Structured AST query (EDN)
GET /api/v1/search?query={:clauses [{:type :fts :field "content" :value "Software"}]}
```

### Redis Protocol

```bash
# Basic search
SEARCH Software

# With options
SEARCH Software LIMIT 10 OFFSET 0 SORT age:asc BRANCH main

# Using FT.SEARCH alias
FT.SEARCH Software LIMIT 10
```

### CLI

```bash
# Basic search
chrondb search --q Software

# With options
chrondb search --q Software --limit 10 --offset 0 --sort age:asc --branch main

# Structured query
chrondb search --query '{:clauses [{:type :fts :field "content" :value "Software"}]}'
```

## Helper Functions

### Building Queries

```clojure
;; With pagination
(ast/with-pagination (ast/query [clauses]) {:limit 10 :offset 0})

;; With sorting
(ast/with-sort (ast/query [clauses]) [(ast/sort-by "age" :asc)])

;; With searchAfter cursor
(ast/with-search-after (ast/query [clauses]) {:doc 123 :score 1.5})
```

### Complex Queries

```clojure
;; Multiple clauses
(ast/query [(ast/fts "content" "Software")
            (ast/range-long "age" 25 65)]
           {:limit 10
            :sort [(ast/sort-by "age" :asc)
                   (ast/sort-by "score" :desc)]
            :branch "main"})
```

## Pagination

### Offset-based Pagination

```clojure
;; First page
(ast/query [clauses] {:limit 10 :offset 0})

;; Second page
(ast/query [clauses] {:limit 10 :offset 10})
```

### Cursor-based Pagination (searchAfter)

More efficient for large result sets:

```clojure
;; Initial query
(let [result (index/search-query index ast-query branch opts)
      next-cursor (:next-cursor result)]
  ;; Use next-cursor for next page
  (ast/query [clauses] {:limit 10 :after next-cursor}))
```

The `next-cursor` is a map with `:doc` (document ID) and `:score` (relevance score) that can be serialized for API responses.

## Sorting

Sort by multiple fields:

```clojure
(ast/query [clauses]
           {:sort [(ast/sort-by "age" :asc)
                   (ast/sort-by "name" :desc)]})
```

Sorting is applied in order: first by age ascending, then by name descending.

## Protocol Consistency

The AST ensures that queries behave identically across all protocols:

* **REST**: `/api/v1/search?q=term&sort=age:asc&limit=10`
* **Redis**: `SEARCH term SORT age:asc LIMIT 10`
* **CLI**: `chrondb search --q term --sort age:asc --limit 10`
* **Direct AST**: `(ast/query [(ast/fts "content" "term")] {:sort [(ast/sort-by "age" :asc)] :limit 10})`

All return the same results with the same ordering and pagination.

## Type Safety

The AST supports type-aware queries:

```clojure
;; Numeric ranges
(ast/range-long "age" 18 65)
(ast/range-double "price" 10.0 100.0)

;; These ensure proper Lucene numeric field handling
```

## Migration Notes

The AST system replaces the legacy query format:

**Legacy:**

```clojure
{:field "content" :value "term"}
```

**New AST:**

```clojure
(ast/fts "content" "term")
```

Both are supported during migration, but new code should use AST.


# Examples Overview

This document provides links to detailed examples for using the various ChronDB interfaces.

ChronDB supports multiple access protocols, allowing you to choose the interface that best suits your needs:

## [Clojure API](/example-applications/examples-clojure)

The native Clojure API offers the most direct way to interact with ChronDB, providing access to all features including CRUD operations, searching, version control, and branching.

```clojure
;; Basic example
(require '[chrondb.core :as chrondb])
(def db (chrondb/create-chrondb))
(chrondb/save db "user:1" {:name "Alice" :email "alice@example.com"})
```

[View complete Clojure API examples →](/example-applications/examples-clojure)

## [REST API](/connection-methods/examples-rest)

The REST API allows you to interact with ChronDB using HTTP requests, making it easy to integrate with any programming language or environment that can make HTTP requests.

```bash
# Create a document using curl
curl -X POST http://localhost:3000/api/v1/documents/user:1 \
  -H "Content-Type: application/json" \
  -d '{"name": "John Doe", "email": "john@example.com"}'
```

[View complete REST API examples →](/connection-methods/examples-rest)

## [Redis Protocol](/connection-methods/examples-redis)

ChronDB implements a subset of the Redis protocol, allowing you to connect using standard Redis clients.

```bash
# Using redis-cli
redis-cli -h localhost -p 6379
> SET user:1 '{"name":"John Doe","email":"john@example.com"}'
```

[View complete Redis protocol examples →](/connection-methods/examples-redis)

## [PostgreSQL Protocol](/connection-methods/examples-postgresql)

ChronDB implements a subset of the PostgreSQL protocol, allowing you to connect using standard PostgreSQL clients and leverage SQL query capabilities.

```sql
-- Create a document using psql
INSERT INTO user (id, name, email)
VALUES ('1', 'John Doe', 'john@example.com');
```

[View complete PostgreSQL protocol examples →](/connection-methods/examples-postgresql)

## Examples by Protocol

For more detailed examples, see:

* [Clojure API Examples](/example-applications/examples-clojure)
* [REST API Examples](/connection-methods/examples-rest)
* [Redis Protocol Examples](/connection-methods/examples-redis)
* [PostgreSQL Protocol Examples](/connection-methods/examples-postgresql)

## SQL History Functions

Here are examples of using the SQL history functions:

### Document History

To retrieve the complete history of a document:

```sql
-- View all changes to a user document
SELECT * FROM chrondb_history('user', '1');

-- Extract only specific history information
SELECT commit_id, timestamp FROM chrondb_history('user', '1');

-- Find the most recent 5 changes
SELECT * FROM chrondb_history('user', '1') LIMIT 5;
```

### Point-in-Time Document Access

To access documents at specific points in time:

```sql
-- View document as it was at a specific commit
SELECT * FROM chrondb_at('user', '1', 'abc123def456');

-- Extract specific fields from a historical version
SELECT name, email FROM chrondb_at('user', '1', 'abc123def456');

-- Compare with current version
SELECT
  current.name AS current_name,
  history.name AS previous_name
FROM
  user AS current,
  chrondb_at('user', '1', 'abc123def456') AS history
WHERE
  current.id = '1';
```

### Document Version Comparison

To compare different versions of a document:

```sql
-- Compare two versions of a document
SELECT * FROM chrondb_diff('user', '1', 'abc123def456', 'def456abc123');

-- Check only what fields changed
SELECT changed FROM chrondb_diff('user', '1', 'abc123def456', 'def456abc123');

-- Get compact view of changes
SELECT
  id,
  commit1,
  commit2,
  CASE
    WHEN added IS NULL THEN 'No additions'
    ELSE added
  END AS added_fields,
  CASE
    WHEN removed IS NULL THEN 'No removals'
    ELSE removed
  END AS removed_fields,
  CASE
    WHEN changed IS NULL THEN 'No changes'
    ELSE changed
  END AS modified_fields
FROM
  chrondb_diff('user', '1', 'abc123def456', 'def456abc123');
```

These examples demonstrate how to leverage ChronDB's history tracking capabilities using the SQL interface.

***

Choose the interface that best suits your needs and refer to the specific documentation for detailed examples and use cases.


# Clojure Examples

This document provides detailed examples for using the ChronDB native Clojure API, which is the most direct way to interact with ChronDB.

## Setup

First, add ChronDB as a dependency in your project:

### deps.edn

```clojure
{:deps {com.github.avelino/chrondb {:git/tag "v0.1.0"
                                     :git/sha "..."}}}
```

### Leiningen (project.clj)

```clojure
:dependencies [[com.github.avelino/chrondb "0.1.0"]]
```

## Basic Usage

### Creating a Database

```clojure
(ns my-app.core
  (:require [chrondb.core :as chrondb]))

;; Create with default configuration (in-memory database)
(def db (chrondb/create-chrondb))

;; Create with custom configuration
(def config {:git {:committer-name "My App"
                  :committer-email "app@example.com"}
             :storage {:data-dir "/path/to/storage"}})
(def db (chrondb/create-chrondb config))
```

### CRUD Operations

```clojure
;; Create/Update documents
(chrondb/save db "user:1" {:name "Alice"
                          :email "alice@example.com"
                          :roles ["admin"]})

(chrondb/save db "user:2" {:name "Bob"
                          :email "bob@example.com"
                          :roles ["user"]})

;; Read a document
(def user (chrondb/get db "user:1"))
(println user) ;; => {:name "Alice", :email "alice@example.com", :roles ["admin"]}

;; Update a document (merge with existing data)
(chrondb/save db "user:1" (assoc user :age 30))

;; Delete a document
(chrondb/delete db "user:2")
(println (chrondb/get db "user:2")) ;; => nil
```

## Searching

ChronDB uses Lucene for powerful search capabilities:

```clojure
;; Basic search by field
(def results (chrondb/search db "name:Alice"))

;; Search with multiple conditions
(def results (chrondb/search db "name:Alice AND age:[25 TO 35]"))

;; Wildcard search
(def results (chrondb/search db "email:*@example.com"))

;; Search with paging
(def page1 (chrondb/search db "roles:admin" {:limit 10 :offset 0}))
(def page2 (chrondb/search db "roles:admin" {:limit 10 :offset 10}))
```

## Version Control Features

### Working with Document History

```clojure
;; Get the history of a document
(def history (chrondb/history db "user:1"))
(println history)
;; => [{:timestamp "2023-05-10T14:30:00Z"
;;      :data {:name "Alice" :email "alice@example.com" :roles ["admin"]}}
;;     {:timestamp "2023-05-11T09:15:00Z"
;;      :data {:name "Alice" :email "alice@example.com" :roles ["admin"] :age 30}}]

;; Get a document at a specific point in time
(def old-version (chrondb/get-at db "user:1" "2023-05-10T14:30:00Z"))
(println old-version) ;; => Version without :age field

;; Compare document versions
(def diff (chrondb/diff db "user:1" "2023-05-10T14:30:00Z" "2023-05-11T09:15:00Z"))
(println diff) ;; => Shows what changed between versions
```

### Working with Branches

```clojure
;; Create a new branch for testing
(def test-db (chrondb/create-branch db "test"))

;; Make changes in the test branch
(chrondb/save test-db "user:1" {:name "Alice (Test)"
                               :email "alice.test@example.com"})

;; The changes are isolated to the test branch
(println (chrondb/get db "user:1")) ;; => Original version
(println (chrondb/get test-db "user:1")) ;; => Modified version

;; Switch the current database to a different branch
(def switched-db (chrondb/switch-branch db "test"))

;; Merge changes from one branch to another
(def merged-db (chrondb/merge-branch db "test" "main"))
```

## Transactions

ChronDB supports atomic transactions:

```clojure
;; Execute multiple operations atomically
(chrondb/with-transaction [db]
  (chrondb/save db "order:1" {:items [{:id "item1" :qty 2}]
                             :total 50.0
                             :status "pending"})

  ;; Update inventory
  (let [inventory (chrondb/get db "inventory:item1")]
    (chrondb/save db "inventory:item1"
                 (update inventory :stock - 2)))

  ;; Update user orders
  (let [user (chrondb/get db "user:1")]
    (chrondb/save db "user:1"
                 (update user :orders conj "order:1"))))

;; If any operation fails, all changes are rolled back
```

You can customise the transaction metadata written to Git notes via the `chrondb.transaction.core` helpers:

```clojure
(require '[chrondb.transaction.core :as tx])

(tx/with-transaction [db {:origin "cli"
                          :user "admin"
                          :flags ["migration"]
                          :metadata {:request "seed-dataset"}}]
  ;; Flags can be appended dynamically as context evolves
  (tx/add-flags! "bulk-load")
  (tx/merge-metadata! {:batch-size 3})

  (doseq [doc [{:id "product:1" :name "Laptop" :price 1200}
               {:id "product:2" :name "Phone" :price 800}
               {:id "product:3" :name "Tablet" :price 600}]]
    (chrondb/save db (:id doc) doc)))

;; Inspect the resulting Git notes with: git log --show-notes=chrondb
```

## Advanced Features

### Custom Hooks

```clojure
;; Register a pre-save hook for validation
(chrondb/register-hook db :pre-save
  (fn [doc]
    (when (and (contains? doc :email)
               (not (re-matches #".*@.*\..+" (:email doc))))
      (throw (ex-info "Invalid email format" {:doc doc})))
    doc))

;; Register a post-save hook for notifications
(chrondb/register-hook db :post-save
  (fn [doc]
    (println "Document saved:" doc)
    doc))
```

### Database Statistics and Maintenance

```clojure
;; Get database statistics
(def stats (chrondb/stats db))
(println stats)
;; => {:documents 100, :branches 2, :size "10MB", ...}

;; Perform database health check
(def health (chrondb/health-check db))
(println health)
;; => {:status "healthy", :issues []}

;; Create a backup
(chrondb/backup db "/path/to/backup.tar.gz")
```

## Integration with Other Systems

### Creating a REST API Server

```clojure
(ns my-app.server
  (:require [chrondb.core :as chrondb]
            [ring.adapter.jetty :as jetty]
            [compojure.core :refer [defroutes GET POST DELETE]]
            [compojure.route :as route]
            [ring.middleware.json :refer [wrap-json-response wrap-json-body]]
            [ring.util.response :refer [response]]))

(def db (chrondb/create-chrondb))

(defroutes app-routes
  (GET "/documents/:key" [key]
       (response (chrondb/get db key)))

  (POST "/documents/:key" [key :as req]
        (let [doc (:body req)]
          (chrondb/save db key doc)
          (response {:status "success"})))

  (DELETE "/documents/:key" [key]
          (chrondb/delete db key)
          (response {:status "success"}))

  (GET "/search" [q]
       (response (chrondb/search db q)))

  (route/not-found (response {:status "error" :message "Not found"})))

(def app
  (-> app-routes
      wrap-json-response
      (wrap-json-body {:keywords? true})))

(defn start-server []
  (jetty/run-jetty app {:port 3000 :join? false}))
```

## Performance Tips

* Use the appropriate branch strategy for your application
* For large datasets, consider indexing only the fields you search frequently
* Use transactions for operations that need to be atomic
* For read-heavy workloads, consider using a caching layer
* Monitor disk usage regularly, as historical data will grow over time
* Use the compact operation periodically to optimize storage


# Overview

ChronDB provides native bindings for multiple programming languages, all generated from a single Rust SDK.

## Architecture: One Codebase, Many Languages

Instead of maintaining separate implementations for each language, ChronDB uses the **Rust SDK as the single source of truth**. All complex logic — worker thread management, GraalVM isolate lifecycle, idle timeout, lock cleanup, shared registry — lives exclusively in Rust. Other language bindings are thin, auto-generated wrappers.

```
libchrondb.so (GraalVM/Clojure — core database engine)
└── Rust SDK (crate chrondb — all logic lives here)
    │
    ├── UniFFI  → Python, Ruby, Kotlin, Swift
    ├── NAPI-RS → Node.js
    ├── dart:ffi → Dart/Flutter
    ├── cgo     → Go
    └── Native  → Rust (direct crate dependency)
```

### Why this approach?

Every feature or bug fix is implemented **once in Rust** and automatically available in all languages. No more reimplementing idle timeout logic in Python, Ruby, and Node.js separately.

### UniFFI (Mozilla)

[UniFFI](https://github.com/mozilla/uniffi-rs) is an open-source tool created by Mozilla for generating multi-language bindings from Rust. It powers production components in **Firefox** — the same tool that generates Kotlin/Swift bindings for Firefox's Rust networking, cryptography, and sync engines is what generates ChronDB's Python and Ruby bindings.

Key references:

* [UniFFI repository](https://github.com/mozilla/uniffi-rs)
* [UniFFI user guide](https://mozilla.github.io/uniffi-rs/)
* [Mozilla's blog on shipping Rust in Firefox](https://hacks.mozilla.org/2019/02/rewriting-a-browser-component-in-rust/)
* [Application Services](https://github.com/mozilla/application-services) — Mozilla's production project using UniFFI to ship Rust code to Kotlin, Swift, and Python

### NAPI-RS

[NAPI-RS](https://napi.rs/) generates native Node.js addons from Rust. Used in production by projects like [SWC](https://swc.rs/) (the JavaScript/TypeScript compiler), [Rspack](https://rspack.dev/), and [Biome](https://biomejs.dev/).

Key references:

* [NAPI-RS repository](https://github.com/napi-rs/napi-rs)
* [NAPI-RS documentation](https://napi.rs/docs/introduction)

## Available Bindings

| Language                                                                           | Tool         | Status | Package                |
| ---------------------------------------------------------------------------------- | ------------ | ------ | ---------------------- |
| [Rust](/language-bindings/rust)                                                    | Native crate | Stable | `chrondb` on crates.io |
| [Python](/language-bindings/python)                                                | UniFFI       | Stable | `chrondb` on PyPI      |
| [Ruby](/language-bindings/ruby)                                                    | UniFFI       | Stable | `chrondb` gem          |
| [Node.js](/language-bindings/nodejs)                                               | NAPI-RS      | Stable | `chrondb` on npm       |
| [Kotlin](/language-bindings/kotlin)                                                | UniFFI       | Stable | GitHub Release tarball |
| [Swift](/language-bindings/swift)                                                  | UniFFI       | Stable | GitHub Release tarball |
| [Dart/Flutter](https://github.com/avelino/chrondb/blob/main/docs/bindings/dart.md) | dart:ffi     | Stable | GitHub Release tarball |
| [Go](https://github.com/avelino/chrondb/blob/main/docs/bindings/go.md)             | cgo          | Stable | GitHub Release tarball |

## API Consistency

All bindings expose the same core API:

| Operation                                             | Description                                     |
| ----------------------------------------------------- | ----------------------------------------------- |
| `open_path(db_path)`                                  | Open a database connection (preferred)          |
| `open(data_path, index_path)`                         | Open with separate paths *(deprecated)*         |
| `open_with_idle_timeout(data_path, index_path, secs)` | Open with idle timeout                          |
| `put(id, doc, branch?)`                               | Save a document                                 |
| `get(id, branch?)`                                    | Get a document by ID                            |
| `delete(id, branch?)`                                 | Delete a document                               |
| `list_by_prefix(prefix, branch?)`                     | List documents by ID prefix                     |
| `list_by_table(table, branch?)`                       | List documents by table                         |
| `history(id, branch?)`                                | Get change history                              |
| `query(query, branch?)`                               | Execute a Lucene query                          |
| `execute_sql(sql, branch?)`                           | Execute a SQL query directly (no server needed) |

Each language wrapper adds idiomatic conveniences (context managers in Python, keyword arguments in Ruby, TypeScript types in Node.js) while the core behavior is identical.

## Building Bindings from Source

All bindings require the `libchrondb` native library (GraalVM native-image) and the Rust toolchain.

```bash
# 1. Build the GraalVM native library
clojure -M:shared-lib
native-image @target/shared-image-args

# 2. Build the Rust SDK
cd bindings/rust && cargo build --release

# 3. Generate UniFFI bindings (Python, Ruby, Kotlin, Swift)
cd bindings/uniffi && cargo build --release
cargo run --bin uniffi-bindgen generate \
  --library target/release/libchrondb_uniffi.dylib \
  --language python --language ruby --language kotlin --language swift \
  --out-dir generated/

# 4. Build Node.js binding
cd bindings/node && npm run build
```


# Rust

Rust client for ChronDB, a time-traveling key/value database built on Git architecture.

## Requirements

* Rust 1.56+ (2021 edition)
* `libclang` (for bindgen to generate FFI bindings)

## Stack Size (Handled Automatically)

ChronDB uses GraalVM native-image with Apache Lucene and JGit, which require deep call stacks (\~64MB).

**This is handled automatically.** The Rust binding spawns a dedicated worker thread with a 64MB stack for all FFI operations. You do not need to configure `RUST_MIN_STACK`, `ulimit`, or any other stack settings.

The worker thread architecture also provides:

* Consistent stack size across all platforms
* No special CI configuration required
* Thread-safe operation via message passing

## Installation

Add `chrondb` from [crates.io](https://crates.io/crates/chrondb):

```bash
cargo add chrondb
```

Or add it directly to your `Cargo.toml`:

```toml
[dependencies]
chrondb = "*"
serde_json = "1"
```

### Native shared library

The crate requires the `libchrondb` native library at runtime. Download it from the [latest GitHub release](https://github.com/avelino/chrondb/releases/tag/latest):

**macOS (Apple Silicon):**

```bash
curl -L https://github.com/avelino/chrondb/releases/download/latest/libchrondb-latest-macos-aarch64.tar.gz | tar xz
```

**Linux (x86\_64):**

```bash
curl -L https://github.com/avelino/chrondb/releases/download/latest/libchrondb-latest-linux-x86_64.tar.gz | tar xz
```

### Configure the runtime library path

The shared library must be discoverable at runtime:

**Linux:**

```bash
export LD_LIBRARY_PATH=/path/to/chrondb-rust/lib:$LD_LIBRARY_PATH
```

**macOS:**

```bash
export DYLD_LIBRARY_PATH=/path/to/chrondb-rust/lib:$DYLD_LIBRARY_PATH
```

### Library path (advanced)

To override the library location at build time, set `CHRONDB_LIB_DIR`:

```bash
export CHRONDB_LIB_DIR=/path/to/dir/with/libchrondb
```

## Quick Start

```rust
use chrondb::ChronDB;
use serde_json::json;

fn main() -> chrondb::Result<()> {
    // Single path (preferred)
    let db = ChronDB::open_path("./mydb")?;

    // Save a document
    db.put("user:1", &json!({"name": "Alice", "age": 30}), None)?;

    // Retrieve it
    let doc = db.get("user:1", None)?;
    println!("{}", doc); // {"name":"Alice","age":30}

    Ok(())
    // db is automatically closed via Drop
}
```

> **Legacy API (deprecated):** `ChronDB::open("/tmp/data", "/tmp/index")` still works but is deprecated. Use `open_path` instead.

## API Reference

### `ChronDB::open_path(db_path) -> Result<ChronDB>`

Opens a database connection using a single directory path. Data and index are stored in subdirectories automatically.

| Parameter | Type   | Description                                          |
| --------- | ------ | ---------------------------------------------------- |
| `db_path` | `&str` | Path for the database (data and index stored inside) |

**Returns:** `Result<ChronDB>`

**Errors:** `IsolateCreationFailed`, `OpenFailed(reason)`

***

### `ChronDB::open(data_path, index_path) -> Result<ChronDB>` *(deprecated)*

> **Deprecated.** Use `open_path` instead. This method still works but will be removed in a future release.

Opens a database connection with separate data and index paths.

| Parameter    | Type   | Description                                |
| ------------ | ------ | ------------------------------------------ |
| `data_path`  | `&str` | Path for the Git repository (data storage) |
| `index_path` | `&str` | Path for the Lucene index                  |

**Returns:** `Result<ChronDB>`

**Errors:** `IsolateCreationFailed`, `OpenFailed(reason)`

***

### `ChronDB::builder(data_path, index_path) -> ChronDBBuilder`

Creates a builder for opening a database with custom options like idle timeout.

| Parameter    | Type   | Description                                |
| ------------ | ------ | ------------------------------------------ |
| `data_path`  | `&str` | Path for the Git repository (data storage) |
| `index_path` | `&str` | Path for the Lucene index                  |

**Returns:** `ChronDBBuilder`

#### `ChronDBBuilder::idle_timeout(duration) -> Self`

Sets the idle timeout for the GraalVM isolate. When no operations happen for the specified duration, the isolate is automatically closed to free CPU and memory. The next operation transparently reopens it.

| Parameter  | Type       | Description                                 |
| ---------- | ---------- | ------------------------------------------- |
| `duration` | `Duration` | Max idle time before suspending the isolate |

#### `ChronDBBuilder::open() -> Result<ChronDB>`

Opens the database with the configured options.

**Errors:** `IsolateCreationFailed`, `OpenFailed(reason)`

#### Example

```rust
use chrondb::ChronDB;
use std::time::Duration;

let db = ChronDB::builder("/tmp/data", "/tmp/index")
    .idle_timeout(Duration::from_secs(120))
    .open()?;

// Use normally — isolate suspends after 120s of inactivity,
// reopens transparently on next operation.
db.put("user:1", &json!({"name": "Alice"}), None)?;
```

***

### `put(&self, id, doc, branch) -> Result<serde_json::Value>`

Saves a document.

| Parameter | Type                 | Description                      |
| --------- | -------------------- | -------------------------------- |
| `id`      | `&str`               | Document ID (e.g., `"user:1"`)   |
| `doc`     | `&serde_json::Value` | Document data as JSON value      |
| `branch`  | `Option<&str>`       | Branch name (`None` for default) |

**Returns:** The saved document as `serde_json::Value`.

**Errors:** `OperationFailed(reason)`

***

### `get(&self, id, branch) -> Result<serde_json::Value>`

Retrieves a document by ID.

| Parameter | Type           | Description |
| --------- | -------------- | ----------- |
| `id`      | `&str`         | Document ID |
| `branch`  | `Option<&str>` | Branch name |

**Returns:** The document as `serde_json::Value`.

**Errors:** `NotFound`, `OperationFailed(reason)`

***

### `delete(&self, id, branch) -> Result<()>`

Deletes a document by ID.

| Parameter | Type           | Description |
| --------- | -------------- | ----------- |
| `id`      | `&str`         | Document ID |
| `branch`  | `Option<&str>` | Branch name |

**Returns:** `Ok(())` on success.

**Errors:** `NotFound`, `OperationFailed(reason)`

***

### `list_by_prefix(&self, prefix, branch) -> Result<serde_json::Value>`

Lists documents whose IDs start with the given prefix.

| Parameter | Type           | Description        |
| --------- | -------------- | ------------------ |
| `prefix`  | `&str`         | ID prefix to match |
| `branch`  | `Option<&str>` | Branch name        |

**Returns:** JSON array of matching documents (empty array if none).

***

### `list_by_table(&self, table, branch) -> Result<serde_json::Value>`

Lists all documents in a table.

| Parameter | Type           | Description |
| --------- | -------------- | ----------- |
| `table`   | `&str`         | Table name  |
| `branch`  | `Option<&str>` | Branch name |

**Returns:** JSON array of matching documents (empty array if none).

***

### `history(&self, id, branch) -> Result<serde_json::Value>`

Returns the change history of a document.

| Parameter | Type           | Description |
| --------- | -------------- | ----------- |
| `id`      | `&str`         | Document ID |
| `branch`  | `Option<&str>` | Branch name |

**Returns:** JSON array of history entries (empty array if none).

***

### `query(&self, query, branch) -> Result<serde_json::Value>`

Executes a query against the Lucene index.

| Parameter | Type                 | Description                |
| --------- | -------------------- | -------------------------- |
| `query`   | `&serde_json::Value` | Query in Lucene AST format |
| `branch`  | `Option<&str>`       | Branch name                |

**Returns:** Results object with `results`, `total`, `limit`, `offset`.

**Errors:** `OperationFailed(reason)`

***

### `execute_sql(&self, sql, branch) -> Result<serde_json::Value>`

Executes a SQL query directly against the database without needing a running server.

| Parameter | Type           | Description                      |
| --------- | -------------- | -------------------------------- |
| `sql`     | `&str`         | SQL query string                 |
| `branch`  | `Option<&str>` | Branch name (`None` for default) |

**Returns:** Result object with `type`, `columns`, `rows`, `count`.

**Errors:** `OperationFailed(reason)`

***

### `setup_remote(&self, remote_url) -> Result<serde_json::Value>`

Configures a remote Git repository URL for syncing.

| Parameter    | Type   | Description                                            |
| ------------ | ------ | ------------------------------------------------------ |
| `remote_url` | `&str` | Remote Git URL (e.g., `"git@github.com:org/repo.git"`) |

**Returns:** `{"type":"ok","remote_url":"..."}` on success.

**Errors:** `OperationFailed(reason)`

***

### `push(&self) -> Result<serde_json::Value>`

Pushes local changes to the configured remote repository.

**Returns:** `{"type":"ok","status":"pushed"}` on success, `{"type":"ok","status":"skipped"}` if no remote is configured.

**Errors:** `OperationFailed(reason)`

***

### `pull(&self) -> Result<serde_json::Value>`

Pulls changes from the configured remote repository. Fetches and fast-forwards the local branch.

**Returns:** `{"type":"ok","status":"pulled|current|skipped|conflict"}`.

**Errors:** `OperationFailed(reason)`

***

### `fetch(&self) -> Result<serde_json::Value>`

Fetches changes from the configured remote without merging.

**Returns:** `{"type":"ok","status":"fetched|skipped"}`.

**Errors:** `OperationFailed(reason)`

***

### `remote_status(&self) -> Result<serde_json::Value>`

Returns whether a remote repository is configured.

**Returns:** `{"type":"ok","configured":true|false}`.

**Errors:** `OperationFailed(reason)`

***

### `last_error(&self) -> Option<String>`

Returns the last error message from the native library, if any.

***

### `Drop`

The `ChronDB` struct implements `Drop`. When it goes out of scope, the database connection is closed and the GraalVM isolate is torn down automatically.

## Document ID Convention

Documents use the format `table:id`:

```rust
db.put("user:123", &json!({"name": "Alice"}), None)?;
db.put("order:456", &json!({"total": 99.90}), None)?;
```

Internally, `user:123` is stored as `user/user_COLON_123.json` in the Git repository.

Use `list_by_table("user")` to retrieve all documents in the `user` table.

## Error Handling

### `ChronDBError` Enum

```rust
pub enum ChronDBError {
    IsolateCreationFailed,   // GraalVM isolate could not be created
    OpenFailed(String),      // Database failed to open (with reason)
    CloseFailed,             // Database failed to close
    NotFound,                // Document does not exist
    OperationFailed(String), // Operation failed (with reason)
    JsonError(String),       // JSON serialization/deserialization error
}
```

All variants implement `Display` and `std::error::Error`.

The crate also provides a type alias:

```rust
pub type Result<T> = std::result::Result<T, ChronDBError>;
```

### Conversion

`serde_json::Error` is automatically converted to `ChronDBError::JsonError` via `From`.

### Example

```rust
use chrondb::{ChronDB, ChronDBError};

fn main() {
    let db = ChronDB::open_path("./mydb").unwrap();

    match db.get("user:999", None) {
        Ok(doc) => println!("Found: {}", doc),
        Err(ChronDBError::NotFound) => println!("Document does not exist"),
        Err(e) => eprintln!("Error: {}", e),
    }
}
```

## Examples

### Full CRUD

```rust
use chrondb::ChronDB;
use serde_json::json;

fn main() -> chrondb::Result<()> {
    let db = ChronDB::open_path("./mydb")?;

    // Create
    db.put("user:1", &json!({"name": "Alice", "email": "alice@example.com"}), None)?;
    db.put("user:2", &json!({"name": "Bob", "email": "bob@example.com"}), None)?;

    // Read
    let alice = db.get("user:1", None)?;

    // Update
    let mut updated = alice.clone();
    updated["age"] = json!(30);
    db.put("user:1", &updated, None)?;

    // Delete
    db.delete("user:2", None)?;

    // List by table
    let users = db.list_by_table("user", None)?;
    println!("Users: {}", users);

    // List by prefix
    let matched = db.list_by_prefix("user:1", None)?;
    println!("Matched: {}", matched);

    Ok(())
}
```

### Query

```rust
use chrondb::ChronDB;
use serde_json::json;

fn main() -> chrondb::Result<()> {
    let db = ChronDB::open_path("./mydb")?;

    db.put("product:1", &json!({"name": "Laptop", "price": 999}), None)?;
    db.put("product:2", &json!({"name": "Mouse", "price": 29}), None)?;

    let results = db.query(&json!({
        "type": "term",
        "field": "name",
        "value": "Laptop"
    }), None)?;

    println!("Total: {}", results["total"]); // 1

    Ok(())
}
```

### History (Time Travel)

```rust
use chrondb::ChronDB;
use serde_json::json;

fn main() -> chrondb::Result<()> {
    let db = ChronDB::open_path("./mydb")?;

    db.put("config:app", &json!({"version": "1.0"}), None)?;
    db.put("config:app", &json!({"version": "2.0"}), None)?;

    let entries = db.history("config:app", None)?;
    println!("History: {}", entries);

    Ok(())
}
```

### SQL Queries

Execute SQL queries directly without needing a running server:

```rust
use chrondb::ChronDB;
use serde_json::json;

fn main() -> chrondb::Result<()> {
    let db = ChronDB::open_path("./mydb")?;

    db.put("user:1", &json!({"name": "Alice", "age": 30}), None)?;
    db.put("user:2", &json!({"name": "Bob", "age": 25}), None)?;

    let result = db.execute_sql("SELECT * FROM user", None)?;
    println!("Columns: {}", result["columns"]);
    println!("Count: {}", result["count"]);

    let result = db.execute_sql("SELECT * FROM user WHERE name = 'Alice'", None)?;
    println!("Rows: {}", result["rows"]);

    Ok(())
}
```

### Remote Sync

```rust
use chrondb::ChronDB;
use serde_json::json;

fn main() -> chrondb::Result<()> {
    let db = ChronDB::open_path("./mydb")?;

    // Configure remote
    db.setup_remote("git@github.com:org/data.git")?;

    // Write data locally
    db.put("user:1", &json!({"name": "Alice"}), None)?;

    // Push to remote
    let result = db.push()?;
    println!("Push: {}", result["status"]); // "pushed"

    // Pull latest from remote
    let result = db.pull()?;
    println!("Pull: {}", result["status"]); // "pulled" or "current"

    // Check remote status
    let status = db.remote_status()?;
    println!("Remote configured: {}", status["configured"]); // true

    Ok(())
}
```

### Using with `Drop` (automatic cleanup)

```rust
use chrondb::ChronDB;
use serde_json::json;

fn do_work() -> chrondb::Result<()> {
    let db = ChronDB::open_path("./mydb")?;
    db.put("temp:1", &json!({"data": "value"}), None)?;
    Ok(())
    // db is dropped here, closing the connection
}
```

### Idle Timeout (long-running services)

ChronDB loads a GraalVM native-image shared library whose internal threads consume CPU even when no operations are in flight. For long-running services with sporadic database access, use `idle_timeout` to automatically suspend the isolate when idle:

```rust
use chrondb::ChronDB;
use serde_json::json;
use std::time::Duration;

fn main() -> chrondb::Result<()> {
    // Isolate suspends after 2 minutes of inactivity
    let db = ChronDB::builder("/tmp/data", "/tmp/index")
        .idle_timeout(Duration::from_secs(120))
        .open()?;

    // Normal usage — no change in API
    db.put("audit:1", &json!({"action": "login"}), None)?;
    let doc = db.get("audit:1", None)?;

    // After 120s without operations, the GraalVM isolate is torn down.
    // The next call to put/get/query transparently reopens it.

    Ok(())
}
```

**When to use:**

* Long-running daemons that write to ChronDB intermittently (e.g., audit logging, MCP servers)
* Services where memory/CPU usage matters during idle periods

**When NOT to use:**

* Short-lived CLI tools (just use `ChronDB::open`)
* High-throughput services with constant database access (the isolate would never go idle)

## Export & Backup

### Export to Directory

```rust
// Export current state to filesystem
let result = db.export_to_directory("/tmp/export", None)?;
println!("Exported {} files", result["files_exported"]);

// Export with options
let opts = serde_json::json!({
    "branch": "main",
    "prefix": "users",
    "format": "json",
    "decode_paths": true,
    "overwrite": true
});
let result = db.export_to_directory("/tmp/export", Some(&opts.to_string()))?;
```

### Backup & Restore

```rust
// Create a full backup
let result = db.create_backup("/backups/full.tar.gz", None)?;
println!("Backup at: {}", result["path"]);

// Create a bundle backup
let opts = r#"{"format":"bundle"}"#;
let result = db.create_backup("/backups/full.bundle", Some(opts))?;

// Restore from backup
let result = db.restore_backup("/backups/full.tar.gz", None)?;

// Export/import git bundle snapshots
let result = db.export_snapshot("/backups/main.bundle", None)?;
let result = db.import_snapshot("/backups/main.bundle", None)?;
```

## Building from Source

```bash
# 1. Build the shared library (requires Java 11+ and GraalVM)
cd chrondb/
clojure -M:shared

# 2. Build the Rust binding
cd bindings/rust/
CHRONDB_LIB_DIR=../../target cargo build

# 3. Run tests
CHRONDB_LIB_DIR=../../target \
  LD_LIBRARY_PATH=../../target \
  cargo test
```

### `build.rs` Behavior

The build script (`build.rs`):

1. Reads `CHRONDB_LIB_DIR` (defaults to `../../target`)
2. Configures `rustc-link-search` and `rustc-link-lib=dylib=chrondb`
3. If `libchrondb.h` and `graal_isolate.h` exist in that directory, generates FFI bindings via `bindgen`
4. Otherwise, uses stub bindings (`src/ffi_stub.rs`) to allow compilation without the native library


# Python

Python client for ChronDB, auto-generated from the Rust SDK via [UniFFI](https://github.com/mozilla/uniffi-rs).

## Requirements

* Python 3.8+

## Installation

Install directly from the [latest GitHub release](https://github.com/avelino/chrondb/releases/tag/latest):

**macOS (Apple Silicon):**

```bash
pip install https://github.com/avelino/chrondb/releases/download/latest/chrondb-latest-py3-none-macosx_14_0_arm64.whl
```

**Linux (x86\_64):**

```bash
pip install https://github.com/avelino/chrondb/releases/download/latest/chrondb-latest-py3-none-manylinux_2_35_x86_64.whl
```

The wheel already includes the native shared library — no extra configuration needed.

### Library path (advanced)

If you need to override the bundled library (e.g., using a custom build), the binding searches in this order:

1. `CHRONDB_LIB_PATH` — full path to the library file
2. `CHRONDB_LIB_DIR` — directory containing the library
3. Bundled `lib/` inside the package
4. System paths: `/usr/local/lib`, `/usr/lib`

```bash
export CHRONDB_LIB_PATH=/path/to/libchrondb.so
```

## Quick Start

```python
from chrondb import ChronDB

# Single path (preferred)
with ChronDB("./mydb") as db:
    # Save a document
    db.put("user:1", {"name": "Alice", "age": 30})

    # Retrieve it
    doc = db.get("user:1")
    print(doc)  # {"name": "Alice", "age": 30}
```

> **Legacy API (deprecated):** `ChronDB("/tmp/data", "/tmp/index")` still works but is deprecated. Use the single-path form instead.

## API Reference

### `ChronDB(db_path: str, idle_timeout: float = None)`

Opens a database connection using a single directory path. Data and index are stored in subdirectories automatically.

| Parameter      | Type              | Description                                                                                                           |
| -------------- | ----------------- | --------------------------------------------------------------------------------------------------------------------- |
| `db_path`      | `str`             | Path for the database (data and index stored inside)                                                                  |
| `idle_timeout` | `Optional[float]` | Seconds of inactivity before suspending the GraalVM isolate. `None` (default) keeps it alive for the entire lifetime. |

When `idle_timeout` is set, the GraalVM isolate is automatically closed after the specified seconds of inactivity, freeing CPU and memory. The next operation transparently reopens it. See [Idle Timeout](#idle-timeout-long-running-services) for details.

**Raises:** `ChronDBError` if the database cannot be opened.

Implements the context manager protocol (`__enter__` / `__exit__`), calling `close()` automatically on exit.

#### Legacy: `ChronDB(data_path: str, index_path: str, idle_timeout: float = None)`

> **Deprecated.** The two-path constructor still works but is deprecated. Use the single-path form above.

| Parameter      | Type              | Description                                                 |
| -------------- | ----------------- | ----------------------------------------------------------- |
| `data_path`    | `str`             | Path for the Git repository (data storage)                  |
| `index_path`   | `str`             | Path for the Lucene index                                   |
| `idle_timeout` | `Optional[float]` | Seconds of inactivity before suspending the GraalVM isolate |

***

### `put(id, doc, branch=None) -> Dict[str, Any]`

Saves a document.

| Parameter | Type             | Description                      |
| --------- | ---------------- | -------------------------------- |
| `id`      | `str`            | Document ID (e.g., `"user:1"`)   |
| `doc`     | `Dict[str, Any]` | Document data                    |
| `branch`  | `Optional[str]`  | Branch name (`None` for default) |

**Returns:** The saved document as a dictionary.

**Raises:** `ChronDBError` on failure.

***

### `get(id, branch=None) -> Dict[str, Any]`

Retrieves a document by ID.

| Parameter | Type            | Description |
| --------- | --------------- | ----------- |
| `id`      | `str`           | Document ID |
| `branch`  | `Optional[str]` | Branch name |

**Returns:** The document as a dictionary.

**Raises:** `DocumentNotFoundError` if not found, `ChronDBError` on failure.

***

### `delete(id, branch=None) -> bool`

Deletes a document by ID.

| Parameter | Type            | Description |
| --------- | --------------- | ----------- |
| `id`      | `str`           | Document ID |
| `branch`  | `Optional[str]` | Branch name |

**Returns:** `True` if deleted.

**Raises:** `DocumentNotFoundError` if not found, `ChronDBError` on failure.

***

### `list_by_prefix(prefix, branch=None) -> List[Dict[str, Any]]`

Lists documents whose IDs start with the given prefix.

| Parameter | Type            | Description        |
| --------- | --------------- | ------------------ |
| `prefix`  | `str`           | ID prefix to match |
| `branch`  | `Optional[str]` | Branch name        |

**Returns:** List of matching documents (empty list if none).

***

### `list_by_table(table, branch=None) -> List[Dict[str, Any]]`

Lists all documents in a table.

| Parameter | Type            | Description |
| --------- | --------------- | ----------- |
| `table`   | `str`           | Table name  |
| `branch`  | `Optional[str]` | Branch name |

**Returns:** List of matching documents (empty list if none).

***

### `history(id, branch=None) -> List[Dict[str, Any]]`

Returns the change history of a document.

| Parameter | Type            | Description |
| --------- | --------------- | ----------- |
| `id`      | `str`           | Document ID |
| `branch`  | `Optional[str]` | Branch name |

**Returns:** List of history entries (empty list if none).

***

### `query(query, branch=None) -> Dict[str, Any]`

Executes a query against the Lucene index.

| Parameter | Type             | Description                |
| --------- | ---------------- | -------------------------- |
| `query`   | `Dict[str, Any]` | Query in Lucene AST format |
| `branch`  | `Optional[str]`  | Branch name                |

**Returns:** Results dict with keys `results`, `total`, `limit`, `offset`.

**Raises:** `ChronDBError` on failure.

***

### `execute(sql, branch=None) -> Dict[str, Any]`

Executes a SQL query directly against the database without needing a running server.

| Parameter | Type            | Description                      |
| --------- | --------------- | -------------------------------- |
| `sql`     | `str`           | SQL query string                 |
| `branch`  | `Optional[str]` | Branch name (`None` for default) |

**Returns:** Result dict with keys `type`, `columns`, `rows`, `count`.

**Raises:** `ChronDBError` on failure.

***

### `setup_remote(remote_url) -> Dict[str, Any]`

Configures a remote Git repository URL for syncing.

| Parameter    | Type  | Description                                            |
| ------------ | ----- | ------------------------------------------------------ |
| `remote_url` | `str` | Remote Git URL (e.g., `"git@github.com:org/repo.git"`) |

**Returns:** `{"type": "ok", "remote_url": "..."}` on success.

**Raises:** `ChronDBError` on failure.

***

### `push() -> Dict[str, Any]`

Pushes local changes to the configured remote repository.

**Returns:** `{"type": "ok", "status": "pushed"}` on success, `{"type": "ok", "status": "skipped"}` if no remote is configured.

**Raises:** `ChronDBError` on failure.

***

### `pull() -> Dict[str, Any]`

Pulls changes from the configured remote repository. Fetches and fast-forwards the local branch.

**Returns:** `{"type": "ok", "status": "pulled|current|skipped|conflict"}`.

**Raises:** `ChronDBError` on failure.

***

### `fetch() -> Dict[str, Any]`

Fetches changes from the configured remote without merging.

**Returns:** `{"type": "ok", "status": "fetched|skipped"}`.

**Raises:** `ChronDBError` on failure.

***

### `remote_status() -> Dict[str, Any]`

Returns whether a remote repository is configured.

**Returns:** `{"type": "ok", "configured": true|false}`.

**Raises:** `ChronDBError` on failure.

***

### `close()`

Closes the database connection and releases native resources. Called automatically when using the context manager.

## Document ID Convention

Documents use the format `table:id`:

```python
db.put("user:123", {"name": "Alice"})
db.put("order:456", {"total": 99.90})
```

Internally, `user:123` is stored as `user/user_COLON_123.json` in the Git repository.

Use `list_by_table("user")` to retrieve all documents in the `user` table.

## Error Handling

### Exception Hierarchy

```
ChronDBError (base)
  └── DocumentNotFoundError
```

### `ChronDBError`

Base exception for all ChronDB errors (connection failures, operation errors, native library issues).

### `DocumentNotFoundError`

Raised by `get()` and `delete()` when the target document does not exist.

### Example

```python
from chrondb import ChronDB, ChronDBError, DocumentNotFoundError

try:
    with ChronDB("./mydb") as db:
        doc = db.get("user:999")
except DocumentNotFoundError:
    print("Document does not exist")
except ChronDBError as e:
    print(f"Database error: {e}")
```

## Examples

### Full CRUD

```python
from chrondb import ChronDB

with ChronDB("./mydb") as db:
    # Create
    db.put("user:1", {"name": "Alice", "email": "alice@example.com"})
    db.put("user:2", {"name": "Bob", "email": "bob@example.com"})

    # Read
    alice = db.get("user:1")

    # Update
    alice["age"] = 30
    db.put("user:1", alice)

    # Delete
    db.delete("user:2")

    # List by table
    users = db.list_by_table("user")

    # List by prefix
    matched = db.list_by_prefix("user:1")
```

### Query

```python
with ChronDB("./mydb") as db:
    db.put("product:1", {"name": "Laptop", "price": 999})
    db.put("product:2", {"name": "Mouse", "price": 29})

    results = db.query({
        "type": "term",
        "field": "name",
        "value": "Laptop"
    })
    print(results["total"])  # 1
```

### History (Time Travel)

```python
with ChronDB("./mydb") as db:
    db.put("config:app", {"version": "1.0"})
    db.put("config:app", {"version": "2.0"})

    entries = db.history("config:app")
    for entry in entries:
        print(entry)
```

### SQL Queries

Execute SQL queries directly without needing a running server:

```python
with ChronDB("./mydb") as db:
    db.put("user:1", {"name": "Alice", "age": 30})
    db.put("user:2", {"name": "Bob", "age": 25})

    result = db.execute("SELECT * FROM user")
    print(result["columns"])  # ["name", "age"]
    print(result["count"])    # 2

    result = db.execute("SELECT * FROM user WHERE name = 'Alice'")
    print(result["rows"])     # [{"name": "Alice", "age": 30}]
```

### Remote Sync

```python
with ChronDB("./mydb") as db:
    # Configure remote
    db.setup_remote("git@github.com:org/data.git")

    # Write data locally
    db.put("user:1", {"name": "Alice"})

    # Push to remote
    result = db.push()
    print(result["status"])  # "pushed"

    # Pull latest from remote
    result = db.pull()
    print(result["status"])  # "pulled" or "current"

    # Check remote status
    status = db.remote_status()
    print(status["configured"])  # True
```

### Idle Timeout (long-running services)

ChronDB loads a GraalVM native-image shared library whose internal threads consume CPU even when no operations are in flight. For long-running services with sporadic database access, use `idle_timeout` to automatically suspend the isolate when idle:

```python
from chrondb import ChronDB

# Isolate suspends after 2 minutes of inactivity
db = ChronDB("./mydb", idle_timeout=120)

# Normal usage — no change in API
db.put("audit:1", {"action": "login"})
doc = db.get("audit:1")

# After 120s without operations, the GraalVM isolate is torn down.
# The next call to put/get/query transparently reopens it.
```

**When to use:**

* Long-running daemons that write to ChronDB intermittently (e.g., audit logging, MCP servers)
* Services where memory/CPU usage matters during idle periods

**When NOT to use:**

* Short-lived scripts (just use `ChronDB(...)` without timeout)
* High-throughput services with constant database access (the isolate would never go idle)

### Pytest Fixture

```python
import pytest
from chrondb import ChronDB

@pytest.fixture
def db(tmp_path):
    with ChronDB(str(tmp_path / "db")) as conn:
        yield conn

def test_put_and_get(db):
    db.put("item:1", {"value": 42})
    doc = db.get("item:1")
    assert doc["value"] == 42
```

## Export & Backup

### Export to Directory

```python
# Export current state to filesystem
result = db.export_to_directory("/tmp/export")
print(f"Exported {result['files_exported']} files")

# Export specific branch with prefix filter
result = db.export_to_directory(
    "/tmp/export",
    branch="feature-x",
    prefix="users",
    format="json",        # "json" (pretty) or "raw"
    decode_paths=True,    # decode encoded paths
    overwrite=True        # overwrite existing directory
)
```

### Backup & Restore

```python
# Create a full backup
result = db.create_backup("/backups/full.tar.gz")
print(f"Backup at: {result['path']}")

# Create a bundle backup
result = db.create_backup("/backups/full.bundle", format="bundle")

# Restore from backup
result = db.restore_backup("/backups/full.tar.gz")
print(f"Restore type: {result['restore_type']}")

# Export/import git bundle snapshots
result = db.export_snapshot("/backups/main.bundle")
result = db.import_snapshot("/backups/main.bundle")
```

## Building from Source

```bash
# 1. Build the shared library (requires Java 11+ and GraalVM)
cd chrondb/
clojure -M:shared

# 2. Install the Python package in development mode
cd bindings/python/
pip install -e .

# 3. Run tests
CHRONDB_LIB_DIR=../../target pytest
```


# Ruby

Ruby client for ChronDB, auto-generated from the Rust SDK via [UniFFI](https://github.com/mozilla/uniffi-rs).

## Requirements

* Ruby 3.0+

## Installation

```bash
gem install chrondb
```

## Quick Start

```ruby
require "chrondb"

# Single path (preferred)
db = ChronDB::Client.new("./mydb")

# Save a document
db.put("user:1", { name: "Alice", age: 30 })

# Retrieve it
doc = db.get("user:1")
puts doc  # {"name"=>"Alice", "age"=>30}
```

> **Legacy API (deprecated):** `ChronDB::Client.new("/tmp/data", "/tmp/index")` still works but is deprecated. Use the single-path form instead.

## API Reference

### `ChronDB::Client.new(db_path, idle_timeout: nil)`

Opens a database connection using a single directory path.

| Parameter      | Type      | Description                                                 |
| -------------- | --------- | ----------------------------------------------------------- |
| `db_path`      | `String`  | Path for the database (data and index stored inside)        |
| `idle_timeout` | `Integer` | Seconds of inactivity before suspending the GraalVM isolate |

**Raises:** `ChronDB::Error` if the database cannot be opened.

#### Legacy: `ChronDB::Client.new(data_path, index_path, idle_timeout: nil)`

> **Deprecated.** The two-path constructor still works but is deprecated. Use the single-path form above.

| Parameter      | Type      | Description                                                 |
| -------------- | --------- | ----------------------------------------------------------- |
| `data_path`    | `String`  | Path for the Git repository (data storage)                  |
| `index_path`   | `String`  | Path for the Lucene index                                   |
| `idle_timeout` | `Integer` | Seconds of inactivity before suspending the GraalVM isolate |

***

### `put(id, doc, branch: nil) -> Hash`

Saves a document.

| Parameter | Type     | Description                     |
| --------- | -------- | ------------------------------- |
| `id`      | `String` | Document ID (e.g., `"user:1"`)  |
| `doc`     | `Hash`   | Document data                   |
| `branch`  | `String` | Branch name (`nil` for default) |

***

### `get(id, branch: nil) -> Hash`

Retrieves a document by ID.

**Raises:** `ChronDB::DocumentNotFoundError` if not found.

***

### `delete(id, branch: nil) -> true`

Deletes a document by ID.

**Raises:** `ChronDB::DocumentNotFoundError` if not found.

***

### `list_by_prefix(prefix, branch: nil) -> Array<Hash>`

Lists documents whose IDs start with the given prefix.

***

### `list_by_table(table, branch: nil) -> Array<Hash>`

Lists all documents in a table.

***

### `history(id, branch: nil) -> Array<Hash>`

Returns the change history of a document.

***

### `query(query, branch: nil) -> Hash`

Executes a query against the Lucene index.

***

### `execute(sql, branch: nil) -> Hash`

Executes a SQL query directly against the database without needing a running server.

| Parameter | Type     | Description                     |
| --------- | -------- | ------------------------------- |
| `sql`     | `String` | SQL query string                |
| `branch`  | `String` | Branch name (`nil` for default) |

**Returns:** Hash with keys `type`, `columns`, `rows`, `count`.

**Raises:** `ChronDB::Error` on failure.

## Error Handling

```ruby
require "chrondb"

begin
  db = ChronDB::Client.new("./mydb")
  doc = db.get("user:999")
rescue ChronDB::DocumentNotFoundError
  puts "Document does not exist"
rescue ChronDB::Error => e
  puts "Database error: #{e.message}"
end
```

## Examples

### Full CRUD

```ruby
require "chrondb"

db = ChronDB::Client.new("./mydb")

# Create
db.put("user:1", { name: "Alice", email: "alice@example.com" })
db.put("user:2", { name: "Bob", email: "bob@example.com" })

# Read
alice = db.get("user:1")

# Update
alice["age"] = 30
db.put("user:1", alice)

# Delete
db.delete("user:2")

# List
users = db.list_by_table("user")
```

### Idle Timeout (long-running services)

```ruby
# Isolate suspends after 2 minutes of inactivity
db = ChronDB::Client.new("./mydb", idle_timeout: 120)

db.put("audit:1", { action: "login" })

# After 120s without operations, the GraalVM isolate is torn down.
# The next call transparently reopens it.
```

### SQL Queries

Execute SQL queries directly without needing a running server:

```ruby
db = ChronDB::Client.new("./mydb")

db.put("user:1", { name: "Alice", age: 30 })
db.put("user:2", { name: "Bob", age: 25 })

result = db.execute("SELECT * FROM user")
puts result["columns"]  # ["name", "age"]
puts result["count"]    # 2

result = db.execute("SELECT * FROM user WHERE name = 'Alice'")
puts result["rows"]     # [{"name"=>"Alice", "age"=>30}]
```

### Query

```ruby
db.put("product:1", { name: "Laptop", price: 999 })

results = db.query({
  type: "term",
  field: "name",
  value: "Laptop"
})

puts results["total"]  # 1
```

### History (Time Travel)

```ruby
db.put("config:app", { version: "1.0" })
db.put("config:app", { version: "2.0" })

entries = db.history("config:app")
entries.each { |entry| puts entry }
```

## Export & Backup

### Export to Directory

```ruby
# Export current state to filesystem
result = db.export_to_directory("/tmp/export")
puts "Exported #{result['files_exported']} files"

# Export with options
result = db.export_to_directory(
  "/tmp/export",
  branch: "main",
  prefix: "users",
  format: "json",
  decode_paths: true,
  overwrite: true
)
```

### Backup & Restore

```ruby
# Create a full backup
result = db.create_backup("/backups/full.tar.gz")
puts "Backup at: #{result['path']}"

# Create a bundle backup
result = db.create_backup("/backups/full.bundle", format: "bundle")

# Restore from backup
result = db.restore_backup("/backups/full.tar.gz")

# Export/import git bundle snapshots
result = db.export_snapshot("/backups/main.bundle")
result = db.import_snapshot("/backups/main.bundle")
```


# Node.js

Node.js client for ChronDB, built with [NAPI-RS](https://napi.rs/) from the Rust SDK.

## Requirements

* Node.js 16+

## Installation

```bash
npm install chrondb
```

## Quick Start

```javascript
const { ChronDB } = require('chrondb')

// Single path (preferred)
const db = new ChronDB('./mydb')

// Save a document
db.put('user:1', { name: 'Alice', age: 30 })

// Retrieve it
const doc = db.get('user:1')
console.log(doc) // { name: 'Alice', age: 30 }
```

> **Legacy API (deprecated):** `new ChronDB('/tmp/data', '/tmp/index')` still works but is deprecated. Use the single-path form instead.

### ES Modules

```javascript
import { ChronDB } from 'chrondb'

const db = new ChronDB('./mydb')
```

## API Reference

### `new ChronDB(dbPath, options?)`

Opens a database connection using a single directory path.

| Parameter             | Type     | Description                                                 |
| --------------------- | -------- | ----------------------------------------------------------- |
| `dbPath`              | `string` | Path for the database (data and index stored inside)        |
| `options.idleTimeout` | `number` | Seconds of inactivity before suspending the GraalVM isolate |

**Throws:** `Error` if the database cannot be opened.

#### Legacy: `new ChronDB(dataPath, indexPath, options?)`

> **Deprecated.** The two-path constructor still works but is deprecated. Use the single-path form above.

| Parameter             | Type     | Description                                                 |
| --------------------- | -------- | ----------------------------------------------------------- |
| `dataPath`            | `string` | Path for the Git repository (data storage)                  |
| `indexPath`           | `string` | Path for the Lucene index                                   |
| `options.idleTimeout` | `number` | Seconds of inactivity before suspending the GraalVM isolate |

***

### `put(id, doc, branch?) -> Object`

Saves a document.

| Parameter | Type             | Description                    |
| --------- | ---------------- | ------------------------------ |
| `id`      | `string`         | Document ID (e.g., `"user:1"`) |
| `doc`     | `Object`         | Document data                  |
| `branch`  | `string \| null` | Branch name                    |

***

### `get(id, branch?) -> Object`

Retrieves a document by ID.

**Throws:** `Error` if not found.

***

### `delete(id, branch?) -> void`

Deletes a document by ID.

***

### `listByPrefix(prefix, branch?) -> Object[]`

Lists documents whose IDs start with the given prefix.

***

### `listByTable(table, branch?) -> Object[]`

Lists all documents in a table.

***

### `history(id, branch?) -> Object[]`

Returns the change history of a document.

***

### `query(query, branch?) -> Object`

Executes a query against the Lucene index.

***

### `execute(sql, branch?) -> Object`

Executes a SQL query directly against the database without needing a running server.

| Parameter | Type             | Description      |
| --------- | ---------------- | ---------------- |
| `sql`     | `string`         | SQL query string |
| `branch`  | `string \| null` | Branch name      |

**Returns:** Result object with `type`, `columns`, `rows`, `count`.

## TypeScript

Full TypeScript definitions are included:

```typescript
import { ChronDB, ChronDBOptions } from 'chrondb'

const db = new ChronDB('./mydb', { idleTimeout: 120 })

const doc: Record<string, unknown> = db.get('user:1')
```

## Error Handling

```javascript
const { ChronDB } = require('chrondb')

try {
  const db = new ChronDB('./mydb')
  const doc = db.get('user:999')
} catch (err) {
  if (err.message.includes('not found')) {
    console.log('Document does not exist')
  } else {
    console.error('Database error:', err.message)
  }
}
```

## Examples

### Full CRUD

```javascript
const { ChronDB } = require('chrondb')

const db = new ChronDB('./mydb')

// Create
db.put('user:1', { name: 'Alice', email: 'alice@example.com' })
db.put('user:2', { name: 'Bob', email: 'bob@example.com' })

// Read
const alice = db.get('user:1')

// Update
db.put('user:1', { ...alice, age: 30 })

// Delete
db.delete('user:2')

// List
const users = db.listByTable('user')
```

### Idle Timeout (long-running services)

```javascript
// Isolate suspends after 2 minutes of inactivity
const db = new ChronDB('./mydb', { idleTimeout: 120 })

db.put('audit:1', { action: 'login' })

// After 120s without operations, the GraalVM isolate is torn down.
// The next call transparently reopens it.
```

### SQL Queries

Execute SQL queries directly without needing a running server:

```javascript
const db = new ChronDB('./mydb')

db.put('user:1', { name: 'Alice', age: 30 })
db.put('user:2', { name: 'Bob', age: 25 })

const result = db.execute('SELECT * FROM user')
console.log(result.columns) // ['name', 'age']
console.log(result.count)   // 2

const filtered = db.execute("SELECT * FROM user WHERE name = 'Alice'")
console.log(filtered.rows)  // [{ name: 'Alice', age: 30 }]
```

### Query

```javascript
db.put('product:1', { name: 'Laptop', price: 999 })

const results = db.query({
  type: 'term',
  field: 'name',
  value: 'Laptop'
})

console.log(results.total) // 1
```

### History (Time Travel)

```javascript
db.put('config:app', { version: '1.0' })
db.put('config:app', { version: '2.0' })

const entries = db.history('config:app')
entries.forEach(entry => console.log(entry))
```

## Export & Backup

### Export to Directory

```javascript
// Export current state to filesystem
const result = db.exportToDirectory('/tmp/export')
console.log(`Exported ${result.files_exported} files`)

// Export with options
const result = db.exportToDirectory('/tmp/export', {
  branch: 'main',
  prefix: 'users',
  format: 'json',
  decodePaths: true,
  overwrite: true
})
```

### Backup & Restore

```javascript
// Create a full backup
let result = db.createBackup('/backups/full.tar.gz')
console.log(`Backup at: ${result.path}`)

// Create a bundle backup
result = db.createBackup('/backups/full.bundle', { format: 'bundle' })

// Restore from backup
result = db.restoreBackup('/backups/full.tar.gz')

// Export/import git bundle snapshots
result = db.exportSnapshot('/backups/main.bundle')
result = db.importSnapshot('/backups/main.bundle')
```


# Kotlin

Kotlin client for ChronDB, auto-generated from the Rust SDK via [UniFFI](https://github.com/mozilla/uniffi-rs).

## Requirements

* Kotlin 1.6+
* JNA (Java Native Access)

## Installation

Download the platform-specific tarball from the [latest GitHub release](https://github.com/avelino/chrondb/releases):

* `chrondb-kotlin-{version}-macos-aarch64.tar.gz` (macOS Apple Silicon)
* `chrondb-kotlin-{version}-linux-x86_64.tar.gz` (Linux x86\_64)

Extract and add as a local dependency:

```kotlin
// In your build.gradle.kts
dependencies {
    implementation(files("path/to/chrondb-kotlin/lib"))
}
```

## Quick Start

```kotlin
import chrondb.ChronDB
import org.json.JSONObject

// Single path (preferred)
val db = ChronDB.openPath("./mydb")

// Save a document
val doc = JSONObject().put("name", "Alice").put("age", 30)
db.put("user:1", doc.toString(), null)

// Retrieve it
val result = db.get("user:1", null)
println(result) // {"name":"Alice","age":30}
```

> **Legacy API (deprecated):** `ChronDB.open("/tmp/data", "/tmp/index")` still works but is deprecated. Use `openPath` instead.

## API Reference

### `ChronDB.openPath(dbPath: String): ChronDB`

Opens a database connection using a single directory path. Data and index are stored in subdirectories automatically.

### `ChronDB.open(dataPath: String, indexPath: String): ChronDB` *(deprecated)*

> **Deprecated.** Use `openPath` instead. This method still works but will be removed in a future release.

### `ChronDB.openWithIdleTimeout(dataPath: String, indexPath: String, idleTimeoutSecs: ULong): ChronDB`

Opens a database with idle timeout. The GraalVM isolate suspends after the specified seconds of inactivity.

### Operations

| Method                          | Description                                       |
| ------------------------------- | ------------------------------------------------- |
| `put(id, jsonDoc, branch?)`     | Save a document (JSON string in, JSON string out) |
| `get(id, branch?)`              | Get a document by ID                              |
| `delete(id, branch?)`           | Delete a document                                 |
| `listByPrefix(prefix, branch?)` | List documents by ID prefix                       |
| `listByTable(table, branch?)`   | List documents by table                           |
| `history(id, branch?)`          | Get change history                                |
| `query(queryJson, branch?)`     | Execute a Lucene query                            |
| `executeSql(sql, branch?)`      | Execute a SQL query directly                      |

## Error Handling

```kotlin
import chrondb.ChronDB
import chrondb.ChronDBError

try {
    val db = ChronDB.openPath("./mydb")
    val doc = db.get("user:999", null)
} catch (e: ChronDBError.NotFound) {
    println("Document does not exist")
} catch (e: ChronDBError) {
    println("Database error: ${e.message}")
}
```

## Examples

### SQL Queries

Execute SQL queries directly without needing a running server:

```kotlin
val db = ChronDB.openPath("./mydb")

db.put("user:1", """{"name": "Alice", "age": 30}""", null)
db.put("user:2", """{"name": "Bob", "age": 25}""", null)

val result = db.executeSql("SELECT * FROM user", null)
println(result) // {"type":"select","columns":[...],"rows":[...],"count":2}
```

### Idle Timeout

```kotlin
// Isolate suspends after 2 minutes of inactivity
val db = ChronDB.openWithIdleTimeout("/tmp/data", "/tmp/index", 120u)

db.put("audit:1", """{"action": "login"}""", null)
```


# Swift

Swift client for ChronDB, auto-generated from the Rust SDK via [UniFFI](https://github.com/mozilla/uniffi-rs).

## Requirements

* Swift 5.5+
* macOS 12+ or Linux

## Installation

Download the platform-specific tarball from the [latest GitHub release](https://github.com/avelino/chrondb/releases):

* `chrondb-swift-{version}-macos-aarch64.tar.gz` (macOS Apple Silicon)
* `chrondb-swift-{version}-linux-x86_64.tar.gz` (Linux x86\_64)

Extract and add as a local Swift package dependency:

```swift
// In your Package.swift
.package(path: "path/to/chrondb-swift")
```

## Quick Start

```swift
import ChronDB

// Single path (preferred)
let db = try ChronDB.openPath(dbPath: "./mydb")

// Save a document
try db.put(id: "user:1", jsonDoc: #"{"name": "Alice", "age": 30}"#, branch: nil)

// Retrieve it
let doc = try db.get(id: "user:1", branch: nil)
print(doc) // {"name":"Alice","age":30}
```

> **Legacy API (deprecated):** `ChronDB.open(dataPath:indexPath:)` still works but is deprecated. Use `openPath` instead.

## API Reference

### `ChronDB.openPath(dbPath:) throws -> ChronDB`

Opens a database connection using a single directory path. Data and index are stored in subdirectories automatically.

### `ChronDB.open(dataPath:indexPath:) throws -> ChronDB` *(deprecated)*

> **Deprecated.** Use `openPath` instead. This method still works but will be removed in a future release.

### `ChronDB.openWithIdleTimeout(dataPath:indexPath:idleTimeoutSecs:) throws -> ChronDB`

Opens a database with idle timeout.

### Operations

| Method                         | Description                  |
| ------------------------------ | ---------------------------- |
| `put(id:jsonDoc:branch:)`      | Save a document              |
| `get(id:branch:)`              | Get a document by ID         |
| `delete(id:branch:)`           | Delete a document            |
| `listByPrefix(prefix:branch:)` | List by ID prefix            |
| `listByTable(table:branch:)`   | List by table                |
| `history(id:branch:)`          | Get change history           |
| `query(queryJson:branch:)`     | Execute a Lucene query       |
| `executeSql(sql:branch:)`      | Execute a SQL query directly |

## Error Handling

```swift
import ChronDB

do {
    let db = try ChronDB.openPath(dbPath: "./mydb")
    let doc = try db.get(id: "user:999", branch: nil)
} catch ChronDBError.NotFound {
    print("Document does not exist")
} catch {
    print("Database error: \(error)")
}
```

## Examples

### SQL Queries

Execute SQL queries directly without needing a running server:

```swift
let db = try ChronDB.openPath(dbPath: "./mydb")

try db.put(id: "user:1", jsonDoc: #"{"name": "Alice", "age": 30}"#, branch: nil)
try db.put(id: "user:2", jsonDoc: #"{"name": "Bob", "age": 25}"#, branch: nil)

let result = try db.executeSql(sql: "SELECT * FROM user", branch: nil)
print(result) // {"type":"select","columns":[...],"rows":[...],"count":2}
```

### Idle Timeout

```swift
// Isolate suspends after 2 minutes of inactivity
let db = try ChronDB.openWithIdleTimeout(
    dataPath: "/tmp/data",
    indexPath: "/tmp/index",
    idleTimeoutSecs: 120
)

try db.put(id: "audit:1", jsonDoc: #"{"action": "login"}"#, branch: nil)
```


# Best Practices

Lessons learned from integrating ChronDB in real-world applications. These apply to **all language bindings** (Rust, Python, Node.js, etc.) since they stem from the ChronDB core (GraalVM native image + Git storage).

## Values must be JSON objects

`put()` requires a JSON object as the document value. Scalar values (numbers, strings, booleans) will fail silently or return an error.

```json
// ❌ Fails
put("meta:counter", 42)

// ✅ Works
put("meta:counter", {"value": 42})
```

This applies to all bindings — always wrap scalars in an object.

## Batch writes: open once, write many, close

Opening ChronDB has overhead (GraalVM isolate startup). For bulk operations, open once and write all documents in a single session.

```
// ✅ Fast — single open/close
db = open_path("./mydb")
for doc in documents:
    db.put(doc.id, doc.data)
// close/drop db

// ❌ Slow — reopening per document
for doc in documents:
    db = open_path("./mydb")
    db.put(doc.id, doc.data)
    // close/drop db
```

The difference is orders of magnitude for large batches. Each `open_path()` creates or attaches to a GraalVM isolate with a dedicated worker thread.

## GraalVM heap exhaustion on large datasets

For sessions with thousands of `put()` calls, the GraalVM native image may run out of heap space:

```
Fatal error: java.lang.OutOfMemoryError: Could not allocate an aligned heap chunk
```

**Workaround:** Split large write batches into smaller sessions. Open, write a batch (e.g., 1000 docs), close, then reopen for the next batch. This allows GraalVM to reclaim memory between sessions.

## Index corruption after interrupted writes

If the process is killed during a write operation (Ctrl+C, SIGKILL, OOM), the Lucene index may be left with a stale `write.lock` file. The next `open_path()` will fail:

```
Failed to open database at <path>. Check that the paths are valid and writable.
```

**Recovery:** Delete the index directory and reopen. ChronDB rebuilds the index automatically from the Git data — no data is lost.

```bash
rm -rf ./mydb/index
# Next open_path() rebuilds the index from git objects
```

For production applications, implement automatic recovery: if `open_path()` fails, delete the index directory and retry.

## Git data directory is a bare repository

ChronDB stores data as a bare Git repository. The data directory contains Git objects directly (`HEAD`, `objects/`, `refs/`) **without** a `.git/` subdirectory.

If you need to interact with the Git repo directly, use `--git-dir`:

```bash
git --git-dir=./mydb/data log --oneline -5
```

## Remote sync

Since v0.2.1, ChronDB supports native remote Git operations via `setup_remote()`, `push()`, `pull()`, and `fetch()`. Use these instead of running Git CLI commands manually.

```
db = open_path("./mydb")
db.setup_remote("git@github.com:org/data.git")
db.pull()   // fetch + fast-forward
// ... write data ...
db.push()   // push to remote
```

**Note:** Passing a Git URL directly to `open_path()` does **not** clone the repository — it creates a literal directory with that name. Always use a local path and configure the remote separately with `setup_remote()`.

## `list_by_prefix` response format

`list_by_prefix()` returns a JSON array of objects. Each object includes an `id` field with the document key:

```json
[
  {"id": "user:1", "name": "Alice", ...},
  {"id": "user:2", "name": "Bob", ...}
]
```

Use the `id` field to extract document keys when iterating results.

## Idle timeout for long-running services

ChronDB's GraalVM isolate consumes CPU and memory even when idle. For services with sporadic database access (daemons, MCP servers, background workers), use `idle_timeout` to automatically suspend the isolate.

The isolate suspends after the timeout and transparently reopens on the next operation.

**Don't use for:** Short-lived CLI tools (just open and close normally) or high-throughput services where the isolate never goes idle.


# Configuration

This document details all configuration options available in ChronDB.

## Configuration File

ChronDB uses an EDN (Extensible Data Notation) configuration file. By default, it looks for a `config.edn` file in the project's root directory.

## Configuration Sections

### 1. Git (:git)

Git storage related configurations:

```clojure
:git {
  :committer-name "ChronDB"      ; Name that will appear in commits
  :committer-email "chrondb@example.com"  ; Email that will appear in commits
  :default-branch "main"         ; Main repository branch
  :sign-commits false            ; GPG commit signing
}
```

#### Details

* `committer-name`: Name used to identify the commit author
* `committer-email`: Email used to identify the commit author
* `default-branch`: Name of the main branch where changes will be saved
* `sign-commits`: If true, commits will be signed with GPG (requires additional setup)

### 2. Storage (:storage)

Data storage related configurations:

```clojure
:storage {
  :data-dir "data"              ; Base storage directory
}
```

#### Details

* `data-dir`: Directory where documents will be stored
  * Documents are saved in JSON format
  * Directory structure is maintained by Git
  * Absolute paths are recommended in production

### 3. Logging (:logging)

Logging system configurations:

```clojure
:logging {
  :level :info                  ; Minimum log level
  :output :stdout               ; Where logs will be written
  :file "chrondb.log"           ; Log file (if output = :file)
}
```

### 4. Servers (:servers)

Configuration for different protocols:

```clojure
:servers {
  :rest {
    :enabled true
    :host "0.0.0.0"
    :port 3000
  }
  :redis {
    :enabled true
    :host "0.0.0.0"
    :port 6379
  }
  :postgresql {
    :enabled true
    :host "0.0.0.0"
    :port 5432
    :username "chrondb"
    :password "chrondb"
  }
}
```

## Complete Example

```clojure
{:git {:committer-name "ChronDB"
       :committer-email "chrondb@example.com"
       :default-branch "main"
       :sign-commits false}
 :storage {:data-dir "/var/lib/chrondb/data"}
 :logging {:level :info
           :output :file
           :file "/var/log/chrondb/chrondb.log"}
 :servers {:rest {:enabled true
                 :host "0.0.0.0"
                 :port 3000}
          :redis {:enabled true
                 :host "0.0.0.0"
                 :port 6379}
          :postgresql {:enabled true
                      :host "0.0.0.0"
                      :port 5432
                      :username "chrondb"
                      :password "chrondb"}}}
```

## Loading Configuration

```clojure
(require '[chrondb.config :as config])

;; Load from default file (config.edn)
(def cfg (config/load-config))

;; Or specify a file
(def cfg (config/load-config "/etc/chrondb/prod.edn"))

;; Or merge with default settings
(def cfg (config/load-config {:logging {:level :debug}}))
```

## Best Practices

1. **Development Environment**:
   * Use `:debug` as log level
   * Set `:output` to `:stdout`
   * Use relative paths for easier development
2. **Production Environment**:
   * Use `:info` or `:warn` as log level
   * Set `:output` to `:file`
   * Use absolute paths for directories
   * Configure external log rotation
3. **Security**:
   * Don't version control `config.edn` with credentials
   * Use environment variables for sensitive information
   * Restrict configuration file permissions


# Deployment Guide

This guide covers deploying ChronDB in production environments, from single-node Docker setups to Kubernetes clusters.

## Docker

### Single Container

```bash
docker run -d \
  --name chrondb \
  -p 3000:3000 \
  -p 6379:6379 \
  -p 5432:5432 \
  -v chrondb-data:/app/data \
  ghcr.io/avelino/chrondb:latest
```

### Docker Compose

A `docker-compose.yml` is provided at the repository root for local development and simple deployments:

```bash
docker compose up -d
```

This starts ChronDB with:

* Persistent data volume (`chrondb-data`)
* All three protocols exposed (REST 3000, Redis 6379, PostgreSQL 5432)
* Health check on `/healthz` every 10 seconds
* Automatic restart on failure
* 1 GB memory limit

Verify the service is running:

```bash
curl http://localhost:3000/healthz
# {"status":"healthy","timestamp":"..."}
```

### Custom Configuration

Mount a custom `config.edn` to override defaults:

```bash
docker run -d \
  --name chrondb \
  -p 3000:3000 \
  -v chrondb-data:/app/data \
  -v ./config.edn:/app/config.edn \
  ghcr.io/avelino/chrondb:latest --config /app/config.edn
```

See [Configuration](https://github.com/avelino/chrondb/blob/main/docs/configuration/README.md) for all available options.

### Docker Image Details

* **Base**: Debian 12 slim (runtime stage)
* **Build**: GraalVM native-image (multi-stage)
* **User**: Non-root `chrondb` user
* **Ports**: 3000 (REST), 6379 (Redis), 5432 (PostgreSQL)
* **Entry point**: `/usr/local/bin/chrondb`

***

## Kubernetes

### StatefulSet

ChronDB stores data on disk (Git repository + Lucene index), so it should be deployed as a StatefulSet with persistent storage.

```yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: chrondb
  labels:
    app: chrondb
spec:
  serviceName: chrondb
  replicas: 1
  selector:
    matchLabels:
      app: chrondb
  template:
    metadata:
      labels:
        app: chrondb
    spec:
      containers:
        - name: chrondb
          image: ghcr.io/avelino/chrondb:latest
          ports:
            - name: rest
              containerPort: 3000
            - name: redis
              containerPort: 6379
            - name: postgresql
              containerPort: 5432
          volumeMounts:
            - name: data
              mountPath: /app/data
          resources:
            requests:
              memory: "512Mi"
              cpu: "250m"
            limits:
              memory: "2Gi"
              cpu: "2000m"
          livenessProbe:
            httpGet:
              path: /healthz
              port: rest
            initialDelaySeconds: 15
            periodSeconds: 10
            timeoutSeconds: 5
          readinessProbe:
            httpGet:
              path: /readyz
              port: rest
            initialDelaySeconds: 10
            periodSeconds: 5
            timeoutSeconds: 3
          startupProbe:
            httpGet:
              path: /startupz
              port: rest
            failureThreshold: 30
            periodSeconds: 5
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 10Gi
```

### Service

Expose ChronDB to other pods in the cluster:

```yaml
apiVersion: v1
kind: Service
metadata:
  name: chrondb
  labels:
    app: chrondb
spec:
  selector:
    app: chrondb
  ports:
    - name: rest
      port: 3000
      targetPort: rest
    - name: redis
      port: 6379
      targetPort: redis
    - name: postgresql
      port: 5432
      targetPort: postgresql
```

For external access, create an additional LoadBalancer or Ingress resource targeting the service.

### ConfigMap

Store your `config.edn` in a ConfigMap:

```yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: chrondb-config
data:
  config.edn: |
    {:git {:committer-name "ChronDB"
           :committer-email "chrondb@example.com"
           :default-branch "main"
           :push-enabled false}
     :storage {:data-dir "/app/data"}
     :logging {:level :info
               :output :stdout}}
```

Mount it in the StatefulSet:

```yaml
volumes:
  - name: config
    configMap:
      name: chrondb-config
containers:
  - name: chrondb
    args: ["--config", "/etc/chrondb/config.edn"]
    volumeMounts:
      - name: config
        mountPath: /etc/chrondb
```

### Prometheus ServiceMonitor

If you use Prometheus Operator, create a ServiceMonitor to scrape ChronDB metrics:

```yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: chrondb
  labels:
    app: chrondb
spec:
  selector:
    matchLabels:
      app: chrondb
  endpoints:
    - port: rest
      path: /metrics
      interval: 15s
```

***

## Resource Guidance

### Memory

| Documents           | Recommended Memory |
| ------------------- | ------------------ |
| < 10,000            | 512 MB             |
| 10,000 - 100,000    | 1 - 2 GB           |
| 100,000 - 1,000,000 | 2 - 4 GB           |
| > 1,000,000         | 4+ GB              |

Memory usage is driven by:

* Lucene index segment buffers (in-memory during writes)
* Git object cache (recently accessed documents)
* JVM heap for query processing (sorts, aggregations, joins)

### Disk

Disk usage grows with both document count and history depth. Every write creates a Git commit, so a document updated 100 times uses \~100x the space of its current size. Run `git gc` periodically (via the `compact` command) to optimize storage.

Rule of thumb: allocate **3-5x** the expected current dataset size for history and Git overhead.

### CPU

ChronDB is mostly I/O bound. A single core handles typical workloads. Allocate additional cores for:

* Concurrent query processing
* Lucene index merges (background)
* Git operations (push/pull to remotes)

***

## Production Checklist

### Data Persistence

* [ ] Data directory is on persistent storage (not ephemeral)
* [ ] Volume is backed up regularly (see [Operations Guide](https://github.com/avelino/chrondb/blob/main/docs/operations/README.md))
* [ ] Disk space monitoring is configured with alerts

### Monitoring

* [ ] Health check endpoint (`/healthz`) is monitored
* [ ] Prometheus scraping is configured for `/metrics`
* [ ] Alerts are set for `chrondb_wal_pending_entries > 100` (WAL backlog)
* [ ] Alerts are set for disk space usage > 80%
* [ ] Alerts are set for memory usage > 85%

### Security

* [ ] ChronDB is not exposed to the public internet without a reverse proxy
* [ ] PostgreSQL protocol has authentication configured (username/password)
* [ ] Data directory has restricted file permissions
* [ ] Remote Git sync uses SSH keys (not passwords)
* [ ] See [Security Best Practices](https://github.com/avelino/chrondb/blob/main/docs/security/README.md) for full guidance

### Backups

* [ ] Automated backup schedule is configured
* [ ] Backup restore has been tested
* [ ] Backups are stored off-host (different disk, S3, etc.)

### Network

* [ ] Only needed protocols are enabled (disable unused ones in `config.edn`)
* [ ] Ports are firewalled to trusted networks
* [ ] Consider a reverse proxy (nginx, Envoy) for REST API TLS termination


# Operations Guide

This guide covers operational aspects of ChronDB, including installation, execution, monitoring, and maintenance.

## Installation

### Requirements

* Java 11+
* Git 2.25.0+
* 1GB+ RAM (recommended)
* Disk space proportional to data volume and history

### Installation via JAR

1. Download the latest JAR file from the releases page:

   ```sh
   curl -LO https://github.com/avelino/chrondb/releases/latest/download/chrondb.jar
   ```
2. Run the JAR:

   ```sh
   java -jar chrondb.jar
   ```

### Manual Compilation

1. Clone the repository:

   ```sh
   git clone https://github.com/avelino/chrondb.git
   cd chrondb
   ```
2. Compile the project:

   ```sh
   clojure -T:build uber
   ```
3. Run the generated JAR:

   ```sh
   java -jar target/chrondb.jar
   ```

## Running ChronDB

### Command Line Options

```sh
java -jar chrondb.jar [options]

Options:
  --config FILE         Path to configuration file
  --data-dir DIR        Directory for data storage
  --http-port PORT      Port for HTTP REST server
  --redis-port PORT     Port for Redis protocol server
  --pg-port PORT        Port for PostgreSQL protocol server
  --log-level LEVEL     Log level (debug, info, warn, error)
  --memory-limit MB     Memory limit in MB
```

### As a Systemd Service

1. Create a service file `/etc/systemd/system/chrondb.service`:

   ```toml
   [Unit]
   Description=ChronDB Chronological Database
   After=network.target

   [Service]
   User=chrondb
   Group=chrondb
   ExecStart=/usr/bin/java -jar /opt/chrondb/chrondb.jar --config /etc/chrondb/config.edn
   Restart=on-failure

   [Install]
   WantedBy=multi-user.target
   ```
2. Reload systemd:

   ```sh
   sudo systemctl daemon-reload
   ```
3. Start the service:

   ```sh
   sudo systemctl start chrondb
   ```

## Monitoring

ChronDB exposes metrics and health information for monitoring.

### Health Check Endpoint

```http
GET /api/v1/health
```

Response:

```json
{
  "status": "healthy",
  "version": "0.1.0",
  "uptime": "10h 30m",
  "memory": {
    "used": "256MB",
    "total": "1024MB"
  },
  "storage": {
    "size": "500MB",
    "documents": 1250
  }
}
```

### Logs

Logs contain detailed information about operations and errors. By default, they are sent to stdout or a file as configured.

### Prometheus Integration

ChronDB exposes metrics in Prometheus format at:

```http
GET /metrics
```

## Backup and Restoration

ChronDB usa Git internamente, então cada gravação gera commits imutáveis. A partir desta versão há suporte oficial a backup/restore completos e incrementais, tanto via CLI quanto via REST.

### Backup (CLI)

* Full tar.gz (default):

  ```bash
  clojure -M:run backup --output backups/full-2025-10-12.tar.gz
  ```
* Full bundle:

  ```bash
  clojure -M:run backup --output backups/full-main.bundle --format bundle
  ```
* Incremental bundle (desde commit base):

  ```bash
  clojure -M:run backup --format bundle --output backups/incr.bundle --base-commit <full-commit>
  ```

### Restore (CLI)

* Tar.gz:

  ```bash
  clojure -M:run restore --input backups/full-2025-10-12.tar.gz
  ```
* Bundle:

  ```bash
  clojure -M:run restore --input backups/full-main.bundle --format bundle
  ```

### Export/Import bundle (CLI)

```bash
clojure -M:run export-snapshot --output backups/main.bundle --refs refs/heads/main
clojure -M:run import-snapshot --input backups/main.bundle
```

### Scheduler (CLI)

```bash
clojure -M:run schedule --mode full --interval 60 --output-dir backups/
clojure -M:run cancel-schedule --id <job-id>
clojure -M:run list-schedules
```

### Backup via REST

* POST `/api/v1/backup` — body JSON `{"output":"/path/to/full.tar.gz","format":"tar.gz"}`
* POST `/api/v1/export` — JSON `{"output":"/path/to/main.bundle","refs":["refs/heads/main"]}`

### Restore via REST

* POST `/api/v1/restore` com upload multipart (campo `file`) e campo opcional `format`
* POST `/api/v1/import` com upload multipart (`file`) e flag `verify`

As respostas retornam status HTTP e o manifesto com checksum, refs inclusas e tipo (:full, :incremental).

> **Dica:** Incrementais suportam apenas formato bundle; para reconstruir completamente, aplique primeiro o full e depois os incrementais na ordem.

## Maintenance

### Compaction

To optimize performance and space usage:

```
java -jar chrondb.jar --command compact
```

This operation:

* Runs git gc to optimize the repository
* Rebuilds indices to improve search performance
* Optimizes the internal structure

### Background Reindexing

ChronDB mantém o índice Lucene sincronizado através de uma tarefa de reindexação em segundo plano. Sempre que `chrondb server` é iniciado, um processo assíncrono percorre commits existentes para garantir que documentos antigos também estejam presentes nos novos índices Lucene. A tarefa roda sem bloquear gravações ou leituras e respeita a ordem cronológica: novas confirmações continuam chegando enquanto a reindexação prossegue.

* **Inicialização resiliente**: após restaurações, importações ou grandes refatorações de esquema, o reindexador faz o catch-up automático para que a busca reflita imediatamente o estado do repositório Git.
* **Janela de manutenção recorrente**: além do bootstrap inicial, uma rotina programada executa a verificação/reindexação a cada hora (valor padrão). Esse intervalo é seguro para ambientes de produção porque processa batches incrementais e atualiza métricas de uso para o planejador de consultas.
* **Observabilidade**: logs no namespace `chrondb.index.lucene` indicam quando a reindexação começa e termina, bem como eventuais erros de documento. Métricas expostas em `chrondb.index.*` ajudam a acompanhar progresso e filas pendentes.
* **Uso recomendado**: após rodar `chrondb restore`, `import-snapshot` ou grandes cargas batch, aguarde a conclusão da tarefa em background ou monitore os logs antes de liberar novas buscas críticas.

### Updating

1. Stop the ChronDB service
2. Backup your data
3. Replace the JAR file with the new version
4. Start the service again

### Branch Management

Unused branches can be removed:

```
java -jar chrondb.jar --command remove-branch --name test-branch
```

### Disk Space Monitoring

ChronDB will grow over time due to the historical storage nature. Monitor disk usage and configure retention policies if necessary.

## FFI / Language Bindings

Export and backup operations are available in all language bindings (Python, Rust, Ruby, Node.js).

### Available Methods

| Operation           | Python                  | Rust                    | Ruby                    | Node.js               |
| ------------------- | ----------------------- | ----------------------- | ----------------------- | --------------------- |
| Export to directory | `export_to_directory()` | `export_to_directory()` | `export_to_directory()` | `exportToDirectory()` |
| Create backup       | `create_backup()`       | `create_backup()`       | `create_backup()`       | `createBackup()`      |
| Restore backup      | `restore_backup()`      | `restore_backup()`      | `restore_backup()`      | `restoreBackup()`     |
| Export snapshot     | `export_snapshot()`     | `export_snapshot()`     | `export_snapshot()`     | `exportSnapshot()`    |
| Import snapshot     | `import_snapshot()`     | `import_snapshot()`     | `import_snapshot()`     | `importSnapshot()`    |

See the individual [binding documentation](/language-bindings/overview) for detailed usage examples.


# Performance and Scalability

ChronDB is built on Git's version control system, which provides excellent performance characteristics for many operations. This document explores the performance aspects of ChronDB and provides guidance for scaling your applications.

## Git-Based Architecture: Performance Implications

ChronDB leverages Git as its storage engine, inheriting many performance characteristics from Git's underlying implementation. This provides several benefits:

1. **Content addressable storage** - Git's object model allows for efficient deduplication
2. **Delta compression** - Only changes are stored, minimizing storage requirements
3. **Local operations** - Most operations occur locally, providing fast response times
4. **Distributed architecture** - Allows for high availability and horizontal scaling

## Read Performance

### Document Retrieval

Direct document retrieval in ChronDB is typically very fast, as Git can efficiently locate and retrieve objects from its repository. When accessing the latest version of a document, ChronDB uses Git's optimized indexing to locate the content quickly.

According to performance studies, Git can retrieve content in microseconds to milliseconds, depending on repository size:

> "In typical repositories, Git read operations like `git cat-file` can retrieve objects with latencies in the 1-10ms range, even in repositories with hundreds of thousands of files." - [Git Performance Benchmarks](https://git-scm.com/book/en/v2/Git-Internals-Packfiles)

### Historical Retrieval

Retrieving historical versions may have higher latency, as Git needs to traverse the commit history. Performance depends on:

* Depth of the history being accessed
* Size of the repository
* Structure of the commit graph

## Write Performance

Write operations in ChronDB involve several steps that affect performance:

1. Converting the document to Git objects
2. Writing objects to the repository
3. Creating a commit with metadata
4. Updating references

For individual document writes, ChronDB typically provides very good performance. However, as with any Git-based system, performance can decrease with repository size and history length.

Research has shown:

> "Git write performance tends to scale with O(log n) where n is the number of objects. Small commits typically complete in 10-50ms, while larger dataset operations can take seconds." - [Microsoft's Analysis of Git Performance](https://devblogs.microsoft.com/devops/scalar-git-performance-at-scale/)

## Lucene Indexing Overhaul

ChronDB's search layer now leans entirely on Apache Lucene, bringing a substantial performance uplift compared to the earlier bespoke indexes. The rework was driven by production users that needed predictable latency under complex filters and full-text workloads. By adopting Lucene's optimized query execution engine, ChronDB now delegates scoring, boolean logic, and range evaluation to proven algorithms instead of reimplementing them in-house.

### Why it matters

* **Lower tail latencies**: Composite and secondary indexes can now be configured per collection, reducing random I/O and eliminating manual fan-out scans.
* **Specialized analyzers**: Tokenization, stemming, and language-specific analyzers are first-class Lucene features and drastically improve relevance quality for full-text search scenarios.
* **Query planner**: ChronDB inspects incoming queries to choose efficient index combinations, avoiding redundant segment scans and materializing only the minimal result set.
* **Result caching**: Frequently used queries populate an eviction-aware cache, trimming recurring response times and protecting the repository from repeated heavy traversals.
* **Geospatial acceleration**: Geohash and BKD tree indexes leverage Lucene's spatial extensions, enabling fast proximity queries without bespoke data structures.

### Operational guidance

* Configure index definitions alongside schema creation so that ChronDB can warm caches and statistics proactively.
* Monitor the new index metrics under `chrondb.index.*` to track cache hit rates, planner fallbacks, and segment merge costs.
* Use the `ANALYZE INDEX` tooling to recompute query statistics when workloads shift—ChronDB will use the new data to refine its execution plans.
* When adding batch ingestion jobs, stage writes behind the asynchronous indexer to prevent cache stampedes and leverage Lucene's bulk segment writers.

## Scaling Strategies

### Repository Size Considerations

While Git repositories can handle millions of files, performance optimizations may be needed as scale increases:

```
# Typical performance characteristics by repository size
Small repos     (<10K docs):     Excellent performance for all operations
Medium repos    (<100K docs):    Good performance with minimal tuning
Large repos     (<1M docs):      May require optimization strategies
Very large repos (>1M docs):     Requires careful planning and partitioning
```

### Optimization Strategies

When scaling ChronDB for large applications, consider these strategies:

1. **Repository Sharding**: Partition data across multiple repositories based on:
   * Natural data boundaries
   * Time-based partitioning
   * Customer/tenant isolation
2. **Read Replicas**: For read-heavy workloads, deploy read-only replicas to distribute load
3. **Caching Layer**: Implement a caching strategy for frequently accessed documents
4. **Branch Management**: Limit the number of active branches to reduce complexity
5. **Regular Maintenance**: Schedule routine maintenance operations:
   * Garbage collection
   * Repository repacking
   * Index optimization

## Synchronization Performance

ChronDB's synchronization operations (similar to Git's push/pull) involve transferring data between repositories. Performance depends on:

1. Network bandwidth and latency
2. Volume of changes being synchronized
3. Repository size and structure

Studies on Git synchronization show:

> "Git's pack transfer protocol is highly efficient, transferring only the minimal delta needed between repositories. A well-tuned Git server can handle hundreds of concurrent clone/fetch/push operations with proper resource allocation." - [GitHub's Engineering Blog on Scaling Git](https://github.blog/2016-04-01-how-github-improved-performance-git-push-operations/)

For large-scale deployments, consider:

```
# Synchronization optimization examples
git gc --aggressive      # Compress repository storage
git repack -a -d -f      # Optimize repository packing
git reflog expire --all  # Clean up reference logs
```

## Performance Benchmarks

ChronDB's performance can be evaluated along several dimensions:

| Operation               | Small DB (<10K docs)                 | Medium DB (<100K docs) | Large DB (>100K docs) |
| ----------------------- | ------------------------------------ | ---------------------- | --------------------- |
| Read (latest)           | <5ms                                 | 5-20ms                 | 10-50ms               |
| Read (historical)       | 5-15ms                               | 15-50ms                | 50-200ms              |
| Write (single doc)      | 10-20ms                              | 20-50ms                | 50-200ms              |
| Batch writes (100 docs) | 200-500ms                            | 500-1500ms             | 1500-5000ms           |
| Synchronization         | Depends on network and change volume |                        |                       |

*Note: These are approximate figures and may vary based on hardware, configuration, and access patterns.*

## Monitoring ChronDB Performance

To ensure optimal performance, monitor key metrics:

```bash
# Example: Check repository size
du -sh /path/to/chrondb/repo

# Example: Count objects in repository
git count-objects -v

# Example: Check recent operations timing
chrondb.stats.timing
```

## Conclusion

ChronDB provides excellent performance for most use cases by leveraging Git's efficient storage model. For large-scale deployments, additional planning and optimization may be required to maintain optimal performance.

By understanding the underlying Git performance characteristics and following the optimization strategies outlined here, you can ensure ChronDB performs well as your data and usage grow.


# Benchmark Results

This document describes how to run and interpret the ChronDB SQL protocol benchmark tests.

## About the Benchmark

The benchmark is designed to measure the performance of ChronDB's SQL protocol with large datasets (1GB+). It tests various SQL operations including:

* SELECT with large data volumes
* SEARCH queries (using WHERE clause with LIKE)
* Queries with INNER JOIN
* Queries with LEFT JOIN
* INSERT operations

## Running the Benchmark

### Locally

To run the benchmark locally, use the provided script:

```bash
./scripts/run_benchmark.sh
```

This will:

1. Generate approximately 1GB of test data
2. Run all benchmark operations
3. Save the results to a timestamped file (`benchmark_results_YYYY-MM-DD_HH-MM-SS.txt`)

### Via GitHub Actions

The benchmark can also be run automatically through GitHub Actions:

1. It runs automatically once a week (Sunday at midnight)
2. It can be manually triggered at any time through the GitHub interface:
   * Go to the "Actions" tab in the repository
   * Select the "SQL Protocol Benchmark" workflow
   * Click "Run workflow"

Results are available as workflow artifacts for 90 days.

## Interpreting the Results

The results file contains detailed information about the performance of each SQL operation:

* **SELECT 1000 records**: Time to fetch 1000 records from the main table
* **SEARCH records**: Time to perform a query with LIKE filter
* **INNER JOIN query**: Time to perform a query with INNER JOIN between tables
* **LEFT JOIN query**: Time to perform a query with LEFT JOIN between tables
* **Average INSERT time**: Average time to insert a complete document

Use these results to:

1. Compare performance between different versions of ChronDB
2. Identify performance bottlenecks in specific operations
3. Validate performance improvements after optimizations

## Important Notes

* The benchmark requires at least 4GB of available memory
* The complete execution may take several minutes
* Results may vary depending on hardware and execution environment
* Benchmark failures do not impact CI/CD workflows (continue-on-error is enabled)


# Troubleshooting

Common issues and their solutions when running ChronDB.

## Startup Issues

### Port Already in Use

```
java.net.BindException: Address already in use
```

Another process is using the port. Check which process holds it and either stop that process or change the ChronDB port:

```bash
# Find what's using port 3000
lsof -i :3000

# Run ChronDB on different ports
clojure -M:run 8080 6380
```

Or disable unused protocols:

```bash
clojure -M:run --disable-redis --disable-sql
```

### Out of Memory on Startup

```
java.lang.OutOfMemoryError: Java heap space
```

ChronDB needs enough memory for the Lucene index and Git operations. Increase JVM heap:

```bash
java -Xmx2g -jar chrondb.jar
```

For Docker deployments, ensure the container memory limit matches the JVM heap:

```yaml
deploy:
  resources:
    limits:
      memory: 2g
```

### Configuration File Not Found

```
Could not find config file: config.edn
```

ChronDB looks for `config.edn` in the current directory by default. Specify the path explicitly:

```bash
java -jar chrondb.jar --config /path/to/config.edn
```

***

## Storage Issues

### Git Lock Files

```
Cannot lock refs/heads/main. Unable to create new lock file
```

A previous ChronDB process crashed or was killed without releasing Git locks. Remove stale lock files:

```bash
# Find lock files in the data directory
find data/ -name "*.lock" -type f

# Remove stale locks (ensure ChronDB is stopped first)
find data/ -name "*.lock" -type f -delete
```

ChronDB automatically detects and removes lock files older than 60 seconds on startup. If you see this error during normal operation, it may indicate concurrent processes accessing the same data directory. Only one ChronDB instance should use a given data directory.

### Disk Space Exhausted

```
No space left on device
```

Every write creates a Git commit. Over time, history accumulates. To reclaim space:

```bash
# Run Git garbage collection via ChronDB
java -jar chrondb.jar --command compact

# Or manually in the data directory
cd data/ && git gc --aggressive
```

Monitor disk usage proactively:

```bash
# Check data directory size
du -sh data/

# Check Git object database size
du -sh data/.git/objects/
```

The `/health` endpoint reports disk space status. Configure alerts for when free disk drops below 5 GB.

### Corrupted Git Repository

```
org.eclipse.jgit.errors.CorruptObjectException
```

If the Git repository is corrupted (e.g., due to a hard crash during a write):

```bash
# Verify repository integrity
cd data/ && git fsck --full

# Attempt automatic repair
cd data/ && git fsck --full --repair
```

If repair fails, restore from a backup:

```bash
clojure -M:run restore --input /path/to/backup.tar.gz
```

Always maintain regular backups. See the [Operations Guide](https://github.com/avelino/chrondb/blob/main/docs/operations/README.md) for backup procedures.

***

## Index Issues

### Lucene Index Corruption

```
org.apache.lucene.index.CorruptIndexException
```

If the Lucene index is corrupted, ChronDB can rebuild it from the Git repository. The background reindexing process runs automatically on startup. To force a full reindex:

1. Stop ChronDB
2. Delete the index directory (default: `data/index/` or the configured index path)
3. Restart ChronDB — the reindexer will rebuild the index from Git commits

```bash
rm -rf data/index/
java -jar chrondb.jar
```

Monitor reindexing progress in the logs:

```
INFO chrondb.index.lucene - Starting background reindexing
INFO chrondb.index.lucene - Reindexing complete: 15000 documents indexed
```

### Search Returns Stale Results

The Lucene Near-Real-Time (NRT) reader refreshes automatically after writes. If search results seem stale:

1. Check that the write operation completed successfully (no errors in logs)
2. Check the `chrondb_wal_pending_entries` metric — a high value means writes are queued
3. If using remote sync, the document may not have been pulled yet

***

## Protocol-Specific Issues

### REST API: Connection Refused

```
curl: (7) Failed to connect to localhost port 3000: Connection refused
```

1. Verify ChronDB is running: check the process and logs
2. Verify the REST protocol is enabled in `config.edn` (or not disabled via `--disable-rest`)
3. Check if it's bound to the correct interface (default `0.0.0.0` accepts all)
4. Check health: `curl http://localhost:3000/healthz`

### Redis: Authentication Error

ChronDB's Redis protocol does not currently support AUTH commands. If your Redis client requires authentication, configure it to connect without a password, or use a proxy that handles authentication.

### PostgreSQL: Connection Error

```
psql: error: connection to server failed
```

1. Verify the PostgreSQL protocol is enabled and the port is correct
2. Use the configured username and password:

```bash
psql -h localhost -p 5432 -U chrondb -W
# Enter password: chrondb (default)
```

3. Check that `config.edn` has the PostgreSQL server enabled:

```clojure
:servers {:postgresql {:enabled true :port 5432 :username "chrondb" :password "chrondb"}}
```

### PostgreSQL: Unsupported SQL Feature

```
ERROR: Unsupported SQL statement
```

ChronDB implements a subset of SQL. See [Protocols](https://github.com/avelino/chrondb/blob/main/docs/protocols/README.md) for the complete list of supported SQL features. Common limitations:

* No subqueries (yet)
* No window functions (yet)
* No HAVING clause (yet)
* Only INNER JOIN and LEFT JOIN supported

***

## Performance Issues

### Slow Read Operations

If GET operations are slow:

1. **Check document count** — read performance degrades with repository size because of path resolution overhead. This is a known limitation (see issue #109).
2. **Check disk I/O** — Git operations are I/O bound. Use SSDs for the data directory.
3. **Check memory** — insufficient memory forces the JVM to GC frequently. Monitor with `/health`:

```bash
curl -s http://localhost:3000/health | jq '.checks[] | select(.component == "memory")'
```

### Slow Queries

If SQL queries or search operations are slow:

1. **Check the query** — ChronDB currently loads all documents for a table before filtering (see issue #115). Avoid `SELECT *` on large tables when possible.
2. **Use targeted queries** — queries that match indexed fields (via Lucene) are faster than full table scans.
3. **Limit results** — always use `LIMIT` in SQL queries and `limit` parameter in search.
4. **Check index health** — the `chrondb_index_documents` gauge should match your expected document count.

### High Memory Usage

1. Check for large WAL backlogs: `chrondb_wal_pending_entries` should be near 0
2. Check for many concurrent connections: `chrondb_active_connections`
3. Run compaction to optimize Git storage: `java -jar chrondb.jar --command compact`
4. Consider increasing the JVM heap if usage is consistently above 85%

***

## Diagnostic Tools

ChronDB includes built-in diagnostic tools:

### Repository Diagnostics

```bash
# Run diagnostics on the data directory
clojure -M:run diagnose --data-dir data/
```

This checks:

* Git repository integrity
* Lock file presence
* Branch configuration
* Remote sync status

### Data Dump

```bash
# Export all documents for inspection
clojure -M:run dump --data-dir data/ --output dump.json
```

### WAL Status

Check the Write-Ahead Log for pending or failed operations:

```bash
curl -s http://localhost:3000/health | jq '.checks[] | select(.component == "wal")'
```

A healthy WAL should show `"pending-entries": 0`. Values above 100 indicate the WAL is backing up, which may mean storage is slow or experiencing errors.

***

## Getting Help

If your issue is not covered here:

1. Check the [FAQ](https://github.com/avelino/chrondb/blob/main/docs/faq/README.md) for common questions
2. Search [existing issues](https://github.com/avelino/chrondb/issues)
3. Open a new issue with:
   * ChronDB version
   * Operating system and Java version
   * Full error message and stack trace
   * Steps to reproduce
4. Join the [discussions](https://github.com/avelino/chrondb/discussions) for questions


# Security Best Practices

This guide covers security considerations for deploying ChronDB in production environments.

## Network Security

### Bind to Specific Interfaces

By default, ChronDB binds to `0.0.0.0` (all interfaces). In production, bind to specific interfaces:

```clojure
:servers {
  :rest {:host "127.0.0.1" :port 3000}       ;; Localhost only
  :redis {:host "10.0.0.5" :port 6379}       ;; Internal network only
  :postgresql {:host "10.0.0.5" :port 5432}  ;; Internal network only
}
```

### Disable Unused Protocols

Only enable the protocols your application uses:

```bash
# REST API only
clojure -M:run --disable-redis --disable-sql

# Redis only
clojure -M:run --disable-rest --disable-sql
```

Or in `config.edn`:

```clojure
:servers {
  :rest {:enabled true}
  :redis {:enabled false}
  :postgresql {:enabled false}
}
```

### Firewall Rules

Restrict access to ChronDB ports from trusted networks only:

```bash
# Allow REST API only from internal network
iptables -A INPUT -p tcp --dport 3000 -s 10.0.0.0/8 -j ACCEPT
iptables -A INPUT -p tcp --dport 3000 -j DROP

# Allow PostgreSQL from application servers only
iptables -A INPUT -p tcp --dport 5432 -s 10.0.1.0/24 -j ACCEPT
iptables -A INPUT -p tcp --dport 5432 -j DROP
```

### TLS Termination

ChronDB does not handle TLS directly. Use a reverse proxy for HTTPS:

**nginx example:**

```nginx
server {
    listen 443 ssl;
    server_name chrondb.internal;

    ssl_certificate /etc/ssl/certs/chrondb.crt;
    ssl_certificate_key /etc/ssl/private/chrondb.key;

    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}
```

For the Redis and PostgreSQL protocols, use [stunnel](https://www.stunnel.org/) or a similar TLS wrapper if encryption in transit is required.

***

## Authentication

### PostgreSQL Protocol

The PostgreSQL wire protocol supports username/password authentication:

```clojure
:servers {
  :postgresql {
    :enabled true
    :port 5432
    :username "chrondb"
    :password "a-strong-password-here"
  }
}
```

Change the default credentials before deploying to production.

### REST API

The REST API does not include built-in authentication. Implement authentication at the reverse proxy layer:

* **Basic Auth**: via nginx `auth_basic` or similar
* **OAuth2/JWT**: via an API gateway (Kong, Envoy, AWS ALB)
* **mTLS**: mutual TLS at the proxy layer for service-to-service communication

### Redis Protocol

The Redis protocol does not currently support the `AUTH` command. Protect it via network isolation (bind to internal interfaces, firewall rules).

***

## File System Permissions

### Data Directory

The data directory contains the Git repository and Lucene index. Restrict access:

```bash
# Create dedicated user
sudo useradd --system --no-create-home chrondb

# Set ownership
sudo chown -R chrondb:chrondb /var/lib/chrondb/data

# Restrict permissions (owner only)
sudo chmod -R 700 /var/lib/chrondb/data
```

### Configuration File

The configuration file may contain credentials (PostgreSQL password, SSH key paths):

```bash
sudo chown chrondb:chrondb /etc/chrondb/config.edn
sudo chmod 600 /etc/chrondb/config.edn
```

***

## Docker Security

The official Docker image follows security best practices:

* **Non-root user**: runs as the `chrondb` user (not root)
* **Minimal base image**: Debian 12 slim with only required dependencies
* **No shell access needed**: entry point is the compiled binary

### Additional Hardening

```yaml
services:
  chrondb:
    image: ghcr.io/avelino/chrondb:latest
    read_only: true
    tmpfs:
      - /tmp:size=100M
    volumes:
      - chrondb-data:/app/data
    security_opt:
      - no-new-privileges:true
    cap_drop:
      - ALL
```

### Kubernetes Security Context

```yaml
securityContext:
  runAsNonRoot: true
  runAsUser: 1000
  readOnlyRootFilesystem: true
  allowPrivilegeEscalation: false
  capabilities:
    drop:
      - ALL
```

***

## Remote Git Security

When using remote Git synchronization, ChronDB connects to a remote repository. Secure this connection:

### SSH Keys

Use deploy keys with minimal permissions:

```bash
# Generate a dedicated key pair
ssh-keygen -t ed25519 -f /etc/chrondb/deploy_key -N "" -C "chrondb-deploy"

# Add the public key to your Git hosting as a deploy key (read/write)
```

Configure in `config.edn`:

```clojure
:git {
  :remote-url "git@github.com:your-org/chrondb-data.git"
  :ssh {:ssh-dir "/etc/chrondb/.ssh"
        :auth-methods "publickey"}
}
```

### Key Permissions

```bash
chmod 700 /etc/chrondb/.ssh
chmod 600 /etc/chrondb/.ssh/deploy_key
chmod 644 /etc/chrondb/.ssh/deploy_key.pub
chown -R chrondb:chrondb /etc/chrondb/.ssh
```

### Repository Access

* Use a **dedicated repository** for ChronDB data (not a shared repo)
* Grant **minimal permissions** (deploy key with write access only to the data repo)
* Enable **branch protection** on the remote to prevent force pushes
* Enable **audit logging** on your Git hosting to track access

***

## Backup Security

### Backup Storage

* Store backups in a different location than the primary data (different disk, cloud storage)
* Encrypt backups at rest if they contain sensitive data:

```bash
# Create and encrypt a backup
clojure -M:run backup --output /tmp/backup.tar.gz
gpg --symmetric --cipher-algo AES256 /tmp/backup.tar.gz
rm /tmp/backup.tar.gz

# Decrypt before restoring
gpg --decrypt backup.tar.gz.gpg > backup.tar.gz
clojure -M:run restore --input backup.tar.gz
```

### Backup Access Control

* Restrict who can trigger backup/restore via the REST API
* The `/api/v1/backup` and `/api/v1/restore` endpoints should be behind authentication
* Consider disabling these endpoints in production and using CLI-based backups instead

***

## Audit Trail

ChronDB's Git-based storage provides a built-in audit trail:

* Every write creates an immutable Git commit with timestamp and metadata
* Transaction metadata (user, origin, flags) is stored in Git notes (`refs/notes/chrondb`)
* History cannot be altered without detection (Git hash chain integrity)

### Inspecting the Audit Trail

```bash
# View commit history with notes
cd data/ && git log --show-notes=chrondb

# View metadata for a specific commit
cd data/ && git notes --ref=chrondb show <commit-hash>

# Via SQL
SELECT * FROM chrondb_history('user', '1');
```

### Enriching the Audit Trail

Always set transaction metadata headers on write operations:

```bash
curl -X POST http://localhost:3000/api/v1/save \
  -H "Content-Type: application/json" \
  -H "X-ChronDB-User: alice@example.com" \
  -H "X-ChronDB-Origin: admin-panel" \
  -H "X-ChronDB-Flags: manual-edit" \
  -H "X-Request-Id: req-abc-123" \
  -d '{"id":"user:1","name":"Alice"}'
```

***

## Security Monitoring

### What to Monitor

| Signal                | Metric / Log                          | Action                                       |
| --------------------- | ------------------------------------- | -------------------------------------------- |
| High error rate       | `chrondb_occ_conflicts_total`         | May indicate concurrent modification attacks |
| Unusual write volume  | `chrondb_documents_saved_total` spike | May indicate unauthorized bulk writes        |
| Failed connections    | Application logs                      | May indicate port scanning or brute force    |
| Disk space exhaustion | `/health` disk check                  | May indicate DoS via document flooding       |
| WAL backlog           | `chrondb_wal_pending_entries`         | May indicate storage issues or attack        |

### Log Review

Regularly review ChronDB logs for:

* Unknown client connections
* Unusual query patterns
* Repeated errors on specific documents
* Operations from unexpected origins (check Git notes)


# Contributing Guide

Thank you for your interest in contributing to ChronDB! This guide will help you get started.

## Prerequisites

| Tool           | Version | Purpose                              |
| -------------- | ------- | ------------------------------------ |
| Java / GraalVM | 17+     | Runtime and native-image compilation |
| Clojure CLI    | 1.11+   | Build tool and dependency management |
| Git            | 2.25+   | Required by ChronDB's storage engine |

Install Clojure CLI: <https://clojure.org/guides/install\\_clojure>

## Development Setup

```bash
# Clone the repository
git clone https://github.com/avelino/chrondb.git
cd chrondb

# Download dependencies
clojure -P

# Run the full server (REST + Redis + SQL)
clojure -M:run

# Run individual protocols
clojure -M:run-rest    # REST API only (port 3000)
clojure -M:run-redis   # Redis only (port 6379)
clojure -M:run-sql     # PostgreSQL only (port 5432)
```

Verify it's working:

```bash
# REST API
curl http://localhost:3000/healthz

# Redis
redis-cli -p 6379 PING

# PostgreSQL
psql -h localhost -p 5432 -U chrondb -c "SHOW TABLES"
```

## Running Tests

```bash
# Full test suite (recommended before submitting a PR)
clojure -M:test

# Core logic only (faster, runs in parallel)
clojure -M:test-non-external-protocol

# Protocol-specific tests (run sequentially due to port binding)
clojure -M:test-redis-sequential
clojure -M:test-sql-only

# Benchmarks
clojure -M:benchmark
```

## Linting and Formatting

```bash
# Static analysis with clj-kondo
clojure -M:lint

# Check formatting
clojure -M:fmt

# Auto-fix formatting
clojure -M:fmt-fix
```

Run both lint and format checks before submitting a PR.

## Project Structure

```
src/chrondb/
  core.clj              # Entry point and CLI dispatcher
  config.clj            # Configuration loader
  api/                  # Protocol implementations (REST, Redis, SQL)
  storage/              # Git-based storage layer
  index/                # Lucene search index
  query/                # Query AST builders
  transaction/          # Transaction context and metadata
  wal/                  # Write-Ahead Log
  backup/               # Backup and restore
  validation/           # JSON Schema validation
  observability/        # Health checks and metrics
  lib/                  # FFI entry points for native bindings
test/chrondb/           # Test suite (mirrors src structure)
bindings/               # Language bindings (Rust, Python, Node.js)
docs/                   # Documentation (served at chrondb.avelino.run)
```

For a detailed architecture overview, see [Architecture](/key-concepts/architecture).

## Code Style

* **Naming**: `kebab-case` for all vars and namespaces
* **Namespaces**: follow `chrondb.*` convention
* **Docstrings**: required for all public functions
* **Functions**: should do one thing well, max \~50 lines
* **Error handling**: fail fast with structured context (include what, where, and relevant data)
* **Formatting**: enforced by `cljfmt` — run `clojure -M:fmt-fix` before committing
* **Immutability**: never rewrite Git commits; create new ones

## Making Changes

### Common Patterns

**Adding a Redis command:**

1. Add the handler function in `src/chrondb/api/redis/core.clj`
2. Register it in the command dispatcher
3. Add tests in `test/chrondb/api/redis/`
4. Update `docs/protocols.md` with the new command

**Adding a SQL feature:**

1. Extend the parser in `src/chrondb/api/sql/parser/`
2. Add execution logic in `src/chrondb/api/sql/execution/`
3. Add tests in `test/chrondb/api/sql/`
4. Update `docs/protocols.md` with the new syntax

**Adding a REST endpoint:**

1. Add the handler in `src/chrondb/api/v1.clj`
2. Add the route in `src/chrondb/api/v1/routes.clj`
3. Add tests in `test/chrondb/api/`
4. Update `docs/protocols.md` and `docs/examples-rest.md`

**Adding an FFI operation:**

1. Add the Clojure function in `src/chrondb/lib/core.clj`
2. Add the Java bridge method in `java/chrondb/lib/ChronDBLib.java`
3. Update the UniFFI definition in `bindings/uniffi/src/chrondb.udl`
4. Update each binding (Rust, Python, Node.js)
5. Add tests in binding test directories

### Building Native Image

```bash
# Generate GraalVM args
clojure -M:shared-lib

# Build native library
native-image @target/shared-image-args -H:Name=libchrondb
```

## Pull Request Process

1. Create a feature branch from `main`
2. Make your changes with clear, focused commits
3. Ensure all tests pass: `clojure -M:test`
4. Ensure code is formatted: `clojure -M:fmt`
5. Ensure linting passes: `clojure -M:lint`
6. Open a PR with a clear description of what changed and why
7. Link any related issues

### PR Checklist

* [ ] Tests added/updated for new behavior
* [ ] All existing tests pass
* [ ] Code formatted with `cljfmt`
* [ ] No linting warnings
* [ ] Documentation updated (if user-facing changes)
* [ ] Consistent behavior across REST, Redis, and SQL protocols (if applicable)

## Questions?

* **Issues**: <https://github.com/avelino/chrondb/issues>
* **Discussions**: <https://github.com/avelino/chrondb/discussions>
* **Documentation**: <https://chrondb.avelino.run>


# Release Notes

This is the first official release of ChronDB, a chronological key/value database based on a Git architecture.

## Main Features

### Schema Validation

* **Optional JSON Schema Validation** - Per-namespace validation with configurable modes (strict/warning/disabled) ([#55](https://github.com/avelino/chrondb/issues/55))
  * REST API endpoints for schema management (`/api/v1/schemas/validation`)
  * Redis commands (`SCHEMA.SET`, `SCHEMA.GET`, `SCHEMA.DEL`, `SCHEMA.LIST`, `SCHEMA.VALIDATE`)
  * SQL DDL statements (`CREATE/DROP/SHOW VALIDATION SCHEMA`)
  * Supports JSON Schema Draft-07, 2019-09, and 2020-12
  * Version-controlled schema history

### Communication Protocols

* **PostgreSQL Protocol Support** - Implementation of the PostgreSQL protocol for communication with ChronDB, allowing compatibility with existing PostgreSQL tools and clients ([#16](https://github.com/avelino/chrondb/pull/16))
* **Redis Protocol Server Implementation** - Support for the Redis protocol, expanding database communication options ([#11](https://github.com/avelino/chrondb/pull/11))

### Storage

* **Virtual Git Storage Implementation** - Rewrite of the Git storage implementation to use virtual commits, improving performance and scalability ([#9](https://github.com/avelino/chrondb/pull/9))
* **File Repository Support** - Added support for file-based storage ([#6](https://github.com/avelino/chrondb/pull/6))
* **Initial JSON Compression** - Implementation of JSON compression to optimize data storage ([#1](https://github.com/avelino/chrondb/pull/1))

## Performance Improvements

### SQL Driver

* **Removal of Unnecessary GROUP BY Operations** - Fixed an issue where the GROUP BY clause was being unnecessarily applied in SQL queries, significantly improving performance ([#17](https://github.com/avelino/chrondb/pull/17))
* **Client Connection Handling Improvements** - Optimization of connection handling for greater stability and performance ([#17](https://github.com/avelino/chrondb/pull/17))

## Tools and Utilities

* **Diagnostic and Dump Tools** - Added new tools for diagnostics and data dumping, facilitating debugging and analysis ([#17](https://github.com/avelino/chrondb/pull/17))

## Refactoring and Internal Improvements

* **New Package Architecture** - Restructuring of the package architecture for better code organization and maintainability ([#7](https://github.com/avelino/chrondb/pull/7))
* **GitHub Actions and Binding Renaming** - Implementation of GitHub Actions for continuous integration and adjustments to binding names ([#2](https://github.com/avelino/chrondb/pull/2))

## Note

This release represents the collaborative effort of the community to create a robust and efficient chronological key/value database. We thank all contributors who made this version possible.

### Licensing Update

* ChronDB is now distributed under the GNU Affero General Public License v3.0 (AGPLv3) to ensure modifications deployed as network services remain available to the community.