Codecs

Codecs handle the marshalling and unmarshalling of Go types to and from byte slices for storage in KV backends. The Store encodes values through the codec before passing them to the driver, and decodes them on retrieval.

Codec Interface

Every codec implements three methods:

type Codec interface {
    // Encode serializes v into bytes.
    Encode(v any) ([]byte, error)

    // Decode deserializes data into v. v must be a pointer.
    Decode(data []byte, v any) error

    // Name returns the codec identifier (e.g., "json", "msgpack", "protobuf").
    Name() string
}

Built-in Codecs

Grove KV ships with four codecs in the github.com/xraph/grove/kv/codec package:

JSON (default)

Uses encoding/json. This is the default codec when no codec is specified.

import "github.com/xraph/grove/kv/codec"

store, err := kv.Open(rdb, kv.WithCodec(codec.JSON()))

• Human-readable output
• Broad compatibility with external systems
• Supports all Go types that encoding/json supports
• Largest payload size of the built-in codecs

MsgPack

Uses github.com/vmihailenco/msgpack/v5. More compact than JSON and faster to encode/decode.

store, err := kv.Open(rdb, kv.WithCodec(codec.MsgPack()))

• 30-50% smaller payloads than JSON
• Faster serialization than JSON
• Binary format (not human-readable)
• Good default for production workloads where payload size matters

Protobuf

Uses google.golang.org/protobuf/proto. Values must implement proto.Message.

store, err := kv.Open(rdb, kv.WithCodec(codec.Protobuf{}))

• Smallest payload size
• Fastest serialization
• Requires .proto schema definitions and code generation
• Values must implement proto.Message -- will return an error otherwise

Gob

Uses encoding/gob. Go-native binary encoding.

store, err := kv.Open(rdb, kv.WithCodec(codec.Gob{}))

• Go-native format with good type fidelity
• Handles interface types (must register with gob.Register)
• Not compatible with non-Go consumers
• Slightly larger than MsgPack

Using a Codec

Store-level

Set the codec when opening the store. All Get and Set operations will use it:

store, err := kv.Open(rdb, kv.WithCodec(codec.MsgPack()))

Per-keyspace

Override the store-level codec for a specific keyspace:

import "github.com/xraph/grove/kv/keyspace"

// This keyspace uses MsgPack even if the store uses JSON.
users := keyspace.New[User](store, "users",
    keyspace.WithCodec(codec.MsgPack()),
)

This is useful when different keyspaces have different serialization requirements -- for example, a session keyspace might use MsgPack for speed while a configuration keyspace uses JSON for debuggability.

Raw bypass

If you need to handle serialization yourself, use GetRaw and SetRaw on the Store to bypass the codec entirely:

raw, err := store.GetRaw(ctx, "key")
err = store.SetRaw(ctx, "key", myBytes)

When to Use Each Codec

Recommendation: Start with JSON for development. Switch to MsgPack for production workloads where payload size or throughput matters. Use Protobuf when you already have .proto schemas. Use Gob for Go-only systems that need to serialize complex interface types.

Implementing a Custom Codec

Implement the Codec interface to add support for any serialization format:

package mycodec

import (
    "github.com/xraph/grove/kv/codec"
    "github.com/fxamacker/cbor/v2"
)

// CBOR implements codec.Codec using CBOR (RFC 8949).
type CBOR struct{}

// Verify interface compliance at compile time.
var _ codec.Codec = CBOR{}

func (CBOR) Name() string { return "cbor" }

func (CBOR) Encode(v any) ([]byte, error) {
    return cbor.Marshal(v)
}

func (CBOR) Decode(data []byte, v any) error {
    return cbor.Unmarshal(data, v)
}

Use it like any built-in codec:

store, err := kv.Open(rdb, kv.WithCodec(mycodec.CBOR{}))

Implementation notes

• Encode receives an any value. It may be a pointer or a value type.
• Decode receives a pointer (v is always a pointer). Unmarshal into it.
• Name should return a stable, lowercase identifier. It is used for logging and diagnostics.
• Return clear errors from Encode and Decode -- the Store wraps them with kv.ErrCodecEncode and kv.ErrCodecDecode for consistent error handling.