storage

package
v1.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 11, 2025 License: MIT Imports: 11 Imported by: 0

Documentation

Overview

Package storage provides storage interfaces and implementations for the Redis in-memory replica.

The storage layer abstracts the underlying data storage mechanism, allowing for different implementations (in-memory, persistent, etc.).

Basic usage:

storage := storage.NewMemory()
err := storage.Set("key", []byte("value"), nil)
value, exists := storage.Get("key")

The package supports:

  • Thread-safe operations
  • Expiration handling
  • Memory usage tracking
  • Key enumeration
  • Atomic operations

Index

Constants

This section is empty.

Variables

View Source
var CleanupConfigBestPerformance = CleanupConfig{
	SampleSize:       100,
	MaxRounds:        10,
	BatchSize:        50,
	ExpiredThreshold: 0.1,
}

CleanupConfigBestPerformance optimized for maximum cleanup throughput Uses larger batches and more aggressive settings for heavy workloads

View Source
var CleanupConfigDefault = CleanupConfig{
	SampleSize:       20,
	MaxRounds:        4,
	BatchSize:        10,
	ExpiredThreshold: 0.25,
}

CleanupConfigDefault provides balanced performance for most use cases Similar to Redis native behavior with good performance/resource balance

View Source
var CleanupConfigLargeDataset = CleanupConfig{
	SampleSize:       50,
	MaxRounds:        8,
	BatchSize:        25,
	ExpiredThreshold: 0.15,
}

CleanupConfigLargeDataset optimized for datasets with > 100,000 keys More aggressive cleanup for large-scale deployments

View Source
var CleanupConfigLowLatency = CleanupConfig{
	SampleSize:       15,
	MaxRounds:        3,
	BatchSize:        8,
	ExpiredThreshold: 0.4,
}

CleanupConfigLowLatency optimized for latency-sensitive applications Minimizes cleanup impact on application response times

View Source
var CleanupConfigMediumDataset = CleanupConfig{
	SampleSize:       25,
	MaxRounds:        5,
	BatchSize:        12,
	ExpiredThreshold: 0.3,
}

CleanupConfigMediumDataset optimized for datasets with 10,000-100,000 keys Balanced approach with moderate resource usage

View Source
var CleanupConfigSmallDataset = CleanupConfig{
	SampleSize:       10,
	MaxRounds:        2,
	BatchSize:        5,
	ExpiredThreshold: 0.5,
}

CleanupConfigSmallDataset optimized for datasets with < 10,000 keys More conservative approach to minimize overhead

Functions

func MatchPatternWithStrategy added in v1.1.0

func MatchPatternWithStrategy(str, pattern string, strategy MatchingStrategy) bool

MatchPatternWithStrategy matches a string against a pattern using the specified strategy

func SetMatchingStrategy added in v1.1.0

func SetMatchingStrategy(strategy MatchingStrategy)

SetMatchingStrategy sets the global matching strategy

Types

type CleanupConfig added in v1.1.0

type CleanupConfig struct {
	// SampleSize is the number of keys to sample per round
	SampleSize int
	// MaxRounds is the maximum number of rounds per cleanup cycle
	MaxRounds int
	// BatchSize is the number of keys to delete in each batch
	BatchSize int
	// ExpiredThreshold continues cleanup if this percentage of sampled keys are expired
	ExpiredThreshold float64
}

CleanupConfig holds configuration for incremental cleanup

type CleanupConfigurableStorage added in v1.1.0

type CleanupConfigurableStorage interface {
	Storage

	// Cleanup configuration
	SetCleanupConfig(config CleanupConfig)
	GetCleanupConfig() CleanupConfig
}

CleanupConfigurableStorage extends Storage with cleanup configuration

type HashValue

type HashValue struct {
	Fields map[string][]byte
}

HashValue represents a hash value

type KeyInfo

type KeyInfo struct {
	Key        string
	Type       ValueType
	TTL        time.Duration // -1 for no expiry, -2 for expired/not found
	Size       int64         // Size in bytes
	LastAccess time.Time
}

KeyInfo provides metadata about a key

type ListValue

type ListValue struct {
	Elements [][]byte
}

ListValue represents a list value

type MatchingConfig added in v1.1.0

type MatchingConfig struct {
	// contains filtered or unexported fields
}

MatchingConfig holds configuration for pattern matching

type MatchingStrategy added in v1.1.0

type MatchingStrategy int

MatchingStrategy defines the algorithm used for pattern matching

const (
	// StrategySimple uses optimized algorithms for common patterns (single wildcard)
	StrategySimple MatchingStrategy = iota
	// StrategyRegex uses regular expressions for complex patterns
	StrategyRegex
	// StrategyAutomaton uses finite state machines for pattern matching
	StrategyAutomaton
	// StrategyGlob uses Go's standard library filepath.Match
	StrategyGlob
)

func GetMatchingStrategy added in v1.1.0

func GetMatchingStrategy() MatchingStrategy

GetMatchingStrategy returns the current global matching strategy

type MemoryLimitedStorage

type MemoryLimitedStorage interface {
	Storage

	// Memory management
	SetMemoryLimit(bytes int64)
	GetMemoryLimit() int64
	EvictLRU(count int) int64
}

MemoryLimitedStorage extends Storage with memory management

type MemoryOption added in v1.4.0

type MemoryOption func(*MemoryStorage)

MemoryOption is a function that configures a MemoryStorage instance

func WithShardCount added in v1.4.0

func WithShardCount(count int) MemoryOption

WithShardCount sets the number of shards for the storage The number is automatically rounded up to the next power of 2 for optimal performance

type MemoryStorage

type MemoryStorage struct {
	// contains filtered or unexported fields
}

MemoryStorage implements an in-memory storage engine

func NewMemory

func NewMemory(opts ...MemoryOption) *MemoryStorage

NewMemory creates a new in-memory storage instance with default number of shards (64)

func NewMemoryWithShards added in v1.4.0

func NewMemoryWithShards(numShards int) *MemoryStorage

NewMemoryWithShards creates a new in-memory storage instance with specified number of shards Deprecated: Use NewMemory(WithShardCount(n)) instead The number of shards is automatically rounded up to the next power of 2 for optimal performance

func (*MemoryStorage) AddObserver

func (s *MemoryStorage) AddObserver(observer StorageObserver)

AddObserver adds a storage observer

func (*MemoryStorage) Close

func (s *MemoryStorage) Close() error

Close shuts down the storage

func (*MemoryStorage) CurrentDB

func (s *MemoryStorage) CurrentDB() int

CurrentDB returns the current database number

func (*MemoryStorage) DatabaseInfo added in v1.4.0

func (s *MemoryStorage) DatabaseInfo() map[int]map[string]interface{}

DatabaseInfo returns information about all databases with keys For databases with many keys, it uses sampling to estimate expired count for performance

func (*MemoryStorage) Del

func (s *MemoryStorage) Del(keys ...string) int64

Del deletes one or more keys

func (*MemoryStorage) EvictLRU

func (s *MemoryStorage) EvictLRU(count int) int64

EvictLRU evicts the least recently used keys

func (*MemoryStorage) Exists

func (s *MemoryStorage) Exists(keys ...string) int64

Exists checks if keys exist

func (*MemoryStorage) Expire

func (s *MemoryStorage) Expire(key string, expiry time.Time) bool

Expire sets expiration for a key

func (*MemoryStorage) FlushAll

func (s *MemoryStorage) FlushAll() error

FlushAll removes all keys from all databases

func (*MemoryStorage) Get

func (s *MemoryStorage) Get(key string) ([]byte, bool)

Get retrieves a value by key

func (*MemoryStorage) GetCleanupConfig added in v1.1.0

func (s *MemoryStorage) GetCleanupConfig() CleanupConfig

GetCleanupConfig returns the current cleanup configuration

func (*MemoryStorage) GetMemoryLimit

func (s *MemoryStorage) GetMemoryLimit() int64

GetMemoryLimit returns the current memory limit

func (*MemoryStorage) Info

func (s *MemoryStorage) Info() map[string]interface{}

Info returns storage information

func (*MemoryStorage) KeyCount

func (s *MemoryStorage) KeyCount() int64

KeyCount returns the number of keys in the current database

func (*MemoryStorage) Keys

func (s *MemoryStorage) Keys(pattern string) []string

Keys returns all keys matching the pattern in the current database Pattern supports glob-style patterns: * matches any number of characters (including zero) ? matches a single character [abc] matches any character in the brackets [a-z] matches any character in the range

func (*MemoryStorage) MemoryUsage

func (s *MemoryStorage) MemoryUsage() int64

MemoryUsage returns current memory usage in bytes

func (*MemoryStorage) PTTL added in v1.3.0

func (s *MemoryStorage) PTTL(key string) time.Duration

PTTL returns the time to live for a key in milliseconds

func (*MemoryStorage) Scan

func (s *MemoryStorage) Scan(cursor int64, match string, count int64) (int64, []string)

Scan provides cursor-based iteration over keys

func (*MemoryStorage) SelectDB

func (s *MemoryStorage) SelectDB(db int) error

SelectDB selects a database

func (*MemoryStorage) Set

func (s *MemoryStorage) Set(key string, value []byte, expiry *time.Time) error

Set stores a value with optional expiration

func (*MemoryStorage) SetCleanupConfig added in v1.1.0

func (s *MemoryStorage) SetCleanupConfig(config CleanupConfig)

SetCleanupConfig updates the cleanup configuration

func (*MemoryStorage) SetMemoryLimit

func (s *MemoryStorage) SetMemoryLimit(bytes int64)

SetMemoryLimit sets the memory limit

func (*MemoryStorage) TTL

func (s *MemoryStorage) TTL(key string) time.Duration

TTL returns the time to live for a key

func (*MemoryStorage) Type

func (s *MemoryStorage) Type(key string) ValueType

Type returns the type of a key

type SetValue

type SetValue struct {
	Members map[string]struct{}
}

SetValue represents a set value

type Snapshot

type Snapshot interface {
	// Iterate over all key-value pairs
	ForEach(ctx context.Context, fn func(key string, value *Value) error) error

	// Get snapshot metadata
	Timestamp() time.Time
	KeyCount() int64

	// Cleanup
	Close() error
}

Snapshot represents a point-in-time storage snapshot

type SnapshotStorage

type SnapshotStorage interface {
	Storage

	// Create a point-in-time snapshot
	Snapshot() (Snapshot, error)

	// Restore from snapshot
	Restore(snapshot Snapshot) error
}

SnapshotStorage extends Storage with snapshot capabilities

type Storage

type Storage interface {
	// String operations
	Get(key string) ([]byte, bool)
	Set(key string, value []byte, expiry *time.Time) error
	Del(keys ...string) int64
	Exists(keys ...string) int64

	// Expiration operations
	Expire(key string, expiry time.Time) bool
	TTL(key string) time.Duration
	PTTL(key string) time.Duration

	// Key operations
	Keys(pattern string) []string
	KeyCount() int64
	FlushAll() error

	// Type operations
	Type(key string) ValueType

	// Memory operations
	MemoryUsage() int64

	// Database operations
	SelectDB(db int) error
	CurrentDB() int

	// Iteration
	Scan(cursor int64, match string, count int64) (int64, []string)

	// Info and stats
	Info() map[string]interface{}
	DatabaseInfo() map[int]map[string]interface{}

	// Shutdown
	Close() error
}

Storage defines the interface for data storage operations

type StorageObserver

type StorageObserver interface {
	OnKeySet(key string, value []byte)
	OnKeyDeleted(key string)
	OnKeyExpired(key string)
	OnKeyAccessed(key string)
}

StorageObserver provides hooks for storage events

type StreamEntry

type StreamEntry struct {
	ID     string
	Fields map[string][]byte
}

StreamEntry represents a stream entry

type StreamValue

type StreamValue struct {
	Entries []StreamEntry
	LastID  string
}

StreamValue represents a stream value

type StringValue

type StringValue struct {
	Data []byte
}

StringValue represents a string value

type Transaction

type Transaction interface {
	// Queue commands
	Set(key string, value []byte, expiry *time.Time) Transaction
	Get(key string) Transaction
	Del(keys ...string) Transaction

	// Execute transaction
	Exec() ([]interface{}, error)
	Discard() error
}

Transaction represents a Redis transaction

type TransactionalStorage

type TransactionalStorage interface {
	Storage

	// Transaction operations
	Multi() Transaction
	Watch(keys ...string) error
	Unwatch() error
}

TransactionalStorage extends Storage with transaction support

type Value

type Value struct {
	Type    ValueType
	Data    interface{}
	Expiry  *time.Time
	Version int64 // For optimistic locking
}

Value represents a stored value with metadata

func (*Value) IsExpired

func (v *Value) IsExpired() bool

IsExpired returns true if the value has expired

type ValueType

type ValueType int

ValueType represents the Redis data type

const (
	ValueTypeString ValueType = iota
	ValueTypeList
	ValueTypeSet
	ValueTypeZSet
	ValueTypeHash
	ValueTypeStream
)

func (ValueType) String

func (vt ValueType) String() string

String returns the Redis-compatible type name

type ZSetMember

type ZSetMember struct {
	Member string
	Score  float64
}

ZSetMember represents a sorted set member with score

type ZSetValue

type ZSetValue struct {
	Members map[string]float64
	Scores  []ZSetMember // Sorted by score
}

ZSetValue represents a sorted set value

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL