Skip to content

jrmd/pg-keystore

Repository files navigation

@jrmd/pg-keystore

A small Redis-like JSON key-value store for applications that already use PostgreSQL and Drizzle ORM. Values are stored in jsonb; reads, writes, hash updates, expiration, and transactional batches use the application's existing database connection.

Important

This is not as fast as Redis. Every operation still pays for PostgreSQL's SQL execution, network protocol, and MVCC storage work. pg-keystore is an interim solution for applications that already have PostgreSQL and want simple, centralized key-value semantics without adding infrastructure. When a project truly scales to needing a dedicated high-throughput cache, queue, or data store, use Redis (or the service designed for that workload) instead.

Install

bun add @jrmd/pg-keystore drizzle-orm

Apply drizzle/0000_create_kv_store.sql, or import the schema from @jrmd/pg-keystore/schema and let Drizzle Kit generate the migration with the rest of your application schema.

Use

import { drizzle } from "drizzle-orm/node-postgres";
import { makeKvService } from "@jrmd/pg-keystore";
import { kvStore } from "@jrmd/pg-keystore/schema";

const db = drizzle(process.env.DATABASE_URL!, {
  schema: { kvStore },
});
const cache = makeKvService(db, { preparedStatements: false });

await cache.set("user:42", { name: "Ada" }, { ttlMs: 60_000 });
const user = await cache.get<{ name: string }>("user:42");

await cache.hset("flags", "new-navigation", true);
const enabled = await cache.hget<boolean>("flags", "new-navigation");

await cache.batch(async (batch) => {
  await batch.set("one", 1);
  await batch.set("two", 2);
});

await cache.setMany([
  { key: "user:42", value: { name: "Ada" } },
  { key: "session:42", value: { userId: "42" }, options: { ttlMs: 60_000 } },
]);
const values = await cache.getMany(["user:42", "session:42"]);
await cache.delMany(["session:42"]);

Missing and expired get/hget calls return null. getMany omits missing keys; many remains a backwards-compatible alias. hgetall returns an empty object when the key is absent or is not a JSON object. Calling set without expiration clears an existing TTL; hash operations preserve an active TTL, matching Redis behavior.

set accepts either { ttlMs } or { expiresAt }. Relative expiration is calculated by PostgreSQL, avoiding drift between the application and database clocks. Stored values must be accepted by JSON.stringify.

Expired-row maintenance

Expired entries are hidden immediately but remain on disk until overwritten or deleted. Run a small maintenance job periodically until it returns 0:

let deleted: number;
do {
  deleted = await cache.purgeExpired(1_000);
} while (deleted > 0);

The purge is bounded to 10,000 rows per call and rechecks expiration in the DELETE, so a concurrent refresh is not removed.

Batch operations

setMany, getMany, and delMany accept at most 10,000 entries. Empty batches do nothing. setMany is one atomic PostgreSQL statement; it validates and serializes every value before writing, and duplicate keys use the final entry. Relative TTLs are calculated from PostgreSQL's clock for the entire batch.

Operational notes

  • PostgreSQL 16 or newer is required. The provided table is logged by default, so it survives an unclean PostgreSQL restart. If every value is disposable and write throughput matters more than crash persistence, you can explicitly use ALTER TABLE kv_store SET UNLOGGED.
  • New installations get a partial BRIN expiry index plus a lower fillfactor and tighter autovacuum thresholds. For existing tables, run 0001_kv_store_hot_updates.sql outside a transaction, executing its statements separately because concurrent index commands cannot run in a transaction. Do not run VACUUM FULL as part of this upgrade.
  • Monitor churn with pg_stat_user_tables (especially HOT updates and dead tuples) and query cost with pg_stat_statements. BRIN and unlogged tables still need regular vacuuming.
  • Pool size belongs to the application deployment: budget connections across every application instance and PostgreSQL itself; this library never changes it. PostgreSQL retains MVCC and SQL/protocol overhead, so isolated single-key requests will generally remain slower than Redis.
  • Keep preparedStatements disabled for transaction-mode proxies unless the proxy explicitly supports named prepared statements. The default portable Drizzle query path is appropriate for those deployments.
  • Keys are limited to 255 Unicode characters and cannot contain a null byte. Hash fields cannot contain a null byte.
  • Generic return types are assertions made by the caller; this package does not perform runtime schema validation. Validate untrusted cached data in your application.
  • batch uses Drizzle's transaction API and therefore depends on transaction support in the selected PostgreSQL driver.

API reference

All methods return promises. Values must be accepted by JSON.stringify; generic result types are caller assertions, not runtime validation.

makeKvService(db, options?) / new KVService(db, options?)

Creates a service from a Drizzle PostgreSQL database instance.

const cache = makeKvService(db, { preparedStatements: false });

options.preparedStatements defaults to false. Set it to true only when the driver and connection/proxy support named prepared statements. It enables versioned prepared queries for get, getMany, and delMany; the portable dynamic-query path remains the default.

get<T>(key)

Returns the active value for key, or null when it is missing or expired.

const user = await cache.get<{ name: string }>("user:42");

set<T>(key, value, options?)

Stores a JSON value. A call without options clears any previous expiry. Repeating a write with the same value and unchanged expiry avoids creating a new PostgreSQL row version.

await cache.set("user:42", { name: "Ada" });
await cache.set("session:42", { userId: "42" }, { ttlMs: 60_000 });
await cache.set("campaign", true, { expiresAt: new Date("2027-01-01T00:00:00Z") });

SetOptions accepts exactly one of:

  • ttlMs: number — a positive, finite millisecond duration, evaluated using the PostgreSQL clock.
  • expiresAt: Date — a valid absolute expiry timestamp.

del(key)

Deletes a key. Deleting a missing key succeeds without error.

await cache.del("session:42");

getMany<T>(keys) / many<T>(keys)

Returns active values keyed by name, omitting missing and expired keys. Duplicate input keys are read once. many is a backwards-compatible alias for getMany.

const values = await cache.getMany<number>(["counter:a", "counter:b"]);
// Partial<Record<string, number>>

setMany<T>(entries)

Writes up to 10,000 entries atomically in a single statement. Every entry is validated and serialized before SQL runs, so invalid input cannot partially write. Duplicate keys use the final entry.

await cache.setMany([
  { key: "feature:dark-mode", value: true },
  { key: "session:42", value: { userId: "42" }, options: { ttlMs: 60_000 } },
]);

Each SetManyEntry<T> is { key: string; value: T; options?: SetOptions }. Relative TTLs use the database clock; entries may mix no expiration, relative TTLs, and absolute expirations. An empty array is a no-op.

delMany(keys)

Deletes up to 10,000 keys in one operation. Duplicate and missing keys are harmless; an empty array is a no-op.

await cache.delMany(["session:42", "session:43"]);

hget<T>(key, field)

Returns one field from an active JSON object, or null if the key/field is absent, expired, or its value is not an object. Arrays are not treated as hashes.

const enabled = await cache.hget<boolean>("flags", "new-navigation");

hset<T>(key, field, value)

Sets a field on a JSON object. For an absent, expired, non-object, or array value, it creates a new object containing the field. It preserves an active key TTL and skips a PostgreSQL update when the field already has the same JSON value.

await cache.hset("flags", "new-navigation", true);

hdel(key, field)

Deletes a field from an active JSON object. It is a no-op if the key is absent, expired, not an object, or does not have that field. Active TTLs are preserved.

await cache.hdel("flags", "new-navigation");

hgetall<T>(key)

Returns an active JSON object, or {} if the key is absent, expired, or is not an object. Arrays are not hashes.

const flags = await cache.hgetall<boolean>("flags");

batch(callback)

Runs the callback in a Drizzle transaction. The callback receives another KVService using the transaction connection and the same service options. If it throws, all of its writes are rolled back.

await cache.batch(async (tx) => {
  await tx.set("one", 1);
  await tx.set("two", 2);
});

The selected driver must support Drizzle transactions.

purgeExpired(limit?)

Physically removes expired rows and returns the number deleted. Expired values are already invisible to reads; this method reclaims storage. limit defaults to 1,000 and must be an integer from 1 through 10,000. The delete rechecks expiration, so a concurrently refreshed key is not removed.

while (await cache.purgeExpired(1_000)) {
  // Continue until the current expired backlog is gone.
}

Input limits and errors

  • Keys must be strings of at most 255 Unicode characters and cannot contain a null byte.
  • Hash fields must be strings without null bytes.
  • Bulk methods accept at most 10,000 input entries/keys.
  • Invalid JSON values, invalid expiration options, and invalid keys/fields reject before a query is issued.

The package is ESM-only and supports Node.js 20 or newer.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages