Package Structure

BigBrotr is organized into five packages, each with a single clear responsibility. This page details the contents and purpose of every package.

models

Pure domain foundations. Frozen dataclasses representing Nostr concepts with invariants enforced at construction.

Module	Contents
relay.py	Relay — URL validation (RFC 3986), network detection (clearnet/tor/i2p/loki/local), local IP rejection
event.py	Event — Nostr event with signature, EventRelay — junction model
metadata.py	Metadata — content-addressed with SHA-256 hash, MetadataType enum (7 types), RelayMetadata — junction model
constants.py	NetworkType, ServiceName (StrEnum), EventKind (IntEnum), EVENT_KIND_MAX
service_state.py	ServiceState — service checkpoint data, ServiceStateType (StrEnum), ServiceStateDbParams (NamedTuple)

Key patterns:

• All models use @dataclass(frozen=True, slots=True) for immutability and memory efficiency.
• All models cache to_db_params() in __post_init__ via a _db_params field (using object.__setattr__ frozen workaround).
• from_db_params() classmethod reconstructs objects from database tuples with full re-validation.
• stdlib logging only — no bigbrotr imports.

core

Infrastructure and lifecycle. Manages the connection pool, database operations, service lifecycle, logging, and metrics.

Module	Contents
pool.py	Pool — asyncpg connection pool with retry/backoff and health-checked acquisition. PoolConfig Pydantic model.
brotr.py	Brotr — database facade. _call_procedure() for stored functions, generic query methods: fetch(), fetchrow(), fetchval(), execute(), transaction(). BrotrConfig with BatchConfig and TimeoutsConfig.
base_service.py	BaseService[ConfigT] — abstract base class. run() cycle, run_forever() loop, graceful shutdown. from_yaml()/from_dict() factory methods.
logger.py	Logger — structured key=value logging with JSON output mode. format_kv_pairs() utility.
metrics.py	Prometheus /metrics endpoint. Four metric types: SERVICE_INFO, SERVICE_GAUGE, SERVICE_COUNTER, CYCLE_DURATION_SECONDS.
yaml.py	YAML configuration file loading with environment variable interpolation.

Key patterns:

• Brotr._pool is private. Services use Brotr methods, never the pool directly.
• BaseService is generic over ConfigT (a Pydantic BaseModel), enabling type-safe configuration.
• Pool implements async with context management for automatic cleanup.

nips

Protocol-aware I/O. Each NIP implementation is a “sensor” that produces typed Metadata objects. The package contains shared base classes and two sub-packages.

Module	Contents
base.py	Shared base classes for NIP implementations
event_builders.py	Nostr event construction for publishing monitoring data
parsing.py	Declarative field parsing utilities
nip11/	Sub-package: nip11.py (main class), data.py (data models), logs.py (result logging), info.py (field extraction)
nip66/	Sub-package: nip66.py (main class), data.py, logs.py, plus one module per test: rtt.py, ssl.py, dns.py, geo.py, net.py, http.py

NIP-66 health tests:

Test	MetadataType	Measures
RTT	NIP66_RTT	3-phase WebSocket latency: open, read, write (milliseconds)
SSL	NIP66_SSL	Certificate validity, expiration, issuer, SANs, cipher, fingerprint (SHA-256)
DNS	NIP66_DNS	A, AAAA, CNAME, NS, PTR records, TTL
Geo	NIP66_GEO	Country, city, coordinates, timezone, geohash (precision 9) via GeoLite2-City
Net	NIP66_NET	AS number, ISP name, CIDR network ranges (IPv4 + IPv6) via GeoLite2-ASN
HTTP	NIP66_HTTP	Server and X-Powered-By headers from WebSocket upgrade handshake

NIP-11 produces MetadataType.NIP11_INFO.

Key patterns:

• Fetch methods never raise exceptions. Always check logs.success on the result.
• Depends on utils (for transport) and models (for data types). Does not depend on core.

utils

Network and crypto primitives. Stateless helper functions with no business logic.

Module	Contents
protocol.py	create_client(), connect_relay(), broadcast_events(), is_nostr_relay() — WebSocket client creation and relay connectivity testing.
transport.py	DEFAULT_TIMEOUT, InsecureWebSocketTransport — HTTP/WebSocket transport with SSL fallback and timeout configuration.
http.py	Bounded JSON reading and file downloads with size limits.
dns.py	DNS resolution utilities for relay URL validation.
keys.py	load_keys_from_env(), KeysConfig Pydantic model — Nostr key management for event signing and publishing.
parsing.py	safe_parse() — tolerant data parsing that catches and logs failures without crashing.
streaming.py	Data-driven windowing algorithm for streaming Nostr events with completeness guarantees and binary-split fallback.

Key patterns:

• All functions are stateless. No class instances, no side effects beyond I/O.
• SOCKS5 proxy support for Tor/I2P/Lokinet relays.
• Depends only on models.

services

Business logic. Eight independent services, each inheriting BaseService[ConfigT] and implementing async def run().

Service	Mode	Purpose
seeder	One-shot	Load relay URLs from seed files and known relay lists
finder	Continuous	Discover relay URLs from event tags and public APIs
validator	Continuous	Test WebSocket connectivity, promote candidates to relay table
monitor	Continuous	NIP-11 + NIP-66 health checks, publish kind 10166/30166 events
refresher	Scheduled	Orchestrate materialized view refresh cycles
synchronizer	Continuous	Cursor-based event collection from validated relays
api	Continuous	FastAPI read-only HTTP API for relay and event data
dvm	Continuous	NIP-90 Data Vending Machine for Nostr-native data access

Shared infrastructure in services/common/:

Module	Contents
configs.py	Per-network Pydantic models: ClearnetConfig, TorConfig, I2pConfig, LokiConfig with timeouts, proxy URLs, concurrency limits
queries.py	Batch insert utilities (batched_insert, upsert_service_states). Domain queries distributed across service-specific modules.
mixins.py	5 cooperative mixins: ConcurrentStreamMixin (TaskGroup streaming), NetworkSemaphoresMixin (per-network concurrency), GeoReaderMixin (GeoIP lifecycle), ClientsMixin (Nostr client pool), CatalogAccessMixin (schema discovery)
catalog.py	Catalog — runtime schema introspection and safe parameterized query builder, raises CatalogError to prevent leaking database internals
types.py	Checkpoint hierarchy (Api, Monitor, Publish, Candidate) and Cursor hierarchy (Sync, Finder)

Next Steps

• Data Flow — how data moves through the system.
• Services Overview — detailed description of each service.