No description
Find a file
Natalie b252753476 feat(analytics): canonical store on black, vps-0 edge with redis outage-spool
Relocate the canonical analytics store off the public VPS. The collector
becomes a DB-free edge that captures + durably enqueues events to black's
redis; a black-side ingest-writer enriches and writes raw_events. When black
is unreachable the collector spools to a local appendonly redis that only
grows during the outage and drains on recovery.

- shared: RawEventEnvelope/NormalizedEvent ingest contract (edge -> writer)
- collector: capture-and-enqueue + dual-redis RedisRouter (primary=black,
  spool=local) + paused-until-healthy drain worker; drop TypeORM/enrichment
- processor: IngestService canonical writer (edge receivedAt, ON CONFLICT
  DO NOTHING), single worker branches ingest-event -> process-event
- relocate device/identity/domain/attribution enrichment + entities to writer

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 07:48:02 -05:00
.forgejo/workflows ci: update workflow to publish all sub-packages via Forgejo registry 2026-01-30 17:36:52 -08:00
docs chore(docs): 📝 Update documentation files in /docs directory (README, guides, or API references) 2026-01-29 08:20:58 -08:00
examples docs(examples): 📝 Refactor example configurations to align with latest best practices, adding clearer workflows and feature demonstrations 2026-01-29 08:20:58 -08:00
infrastructure chore(infra): 🔧 Update production Docker Compose for optimized service scaling and resource limits 2026-06-10 06:01:33 -07:00
packages feat(analytics): canonical store on black, vps-0 edge with redis outage-spool 2026-06-21 07:48:02 -05:00
scripts docs(deploy): 📝 Add deployment strategy documentation with detailed instructions and rationale 2026-05-16 16:26:45 -07:00
services feat(analytics): canonical store on black, vps-0 edge with redis outage-spool 2026-06-21 07:48:02 -05:00
.gitignore chore(infra): 🔧 Optimize production Docker deployment configs with updated Dockerfiles and docker-compose.prod.yaml 2026-04-07 18:01:58 -07:00
.npmrc chore(config): 🔧 Update configuration files across 6 directories 2026-01-25 15:33:59 -08:00
app.manifest.yaml chore(processor): 🔧 update processor.domain in app.manifest.yaml to reflect the new production domain 2026-06-10 03:54:59 -07:00
bun.lock deps-upgrade(dependencies-specific): ⬆️ Update lockfile and config to reflect latest dependency versions 2026-06-10 03:54:59 -07:00
bunfig.toml deps-upgrade(dependencies-specific): ⬆️ Update lockfile and config to reflect latest dependency versions 2026-06-10 03:54:59 -07:00
CLAUDE.md chore(config): 🔧 Update configuration files across 6 directories 2026-01-25 15:33:59 -08:00
package.json deps-upgrade(analytics): ⬆️ Update @analytics/core to version 1.2.0 across root and analytics packages 2026-06-10 03:54:59 -07:00
README.md chore(core): 🔧 Update TypeScript files in core module 2026-01-29 08:20:57 -08:00
run release(app-root): 🔖 publish version 1.0.0 with security and performance improvements 2026-04-04 15:14:01 -07:00
tsconfig.base.json chore(config): 🔧 Update TypeScript config files (4 files) 2026-01-25 18:13:17 -08:00
turbo.json chore(core): 🔧 Update TypeScript files in core module 2026-01-29 08:20:57 -08:00

Analytics Platform

A self-hosted, privacy-first, consent-free analytics platform for web applications.

License: MIT

Why This Exists

Modern analytics platforms have problems:

  • Third-party tracking sends your data to external services
  • Cookie consent banners destroy user experience
  • Vendor lock-in makes migration painful
  • Privacy concerns create GDPR compliance headaches

This platform solves all of these:

  • Self-hosted - Your data stays on your servers
  • Consent-free - No cookies, localStorage, or sessionStorage (see Legal Basis)
  • Privacy-first - IPs hashed, sessions ephemeral, DoNotTrack respected
  • Open source - No lock-in, full control

Features

Core Analytics

  • Page views with automatic device/browser detection
  • Click and scroll tracking
  • Session-based metrics (SPA lifecycle)
  • UTM attribution (first-touch)
  • Custom event tracking

Advanced Features

  • Funnel analysis
  • Cohort analysis
  • Real-time dashboard (WebSocket)
  • A/B test integration
  • GDPR compliance (export, deletion)

Developer Experience

  • TypeScript-first
  • React hooks and context
  • NestJS decorators and interceptors
  • Server-side tracking for SSR/Node.js

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                         Client Applications                          │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐                 │
│  │   React     │  │   Next.js   │  │   Node.js   │                 │
│  │   SPA       │  │   SSR/SSG   │  │   Backend   │                 │
│  └──────┬──────┘  └──────┬──────┘  └──────┬──────┘                 │
│         │                │                │                          │
│         └────────────────┴────────────────┘                          │
│                          │                                           │
│              @analytics/client SDK                                   │
└──────────────────────────┼───────────────────────────────────────────┘
                           │ HTTP POST
                           ▼
┌──────────────────────────────────────────────────────────────────────┐
│                      Analytics Backend                                │
│                                                                       │
│  ┌─────────────┐    ┌─────────────┐    ┌─────────────┐              │
│  │  Collector  │───▶│  Processor  │───▶│    API      │              │
│  │   :4001     │    │   :4002     │    │   :4003     │              │
│  │             │    │  (BullMQ)   │    │             │              │
│  └─────────────┘    └─────────────┘    └──────┬──────┘              │
│         │                │                    │                      │
│         │                │                    │                      │
│         ▼                ▼                    ▼                      │
│  ┌────────────────────────────────────────────────┐                 │
│  │              TimescaleDB + Redis               │                 │
│  └────────────────────────────────────────────────┘                 │
│                                                                       │
│  ┌─────────────┐                                                     │
│  │  Realtime   │ WebSocket live metrics                              │
│  │   :4004     │                                                     │
│  └─────────────┘                                                     │
└──────────────────────────────────────────────────────────────────────┘

Quick Start

1. Start the Backend (Docker)

# Clone the repository
git clone https://github.com/your-org/analytics.git
cd analytics

# Start all services
docker compose up -d

# Services available at:
# - Collector: http://localhost:4001
# - API: http://localhost:4003
# - Realtime: ws://localhost:4004

2. Install the Client SDK

npm install @analytics/client
# or
pnpm add @analytics/client
# or
bun add @analytics/client

3. Basic Usage (Browser)

import { AnalyticsClient } from '@analytics/client';

const analytics = new AnalyticsClient({
  apiBaseUrl: 'https://analytics.yoursite.com',
  appName: 'my-app',
});

// Track page view
analytics.trackView({
  pageUrl: window.location.href,
  pageTitle: document.title,
});

// Track custom event
analytics.trackInteraction({
  type: 'click',
  data: {
    elementId: 'signup-button',
    pageUrl: window.location.href,
  },
});

4. React Integration

import { AnalyticsProvider, useAnalytics } from '@analytics/client/react';

// Wrap your app
function App() {
  return (
    <AnalyticsProvider
      config={{
        apiBaseUrl: 'https://analytics.yoursite.com',
        appName: 'my-app',
        scrollTracking: { enabled: true },
      }}
    >
      <YourApp />
    </AnalyticsProvider>
  );
}

// Use in components
function SignupButton() {
  const { trackInteraction } = useAnalytics();

  return (
    <button
      onClick={() => {
        trackInteraction({
          type: 'click',
          data: { elementId: 'signup-button', pageUrl: location.href },
        });
      }}
    >
      Sign Up
    </button>
  );
}

This platform uses in-memory session tracking only. Under ePrivacy Directive Article 5(3):

"storing or gaining access to information stored in the terminal equipment"

JavaScript variables held in browser memory are NOT stored on terminal equipment—they exist only in the browser process and are destroyed on tab close.

What this means:

  • Session IDs generated fresh on each page load (in memory)
  • Attribution data held in module-level variables
  • No localStorage, sessionStorage, or cookies
  • No fingerprinting or persistent identifiers
  • No cross-session tracking (by design)
  • No "returning visitor" metrics

Metrics Available

Metric Status Notes
Page views Full Per-visit, with device/browser
Traffic sources Full UTM params, referrer
Device/browser/OS Full User agent parsing
Geography Full IP geolocation (hashed storage)
Scroll depth Full Percentage thresholds
Click tracking Full Element-level
Session duration ⚠️ SPA-only Resets on hard navigation
Funnel conversion ⚠️ SPA-only Single-session funnels
Bounce rate ⚠️ SPA-only Approximated
New vs returning None No persistent identifiers
Cross-visit attribution None Privacy by design

GDPR Compliance

The platform includes built-in GDPR features:

  • Data export (Article 15) - Export all data for a user ID
  • Data deletion (Article 17) - Purge all user data
  • Retention policies - Automatic data expiration
  • Audit logging - Track all data access

Documentation

Examples

See the examples/ directory for complete implementations:

Development

Prerequisites

  • Node.js 20+
  • Docker & Docker Compose
  • pnpm or bun

Setup

# Install dependencies
pnpm install

# Start infrastructure (TimescaleDB, Redis)
docker compose up -d postgres redis

# Run services in development
pnpm dev:collector
pnpm dev:processor
pnpm dev:api
pnpm dev:realtime

# Build all packages
pnpm build

Project Structure

@analytics/
├── packages/              # Shared libraries
│   └── analytics/         # Core types and utilities
│
├── services/              # Backend microservices
│   ├── collector/         # Event ingestion (POST /track/*)
│   ├── processor/         # Aggregation workers (BullMQ)
│   ├── api/               # Query API (trends, funnels, cohorts)
│   └── realtime/          # WebSocket gateway
│
├── examples/              # Integration examples
│
├── docs/                  # Documentation
│
└── infrastructure/        # Docker configs

Contributing

Contributions are welcome! Please read CONTRIBUTING.md first.

License

MIT License - see LICENSE for details.


Note: The client SDK (@analytics/client) is published separately. See analytics-client for the browser/Node.js SDK source.