MidКейс23 min

Система уведомлений

Проектирование масштабируемой системы уведомлений: push, email, SMS, in-app с приоритизацией и retry

Система уведомлений -- один из ключевых компонентов современных приложений. Она должна поддерживать множество каналов доставки, гарантировать доставку и не раздражать пользователей.

Шаг 1: Требования

Функциональные требования

  1. Поддержка каналов: push-уведомления (iOS/Android), email, SMS, in-app
  2. Шаблоны уведомлений с переменными
  3. Приоритизация (critical, high, normal, low)
  4. Настройки пользователя: какие уведомления получать и по каким каналам
  5. Retry при неудачной доставке
  6. Планирование отправки (scheduled notifications)
  7. Batch-отправка (маркетинговые рассылки)

Нефункциональные требования

  1. Гарантия доставки (at-least-once)
  2. Задержка < 5 секунд для критических уведомлений
  3. Масштабирование до 1M уведомлений в минуту
  4. Дедупликация (одно уведомление не приходит дважды)
  5. Rate limiting для каждого пользователя

Шаг 2: Оценка нагрузки

Метрика Значение
DAU 50M
Уведомлений на пользователя в день ~10
Всего уведомлений в день 500M
Peak QPS ~17,000
Размер уведомления ~500 bytes
Хранение (30 дней) ~7.5 TB

Шаг 3: High-Level архитектура

┌──────────────┐     ┌──────────────────┐     ┌──────────────────┐
│  Trigger      │────>│  Notification    │────>│  Priority Queue  │
│  (API/Event)  │     │  Service         │     │  (RabbitMQ)      │
└──────────────┘     └──────────────────┘     └────────┬─────────┘
                                                       │
                            ┌──────────────────────────┼────────────────────┐
                            │                          │                    │
                     ┌──────▼──────┐     ┌─────────────▼───┐    ┌──────────▼────┐
                     │  Push       │     │  Email          │    │  SMS          │
                     │  Worker     │     │  Worker         │    │  Worker       │
                     └──────┬──────┘     └────────┬────────┘    └──────┬────────┘
                            │                     │                    │
                     ┌──────▼──────┐     ┌────────▼────────┐   ┌──────▼────────┐
                     │  APNs/FCM   │     │  SES/SendGrid   │   │  Twilio/      │
                     │             │     │                  │   │  Vonage       │
                     └─────────────┘     └─────────────────┘   └───────────────┘
                            │                     │                    │
                            └─────────────────────┼────────────────────┘
                                                  │
                                        ┌─────────▼─────────┐
                                        │  Delivery Status  │
                                        │  Tracker          │
                                        └─────────┬─────────┘
                                                  │
                                        ┌─────────▼─────────┐
                                        │  PostgreSQL +     │
                                        │  Redis            │
                                        └───────────────────┘

Шаг 4: Схема данных

-- Notification templates
CREATE TABLE notification_templates (
    id          UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    code        VARCHAR(100) NOT NULL UNIQUE,
    channel     VARCHAR(20)  NOT NULL, -- push, email, sms, in_app
    subject     TEXT,
    body        TEXT         NOT NULL,
    variables   JSONB        NOT NULL DEFAULT '[]',
    created_at  TIMESTAMPTZ  NOT NULL DEFAULT now()
);

-- Notification log
CREATE TABLE notifications (
    id              UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    user_id         UUID         NOT NULL,
    template_id     UUID         NOT NULL REFERENCES notification_templates(id),
    channel         VARCHAR(20)  NOT NULL,
    priority        SMALLINT     NOT NULL DEFAULT 2, -- 0=critical, 1=high, 2=normal, 3=low
    status          VARCHAR(20)  NOT NULL DEFAULT 'pending',
    payload         JSONB        NOT NULL DEFAULT '{}',
    scheduled_at    TIMESTAMPTZ,
    sent_at         TIMESTAMPTZ,
    delivered_at    TIMESTAMPTZ,
    failed_at       TIMESTAMPTZ,
    retry_count     SMALLINT     NOT NULL DEFAULT 0,
    error_message   TEXT,
    idempotency_key VARCHAR(64)  UNIQUE,
    created_at      TIMESTAMPTZ  NOT NULL DEFAULT now()
);

CREATE INDEX idx_notifications_user_status ON notifications (user_id, status);
CREATE INDEX idx_notifications_scheduled ON notifications (scheduled_at)
    WHERE status = 'scheduled' AND scheduled_at IS NOT NULL;
CREATE INDEX idx_notifications_retry ON notifications (status, retry_count)
    WHERE status = 'failed' AND retry_count < 3;

-- User notification preferences
CREATE TABLE notification_preferences (
    user_id     UUID        NOT NULL,
    category    VARCHAR(50) NOT NULL, -- marketing, transactional, social
    channel     VARCHAR(20) NOT NULL,
    enabled     BOOLEAN     NOT NULL DEFAULT true,
    quiet_start TIME,                 -- Do not disturb start
    quiet_end   TIME,                 -- Do not disturb end
    PRIMARY KEY (user_id, category, channel)
);

-- Device tokens for push notifications
CREATE TABLE device_tokens (
    id          UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    user_id     UUID        NOT NULL,
    platform    VARCHAR(10) NOT NULL, -- ios, android, web
    token       TEXT        NOT NULL,
    is_active   BOOLEAN     NOT NULL DEFAULT true,
    created_at  TIMESTAMPTZ NOT NULL DEFAULT now(),
    updated_at  TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE INDEX idx_device_tokens_user ON device_tokens (user_id) WHERE is_active = true;

Шаг 5: Детальный дизайн

5.1 Notification Service (диспатчер)

<?php

declare(strict_types=1);

final class NotificationService
{
    public function __construct(
        private readonly NotificationRepository $repository,
        private readonly TemplateRenderer $templateRenderer,
        private readonly PreferenceChecker $preferenceChecker,
        private readonly QueuePublisher $queue,
        private readonly DeduplicationService $dedup,
        private readonly RateLimiter $rateLimiter,
    ) {}

    public function send(NotificationRequest $request): void
    {
        // 1. Deduplicate
        if (!$this->dedup->isUnique($request->idempotencyKey)) {
            return; // Already sent
        }

        // 2. Check user preferences
        if (!$this->preferenceChecker->canSend(
            $request->userId,
            $request->category,
            $request->channel,
        )) {
            return; // User opted out
        }

        // 3. Check quiet hours
        if ($this->preferenceChecker->isQuietHour($request->userId, $request->channel)) {
            // Schedule for after quiet hours
            $this->scheduleAfterQuietHours($request);
            return;
        }

        // 4. Rate limit per user
        $rateLimitResult = $this->rateLimiter->allow(
            key: "notif:{$request->userId}:{$request->channel}",
            capacity: $this->getChannelLimit($request->channel),
            rate: 1.0,
        );

        if (!$rateLimitResult['allowed']) {
            // Queue for later
            $this->scheduleWithDelay($request, 60);
            return;
        }

        // 5. Render template
        $rendered = $this->templateRenderer->render(
            $request->templateCode,
            $request->variables,
        );

        // 6. Save to database
        $notification = $this->repository->create(
            userId: $request->userId,
            templateId: $rendered->templateId,
            channel: $request->channel,
            priority: $request->priority,
            payload: $rendered->toArray(),
            idempotencyKey: $request->idempotencyKey,
        );

        // 7. Publish to priority queue
        $this->queue->publish(
            queue: $this->getQueueName($request->channel, $request->priority),
            message: new NotificationMessage(
                notificationId: $notification->id,
                channel: $request->channel,
                payload: $rendered->toArray(),
            ),
        );
    }

    public function sendBatch(BatchNotificationRequest $request): string
    {
        $batchId = bin2hex(random_bytes(16));

        // Publish batch job to dedicated queue
        $this->queue->publish('notification.batch', [
            'batch_id' => $batchId,
            'template_code' => $request->templateCode,
            'user_segment' => $request->userSegment,
            'variables' => $request->variables,
            'channel' => $request->channel,
            'scheduled_at' => $request->scheduledAt?->format('c'),
        ]);

        return $batchId;
    }

    private function getQueueName(string $channel, int $priority): string
    {
        $priorityName = match ($priority) {
            0 => 'critical',
            1 => 'high',
            2 => 'normal',
            3 => 'low',
            default => 'normal',
        };

        return "notification.{$channel}.{$priorityName}";
    }

    private function getChannelLimit(string $channel): int
    {
        return match ($channel) {
            'push' => 30,   // 30 per hour
            'email' => 10,  // 10 per hour
            'sms' => 5,     // 5 per hour
            'in_app' => 100,
            default => 20,
        };
    }
}
### 5.2 Template Renderer
<?php

declare(strict_types=1);

final class TemplateRenderer
{
    public function __construct(
        private readonly TemplateRepository $templates,
    ) {}

    public function render(string $templateCode, array $variables): RenderedNotification
    {
        $template = $this->templates->findByCode($templateCode);

        if ($template === null) {
            throw new TemplateNotFoundException("Template not found: {$templateCode}");
        }

        // Validate required variables
        $requiredVars = json_decode($template->variables, true);
        foreach ($requiredVars as $var) {
            if (!isset($variables[$var])) {
                throw new MissingVariableException("Missing variable: {$var}");
            }
        }

        // Render subject and body
        $subject = $this->interpolate($template->subject ?? '', $variables);
        $body = $this->interpolate($template->body, $variables);

        return new RenderedNotification(
            templateId: $template->id,
            subject: $subject,
            body: $body,
            channel: $template->channel,
        );
    }

    private function interpolate(string $template, array $variables): string
    {
        return preg_replace_callback(
            '/\{\{(\w+)\}\}/',
            fn (array $matches) => $variables[$matches[1]] ?? $matches[0],
            $template,
        );
    }
}

final readonly class RenderedNotification
{
    public function __construct(
        public string $templateId,
        public string $subject,
        public string $body,
        public string $channel,
    ) {}

    public function toArray(): array
    {
        return [
            'template_id' => $this->templateId,
            'subject' => $this->subject,
            'body' => $this->body,
            'channel' => $this->channel,
        ];
    }
}
### 5.3 Channel Workers
<?php

declare(strict_types=1);

interface NotificationSender
{
    public function send(string $notificationId, array $payload): DeliveryResult;
}

final class PushNotificationSender implements NotificationSender
{
    public function __construct(
        private readonly DeviceTokenRepository $deviceTokens,
        private readonly ApnsClient $apns,
        private readonly FcmClient $fcm,
    ) {}

    public function send(string $notificationId, array $payload): DeliveryResult
    {
        $userId = $payload['user_id'];
        $tokens = $this->deviceTokens->getActiveTokens($userId);

        $results = [];
        foreach ($tokens as $token) {
            try {
                $result = match ($token->platform) {
                    'ios' => $this->apns->send($token->token, [
                        'title' => $payload['subject'],
                        'body' => $payload['body'],
                        'data' => $payload['data'] ?? [],
                    ]),
                    'android' => $this->fcm->send($token->token, [
                        'title' => $payload['subject'],
                        'body' => $payload['body'],
                        'data' => $payload['data'] ?? [],
                    ]),
                    default => throw new \InvalidArgumentException("Unknown platform: {$token->platform}"),
                };

                $results[] = $result;
            } catch (InvalidTokenException $e) {
                // Deactivate invalid token
                $this->deviceTokens->deactivate($token->id);
            }
        }

        $success = count(array_filter($results, fn ($r) => $r->isSuccess()));

        return new DeliveryResult(
            success: $success > 0,
            deliveredCount: $success,
            totalCount: count($tokens),
        );
    }
}

final class EmailNotificationSender implements NotificationSender
{
    public function __construct(
        private readonly EmailClient $emailClient,
        private readonly UserRepository $users,
    ) {}

    public function send(string $notificationId, array $payload): DeliveryResult
    {
        $user = $this->users->find($payload['user_id']);

        $result = $this->emailClient->send(
            to: $user->email,
            subject: $payload['subject'],
            htmlBody: $payload['body'],
            headers: [
                'X-Notification-Id' => $notificationId,
            ],
        );

        return new DeliveryResult(
            success: $result->isSuccess(),
            deliveredCount: $result->isSuccess() ? 1 : 0,
            totalCount: 1,
        );
    }
}
### 5.4 Retry с Exponential Backoff
<?php

declare(strict_types=1);

final class NotificationWorker
{
    private const MAX_RETRIES = 3;
    private const BASE_DELAY_SECONDS = 60;

    public function __construct(
        private readonly NotificationSender $sender,
        private readonly NotificationRepository $repository,
        private readonly QueuePublisher $queue,
    ) {}

    public function process(NotificationMessage $message): void
    {
        try {
            $result = $this->sender->send(
                $message->notificationId,
                $message->payload,
            );

            if ($result->success) {
                $this->repository->markDelivered($message->notificationId);
            } else {
                $this->handleFailure($message, 'Delivery failed');
            }
        } catch (\Throwable $e) {
            $this->handleFailure($message, $e->getMessage());
        }
    }

    private function handleFailure(NotificationMessage $message, string $error): void
    {
        $notification = $this->repository->find($message->notificationId);

        if ($notification->retryCount >= self::MAX_RETRIES) {
            $this->repository->markFailed(
                $message->notificationId,
                $error,
            );
            return;
        }

        // Exponential backoff: 60s, 120s, 240s
        $delay = self::BASE_DELAY_SECONDS * (2 ** $notification->retryCount);

        $this->repository->incrementRetry($message->notificationId, $error);

        // Re-queue with delay
        $this->queue->publishWithDelay(
            queue: "notification.{$message->channel}.retry",
            message: $message,
            delaySeconds: $delay,
        );
    }
}
### 5.5 Дедупликация
<?php

declare(strict_types=1);

final class DeduplicationService
{
    private const TTL = 86400; // 24 hours

    public function __construct(
        private readonly \Redis $redis,
    ) {}

    public function isUnique(string $idempotencyKey): bool
    {
        $key = "dedup:{$idempotencyKey}";

        // SET NX returns true only if key didn't exist
        $result = $this->redis->set($key, '1', ['NX', 'EX' => self::TTL]);

        return $result !== false;
    }
}
## Шаг 6: Масштабирование

Приоритетные очереди

Queue: notification.push.critical   -- обрабатывается ПЕРВЫМ (8 workers)
Queue: notification.push.high       -- 4 workers
Queue: notification.push.normal     -- 2 workers
Queue: notification.push.low        -- 1 worker

Queue: notification.email.critical
Queue: notification.email.high
... и так далее для каждого канала

Аналитика уведомлений

Метрика Как считать
Delivery Rate delivered / total * 100%
Open Rate (push) opened / delivered * 100%
Click Rate (email) clicked / delivered * 100%
Unsubscribe Rate unsubscribed / delivered * 100%

Возможные вопросы интервьюера

  1. Как гарантировать exactly-once доставку?

    • Строго: невозможно в распределённой системе
    • Практически: at-least-once + дедупликация на стороне клиента
  2. Как обрабатывать миллионы уведомлений для маркетинговых рассылок?

    • Batch processing с throttling
    • Разбивка на сегменты, каждый -- отдельная задача
    • Постепенная раскатка (1%, 10%, 100%)
  3. Как предотвратить notification fatigue?

    • Rate limiting per user per channel
    • Aggregation (10 лайков -> 1 уведомление)
    • Priority-based filtering
  4. Как обеспечить доставку критических уведомлений?

    • Отдельная очередь с высшим приоритетом
    • Fallback: push -> SMS -> email
    • Мониторинг и алерты на задержки
  5. Как масштабировать систему уведомлений?

    • Горизонтальное масштабирование workers
    • Шардирование очередей по каналам и приоритетам
    • Кэширование шаблонов и пользовательских настроек