Skip to content

ducdmd152/dsocial-distributed-system

Repository files navigation

DSocial Documentation

DSocial is a distributed-based social network platform featuring a community feed, comments, user profiles, real-time notifications, and real-time chat. With local development and self-hosted environments, the services are orchestrated with Docker Compose: a NestJS API, a Socket.IO chat server, a notification worker, and a React frontend.

1. System Architecture

1.1 Big Picture

DSocial is split into 4 runtime components and 4 infrastructure services:

  • DSocial Frontend (React + TypeScript + Vite)
  • DSocial API (NestJS)
  • DSocial Chatting Service (Express + Socket.IO)
  • DSocial Notification Worker (NestJS RabbitMQ consumer)
  • MongoDB (primary persistence)
  • Redis (cache + pub/sub + presence)
  • RabbitMQ (notification event bus)
  • MinIO (media object storage)

1.2 Tech Stack

Layer Technology
Frontend React 18, TypeScript, Vite, Chakra UI, Zustand, Socket.IO Client
API Gateway / Backend NestJS 11, Mongoose, Swagger, Passport JWT, RabbitMQ client, ioredis
Chat Service Express 5, Socket.IO 4, JWT, Mongoose, Redis
Notification Worker NestJS, RabbitMQ subscriber, MongoDB, Redis Pub/Sub
Message Broker RabbitMQ topic exchange (dsocial.notifications)
Cache + Realtime Redis (unread counter, online status, pub/sub channel fanout)
Object Storage MinIO (media upload/download for posts & avatars)
Database MongoDB 6

1.3 High-Level Data Path

  1. The frontend calls DSocial API REST endpoints for auth, feed, users, posts, comments, profile, and notifications.
  2. When an interaction event occurs (for example mention/love/comment), the API publishes a message to the RabbitMQ exchange dsocial.notifications.
  3. The Notification Worker consumes from the notification-service queue, stores data in MongoDB, updates the Redis unread counter, then publishes to Redis channel notifications:user:{userId}.
  4. The API exposes SSE endpoint /notifications/sse, subscribes to Redis pattern notifications:user:*, and forwards real-time events to browsers.
  5. Real-time chat runs separately in dsocial-chatting via Socket.IO and REST history endpoints.

2. Component Roles

2.1 DSocial Frontend

  • Manages community, login/register, and messenger UI.
  • Calls APIs with Authorization: Bearer <accessToken>.
  • Opens SSE connection at /notifications/sse for real-time notifications.
  • Connects to the chat service via Socket.IO for real-time messaging.

2.2 DSocial API (NestJS)

  • Provides REST endpoints:
    • auth, users, profile, posts, comments, feed, notifications, minio.
  • Applies a global JWT guard, except endpoints marked with SkipAuth.
  • Uses a transform interceptor to normalize responses (except endpoints marked with SkipTransform).
  • Reads/writes data in MongoDB.
  • Publishes notification events to RabbitMQ.
  • Relays real-time notifications via SSE from Redis pub/sub.

2.3 DSocial Chatting Service

  • Handles real-time chat through Socket.IO with JWT auth middleware.
  • Provides REST endpoints for user list, profile, chat history, room list, and unread count.
  • Stores messages in MongoDB.
  • Uses Redis for online presence (online:{userId}) with TTL refresh.

2.4 DSocial Notification Worker

  • Subscribes to RabbitMQ routing key notification.create.
  • Persists notifications to MongoDB notifications collection.
  • Updates per-user unread counters in Redis (user:{userId}:unread).
  • Publishes messages to Redis channel notifications:user:{userId}.

3. Main Flows

3.1 Notification Flow (RabbitMQ -> Redis -> SSE)

  1. The API emits a domain event (post/comment/love/mention).
  2. The API publishes the event to RabbitMQ topic notification.create.
  3. The worker consumes the event and stores a notification document in MongoDB.
  4. The worker increments unread counters in Redis.
  5. The worker publishes payloads to channel notifications:user:{consumerId}.
  6. The API SSE service receives pub/sub messages and forwards them to clients via /notifications/sse.
  7. The frontend shows toast notifications and increments unread badges in real time.

3.2 Community Feed Flow

  1. The frontend calls /feed to load cursor-based post lists.
  2. The API reads posts, users, and comment summaries from MongoDB.
  3. The frontend calls refresh/refetch endpoints to sync new data.
  4. Post/comment interactions trigger corresponding notification events.

3.3 Chat Realtime Flow

  1. The frontend gets a JWT token at login and connects Socket.IO to the chat service.
  2. Socket middleware verify token, attach socket.user.
  3. Users join pair-specific rooms ({userA}_{userB}).
  4. send-message events are persisted to MongoDB, then broadcast as receive-message / new-message.
  5. The frontend can call REST /api/chat/history/:partnerId to backfill message history.

4. API Surface

4.1 DSocial API (:3001)

Authentication

  • POST /auth/register
  • POST /auth/login

Feed

  • GET /feed?cursor=&limit=
  • GET /feed/refresh?sinceCursor=
  • GET /feed/refetch?sinceCursor=

Users & Profile

  • GET /users?text=&limit=
  • GET /users/:id
  • GET /users/:id/posts?cursor=&limit=
  • GET /users/:id/posts/refresh?sinceCursor=
  • GET /users/:id/posts/refetch?sinceCursor=
  • GET /profile
  • PUT /profile/bio
  • POST /profile/avatar (multipart)

Posts & Comments

  • GET /posts
  • GET /posts/:id
  • POST /posts
  • POST /posts/media (multipart)
  • PATCH /posts/:id
  • DELETE /posts/:id
  • PUT /posts/:id/love
  • PUT /posts/:id/unlove
  • POST /comments
  • PUT /comments/:id
  • DELETE /comments/:id

Notifications

  • GET /notifications
  • GET /notifications/unread-count
  • PATCH /notifications/:id/read
  • PUT /notifications/read-all
  • GET /notifications/sse (Server-Sent Events)

Media

  • GET /minio/files/:key

API Docs

  • GET /api/docs

4.2 Chat Service (:7001)

REST

  • GET /health
  • POST /api/auth/login
  • GET /api/profile
  • GET /api/users
  • GET /api/chat/history/:partnerId?page=&limit=
  • GET /api/chat/rooms
  • POST /api/user/online
  • GET /api/user/:userId/online
  • GET /api/unread-count

Socket Events

Client -> Server:

  • join-chat(partnerId)
  • join-room(roomId)
  • send-message({ message, partnerId, _id? })
  • read_receipt({ messageId, roomId, readAt })

Server -> Client:

  • room-chatted
  • room-joined
  • receive-message
  • new-message
  • sent-message
  • message_read

5. Data Storage & Cache Strategy

5.1 MongoDB Collections (primary)

Collection Purpose
users accounts, profiles, roles, avatars
posts community posts
comments comments and replies on posts
messages chat history between user pairs
notifications user-targeted notifications

5.2 Redis Keys & Channels

Key / Channel Type Meaning
user:{userId}:unread string counter unread notification count
online:{userId} string + TTL chat online presence
notifications:user:{userId} pub/sub channel real-time notification channel

5.3 Consistency Model

  • Notification history is durable in MongoDB.
  • Real-time delivery uses Redis pub/sub (non-durable).
  • Unread count follows a cache-first strategy, with DB recomputation on cache miss.
  • Online presence is TTL-based and may have short-lived drift on abrupt disconnects.

6. Runtime Ports & Services

Service Port Note
dsocial-frontend 8081 frontend container
dsocial-api 3001 REST API + SSE + Swagger
dsocial-chatting 7001 Socket.IO + chat REST
dsocial-noti-worker internal RabbitMQ consumer worker
mongo 27017 MongoDB
redis 6379 cache / pubsub
rabbitmq 5672, 15672 broker + management UI
minio 9007, 9008 object storage + console

7. How To Get Started

7.1 Development Mode

Run the development stack:

docker compose -f docker-compose.dev.yml up --build -d

Stop the development stack:

docker compose -f docker-compose.dev.yml down

7.2 Production-Style Mode

docker compose -f docker-compose.yml up --build -d

7.3 Useful Root Scripts

npm run dev
npm run dev:build
npm run start
npm run stop
npm run logs
npm run clean

8. Environment Variables

Key environment variables in the current compose setup:

  • PORT, PUBLIC_PORT, BASE_URL
  • JWT_SECRET
  • MONGO_URI
  • REDIS_URI
  • RABBITMQ_URI
  • MINIO_ENDPOINT, MINIO_PORT, MINIO_USE_SSL
  • MINIO_ACCESS_KEY, MINIO_SECRET_KEY, MINIO_BUCKET_NAME
  • Frontend build/runtime:
    • VITE_API_URL
    • VITE_CHAT_URL

Notes

  • The API and chat service both use JWT; keep JWT_SECRET consistent.
  • The chat service includes a separate demo login endpoint (/api/auth/login) for messenger flow.
  • The SSE notification endpoint requires a Bearer token and sends periodic heartbeat events.
  • In local environments, stateful data is mounted under the root data/ directory.

License & Copyright

© 2026 Duc Dao ducdmd152 Licensed under the GNU LICENSE.

🤟 Take a star if you find something interesting 🤟

About

DSocial is a distributed-based social network platform featuring a community feed, comments, user profiles, real-time notifications, and real-time chat.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors