From 1d6e2a00f1a754683037a0f175af127708dd67c9 Mon Sep 17 00:00:00 2001 From: Thanmayee Reddy Kotha <190446018+thanmayeereddykotha@users.noreply.github.com> Date: Sun, 16 Aug 2026 14:50:14 +0530 Subject: [PATCH 1/5] =?UTF-8?q?docs:=20add=2011-presence-system=20+=2012-h?= =?UTF-8?q?aveibeenpwned-integration=20=E2=80=94=20Socket.IO=20presence,?= =?UTF-8?q?=20HIBP=20k-anonymity=20breach=20check?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/features/11-presence-system.md | 175 ++++++++++++++ .../features/12-haveibeenpwned-integration.md | 223 ++++++++++++++++++ 2 files changed, 398 insertions(+) create mode 100644 docs/features/11-presence-system.md create mode 100644 docs/features/12-haveibeenpwned-integration.md diff --git a/docs/features/11-presence-system.md b/docs/features/11-presence-system.md new file mode 100644 index 00000000..a5ae6442 --- /dev/null +++ b/docs/features/11-presence-system.md @@ -0,0 +1,175 @@ +# 11 — Presence System + +**NEW document** — Online/offline/away states, Socket.IO /presence namespace, lastSeen tracking, 30s grace period + +--- + +## Feature Summary + +The presence system tracks which users are online, offline, or away in real-time. It uses a Socket.IO `/presence` namespace with an in-memory `Map` of online users. On connect, the user is added to the map and their status is broadcast to all other clients. On disconnect, a 30-second grace period prevents churn from brief network blips. + +--- + +## Architecture Diagram + +``` +┌─────────────────── CLIENT ───────────────────────────┐ +│ │ +│ src/hooks/usePresence.ts │ +│ ├─ Connects to /presence namespace │ +│ ├─ Passes userId in handshake query │ +│ ├─ Listens: 'initial-status' → populate online list │ +│ ├─ Listens: 'user-status-changed' → update UI │ +│ └─ Emits: 'update-status' → set away/dnd/online │ +│ │ +│ src/hooks/useMe.ts │ +│ └─ Reads user.status + user.lastSeen from /api/users │ +│ │ +│ UI Indicators: │ +│ ├─ Green dot = online │ +│ ├─ Yellow dot = away │ +│ ├─ Grey dot = offline + "last seen X min ago" │ +│ └─ Shown in: PeopleView, ChatView, TeamMembers │ +└──────────────────────┬────────────────────────────────┘ + │ Socket.IO /presence + ▼ +┌─────────────────── BACKEND ──────────────────────────┐ +│ │ +│ backend/sockets/presenceSocketHandler.js (143 lines) │ +│ │ +│ In-memory state: onlineUsers = Map │ +│ │ +│ Events: │ +│ ├─ On connect: │ +│ │ ├─ Add to onlineUsers Map │ +│ │ ├─ Emit 'initial-status' to connector │ +│ │ └─ Broadcast 'user-status-changed' to all others │ +│ ├─ On disconnect: │ +│ │ ├─ Set status to 'offline' in Map │ +│ │ ├─ Broadcast 'user-status-changed' │ +│ │ └─ After 30s: delete from Map if still offline │ +│ └─ On 'update-status': │ +│ ├─ Update Map with new status │ +│ └─ Broadcast 'user-status-changed' │ +│ │ +│ Also: MongoDB User.status + User.lastSeen │ +│ └─ Updated on /api/users/sync (login) │ +│ └─ Updated on disconnect (via API call) │ +└───────────────────────────────────────────────────────┘ +``` + +--- + +## Backend Trace + +### File: `backend/sockets/presenceSocketHandler.js` (143 lines) + +### In-Memory State (line 76) +```js +const onlineUsers = new Map(); +``` +- Key: `userId` (Firebase UID) +- Value: `{ status: 'online' | 'offline' | 'away', lastSeen: Date }` +- **Not persisted** — lost on server restart (clients reconnect and repopulate) + +### Namespace Setup (line 79) +```js +const presenceNamespace = io.of('/presence'); +``` +- Isolated from `/chat`, `/notes`, `/tasks` namespaces +- Registered in `backend/index.js:148`: `require('./sockets/presenceSocketHandler')(io)` + +### Connection Handler (lines 81-141) + +#### On Connect (lines 81-109) +1. **Extract userId** from `socket.handshake.query` (line 82) +2. **Validate** — disconnect if no userId (lines 84-87) +3. **Join room** with userId — enables targeted events (line 89) +4. **Update Map** — `onlineUsers.set(userId, { status: 'online', lastSeen: now })` (line 93) +5. **Build initial status snapshot** — iterate all online users, exclude self (lines 96-101) +6. **Emit 'initial-status'** to connecting user only (line 102) + - Payload: `[{ uid, status, lastSeen }, ...]` +7. **Broadcast 'user-status-changed'** to all other clients (lines 105-109) + - Payload: `{ userId, status: 'online', lastSeen: now }` + +#### On Disconnect (lines 111-128) +1. **Update Map** — set status to 'offline' with current timestamp (line 113) +2. **Broadcast 'user-status-changed'** with offline status (lines 115-119) +3. **30-second grace period** (lines 122-127): + - `setTimeout(30000)` — wait 30 seconds + - Check if user is still offline in Map + - If still offline: `onlineUsers.delete(userId)` — free memory + - If reconnected (status changed back to 'online'): keep in Map + - **Purpose:** prevents churn from brief network blips, tab switches, etc. + +#### On 'update-status' Event (lines 131-140) +1. **Update Map** with new status (`'away'`, `'dnd'`, `'online'`, etc.) (line 133) +2. **Broadcast 'user-status-changed'** to all other clients (lines 135-139) + - Payload: `{ userId, status: newStatus, lastSeen: now }` + +--- + +## Frontend Trace + +### usePresence Hook +**File:** `src/hooks/usePresence.ts` +- Connects to `/presence` namespace via `socket.io-client` +- Passes `userId` in connection query +- Maintains local state of online users +- Exposes `onlineUsers` map and `updateStatus()` function + +### UI Components Using Presence +| Component | File | Usage | +|---|---|---| +| PeopleView | `src/components/views/PeopleView.tsx` | Green/grey dots on user cards | +| ChatView | `src/components/views/ChatView.tsx` | Online indicator on chat header | +| MessagesPage | `src/components/views/MessagesPage.tsx` | Online status in conversation list | +| TeamMembers | `src/components/views/PeopleView.tsx` | Team member presence | +| DashboardHome | `src/components/views/DashboardHome.tsx` | Quick presence overview | + +--- + +## Socket Events Reference + +| Event | Direction | Payload | Purpose | +|---|---|---|---| +| `initial-status` | Server → Client (on connect) | `[{ uid, status, lastSeen }, ...]` | Snapshot of all online users | +| `user-status-changed` | Server → All (broadcast) | `{ userId, status, lastSeen }` | Notify status change | +| `update-status` | Client → Server | `string` (e.g., 'away', 'online') | User manually changes status | +| `disconnect` | Client → Server | — | User disconnected (tab close, network loss) | + +--- + +## Database Persistence + +### MongoDB User Document +| Field | Type | Updated When | Source | +|---|---|---|---| +| `status` | String | Login (`/api/users/sync`) | Set to `'online'` | +| `lastSeen` | Date | Login, activity | `new Date()` | + +- MongoDB persistence is **secondary** to the in-memory Map +- MongoDB `lastSeen` is used for "last seen X ago" when user is offline and server has restarted (Map is empty) +- The in-memory Map is the real-time source of truth + +--- + +## Edge Cases & Error Handling + +| Scenario | Behavior | +|---|---| +| Server restart | All presence data lost. Clients reconnect, Map repopulates. MongoDB `lastSeen` fills gap. | +| Brief network blip (<30s) | Grace period keeps user in Map. On reconnect, status returns to 'online'. | +| Multiple tabs | Each tab creates a separate socket connection. User appears online as long as one tab is open. | +| Mobile app backgrounded | Socket disconnects → user goes offline after 30s grace period. | +| No userId in handshake | Socket immediately disconnected (line 85-87). | + +--- + +## Cross-References + +- [06-middleware-stack.md](./06-middleware-stack.md) — Socket.IO setup in index.js +- [08-firebase-auth-flow.md](./08-firebase-auth-flow.md) — User sync on login sets initial status +- [09-user-profile-management.md](./09-user-profile-management.md) — /api/users/sync updates status +- [26-instant-chat-system.md](./26-instant-chat-system.md) — Chat uses presence for online indicators diff --git a/docs/features/12-haveibeenpwned-integration.md b/docs/features/12-haveibeenpwned-integration.md new file mode 100644 index 00000000..0c932e77 --- /dev/null +++ b/docs/features/12-haveibeenpwned-integration.md @@ -0,0 +1,223 @@ +# 12 — HaveIBeenPwned Integration + +**NEW document** — K-anonymity SHA-256 prefix matching for password breach checks + +--- + +## Feature Summary + +Zync integrates the Have I Been Pwned (HIBP) Pwned Passwords API to check if a user's password has appeared in known data breaches. The integration uses k-anonymity: only the first 5 characters of the SHA-1 hash are sent to the API, ensuring the actual password or full hash is never transmitted. The service fails open — if the API is down, users are not blocked. + +--- + +## Architecture Diagram + +``` +┌─────────────────── FRONTEND (Signup/Settings) ───────────┐ +│ │ +│ User enters password │ +│ │ │ +│ ▼ │ +│ POST /api/users/check-breached-password │ +│ { password: "user_input" } │ +│ │ │ +│ ▼ │ +│ ┌─────────────────── BACKEND ─────────────────────────┐ │ +│ │ │ │ +│ │ haveIBeenPwnedService.js │ │ +│ │ │ │ +│ │ Step 1: SHA-1 hash the password │ │ +│ │ crypto.createHash('sha1') │ │ +│ │ .update(password).digest('hex').toUpperCase() │ │ +│ │ → e.g., "5BAA61E4C9B93F3F0682250B6CF8331B7EE68FD8"│ │ +│ │ │ │ +│ │ Step 2: Split into prefix + suffix │ │ +│ │ prefix = first 5 chars → "5BAA6" │ │ +│ │ suffix = remaining 35 chars → "1E4C9B93F3F..." │ │ +│ │ │ │ +│ │ Step 3: Query HIBP API with prefix ONLY │ │ +│ │ GET https://api.pwnedpasswords.com/range/5BAA6 │ │ +│ │ Headers: { 'Add-Padding': 'true' } │ │ +│ │ Timeout: 5000ms │ │ +│ │ │ │ +│ │ Step 4: API returns ~500 hash suffixes │ │ +│ │ "1E4C9B93F3F0682250B6CF8331B7EE68FD8:3" │ │ +│ │ "2BAA61E4C9B93F3F0682250B6CF8331B7EE68FD8:1" │ │ +│ │ ... │ │ +│ │ │ │ +│ │ Step 5: Local match — find our suffix in results │ │ +│ │ if hashSuffix === suffix → COMPROMISED │ │ +│ │ return { isCompromised: true, count: 3 } │ │ +│ │ │ │ +│ │ Fail-open: on API error → { isCompromised: false } │ │ +│ └──────────────────────────────────────────────────────┘ │ +│ │ +│ Frontend shows warning if isCompromised === true │ +│ "This password has been found in N data breaches" │ +└────────────────────────────────────────────────────────────┘ +``` + +--- + +## Backend Trace + +### File: `backend/services/haveIBeenPwnedService.js` (122 lines) + +### Imports (lines 81-82) +```js +const crypto = require('crypto'); +const axios = require('axios'); +``` + +### API Endpoint (line 84) +```js +const PWNED_PASSWORDS_BASE = 'https://api.pwnedpasswords.com/range/'; +``` + +### checkPassword Function (lines 94-119) + +#### Step 1: SHA-1 Hash (line 95) +```js +const sha1 = crypto.createHash('sha1').update(password).digest('hex').toUpperCase(); +``` +- Uses Node.js built-in `crypto` module (no external dependency) +- Uppercase required by HIBP API specification + +#### Step 2: Split Hash (lines 96-97) +```js +const prefix = sha1.substring(0, 5); +const suffix = sha1.substring(5); +``` +- **Prefix (5 chars):** Sent to API — shared by ~500 other hashes +- **Suffix (35 chars):** Kept locally — used for matching + +#### Step 3: API Request (lines 100-103) +```js +const response = await axios.get(`${PWNED_PASSWORDS_BASE}${prefix}`, { + headers: { 'Add-Padding': 'true' }, + timeout: 5000, +}); +``` +- **Add-Padding header:** Obfuscates actual match count — adds zero-count entries to prevent timing attacks +- **5-second timeout:** Prevents backend from hanging if API is unresponsive + +#### Step 4: Parse Response (lines 105-111) +```js +const lines = response.data.split('\n'); +for (const line of lines) { + const [hashSuffix, count] = line.trim().split(':'); + if (hashSuffix === suffix) { + return { isCompromised: true, count: parseInt(count, 10) }; + } +} +``` +- Response is plain text, one hash suffix per line +- Format: `SUFFIX:COUNT` (e.g., `1E4C9B93F3F0682250B6CF8331B7EE68FD8:3`) +- Local comparison only — full hash never leaves the server + +#### Step 5: No Match (line 113) +```js +return { isCompromised: false, count: 0 }; +``` + +#### Error Handling — Fail Open (lines 114-118) +```js +catch (error) { + console.error('HIBP password check failed:', error.message); + return { isCompromised: false, count: 0 }; +} +``` +- **Fail-open design:** If HIBP API is down, return "not compromised" +- **Rationale:** Don't block user registration/login because a third-party API is unavailable +- Error is logged for monitoring + +--- + +### Route Integration +**File:** `backend/routes/userRoutes.js:161-174` + +```js +router.post('/check-breached-password', async (req, res) => { + const { password } = req.body; + if (!password || typeof password !== 'string') { + return res.status(400).json({ message: 'Password is required' }); + } + try { + const result = await checkPassword(password); + res.json(result); + } catch (error) { + console.error('Breached password check error:', error.message); + res.status(429).json({ message: error.message }); + } +}); +``` + +- **No auth required** — endpoint is called during signup before user exists +- **Input validation:** password must be a non-empty string +- **429 on rate limit:** HIBP API may rate-limit aggressive callers + +--- + +## Frontend Trace + +### Signup Page +**File:** `src/pages/Signup.tsx` +- Password input field with real-time breach check +- On password entry (debounced), calls `POST /api/users/check-breached-password` +- If `isCompromised === true`: shows warning banner + - "This password has been found in {count} data breaches. Please choose a different password." +- If `isCompromised === false`: shows green checkmark +- User can still proceed even with compromised password (warning, not block) + +### SettingsView — Security Tab +**File:** `src/components/views/SettingsView.tsx` +- Password change form includes breach check +- Same warning UI as signup + +--- + +## Privacy & Security Analysis + +### K-Anonymity Model +1. **What is sent:** Only first 5 chars of SHA-1 hash (e.g., "5BAA6") +2. **What is NOT sent:** Password, full hash, user identity, IP address (axios doesn't forward) +3. **API response:** ~500 hash suffixes matching the prefix +4. **Local matching:** Full hash suffix compared locally — HIBP never knows which hash was queried +5. **Result:** HIBP cannot determine which password was checked — privacy preserved + +### Add-Padding Header +- Without padding: response size correlates with match count → timing attack possible +- With padding: all responses have similar size → timing attack mitigated +- Adds fake zero-count entries to response + +### Fail-Open Design +- If HIBP API is unavailable, the check returns "not compromised" +- User experience is not degraded by third-party outage +- Trade-off: a compromised password might be accepted during API downtime + +--- + +## Error Paths + +| Scenario | HTTP Status | Response | User Impact | +|---|---|---|---| +| No password provided | 400 | `{ message: "Password is required" }` | Validation error | +| HIBP API timeout (>5s) | 200 | `{ isCompromised: false, count: 0 }` | No warning shown (fail-open) | +| HIBP API error | 200 | `{ isCompromised: false, count: 0 }` | No warning shown (fail-open) | +| HIBP API rate limit | 429 | `{ message: error.message }` | Error toast shown | +| Password compromised | 200 | `{ isCompromised: true, count: N }` | Warning banner shown | +| Password safe | 200 | `{ isCompromised: false, count: 0 }` | Green checkmark shown | + +--- + +## Environment Variables + +None required — HIBP Pwned Passwords API is free and public. + +--- + +## Cross-References + +- [02-security-auth-architecture.md](./02-security-auth-architecture.md) — Security overview +- [08-firebase-auth-flow.md](./08-firebase-auth-flow.md) — Signup flow where breach check is used +- [09-user-profile-management.md](./09-user-profile-management.md) — Settings security tab From ce5f7eb6fe01cc53fc56a989b377ee0435b004d0 Mon Sep 17 00:00:00 2001 From: Thanmayee Reddy Kotha <190446018+thanmayeereddykotha@users.noreply.github.com> Date: Sun, 16 Aug 2026 15:01:33 +0530 Subject: [PATCH 2/5] =?UTF-8?q?docs:=20add=2013-linkedin-oauth=20+=2014-pr?= =?UTF-8?q?oject-crud=20=E2=80=94=20LinkedIn=20OAuth=20flow,=20project=20C?= =?UTF-8?q?RUD=20with=2018=20endpoints,=20GitHub=20repo=20linking?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../features/13-linkedin-oauth-integration.md | 283 +++++++++++++ docs/features/14-project-crud.md | 379 ++++++++++++++++++ 2 files changed, 662 insertions(+) create mode 100644 docs/features/13-linkedin-oauth-integration.md create mode 100644 docs/features/14-project-crud.md diff --git a/docs/features/13-linkedin-oauth-integration.md b/docs/features/13-linkedin-oauth-integration.md new file mode 100644 index 00000000..111c8338 --- /dev/null +++ b/docs/features/13-linkedin-oauth-integration.md @@ -0,0 +1,283 @@ +# 13 — LinkedIn OAuth Integration + +**NEW document** — LinkedIn OAuth flow, custom token generation, Firebase user creation, profile sync + +--- + +## Feature Summary + +LinkedIn sign-in uses a server-side OAuth 2.0 flow. Unlike Google/GitHub which use Firebase's `signInWithPopup`, LinkedIn requires a full redirect-based OAuth flow handled by the backend. The backend exchanges the LinkedIn auth code for an access token, fetches the user's LinkedIn profile, creates or retrieves a Firebase user, generates a Firebase custom token, and redirects the frontend to `/login?customToken=`. + +--- + +## Architecture Diagram + +``` +┌──────────────── FRONTEND ──────────────────────────────┐ +│ │ +│ Login.tsx → LinkedinSignInButton component │ +│ └─ Links to: /api/linkedin/auth (full page redirect) │ +│ │ +│ After redirect back: │ +│ Login.tsx useEffect reads URL params: │ +│ ├─ ?customToken= → signInWithCustomToken(auth) │ +│ ├─ ?error= → toast error │ +│ └─ postLoginRedirect(navigate, user) │ +│ │ +└─────────────────────────────────────────────────────────┘ + │ + ▼ +┌──────────────── BACKEND (linkedinRoutes.js) ───────────┐ +│ │ +│ GET /api/linkedin/auth │ +│ ├─ Build LinkedIn OAuth URL with: │ +│ │ client_id, redirect_uri, state, scope │ +│ └─ res.redirect(authUrl) → user goes to LinkedIn │ +│ │ +│ GET /api/linkedin/callback │ +│ ├─ Receive ?code= from LinkedIn │ +│ ├─ POST https://www.linkedin.com/oauth/v2/accessToken │ +│ │ Exchange code for access_token │ +│ ├─ GET https://api.linkedin.com/v2/userinfo │ +│ │ Fetch profile: email, name, picture, sub │ +│ ├─ Firebase Admin: getUserByEmail(email) │ +│ │ ├─ If exists: use existing userRecord │ +│ │ └─ If not: createUser({ uid, email, displayName }) │ +│ ├─ getAuth().createCustomToken(userRecord.uid) │ +│ └─ res.redirect(FRONTEND_URL/login?customToken=token) │ +│ │ +└─────────────────────────────────────────────────────────┘ +``` + +--- + +## Sequence Diagram + +``` +User Frontend Backend LinkedIn API Firebase Admin + │ │ │ │ │ + │ Click LinkedIn│ │ │ │ + │─────────────►│ │ │ │ + │ │ Redirect to /api/linkedin/auth │ + │ │───────────►│ │ │ + │ │ │ Build OAuth URL │ + │ │ │──────────────►│ │ + │ Redirected to LinkedIn login │ │ + │◄──────────────────────────────────────────►│ │ + │ Authorizes app │ │ │ + │ │ │ │ │ + │ Redirected to /api/linkedin/callback?code=XXX │ + │───────────────────────────►│ │ │ + │ │ │ Exchange code for token │ + │ │ │──────────────►│ │ + │ │ │ access_token │ │ + │ │ │◄──────────────│ │ + │ │ │ Fetch userinfo │ │ + │ │ │──────────────►│ │ + │ │ │ Profile data │ │ + │ │ │◄──────────────│ │ + │ │ │ getUserByEmail │ │ + │ │ │───────────────────────────────►│ + │ │ │ userRecord │ │ + │ │ │◄───────────────────────────────│ + │ │ │ createCustomToken │ + │ │ │───────────────────────────────►│ + │ │ │ custom token │ │ + │ │ │◄───────────────────────────────│ + │ │ │ Redirect to /login?customToken │ + │◄──────────────────────────│ │ │ + │ │ │ │ │ + │ Frontend: signInWithCustomToken(auth, token) │ + │ │───────────────────────────────────────────►│ + │ │ │ │ Firebase user │ + │ │◄───────────────────────────────────────────│ + │ │ postLoginRedirect → /dashboard │ + │◄─────────────│ │ │ │ +``` + +--- + +## Backend Trace + +### File: `backend/routes/linkedinRoutes.js` (302 lines) + +### Imports (lines 78-86) +```js +const express = require('express'); +const router = express.Router(); +const axios = require('axios'); +const { getApps, initializeApp } = require('firebase-admin/app'); +const { getAuth } = require('firebase-admin/auth'); +``` + +### Configuration (lines 88-106) +| Variable | Source | Default | Purpose | +|---|---|---|---| +| `LINKEDIN_CLIENT_ID` | env | — | OAuth client ID | +| `LINKEDIN_CLIENT_SECRET` | env | — | OAuth client secret | +| `FRONTEND_URL` | env | `http://localhost:5173` | Frontend redirect URL | + +### Endpoint: GET /auth (lines 110-128) +1. **Build redirect URI:** `${req.protocol}://${req.get('host')}/api/linkedin/callback` + - Dynamic — works for both localhost and production +2. **Scope:** `'openid profile email'` — OpenID Connect scopes +3. **State:** Random string for CSRF protection: `Math.random().toString(36).substring(7)` +4. **Auth URL:** `https://www.linkedin.com/oauth/v2/authorization?response_type=code&client_id=...&redirect_uri=...&state=...&scope=...` +5. **Response:** `res.redirect(authUrl)` — browser follows redirect to LinkedIn + +### Endpoint: GET /callback (lines 132-298) + +#### Error Handling (lines 135-155) +- If `error` in query params: redirect to `${FRONTEND_URL}/login?error=` +- If no `code` in query params: redirect to `${FRONTEND_URL}/login?error=NoCodeProvided` + +#### Step 1: Exchange Code for Token (lines 166-200) +```js +const tokenResponse = await axios.post( + 'https://www.linkedin.com/oauth/v2/accessToken', + null, + { + params: { + grant_type: 'authorization_code', + code, + redirect_uri: redirectUri, + client_id: LINKEDIN_CLIENT_ID, + client_secret: LINKEDIN_CLIENT_SECRET, + }, + headers: { 'Content-Type': 'application/x-www-form-urlencoded' }, + } +); +const accessToken = tokenResponse.data.access_token; +``` + +#### Step 2: Fetch LinkedIn Profile (lines 208-221) +```js +const userinfoResponse = await axios.get( + 'https://api.linkedin.com/v2/userinfo', + { headers: { Authorization: `Bearer ${accessToken}` } } +); +const profile = userinfoResponse.data; +``` +- Profile fields: `email`, `sub` (LinkedIn user ID), `name`, `given_name`, `family_name`, `picture` + +#### Step 3: Create or Retrieve Firebase User (lines 242-277) +```js +let userRecord; +try { + userRecord = await getAuth().getUserByEmail(email); +} catch (error) { + if (error.code === 'auth/user-not-found') { + userRecord = await getAuth().createUser({ + uid: `linkedin:${profile.sub}`, + email: email, + emailVerified: true, + displayName: displayName, + photoURL: photoURL, + }); + } else { + throw error; + } +} +``` +- **UID format:** `linkedin:` — prefixed to distinguish from other providers +- **Email verified:** Set to `true` since LinkedIn verified the email +- **Existing user:** If email matches an existing Firebase user, links to that account + +#### Step 4: Generate Custom Token (line 281) +```js +const customToken = await getAuth().createCustomToken(userRecord.uid); +``` + +#### Step 5: Redirect to Frontend (line 285) +```js +res.redirect(`${FRONTEND_URL}/login?customToken=${customToken}`); +``` + +#### Error Fallback (lines 286-297) +- Any unhandled error: redirect to `${FRONTEND_URL}/login?error=LinkedIn Login Failed` +- Error logged: `console.error('LinkedIn OAuth Error:', err?.response?.data || err.message)` + +--- + +## Frontend Trace + +### LinkedinSignInButton Component +**File:** `src/components/auth/LinkedinSignInButton.tsx` +- Renders a LinkedIn-branded button +- On click: `window.location.href = '/api/linkedin/auth'` (full page redirect) +- No popup — LinkedIn OAuth requires full redirect flow + +### Login Page — Custom Token Handling +**File:** `src/pages/Login.tsx:174-196` +```js +useEffect(() => { + const params = new URLSearchParams(location.search); + const customToken = params.get('customToken'); + const authError = params.get('error'); + + if (authError) { + toast({ variant: 'destructive', title: 'Login Error', description: decodeURIComponent(authError) }); + navigate('/login', { replace: true }); + } else if (customToken) { + signInWithCustomToken(auth, customToken) + .then(async (cred) => { + toast({ title: 'Success', description: 'Logged in successfully' }); + await postLoginRedirect(navigate, cred.user); + }) + .catch((error) => { + toast({ variant: 'destructive', title: 'Login Error', description: error.message }); + }); + } +}, [location, navigate, toast]); +``` + +--- + +## OAuth Scopes + +| Scope | Access | Purpose | +|---|---|---| +| `openid` | OpenID Connect | Standard OIDC scope | +| `profile` | name, given_name, family_name, picture, sub | User profile data | +| `email` | email address | User email for Firebase account | + +--- + +## Error Paths + +| Scenario | Handling | User Sees | +|---|---|---| +| User denies permission | Redirect to `/login?error=` | Error toast | +| No code returned | Redirect to `/login?error=NoCodeProvided` | Error toast | +| Token exchange fails | Redirect to `/login?error=LinkedIn Login Failed` | Error toast | +| Userinfo fetch fails | Redirect to `/login?error=LinkedIn Login Failed` | Error toast | +| Firebase user creation fails | Redirect to `/login?error=LinkedIn Login Failed` | Error toast | +| Custom token generation fails | Redirect to `/login?error=LinkedIn Login Failed` | Error toast | +| Frontend: custom token invalid | `signInWithCustomToken` rejects | Error toast | + +--- + +## Environment Variables + +| Variable | Required | Description | +|---|---|---| +| `LINKEDIN_CLIENT_ID` | Yes | LinkedIn OAuth client ID | +| `LINKEDIN_CLIENT_SECRET` | Yes | LinkedIn OAuth client secret | +| `FRONTEND_URL` | Yes | Frontend URL for redirect (e.g., `https://zync-meet.vercel.app`) | + +--- + +## LinkedIn App Configuration + +### Required OAuth 2.0 Settings +- **Redirect URL:** `https:///api/linkedin/callback` (production) + `http://localhost:5000/api/linkedin/callback` (development) +- **Scopes:** `openid`, `profile`, `email` +- **Products:** "Sign In with LinkedIn using OpenID Connect" + +--- + +## Cross-References + +- [08-firebase-auth-flow.md](./08-firebase-auth-flow.md) — Custom token flow in Login.tsx +- [02-security-auth-architecture.md](./02-security-auth-architecture.md) — OAuth security +- [09-user-profile-management.md](./09-user-profile-management.md) — User sync after LinkedIn login diff --git a/docs/features/14-project-crud.md b/docs/features/14-project-crud.md new file mode 100644 index 00000000..708e7371 --- /dev/null +++ b/docs/features/14-project-crud.md @@ -0,0 +1,379 @@ +# 14 — Project CRUD + +**NEW document** — Project creation, list, detail, update, delete, GitHub repo linking, architecture analysis + +--- + +## Feature Summary + +Projects are the top-level organizational unit in Zync. Each project can be linked to a GitHub repository, has a multi-step pipeline (Step model), and contains tasks (ProjectTask model). The project routes handle CRUD operations, GitHub repo creation/linking, AI architecture analysis, team member management, collaborator invites, and task branch automation. + +--- + +## Architecture Diagram + +``` +┌─────────────────── FRONTEND ───────────────────────────┐ +│ │ +│ DashboardHome.tsx │ +│ ├─ Project cards grid │ +│ ├─ "New Project" button → CreateProjectDialog.tsx │ +│ └─ Project search/filter │ +│ │ +│ ProjectWorkspace.tsx │ +│ ├─ Pipeline view (Steps + Tasks) │ +│ ├─ GitHub integration panel │ +│ ├─ Team members panel │ +│ ├─ Architecture analysis viewer │ +│ └─ Task management (Kanban + list) │ +│ │ +│ Hooks: │ +│ ├─ useProjects.ts — TanStack Query for project list │ +│ ├─ useProject.ts — single project with steps │ +│ └─ useProjectTasks.ts — tasks per project │ +│ │ +│ Services: │ +│ └─ projectService.ts — API client wrappers │ +└──────────────────────┬──────────────────────────────────┘ + │ + ▼ +┌─────────────────── BACKEND ROUTES ──────────────────────┐ +│ │ +│ backend/routes/projectRoutes.js (2090 lines) │ +│ │ +│ POST / → create project │ +│ GET / → list user projects │ +│ GET /:id → get project detail │ +│ PATCH /:id → update project │ +│ DELETE /:id → delete project │ +│ POST /new-repo → create GitHub repo │ +│ POST /sync → sync GitHub repos │ +│ POST /:id/analyze-architecture → AI analysis │ +│ POST /:id/team → add team member │ +│ POST /:projectId/steps/:stepId/tasks → create task │ +│ PUT /:projectId/steps/:stepId/tasks/:taskId → update│ +│ DELETE /:projectId/steps/:stepId/tasks/:taskId → delete│ +│ GET /tasks/search → search tasks │ +│ POST /:projectId/quick-task → quick task creation │ +│ GET /:projectId/collaborator-assignees → list │ +│ POST /:projectId/invite-collaborator → invite │ +│ PATCH /:id/github-settings → edit GitHub repo │ +│ GET /:projectId/.../git-activity → commit history │ +│ POST /tasks/:taskId/merge-pr → merge PR + delete │ +│ │ +│ Services used: │ +│ ├─ kiloCodeGateway.js — AI architecture analysis │ +│ ├─ usageService.js — generation quota management │ +│ ├─ githubInstallation.js — Octokit builder │ +│ ├─ mailer.js — task assignment emails │ +│ └─ cache.js — Redis project caching │ +└─────────────────────────────────────────────────────────┘ +``` + +--- + +## Backend Trace + +### File: `backend/routes/projectRoutes.js` (2090 lines) + +### Imports (lines 76-102) +```js +const express = require('express'); +const router = express.Router(); +const { sendZyncEmail } = require('../services/mailer'); +const { getTaskAssignmentEmailHtml } = require('../utils/emailTemplates'); +const { escapeRegExp } = require('../utils/regexUtils'); +const User = require('../models/User'); +const Team = require('../models/Team'); +const Project = require('../models/Project'); +const Step = require('../models/Step'); +const ProjectTask = require('../models/ProjectTask'); +const axios = require('axios'); +const CryptoJS = require('crypto-js'); +const mongoose = require('mongoose'); +const authMiddleware = require('../middleware/authMiddleware'); +const { normalizeDoc, normalizeDocs } = require('../utils/normalize'); +const { paginateArray, setPaginationHeaders } = require('../utils/pagination'); +const { getProjectWithSteps, getProjectsWithSteps } = require('../utils/projectHelper'); +const cache = require('../utils/cache'); +const { analyzeArchitectureWithKilo } = require('../services/kiloCodeGateway'); +const { checkAndReserveGen, refundGen } = require('../services/usageService'); +const { getInstallationOctokit, invalidateInstallationCaches } = require('../utils/githubInstallation'); +``` + +### Cache Invalidation Helper (lines 104-109) +```js +async function invalidateProjectCache(project, additionalUids = []) { + const uids = [...new Set([project.ownerUid, ...(project.team || []), ...additionalUids].filter(Boolean))]; + const keys = uids.map((uid) => `projects:${uid}`); + await cache.invalidate(...keys); +} +``` +- Invalidates `projects:{uid}` for owner + all team members +- Called after every project mutation (create, update, delete, task change) + +### Architecture Analysis Cache (lines 118-202) +- **In-memory Map:** `architectureAnalysisCache` with TTL (default 6 hours) +- **Max entries:** 100 (configurable via `ARCHITECTURE_CACHE_MAX_ENTRIES`) +- **Pruning:** Removes expired entries + evicts oldest if over max +- **Cache key:** `${projectId}:${repoCacheKey}` (repo freshness fingerprint) + +--- + +### Endpoint: POST /new-repo (lines 414-525) +- **Auth:** required +- **Input:** `{ name, description, isPrivate }` +- **Logic:** + 1. Get owner's GitHub installation Octokit + 2. Create new GitHub repository via Octokit + 3. Return repo details (name, owner, URL) +- **Used by:** CreateProjectDialog when user opts to create a new repo + +### Endpoint: POST / (lines 527-596) +- **Auth:** required +- **Input:** `{ name, description, githubRepoName?, githubRepoOwner? }` +- **Logic:** + 1. Create Project document: `{ name, description, ownerUid, githubRepoName, githubRepoOwner }` + 2. Create default Step documents (pipeline stages) + 3. Invalidate cache for owner +- **Response:** Project document with steps + +### Endpoint: POST /:id/analyze-architecture (lines 598-752) +- **Auth:** required +- **Input:** `{ id }` (project ID), `?forceRefresh=true` +- **Logic:** + 1. Load project + GitHub repo details + 2. Check in-memory architecture cache (unless forceRefresh) + 3. Build repo freshness key (repo full_name, default_branch, pushed_at, updated_at) + 4. If cache hit and freshness matches: return cached architecture + 5. If cache miss: `checkAndReserveGen(uid)` — check AI generation quota + 6. Fetch repo file tree + interesting files (package.json, README.md, etc.) + 7. Call `analyzeArchitectureWithKilo(repoContext)` — AI analysis + 8. Store result in memory cache with TTL + 9. Return architecture analysis +- **Quota:** Uses `usageService.checkAndReserveGen()` — refunds on failure + +### Endpoint: POST /sync (lines 755-812) +- **Auth:** required +- **Logic:** + 1. Get owner's GitHub installation Octokit + 2. List all repos for the installation + 3. Return list of repos available for linking + +### Endpoint: GET / (lines 815-898) +- **Auth:** required +- **Cache:** Redis `projects:{uid}` with 300s TTL +- **Logic:** + 1. Check cache → return if hit + 2. Find projects where `ownerUid = uid` OR `uid` in `team` array + 3. Also fetch team projects (via Team membership) + 4. `getProjectsWithSteps()` — enrich with Step data + 5. Cache result +- **Response:** Array of projects with steps + +### Endpoint: GET /:id (lines 931-989) +- **Auth:** required +- **Logic:** + 1. `Project.findById(id)` + 2. Verify access: owner or team member + 3. `getProjectWithSteps()` — enrich with steps + tasks +- **Response:** Full project detail with steps and tasks + +### Endpoint: DELETE /:id (lines 992-1017) +- **Auth:** required +- **Logic:** + 1. Find project, verify ownership + 2. Delete all Step documents for project + 3. Delete all ProjectTask documents for project + 4. Delete Project document + 5. Invalidate cache +- **Response:** `{ message: "Project deleted" }` + +### Endpoint: PATCH /:id (lines 1019-1057) +- **Auth:** required +- **Input:** Partial project fields (`name`, `description`, etc.) +- **Logic:** + 1. Find project, verify ownership + 2. `Project.findByIdAndUpdate(id, { $set: updates }, { new: true })` + 3. Invalidate cache +- **Response:** Updated project + +### Endpoint: POST /:id/team (lines 901-929) +- **Auth:** required +- **Input:** `{ userId }` — UID to add to team +- **Logic:** + 1. Find project, verify ownership + 2. Add userId to `project.team` array (if not already present) + 3. Invalidate cache +- **Response:** Updated project + +--- + +### Task Endpoints + +#### POST /:projectId/steps/:stepId/tasks (lines 1059-1163) +- **Input:** `{ title, description, assignedTo?, assignedToName? }` +- **Logic:** + 1. Verify project + step exist + 2. Create ProjectTask: `{ title, description, stepId, projectId, assignedTo, assignedToName }` + 3. If `assignedTo`: `handleTaskAssignment()` — creates GitHub branch + 4. Send task assignment email via `sendZyncEmail()` + 5. Invalidate cache + +#### handleTaskAssignment (lines 236-284) +1. Generate branch name: `task/${slug}-${taskId}` +2. Generate completion commit message: `Complete Task: ${taskId}` +3. If project has GitHub repo: + - Get installation Octokit for owner + - Fetch default branch SHA + - Create new branch from default branch SHA + - Non-blocking: failure logged, DB still stores branch name + +#### PUT /:projectId/steps/:stepId/tasks/:taskId (lines 1165-1258) +- Update task fields (title, description, status, assignedTo) +- If assignment changes: `handleTaskAssignment()` for new assignee +- Invalidate cache + +#### DELETE /:projectId/steps/:stepId/tasks/:taskId (lines 1260-1340) +- Delete ProjectTask document +- Optionally delete GitHub branch +- Invalidate cache + +#### GET /tasks/search (lines 1342-1425) +- Search tasks by title across all user's projects +- Returns paginated results + +#### POST /:projectId/quick-task (lines 1427-1540) +- Create a task without a specific step (goes to inbox/default step) +- Simplified input: `{ title, description, assignedTo?, assignedToName? }` + +#### GET /:projectId/collaborator-assignees (lines 1542-1718) +- List GitHub collaborators + team members who can be assigned tasks +- Merges GitHub collaborators with Zync team members + +#### POST /:projectId/invite-collaborator (lines 1720-1853) +- Invite a GitHub user as collaborator to the project's repo +- Uses installation Octokit to send invitation + +#### PATCH /:id/github-settings (lines 1855-1934) +- Update GitHub repo settings: description, homepage, topics +- Uses installation Octokit + +#### GET /:projectId/.../git-activity (lines 1937-2003) +- Fetch commit count + messages for a task's branch +- Live from GitHub API + +#### POST /tasks/:taskId/merge-pr (lines 2006-2090) +- Merge PR for task branch + delete branch after merge +- Update task status to 'completed' + +--- + +## Frontend Trace + +### useProjects Hook +**File:** `src/hooks/useProjects.ts` +- TanStack Query: `useQuery({ queryKey: ['projects'], queryFn: fetchProjects })` +- `staleTime: 60_000` (1 min) +- Returns array of projects + +### useProject Hook +**File:** `src/hooks/useProject.ts` +- TanStack Query: `useQuery({ queryKey: ['project', id], queryFn: () => fetchProject(id) })` +- Returns single project with steps and tasks + +### CreateProjectDialog +**File:** `src/components/projects/CreateProjectDialog.tsx` +- Modal dialog for creating new projects +- Options: new GitHub repo, link existing repo, no GitHub +- Calls `POST /api/projects` or `POST /api/projects/new-repo` first + +### ProjectWorkspace +**File:** `src/components/views/ProjectWorkspace.tsx` +- Main project view with tabbed interface +- Tabs: Pipeline, Tasks, GitHub, Team, Architecture +- Pipeline: drag-and-drop steps, task cards per step +- Architecture: renders AI analysis result + +--- + +## Database Layer + +### Project Model (Mongoose) +**File:** `backend/models/Project.js` + +| Field | Type | Index | Notes | +|---|---|---|---| +| `name` | String | text | Project name | +| `description` | String | — | | +| `ownerUid` | String | yes | Firebase UID of owner | +| `team` | String[] | — | Array of team member UIDs | +| `githubRepoName` | String | — | Linked GitHub repo | +| `githubRepoOwner` | String | — | GitHub org/user | +| `webhookSecret` | String | — | For GitHub webhooks | +| `createdAt` | Date | — | | +| `updatedAt` | Date | — | | + +### Step Model +| Field | Type | Notes | +|---|---|---| +| `projectId` | ObjectId | Ref: Project | +| `title` | String | Step name (e.g., "Backlog", "In Progress") | +| `order` | Number | Sort order | + +### ProjectTask Model +| Field | Type | Notes | +|---|---|---| +| `projectId` | ObjectId | Ref: Project | +| `stepId` | ObjectId | Ref: Step | +| `title` | String | | +| `description` | String | | +| `assignedTo` | String? | Firebase UID | +| `assignedToName` | String? | Display name | +| `status` | String | pending/in_progress/completed | +| `githubBranchName` | String? | Auto-generated branch | +| `completionCommitMessage` | String? | Auto-generated | + +--- + +## Caching Strategy + +| Endpoint | Cache Key | TTL | Invalidation Trigger | +|---|---|---|---| +| GET / | `projects:{uid}` | 300s | Any project mutation | +| Architecture analysis | In-memory Map | 6h (configurable) | forceRefresh param or TTL expiry | + +--- + +## Error Paths + +| Scenario | HTTP Status | Response | +|---|---|---| +| No token | 401 | Unauthorized | +| Project not found | 404 | `{ message: "Project not found" }` | +| Not owner (delete/update) | 403 | `{ message: "Not authorized" }` | +| GitHub repo creation fails | 500 | Error message from GitHub | +| AI quota exceeded | 429 | `{ message: "Generation limit reached" }` | +| GitHub branch creation fails | — (logged) | Task still created, branch name stored | +| Server error | 500 | `{ message: "Server error" }` | + +--- + +## Environment Variables + +| Variable | Required | Default | Description | +|---|---|---|---| +| `ENCRYPTION_KEY` | Yes (prod) | `dev-only-encryption-key-123` | GitHub token encryption | +| `ARCHITECTURE_CACHE_TTL_MS` | No | `21600000` (6h) | Architecture cache TTL | +| `ARCHITECTURE_CACHE_MAX_ENTRIES` | No | `100` | Max cached analyses | + +--- + +## Cross-References + +- [05-database-schema-and-models.md](./05-database-schema-and-models.md) — Project, Step, ProjectTask models +- [15-project-steps-pipeline.md](./15-project-steps-pipeline.md) — Step pipeline detail +- [16-task-management.md](./16-task-management.md) — Task CRUD detail +- [22-github-oauth-integration.md](./22-github-oauth-integration.md) — GitHub installation Octokit +- [25-ai-architecture-analysis.md](./25-ai-architecture-analysis.md) — Kilo Code Gateway +- [41-team-crud-and-invites.md](./41-team-crud-and-invites.md) — Team member management From 60a58f0aa83de6a930c81c15cc1f8b894c7894d0 Mon Sep 17 00:00:00 2001 From: Thanmayee Reddy Kotha <190446018+thanmayeereddykotha@users.noreply.github.com> Date: Sun, 16 Aug 2026 15:02:25 +0530 Subject: [PATCH 3/5] =?UTF-8?q?docs:=20add=2015-project-steps-pipeline=20+?= =?UTF-8?q?=2016-task-management=20=E2=80=94=20Kanban=20pipeline,=20task?= =?UTF-8?q?=20CRUD,=20GitHub=20branch=20automation,=20PR=20merge?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/features/15-project-steps-pipeline.md | 174 +++++++++++++ docs/features/16-task-management.md | 287 +++++++++++++++++++++ 2 files changed, 461 insertions(+) create mode 100644 docs/features/15-project-steps-pipeline.md create mode 100644 docs/features/16-task-management.md diff --git a/docs/features/15-project-steps-pipeline.md b/docs/features/15-project-steps-pipeline.md new file mode 100644 index 00000000..64938b2d --- /dev/null +++ b/docs/features/15-project-steps-pipeline.md @@ -0,0 +1,174 @@ +# 15 — Project Steps Pipeline + +**NEW document** — Step model, pipeline stages, drag-and-drop ordering, task-to-step assignment + +--- + +## Feature Summary + +Each project has a multi-step pipeline (Kanban-style). Steps represent stages like "Backlog", "In Progress", "Review", "Done". Tasks are assigned to steps and can be moved between steps via drag-and-drop. The pipeline is the primary project workspace view. + +--- + +## Architecture Diagram + +``` +┌─────────────────── FRONTEND ───────────────────────────┐ +│ │ +│ ProjectWorkspace.tsx → Pipeline tab │ +│ ├─ KanbanBoard.tsx │ +│ │ ├─ Column per Step (ordered by Step.order) │ +│ │ ├─ TaskCard.tsx per task in each column │ +│ │ ├─ Drag-and-drop: @dnd-kit/core │ +│ │ └─ On drop: PATCH task's stepId │ +│ ├─ AddStepButton → creates new Step │ +│ └─ StepHeader → rename, delete, reorder │ +│ │ +│ Hooks: │ +│ ├─ useProject.ts → includes steps + tasks │ +│ ├─ useUpdateTaskStep.ts → mutation to change stepId │ +│ └─ useReorderSteps.ts → mutation to update Step.order │ +│ │ +└──────────────────────┬──────────────────────────────────┘ + │ + ▼ +┌─────────────────── BACKEND ────────────────────────────┐ +│ │ +│ projectRoutes.js (task endpoints also handle steps) │ +│ │ +│ Step model (backend/models/Step.js) │ +│ ├─ projectId: ObjectId → Project │ +│ ├─ title: String │ +│ ├─ order: Number │ +│ └─ createdAt: Date │ +│ │ +│ ProjectTask model (backend/models/ProjectTask.js) │ +│ ├─ stepId: ObjectId → Step │ +│ └─ Moving tasks = updating stepId │ +│ │ +│ Default steps created on project creation: │ +│ 1. "Backlog" (order: 0) │ +│ 2. "To Do" (order: 1) │ +│ 3. "In Progress" (order: 2) │ +│ 4. "Review" (order: 3) │ +│ 5. "Done" (order: 4) │ +│ │ +└─────────────────────────────────────────────────────────┘ +``` + +--- + +## Backend Trace + +### Step Creation on Project Init +**File:** `backend/routes/projectRoutes.js:527-596` + +When a project is created (`POST /`), default steps are automatically generated: +``` +Step 1: "Backlog" (order: 0) +Step 2: "To Do" (order: 1) +Step 3: "In Progress" (order: 2) +Step 4: "Review" (order: 3) +Step 5: "Done" (order: 4) +``` + +### Step Enrichment +**File:** `backend/utils/projectHelper.js` + +#### `getProjectWithSteps(projectId)` +1. Fetch Project by ID +2. Fetch all Steps for project, sorted by `order` +3. Fetch all ProjectTasks for project +4. Group tasks by `stepId` +5. Return: `{ ...project, steps: [{ ...step, tasks: [...] }] }` + +#### `getProjectsWithSteps(uid)` +1. Fetch all projects for user (owned + team) +2. For each project, fetch steps + tasks +3. Return enriched array + +### Task Movement Between Steps +**File:** `backend/routes/projectRoutes.js:1165-1258` + +`PUT /:projectId/steps/:stepId/tasks/:taskId`: +- Accepts `stepId` in body to move task to a different step +- Updates `ProjectTask.stepId` +- Invalidates project cache + +--- + +## Frontend Trace + +### KanbanBoard Component +**File:** `src/components/projects/KanbanBoard.tsx` +- Uses `@dnd-kit/core` for drag-and-drop +- Each column is a droppable area +- Each task card is a draggable item +- On drop: calls `useUpdateTaskStep` mutation + +### useUpdateTaskStep Hook +**File:** `src/hooks/useUpdateTaskStep.ts` +- TanStack Query mutation +- `PUT /api/projects/:projectId/steps/:stepId/tasks/:taskId` with new `stepId` +- On success: invalidates `['project', projectId]` query + +### Step Management UI +- **Add step:** Button at end of board → `POST /api/projects/:id/steps` +- **Rename step:** Inline edit on step header → `PATCH /api/projects/:id/steps/:stepId` +- **Delete step:** Context menu → `DELETE /api/projects/:id/steps/:stepId` (tasks moved to previous step) +- **Reorder:** Drag step header → `PATCH` with updated `order` values + +--- + +## Database Layer + +### Step Model +**File:** `backend/models/Step.js` + +| Field | Type | Required | Index | Notes | +|---|---|---|---|---| +| `projectId` | ObjectId | yes | yes | Ref: Project | +| `title` | String | yes | — | Display name | +| `order` | Number | yes | — | Sort position (0-based) | +| `createdAt` | Date | auto | — | | +| `updatedAt` | Date | auto | — | | + +**Index:** `{ projectId: 1, order: 1 }` — efficient sorted fetch + +### ProjectTask → Step Relationship +- `ProjectTask.stepId` references `Step._id` +- When step is deleted, tasks are moved to the previous step (or first step if deleting first) +- No cascade delete — tasks are preserved + +--- + +## Default Pipeline Configuration + +| Order | Title | Purpose | +|---|---|---| +| 0 | Backlog | Unstarted work, ideas | +| 1 | To Do | Prioritized work ready to start | +| 2 | In Progress | Active work | +| 3 | Review | Code review / QA | +| 4 | Done | Completed work | + +Users can customize: add, rename, delete, and reorder steps. + +--- + +## Error Paths + +| Scenario | HTTP Status | Response | +|---|---|---| +| Step not found | 404 | `{ message: "Step not found" }` | +| Project not found | 404 | `{ message: "Project not found" }` | +| Not authorized | 403 | `{ message: "Not authorized" }` | +| Server error | 500 | `{ message: "Server error" }` | + +--- + +## Cross-References + +- [14-project-crud.md](./14-project-crud.md) — Project creation triggers default step creation +- [16-task-management.md](./16-task-management.md) — Tasks live within steps +- [05-database-schema-and-models.md](./05-database-schema-and-models.md) — Step + ProjectTask models diff --git a/docs/features/16-task-management.md b/docs/features/16-task-management.md new file mode 100644 index 00000000..20856390 --- /dev/null +++ b/docs/features/16-task-management.md @@ -0,0 +1,287 @@ +# 16 — Task Management + +**NEW document** — Task CRUD, assignment, GitHub branch automation, quick tasks, task search, PR merge + +--- + +## Feature Summary + +Tasks are the atomic work units within a project. Each task belongs to a Step in the pipeline, can be assigned to a team member, and can auto-create a GitHub branch for the assignee. Tasks support search, quick creation, status tracking, and PR merging with branch cleanup. + +--- + +## Architecture Diagram + +``` +┌─────────────────── FRONTEND ───────────────────────────┐ +│ │ +│ KanbanBoard.tsx │ +│ ├─ TaskCard.tsx — title, assignee, status, branch │ +│ ├─ Drag between steps (changes stepId) │ +│ └─ Click card → TaskDetailDialog.tsx │ +│ │ +│ TaskDetailDialog.tsx │ +│ ├─ Edit title/description │ +│ ├─ Assign to team member │ +│ ├─ View GitHub branch + commits │ +│ ├─ Merge PR button → POST /tasks/:id/merge-pr │ +│ └─ Delete task │ +│ │ +│ QuickAddTask.tsx │ +│ └─ Inline input → POST /:projectId/quick-task │ +│ │ +│ TaskSearch.tsx │ +│ └─ Debounced input → GET /tasks/search?query=... │ +│ │ +│ Hooks: │ +│ ├─ useCreateTask.ts → POST /:projectId/steps/:stepId/tasks │ +│ ├─ useUpdateTask.ts → PUT /:projectId/steps/:stepId/tasks/:taskId │ +│ ├─ useDeleteTask.ts → DELETE /:projectId/steps/:stepId/tasks/:taskId │ +│ ├─ useQuickTask.ts → POST /:projectId/quick-task │ +│ ├─ useSearchTasks.ts → GET /tasks/search │ +│ └─ useMergePR.ts → POST /tasks/:taskId/merge-pr │ +│ │ +└──────────────────────┬──────────────────────────────────┘ + │ + ▼ +┌─────────────────── BACKEND ────────────────────────────┐ +│ │ +│ projectRoutes.js — Task endpoints (lines 1059-2090) │ +│ │ +│ Task lifecycle: │ +│ 1. Create task → assigned? → create GitHub branch │ +│ 2. Update task → reassign? → create branch for new │ +│ 3. Move task → update stepId │ +│ 4. Search tasks → regex on title across projects │ +│ 5. View git activity → fetch commits from branch │ +│ 6. Merge PR → merge branch to default + delete branch │ +│ 7. Delete task → optionally delete branch │ +│ │ +│ GitHub branch automation: │ +│ ├─ Branch name: task/{slug-title}-{taskId} │ +│ ├─ Created from default branch SHA │ +│ ├─ Completion commit msg: "Complete Task: {taskId}" │ +│ └─ Non-blocking: DB stores branch name even if GH fails│ +│ │ +│ Services: │ +│ ├─ githubInstallation.js → Octokit for branch ops │ +│ ├─ mailer.js → task assignment email │ +│ └─ usageService.js → not used for tasks (no AI) │ +│ │ +└─────────────────────────────────────────────────────────┘ +``` + +--- + +## Backend Trace + +### File: `backend/routes/projectRoutes.js` + +### POST /:projectId/steps/:stepId/tasks (lines 1059-1163) +- **Auth:** required +- **Input:** `{ title, description, assignedTo?, assignedToName? }` +- **Logic:** + 1. Verify project exists + user has access + 2. Verify step exists in project + 3. Create ProjectTask: `{ title, description, projectId, stepId, assignedTo, assignedToName, status: 'pending' }` + 4. If `assignedTo`: call `handleTaskAssignment(project, task, assignedTo, assignedToName)` + 5. Send assignment email via `sendZyncEmail()` with `getTaskAssignmentEmailHtml()` + 6. Invalidate project cache +- **Response:** Created task document + +### handleTaskAssignment (lines 236-284) +``` +1. Generate slug from task title: slugify(title).substring(0, 30) +2. Generate branch name: `task/${slug}-${task._id}` +3. Generate completion commit message: `Complete Task: ${task._id}` +4. If project has GitHub repo (githubRepoOwner + githubRepoName): + a. Get installation Octokit for project.ownerUid + b. GET /repos/{owner}/{repo} → fetch default_branch + c. GET /repos/{owner}/{repo}/git/ref/heads/{defaultBranch} → get SHA + d. POST /repos/{owner}/{repo}/git/refs → create branch from SHA + e. Non-blocking: if fails, log error, continue (DB has branch name) +5. Return { assignedTo, assignedToName, githubBranchName, completionCommitMessage } +``` + +### PUT /:projectId/steps/:stepId/tasks/:taskId (lines 1165-1258) +- **Auth:** required +- **Input:** Partial task fields (`title`, `description`, `status`, `assignedTo`, `stepId`) +- **Logic:** + 1. Find task by ID + 2. If `stepId` changed: update step reference (move between pipeline columns) + 3. If `assignedTo` changed: call `handleTaskAssignment()` for new assignee + 4. Update task fields + 5. Invalidate cache +- **Response:** Updated task + +### DELETE /:projectId/steps/:stepId/tasks/:taskId (lines 1260-1340) +- **Auth:** required +- **Logic:** + 1. Find task, verify project access + 2. If task has `githubBranchName` and project has GitHub repo: + - Delete branch via Octokit (non-blocking, failure logged) + 3. Delete ProjectTask document + 4. Invalidate cache +- **Response:** `{ message: "Task deleted" }` + +### GET /tasks/search (lines 1342-1425) +- **Auth:** required +- **Input:** `?query=&page=1&limit=20` +- **Logic:** + 1. Get all project IDs for user (owned + team) + 2. Regex search on `ProjectTask.title` (case-insensitive) + 3. Filter by user's projects + 4. Paginate results + 5. Set pagination headers +- **Response:** Array of matching tasks with project info + +### POST /:projectId/quick-task (lines 1427-1540) +- **Auth:** required +- **Input:** `{ title, description?, assignedTo?, assignedToName? }` +- **Logic:** + 1. Find project, verify access + 2. Find first step (lowest `order`) in project + 3. Create task in first step + 4. If assigned: `handleTaskAssignment()` + 5. Send email if assigned + 6. Invalidate cache +- **Purpose:** Fast task creation without specifying a step + +### GET /:projectId/collaborator-assignees (lines 1542-1718) +- **Auth:** required +- **Logic:** + 1. Get GitHub collaborators via Octokit + 2. Get Zync team members + 3. Merge lists, deduplicate by GitHub username + 4. Return combined list with display names + avatars +- **Used by:** Task assignee dropdown + +### GET /:projectId/steps/:stepId/tasks/:taskId/git-activity (lines 1937-2003) +- **Auth:** required +- **Logic:** + 1. Find task, verify project access + 2. If task has `githubBranchName`: + - GET /repos/{owner}/{repo}/commits?sha={branchName} + - Extract commit count + messages + 3. Return `{ commitCount, commits: [{ sha, message, author, date }] }` +- **Used by:** TaskDetailDialog git activity panel + +### POST /tasks/:taskId/merge-pr (lines 2006-2090) +- **Auth:** required +- **Input:** `{ taskId }` (from URL) +- **Logic:** + 1. Find task by ID + 2. Find project for task + 3. Verify user is project owner or task assignee + 4. Get installation Octokit + 5. Check if PR exists for branch (via GitHub API) + 6. If PR exists: merge PR via Octokit + 7. Delete branch: `DELETE /repos/{owner}/{repo}/git/refs/heads/{branchName}` + 8. Update task status to `'completed'` + 9. Invalidate cache +- **Response:** `{ message: "PR merged and branch deleted", task }` + +--- + +## Frontend Trace + +### TaskCard Component +**File:** `src/components/projects/TaskCard.tsx` +- Displays: title, assignee avatar, status badge, branch name +- Draggable via `@dnd-kit` +- Click opens `TaskDetailDialog` + +### TaskDetailDialog +**File:** `src/components/projects/TaskDetailDialog.tsx` +- Tabs: Details, Git Activity, Comments +- **Details:** Edit title, description, assignee, status +- **Git Activity:** Shows commits from GitHub branch +- **Comments:** Thread stored in MongoDB (task.comments array) +- Actions: Save, Delete, Merge PR + +### QuickAddTask +**File:** `src/components/projects/QuickAddTask.tsx` +- Inline text input at top of Kanban board +- Enter key creates task via `POST /:projectId/quick-task` +- No step selection needed — goes to first step + +### TaskSearch +**File:** `src/components/projects/TaskSearch.tsx` +- Debounced search input (300ms) +- Calls `GET /tasks/search?query=...` +- Results dropdown with project name + step + +--- + +## Database Layer + +### ProjectTask Model +**File:** `backend/models/ProjectTask.js` + +| Field | Type | Required | Index | Notes | +|---|---|---|---|---| +| `projectId` | ObjectId | yes | yes | Ref: Project | +| `stepId` | ObjectId | yes | yes | Ref: Step | +| `title` | String | yes | text | | +| `description` | String | no | — | | +| `assignedTo` | String | no | — | Firebase UID | +| `assignedToName` | String | no | — | Display name | +| `status` | String | no | — | pending/in_progress/completed | +| `githubBranchName` | String | no | — | Auto-generated | +| `completionCommitMessage` | String | no | — | Auto-generated | +| `comments` | Mixed | no | — | Array of comment objects | +| `createdAt` | Date | auto | — | | +| `updatedAt` | Date | auto | — | | + +**Text Index:** `{ title: 'text' }` — for search + +--- + +## GitHub Branch Automation + +| Step | GitHub API Call | Blocking? | +|---|---|---| +| Create branch | `POST /repos/{owner}/{repo}/git/refs` | No — failure logged | +| Fetch commits | `GET /repos/{owner}/{repo}/commits?sha={branch}` | Yes — for git activity view | +| Merge PR | `PUT /repos/{owner}/{repo}/pulls/{pr_number}/merge` | Yes — must succeed | +| Delete branch | `DELETE /repos/{owner}/{repo}/git/refs/heads/{branch}` | No — failure logged | + +### Branch Naming Convention +``` +task/{slugified-title-30-chars}-{mongodb-objectid} +``` +Example: `task/add-login-page-507f191e810c19729de860ea` + +--- + +## Error Paths + +| Scenario | HTTP Status | Response | +|---|---|---| +| Task not found | 404 | `{ message: "Task not found" }` | +| Project not found | 404 | `{ message: "Project not found" }` | +| Not authorized | 403 | `{ message: "Not authorized" }` | +| Branch creation fails | 201 | Task still created (branch name stored in DB) | +| PR merge fails | 500 | Error from GitHub API | +| Branch deletion fails | 200 | PR merged, branch deletion logged as warning | +| Server error | 500 | `{ message: "Server error" }` | + +--- + +## Environment Variables + +| Variable | Required | Description | +|---|---|---| +| `ENCRYPTION_KEY` | Yes (prod) | GitHub token decryption | +| `GITHUB_APP_ID` | Yes | GitHub App for installation Octokit | +| `GITHUB_PRIVATE_KEY` | Yes | GitHub App private key | + +--- + +## Cross-References + +- [14-project-crud.md](./14-project-crud.md) — Parent project routes +- [15-project-steps-pipeline.md](./15-project-steps-pipeline.md) — Steps that contain tasks +- [22-github-oauth-integration.md](./22-github-oauth-integration.md) — GitHub installation Octokit +- [23-github-webhook-handler.md](./23-github-webhook-handler.md) — Webhooks for branch/PR events +- [05-database-schema-and-models.md](./05-database-schema-and-models.md) — ProjectTask model From 03f687bd47ea756a589f8e947b6be75215acde08 Mon Sep 17 00:00:00 2001 From: Thanmayee Reddy Kotha <190446018+thanmayeereddykotha@users.noreply.github.com> Date: Sun, 16 Aug 2026 15:03:21 +0530 Subject: [PATCH 4/5] =?UTF-8?q?docs:=20add=2017-notes-system=20+=2018-fold?= =?UTF-8?q?ers-and-organization=20=E2=80=94=20note=20CRUD,=20rich=20text,?= =?UTF-8?q?=20folder=20hierarchy,=20sharing=20model?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/features/17-notes-system.md | 193 +++++++++++++++++ docs/features/18-folders-and-organization.md | 214 +++++++++++++++++++ 2 files changed, 407 insertions(+) create mode 100644 docs/features/17-notes-system.md create mode 100644 docs/features/18-folders-and-organization.md diff --git a/docs/features/17-notes-system.md b/docs/features/17-notes-system.md new file mode 100644 index 00000000..9bb1e69c --- /dev/null +++ b/docs/features/17-notes-system.md @@ -0,0 +1,193 @@ +# 17 — Notes System + +**NEW document** — Note CRUD, rich text editing, sharing, folder organization, project-scoped notes + +--- + +## Feature Summary + +Notes are rich text documents that users can create, edit, share, and organize into folders. Notes can be standalone or associated with a project. The system supports real-time collaborative editing via Socket.IO, with notes stored in MongoDB. + +--- + +## Architecture Diagram + +``` +┌─────────────────── FRONTEND ───────────────────────────┐ +│ │ +│ NotesView.tsx │ +│ ├─ Sidebar: Folder tree + note list │ +│ ├─ Editor: TipTap / Lexical rich text editor │ +│ ├─ Share dialog: add collaborators by UID │ +│ └─ Project notes tab (when in project workspace) │ +│ │ +│ Hooks: │ +│ ├─ useNotes.ts — list notes (TanStack Query) │ +│ ├─ useNote.ts — single note with real-time sync │ +│ ├─ useCreateNote.ts — create mutation │ +│ ├─ useUpdateNote.ts — update mutation │ +│ └─ useDeleteNote.ts — delete mutation │ +│ │ +│ Real-time: │ +│ └─ Socket.IO /notes namespace for collaborative edit │ +│ │ +└──────────────────────┬──────────────────────────────────┘ + │ + ▼ +┌─────────────────── BACKEND ROUTES ──────────────────────┐ +│ │ +│ backend/routes/noteRoutes.js (393 lines) │ +│ │ +│ POST / → create note │ +│ GET / → list notes (with folder filter) │ +│ GET /:id → get single note │ +│ PUT /:id → update note (title/content) │ +│ DELETE /:id → delete note (owner only) │ +│ │ +│ Folder endpoints: │ +│ POST /folders → create folder │ +│ GET /folders → list folders │ +│ PUT /folders/:id → update folder │ +│ DELETE /folders/:id → delete folder │ +│ POST /folders/:id/share → share folder │ +│ POST /folders/:id/unshare → unshare folder │ +│ │ +└─────────────────────────────────────────────────────────┘ +``` + +--- + +## Backend Trace + +### File: `backend/routes/noteRoutes.js` (393 lines) + +### Imports (lines 76-82) +```js +const express = require('express'); +const router = express.Router(); +const verifyToken = require('../middleware/authMiddleware'); +const Note = require('../models/Note'); +const Folder = require('../models/Folder'); +const { normalizeDoc, normalizeDocs } = require('../utils/normalize'); +const { paginateArray, setPaginationHeaders } = require('../utils/pagination'); +``` + +### Endpoint: POST / (lines 235-258) +- **Auth:** required +- **Input:** `{ title, content, ownerId, folderId?, projectId? }` +- **Logic:** + 1. Create Note with provided fields + 2. `ownerId` set to `req.user.uid` (or from body if provided) + 3. If `folderId`: note is placed in that folder + 4. If `projectId`: note is scoped to that project +- **Response:** Created note document + +### Endpoint: GET / (lines 260-310) +- **Auth:** required +- **Query params:** `?folderId=&page=1&limit=20` +- **Logic:** + 1. Find notes where `ownerId = uid` OR `uid` in `sharedWith` array + 2. If `folderId` provided: filter by folder + 3. Paginate results + 4. Set pagination headers +- **Response:** Array of notes + +### Endpoint: GET /:id (lines 312-342) +- **Auth:** required +- **Logic:** + 1. Find note by ID + 2. Check access: `note.ownerId === uid` OR `note.sharedWith.includes(uid)` + 3. If no access: 403 +- **Response:** Single note document + +### Endpoint: PUT /:id (lines 344-373) +- **Auth:** required +- **Input:** `{ title?, content?, folderId? }` +- **Logic:** + 1. Find note by ID + 2. Check access: owner or shared + 3. Build `updateData` from provided fields (partial update) + 4. `Note.findByIdAndUpdate(id, { $set: updateData }, { returnDocument: 'after' })` +- **Response:** Updated note + +### Endpoint: DELETE /:id (lines 375-391) +- **Auth:** required +- **Logic:** + 1. Find note by ID + 2. **Owner only:** `note.ownerId !== req.user.uid` → 403 + 3. Delete note +- **Response:** `{ message: "Note deleted successfully" }` + +--- + +## Frontend Trace + +### NotesView Component +**File:** `src/components/views/NotesView.tsx` +- Split-pane layout: folder sidebar + note list + editor +- Folder tree with expand/collapse +- Note list with title preview + last modified +- Rich text editor (TipTap or Lexical) + +### Note Editor +- Rich text with formatting: bold, italic, headings, lists, code blocks +- Auto-save on content change (debounced 1s) +- Real-time collaboration via Socket.IO + +### Real-Time Collaboration +- Socket.IO `/notes` namespace +- Events: `note:join`, `note:leave`, `note:edit`, `note:cursor` +- Broadcasts edits to all connected clients +- Conflict resolution via operational transform or CRDT + +--- + +## Database Layer + +### Note Model +**File:** `backend/models/Note.js` + +| Field | Type | Required | Index | Notes | +|---|---|---|---|---| +| `title` | String | yes | text | Note title | +| `content` | String | no | — | Rich text HTML/JSON | +| `ownerId` | String | yes | yes | Firebase UID | +| `folderId` | ObjectId | no | yes | Ref: Folder | +| `projectId` | ObjectId | no | yes | Ref: Project (optional) | +| `sharedWith` | String[] | no | — | Array of Firebase UIDs | +| `createdAt` | Date | auto | — | | +| `updatedAt` | Date | auto | — | | + +### Folder Model +**File:** `backend/models/Folder.js` + +| Field | Type | Required | Index | Notes | +|---|---|---|---|---| +| `name` | String | yes | — | Folder name | +| `ownerId` | String | yes | yes | Firebase UID | +| `parentId` | ObjectId | no | — | Parent folder (nested) | +| `type` | String | no | — | Folder category | +| `projectId` | ObjectId | no | — | Project-scoped folder | +| `color` | String | no | — | UI color tag | +| `collaboratorIds` | String[] | no | — | Shared with UIDs | +| `createdAt` | Date | auto | — | | + +--- + +## Error Paths + +| Scenario | HTTP Status | Response | +|---|---|---| +| No token | 401 | Unauthorized | +| Note not found | 404 | `{ error: "Note not found" }` | +| Not owner/shared | 403 | `{ error: "Unauthorized" }` | +| Only owner can delete | 403 | `{ error: "Unauthorized: Only owner can delete note" }` | +| Server error | 500 | `{ error: error.message }` | + +--- + +## Cross-References + +- [18-folders-and-organization.md](./18-folders-and-organization.md) — Folder CRUD and sharing +- [05-database-schema-and-models.md](./05-database-schema-and-models.md) — Note + Folder models +- [14-project-crud.md](./14-project-crud.md) — Project-scoped notes diff --git a/docs/features/18-folders-and-organization.md b/docs/features/18-folders-and-organization.md new file mode 100644 index 00000000..2d49f53b --- /dev/null +++ b/docs/features/18-folders-and-organization.md @@ -0,0 +1,214 @@ +# 18 — Folders and Organization + +**NEW document** — Folder CRUD, nested folders, sharing, collaborator management, project-scoped folders + +--- + +## Feature Summary + +Folders organize notes into a hierarchical structure. Folders can be nested (parent-child), shared with collaborators, color-tagged, and scoped to a project. The folder system provides the sidebar tree view in the Notes feature. + +--- + +## Architecture Diagram + +``` +┌─────────────────── FRONTEND ───────────────────────────┐ +│ │ +│ NotesView.tsx → Sidebar │ +│ ├─ FolderTree.tsx │ +│ │ ├─ Recursive rendering of nested folders │ +│ │ ├─ Expand/collapse per folder │ +│ │ ├─ Color dot per folder (Folder.color) │ +│ │ ├─ Context menu: rename, delete, share │ +│ │ └─ Drag notes into folders │ +│ ├─ "New Folder" button → CreateFolderDialog.tsx │ +│ └─ Shared folders section (shared by others) │ +│ │ +│ Hooks: │ +│ ├─ useFolders.ts — list folders (TanStack Query) │ +│ ├─ useCreateFolder.ts — create mutation │ +│ ├─ useUpdateFolder.ts — update mutation │ +│ ├─ useDeleteFolder.ts — delete mutation │ +│ ├─ useShareFolder.ts — share mutation │ +│ └─ useUnshareFolder.ts — unshare mutation │ +│ │ +└──────────────────────┬──────────────────────────────────┘ + │ + ▼ +┌─────────────────── BACKEND ROUTES ──────────────────────┐ +│ │ +│ backend/routes/noteRoutes.js — Folder endpoints │ +│ │ +│ POST /folders → create folder │ +│ GET /folders → list folders │ +│ PUT /folders/:id → update folder │ +│ DELETE /folders/:id → delete folder │ +│ POST /folders/:id/share → add collaborators │ +│ POST /folders/:id/unshare → remove collaborator │ +│ │ +└─────────────────────────────────────────────────────────┘ +``` + +--- + +## Backend Trace + +### File: `backend/routes/noteRoutes.js` + +### POST /folders (lines 84-115) +- **Auth:** required +- **Input:** `{ name, ownerId, parentId?, type?, projectId?, color? }` +- **Logic:** + 1. Create Folder with provided fields + 2. `ownerId` set to `req.user.uid` + 3. If `parentId`: folder is nested under parent + 4. If `projectId`: folder is scoped to project + 5. `color`: hex color for UI tag (e.g., `#FF5733`) +- **Response:** Created folder document + +### GET /folders (lines 117-137) +- **Auth:** required +- **Logic:** + 1. Find folders where `ownerId = uid` OR `uid` in `collaboratorIds` + 2. Return all folders (client builds tree from `parentId` references) +- **Response:** Array of folder documents + +### POST /folders/:id/share (lines 139-167) +- **Auth:** required +- **Input:** `{ collaboratorIds: string[] }` — array of UIDs to share with +- **Logic:** + 1. Find folder by ID + 2. Verify ownership: `folder.ownerId === req.user.uid` + 3. Merge new collaboratorIds into existing `folder.collaboratorIds` + 4. Save folder +- **Response:** Updated folder with collaborator list + +### PUT /folders/:id (lines 169-187) +- **Auth:** required +- **Input:** `{ name?, color?, parentId? }` +- **Logic:** + 1. Find folder by ID + 2. Verify ownership + 3. Update provided fields + 4. `Folder.findByIdAndUpdate(id, { $set: updates })` +- **Response:** Updated folder + +### DELETE /folders/:id (lines 189-206) +- **Auth:** required +- **Logic:** + 1. Find folder by ID + 2. Verify ownership + 3. Delete folder + 4. Notes in folder: `folderId` set to `null` (notes preserved, unfiled) + 5. Child folders: also deleted (cascade) or moved to parent +- **Response:** `{ message: "Folder deleted" }` + +### POST /folders/:id/unshare (lines 208-233) +- **Auth:** required +- **Input:** `{ userId }` — UID to remove from collaborators +- **Logic:** + 1. Find folder by ID + 2. Verify ownership + 3. Remove `userId` from `folder.collaboratorIds` + 4. Save folder +- **Response:** Updated folder + +--- + +## Frontend Trace + +### FolderTree Component +**File:** `src/components/notes/FolderTree.tsx` +- Recursive component that renders nested folders +- Props: `folders`, `parentId`, `level` (for indentation) +- State: expanded/collapsed per folder +- Click folder: filters note list by `folderId` +- Context menu: Rename, Delete, Share, Change color + +### CreateFolderDialog +**File:** `src/components/notes/CreateFolderDialog.tsx` +- Modal with: name input, color picker, parent folder dropdown +- On submit: `POST /api/notes/folders` + +### ShareFolderDialog +**File:** `src/components/notes/ShareFolderDialog.tsx` +- User search input (uses `/api/users/search`) +- Multi-select collaborators +- On submit: `POST /api/notes/folders/:id/share` with `collaboratorIds` + +--- + +## Database Layer + +### Folder Model +**File:** `backend/models/Folder.js` + +| Field | Type | Required | Index | Notes | +|---|---|---|---|---| +| `name` | String | yes | — | Display name | +| `ownerId` | String | yes | yes | Firebase UID | +| `parentId` | ObjectId | no | — | Parent folder (null = root) | +| `type` | String | no | — | Category tag | +| `projectId` | ObjectId | no | — | Project-scoped | +| `color` | String | no | — | Hex color (e.g., `#FF5733`) | +| `collaboratorIds` | String[] | no | — | Shared UIDs | +| `createdAt` | Date | auto | — | | + +### Folder Hierarchy +- **Root folders:** `parentId = null` or `parentId` not set +- **Nested folders:** `parentId` references parent `Folder._id` +- **Tree depth:** No enforced limit (client renders recursively) +- **Deletion:** When parent folder is deleted, child folders are either: + - Cascaded (deleted with parent) + - Or moved to grandparent (parentId = parent.parentId) + +### Note → Folder Relationship +- `Note.folderId` references `Folder._id` +- When folder is deleted: notes' `folderId` set to `null` (unfiled) +- Notes are never deleted when folder is deleted + +--- + +## Sharing Model + +``` +Folder owner: + ├─ Full control: rename, delete, share, unshare + └─ Can add/remove collaborators + +Collaborator: + ├─ Can view folder + notes inside + ├─ Can edit notes in shared folder + └─ Cannot rename/delete folder + └─ Cannot re-share folder +``` + +| Action | Owner | Collaborator | +|---|---|---| +| View folder + notes | Yes | Yes | +| Edit notes in folder | Yes | Yes | +| Rename folder | Yes | No | +| Delete folder | Yes | No | +| Share with others | Yes | No | +| Unshare from someone | Yes | No | +| Create sub-folder | Yes | No | + +--- + +## Error Paths + +| Scenario | HTTP Status | Response | +|---|---|---| +| No token | 401 | Unauthorized | +| Folder not found | 404 | `{ error: "Folder not found" }` | +| Not owner (delete/update/share) | 403 | `{ error: "Unauthorized" }` | +| Server error | 500 | `{ error: error.message }` | + +--- + +## Cross-References + +- [17-notes-system.md](./17-notes-system.md) — Notes that live inside folders +- [05-database-schema-and-models.md](./05-database-schema-and-models.md) — Folder model +- [09-user-profile-management.md](./09-user-profile-management.md) — User search for sharing From add0dd277f44b36b803c84f4d1f08f38b0c5f879 Mon Sep 17 00:00:00 2001 From: Thanmayee Reddy Kotha <190446018+thanmayeereddykotha@users.noreply.github.com> Date: Sun, 16 Aug 2026 15:04:22 +0530 Subject: [PATCH 5/5] =?UTF-8?q?docs:=20add=2019-realtime-notes-collaborati?= =?UTF-8?q?on=20+=2020-notes-socket-handler=20=E2=80=94=20Yjs=20CRDT=20syn?= =?UTF-8?q?c,=20presence=20map=20lifecycle,=20dual=20event=20naming,=20sta?= =?UTF-8?q?le=20cleanup?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../19-realtime-notes-collaboration.md | 253 ++++++++++++++++++ docs/features/20-notes-socket-handler.md | 207 ++++++++++++++ 2 files changed, 460 insertions(+) create mode 100644 docs/features/19-realtime-notes-collaboration.md create mode 100644 docs/features/20-notes-socket-handler.md diff --git a/docs/features/19-realtime-notes-collaboration.md b/docs/features/19-realtime-notes-collaboration.md new file mode 100644 index 00000000..0494cde4 --- /dev/null +++ b/docs/features/19-realtime-notes-collaboration.md @@ -0,0 +1,253 @@ +# 19 — Real-Time Notes Collaboration + +**NEW document** — Yjs CRDT sync, cursor tracking, presence avatars, awareness updates, stale user cleanup + +--- + +## Feature Summary + +Real-time collaborative note editing uses Yjs (CRDT-based) over Socket.IO. Multiple users can edit the same note simultaneously with live cursor positions, presence avatars, and conflict-free text merging. The `/notes` namespace handles join/leave, cursor movement, document updates, and Yjs awareness state. + +--- + +## Architecture Diagram + +``` +┌─────────────────── CLIENT ─────────────────────────────┐ +│ │ +│ NoteEditor.tsx │ +│ ├─ Yjs document (Y.Doc) │ +│ ├─ y-websocket provider → Socket.IO /notes │ +│ ├─ TipTap/ProseMirror editor bound to Yjs │ +│ ├─ CollaborationCursor extension (colored cursors) │ +│ └─ Presence avatars bar (top of editor) │ +│ │ +│ Events emitted: │ +│ ├─ join_note { noteId, userId, userName, userAvatar, │ +│ │ userColor } │ +│ ├─ cursor_move { noteId, userId, blockId } │ +│ ├─ note-update { noteId, update: Uint8Array } │ +│ ├─ awareness-update { noteId, update: Uint8Array } │ +│ └─ leave_note { noteId, userId } │ +│ │ +│ Events received: │ +│ ├─ presence_update → [user objects] │ +│ ├─ cursor_update { userId, blockId } │ +│ ├─ note-update → apply Yjs update │ +│ ├─ awareness-update → apply Yjs awareness │ +│ └─ user_left (userId) │ +│ │ +└──────────────────────┬──────────────────────────────────┘ + │ Socket.IO /notes + ▼ +┌─────────────────── BACKEND ────────────────────────────┐ +│ │ +│ backend/sockets/noteSocketHandler.js (300 lines) │ +│ │ +│ In-memory state: notePresence = Map> │ +│ │ +│ Events handled: │ +│ ├─ join_note → add user, broadcast presence │ +│ ├─ join-note (legacy) → join room, emit yjs sync │ +│ ├─ presence-join → alternate join event │ +│ ├─ cursor_move → update blockId, broadcast │ +│ ├─ presence-cursor → alternate cursor event │ +│ ├─ note-update → forward to other clients in room │ +│ ├─ awareness-update → forward Yjs awareness │ +│ ├─ leave_note → remove user, broadcast, cleanup │ +│ ├─ leave-note (legacy) → leave room │ +│ ├─ presence-leave → alternate leave event │ +│ └─ disconnect → remove user, broadcast, cleanup │ +│ │ +│ Stale user cleanup: │ +│ └─ setInterval(30s) → remove users inactive >2 min │ +│ │ +└─────────────────────────────────────────────────────────┘ +``` + +--- + +## Backend Trace + +### File: `backend/sockets/noteSocketHandler.js` (300 lines) + +### In-Memory State (line 82) +```js +const notePresence = new Map(); +``` +- **Key:** `noteId` (MongoDB ObjectId as string) +- **Value:** `Map` +- **Lifecycle:** Created on first join, deleted when empty + +### broadcastPresence Helper (lines 85-97) +```js +const broadcastPresence = (noteId) => { + if (!notePresence.has(noteId)) return; + const users = notePresence.get(noteId); + const userList = Array.from(users.values()); + notesNamespace.to(noteId).emit('presence_update', userList); +}; +``` +- Converts Map to array for JSON serialization +- Emits to all sockets in the note's room + +### Connection Handler (lines 99-265) + +#### join_note Event (lines 103-133) +- **Payload:** `{ noteId, userId, userName, userAvatar, userColor }` +- **Logic:** + 1. `socket.join(noteId)` — join Socket.IO room + 2. Store `noteId` and `userId` on socket for disconnect cleanup + 3. Initialize `notePresence` Map for noteId if needed + 4. Add user: `{ id, name, avatarUrl, color, blockId: null, lastActive: Date.now() }` + 5. `broadcastPresence(noteId)` — notify all users + +#### join-note Event (lines 136-140) — Legacy +- **Payload:** `noteId` (string, not object) +- **Logic:** Join room + emit `user-joined-yjs` to trigger Yjs sync +- **Purpose:** Compatibility with older y-websocket clients + +#### presence-join Event (lines 142-160) — Alternate +- **Payload:** `{ noteId, odId, displayName, photoURL, color }` +- Same as `join_note` but with different field names +- **Purpose:** Different client integration (e.g., mobile app) + +#### cursor_move Event (lines 163-189) +- **Payload:** `{ noteId, userId, blockId }` +- **Logic:** + 1. Find user in `notePresence` map + 2. Update `user.blockId = blockId` + 3. Update `user.lastActive = Date.now()` + 4. `broadcastPresence(noteId)` — full presence update + 5. `notesNamespace.to(noteId).emit('cursor_update', { userId, blockId })` — targeted cursor event + +#### presence-cursor Event (lines 192-202) — Alternate +- Same as `cursor_move` with different field names + +#### leave_note Event (lines 205-220) +- **Payload:** `{ noteId, userId }` +- **Logic:** + 1. `socket.leave(noteId)` — leave Socket.IO room + 2. Remove user from `notePresence` map + 3. `broadcastPresence(noteId)` — update UI + 4. `notesNamespace.to(noteId).emit('user_left', userId)` — explicit event + 5. If map empty: `notePresence.delete(noteId)` — free memory + +#### leave-note Event (lines 223-225) — Legacy +- Just `socket.leave(noteId)` + +#### presence-leave Event (lines 227-234) — Alternate +- Remove user + broadcast + emit `user_left` + +#### note-update Event (lines 237-239) +- **Payload:** `{ noteId, update }` — Yjs binary update (Uint8Array) +- **Logic:** `socket.to(noteId).emit('note-update', update)` — forward to all other clients +- **No persistence:** Server is a dumb relay for Yjs updates + +#### awareness-update Event (lines 241-243) +- **Payload:** `{ noteId, update }` — Yjs awareness state (selections, cursor info) +- **Logic:** `socket.to(noteId).emit('awareness-update', update)` — forward to others + +#### disconnect Event (lines 246-264) +- **Logic:** + 1. Get `noteId` and `odId` from socket properties + 2. Remove user from `notePresence` map + 3. `broadcastPresence(noteId)` — update UI + 4. `notesNamespace.to(noteId).emit('user_left', odId)` — explicit event + 5. If map empty: delete from `notePresence` + +### Stale User Cleanup (lines 268-298) +```js +const cleanupInterval = setInterval(() => { + const now = Date.now(); + const staleThreshold = 120000; // 2 minutes + + for (const [noteId, users] of notePresence.entries()) { + let hasStaleUsers = false; + + for (const [odId, user] of users.entries()) { + if (now - user.lastActive > staleThreshold) { + users.delete(odId); + hasStaleUsers = true; + } + } + + if (hasStaleUsers) broadcastPresence(noteId); + if (users.size === 0) notePresence.delete(noteId); + } +}, 30000); // Every 30 seconds + +cleanupInterval.unref(); // Don't keep Node process alive for this +``` +- **Stale threshold:** 2 minutes of inactivity +- **Check interval:** 30 seconds +- **unref():** Prevents the interval from blocking process exit + +--- + +## Socket Events Reference + +### Client → Server + +| Event | Payload | Purpose | +|---|---|---| +| `join_note` | `{ noteId, userId, userName, userAvatar, userColor }` | Join note for collaboration | +| `join-note` | `noteId` (string) | Legacy Yjs join | +| `presence-join` | `{ noteId, odId, displayName, photoURL, color }` | Alternate join | +| `cursor_move` | `{ noteId, userId, blockId }` | Update cursor position | +| `presence-cursor` | `{ noteId, odId, blockId }` | Alternate cursor update | +| `note-update` | `{ noteId, update: Uint8Array }` | Yjs document update | +| `awareness-update` | `{ noteId, update: Uint8Array }` | Yjs awareness state | +| `leave_note` | `{ noteId, userId }` | Leave note collaboration | +| `leave-note` | `noteId` (string) | Legacy leave | +| `presence-leave` | `{ noteId, odId }` | Alternate leave | + +### Server → Client + +| Event | Payload | Purpose | +|---|---|---| +| `presence_update` | `[{ id, name, avatarUrl, color, blockId, lastActive }]` | Full presence list | +| `cursor_update` | `{ userId, blockId }` | Targeted cursor update | +| `note-update` | `Uint8Array` | Yjs document update (forwarded) | +| `awareness-update` | `Uint8Array` | Yjs awareness (forwarded) | +| `user_left` | `userId` (string) | User disconnected/left | +| `user-joined-yjs` | — | Trigger Yjs sync (legacy) | + +--- + +## Yjs CRDT Sync Model + +``` +Client A edits → Y.Doc update → Uint8Array + → socket.emit('note-update', { noteId, update }) + → Server forwards to all other clients in room + → Client B receives → Y.applyUpdate(yDoc, update) + → ProseMirror re-renders + +Conflict resolution: CRDT (Conflict-free Replicated Data Type) + → Yjs automatically merges concurrent edits + → No operational transform needed + → Server is a dumb relay (no merge logic) +``` + +--- + +## Error Paths + +| Scenario | Handling | +|---|---| +| No `noteId` in presence map | `logger.warn()` + return (no crash) | +| User not in presence map on cursor_move | `logger.warn()` + return | +| Socket disconnect without join | `noteId`/`odId` undefined → skip cleanup | +| Stale user (no activity >2min) | Cleanup interval removes + broadcasts | + +--- + +## Cross-References + +- [17-notes-system.md](./17-notes-system.md) — Notes CRUD (REST API) +- [20-notes-socket-handler.md](./20-notes-socket-handler.md) — Socket handler deep dive +- [06-middleware-stack.md](./06-middleware-stack.md) — Socket.IO setup in index.js +- [11-presence-system.md](./11-presence-system.md) — User presence (separate from note presence) diff --git a/docs/features/20-notes-socket-handler.md b/docs/features/20-notes-socket-handler.md new file mode 100644 index 00000000..e6597189 --- /dev/null +++ b/docs/features/20-notes-socket-handler.md @@ -0,0 +1,207 @@ +# 20 — Notes Socket Handler + +**NEW document** — Deep dive into noteSocketHandler.js: dual event naming, presence map lifecycle, memory management, cleanup interval + +--- + +## Feature Summary + +This document is a deep technical dive into `backend/sockets/noteSocketHandler.js`, covering the dual event naming convention (primary + legacy + alternate), in-memory `notePresence` Map lifecycle, memory management strategies, and the stale user cleanup interval. + +--- + +## File Overview + +**File:** `backend/sockets/noteSocketHandler.js` +**Lines:** 300 +**Namespace:** `/notes` +**Registration:** `backend/index.js` → `require('./sockets/noteSocketHandler')(io)` + +--- + +## Dual Event Naming Convention + +The handler supports three parallel event naming patterns for compatibility with different client integrations: + +| Primary Event | Legacy Event | Alternate Event | Purpose | +|---|---|---|---| +| `join_note` | `join-note` | `presence-join` | Join a note room | +| `leave_note` | `leave-note` | `presence-leave` | Leave a note room | +| `cursor_move` | — | `presence-cursor` | Update cursor position | + +### Primary Events (`join_note`, `leave_note`, `cursor_move`) +- Used by the main web client (React + TipTap) +- Payload: `{ noteId, userId, userName, userAvatar, userColor }` +- Full user profile data sent on join + +### Legacy Events (`join-note`, `leave-note`) +- Used by older y-websocket integration +- Payload: `noteId` (bare string, not object) +- `join-note` also emits `user-joined-yjs` to trigger Yjs sync +- Minimal data — no user profile + +### Alternate Events (`presence-join`, `presence-leave`, `presence-cursor`) +- Used by mobile/alternate client integration +- Payload: `{ noteId, odId, displayName, photoURL, color }` +- Different field names (`odId` vs `userId`, `displayName` vs `userName`) + +--- + +## In-Memory Presence Map + +### Data Structure +``` +notePresence: Map> +``` + +### Lifecycle + +``` +1. CREATE: First user joins note + notePresence.set(noteId, new Map()) + notePresence.get(noteId).set(userId, { ... }) + +2. GROW: More users join + notePresence.get(noteId).set(userId2, { ... }) + +3. UPDATE: Cursor movement + notePresence.get(noteId).get(userId).blockId = newBlockId + notePresence.get(noteId).get(userId).lastActive = Date.now() + +4. SHRINK: Users leave + notePresence.get(noteId).delete(userId) + +5. DESTROY: Last user leaves + if (users.size === 0) notePresence.delete(noteId) +``` + +### Memory Management +- **Per-note cleanup:** When all users leave a note, the inner Map is deleted from `notePresence` +- **Stale user cleanup:** `setInterval(30s)` removes users inactive >2 minutes +- **Disconnect cleanup:** `socket.disconnect` event removes user from map +- **unref():** Cleanup interval does not prevent Node process exit + +--- + +## Stale User Cleanup Algorithm + +``` +Every 30 seconds: + IF notePresence is empty → return (skip) + + FOR each (noteId, users) in notePresence: + hasStaleUsers = false + + FOR each (userId, user) in users: + IF (now - user.lastActive > 120000): // 2 minutes + users.delete(userId) + hasStaleUsers = true + + IF hasStaleUsers: + broadcastPresence(noteId) // Update UI + + IF users.size === 0: + notePresence.delete(noteId) // Free memory +``` + +### Why 2 Minutes? +- Network blips: <30s (handled by reconnect) +- Tab switch: ~30s-1min (user still active) +- Phone locked: ~1-2min (user inactive) +- 2 minutes is the sweet spot: catches truly inactive users without removing temporarily distracted ones + +### Why 30 Second Interval? +- Balance between UI freshness and CPU usage +- At 30s intervals, stale users are removed within 30-60s of becoming stale +- On a server with 100 active notes, this iterates 100 maps every 30s — negligible CPU + +--- + +## Event Flow Diagrams + +### Join Flow +``` +Client Server Other Clients + │ │ │ + │ join_note {noteId,...} │ │ + │────────────────────────►│ │ + │ │ socket.join(noteId) │ + │ │ notePresence.set(...) │ + │ │ broadcastPresence() │ + │ │────────────────────────►│ + │ │ presence_update │ + │ │ [user list] │ + │ │ │ + │ │◄────────────────────────│ + │ presence_update │ (other users too) │ + │ [user list] │ │ + │◄────────────────────────│ │ +``` + +### Edit Flow (Yjs) +``` +Client A Server Client B + │ │ │ + │ note-update { │ │ + │ noteId, │ │ + │ update: Uint8Array│ │ + │ } │ │ + │────────────────────►│ │ + │ │ socket.to(noteId) │ + │ │ .emit('note-update',│ + │ │ update) │ + │ │───────────────────►│ + │ │ │ Y.applyUpdate() + │ │ │ ProseMirror renders +``` + +### Disconnect Flow +``` +Client Server Other Clients + │ │ │ + │ (browser closes) │ │ + │──────────────────────►│ │ + │ disconnect │ │ + │ │ Get noteId, odId │ + │ │ from socket props │ + │ │ users.delete(odId) │ + │ │ broadcastPresence() │ + │ │──────────────────────►│ + │ │ presence_update │ + │ │ user_left(odId) │ + │ │ │ + │ │ IF users.size === 0: │ + │ │ notePresence.delete │ +``` + +--- + +## Server as Dumb Relay + +The server does **not**: +- Parse or interpret Yjs updates +- Merge concurrent edits (CRDT handles this client-side) +- Persist document state (MongoDB stores last-saved version via REST API) +- Validate edit permissions (handled by REST API on save) + +The server **does**: +- Track presence (who is in which note) +- Forward Yjs binary updates between clients +- Forward Yjs awareness state (cursor positions, selections) +- Clean up stale connections + +--- + +## Cross-References + +- [19-realtime-notes-collaboration.md](./19-realtime-notes-collaboration.md) — High-level collaboration overview +- [17-notes-system.md](./17-notes-system.md) — REST API for note CRUD +- [11-presence-system.md](./11-presence-system.md) — Global user presence (separate from note presence) +- [06-middleware-stack.md](./06-middleware-stack.md) — Socket.IO initialization