diff --git a/docs/features/51-middleware-stack-overview.md b/docs/features/51-middleware-stack-overview.md new file mode 100644 index 00000000..d2d2fc68 --- /dev/null +++ b/docs/features/51-middleware-stack-overview.md @@ -0,0 +1,141 @@ +# 51 — Middleware Stack Overview + +**NEW document** — Express middleware chain, auth middleware, CORS, rate limiting, raw body parsing for webhooks + +--- + +## Feature Summary + +Zync's Express middleware stack handles authentication (Firebase JWT verification), CORS, request parsing, rate limiting, and webhook signature verification. Middleware is applied globally and per-route, with special raw body parsing for GitHub webhooks. + +--- + +## Architecture Diagram + +``` +┌─────────────────── EXPRESS APP ─────────────────────────┐ +│ │ +│ Global Middleware (applied to all routes): │ +│ ┌─────────────────────────────────────────────────┐ │ +│ │ 1. cors({ origin: FRONTEND_URL, credentials }) │ │ +│ │ 2. express.json({ limit: '10mb }) │ │ +│ │ 3. express.urlencoded({ extended: true }) │ │ +│ │ 4. cookieParser() │ │ +│ │ 5. helmet() (security headers) │ │ +│ │ 6. rateLimit({ windowMs, max }) │ │ +│ └─────────────────────────────────────────────────┘ │ +│ │ +│ Per-Route Middleware: │ +│ ┌─────────────────────────────────────────────────┐ │ +│ │ verifyToken (authMiddleware) │ │ +│ │ → Verifies Firebase JWT from Authorization │ │ +│ │ → Sets req.user = { uid, email, ... } │ │ +│ │ → 401 if invalid/missing │ │ +│ ├─────────────────────────────────────────────────┤ │ +│ │ verifyGithub │ │ +│ │ → HMAC SHA-256 webhook verification │ │ +│ │ → Uses WEBHOOK_SECRET │ │ +│ │ → 401 if signature mismatch │ │ +│ ├─────────────────────────────────────────────────┤ │ +│ │ requireDb (chatRoutes only) │ │ +│ │ → Checks mongoose.connection.readyState === 1 │ │ +│ │ → 503 if DB disconnected │ │ +│ ├─────────────────────────────────────────────────┤ │ +│ │ multer (upload routes) │ │ +│ │ → Parses multipart/form-data │ │ +│ │ → Memory storage, 10MB limit │ │ +│ └─────────────────────────────────────────────────┘ │ +│ │ +│ Webhook Raw Body Parsing: │ +│ ┌─────────────────────────────────────────────────┐ │ +│ │ express.json({ verify: (req) => { │ │ +│ │ return req.path.includes('/webhook'); │ │ +│ │ }}) │ │ +│ │ → Stores raw body on req.rawBody for HMAC │ │ +│ └─────────────────────────────────────────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────┘ +``` + +--- + +## Middleware Details + +### authMiddleware (verifyToken) +**File:** `backend/middleware/authMiddleware.js` + +```js +const admin = require('firebase-admin'); + +const verifyToken = async (req, res, next) => { + const authHeader = req.headers.authorization; + if (!authHeader || !authHeader.startsWith('Bearer ')) { + return res.status(401).json({ error: 'Unauthorized' }); + } + const token = authHeader.split(' ')[1]; + try { + const decoded = await admin.auth().verifyIdToken(token); + req.user = { uid: decoded.uid, email: decoded.email }; + next(); + } catch (error) { + return res.status(401).json({ error: 'Invalid token' }); + } +}; +``` + +- **Firebase Admin SDK:** Verifies JWT against Firebase +- **req.user:** Sets `uid` and `email` for downstream handlers +- **Bearer token:** Extracted from `Authorization` header + +### verifyGithub +**File:** `backend/middleware/verifyGithub.js` +- HMAC SHA-256 verification using `WEBHOOK_SECRET` +- Timing-safe comparison to prevent timing attacks +- Detailed in [22-github-webhook-handler.md](./22-github-webhook-handler.md) + +### CORS Configuration +```js +const cors = require('cors'); +app.use(cors({ + origin: process.env.FRONTEND_URL || 'http://localhost:3000', + credentials: true, +})); +``` +- Single origin (not wildcard) for security +- Credentials enabled for cookies/auth + +### Rate Limiting +```js +const rateLimit = require('express-rate-limit'); +const limiter = rateLimit({ + windowMs: 15 * 60 * 1000, // 15 minutes + max: 100, // 100 requests per window per IP + message: { error: 'Too many requests' }, +}); +app.use('/api/', limiter); +``` +- Applied to all `/api/` routes +- 100 requests per 15 minutes per IP +- Webhook routes exempt (GitHub needs fast response) + +--- + +## Error Paths + +| Scenario | HTTP Status | Response | +|---|---|---| +| No Authorization header | 401 | `{ error: "Unauthorized" }` | +| Invalid JWT | 401 | `{ error: "Invalid token" }` | +| Expired JWT | 401 | `{ error: "Invalid token" }` | +| Rate limited | 429 | `{ error: "Too many requests" }` | +| DB disconnected (requireDb) | 503 | `{ error: "Database not available" }` | +| Webhook signature invalid | 401 | Unauthorized | + +--- + +## Cross-References + +- [08-firebase-auth-flow.md](./08-firebase-auth-flow.md) — Firebase JWT verification +- [22-github-webhook-handler.md](./22-github-webhook-handler.md) — HMAC verification +- [02-security-auth-architecture.md](./02-security-auth-architecture.md) — Security overview +- [50-socket-io-initialization.md](./50-socket-io-initialization.md) — Socket.IO setup diff --git a/docs/features/52-database-schema-models.md b/docs/features/52-database-schema-models.md new file mode 100644 index 00000000..1fde9cfa --- /dev/null +++ b/docs/features/52-database-schema-models.md @@ -0,0 +1,140 @@ +# 52 — Database Schema & Models + +**NEW document** — Mongoose models overview, schema definitions, indexes, relationships, dual ORM strategy + +--- + +## Feature Summary + +Zync uses MongoDB with Mongoose as the primary ODM. All models are defined in `backend/models/` with explicit schemas, indexes, and validation. The database stores users, projects, tasks, steps, notes, folders, messages, teams, sessions, meetings, activities, and collaborators. + +--- + +## Model Inventory + +| Model | File | Collection | Purpose | +|---|---|---|---| +| User | `User.js` | users | User profiles, integrations, settings | +| Project | `Project.js` | projects | Project metadata, GitHub repo link | +| Step | `Step.js` | steps | Kanban pipeline stages | +| ProjectTask | `ProjectTask.js` | projecttasks | Tasks within project steps | +| Note | `Note.js` | notes | Rich text notes | +| Folder | `Folder.js` | folders | Note organization, sharing | +| Message | `Message.js` | messages | Chat messages | +| Team | `Team.js` | teams | Team groups, roles, invites | +| Activity | `Activity.js` | activities | Team activity logs | +| Session | `Session.js` | sessions | Work/meeting sessions | +| Meeting | `Meeting.js` | meetings | Google Meet meetings | +| Collaborator | `Collaborator.js` | collaborators | Beta applications | + +--- + +## Key Schema Details + +### User Model +``` +{ + uid: String (Firebase UID, primary key), + email: String (unique, indexed), + displayName: String, + photoURL: String, + bio: String, + location: { city, country, lat, lng, timezone, manual }, + githubIntegration: { connected, accessToken (encrypted), username, installationId }, + googleIntegration: { connected, accessToken (encrypted), refreshToken, expiryDate }, + linkedinIntegration: { connected, profileUrl }, + securityPin: String (hashed), + createdAt, updatedAt +} +``` +**Indexes:** `email` (unique), `uid` (unique), `displayName` (text) + +### Project Model +``` +{ + name: String, + description: String, + ownerUid: String (indexed), + team: [String] (member UIDs), + githubRepoId: Number, + githubRepoName: String, + githubRepoOwner: String, + githubDefaultBranch: String, + steps: [ObjectId] (ref: Step), + createdAt, updatedAt +} +``` +**Indexes:** `ownerUid`, `team` + +### Message Model +``` +{ + chatId: String (format: uidA_uidB, indexed), + senderId: String (indexed), + receiverId: String (indexed), + text: String, + type: String (text/image/file), + fileUrl, fileName, fileSize, + senderName, senderPhotoURL, + projectId, projectName, projectOwnerId, + delivered: Boolean, + deliveredAt: Date, + seen: Boolean, + seenAt: Date, + createdAt +} +``` +**Indexes:** `{ chatId: 1, createdAt: 1 }`, `{ receiverId: 1, seen: 1 }`, `{ receiverId: 1, delivered: 1 }` + +### Note Model +``` +{ + title: String, + content: String (rich text HTML), + ownerId: String (indexed), + folderId: ObjectId (ref: Folder), + projectId: ObjectId (ref: Project), + sharedWith: [String], + isPublic: Boolean, + createdAt, updatedAt +} +``` + +### Team Model +``` +{ + name: String, + ownerUid: String (indexed), + admins: [String], + members: [String], + pendingMembers: [String], + inviteCode: String (unique), + type: String, + pin: String (hashed), + createdAt +} +``` + +--- + +## Index Strategy + +### Compound Indexes +- Message: `{ chatId: 1, createdAt: 1 }` — efficient history queries +- Message: `{ receiverId: 1, seen: 1 }` — unread count queries +- Message: `{ receiverId: 1, delivered: 1 }` — delivery catch-up queries + +### Text Indexes +- User: `{ displayName: 'text', email: 'text' }` — user search + +### Unique Indexes +- User: `email`, `uid` +- Team: `inviteCode` + +--- + +## Cross-References + +- [03-performance-caching-strategy.md](./03-performance-caching-strategy.md) — Redis caching on top of MongoDB +- [05-database-schema-and-models.md](./05-database-schema-and-models.md) — Original schema doc +- All feature docs reference their respective models diff --git a/docs/features/53-frontend-routing-layout.md b/docs/features/53-frontend-routing-layout.md new file mode 100644 index 00000000..fbd248a9 --- /dev/null +++ b/docs/features/53-frontend-routing-layout.md @@ -0,0 +1,134 @@ +# 53 — Frontend Routing & Layout + +**NEW document** — React Router structure, lazy loading, protected routes, layout components, navigation + +--- + +## Feature Summary + +The Zync frontend uses React Router DOM for client-side routing. Routes are organized into public (login, register), protected (dashboard, projects, settings), and modal-based routes. Layout components provide the app shell with sidebar navigation, top bar, and content area. + +--- + +## Architecture Diagram + +``` +┌─────────────────── REACT ROUTER ────────────────────────┐ +│ │ +│ App.tsx │ +│ └─ │ +│ └─ │ +│ ├─ / (Public) │ +│ │ ├─ /login → Login.tsx │ +│ │ ├─ /register → Register.tsx │ +│ │ └─ /beta → BetaApplication.tsx │ +│ │ │ +│ ├─ / (Protected, wrapped in ProtectedRoute) │ +│ │ └─ AppLayout.tsx │ +│ │ ├─ Sidebar (navigation) │ +│ │ ├─ TopBar (search, notifications) │ +│ │ └─ │ +│ │ ├─ /dashboard → DashboardHome.tsx │ +│ │ ├─ /projects → ProjectsView.tsx │ +│ │ ├─ /projects/:id → ProjectWorkspace │ +│ │ ├─ /messages → MessagesPage.tsx │ +│ │ ├─ /notes → NotesView.tsx │ +│ │ ├─ /teams → TeamsView.tsx │ +│ │ ├─ /meetings → MeetingsView.tsx │ +│ │ ├─ /settings → SettingsView.tsx │ +│ │ └─ /calendar → CalendarView.tsx │ +│ │ │ +│ └─ * → NotFound.tsx │ +│ │ +└─────────────────────────────────────────────────────────┘ +``` + +--- + +## Route Protection + +### ProtectedRoute Component +```tsx +const ProtectedRoute = ({ children }) => { + const { user, loading } = useAuth(); + if (loading) return ; + if (!user) return ; + return children; +}; +``` +- Checks Firebase Auth state +- Redirects to `/login` if not authenticated +- Shows spinner during auth state loading + +### Auth Context +**File:** `src/context/AuthContext.tsx` +- Uses `onAuthStateChanged` from Firebase Auth +- Provides `{ user, loading, login, logout }` +- All protected routes consume this context + +--- + +## Lazy Loading + +```tsx +const DashboardHome = lazy(() => import('./views/DashboardHome')); +const ProjectWorkspace = lazy(() => import('./views/ProjectWorkspace')); +const MessagesPage = lazy(() => import('./views/MessagesPage')); + +// Wrapped in Suspense: +}> + + } /> + ... + + +``` +- Reduces initial bundle size +- Each view loaded on demand +- Suspense fallback shows loading spinner + +--- + +## Layout Components + +### AppLayout +**File:** `src/components/layout/AppLayout.tsx` +- App shell with sidebar + topbar + content area +- Uses `` for nested routes +- Responsive: sidebar collapses on mobile + +### Sidebar +**File:** `src/components/layout/Sidebar.tsx` +- Navigation links: Dashboard, Projects, Messages, Notes, Teams, Meetings, Calendar, Settings +- Active route highlighting +- User avatar at bottom +- Collapsible on mobile (hamburger menu) + +### TopBar +**File:** `src/components/layout/TopBar.tsx` +- Global search bar +- Notifications bell (unread chat count) +- User dropdown menu (profile, settings, logout) + +--- + +## Navigation Flow + +``` +Login → Auth check → Redirect to /dashboard + ↓ +Dashboard → Sidebar navigation + ├─ Projects → Project list → Click project → ProjectWorkspace + ├─ Messages → Conversation list → Click chat → Chat window + ├─ Notes → Note list → Click note → Note editor + ├─ Teams → Team list → Click team → Team detail + └─ Settings → Profile/Integrations/Security tabs +``` + +--- + +## Cross-References + +- [08-firebase-auth-flow.md](./08-firebase-auth-flow.md) — Auth context and protected routes +- [01-frontend-architecture.md](./01-frontend-architecture.md) — Frontend structure overview +- [50-socket-io-initialization.md](./50-socket-io-initialization.md) — Socket context provider diff --git a/docs/features/54-frontend-state-management.md b/docs/features/54-frontend-state-management.md new file mode 100644 index 00000000..5a12d774 --- /dev/null +++ b/docs/features/54-frontend-state-management.md @@ -0,0 +1,173 @@ +# 54 — Frontend State Management + +**NEW document** — TanStack Query, Zustand stores, context providers, optimistic updates, cache invalidation + +--- + +## Feature Summary + +Zync uses TanStack Query (React Query) for server state management and Zustand for client-side UI state. TanStack Query handles data fetching, caching, optimistic updates, and background refetching. Zustand manages UI-only state like theme, sidebar visibility, and active modals. + +--- + +## Architecture Diagram + +``` +┌─────────────────── STATE LAYERS ────────────────────────┐ +│ │ +│ Server State (TanStack Query) │ +│ ├─ QueryClient with global defaults │ +│ │ ├─ staleTime: 30s │ +│ │ ├─ refetchOnWindowFocus: true │ +│ │ └─ retry: 2 │ +│ ├─ Custom hooks: useProjects, useTasks, useNotes │ +│ ├─ Mutations: useCreateProject, useUpdateTask │ +│ └─ Query keys: ['projects'], ['projects', id], etc. │ +│ │ +│ Client State (Zustand) │ +│ ├─ useUIStore: sidebarOpen, theme, activeModal │ +│ ├─ useAuthStore: user, loading (mirror of context) │ +│ └─ useEditorStore: activeNote, isEditing │ +│ │ +│ Context Providers │ +│ ├─ AuthContext: Firebase auth state │ +│ ├─ SocketContext: Socket.IO connections │ +│ └─ ThemeContext: dark/light mode │ +│ │ +└─────────────────────────────────────────────────────────┘ +``` + +--- + +## TanStack Query Setup + +### QueryClient Configuration +**File:** `src/lib/queryClient.ts` +```ts +import { QueryClient } from '@tanstack/react-query'; + +export const queryClient = new QueryClient({ + defaultOptions: { + queries: { + staleTime: 30 * 1000, // 30 seconds + refetchOnWindowFocus: true, + retry: 2, + retryDelay: 1000, + }, + mutations: { + retry: 0, + }, + }, +}); +``` + +### Query Key Convention +``` +['projects'] → list of projects +['projects', projectId] → single project +['projects', projectId, 'tasks'] → tasks in project +['projects', projectId, 'steps'] → steps in project +['conversations'] → chat conversations +['notes', folderId] → notes in folder +['user', 'me'] → current user profile +['github', 'repos'] → GitHub repos +``` + +### Custom Hooks Example +**File:** `src/hooks/useProjects.ts` +```ts +export const useProjects = () => { + return useQuery({ + queryKey: ['projects'], + queryFn: () => api.get('/projects').then(res => res.data), + }); +}; + +export const useCreateProject = () => { + const queryClient = useQueryClient(); + return useMutation({ + mutationFn: (data) => api.post('/projects', data), + onSuccess: () => { + queryClient.invalidateQueries({ queryKey: ['projects'] }); + }, + }); +}; +``` + +### Optimistic Updates +```ts +export const useUpdateTask = () => { + const queryClient = useQueryClient(); + return useMutation({ + mutationFn: (data) => api.put(`/tasks/${data.id}`, data), + onMutate: async (newData) => { + await queryClient.cancelQueries({ queryKey: ['projects', newData.projectId, 'tasks'] }); + const previous = queryClient.getQueryData(['projects', newData.projectId, 'tasks']); + queryClient.setQueryData(['projects', newData.projectId, 'tasks'], (old) => + old.map(t => t.id === newData.id ? { ...t, ...newData } : t) + ); + return { previous }; + }, + onError: (err, newData, context) => { + queryClient.setQueryData(['projects', newData.projectId, 'tasks'], context.previous); + }, + onSettled: (data, error, variables) => { + queryClient.invalidateQueries({ queryKey: ['projects', variables.projectId, 'tasks'] }); + }, + }); +}; +``` + +--- + +## Zustand Stores + +### useUIStore +**File:** `src/stores/uiStore.ts` +```ts +export const useUIStore = create((set) => ({ + sidebarOpen: true, + toggleSidebar: () => set((s) => ({ sidebarOpen: !s.sidebarOpen })), + activeModal: null, + setActiveModal: (modal) => set({ activeModal: modal }), +})); +``` + +### useEditorStore +**File:** `src/stores/editorStore.ts` +```ts +export const useEditorStore = create((set) => ({ + activeNoteId: null, + isEditing: false, + setActiveNote: (id) => set({ activeNoteId: id, isEditing: true }), + clearActiveNote: () => set({ activeNoteId: null, isEditing: false }), +})); +``` + +--- + +## Socket.IO + TanStack Query Integration + +When Socket.IO events arrive, query caches are updated: + +```ts +// In useChatSocket hook +useEffect(() => { + chatSocket.on('new-message', (msg) => { + // Optimistically add to messages query + queryClient.setQueryData(['chat', msg.chatId], (old) => [...old, msg]); + // Invalidate conversations list + queryClient.invalidateQueries({ queryKey: ['conversations'] }); + }); +}, []); +``` + +--- + +## Cross-References + +- [01-frontend-architecture.md](./01-frontend-architecture.md) — Frontend structure +- [53-frontend-routing-layout.md](./53-frontend-routing-layout.md) — Routing +- [50-socket-io-initialization.md](./50-socket-io-initialization.md) — Socket context +- [14-project-crud.md](./14-project-crud.md) — Project hooks +- [16-task-management.md](./16-task-management.md) — Task hooks with optimistic updates diff --git a/docs/features/55-api-client-interceptors.md b/docs/features/55-api-client-interceptors.md new file mode 100644 index 00000000..b8991bf5 --- /dev/null +++ b/docs/features/55-api-client-interceptors.md @@ -0,0 +1,154 @@ +# 55 — API Client & Interceptors + +**NEW document** — Axios instance, request/response interceptors, auth token injection, error normalization + +--- + +## Feature Summary + +The frontend uses a centralized Axios instance with interceptors for automatic JWT injection, error normalization, and 401 handling. All API calls go through this client, ensuring consistent auth headers and error handling across the app. + +--- + +## Architecture Diagram + +``` +┌─────────────────── FRONTEND ────────────────────────────┐ +│ │ +│ src/lib/api.ts (Axios instance) │ +│ │ +│ Request Interceptor: │ +│ ├─ Get Firebase JWT from auth.currentUser │ +│ ├─ Set Authorization: Bearer │ +│ └─ Set Content-Type: application/json │ +│ │ +│ Response Interceptor: │ +│ ├─ On 2xx: return response.data │ +│ ├─ On 401: sign out user, redirect to /login │ +│ ├─ On 429: return rate limit error │ +│ ├─ On 500: return server error │ +│ └─ On network error: return connection error │ +│ │ +│ Base URL: VITE_API_URL || http://localhost:5000 │ +│ Timeout: 30000ms (30s) │ +│ withCredentials: true (for cookies) │ +│ │ +└─────────────────────────────────────────────────────────┘ +``` + +--- + +## Implementation + +### File: `src/lib/api.ts` +```ts +import axios from 'axios'; +import { getAuth } from 'firebase/auth'; + +const api = axios.create({ + baseURL: import.meta.env.VITE_API_URL || 'http://localhost:5000', + timeout: 30000, + withCredentials: true, +}); + +// Request interceptor: inject JWT +api.interceptors.request.use(async (config) => { + const auth = getAuth(); + const user = auth.currentUser; + if (user) { + const token = await user.getIdToken(); + config.headers.Authorization = `Bearer ${token}`; + } + return config; +}); + +// Response interceptor: normalize errors +api.interceptors.response.use( + (response) => response.data, + (error) => { + if (error.response?.status === 401) { + // Token expired or invalid + auth.signOut(); + window.location.href = '/login'; + } + const normalized = { + status: error.response?.status || 0, + message: error.response?.data?.error || error.response?.data?.message || 'Network error', + data: error.response?.data, + }; + return Promise.reject(normalized); + } +); + +export default api; +``` + +--- + +## Usage Pattern + +### In Custom Hooks +```ts +import api from '@/lib/api'; + +// Query +const data = await api.get('/projects'); +// data is already response.data (interceptor strips it) + +// Mutation +const result = await api.post('/projects', { name, description }); + +// Error handling +try { + await api.delete(`/projects/${id}`); +} catch (err) { + // err is normalized: { status, message, data } + showToast(err.message); +} +``` + +### File Uploads +```ts +const formData = new FormData(); +formData.append('file', file); +await api.post('/upload', formData, { + headers: { 'Content-Type': 'multipart/form-data' }, +}); +``` + +--- + +## Token Refresh + +Firebase Auth automatically refreshes expired JWTs: +```ts +// In request interceptor: +const token = await user.getIdToken(); +// getIdToken() automatically refreshes if token is expired +``` +- No manual refresh logic needed +- Firebase handles token lifecycle internally +- Interceptor always gets a fresh token + +--- + +## Error Normalization + +All errors are normalized to a consistent shape: +```ts +{ + status: number, // HTTP status code (0 for network errors) + message: string, // Human-readable error message + data: any, // Original response data (if any) +} +``` + +This allows UI components to always access `err.message` without checking error shape. + +--- + +## Cross-References + +- [08-firebase-auth-flow.md](./08-firebase-auth-flow.md) — Firebase Auth token management +- [54-frontend-state-management.md](./54-frontend-state-management.md) — TanStack Query uses this client +- [51-middleware-stack-overview.md](./51-middleware-stack-overview.md) — Backend auth middleware diff --git a/docs/features/56-environment-variables-reference.md b/docs/features/56-environment-variables-reference.md new file mode 100644 index 00000000..a133a53d --- /dev/null +++ b/docs/features/56-environment-variables-reference.md @@ -0,0 +1,188 @@ +# 56 — Environment Variables Reference + +**NEW document** — Complete environment variable catalog, required vs optional, defaults, descriptions + +--- + +## Feature Summary + +This document is a comprehensive reference for all environment variables used across the Zync backend and frontend. Variables are organized by category: Firebase, Database, Redis, GitHub, Google, Cloudinary, SMTP, AI Gateway, Security, and Frontend. + +--- + +## Complete Environment Variable Catalog + +### Firebase +| Variable | Required | Default | Description | +|---|---|---|---| +| `FIREBASE_PROJECT_ID` | Yes | — | Firebase project ID | +| `FIREBASE_CLIENT_EMAIL` | Yes | — | Firebase service account email | +| `FIREBASE_PRIVATE_KEY` | Yes | — | Firebase private key (PEM) | +| `FIREBASE_API_KEY` | Yes | — | Firebase Web API key | +| `FIREBASE_AUTH_DOMAIN` | Yes | — | Firebase auth domain | +| `FIREBASE_STORAGE_BUCKET` | No | — | Firebase storage bucket | + +### Database (MongoDB) +| Variable | Required | Default | Description | +|---|---|---|---| +| `MONGODB_URI` | Yes | — | MongoDB connection string | + +### Redis +| Variable | Required | Default | Description | +|---|---|---|---| +| `REDIS_URL` | Yes (prod) | — | Redis connection URL | +| `REDIS_HOST` | No | localhost | Redis host (alternative to URL) | +| `REDIS_PORT` | No | 6379 | Redis port | + +### GitHub +| Variable | Required | Default | Description | +|---|---|---|---| +| `GITHUB_CLIENT_ID` | Yes | — | OAuth app client ID | +| `GITHUB_CLIENT_SECRET` | Yes | — | OAuth app client secret | +| `GITHUB_APP_ID` | Yes | — | GitHub App ID | +| `GITHUB_PRIVATE_KEY` | Yes | — | GitHub App private key (PEM) | +| `GITHUB_APP_REDIRECT_URI` | No | — | OAuth callback URL | +| `WEBHOOK_SECRET` | Yes | — | HMAC secret for webhook verification | + +### Google +| Variable | Required | Default | Description | +|---|---|---|---| +| `GOOGLE_CLIENT_ID` | Yes | — | Google OAuth client ID | +| `GOOGLE_CLIENT_SECRET` | Yes | — | Google OAuth client secret | +| `GOOGLE_REDIRECT_URI` | Yes | — | OAuth callback URL | + +### LinkedIn +| Variable | Required | Default | Description | +|---|---|---|---| +| `LINKEDIN_CLIENT_ID` | Yes | — | LinkedIn OAuth client ID | +| `LINKEDIN_CLIENT_SECRET` | Yes | — | LinkedIn OAuth client secret | +| `LINKEDIN_REDIRECT_URI` | Yes | — | OAuth callback URL | + +### Cloudinary +| Variable | Required | Default | Description | +|---|---|---|---| +| `CLOUDINARY_CLOUD_NAME` | Yes | — | Cloudinary cloud name | +| `CLOUDINARY_API_KEY` | Yes | — | Cloudinary API key | +| `CLOUDINARY_API_SECRET` | Yes | — | Cloudinary API secret | + +### SMTP / Email +| Variable | Required | Default | Description | +|---|---|---|---| +| `SMTP_USER` | Yes | — | Gmail/SMTP email address | +| `SMTP_PASS` | Yes | — | Gmail app password | +| `SMTP_SERVICE` | No | gmail | SMTP service name | +| `SUPPORT_EMAIL` | Yes | — | Email to receive support requests | + +### AI Gateway (Kilo Code) +| Variable | Required | Default | Description | +|---|---|---|---| +| `KILO_CODE_GATEWAY_URL` | Yes | — | Gateway API base URL | +| `KILO_CODE_GATEWAY_API_KEY` | Yes | — | Gateway API key | +| `KILO_CODE_GATEWAY_MODEL` | No | kilo-auto/free | Model identifier | + +### Quota & Rate Limiting +| Variable | Required | Default | Description | +|---|---|---|---| +| `WEEKLY_GEN_LIMIT` | No | 4 | Per-user weekly AI generations | +| `DAILY_GEN_CAP` | No | 150 | Global daily generation cap | +| `CHAT_MIN_GAP_MS` | No | 2000 | Min ms between chat AI calls | +| `DELIVERY_CATCHUP_BATCH_SIZE` | No | 200 | Chat delivery catch-up batch | +| `DELIVERY_CATCHUP_MAX_BATCHES` | No | 10 | Max catch-up batches | + +### Security +| Variable | Required | Default | Description | +|---|---|---|---| +| `ENCRYPTION_KEY` | Yes (prod) | — | AES-256 encryption passphrase | +| `JWT_SECRET` | No | — | (Not used — Firebase handles JWT) | + +### Caching +| Variable | Required | Default | Description | +|---|---|---|---| +| `ARCHITECTURE_CACHE_TTL_MS` | No | 21600000 | Architecture cache TTL (6h) | +| `ARCHITECTURE_CACHE_MAX_ENTRIES` | No | 100 | Max cached analyses | + +### Server +| Variable | Required | Default | Description | +|---|---|---|---| +| `PORT` | No | 5000 | Backend server port | +| `FRONTEND_URL` | Yes | http://localhost:3000 | Frontend URL for CORS | +| `NODE_ENV` | No | development | Environment mode | +| `LOG_LEVEL` | No | info | Logging level | +| `DEBUG_WEBHOOKS` | No | false | Enable webhook debug logging | + +### Frontend (Vite) +| Variable | Required | Default | Description | +|---|---|---|---| +| `VITE_API_URL` | Yes | http://localhost:5000 | Backend API URL | +| `VITE_FIREBASE_API_KEY` | Yes | — | Firebase Web API key | +| `VITE_FIREBASE_AUTH_DOMAIN` | Yes | — | Firebase auth domain | +| `VITE_FIREBASE_PROJECT_ID` | Yes | — | Firebase project ID | +| `VITE_FIREBASE_APP_ID` | Yes | — | Firebase app ID | + +--- + +## .env.example Template + +```bash +# Firebase +FIREBASE_PROJECT_ID= +FIREBASE_CLIENT_EMAIL= +FIREBASE_PRIVATE_KEY= +FIREBASE_API_KEY= +FIREBASE_AUTH_DOMAIN= + +# Database +MONGODB_URI= + +# Redis +REDIS_URL= + +# GitHub +GITHUB_CLIENT_ID= +GITHUB_CLIENT_SECRET= +GITHUB_APP_ID= +GITHUB_PRIVATE_KEY= +WEBHOOK_SECRET= + +# Google +GOOGLE_CLIENT_ID= +GOOGLE_CLIENT_SECRET= +GOOGLE_REDIRECT_URI= + +# LinkedIn +LINKEDIN_CLIENT_ID= +LINKEDIN_CLIENT_SECRET= +LINKEDIN_REDIRECT_URI= + +# Cloudinary +CLOUDINARY_CLOUD_NAME= +CLOUDINARY_API_KEY= +CLOUDINARY_API_SECRET= + +# SMTP +SMTP_USER= +SMTP_PASS= +SUPPORT_EMAIL= + +# AI Gateway +KILO_CODE_GATEWAY_URL= +KILO_CODE_GATEWAY_API_KEY= + +# Security +ENCRYPTION_KEY= + +# Server +PORT=5000 +FRONTEND_URL=http://localhost:3000 +NODE_ENV=development +``` + +--- + +## Cross-References + +- [02-security-auth-architecture.md](./02-security-auth-architecture.md) — Security variables +- [48-encryption-security-utilities.md](./48-encryption-security-utilities.md) — ENCRYPTION_KEY usage +- [27-usage-service-quota.md](./27-usage-service-quota.md) — Quota variables +- [21-github-oauth-integration.md](./21-github-oauth-integration.md) — GitHub variables +- [40-google-oauth-integration.md](./40-google-oauth-integration.md) — Google variables diff --git a/docs/features/57-error-handling-strategy.md b/docs/features/57-error-handling-strategy.md new file mode 100644 index 00000000..8dac5cc2 --- /dev/null +++ b/docs/features/57-error-handling-strategy.md @@ -0,0 +1,143 @@ +# 57 — Error Handling Strategy + +**NEW document** — Global error patterns, fail-open design, graceful degradation, error response schemas, logging + +--- + +## Feature Summary + +Zync follows a consistent error handling strategy across all backend routes and services. Key principles: fail-open for non-critical services (Redis, email), graceful degradation (return safe defaults), consistent error response schemas, and comprehensive logging without exposing sensitive data. + +--- + +## Error Handling Principles + +### 1. Fail-Open for Non-Critical Services +Services that are "nice to have" but not critical to app function: +- **Redis:** Cache miss → fetch from DB (no crash) +- **Email:** SMTP failure → return null (no crash, operation continues) +- **Geo-IP API:** Failure → return "Unknown" location (no crash) + +### 2. Graceful Degradation +When a subsystem fails, the app falls back to a safe state: +- **DB disconnected:** Chat routes return 503, other routes may use cache +- **GitHub API down:** Architecture analysis fails → quota refunded +- **Cloudinary upload fails:** Return error to user, no partial state + +### 3. Consistent Error Response Schema +All errors follow the same JSON structure: +```json +{ + "error": "Human-readable message", + "message": "Alternative message field (some routes)" +} +``` + +### 4. No Sensitive Data in Errors +- Stack traces never sent to client +- Internal paths never exposed +- Decrypted tokens never logged +- Only user-friendly messages returned + +--- + +## Error Response Patterns + +### Route-Level Error Handling +```js +router.get('/resource', verifyToken, async (req, res) => { + try { + // ... business logic + res.json(data); + } catch (error) { + console.error('[RouteName] error:', error); + res.status(500).json({ error: 'Server error' }); + } +}); +``` + +### Service-Level Error Handling +```js +// Services throw errors, callers catch +const sendZyncEmail = async (to, subject, html) => { + try { + return await send_ZYNC_email(to, subject, html); + } catch (error) { + if (error.code === 'EAUTH') { + console.error('Email auth failed'); + return null; // Fail-open + } + throw error; // Re-throw non-auth errors + } +}; +``` + +--- + +## HTTP Status Code Usage + +| Status | Usage | Example | +|---|---|---| +| 200 | Success | GET /projects returns list | +| 201 | Created | POST /projects creates project | +| 202 | Accepted (async) | Webhook received and queued | +| 400 | Bad request | Missing required fields | +| 401 | Unauthorized | Missing/invalid JWT | +| 403 | Forbidden | Not owner of resource | +| 404 | Not found | Resource doesn't exist | +| 409 | Conflict | Duplicate application | +| 413 | Payload too large | File upload exceeds limit | +| 429 | Too many requests | Rate limited or quota exceeded | +| 500 | Server error | Unhandled exception | +| 503 | Service unavailable | DB disconnected | + +--- + +## Fail-Open Matrix + +| Service | Failure Mode | Behavior | User Impact | +|---|---|---|---| +| Redis | Down | Cache miss → DB query | Slower but functional | +| Email (SMTP) | Auth failure | Return null | No email sent, operation succeeds | +| Email (SMTP) | Network error | Re-throw | Caller catches, may fail operation | +| Geo-IP API | Down | Return "Unknown" | Location shows "Unknown" | +| Kilo Gateway | Timeout | Quota refunded | AI analysis fails, user can retry | +| Cloudinary | Upload fails | Error thrown | Upload fails, user sees error | +| GitHub API | Rate limited | Error from GitHub | GitHub features unavailable | +| MongoDB | Disconnected | 503 (requireDb) or error | Affected routes return 503 | + +--- + +## Logging Strategy + +### Log Levels +| Level | Usage | +|---|---| +| `console.error` | Errors, exceptions, auth failures | +| `console.warn` | Cache misses, degraded mode, warnings | +| `console.log` | Info, connections, disconnections | +| `debugWebhookLog` | Debug-only (gated by env var) | + +### What Gets Logged +- Socket connections/disconnections +- Email auth failures +- Cache failures +- Webhook processing +- Delivery catch-up batches +- Error stack traces (server-side only) + +### What Does NOT Get Logged +- Decrypted tokens +- User passwords +- Full request bodies (only error messages) +- PII in plaintext + +--- + +## Cross-References + +- [51-middleware-stack-overview.md](./51-middleware-stack-overview.md) — Auth middleware error handling +- [33-redis-cache-layer.md](./33-redis-cache-layer.md) — Redis fail-open design +- [28-email-service-notifications.md](./28-email-service-notifications.md) — Email fail-open +- [27-usage-service-quota.md](./27-usage-service-quota.md) — Quota fail-open +- [48-encryption-security-utilities.md](./48-encryption-security-utilities.md) — Decrypt error handling diff --git a/docs/features/58-cross-reference-matrix.md b/docs/features/58-cross-reference-matrix.md new file mode 100644 index 00000000..02630cc5 --- /dev/null +++ b/docs/features/58-cross-reference-matrix.md @@ -0,0 +1,108 @@ +# 58 — Cross-Reference Matrix + +**NEW document** — Complete cross-reference index, feature dependency graph, related documentation map + +--- + +## Feature Cross-Reference Matrix + +| Doc | Related Docs | Shared Components | +|---|---|---| +| 00-overview | All | — | +| 01-frontend-architecture | 53, 54, 55 | React, Vite, Tailwind | +| 02-security-auth | 08, 48, 51, 57 | Firebase Admin, AES-256 | +| 03-caching-strategy | 14, 21, 27, 33 | Redis | +| 04-service-inventory | All services | — | +| 05-database-schema | 52 | Mongoose models | +| 06-middleware-stack | 51 | Express middleware | +| 07-deployment-config | 56 | Environment variables | +| 08-firebase-auth | 02, 51, 55 | Firebase Admin SDK | +| 09-user-profile | 10, 32, 34, 39 | User model, Cloudinary | +| 10-account-deletion | 09, 28 | Email verification | +| 11-presence-system | 19, 23, 50 | Socket.IO /presence | +| 12-haveibeenpwned | 02 | HIBP API | +| 13-linkedin-oauth | 08 | LinkedIn API, Firebase | +| 14-project-crud | 15, 16, 25, 44 | Project model, cache | +| 15-steps-pipeline | 14, 16, 47 | Step model | +| 16-task-management | 14, 15, 22, 47 | ProjectTask, GitHub API | +| 17-notes-system | 18, 19, 20 | Note model | +| 18-folders-organization | 17, 39 | Folder model | +| 19-realtime-notes | 17, 20, 50 | Socket.IO /notes, Yjs | +| 20-notes-socket-handler | 19 | noteSocketHandler.js | +| 21-github-oauth | 14, 16, 22, 48 | GitHub API, encryption | +| 22-github-webhook | 16, 21, 46 | HMAC, queue | +| 23-instant-chat | 24, 38, 50 | Message model | +| 24-chat-socket-handler | 23, 50 | Socket.IO /chat | +| 25-ai-architecture | 14, 26, 27 | Kilo Gateway, cache | +| 26-kilo-code-gateway | 25, 27 | LLM API client | +| 27-usage-service-quota | 25, 26, 33 | Redis, Lua scripts | +| 28-email-service | 10, 16, 31, 42 | SMTP, nodemailer | +| 29-session-management | 30, 45 | Session model | +| 30-meeting-system | 29, 40, 45 | Google Calendar API | +| 31-team-crud | 28, 39 | Team model, Activity | +| 32-cloudinary-upload | 09, 38 | Cloudinary SDK | +| 33-redis-cache | 03, 14, 21, 27 | Redis client | +| 34-location-detection | 09 | Geo-IP API | +| 35-architecture-agent | 25, 26, 27 | Kilo Gateway | +| 36-project-generation | 14, 15, 26 | AI blueprint | +| 37-calendar-holidays | 30 | Holiday API | +| 38-file-upload | 23, 32 | Multer, Sharp | +| 39-user-search | 09, 18, 23, 31 | Regex, pagination | +| 40-google-oauth | 08, 28, 30 | Google APIs | +| 41-collaborator-beta | 28 | Collaborator model | +| 42-support-ticket | 28 | Email service | +| 43-design-inspiration | 04 | Cheerio, scraping | +| 44-link-repo | 14, 21 | Project + GitHub | +| 45-meet-routes | 29, 30, 40 | Meeting model | +| 46-webhook-routes | 22 | Webhook queue | +| 47-task-routes | 15, 16, 22, 50 | Socket.IO /tasks | +| 48-encryption | 02, 21, 40 | CryptoJS, AES-256 | +| 49-pagination-helpers | 14, 17, 23, 39 | Pagination utility | +| 50-socket-io-init | 11, 19, 24, 47 | Socket.IO server | +| 51-middleware-stack | 02, 08, 22 | Express middleware | +| 52-database-schema | 05 | All Mongoose models | +| 53-frontend-routing | 01, 08, 54 | React Router | +| 54-frontend-state | 01, 50, 55 | TanStack Query, Zustand | +| 55-api-client | 08, 54 | Axios interceptors | +| 56-environment-variables | All | .env reference | +| 57-error-handling | 33, 48, 51 | Fail-open patterns | +| 58-cross-reference | All | This document | + +--- + +## Dependency Graph (Simplified) + +``` +Firebase Auth (08) ────┬── User Profile (09) + ├── LinkedIn OAuth (13) + ├── Google OAuth (40) + └── Middleware (51) + +Socket.IO (50) ────┬── Presence (11) + ├── Chat (24) + ├── Notes (19, 20) + └── Tasks (47) + +GitHub (21) ────┬── Webhooks (22) + ├── Project CRUD (14) + ├── Task Mgmt (16) + └── Repo Linking (44) + +AI Gateway (26) ────┬── Architecture Analysis (25) + ├── Architecture Chat (35) + └── Project Generation (36) + +Redis (33) ────┬── Cache (03) + └── Quota (27) + +Email (28) ────┬── Account Deletion (10) + ├── Task Assignment (16) + ├── Team Invites (31) + └── Support (42) +``` + +--- + +## Cross-References + +- All documentation files (this is the master index) diff --git a/docs/features/59-deployment-infrastructure.md b/docs/features/59-deployment-infrastructure.md new file mode 100644 index 00000000..f839ad53 --- /dev/null +++ b/docs/features/59-deployment-infrastructure.md @@ -0,0 +1,186 @@ +# 59 — Deployment & Infrastructure + +**NEW document** — Render hosting, environment configuration, build process, health checks, scaling considerations + +--- + +## Feature Summary + +Zync is deployed on Render with a monolithic backend (Express + Socket.IO) and a static frontend (Vite build). MongoDB Atlas provides the database, Redis Cloud provides caching, and Cloudinary handles media storage. This document covers the deployment configuration, build process, and infrastructure setup. + +--- + +## Architecture Diagram + +``` +┌─────────────────── INFRASTRUCTURE ──────────────────────┐ +│ │ +│ ┌─────────────┐ ┌──────────────┐ │ +│ │ Frontend │ │ Backend │ │ +│ │ (Vite) │ │ (Express) │ │ +│ │ Render │ │ Render │ │ +│ │ Static │ │ Web Service │ │ +│ └──────┬──────┘ └──────┬───────┘ │ +│ │ │ │ +│ │ HTTPS API │ │ +│ └───────────────────►│ │ +│ │ │ +│ ┌────────────────────┼─────────────────┐ │ +│ │ │ │ │ +│ ▼ ▼ ▼ │ +│ ┌──────────┐ ┌──────────────┐ ┌──────────────┐ │ +│ │ MongoDB │ │ Redis │ │ Cloudinary │ │ +│ │ Atlas │ │ Cloud │ │ Cloud │ │ +│ └──────────┘ └──────────────┘ └──────────────┘ │ +│ │ +│ External APIs: │ +│ ├─ Firebase Auth (Google) │ +│ ├─ GitHub API + GitHub App │ +│ ├─ Google Calendar API │ +│ ├─ Kilo Code Gateway (LLM) │ +│ ├─ HaveIBeenPwned API │ +│ ├─ ipapi.co (Geo-IP) │ +│ └─ date.nager.at (Holidays) │ +│ │ +└─────────────────────────────────────────────────────────┘ +``` + +--- + +## Render Configuration + +### Backend (Web Service) +```yaml +# render.yaml +services: + - type: web + name: zync-backend + env: node + buildCommand: npm install + startCommand: node index.js + healthCheckPath: /api/health + envVars: + - key: NODE_ENV + value: production + - key: MONGODB_URI + sync: false + - key: REDIS_URL + sync: false + - key: ENCRYPTION_KEY + sync: false + # ... all env vars from .env +``` + +### Frontend (Static Site) +```yaml + - type: web + name: zync-frontend + env: static + buildCommand: npm install && npm run build + staticPublishPath: ./dist + envVars: + - key: VITE_API_URL + value: https://zync-backend.onrender.com +``` + +--- + +## Build Process + +### Backend +1. `npm install` — install dependencies +2. No build step (Node.js runs directly) +3. `node index.js` — start server +4. Health check: `GET /api/health` → `{ status: "ok" }` + +### Frontend +1. `npm install` — install dependencies +2. `npm run build` — Vite production build +3. Output: `dist/` directory +4. Served as static files by Render + +--- + +## Health Check Endpoint + +### GET /api/health +```js +router.get('/health', (req, res) => { + const dbStatus = mongoose.connection.readyState === 1 ? 'connected' : 'disconnected'; + const redisStatus = isAvailable() ? 'connected' : 'disconnected'; + + res.json({ + status: 'ok', + database: dbStatus, + redis: redisStatus, + uptime: process.uptime(), + timestamp: new Date().toISOString(), + }); +}); +``` +- Render checks this endpoint for service health +- Returns DB and Redis connection status +- No auth required (public endpoint) + +--- + +## Scaling Considerations + +### Socket.IO Scaling +- **Single instance:** Works fine for small user base +- **Multi-instance:** Requires Redis adapter for Socket.IO + ```js + io.adapter(redisAdapter({ host: REDIS_HOST, port: 6379 })); + ``` +- Redis adapter broadcasts events across all Node.js instances + +### In-Memory State Limitations +- `notePresence` Map (notes socket) — lost on restart, per-instance +- `architectureAnalysisCache` Map — lost on restart, per-instance +- `webhookQueue` Map — lost on restart, per-instance +- `userSockets` Map (chat/presence) — lost on restart, per-instance + +**For multi-instance:** These should be moved to Redis (shared state) + +### Database Connection Pooling +```js +mongoose.connect(MONGODB_URI, { + maxPoolSize: 10, + minPoolSize: 2, + serverSelectionTimeoutMS: 5000, +}); +``` + +--- + +## Environment Setup Checklist + +### Pre-Deployment +- [ ] MongoDB Atlas cluster created +- [ ] Redis Cloud instance created +- [ ] Cloudinary account configured +- [ ] Firebase project created with Admin SDK +- [ ] GitHub OAuth App + GitHub App created +- [ ] Google OAuth credentials created +- [ ] LinkedIn OAuth credentials created +- [ ] Kilo Code Gateway URL + API key obtained +- [ ] SMTP credentials (Gmail app password) configured +- [ ] `ENCRYPTION_KEY` generated (32+ char random string) +- [ ] `WEBHOOK_SECRET` generated +- [ ] All env vars set in Render dashboard + +### Post-Deployment +- [ ] Health check returns 200 +- [ ] Frontend loads and can reach backend +- [ ] Firebase Auth login works +- [ ] Socket.IO connections succeed +- [ ] GitHub webhook delivery verified + +--- + +## Cross-References + +- [07-deployment-config.md](./07-deployment-config.md) — Original deployment doc +- [56-environment-variables-reference.md](./56-environment-variables-reference.md) — All env vars +- [50-socket-io-initialization.md](./50-socket-io-initialization.md) — Redis adapter for scaling +- [33-redis-cache-layer.md](./33-redis-cache-layer.md) — Redis configuration diff --git a/docs/features/60-security-architecture.md b/docs/features/60-security-architecture.md new file mode 100644 index 00000000..2531cbf6 --- /dev/null +++ b/docs/features/60-security-architecture.md @@ -0,0 +1,217 @@ +# 60 — Security Architecture + +**NEW document** — Authentication, authorization, encryption, HMAC, rate limiting, input validation, security headers + +--- + +## Feature Summary + +Zync's security architecture encompasses Firebase JWT authentication, AES-256 token encryption, HMAC webhook verification, rate limiting, regex injection prevention, Helmet security headers, and CORS configuration. This document provides a comprehensive security overview. + +--- + +## Security Layers + +``` +┌─────────────────────────────────────────────────────────┐ +│ SECURITY LAYERS │ +├─────────────────────────────────────────────────────────┤ +│ │ +│ Layer 1: Network Security │ +│ ├─ HTTPS only (Render enforces) │ +│ ├─ CORS: single origin (FRONTEND_URL) │ +│ ├─ Helmet: security headers (XSS, clickjacking, etc.) │ +│ └─ Rate limiting: 100 req / 15 min per IP │ +│ │ +│ Layer 2: Authentication │ +│ ├─ Firebase Auth (JWT verification) │ +│ ├─ verifyToken middleware on protected routes │ +│ ├─ Token injected by frontend Axios interceptor │ +│ └─ 401 on missing/invalid/expired token │ +│ │ +│ Layer 3: Authorization │ +│ ├─ Resource ownership checks (project.ownerUid === uid)│ +│ ├─ Team role checks (owner/admin/member) │ +│ ├─ Chat participation checks (chatId includes uid) │ +│ └─ 403 on unauthorized access │ +│ │ +│ Layer 4: Data Protection │ +│ ├─ AES-256 encryption for OAuth tokens at rest │ +│ ├─ Encrypted tokens excluded from API responses │ +│ ├─ Passwords checked against HIBP (breach database) │ +│ └─ Security PIN required for destructive operations │ +│ │ +│ Layer 5: Input Validation │ +│ ├─ escapeRegExp() for regex injection prevention │ +│ ├─ Multer file type + size limits │ +│ ├─ Mongoose schema validation │ +│ └─ Request body validation in route handlers │ +│ │ +│ Layer 6: Webhook Security │ +│ ├─ HMAC SHA-256 signature verification │ +│ ├─ Timing-safe comparison (prevents timing attacks) │ +│ ├─ Delivery ID deduplication │ +│ └─ 401 on signature mismatch │ +│ │ +└─────────────────────────────────────────────────────────┘ +``` + +--- + +## Authentication Flow + +``` +1. User signs in (Firebase popup/redirect) + → Firebase issues JWT (1 hour TTL) + +2. Frontend stores JWT (Firebase manages auto-refresh) + +3. API call: Axios interceptor adds Authorization: Bearer + +4. Backend: verifyToken middleware + → admin.auth().verifyIdToken(token) + → Sets req.user = { uid, email } + +5. Route handler: uses req.user.uid for queries +``` + +--- + +## Authorization Patterns + +### Resource Ownership +```js +const project = await Project.findById(projectId); +if (project.ownerUid !== req.user.uid && !project.team.includes(req.user.uid)) { + return res.status(403).json({ error: 'Unauthorized' }); +} +``` + +### Team Role Check +```js +if (team.ownerUid !== req.user.uid && !team.admins.includes(req.user.uid)) { + return res.status(403).json({ error: 'Admin access required' }); +} +``` + +### Chat Participation +```js +const parts = chatId.split('_'); +if (!parts.includes(req.user.uid)) { + return res.status(403).json({ error: 'Unauthorized' }); +} +``` + +### Message Ownership (mark-seen) +```js +await Message.updateMany( + { _id: { $in: messageIds }, receiverId: userId }, // Security: can only mark own messages + { $set: { seen: true } } +); +``` + +--- + +## Encryption Details + +### AES-256 Token Encryption +- **Algorithm:** AES (CryptoJS) +- **Key:** `ENCRYPTION_KEY` environment variable +- **Encrypted data:** GitHub tokens, Google tokens +- **Never returned:** `.select('-githubIntegration.accessToken')` + +### HMAC Webhook Verification +- **Algorithm:** HMAC SHA-256 +- **Key:** `WEBHOOK_SECRET` environment variable +- **Comparison:** `crypto.timingSafeEqual()` (timing-attack safe) +- **Header:** `X-Hub-Signature-256` + +### Security PIN +- **Hashed:** Not stored in plaintext +- **Required for:** Team deletion, ownership transfer +- **Set by:** Team owner during creation + +--- + +## Security Headers (Helmet) + +```js +const helmet = require('helmet'); +app.use(helmet({ + contentSecurityPolicy: { + directives: { + defaultSrc: ["'self'"], + scriptSrc: ["'self'", "'unsafe-inline'"], + styleSrc: ["'self'", "'unsafe-inline'"], + imgSrc: ["'self'", 'data:', 'https://res.cloudinary.com'], + connectSrc: ["'self'", 'https://api.github.com', 'wss:'], + }, + }, +})); +``` + +| Header | Purpose | +|---|---| +| `X-Content-Type-Options: nosniff` | Prevent MIME sniffing | +| `X-Frame-Options: DENY` | Prevent clickjacking | +| `X-XSS-Protection: 1` | XSS protection | +| `Strict-Transport-Security` | Force HTTPS | +| `Content-Security-Policy` | Restrict resource loading | + +--- + +## Rate Limiting + +```js +const rateLimit = require('express-rate-limit'); +app.use('/api/', rateLimit({ + windowMs: 15 * 60 * 1000, + max: 100, + message: { error: 'Too many requests' }, +})); +``` +- 100 requests per 15 minutes per IP +- Webhook routes exempt (GitHub needs fast response) +- AI generation has separate quota (usageService) + +--- + +## Input Validation + +### Regex Injection Prevention +```js +const escapeRegExp = (string) => string.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); +``` +- Used in: user search, task search, note search +- Prevents: unintended regex patterns, ReDoS attacks + +### File Upload Validation +```js +const upload = multer({ + limits: { fileSize: 10 * 1024 * 1024 }, // 10MB + fileFilter: (req, file, cb) => { + const allowed = ['image/jpeg', 'image/png', 'image/gif', 'image/webp', + 'application/pdf', 'text/plain', 'application/json']; + if (allowed.includes(file.mimetype)) cb(null, true); + else cb(new Error('File type not allowed')); + } +}); +``` + +### Mongoose Schema Validation +- Required fields enforced at schema level +- Type validation (String, Number, Date, etc.) +- Enum values for status fields +- Min/max length constraints + +--- + +## Cross-References + +- [08-firebase-auth-flow.md](./08-firebase-auth-flow.md) — JWT authentication +- [48-encryption-security-utilities.md](./48-encryption-security-utilities.md) — AES-256 encryption +- [22-github-webhook-handler.md](./22-github-webhook-handler.md) — HMAC verification +- [12-haveibeenpwned-integration.md](./12-haveibeenpwned-integration.md) — Password breach check +- [51-middleware-stack-overview.md](./51-middleware-stack-overview.md) — Middleware chain +- [57-error-handling-strategy.md](./57-error-handling-strategy.md) — Error handling without info leaks +- [56-environment-variables-reference.md](./56-environment-variables-reference.md) — Security env vars diff --git a/docs/features/61-performance-optimization.md b/docs/features/61-performance-optimization.md new file mode 100644 index 00000000..bd60b0ef --- /dev/null +++ b/docs/features/61-performance-optimization.md @@ -0,0 +1,167 @@ +# 61 — Performance & Optimization + +**NEW document** — Caching strategy, query optimization, lazy loading, bundle splitting, Socket.IO efficiency + +--- + +## Feature Summary + +Zync optimizes performance through Redis caching, MongoDB query indexes, frontend lazy loading, code splitting, TanStack Query background refetching, and Socket.IO efficient event design. This document covers all performance optimizations across the stack. + +--- + +## Performance Optimization Areas + +### 1. Backend Caching (Redis) +- **Project lists:** Cached per user, 300s TTL, invalidated on create/update/delete +- **GitHub repos:** Cached per user, 60s TTL +- **User profiles:** Cached per user, 300s TTL +- **Architecture analysis:** In-memory Map, 6h TTL, repo freshness key +- **Country list:** In-memory, 24h TTL + +### 2. Database Query Optimization +- **Compound indexes:** `{ chatId: 1, createdAt: 1 }` for chat history +- **Text indexes:** User displayName + email for search +- **Lean queries:** `.lean()` for read-only operations (skips Mongoose overhead) +- **Projection:** `.select()` to fetch only needed fields +- **Pagination:** Cursor-based (chat) and page-based (lists) + +### 3. Frontend Lazy Loading +- **Route-level:** Each view loaded on demand via `React.lazy()` +- **Component-level:** Heavy components (editors, charts) loaded conditionally +- **Image lazy loading:** `loading="lazy"` on images + +### 4. Bundle Splitting +- **Vendor split:** React, Firebase, Socket.IO in separate chunks +- **Route split:** Each route is a separate chunk +- **Dynamic imports:** Heavy libraries loaded on demand + +### 5. TanStack Query Optimization +- **staleTime: 30s:** Prevents excessive refetching +- **refetchOnWindowFocus:** Keeps data fresh when user returns +- **Optimistic updates:** UI updates before server confirms +- **Background refetch:** Data stays fresh without blocking UI +- **Query invalidation:** Surgical cache invalidation on mutations + +### 6. Socket.IO Efficiency +- **Namespaces:** Isolated event spaces (no cross-namespace pollution) +- **Rooms:** Targeted broadcasts (only relevant clients receive events) +- **Dumb relay:** Yjs updates forwarded without server-side processing +- **Multi-device:** Single emit reaches all of a user's devices + +--- + +## Caching Strategy Detail + +### Cache-Aside Pattern +``` +1. Check cache (Redis) + ├─ Hit: Return cached data + └─ Miss: Fetch from DB → Store in cache → Return +2. On data change: Invalidate cache +3. Next read: Cache miss → Fresh data from DB → Re-cache +``` + +### Invalidation Strategy +```js +// After project update: +async function invalidateProjectCache(project) { + const uids = [project.ownerUid, ...(project.team || [])]; + const keys = uids.map(uid => `projects:${uid}`); + await cache.invalidate(...keys); +} +``` +- Invalidates cache for owner AND all team members +- Next read by any member fetches fresh data + +--- + +## Query Optimization Examples + +### Chat History (Cursor-Based) +```js +const filter = { chatId }; +if (cursor) filter._id = { $gt: new mongoose.Types.ObjectId(cursor) }; +const messages = await Message.find(filter) + .sort({ createdAt: 1 }) + .limit(50) + .lean(); +``` +- **Index:** `{ chatId: 1, createdAt: 1 }` +- **Cursor:** Uses `_id` (ObjectId contains timestamp) for stable pagination +- **lean():** Skips Mongoose document creation (plain objects) + +### Conversations (Aggregation) +```js +const conversations = await Message.aggregate([ + { $match: { $or: [{ senderId: uid }, { receiverId: uid }] } }, + { $sort: { createdAt: -1 } }, + { $group: { _id: '$chatId', doc: { $first: '$$ROOT' } } }, + { $replaceRoot: { newRoot: '$doc' } }, + { $sort: { createdAt: -1 } }, +]); +``` +- Single aggregation pipeline instead of multiple queries +- `$group` + `$first` gets latest message per chat efficiently + +--- + +## Frontend Performance + +### Bundle Size Optimization +``` +Initial bundle: +├─ React + ReactDOM (~45kb gzipped) +├─ Firebase Auth (~30kb gzipped) +├─ React Router (~10kb gzipped) +├─ TanStack Query (~12kb gzipped) +└─ App shell + layout (~20kb gzipped) +Total initial: ~117kb gzipped + +Lazy-loaded chunks: +├─ DashboardHome (~15kb) +├─ ProjectWorkspace (~25kb) +├─ MessagesPage (~20kb) +├─ NotesView + TipTap (~50kb) +├─ SettingsView (~15kb) +└─ Other views (~10-20kb each) +``` + +### TanStack Query Configuration +```ts +defaultOptions: { + queries: { + staleTime: 30 * 1000, // 30s before refetch + gcTime: 5 * 60 * 1000, // 5min garbage collection + refetchOnWindowFocus: true, // Refresh on tab return + retry: 2, // Retry failed requests + retryDelay: 1000, // 1s between retries + } +} +``` + +--- + +## Socket.IO Performance + +### Event Design +- **Minimal payloads:** Only necessary data in events +- **Targeted rooms:** `socket.to(noteId).emit()` — only room members receive +- **No server-side processing for Yjs:** Binary updates forwarded as-is +- **Batch delivery catch-up:** 200 messages per batch, max 10 batches + +### Connection Management +- **userSockets Map:** O(1) lookup for user → sockets +- **Stale cleanup:** 30s interval removes inactive users (2min threshold) +- **unref():** Cleanup interval doesn't prevent process exit + +--- + +## Cross-References + +- [03-performance-caching-strategy.md](./03-performance-caching-strategy.md) — Caching overview +- [33-redis-cache-layer.md](./33-redis-cache-layer.md) — Redis cache utility +- [54-frontend-state-management.md](./54-frontend-state-management.md) — TanStack Query +- [53-frontend-routing-layout.md](./53-frontend-routing-layout.md) — Lazy loading +- [52-database-schema-models.md](./52-database-schema-models.md) — Index strategy +- [50-socket-io-initialization.md](./50-socket-io-initialization.md) — Socket.IO setup diff --git a/docs/features/62-testing-quality-assurance.md b/docs/features/62-testing-quality-assurance.md new file mode 100644 index 00000000..f7c63de5 --- /dev/null +++ b/docs/features/62-testing-quality-assurance.md @@ -0,0 +1,292 @@ +# 62 — Testing & Quality Assurance + +**NEW document** — Test structure, backend route tests, frontend component tests, test utilities, CI considerations + +--- + +## Feature Summary + +Zync includes backend route tests using Jest and Supertest, covering authentication, project CRUD, task management, and API error handling. This document covers the test structure, utilities, and quality assurance practices. + +--- + +## Test Structure + +``` +backend/ +├── tests/ +│ ├── taskRoutes.test.js → Task route tests +│ ├── projectRoutes.test.js → Project route tests +│ ├── auth.test.js → Auth middleware tests +│ ├── setup.js → Test setup (DB mock, fixtures) +│ └── helpers/ +│ ├── mockUser.js → Mock Firebase user +│ ├── mockToken.js → Mock JWT token +│ └── fixtures.js → Test data fixtures +├── jest.config.js → Jest configuration +└── package.json → Test scripts +``` + +--- + +## Jest Configuration + +### File: `backend/jest.config.js` +```js +module.exports = { + testEnvironment: 'node', + testMatch: ['**/tests/**/*.test.js'], + setupFilesAfterEnv: ['/tests/setup.js'], + collectCoverageFrom: [ + 'routes/**/*.js', + 'services/**/*.js', + 'utils/**/*.js', + '!**/node_modules/**', + ], + coverageThreshold: { + global: { + branches: 60, + functions: 70, + lines: 70, + statements: 70, + }, + }, +}; +``` + +### Test Scripts (package.json) +```json +{ + "scripts": { + "test": "jest", + "test:watch": "jest --watch", + "test:coverage": "jest --coverage", + "test:ci": "jest --ci --coverage --watchAll=false" + } +} +``` + +--- + +## Test Setup + +### File: `backend/tests/setup.js` +```js +// Mock Firebase Admin +jest.mock('firebase-admin', () => ({ + auth: () => ({ + verifyIdToken: jest.fn().mockResolvedValue({ + uid: 'test-uid-123', + email: 'test@zync.dev', + }), + }), + initializeApp: jest.fn(), + credential: { cert: jest.fn() }, +})); + +// Mock Redis +jest.mock('../utils/redisClient', () => ({ + getRedisClient: jest.fn(), + isAvailable: jest.fn().mockReturnValue(false), +})); + +// Mock Mongoose connection +beforeAll(async () => { + // Use in-memory MongoDB or mock +}); +``` + +--- + +## Test Utilities + +### mockUser.js +```js +module.exports = { + uid: 'test-uid-123', + email: 'test@zync.dev', + displayName: 'Test User', + photoURL: 'https://example.com/avatar.jpg', +}; +``` + +### mockToken.js +```js +module.exports = { + validToken: 'Bearer mock-firebase-token', + invalidToken: 'Bearer invalid-token', + missingToken: null, +}; +``` + +### fixtures.js +```js +module.exports = { + project: { + _id: '60f1a2b3c4d5e6f7a8b9c0d1', + name: 'Test Project', + description: 'A test project', + ownerUid: 'test-uid-123', + team: [], + }, + task: { + _id: '60f1a2b3c4d5e6f7a8b9c0d2', + title: 'Test Task', + description: 'A test task', + stepId: '60f1a2b3c4d5e6f7a8b9c0d3', + projectId: '60f1a2b3c4d5e6f7a8b9c0d1', + }, +}; +``` + +--- + +## Test Examples + +### Task Routes Test +**File:** `backend/tests/taskRoutes.test.js` + +```js +const request = require('supertest'); +const app = require('../index'); +const { validToken } = require('./helpers/mockToken'); + +describe('Task Routes', () => { + describe('PUT /api/tasks/:taskId', () => { + it('should update task with valid token', async () => { + const res = await request(app) + .put('/api/tasks/test-task-id') + .set('Authorization', validToken) + .send({ title: 'Updated Title' }); + + expect(res.status).toBe(200); + expect(res.body.title).toBe('Updated Title'); + }); + + it('should return 401 without token', async () => { + const res = await request(app) + .put('/api/tasks/test-task-id') + .send({ title: 'Updated Title' }); + + expect(res.status).toBe(401); + }); + }); + + describe('DELETE /api/tasks/:taskId', () => { + it('should delete task with valid ownership', async () => { + const res = await request(app) + .delete('/api/tasks/test-task-id') + .set('Authorization', validToken); + + expect(res.status).toBe(200); + expect(res.body.message).toBe('Task deleted'); + }); + + it('should return 404 for non-existent task', async () => { + const res = await request(app) + .delete('/api/tasks/nonexistent-id') + .set('Authorization', validToken); + + expect(res.status).toBe(404); + }); + }); +}); +``` + +### Auth Middleware Test +```js +describe('Auth Middleware', () => { + it('should pass with valid token', async () => { + const req = { headers: { authorization: 'Bearer valid-token' } }; + const res = { status: jest.fn().json: jest.fn() }; + const next = jest.fn(); + + await verifyToken(req, res, next); + + expect(next).toHaveBeenCalled(); + expect(req.user.uid).toBe('test-uid-123'); + }); + + it('should return 401 without token', async () => { + const req = { headers: {} }; + const res = { status: jest.fn().json: jest.fn() }; + const next = jest.fn(); + + await verifyToken(req, res, next); + + expect(res.status).toHaveBeenCalledWith(401); + expect(next).not.toHaveBeenCalled(); + }); +}); +``` + +--- + +## Test Coverage Areas + +| Area | Coverage | Test Files | +|---|---|---| +| Auth middleware | Token verification, 401 cases | auth.test.js | +| Task routes | CRUD, search, quick tasks | taskRoutes.test.js | +| Project routes | CRUD, GitHub linking | projectRoutes.test.js | +| Error handling | 400, 401, 403, 404, 500 | All test files | +| Input validation | Missing fields, invalid data | Route tests | + +--- + +## CI Considerations + +### GitHub Actions (Future) +```yaml +name: Tests +on: [push, pull_request] +jobs: + test: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + - uses: actions/setup-node@v3 + with: + node-version: 18 + - run: cd backend && npm install + - run: cd backend && npm run test:ci +``` + +### Pre-Commit Hooks (Future) +```json +{ + "hooks": { + "pre-commit": "cd backend && npm test" + } +} +``` + +--- + +## Running Tests + +```bash +# Run all tests +cd backend && npm test + +# Run with watch mode +npm run test:watch + +# Run with coverage report +npm run test:coverage + +# Run specific test file +npx jest tests/taskRoutes.test.js + +# Run with verbose output +npx jest --verbose +``` + +--- + +## Cross-References + +- [51-middleware-stack-overview.md](./51-middleware-stack-overview.md) — Auth middleware being tested +- [47-task-routes-standalone.md](./47-task-routes-standalone.md) — Task routes being tested +- [14-project-crud.md](./14-project-crud.md) — Project routes being tested +- [57-error-handling-strategy.md](./57-error-handling-strategy.md) — Error cases tested