Skip to content

Latest commit

Β 

History

10 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

☁️ CloudVault Sentinel

Secure Cloud File Storage & Collaboration Platform

CloudVault is a production-oriented, open-source cloud file storage and collaboration platform designed to help individuals, teams, and organizations securely store, organize, access, share, version, and manage digital files from a centralized platform.


image image image image image image

πŸ“Œ Project Overview

Modern users and organizations manage digital files across laptops, mobile devices, email attachments, messaging platforms, USB drives, local servers, and multiple cloud services.

This creates several real-world problems:

  • Scattered files
  • Difficult file discovery
  • Unauthorized access
  • Accidental deletion
  • Lack of file version control
  • Difficult collaboration
  • Uncontrolled file sharing
  • Limited visibility into file activity
  • Storage management problems
  • Slow processing of large files

CloudVault aims to solve these problems by providing a centralized, secure, scalable, and cloud-ready file management platform.

The project is designed not merely as a CRUD application, but as a practical backend and cloud engineering project demonstrating:

  • Backend development
  • REST API design
  • Database architecture
  • Cloud object storage
  • Authentication and authorization
  • Asynchronous processing
  • Caching
  • File security
  • Organization management
  • DevOps
  • Containerization
  • CI/CD
  • Production deployment
  • System design

🎯 Problem Statement

Individuals and organizations need a reliable way to securely store, organize, share, and manage digital files.

Traditional approaches often distribute files across:

Laptop
   β”‚
   β”œβ”€β”€ Documents
   β”œβ”€β”€ Projects
   └── Photos

Mobile
   β”‚
   β”œβ”€β”€ Images
   └── PDFs

Email
   β”‚
   └── Attachments

USB Drives
   β”‚
   └── Backup Files

Multiple Cloud Services

This makes it difficult to maintain:

  • Centralized storage
  • Access control
  • File ownership
  • File history
  • Collaboration
  • Security
  • Auditability
  • Backup and recovery

CloudVault provides a centralized solution.


πŸ’‘ Vision

The vision of CloudVault is to build a secure and scalable platform where users can:

STORE
  ↓
ORGANIZE
  ↓
SEARCH
  ↓
SHARE
  ↓
COLLABORATE
  ↓
PROTECT
  ↓
MONITOR
  ↓
RECOVER

πŸ‘₯ Target Users

CloudVault is designed for multiple types of users.

πŸŽ“ Students

Students can manage:

  • Notes
  • Assignments
  • Projects
  • Certificates
  • Resume
  • Academic documents

Example:

My Drive/
β”œβ”€β”€ College/
β”‚   β”œβ”€β”€ Notes/
β”‚   β”œβ”€β”€ Assignments/
β”‚   └── Marksheets/
β”‚
β”œβ”€β”€ Projects/
β”‚   β”œβ”€β”€ CloudVault/
β”‚   └── ML Project/
β”‚
β”œβ”€β”€ Certificates/
└── Resume/

πŸ‘¨β€πŸ’» Professionals

Professionals can manage:

  • Resumes
  • Certificates
  • Work documents
  • Reports
  • Contracts
  • Project files

πŸ‘₯ Teams

Teams can collaborate through:

  • Shared folders
  • Shared files
  • Team permissions
  • Organization workspaces
  • File versioning

🏒 Organizations

Organizations can create:

ABC Technologies
β”‚
β”œβ”€β”€ Engineering
β”œβ”€β”€ HR
β”œβ”€β”€ Finance
β”œβ”€β”€ Sales
└── Management

Each department can have controlled access.


✨ Core Features

πŸ” Authentication & Account Management

  • User registration
  • Login/logout
  • JWT authentication
  • Access and refresh tokens
  • Password hashing
  • Password reset
  • Email verification
  • Change password
  • Account management
  • Active session management
  • Login history
  • Optional OAuth authentication
  • Optional two-factor authentication

πŸ‘€ User Management

Users can:

  • Create profiles
  • Update profiles
  • Upload avatars
  • Manage account settings
  • View storage usage
  • Manage active sessions
  • Configure security preferences

πŸ“ File Management

CloudVault provides complete file lifecycle management.

Upload

  • Single file upload
  • Multiple file upload
  • Large file support
  • File validation
  • MIME type validation
  • File size validation
  • Duplicate detection

File Operations

  • Upload
  • Download
  • Preview
  • Rename
  • Move
  • Copy
  • Delete
  • Restore
  • Favorite
  • Metadata management

πŸ“‚ Folder Management

Users can:

  • Create folders
  • Rename folders
  • Delete folders
  • Move folders
  • Create nested folders
  • Share folders
  • Manage folder permissions

Example:

My Drive/
β”‚
β”œβ”€β”€ Projects/
β”‚   β”œβ”€β”€ CloudVault/
β”‚   β”‚   β”œβ”€β”€ Backend/
β”‚   β”‚   β”œβ”€β”€ Frontend/
β”‚   β”‚   β”œβ”€β”€ Documentation/
β”‚   β”‚   └── Deployment/
β”‚   β”‚
β”‚   └── ML Project/
β”‚
β”œβ”€β”€ Certificates/
β”œβ”€β”€ Resume/
└── Personal/

☁️ Cloud Storage

CloudVault uses object storage for actual file storage.

The recommended architecture is:

PostgreSQL
    β”‚
    └── File Metadata

AWS S3
    β”‚
    └── Actual File Objects

The database stores information such as:

File ID
File Name
File Size
Owner
MIME Type
Checksum
Storage Key
Created At
Updated At

AWS S3 stores the actual file.

This avoids storing large binary files directly inside PostgreSQL.


πŸ”— Secure File Sharing

CloudVault supports controlled file sharing.

Users can share files with:

  • Individual users
  • Teams
  • Organizations
  • Public links

Share links can support:

  • View-only permission
  • Download permission
  • Edit permission
  • Password protection
  • Expiration date
  • Download limits
  • Revocation

Example:

Share File
    β”‚
    β”œβ”€β”€ User
    β”œβ”€β”€ Team
    └── Public Link
            β”‚
            β”œβ”€β”€ Password
            β”œβ”€β”€ Expiration
            └── Download Limit

πŸ›‘οΈ Role-Based Access Control

CloudVault uses RBAC to control access.

Example roles:

Organization Owner
        β”‚
        β”œβ”€β”€ Administrator
        β”œβ”€β”€ Manager
        β”œβ”€β”€ Editor
        └── Viewer

Possible permissions:

view
download
upload
edit
rename
move
delete
share
manage_members
manage_organization

Example:

Role View Upload Edit Delete Share Admin
Owner βœ… βœ… βœ… βœ… βœ… βœ…
Admin βœ… βœ… βœ… βœ… βœ… βœ…
Manager βœ… βœ… βœ… Limited βœ… ❌
Editor βœ… βœ… βœ… Limited Limited ❌
Viewer βœ… ❌ ❌ ❌ ❌ ❌

🏒 Organizations & Teams

CloudVault supports multi-user organizations.

An organization can contain:

Organization
β”‚
β”œβ”€β”€ Members
β”œβ”€β”€ Teams
β”œβ”€β”€ Departments
β”œβ”€β”€ Shared Drives
β”œβ”€β”€ Storage Quota
β”œβ”€β”€ Roles
β”œβ”€β”€ Permissions
└── Audit Logs

Example:

ABC Technologies
β”‚
β”œβ”€β”€ Engineering
β”‚   β”œβ”€β”€ Backend Team
β”‚   └── Frontend Team
β”‚
β”œβ”€β”€ HR
β”‚
β”œβ”€β”€ Finance
β”‚
└── Management

This allows CloudVault to support both personal and business use cases.


πŸ• File Versioning

CloudVault can maintain multiple versions of a file.

Example:

Project_Report.pdf

Version 1
Version 2
Version 3
Version 4 ← Current

Users can:

  • View versions
  • Download versions
  • Restore versions
  • Delete old versions

If a user accidentally uploads an incorrect version, an older version can be restored.


πŸ—‘οΈ Trash & Recovery

Deleted files are moved to Trash instead of being immediately destroyed.

Delete File
     ↓
   Trash
     ↓
Retention Period
     ↓
Restore / Permanent Delete

Features:

  • Restore files
  • Permanently delete files
  • Empty trash
  • Automatic cleanup

πŸ”Ž Search & Filtering

CloudVault provides file discovery through:

  • Filename search
  • File type
  • Folder
  • Owner
  • Size
  • Date
  • Tags
  • Metadata

Future versions can support semantic search.

Example:

"Find my AWS deployment documents"

πŸ“Š Storage Analytics

Users can monitor:

Storage Used
Storage Available
Total Files
Total Folders
Uploads
Downloads
Shared Files

Example:

Storage

β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘β–‘β–‘

Used: 78 GB
Available: 22 GB

Organization administrators can also monitor storage consumption.


πŸ“œ Audit Logging

CloudVault records important security and file activities.

Example actions:

LOGIN
LOGOUT
UPLOAD
DOWNLOAD
VIEW
SHARE
RENAME
MOVE
DELETE
RESTORE
PASSWORD_CHANGE
PERMISSION_CHANGE

Example audit record:

User: Raj
Action: DOWNLOAD
File: project.pdf
Time: 10:42 AM
IP: ********
Device: Chrome / Windows

Audit logs help organizations understand:

Who performed which action, on which resource, and when?


πŸ”” Notifications

CloudVault can provide:

  • In-app notifications
  • Email notifications

Examples:

A user shared a file with you.

You have been invited to an organization.

Your password was changed.

Your shared link is about to expire.

Suspicious activity was detected.

βš™οΈ Background Processing

CloudVault uses asynchronous processing for tasks that should not block normal HTTP requests.

Examples:

  • Thumbnail generation
  • Metadata extraction
  • Email notifications
  • File scanning
  • Cleanup jobs
  • Report generation
  • Large file processing

Architecture:

Django
   β”‚
   β–Ό
Redis
   β”‚
   β–Ό
Celery Worker
   β”‚
   β”œβ”€β”€ Thumbnail
   β”œβ”€β”€ Email
   β”œβ”€β”€ File Scan
   β”œβ”€β”€ Metadata
   └── Cleanup

πŸš€ Redis

Redis is used for high-speed temporary data and background processing infrastructure.

Possible uses:

  • Celery broker
  • Caching
  • Rate limiting
  • Session storage
  • Temporary tokens
  • Frequently accessed metadata

πŸ”„ Celery

Celery is responsible for background task execution.

Example:

generate_thumbnail.delay(file_id)

The task is placed into a queue.

Django
   β”‚
   β–Ό
Redis
   β”‚
   β–Ό
Celery Worker
   β”‚
   β–Ό
Generate Thumbnail

This prevents slow operations from blocking user requests.


⏰ Celery Beat

Celery Beat can schedule recurring jobs.

Example:

Every day at 02:00 AM
        ↓
Celery Beat
        ↓
Delete expired files
        ↓
Redis
        ↓
Celery Worker

Possible scheduled jobs:

  • Delete expired trash
  • Remove expired share links
  • Generate daily reports
  • Clean temporary files
  • Recalculate storage statistics

πŸ” Security

Security is a core part of CloudVault.

Security mechanisms include:

  • Secure password hashing
  • JWT authentication
  • Role-based access control
  • Object-level permissions
  • File validation
  • MIME type validation
  • File size restrictions
  • Rate limiting
  • Secure signed URLs
  • HTTPS
  • CSRF protection
  • CORS configuration
  • Security headers
  • Audit logging
  • Secure environment variables
  • Malware scanning
  • Storage access control

πŸ—οΈ System Architecture

High-level architecture:

                         INTERNET
                            β”‚
                            β–Ό
                        Load Balancer
                            β”‚
                            β–Ό
                          Nginx
                            β”‚
                            β–Ό
                       Gunicorn
                            β”‚
                            β–Ό
                         Django
                            β”‚
          β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
          β”‚                 β”‚                 β”‚
          β–Ό                 β–Ό                 β–Ό
     PostgreSQL           Redis              S3
          β”‚                 β”‚                 β”‚
          β”‚                 β–Ό                 β”‚
          β”‚          Celery Workers            β”‚
          β”‚                 β”‚                 β”‚
          β”‚        β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”        β”‚
          β”‚        β–Ό        β–Ό        β–Ό        β”‚
          β”‚      Email   Thumbnail  Scan      β”‚
          β”‚                                   β”‚
          β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

πŸ“€ File Upload Workflow

The main CloudVault workflow is:

User
 β”‚
 β–Ό
Select File
 β”‚
 β–Ό
Django API
 β”‚
 β–Ό
Authentication
 β”‚
 β–Ό
Permission Check
 β”‚
 β–Ό
File Validation
 β”‚
 β–Ό
Upload to S3
 β”‚
 β–Ό
Save Metadata in PostgreSQL
 β”‚
 β–Ό
Create Audit Log
 β”‚
 β–Ό
Queue Background Tasks
 β”‚
 β–Ό
Redis
 β”‚
 β–Ό
Celery Worker
 β”‚
 β”œβ”€β”€ Generate Thumbnail
 β”œβ”€β”€ Extract Metadata
 β”œβ”€β”€ Scan File
 └── Process Preview
 β”‚
 β–Ό
Update Database
 β”‚
 β–Ό
Notify User

πŸ“₯ File Download Workflow

User
 β”‚
 β–Ό
Request Download
 β”‚
 β–Ό
Django API
 β”‚
 β–Ό
Authentication
 β”‚
 β–Ό
Permission Check
 β”‚
 β–Ό
Generate Secure/Signed URL
 β”‚
 β–Ό
AWS S3
 β”‚
 β–Ό
File Download
 β”‚
 β–Ό
Audit Log

πŸ”— File Sharing Workflow

Owner
 β”‚
 β–Ό
Select File
 β”‚
 β–Ό
Share
 β”‚
 β–Ό
Set Permission
 β”‚
 β”œβ”€β”€ View
 β”œβ”€β”€ Download
 └── Edit
 β”‚
 β–Ό
Optional Security
 β”‚
 β”œβ”€β”€ Password
 β”œβ”€β”€ Expiration
 └── Download Limit
 β”‚
 β–Ό
Generate Share Link
 β”‚
 β–Ό
Recipient

🧩 Technology Stack

Backend

  • Python
  • Django
  • Django REST Framework

Database

  • PostgreSQL
  • SQLite for local development/testing when appropriate

Storage

  • AWS S3

Caching & Queue

  • Redis
  • Celery
  • Celery Beat

Web Server

  • Nginx
  • Gunicorn

Frontend

The frontend can be implemented using:

  • React
  • TypeScript
  • Vite
  • Tailwind CSS

DevOps

  • Docker
  • Docker Compose
  • GitHub Actions
  • AWS

Documentation

  • OpenAPI
  • Swagger
  • Markdown

πŸ“ Project Structure

cloudvault/
β”‚
β”œβ”€β”€ backend/
β”‚   β”‚
β”‚   β”œβ”€β”€ config/
β”‚   β”‚   β”œβ”€β”€ settings/
β”‚   β”‚   β”‚   β”œβ”€β”€ base.py
β”‚   β”‚   β”‚   β”œβ”€β”€ development.py
β”‚   β”‚   β”‚   └── production.py
β”‚   β”‚   β”‚
β”‚   β”‚   β”œβ”€β”€ urls.py
β”‚   β”‚   β”œβ”€β”€ asgi.py
β”‚   β”‚   β”œβ”€β”€ wsgi.py
β”‚   β”‚   └── celery.py
β”‚   β”‚
β”‚   β”œβ”€β”€ apps/
β”‚   β”‚   β”œβ”€β”€ accounts/
β”‚   β”‚   β”œβ”€β”€ organizations/
β”‚   β”‚   β”œβ”€β”€ teams/
β”‚   β”‚   β”œβ”€β”€ files/
β”‚   β”‚   β”œβ”€β”€ folders/
β”‚   β”‚   β”œβ”€β”€ sharing/
β”‚   β”‚   β”œβ”€β”€ permissions/
β”‚   β”‚   β”œβ”€β”€ versions/
β”‚   β”‚   β”œβ”€β”€ trash/
β”‚   β”‚   β”œβ”€β”€ notifications/
β”‚   β”‚   β”œβ”€β”€ activities/
β”‚   β”‚   β”œβ”€β”€ search/
β”‚   β”‚   └── analytics/
β”‚   β”‚
β”‚   β”œβ”€β”€ common/
β”‚   β”‚   β”œβ”€β”€ permissions/
β”‚   β”‚   β”œβ”€β”€ exceptions/
β”‚   β”‚   β”œβ”€β”€ pagination/
β”‚   β”‚   β”œβ”€β”€ storage/
β”‚   β”‚   └── utils/
β”‚   β”‚
β”‚   β”œβ”€β”€ tests/
β”‚   β”‚
β”‚   β”œβ”€β”€ manage.py
β”‚   β”‚
β”‚   └── requirements/
β”‚       β”œβ”€β”€ base.txt
β”‚       β”œβ”€β”€ development.txt
β”‚       └── production.txt
β”‚
β”œβ”€β”€ frontend/
β”‚
β”œβ”€β”€ infrastructure/
β”‚   β”œβ”€β”€ docker/
β”‚   β”œβ”€β”€ nginx/
β”‚   β”œβ”€β”€ aws/
β”‚   └── scripts/
β”‚
β”œβ”€β”€ docs/
β”‚   β”œβ”€β”€ architecture/
β”‚   β”œβ”€β”€ api/
β”‚   β”œβ”€β”€ database/
β”‚   β”œβ”€β”€ security/
β”‚   └── deployment/
β”‚
β”œβ”€β”€ .github/
β”‚   └── workflows/
β”‚       β”œβ”€β”€ ci.yml
β”‚       └── cd.yml
β”‚
β”œβ”€β”€ docker-compose.yml
β”œβ”€β”€ Dockerfile
β”œβ”€β”€ .env.example
β”œβ”€β”€ .gitignore
β”œβ”€β”€ LICENSE
β”œβ”€β”€ CONTRIBUTING.md
β”œβ”€β”€ SECURITY.md
β”œβ”€β”€ CODE_OF_CONDUCT.md
β”œβ”€β”€ CHANGELOG.md
└── README.md

πŸ—„οΈ Database Design

The main entities include:

User
 β”‚
 β”œβ”€β”€ Organization Membership
 β”œβ”€β”€ Files
 β”œβ”€β”€ Folders
 β”œβ”€β”€ Shares
 β”œβ”€β”€ Notifications
 └── Activity Logs

Organization
 β”‚
 β”œβ”€β”€ Members
 β”œβ”€β”€ Teams
 β”œβ”€β”€ Roles
 β”œβ”€β”€ Permissions
 └── Shared Files

File
 β”‚
 β”œβ”€β”€ Versions
 β”œβ”€β”€ Shares
 β”œβ”€β”€ Metadata
 β”œβ”€β”€ Activity Logs
 └── Storage Object

Conceptual relationship:

User
 β”‚
 β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
 β”‚               β”‚
 β–Ό               β–Ό
Organization    File
 β”‚               β”‚
 β–Ό               β”œβ”€β”€ Version
Team             β”œβ”€β”€ Share
 β”‚               β”œβ”€β”€ Metadata
 β–Ό               └── Activity
Membership

🌐 API Design

CloudVault follows RESTful API principles.

Example endpoints:

Authentication

POST   /api/v1/auth/register/
POST   /api/v1/auth/login/
POST   /api/v1/auth/refresh/
POST   /api/v1/auth/logout/
POST   /api/v1/auth/password/reset/

Files

GET    /api/v1/files/
POST   /api/v1/files/
GET    /api/v1/files/{id}/
PATCH  /api/v1/files/{id}/
DELETE /api/v1/files/{id}/
GET    /api/v1/files/{id}/download/

Folders

GET    /api/v1/folders/
POST   /api/v1/folders/
GET    /api/v1/folders/{id}/
PATCH  /api/v1/folders/{id}/
DELETE /api/v1/folders/{id}/

Sharing

POST   /api/v1/shares/
GET    /api/v1/shares/
DELETE /api/v1/shares/{id}/

Organizations

GET    /api/v1/organizations/
POST   /api/v1/organizations/
GET    /api/v1/organizations/{id}/
PATCH  /api/v1/organizations/{id}/
DELETE /api/v1/organizations/{id}/

API documentation will be provided using OpenAPI/Swagger.


🐳 Docker Architecture

CloudVault can run locally through Docker Compose.

Example services:

cloudvault-web
cloudvault-db
cloudvault-redis
cloudvault-worker
cloudvault-beat
cloudvault-nginx

Architecture:

                    Docker Compose
                          β”‚
       β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
       β–Ό          β–Ό       β–Ό        β–Ό           β–Ό
     Django   PostgreSQL Redis   Celery     Nginx
                                Worker
                                  β”‚
                               Celery Beat

βš™οΈ Local Development

1. Clone the repository

git clone https://github.com/<your-username>/cloudvault.git
cd cloudvault

2. Create virtual environment

Windows

python -m venv .venv
.venv\Scripts\activate

Linux/macOS

python3 -m venv .venv
source .venv/bin/activate

3. Install dependencies

pip install -r backend/requirements/development.txt

4. Configure environment

Copy:

.env.example

to:

.env

Configure:

SECRET_KEY=
DEBUG=
DATABASE_URL=
REDIS_URL=
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_STORAGE_BUCKET_NAME=
AWS_REGION=

5. Run migrations

python backend/manage.py migrate

6. Create superuser

python backend/manage.py createsuperuser

7. Start Django

python backend/manage.py runserver

🐳 Docker Development

Build containers:

docker compose build

Start services:

docker compose up

Run in background:

docker compose up -d

Run migrations:

docker compose exec web python manage.py migrate

Create superuser:

docker compose exec web python manage.py createsuperuser

Stop:

docker compose down

πŸ§ͺ Testing

CloudVault should maintain automated tests for:

  • Authentication
  • Authorization
  • File uploads
  • File downloads
  • File permissions
  • Folder operations
  • Sharing
  • Versioning
  • Trash
  • Organizations
  • API endpoints
  • Celery tasks
  • Storage operations

Run tests:

pytest

or:

python manage.py test

πŸ”„ CI/CD

GitHub Actions can automatically run:

Push
 β”‚
 β–Ό
GitHub Actions
 β”‚
 β”œβ”€β”€ Lint
 β”œβ”€β”€ Unit Tests
 β”œβ”€β”€ Integration Tests
 β”œβ”€β”€ Security Checks
 β”œβ”€β”€ Docker Build
 └── Deployment

Production workflow:

Developer
    β”‚
    β–Ό
Feature Branch
    β”‚
    β–Ό
Pull Request
    β”‚
    β–Ό
Code Review
    β”‚
    β–Ό
Merge
    β”‚
    β–Ό
CI
    β”‚
    β–Ό
Build Docker Image
    β”‚
    β–Ό
Deploy

🌍 Production Deployment

A production deployment can use:

                    AWS
                     β”‚
          β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
          β–Ό          β–Ό          β–Ό
        EC2         S3       PostgreSQL
          β”‚
        Nginx
          β”‚
      Gunicorn
          β”‚
        Django
          β”‚
        Redis
          β”‚
       Celery

Production infrastructure can later evolve to:

Route 53
   ↓
CloudFront
   ↓
Load Balancer
   ↓
EC2 / ECS
   ↓
Django

πŸ“ˆ Scalability Strategy

CloudVault should be designed so application components can scale independently.

Example:

                  Load Balancer
                       β”‚
          β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
          β–Ό            β–Ό            β–Ό
       Django 1     Django 2     Django 3
          β”‚            β”‚            β”‚
          β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                       β–Ό
                     Redis
                       β”‚
          β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
          β–Ό            β–Ό            β–Ό
      Worker 1      Worker 2      Worker 3

AWS S3 provides scalable object storage while application servers remain responsible primarily for metadata and API operations.


πŸ” Environment Variables

Never commit secrets to Git.

Use:

.env

Example:

SECRET_KEY=your-secret-key
DEBUG=False

DATABASE_URL=postgresql://user:password@db:5432/cloudvault

REDIS_URL=redis://redis:6379/0

AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-key
AWS_STORAGE_BUCKET_NAME=cloudvault
AWS_REGION=ap-south-1

Only commit:

.env.example

πŸ“‹ Git Workflow

Recommended branch structure:

main
 β”‚
 └── develop
       β”‚
       β”œβ”€β”€ feature/authentication
       β”œβ”€β”€ feature/file-upload
       β”œβ”€β”€ feature/file-sharing
       β”œβ”€β”€ feature/versioning
       β”œβ”€β”€ feature/organizations
       └── feature/analytics

Development process:

Issue
 ↓
Feature Branch
 ↓
Implementation
 ↓
Testing
 ↓
Pull Request
 ↓
Code Review
 ↓
Merge
 ↓
CI/CD
 ↓
Release

πŸ“ Commit Convention

CloudVault follows conventional commit-style messages.

Examples:

feat: add JWT authentication

feat: implement file upload API

feat: integrate AWS S3 storage

feat: add organization management

feat: implement secure file sharing

feat: add file versioning

fix: validate uploaded file MIME type

fix: resolve storage quota calculation

test: add file upload API tests

docs: add deployment documentation

chore: configure Docker environment

Avoid commits such as:

update
changes
final
done

πŸ“¦ Release & Versioning

CloudVault follows Semantic Versioning:

MAJOR.MINOR.PATCH

Examples:

v0.1.0
v0.2.0
v0.5.0
v1.0.0
v1.1.0
v2.0.0

Version meaning

MAJOR

Breaking changes.

v1.x.x β†’ v2.x.x

MINOR

New backward-compatible features.

v1.1.0 β†’ v1.2.0

PATCH

Bug fixes.

v1.2.0 β†’ v1.2.1

πŸ—ΊοΈ Roadmap

Phase 1 β€” Foundation

  • Django project setup
  • PostgreSQL
  • User model
  • Authentication
  • REST API
  • Basic permissions

Phase 2 β€” File Management

  • File upload
  • File download
  • File rename
  • File delete
  • Folder management
  • Trash
  • File metadata

Phase 3 β€” Cloud Storage

  • AWS S3
  • Signed URLs
  • Storage quotas
  • Large file support
  • File checksums

Phase 4 β€” Sharing

  • User sharing
  • Team sharing
  • Public links
  • Password-protected links
  • Expiring links
  • Download limits

Phase 5 β€” Background Processing

  • Redis
  • Celery
  • Celery Beat
  • Thumbnail generation
  • Email processing
  • Cleanup jobs

Phase 6 β€” Organizations

  • Organizations
  • Teams
  • Invitations
  • RBAC
  • Organization storage
  • Shared drives

Phase 7 β€” Advanced Features

  • File versioning
  • Audit logs
  • Notifications
  • Advanced search
  • Analytics
  • 2FA
  • Security monitoring

Phase 8 β€” DevOps

  • Docker
  • Nginx
  • Gunicorn
  • AWS deployment
  • CI/CD
  • Monitoring
  • Logging

Phase 9 β€” Intelligent Features

  • OCR
  • Semantic search
  • AI document assistant
  • Duplicate detection
  • Intelligent document classification
  • Anomaly detection

🧠 Future AI Features

Future versions may include:

Semantic Search

Instead of only searching filenames:

"Find my AWS deployment documents"

CloudVault can search document content and metadata.

Document Classification

Automatically categorize:

Resume
Invoice
Certificate
Contract
Report
Assignment

OCR

Extract text from scanned documents.

AI Document Assistant

Users could ask:

"Summarize this document."

"Find the payment deadline."

"What are the important points?"

These features are planned for future versions and are not required for the initial MVP.


🌟 What Makes CloudVault Different?

CloudVault is not intended to be just:

Login
Upload
Download
Delete

Instead, the platform combines:

Cloud Storage
      +
Secure Sharing
      +
RBAC
      +
Organizations
      +
Teams
      +
Version Control
      +
Audit Logging
      +
Background Processing
      +
Analytics
      +
Developer API
      +
DevOps

This makes CloudVault a practical project for learning production-oriented backend and cloud engineering.


🎯 Real-World Use Cases

Students

Store:

  • Notes
  • Projects
  • Certificates
  • Academic documents

Professionals

Store:

  • Resumes
  • Contracts
  • Reports
  • Work documents

Teams

Manage:

  • Project files
  • Documentation
  • Shared resources

Small Businesses

Manage:

  • HR documents
  • Finance documents
  • Business reports
  • Contracts

Organizations

Manage:

  • Departments
  • Users
  • Permissions
  • Shared drives
  • Audit logs

πŸ† Learning Objectives

Building CloudVault provides practical experience with:

Python

  • OOP
  • Modules
  • Exception handling
  • Async/background processing concepts

Django

  • Models
  • Views
  • URLs
  • Middleware
  • Authentication
  • Permissions
  • Django ORM
  • REST APIs

Databases

  • PostgreSQL
  • Relationships
  • Indexing
  • Transactions
  • Query optimization

Cloud

  • AWS S3
  • EC2
  • IAM
  • Networking concepts
  • Cloud architecture

Backend Engineering

  • REST API
  • Authentication
  • Authorization
  • Caching
  • File processing
  • Distributed task processing

DevOps

  • Docker
  • Nginx
  • Gunicorn
  • CI/CD
  • Linux
  • Deployment

System Design

  • Scalability
  • Reliability
  • Security
  • Fault tolerance
  • Background processing

🀝 Contributing

CloudVault is intended to be open source and welcomes contributions.

Contribution workflow

Fork
 ↓
Clone
 ↓
Create Branch
 ↓
Implement Feature
 ↓
Write Tests
 ↓
Commit
 ↓
Push
 ↓
Pull Request
 ↓
Code Review

Example:

git clone https://github.com/<your-username>/cloudvault.git

cd cloudvault

git checkout -b feature/file-sharing

After development:

git add .
git commit -m "feat: implement secure file sharing"
git push origin feature/file-sharing

Then open a Pull Request.


πŸ“œ Open Source

CloudVault is built using widely adopted open-source technologies.

Core technologies include:

  • Python
  • Django
  • Django REST Framework
  • PostgreSQL
  • Redis
  • Celery
  • React
  • Docker
  • Nginx
  • Gunicorn

Third-party dependencies should be used according to their respective licenses.


πŸ“„ License

This project is licensed under the MIT License.

See:

LICENSE

for more information.


πŸ”’ Security Policy

If you discover a security vulnerability, please do not publicly disclose it through a GitHub issue.

Instead, follow the instructions in:

SECURITY.md

Security-related contributions are highly appreciated.


πŸ“š Documentation

Detailed documentation will be maintained under:

docs/
β”œβ”€β”€ architecture/
β”œβ”€β”€ api/
β”œβ”€β”€ database/
β”œβ”€β”€ security/
└── deployment/

Planned documentation:

  • System Architecture
  • Database Design
  • API Documentation
  • Authentication Flow
  • Authorization Model
  • File Upload Flow
  • AWS S3 Architecture
  • Redis Architecture
  • Celery Architecture
  • Docker Setup
  • Production Deployment
  • Security Guide
  • Contribution Guide

πŸ“Š Project Status

Status: 🚧 Active Development
Version: 0.1.0

CloudVault is being developed incrementally, starting with the core file-management system and gradually expanding toward a production-oriented cloud platform.


πŸ“Œ Development Philosophy

CloudVault follows these principles:

Security First
    ↓
Clean Architecture
    ↓
Testable Code
    ↓
API-First Design
    ↓
Scalable Infrastructure
    ↓
Automation
    ↓
Documentation

The goal is not simply to build more features, but to understand why each component exists and how the complete system works together.


πŸš€ Long-Term Vision

The long-term goal is to evolve CloudVault into a complete cloud document and collaboration platform.

                         CLOUDVAULT
                             β”‚
        β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
        β–Ό                    β–Ό                    β–Ό
   Personal Storage      Team Storage       Organization
        β”‚                    β”‚                    β”‚
        β–Ό                    β–Ό                    β–Ό
      Files              Collaboration          RBAC
        β”‚                    β”‚                    β”‚
        β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                             β–Ό
                        Secure Sharing
                             β”‚
                             β–Ό
                         Versioning
                             β”‚
                             β–Ό
                        Audit Logging
                             β”‚
                             β–Ό
                    Background Processing
                             β”‚
                             β–Ό
                       Cloud Storage
                             β”‚
                             β–Ό
                      Scalable Backend
                             β”‚
                             β–Ό
                       Intelligent Search

⭐ Why CloudVault?

CloudVault is designed as more than a traditional Django CRUD project.

It demonstrates how a modern application can combine:

Python + Django + REST APIs + PostgreSQL + Redis + Celery + AWS S3 + Docker + Nginx + Gunicorn + CI/CD + Security + System Design

into one cohesive production-oriented platform.


πŸ‘¨β€πŸ’» Project Author

Raj Shekhar

Computer Science Engineering Student Backend & Cloud Engineering Enthusiast

Interested in:

  • Python
  • Django
  • Backend Engineering
  • Cloud Computing
  • DevOps
  • System Design
  • Distributed Systems

⭐ Support the Project

If you find CloudVault useful:

  • ⭐ Star the repository
  • 🍴 Fork the project
  • πŸ› Report issues
  • πŸ’‘ Suggest features
  • πŸ”§ Submit pull requests
  • πŸ“– Improve documentation

☁️ CloudVault

Store securely. Share confidently. Collaborate efficiently.

Built with Python, Django, PostgreSQL, Redis, Celery, Docker, and AWS.

Quick start (local)

  1. Create and activate a virtualenv:
python -m venv .venv
.\.venv\Scripts\Activate.ps1
  1. Install dependencies:
python -m pip install -r requirements.txt
  1. Run migrations:
python manage.py makemigrations
python manage.py migrate
  1. Start ASGI server (recommended) to enable WebSockets:
.\run_asgi.ps1    # uses daphne by default
# or: .\run_asgi.ps1 -Server uvicorn
  1. Open http://127.0.0.1:8000/ and log in.

Pushing to GitHub (recommended automated commits)

  • Use the provided script to create up to 100 professional commits and push them.
# Dry run (preview planned commits)
.\scripts\create_commits.ps1 -RemoteUrl "git@github.com:USERNAME/REPO_NAME.git" -MaxCommits 100 -DryRun

# Execute and push
.\scripts\create_commits.ps1 -RemoteUrl "git@github.com:USERNAME/REPO_NAME.git" -MaxCommits 100 -PushAfterCommit

CI

  • A basic GitHub Actions workflow is included in .github/workflows/ci.yml that installs requirements, runs migrations and runs tests.

Notes

  • Ensure .env or other secret files are never committed. .gitignore includes .env.
  • For production, configure SECRET_KEY, DEBUG=False, ALLOWED_HOSTS, and a production-ready DATABASES (Postgres) and REDIS_URL for Channels.

License

  • Internal/Proprietary (change as needed)

About

CloudVault is a secure cloud-based file management and collaboration platform that enables individuals, teams, and organizations to store, organize, share, version, and manage digital files with controlled access, audit logging, background processing, and scalable cloud infrastructure.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages