📝 Background & Objective
Our various Zoomcamp courses (Machine Learning, Data Engineering, MLOps, LLM Zoomcamp, and AI Dev Tools) contain hundreds of hours of highly technical video content.
Currently, navigating this material requires watching or re-watching videos. We want to build a full knowledge layer on top of all Zoomcamp courses. The goal is to generate structured, high-quality written notes for every video, create a shared vocabulary of key concepts, and introduce an automated internal linking system.
By combining written notes, a global glossary, cross-linking, and structured SEO data, we will drastically improve course navigation, the student learning experience, and our long-term search engine visibility.
✅ Core Requirements & Implementation Steps
This is a large-scale project that touches data pipeline automation, Markdown generation, and SEO architecture. You can contribute to the entire system or focus on a specific module below.
1. Automated Notes Generation Pipeline
We need an automated process that turns raw video content into structured written drafts.
- Extraction: Fetch automated transcripts directly from the YouTube videos across our various Zoomcamp playlists.
- Processing & LLM Generation: Preprocess the transcripts and use an LLM to produce structured notes.
- CI/CD Integration: Eventually, this pipeline should be packaged as a GitHub Action that can automatically run and refresh notes whenever courses update or new videos are published. (Note: Automation generates initial drafts; contributors will later manually refine them).
2. Unified Notes Repository & Structure
The output of the automated pipeline must be cleanly organized into a dedicated repository structure.
- Folder Hierarchy: Organized logically by
Course → Module → Video.
- Markdown Formatting: Every generated note must follow a unified Markdown template that includes:
- A clear summary of the video.
- Key concepts and detailed explanations.
- Links to related notes and global vocabulary entries.
3. Internal Linking System (Knowledge Graph)
To make navigation seamless, the system must automatically connect related concepts across different modules and courses.
- Detection: Automatically detect related notes using concept extraction, keyword matching, or text embeddings.
- Output: Automatically append a “Related Notes” section to the bottom of each generated Markdown file. This creates a lightweight knowledge graph inside the repository, helping learners and search engines understand how lessons relate to one another.
4. SEO & Structured Data Architecture
To maximize discoverability, the generated notes must be built for modern search engines.
- JSON-LD: Inject JSON-LD structured data into every notes page and vocabulary term. This specifically helps Google interpret the pages as educational resources, definitions, or course materials.
- Metadata: Programmatically generate clear metadata (SEO titles, meta-descriptions, alt text).
- Sitemap: Ensure the pipeline updates or generates a sitemap for the new knowledge layer.
Sources
The automated pipeline will extract lesson videos directly from the modules linked within each course's respective GitHub repository.
This project covers the following Zoomcamps:
📝 Background & Objective
Our various Zoomcamp courses (Machine Learning, Data Engineering, MLOps, LLM Zoomcamp, and AI Dev Tools) contain hundreds of hours of highly technical video content.
Currently, navigating this material requires watching or re-watching videos. We want to build a full knowledge layer on top of all Zoomcamp courses. The goal is to generate structured, high-quality written notes for every video, create a shared vocabulary of key concepts, and introduce an automated internal linking system.
By combining written notes, a global glossary, cross-linking, and structured SEO data, we will drastically improve course navigation, the student learning experience, and our long-term search engine visibility.
✅ Core Requirements & Implementation Steps
This is a large-scale project that touches data pipeline automation, Markdown generation, and SEO architecture. You can contribute to the entire system or focus on a specific module below.
1. Automated Notes Generation Pipeline
We need an automated process that turns raw video content into structured written drafts.
2. Unified Notes Repository & Structure
The output of the automated pipeline must be cleanly organized into a dedicated repository structure.
Course→Module→Video.3. Internal Linking System (Knowledge Graph)
To make navigation seamless, the system must automatically connect related concepts across different modules and courses.
4. SEO & Structured Data Architecture
To maximize discoverability, the generated notes must be built for modern search engines.
Sources
The automated pipeline will extract lesson videos directly from the modules linked within each course's respective GitHub repository.
This project covers the following Zoomcamps: