Midra

简体中文 | English

Midra is an agentic prompt-to-code MIDI composition framework for editable and controllable music generation.

It turns natural language prompts into structured music code, then renders transparent MIDI files that can be inspected, modified, and reused — unlike opaque end-to-end AI audio generation.

Prompt it. Code it. MIDI it.

Why Midra

Most music-generation systems produce audio directly from prompts, which is powerful but often opaque.

Midra introduces a code-first composition primitive:

• It compiles natural language prompts into structured, inspectable music code.
• It keeps intermediate planning artifacts as checkpoint JSON files.
• It renders editable MIDI so composition can be iterated with full control.

This makes Midra suitable for experiments in agentic composition, human-in-the-loop editing, and reproducible prompt-to-MIDI workflows.

Core Modes

1) LLM Note Mode (--note-mode llm)

Per-track note events are generated by model planning for maximum prompt alignment and variation.

2) Rule Note Mode (--note-mode rule)

Per-track note events are generated by deterministic generators for stable and reproducible outputs.

What makes it different

• Prompt-to-code first: turns prompts into structured symbolic music, not opaque waveforms.
• Checkpoint-native pipeline: each stage is persisted and resumable.
• Editable artifacts: MIDI + JSON outputs can be inspected, modified, and reused.
• Agentic orchestration: planning and rendering are coordinated in explicit stages.

Architecture at a glance

• music_agent/agents/: intent/song/arrangement/note planning agents
• music_agent/prompts/: centralized prompt templates
• music_agent/generators/: rule generators for drums/bass/chords/lead
• music_agent/core/: schema, MIDI IR assembly, renderer, validator
• music_agent/utils/: audio rendering helpers (fluidsynth / ffmpeg)
• outputs/projects/: per-project checkpoints and final outputs

Quick Start

1) Install Dependencies

sudo apt install ffmpeg
sudo apt install fluidsynth fluid-soundfont-gm
pip install -r requirements.txt
pip install -e .

2) Configure Environment

cp .env.example .env

Use defaults from .env.example:

• OPENAI_API_KEY
• OPENAI_BASE_URL
• OPENAI_MODEL

Optional:

• MAX_MUSIC_DURATION_SECONDS

3) Docker (Frontend + Backend)

Required configuration

1. Configure backend runtime values in backend/config.yaml.
2. Provide OpenAI credentials via shell env (or .env loaded by Docker Compose):

export OPENAI_API_KEY="your_key"
export OPENAI_BASE_URL="https://api.openai.com/v1"
export OPENAI_MODEL="gpt-5.5"

3. Ensure host paths exist for mounted data:

mkdir -p backend/data outputs

Start with Docker Compose

docker compose up --build -d

Access URLs

• Frontend: http://localhost:5173
• Backend API: http://localhost:8000

Stop

docker compose down

4) Run (CLI)

midra "generate a 30 seconds cyberpunk boss battle bgm with drums bass chords and lead" --project-name demo --project-id test001

5) Resume (CLI)

midra "generate a 30 seconds cyberpunk boss battle bgm with drums bass chords and lead" --project-name demo --project-id test001 --resume

Output

Generated artifacts are stored under:

outputs/projects/demo_test001/

This folder contains checkpoint JSON stages and final rendered outputs.

Roadmap

• Better long-form composition structure and motif consistency
• Stronger controllability with richer prompt constraints
• Enhanced track-level diagnostics and observability
• More export/render options and workflow integrations

Star This Project

If the idea of agentic, editable prompt-to-MIDI generation resonates with you, please star Midra.

It helps the project attract contributors and accelerate open research in controllable music agents.

License

See LICENSE.