v1.0 · DiffSinger + SoundFont + LLM Agent

The AI-Powered
Music Creation IDE

MusIDE is a professional DAW alternative with AI at its core. Compose entire songs from lyrics — singing synthesis via DiffSinger, realistic instrument rendering via SoundFont, and full-process orchestration by an LLM Agent with 20+ tool calls.

Get Started → Learn More
DiffSinger
AI Singing Synthesis
5 SF2
SoundFont Libraries
20+
LLM Agent Tools
16
DAW Tracks

Core Features

From a single lyric to a complete song — MusIDE handles the entire creative process with AI.

🎤

DiffSinger AI Singing

Industry-grade DiffSinger (AAAI-2022) singing voice synthesis. Input lyrics + MIDI notes/durations to generate singing vocals. Uses OpenCPOP pretrained models, supports K_step=100 ultra-fast diffusion, NSF-HiFiGAN vocoder, and pitch shifting via librosa.

🎹

SoundFont Realistic Rendering

Professional instrument synthesis via FluidSynth + SF2 SoundFont. 5 premium tone libraries covering all scenarios: Salamander C5 (24MB), GigaPiano (17MB), FluidR3 GM+GS full set (144MB), GeneralUser GS (30MB), TimGM6mb minimal (6MB). Full 128 General MIDI programs + drum kits supported.

🤖

LLM Intelligent Orchestration

Full-context LLM dialogue intelligence with support for any OpenAI-compatible API. The AI Agent executes professional system prompts, with 20+ tool calls covering audio processing/editing/synthesis, project control, AI composition, etc. Generate complete songs directly from conversation.

🎚️

Multi-Track DAW Editor

Professional multi-track editor built with Web Audio API and Canvas, supporting up to 16 tracks. Each track has independent properties: name, color, icon, instrument, volume, pan. Standard operations: add/delete/drag-move/cut, double-click for detailed note editing.

⏱️

Timeline & Playback

Horizontal timeline with BPM-based bar/beat grid (default 120 BPM, 4/4 time). Real-time moving playhead, timeline zoom, loop sections. Complete transport controls: play/pause/stop/record/loop, time display, playback control, BPM real-time adjustment.

🎵

Note Editing & Piano Roll

Piano roll note editor with drag-select, batch move/delete, snap-to-grid. Supports 3 octave ranges (C3-B5), pitch adjustment, note add/delete. Real-time pitch and duration feedback during editing.

📊

Mixing Console

Tab-style vertical mixing console view for each channel: independent volume, pan slider, VU meter peak values real-time updated. Mute/solo buttons, channel grouping, batch operations. Real-time audio via AudioContext, supporting multiple simultaneous plays.

🎼

Music Theory Engine

Built-in complete music theory module supporting 18 keys (major/minor/Harmonic minor/Japanese scale, etc.), 21 chord types, 20+ chord progressions, 7 cadence types, 4 typical structure templates. AI Agent automatically selects appropriate key, mode, and chord progressions based on style during composition.

🎙️

Audio Source Separation

AI source separation via Demucs (customizable: vocals/drums/bass/other). Whisper-based speech recognition for audio-to-text transcription. Background thread async processing with real-time progress updates.

📁

Project Management

Full file manager: create/upload/download/delete with auto audio file detection. Complete Git integration (status/log/branch/stage/commit/push/pull/checkout/clone/diff/stash). Built-in terminal, project settings, search & replace, mobile-optimized responsive UI.

Quick Start

One-command install on any platform. Default address: http://localhost:12346

# Windows (PowerShell) irm https://music.samai.cc/install.ps1 | iex # Linux / macOS / Termux curl -fsSL https://music.samai.cc/install.sh | bash # Manual install pip install flask flask-cors # Clone from your authorized repository git clone cd muside python muside_server.py # → open http://localhost:12346 # Custom install directory MUSIDE_INSTALL_DIR=~/my-muside curl -fsSL https://music.samai.cc/install.sh | bash # Custom port MUSIDE_PORT=8080 curl -fsSL https://music.samai.cc/install.sh | bash # Docker docker run -d -p 12346:12346 -v ~/muside_workspace:/workspace python:3.12-slim bash -c \ "pip install flask flask-cors && git clone --depth 1 /muside && cd /muside && MUSIDE_WORKSPACE=/workspace python3 muside_server.py"

Cross-Platform

One codebase, runs everywhere Python runs.

🪟
Windows 10/11
PowerShell one-line install
🐧
Linux
Ubuntu / Debian / Fedora / CentOS / Arch / openSUSE / Alpine
🍎
macOS
Homebrew Python
📱
Termux (Android)
pkg install python
🐳
Docker
python:3.12-slim image
💻
WSL
Windows Subsystem for Linux

Architecture

A modular Flask service with rich route blueprints and a Catppuccin-themed web UI.

samaidev/muside/ ├── muside_server.py # Flask app entry, registers blueprints ├── utils.py # Common utilities ├── music_theory.py # Music theory (keys/chords/progressions/structures) ├── sf2_renderer.py # SF2 SoundFont renderer + MIDIBuilder + GM mapping ├── diffsinger_svs.py # DiffSinger AI singing synthesis (AAAI-2022) ├── audio_tools.py # Audio processing utilities ├── requirements.txt # Python dependencies ├── install.sh / install.ps1 # Full-auto installers (with DiffSinger) ├── start.sh # Launch script ├── soundfonts/ # SF2 tone libraries │ ├── SalC5Light2.sf2 # Salamander C5 (24MB) │ ├── GigaPiano.sf2 # Giga Piano (17MB) │ ├── FluidR3_GM_GS.sf2 # FluidR3 full set (144MB) │ ├── GeneralUser_GS.sf2 # GeneralUser GS (30MB) │ └── TimGM6mb.sf2 # TimGM6mb minimal (6MB) ├── diffsinger_repo/ # DiffSinger model repo (AAAI-2022) │ └── checkpoints/ # Pretrained models (OpenCPOP, NSF-HiFiGAN, etc.) ├── routes/ │ ├── files.py # File CRUD │ ├── run.py # Code execution │ ├── git.py # Git operations │ ├── chat.py # LLM chat + AI Agent (20+ tools) │ ├── audio.py # Audio: AI synthesis, source separation │ ├── venv.py # Virtual environment management │ ├── server_mgmt.py # Server management │ └── browser.py # Browser toolkit └── static/ ├── index.html # Main DAW interface ├── css/style.css # Catppuccin theme, responsive └── js/ ├── app.js # Main controller ├── track-editor.js # Multi-track DAW editor ├── chat.js # LLM chat + Agent ├── files.js # File manager ├── git.js # Git panel ├── search.js # Global search & replace └── terminal.js # Built-in terminal