Nightingale

Karaoke from any song in your music library, powered by neural networks.

Nightingale scans your music folder, separates lead vocals from instrumentals using the UVR Karaoke model (or Demucs), transcribes lyrics with word-level timestamps via WhisperX, and plays it all back with synchronized highlighting, pitch scoring, key/tempo controls, profiles, and dynamic backgrounds.

Ships as a single binary. No manual installation of Python, ffmpeg, or ML models required — everything is downloaded and bootstrapped automatically on first launch.

Key Features

• Stem Separation — isolates lead vocals from instrumentals
• Word-Level Lyrics — automatic transcription with alignment
• CJK Lyrics — Japanese / Chinese / Korean songs get per-character forced alignment and romanized readings (Hepburn, pinyin, Revised Romanization) above each token
• Pluggable ASR — Whisper (default) or Parakeet v3 (experimental, ~25 European languages)
• UltraStar Deluxe Songs (experimental) — drop USDX folders into your library and play them with their built-in pitch + lyric data
• Pitch Scoring — real-time microphone input with star ratings
• Key & Tempo Shifts — adjust analyzed songs to better fit your voice
• Profiles — per-player score tracking
• Video Files — use video files with synchronized background playback
• Audio-Reactive Backgrounds — 10 GPU shaders that react to your mic, 5 Pixabay video flavors, source-video for video files
• Sidebar Filters + Analyze All — quickly browse and batch-analyze your library
• Mic Mirroring — optional low-latency live mic monitoring with adjustable gain (0–200%)
• Gamepad Support — full navigation via gamepad
• Self-Contained — zero manual dependency setup

Supported Platforms

Getting Started

Download

Download the latest version from the Releases page.

Supported audio formats: .mp3, .flac, .ogg, .wav, .m4a, .aac, .wma.

Supported video formats: .mp4, .mkv, .avi, .webm, .mov, .m4v.

UltraStar Deluxe songs (.usdx, plus .txt files whose contents look like USDX) are also picked up automatically and bypass the analyzer pipeline entirely. See UltraStar Deluxe for the supported tags and folder layout.

macOS: Removing the Quarantine Flag

macOS automatically adds a quarantine attribute to files downloaded from the internet. Since Nightingale is not signed with an Apple Developer ID, Gatekeeper will block it with a message like “app is damaged and can’t be opened” or “Apple cannot check it for malicious software”.

To fix this, remove the quarantine attribute after moving the Nightingale.app to Applications:

xattr -cr /Applications/Nightingale.app

This tells macOS to clear (-c) all extended attributes recursively (-r) from the app bundle, which removes the com.apple.quarantine flag that triggers Gatekeeper. The app itself is safe — it’s just not code-signed.

First Launch

On first launch, Nightingale will guide you through setup:

1. Choose data folder — select where cache, models, videos, vendor tools, and the library database are stored
2. Downloads ffmpeg — needed for audio/video processing
3. Downloads uv — Python package manager
4. Installs Python 3.10 — via uv, isolated from your system Python
5. Creates virtual environment — with PyTorch, WhisperX, Demucs, and UVR models
6. Downloads ML models — stem separation and transcription models
7. Pre-downloads video backgrounds — Pixabay videos for the first session

This process takes a few minutes and shows a progress screen. After setup completes, Nightingale is ready to use.

Adding Music

When prompted, select your music folder. Nightingale scans it for supported audio and video files. You can change this folder later from the sidebar actions menu.

Analysis

Before a song can be played as karaoke, it needs to be analyzed:

1. Select a song from the library
2. Analysis runs automatically (stem separation → lyrics → transcription)
3. Results are cached — subsequent plays are instant

You can also batch analysis with Analyze All from the song list toolbar.

Updating

On macOS and Windows, Nightingale checks for new releases once at launch. When one is available:

1. The sidebar avatar grows a small green dot.
2. The Update entry in the sidebar dropdown menu also shows the badge and opens a dialog with the version, release date, and notes.
3. Click Install & Restart. The signed bundle downloads (with a progress bar), installs, and the app relaunches.

Platform notes:

• macOS: the bundle is replaced in place and the app relaunches.
• Windows: the installer runs in passive mode — a small progress window appears, the app exits, and it comes back automatically once the install finishes.

Linux

The Linux build ships without the in-app updater. The Update entry is still in the sidebar menu, but instead of fetching a bundle it opens a dialog that explains this and gives you an Open GitHub Releases button. Pick the .deb or .rpm for your distro from the Releases page and install it the usual way.

There is no update badge on the sidebar avatar on Linux.

If the macOS or Windows dialog reports an error, see Troubleshooting → Updates.

Force Re-setup

If something goes wrong with setup or dependencies, open the sidebar actions menu and select Re-run Setup.

Controls

Nightingale supports both keyboard and gamepad input. The UI adapts to your input method automatically.

Navigation

Playback

Key and tempo are adjusted from the song list controls after a song has been analyzed.

Gamepad Notes

• Full navigation of menus, song selection, and settings via gamepad
• D-pad and left stick both work for navigation
• Face buttons map to confirm (A/South) and cancel (B/East)

Keyboard & Gamepad Reference

This page provides a complete reference of all keyboard shortcuts and gamepad mappings.

Menu Navigation

In the main menu, sidebar, and settings screens:

• Arrow keys / D-pad: Move focus between items
• Enter / A button: Select the focused item
• Escape / B button: Go back or close overlays
• Tab: Switch between sidebar and main content area
• Type any text: Filter/search the song list
• Analyze All button: Queue analysis for the current filtered list

During Playback

While a song is playing:

• Space / Start: Pause or resume playback
• Escape / B: Exit back to the song menu
• G: Toggle guide vocals on/off
• + / -: Adjust guide vocal volume
• T: Cycle through background themes (shaders, video, source)
• F: Cycle through Pixabay video flavors (Nature, Underwater, Space, City, Countryside)
• M: Toggle microphone for pitch scoring
• N: Switch to the next available microphone
• R: Toggle mic mirroring (live monitor during playback)
• F11: Toggle fullscreen

Song List Controls

After a song is analyzed, you can use the on-row controls to shift:

• Tempo: adjust playback speed in small steps
• Key: transpose to a more comfortable vocal range

Skip Buttons

During playback, on-screen skip buttons appear for intro and outro sections. These can be activated with Enter/A or clicked.

How It Works

Nightingale’s pipeline transforms any audio or video file into a karaoke experience through several stages.

Pipeline Overview

flowchart TD
    A["🎵 Audio or video file"] --> B["UVR Karaoke / Demucs"]
    A2["🎼 USDX bundle (.txt / .usdx)"] --> E["Tauri App (Rust + React)"]
    B --> |"vocals + instrumental"| C["LRCLIB"]
    C --> |"synced lyrics if available"| D["WhisperX (large-v3) or Parakeet v3 (exp.)"]
    D --> |"word-level alignment, CJK readings"| E
    E --> F["🎤 Plays instrumental + synced lyrics\nwith pitch scoring, key/tempo controls,\nmic mirroring, and audio-reactive backgrounds"]

USDX bundles bypass stem separation and transcription entirely — the .txt is parsed into a transcript JSON shaped exactly like the analyzer cache, so playback reuses the existing pipeline. See UltraStar Deluxe.

Analyzer Server

The analyzer is a long-lived Python process that Nightingale spawns once on startup and talks to over a token-authenticated loopback TCP socket using newline-delimited JSON (NDJSON). Per-song startup costs (model load, CUDA init, Python imports) are paid once at boot, after which analyze requests stream progress events and complete with done or error messages. This makes back-to-back analyses noticeably faster than the previous per-song subprocess model.

Caching

Analysis results are cached in your configured data folder (cache/) using blake3 file hashes. Re-analysis only happens if the source file changes, if you trigger it manually, or when creating shifted key/tempo variants.

Hardware Acceleration

The Python analyzer uses PyTorch and auto-detects the best backend:

The UVR Karaoke model uses ONNX Runtime and enables CUDA acceleration automatically on NVIDIA GPUs, or CoreML on Apple Silicon.

A song typically takes 2–5 minutes on GPU, 10–20 minutes on CPU.

Stem Separation

Nightingale separates lead vocals from instrumentals so you can sing along to the backing track.

Models

UVR Karaoke (Default)

The UVR (Ultimate Vocal Remover) Karaoke model is optimized specifically for karaoke use. It preserves backing vocals in the instrumental track, giving a more natural karaoke experience. Uses ONNX Runtime for inference, with automatic CUDA (NVIDIA) or CoreML (Apple Silicon) acceleration.

Demucs

Demucs by Facebook Research provides an alternative separation model. You can switch between models in the settings.

Video Files

When processing video files (.mp4, .mkv, etc.), Nightingale first extracts the audio track using ffmpeg, then runs stem separation on the extracted audio. During playback, Nightingale uses the source video when available and falls back to a playable converted video when needed.

Guide Vocals

After separation, you can control how much of the lead vocals bleed through the instrumental:

• Toggle: Press G to turn guide vocals on/off
• Volume: Press + / - to adjust the guide vocal level

This is useful for learning new songs or for singers who want a reference pitch.

UltraStar Bundles Skip Stem Separation

UltraStar Deluxe songs that ship with #VOCALS and #INSTRUMENTAL files are used directly during playback — no stem separation is run. If only #AUDIO is provided, Nightingale falls back to running the configured separator on that file. See UltraStar Deluxe for the supported tags.

Lyrics & Transcription

Nightingale provides word-level synchronized lyrics through two sources.

LRCLIB

LRCLIB is queried first for existing synced lyrics. When a match is found, lyrics are used directly without needing transcription. This is faster and often more accurate for well-known songs.

WhisperX Transcription

When LRCLIB doesn’t have lyrics for a song, Nightingale runs ASR over the isolated vocals to:

1. Transcribe the audio into text
2. Align each word to precise timestamps

This produces word-level timing information that drives the karaoke highlighting during playback.

Choosing the ASR Engine

Two ASR engines are available, switchable from Settings → Analysis:

Whisper (default)

Uses WhisperX with the large-v3 model. Broad language coverage, robust on noisy or multilingual material. This is the default and the recommended choice for most users.

Parakeet v3 (experimental)

Parakeet TDT 0.6B v3 by NVIDIA. Smaller and noticeably faster than Whisper large-v3. Two interchangeable backends are picked automatically based on your runtime device:

• CUDA → NVIDIA NeMo (nemo_toolkit[asr])
• CPU / MPS → ONNX Runtime via onnx-asr

Parakeet is supported for the following 25 European languages: Bulgarian, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, German, Greek, Hungarian, Italian, Latvian, Lithuanian, Maltese, Polish, Portuguese, Romanian, Russian, Slovak, Slovenian, Spanish, Swedish, Ukrainian.

If Parakeet is selected for an unsupported language, or it produces no usable words for a song, Nightingale falls back to Whisper for that file. Word-level alignment after Parakeet still uses wav2vec2 forced alignment, so timing accuracy is comparable.

Language Support

The language is auto-detected from the audio. You can override it per song from the song-list controls. Nightingale includes Noto Sans CJK fonts for Chinese, Japanese, and Korean lyrics.

CJK Languages

Japanese (ja) and Chinese (zh) take a dedicated forced-alignment path because their wav2vec2 alignment models are character-level CTC checkpoints, not word/space-segmented. Nightingale:

1. Transcribes with Whisper or Parakeet as usual.
2. Cleans the text down to the alignment vocab (drops punctuation and other out-of-vocab symbols).
3. Aligns per character with a wav2vec2 model:
   • Japanese: vumichien/wav2vec2-large-xlsr-japanese-hiragana — feeds fugashi-derived hiragana readings into a hiragana-vocab CTC model. This sidesteps the dense kanji vocabulary of the default checkpoint and matches the acoustic prior of natural Japanese speech far better.
   • Chinese: jonatasgrosman/wav2vec2-large-xlsr-53-chinese-zh-cn with jieba tokenization for display.
4. Reattributes per-character timing back onto fugashi (ja) or jieba (zh) tokens for word-level highlighting.

Korean (ko) uses kresnik/wav2vec2-large-xlsr-korean, which is already eojeol-segmented and bypasses the character-retokenization step.

For all three languages, every word is annotated with a romanized reading that appears above the original token during playback:

• Japanese — Hepburn romaji via pykakasi
• Chinese — tone-mark pinyin via pypinyin
• Korean — academic Revised Romanization via hangul-romanize

Heavy CJK modules are imported lazily on first use, so non-CJK songs don’t pay the startup cost.

Highlighting

During playback, lyrics are displayed with word-by-word highlighting:

• Current word — highlighted in the accent color
• Sung words — shown in a completed state
• Upcoming words — shown in a dimmer color
• Next line — previewed below the current line
• Reading — for CJK songs, the romanized reading is shown above each token in a smaller weight

UltraStar Deluxe

Nightingale can play UltraStar Deluxe songs directly from your library, using the pitch and lyric data shipped with each song instead of running its own analyzer. This support is experimental.

How songs are detected

The library scanner picks up two kinds of USDX files:

• *.usdx files — always treated as USDX.
• *.txt files — only when the file’s contents look like a USDX header (so unrelated .txt notes in your music folder are ignored).

Sibling media referenced from the file (#AUDIO, #MP3, #VOCALS, #INSTRUMENTAL, #VIDEO) are de-duped from the scan, so they don’t appear as standalone tracks alongside the USDX song.

Folder layout

A typical USDX bundle keeps every file in one folder:

my-songs/
└── Some Artist - Some Song/
    ├── song.txt
    ├── song.mp3        # referenced via #AUDIO / #MP3
    ├── song.vocals.mp3 # referenced via #VOCALS  (optional)
    ├── song.instr.mp3  # referenced via #INSTRUMENTAL (optional)
    ├── song.mp4        # referenced via #VIDEO (optional)
    └── cover.jpg       # referenced via #COVER (optional)

All sibling paths are resolved relative to the folder containing the .txt. If a referenced file is missing, the corresponding feature is skipped (e.g. no #VIDEO means no source-video background).

Supported tags

Nightingale reads the following header tags. Anything else is ignored:

*Either #AUDIO or #MP3 must resolve to a real file.

Timing model

UltraStar’s #BPM counts quarter-note beats per minute, and a note beat is 1/4 of one of those — so seconds_per_note_beat = 60 / (BPM * 4). #GAP is a flat millisecond offset added to every timestamp. This matches the formula used by UltraStar Deluxe, Vocaluxe, and Performous, so files authored against any of those tools should play in sync.

Note kinds (: regular, * golden, F freestyle, R / G rap) are recognised by the parser, but pitch-bonus weighting is currently uniform across all of them.

How USDX songs flow through the pipeline

1. The .txt is parsed into a transcript JSON whose shape is identical to what the analyzer cache writes for non-USDX songs, so playback reuses the existing transcript and lyric path with no special-casing.
2. If #VOCALS and #INSTRUMENTAL are both provided, stem separation is skipped entirely. Otherwise, the configured separator runs on #AUDIO once and the result is cached like any other song.
3. If #VIDEO is set, it plays as the source background. The other shader and Pixabay backgrounds are still available with T.
4. Re-scanning the library does not re-parse unchanged USDX files; transcript synthesis is keyed off the file’s blake3 hash like every other cache.

Limitations

• USDX support is experimental. File a bug if you hit a song that doesn’t parse.
• Medley sections are not yet supported.
• Duet voice tracks (P1 / P2) are parsed as a single voice; per-singer scoring is not differentiated.
• Encodings beyond UTF-8 and the most common Latin code pages may fail to decode cleanly.
• Songs missing any of the required tags (#TITLE, #ARTIST, #AUDIO/#MP3, #BPM) are rejected at scan time.

Where to find songs

• usdb.animux.de — large community song database.
• usdx.eu/format — the format reference, useful if you want to author your own files.

Pitch Scoring

Nightingale includes real-time pitch scoring to gamify the karaoke experience.

How It Works

1. Microphone input — select your microphone and toggle it on with M
2. Pitch detection — your vocal pitch is analyzed in real-time
3. Comparison — your pitch is compared against the reference vocal track
4. Scoring — accuracy is tracked throughout the song

Star Ratings

At the end of each song, you receive a star rating based on your overall pitch accuracy. Ratings are saved to your profile’s scoreboard.

Results

The results screen appears at the end of a song whenever your score is above 0. If an active profile is set, the score is saved to the leaderboard.

Microphone Selection

• Press M to toggle the microphone on/off
• Press N to cycle through available microphones
• Press R to toggle mic mirroring during playback
• You can also select a preferred microphone in Settings
• The active microphone is shown in the HUD during playback

Per-Song Scoreboards

Each song maintains a scoreboard of past performances. Scores are tracked per profile, so multiple singers can compete on the same songs.

Backgrounds

Nightingale offers a deep selection of background themes during playback, cycled with the T key.

10 GPU Shader Backgrounds

Ten backgrounds are rendered in real-time using GPU shaders (GLSL):

1. Plasma — flowing colorful plasma effect
2. Waves — undulating wave patterns
3. Nebula — cosmic nebula clouds
4. Starfield — deep space star field
5. Sonar — radial pulse sweeps
6. Voronoi — animated cellular tessellation
7. Vortex — swirling color tunnels
8. Metaballs — fluid blob morphs
9. Spectrum — frequency-bar visualizer
10. Oscilloscope — waveform line trace

These run at full frame rate and adapt to your display resolution.

Shaders are audio-reactive when the microphone is enabled: a real-time analyzer drives shared uniforms (level, low/mid/high band energy, beat impulses) so louder vocals push the visuals harder. With the mic off, the shaders animate on their own time-based clock.

Pixabay Video Backgrounds

Pre-downloaded video backgrounds from Pixabay in 5 flavors, cycled with the F key:

1. Nature — forests, mountains, rivers
2. Underwater — ocean, coral, sea life
3. Space — galaxies, nebulae, Earth from orbit
4. City — urban skylines, night cityscapes
5. Countryside — rolling fields, sunsets

Videos are pre-downloaded during setup so they’re ready instantly.

Source Video Playback

When playing a video file (.mp4, .mkv, etc.), the source video plays as the background automatically. If the source is not directly playable, Nightingale generates a compatible playback version in cache.

Source video background timing follows playback tempo, so visual sync stays consistent when tempo is adjusted.

Profiles

Nightingale supports multiple player profiles for tracking scores across different singers.

Creating Profiles

Create a new profile from the main menu. Each profile stores:

• Player name
• Per-song pitch scores and star ratings
• Score history

Switching Profiles

Switch between profiles from the sidebar. The active profile is shown in the UI and all new scores are saved to it.

Score Tracking

Scores are stored in profiles.json inside your selected data folder (default ~/.nightingale/profiles.json, or <your-data-folder>/profiles.json if you picked a custom location during setup). Each profile maintains separate scoreboards for every song, so multiple singers can compete on the same library.

Configuration

Nightingale stores app settings in ~/.nightingale/config.json.

Data Storage

During setup, you can choose a custom data folder. Most runtime data lives in that selected folder. config.json and nightingale.log remain in the default ~/.nightingale path.

Typical selected data folder layout:

<selected-data-folder>/
├── cache/               # Stems, transcripts, lyrics, shifted variants, covers, playable videos
├── songs.db             # Song library and analysis metadata (SQLite)
├── profiles.json        # Player profiles and scores
├── videos/              # Cached Pixabay video backgrounds
├── sounds/              # Sound effects
├── vendor/
│   ├── ffmpeg           # Downloaded ffmpeg binary
│   ├── uv               # Downloaded uv binary
│   ├── python/          # Python 3.10 installed via uv
│   ├── venv/            # Virtual environment with ML packages
│   ├── analyzer/        # Extracted analyzer Python scripts
│   └── .ready           # Marker indicating setup is complete
└── models/
    ├── torch/           # Demucs model cache
    ├── huggingface/     # WhisperX model cache
    └── audio_separator/ # UVR Karaoke model cache

Video Backgrounds

Pixabay video backgrounds use the Pixabay API. In development, create a .env file at the project root with:

PIXABAY_API_KEY=your_key_here

Theme

Toggle between dark and light themes from the sidebar. The theme preference is saved in the config.

Notable Settings

config.json is written by the app — you’ll usually change these from Settings rather than by editing the file directly. A few keys worth knowing:

Building from Source

Prerequisites

Development

git clone <repo-url> nightingale
cd nightingale
cargo desktop dev

This starts the Tauri development server with hot-reload for the React frontend.

Release Build

cargo desktop build

Builds the production app bundle for your current platform using Tauri’s bundler.

Re-running Setup

If something goes wrong with the vendor environment, you can force a fresh setup from the sidebar actions menu inside the app by selecting Re-run Setup.

Troubleshooting

First Launch Takes Too Long

The initial setup downloads Python, PyTorch, ML models, and video backgrounds. On a slow connection this can take 10–20 minutes. Subsequent launches skip setup entirely.

Analysis Fails

If song analysis fails:

1. Check the error message in the UI for details
2. Ensure you have enough disk space (~5 GB for models and cache)
3. Try selecting Re-run Setup from the sidebar actions menu to reset the vendor environment
4. GPU memory errors may occur with very long songs — CPU fallback will be used automatically

No Sound

• Verify your audio output device is correctly configured
• Check that the audio file format is supported (.mp3, .flac, .ogg, .wav, .m4a, .aac, .wma)
• Try a different audio file to rule out file-specific issues

Microphone Not Detected

• Press N to cycle through available microphones
• Press R to toggle mic mirroring if you want live monitor audio
• Ensure microphone permissions are granted to the application
• On macOS, check System Settings > Privacy & Security > Microphone

GPU Acceleration Not Working

The analyzer auto-detects the best backend:

• NVIDIA GPU: Requires CUDA-compatible drivers
• Apple Silicon: Uses MPS backend (some operations fall back to CPU)
• CPU: Always works as a fallback

Check the setup progress screen for which backend was detected.

Updates

The auto-updater runs on macOS and Windows only — Linux ships without it. If you’re on Linux and the Update dialog says “Auto-update isn’t supported on Linux”, that’s expected. Click Open GitHub Releases to grab the new .deb or .rpm and install it the normal way for your distro.

If the Update dialog reports a problem on macOS or Windows:

• Stuck on “Checking for updates…” — the update server couldn’t be reached. Confirm you have a working internet connection and that GitHub is reachable; the manifest is fetched from github.com/rzru/nightingale/releases/latest. If the dialog shows a Wi-Fi-off icon, you’re offline.
• “Couldn’t reach the update server” / network error — same cause as above. Click Retry once your connection is back, or download the new build manually from the Releases page.
• “Signature error” — the downloaded bundle didn’t match the public key baked into the app. This usually means you’re running an unofficial build. Re-download Nightingale from the Releases page and try again.

Reset Everything

To completely reset Nightingale, delete your selected data directory. If you use the default location:

rm -rf ~/.nightingale

If you use a custom data folder, delete that folder instead, then relaunch and run setup again.