Performs local music analysis to extract stems, timing, harmony, and lyrics, and identifies loopable sections for short-form video content.
Also known as kieransimkin/stemlab, stemlab, danceflow-stemlab
- Stars
- 0
- Tools
- —
- Downloads, 30 days
- 2,023
- Latest version
- 1.14.3
- Last commit
- Nov 13, 2026
- License
- MIT
What is StemLab MCP?
StemLab performs local music analysis to extract stems, timing, harmony, and lyrics, and identifies loopable sections for short-form video content.
Summary generated from the project's README
About StemLab MCP server
StemLab is a local music analysis toolkit designed for creators who need to extract precise musical data for short-form video content. It processes master recordings to identify stems, timing, harmony, lyrics, and MIDI, specifically searching for complete-bar loopable sections suitable for Shorts. The system functions as a CLI, Python library, web service, or container, and can be integrated as a local Codex MCP plugin.
Users interact with the tool through various commands such as stemlab.lyric_batch for lyric processing or stemlab compare-beats for rhythm analysis. It avoids silent assumptions by providing traceable evidence through JSON, TSV, and NPZ artifacts. The toolkit does not redistribute model weights; instead, it fetches them from upstream registries during the first use. This ensures that all analysis, from beat counting to motion-rig rhythm estimation, remains grounded in verifiable source hashes and manifests.
What you can do with it
- 1Extract loopable video segmentsIdentify complete-bar sections of a song that are suitable for use in short-form video content.
- 2Analyze rhythmic motion dataEvaluate timestamped dance captures to estimate kinematic movement accents and BPM candidates.
- 3Perform batch lyric extractionProcess lyrics using a method that preserves native speech recognition evidence without interpolating missing lines.
- 4Compare different beat modelsCompare model-free audio against various rhythm decoders and existing model result JSON files.
Written from the project's README. Every use case is checked against a quote from it.
How to install StemLab MCP
Formats verified 10 Nov 2026Add to CursorAdd to GooseInstall StemLab MCP in Claude Code
$ claude mcp add --env 'STEMLAB_CODEX_WORKSPACE=<STEMLAB_CODEX_WORKSPACE>' stemlab -- uvx danceflow-stemlabInstall StemLab MCP in Cursor
~/.cursor/mcp.json
{
"mcpServers": {
"stemlab": {
"command": "uvx",
"args": [
"danceflow-stemlab"
],
"env": {
"STEMLAB_CODEX_WORKSPACE": "<STEMLAB_CODEX_WORKSPACE>"
}
}
}
}Install StemLab MCP in VS Code
.vscode/mcp.json
{
"servers": {
"stemlab": {
"type": "stdio",
"command": "uvx",
"args": [
"danceflow-stemlab"
],
"env": {
"STEMLAB_CODEX_WORKSPACE": "<STEMLAB_CODEX_WORKSPACE>"
}
}
}
}Install StemLab MCP in VS Code CLI
$ code --add-mcp '{"name":"stemlab","type":"stdio","command":"uvx","args":["danceflow-stemlab"],"env":{"STEMLAB_CODEX_WORKSPACE":"<STEMLAB_CODEX_WORKSPACE>"}}'Install StemLab MCP in Claude Desktop
claude_desktop_config.json
{
"mcpServers": {
"stemlab": {
"command": "uvx",
"args": [
"danceflow-stemlab"
],
"env": {
"STEMLAB_CODEX_WORKSPACE": "<STEMLAB_CODEX_WORKSPACE>"
}
}
}
}Install StemLab MCP in Codex
$ codex mcp add stemlab --env 'STEMLAB_CODEX_WORKSPACE=<STEMLAB_CODEX_WORKSPACE>' -- uvx danceflow-stemlabInstall StemLab MCP in Codex config.toml
~/.codex/config.toml
[mcp_servers.stemlab] command = "uvx" args = ["danceflow-stemlab"] [mcp_servers.stemlab.env] STEMLAB_CODEX_WORKSPACE = "<STEMLAB_CODEX_WORKSPACE>"
Install StemLab MCP in Gemini CLI
$ gemini mcp add -s user -e 'STEMLAB_CODEX_WORKSPACE=<STEMLAB_CODEX_WORKSPACE>' stemlab uvx -- danceflow-stemlabInstall StemLab MCP in Gemini settings.json
~/.gemini/settings.json
{
"mcpServers": {
"stemlab": {
"command": "uvx",
"args": [
"danceflow-stemlab"
],
"env": {
"STEMLAB_CODEX_WORKSPACE": "<STEMLAB_CODEX_WORKSPACE>"
}
}
}
}Configuration
STEMLAB_CODEX_WORKSPACERequiredAbsolute, existing local directory containing authorised audio and results. Never use a filesystem root.
| Name | Description | |
|---|---|---|
| STEMLAB_CODEX_WORKSPACE | Absolute, existing local directory containing authorised audio and results. Never use a filesystem root. | Required |
Automated checks
Ran Nov 12, 2026Full report →Passed our automated checks. That is not a security audit: read the code and give the server the narrowest access it needs. How checks work
- Source repository linked and reachablePass
- Open source license (OSI)Pass
- Commit or release in the last yearPass
- Package name not close to a popular packagePass
- Remote endpoint uses HTTPSNot checked
- Destructive tools are labelledNot checked
- No hidden instructions to the modelPass
- No critical advisory in the package or its dependenciesPass
README
View on GitHubStemLab
By Kieran Simkin · DanceFlow ecosystem · Vector logo and usage guide.
Local music analysis for stems, timing, beats, harmony, lyrics, MIDI and loopable sections for Shorts. https://kieransimkin.co.uk/
Complete lyric-line gates and separate vocal parts
Canonical LRC export rejects missing anchors, implausibly long cross-gap phrases,
compressed line clusters and rounded onset collisions before ranking candidates.
The repetition-safe python -m stemlab.lyric_batch route preserves native ASR
evidence and holds complete export when a section fails; it never interpolates
missing lines. Optional python -m stemlab.lyric_acoustic reuses existing local
Whisper weights for known-text alignment. Explicit separate-part line selections
can retain simultaneous onsets without serializing backing vocals.
Read the commands, synthetic demonstration and limits. Recognition scores and forced text coverage do not prove sung-word accuracy or voice identity. All exports remain drafts until artist listening confirmation; existing files, canonical lyrics and selected EPK timings are preserved.
Canonical song example — Star Lights Glow (Mystery Liquid Kick)
examples/star-lights-glow-mystery-liquid-kick/ holds Bass Ben & Friends'
Star Lights Glow (Mystery Liquid Kick) as a verified 192 kbps CBR MP3
listening/analysis reference with the original supplied front-cover JPEG and ID3
metadata. The original lossless WAV remains the canonical source master,
identified by SHA-256 in evidence.json rather than redistributed as a second
large binary. The mix is separate from Star Lights Glow (Anthem); BPM, phase,
release date and ISRC are unconfirmed. Verify with
python scripts/verify_star_lights_reference.py and consult the example's
README. The MIT code
licence does not license the music or artwork.
Independent beat finding and counting (experimental)
Tempo precision update: Classical autocorrelation and comb-filter lag peaks now retain their original 100 Hz integer-bin BPM while also reporting a bounded sub-frame estimate. Local tempo windows use the same refinement, and tempo families expose source-wide observed-onset-path coverage as a review diagnostic, not as calibrated confidence or automatic BPM approval. Predicted beats, observed onset candidates and inferred ordinal gaps remain distinct. See beat-evidence documentation.
stemlab compare-beats now compares model-free audio, N-dimensional
DanceRudiments recordings, rig captures and verified motion-package accents.
Five complementary classical rhythm decoders produce reviewable BPM, beat count,
phase, accent-cycle and local-tempo hypotheses. It can also import existing
Beat This!/BeatNet/Beat Transformer/All-In-One model result JSON without
re-running models; no model weights are downloaded. Cross-source phase scores
require explicit clock offsets. No inferred downbeat or predicted grid is
silently accepted as a real detection or canonical timeline. See
usage and evidence schema and the
model and implementation research roadmap.
The review now groups half/double tempo families rather than majority-voting
correlated algorithms, scores DP coverage across the original source, and fits
original saved model timestamps with robust residual/drift guards. Optional
--annotation NAME=FILE evaluates beat timing only against a matching,
human-reviewed, SHA-bound annotation; separately-held-out reliability calibration
abstains without enough independent annotated songs. The non-destructive
scripts/benchmark_example_beats.py handles Arcadians, The Truth is a Virus,
and the separately pinned Star Lights Glow (Mystery Liquid Kick) derivative.
No estimator approves beat 1, changes an audio master or replaces a canonical
beat map. See beat evidence.
Rigged mesh rhythm and loop evidence
The optional stemlab analyze-rig-motion command analyses timestamped
DanceRudiments rig captures or verified time-only motion packages
from animated FBX/BVH/glTF/BLEND Actions. It
estimates kinematic movement accents and BPM candidates, discovers repeated
pose cycles, and suggests reviewable loop seams with velocity and root-drift
checks. It can export an explicitly unverified source beat map for the
existing DanceRudiments 64-pip compiler. This is not verified musical beat 1,
audio synchronization, or automatic looping/mesh-seam repair. No model downloads
or native sampler are involved. The optional DanceRudiments authoring loader
validates all motion-package chunk hashes. Tempo ranking uses periodicity and spectral support to distinguish plausible
half- and double-time pulses. See rig-motion analysis.
Python callers with an independently aligned beat grid can also use
stemlab.analysis.motion_loop_seams.rank_aligned_motion_loops to rank hard-cut
recording loops of a fixed beat count. It reuses the pose, velocity and root
gates, can reject large PCM join steps, and accepts optional full-body and
lower-body camera measurements. It keeps both endpoints within a bounded
shared shift from the supplied beats. See beat-constrained seam ranking.
For original captures, --world-motion world-motion-evidence.json consumes the
source-hash-bound world-space/foot-contact sidecar produced by DanceRudiments.
It checks the exact capture, rig profile, timestamps and coverage, retains the
original control analysis, and exposes contact/world tempo alternatives plus
contact-state loop diagnostics. --tempo-strategy consensus prefers a contact
pulse corroborated by control/world timing; contradictory families abstain.
Explicit control, world and contact selections remain available and never
approve music timing. Root/world/height traces share one kinematic lineage;
correlated streams are not counted as independent model votes.
Selecting an alternative consensus candidate checks corroboration for that
exact candidate. Source coverage is recorded from the original timestamps even
without a world sidecar; gaps and isolated frames cannot become interpolated
alignment events or selected world-aware loop spans. Original control evidence
remains available for review.
prepare-alignment-events, align-motion-audio and confirm-motion-alignment
complete the source-clock handoff: import actual observed accents or saved audio
detections, verify original source bytes, supply an explicit clock map, compare
offset/half-speed/double-speed and bounded variable-tempo proposals, then write
a separate review revision. Gaps remain unmapped, periodic phase ambiguity is
reported, and beat one/meter require separate explicit confirmations. Generated
affine endpoints receive bounded floating-point roundoff correction before
review; reviewed mappings still require exact source bounds. Nothing
retimes source media or silently changes a canonical beat map. The offline
modules use NumPy; python -m stemlab.rig_motion_cli --help also exposes this
workflow without importing the audio/model pipeline. See the alignment guide,
JSON contracts and reproducible example.
The motion-evidence CI job builds the actual wheel, verifies isolated offline
imports without audio/model dependencies, and tests source binding, world/contact
sidecars, timing candidates and review revisions on Python 3.10/3.12. Production
companion dependencies still require official published releases; this source
gate does not claim listening, retargeted-character or deployment acceptance.
The real-song reference timeline preserves its explicitly hash-pinned public
detector JSON byte for byte, including newlines, on Windows and Linux. Generic
exports still remove private paths. Container model weights and model-specific
caches live under the owning project's ignored /opt/stemlab/models/ directory;
they are excluded from the build context and source/package artifacts.
StemLab by Kieran Simkin · My Songs portfolio · Arcadians EPK · Source
Packaging identity: the canonical project name is StemLab. StemLab is part of the Dance Flow project. The PyPI distribution is named
danceflow-stemlabsolely because Python package-registry names are globally unique. The Python import, CLI, GitHub repository and container image remainstemlab.
Before every new workflow
Check the latest stable official release and the actual installed/imported version before starting; do not infer currency from a directory name or a development checkout. Read the current release notes, tagged capability documentation and relevant Potential problems. Use the DanceFlow release preflight and complete capability references.
People and agents should consider the full ecosystem, use all capabilities that materially serve the task, compose supported files/APIs, and improve a demonstrated reusable gap in its canonical tool. Record why no worthwhile change is justified when that is the result. Keep independent tool identities, compatibility, source clocks, privacy, model/download costs and publication/deployment permissions intact. Store weights and model-specific caches in the owning project's ignored models/.
Find your route
Synopsis · Install · CLI · Output layout · Analysis performed · Curated default model set (September 2026) · Speech transcription · Comprehensive sonic, harmonic, rhythmic and semantic analysis · Find DanceRudiments matching candidates from each stem
Synopsis
StemLab is a local, evidence-led music analysis toolkit that turns a master recording into inspectable stems, timing, structure, harmony, lyric and sonic evidence, then finds complete-bar loopable sections for Shorts and other short-form videos.
It combines independent separation and music-information-retrieval routes rather than treating any single model as the answer. Results remain traceable through source hashes, manifests, JSON/TSV/NPZ artefacts, portable Sonic Visualiser sessions and a shared React timeline for playback and comparison. StemLab can run as a CLI, Python library, web service, container or local Codex MCP plugin. It is the music-understanding layer of Kieran Simkin's DanceFlow workflow and can supply downstream motion systems such as the WordPress DanceMoves plugin, while remaining useful on its own.
Model weights are not redistributed by this project. They are fetched from their upstream registries/releases on first use. This avoids silently republishing checkpoints whose licensing may differ from the source code license, and lets upstream integrity metadata be used where available.
Install
Python 3.11 is recommended for the broad optional-model ecosystem. The beats and all extras install Beat This! plus the compatible madmom-prebuilt decoder. BeatNet 1.1.3 itself still publishes obsolete NumPy/Numba dependency pins, so StemLab downloads its official pure-Python wheel separately, verifies the published SHA-256, and exposes only the BeatNet code and bundled model weights from its cache. This route is supported on Python 3.11-3.13; a missing BeatNet result must not be mistaken for a successful detector run.
From PyPI, install the released StemLab distribution with:
pip install danceflow-stemlab
The installed Python package and command remain stemlab. For development from a source checkout:
python -m venv .venv
. .venv/bin/activate # Windows: .venv\Scripts\activate
python -m pip install -U pip
pip install -e ".[all]"
SCNet and Beat Transformer are research repositories rather than stable pip inference APIs. They are isolated/downloaded on demand. To prepare everything in advance:
stemlab bootstrap all
For BeatNet alone:
pip install "danceflow-stemlab[beats]"
stemlab bootstrap beatnet
stemlab doctor
stemlab doctor reports BeatNet as ready only when the normal package or the
verified StemLab cache can actually be found. analyze --no-bootstrap never
downloads BeatNet and records the unavailable backend instead.
For only the Vamp analysis stack:
stemlab bootstrap vamp
bootstrap vamp downloads the pinned Sonic Annotator runtime and launches the
official Vamp Plugin Pack installer. Complete the installer once, then rerun the
command if needed to verify that the requested plugin outputs are visible.
CLI
Full requested analysis:
stemlab analyze master.wav --output ./analysis-master --profile full --device auto
Useful alternatives:
# Skip the 53-stem model for a materially lighter run
stemlab analyze master.wav -o ./analysis-master --profile practical
# Run only selected separators
stemlab analyze master.wav -o ./analysis-master \
--model bs_roformer_sw --model scnet_xl_ihf --model openunmix_umxhq
# Fail immediately rather than recording a backend failure and continuing
stemlab analyze master.wav -o ./analysis-master --strict
# Skip Vamp if the plugin pack is intentionally not installed
stemlab analyze master.wav -o ./analysis-master --no-vamp
stemlab models
stemlab doctor
By default model weights and external research code are cached outside the output folder; all generated analysis artifacts and datasets are written inside the output folder. The copied master is also placed in input/ so the Sonic Visualiser session is portable as one directory.
Output layout
analysis-master/
input/master.wav
stems/
bs_roformer_sw/*.wav
mvsep_mega53/*.wav
scnet_xl_ihf/*.wav
htdemucs_ft/*.wav
openunmix_umxhq/*.wav
spectrograms/
master.png
master.npz
<model>/<stem>.png
<model>/<stem>.npz
speech/spoken_word.png
speech/spoken_word.npz
speech/
spoken_word.wav
speech_regions.json
whisper.json
transcript.txt
transcript.srt
words.tsv
beats/
beatnet.json / beatnet.tsv
beat_this.json / beat_this.tsv
beat_transformer.json / beat_transformer.tsv
beat_transformer_activations.npz
vamp/
report.json
transforms/*.n3
raw/*.csv
data/*.json
data/*.npz
plots/*.png
deep/
sonic/
rhythm/
harmony/
structure/
lyrics/
semantic_text/
song_map/
rudiments/ # optional native DanceRudiments candidate report
report.json
index.html
overview.png
summary.json
sonic_visualiser/
session.sv
session.xml
open_sonic_visualiser.bat
open_sonic_visualiser.sh
analysis.json
manifest.json
session.sv is genuine Sonic Visualiser bzip2-compressed session XML. It contains a master pane with beat/downbeat/Whisper time-instant layers and one synchronized waveform + Sonic Visualiser spectrogram pane for every separated stem. The uncompressed session.xml is retained for inspection/debugging. The Windows .bat looks on PATH and in the usual Program Files locations.
Analysis performed
For every successful separator, every WAV it emits is retained. Stem names are discovered from files rather than truncated to a hard-coded four-stem schema, which is important for MVSep Mega 53. For the master and every stem, StemLab writes both a PNG spectrogram and compressed .npz spectrogram data (time axis, frequency axis, dB matrix, FFT metadata).
Full-resolution spectrogram evidence stays in the .npz; the PNG renderer is bounded to 8,192 time frames so long tracks cannot expand into multi-gigabyte Matplotlib RGBA buffers. If an older run retained the NPZ but failed while drawing the PNG, repair it in place without rerunning models:
stemlab repair-spectrograms "path\to\analysis-results"
Beat-grid comparison also retains an explicit inferred-index hypothesis when saved detections skip whole beats. It reuses the constant-grid fitter with integer gaps, preserving every observed timestamp and the raw half/double-time tempo. A poor sequential fit is replaced only by a better first-anchored gap fit within the existing error guard; both fits remain in the comparison report. Missing detections are not newly detected beats or canonical approval.
Curated default model set (September 2026)
| StemLab id | Role | Outputs | Why it is included |
|---|---|---|---|
bs_roformer_sw |
high-capacity | vocals, drums, bass, guitar, piano, other (+ upstream instrumental) | Current production-oriented BS-RoFormer inference package recommends this six-stem checkpoint. |
mvsep_mega53 |
high-capacity / broad taxonomy | 53 raw stems (discovered dynamically) | Extremely broad one-checkpoint stem inventory for later assessment. Upstream warns it is memory-heavy and recommends at least 16 GB VRAM. |
scnet_xl_ihf |
high-capacity | vocals, drums, bass, other | The MSST published checkpoint table reports a 10.08 dB average MUSDB test SDR and 9.92 dB Multisong average for this model. |
htdemucs_ft |
high-capacity established baseline | drums, bass, other, vocals | Fine-tuned HTDemucs remains a useful independent architecture/baseline rather than another RoFormer variant. |
openunmix_umxhq |
compact / low-latency candidate | vocals, drums, bass, other | Compact PyTorch/Open-Unmix baseline. StemLab runs niter=0 to favour latency. “Realtime” is hardware- and buffer-dependent: benchmark it on the intended target. |
An optional htdemucs_6s model is registered for guitar/piano comparison but is not part of the default top-four group.
Relevant upstreams:
- https://github.com/openmirlab/bs-roformer-infer
- https://github.com/ZFTurbo/Music-Source-Separation-Training
- https://github.com/adefossez/demucs
- https://github.com/sigsep/open-unmix-pytorch
Speech transcription
The preferred vocal stem is passed through the Silero VAD implementation bundled with faster-whisper. Speech regions are used to create a timeline-preserving spoken_word.wav (non-speech is zeroed rather than concatenated), then Whisper is run with word timestamps. Outputs include whisper.json, speech_regions.json, transcript.txt, transcript.srt, and words.tsv.
For singing, compare --whisper-vad-mode full when the speech mask removes
musical vocal phrases. This retains every input sample in a separate normalized
16 kHz mono transcription_input.wav using faster-whisper's own PyAV decoder;
this full-audio path needs no Torch import. It does not concatenate regions or change
source timestamps. The default remains speech. Optionally set
--whisper-language en when the language is known. Saved evidence records the
mode, whether a mask was applied and the requested language. The public Python
API isolate_and_transcribe(..., vad_mode="full", language="en") also accepts
an exact master mix or saved vocal stem and a local model-directory path.

These are measured word-reconciliation scores from the same pinned large-v3 model, not acoustic accuracy or listening approval. Canonical wording stays locked; the 70% threshold and observed-anchor requirements still apply. A line with no observed timing remains a hold. Chart context and hashes.
Canonical-word reconciliation accepts edge punctuation on one observed lexical
word, so Go, can match canonical Go without changing either text or timing.
It does not split one multiword ASR timestamp into invented per-word timings.
Whisper runs in repetition-safe mode by default (condition_on_previous_text=False) to prevent a repeated syllable in one window contaminating later windows. whisper.json records that setting and includes repeated-token diagnostics. Use --whisper-condition-on-previous-text only for an explicit comparison run; a flagged transcript remains evidence requiring section-wise recovery or listening QA, not a trustworthy lyric source.
Beat analysis runs:
- BeatNet in offline/DBN mode through StemLab's checksum-verified runtime bootstrap. StemLab uses the maintained
madmom-prebuiltwheel and bypasses only BeatNet 1.1.3's obsolete NumPy/Numba package metadata. - Beat This! using the
final0checkpoint and its minimal postprocessor. - Beat Transformer, using the original released model code/checkpoints. The model was trained on five demixed mel streams, so StemLab maps BS-RoFormer-SW to vocals, drums, bass, piano, and
other + guitar, makes 128-bin mel-power spectrograms at the original 44.1 kHz / 4096 FFT / 1024-hop settings, and averages all eight released fold checkpoints by default. It uses the original madmom DBN decoder if madmom is importable, otherwise a documented SciPy peak-picking fallback.
Beat outputs are stored as JSON, TSV, and (for Beat Transformer) activation NPZ data.
StemLab also supports the official Vamp Plugin Pack, executed through
Sonic Annotator. The curated Vamp pass focuses on musically useful outputs:
Chordino chord transcription and harmonic-change likelihood; NNLS chroma and
bass chroma; Queen Mary key and tonal-change detection; concert-pitch tuning;
pYIN melody/F0 and monophonic note transcription on the preferred separated
vocal stem; Silvet polyphonic note transcription; Segmentino song-structure
segmentation; and the Queen Mary Vamp beat/bar tracker. Raw CSV, pinned
transform files, JSON, NPZ and plots are retained under vamp/, with the most
useful melody/harmony layers also embedded in the Sonic Visualiser session.
StemLab MCP questions
How do I add StemLab MCP to Claude Code?
Run claude mcp add --env 'STEMLAB_CODEX_WORKSPACE=<STEMLAB_CODEX_WORKSPACE>' stemlab -- uvx danceflow-stemlab in your terminal. Add --scope user to make it available in every project.
How do I add StemLab MCP to Cursor?
Open ~/.cursor/mcp.json and add the server under mcpServers, using the Cursor config shown in the install section above.
Does StemLab redistribute model weights?
No, model weights are not redistributed by the project; they are fetched from their upstream registries on first use to ensure licensing integrity.
Can StemLab find loopable sections for videos?
Yes, it identifies complete-bar loopable sections specifically designed for Shorts and other short-form videos.
Does the tool automatically approve beat detection?
No, the tool provides hypotheses and candidates for review, but it does not silently accept inferred downbeats or predicted grids as canonical.
