RESEARCH

Music structure & human intention

Current questions, research prototypes, and clearly scoped pilot evidence.

RESEARCH

Research

Current focus — 2026

Symbolic music structure analysis · Musicology-informed machine learning · Human–AI creative systems

How can musical intent become editable constraints while form, sound, and authorship remain in the creator’s hands?

I begin with musical questions and build tools that make creative decisions visible and revisable. My research connects symbolic musical structure with interactive composition and local revision.

Research interests

Human–AI music co-creation
Computational music theory
Symbolic music representation

Controllable music & multimodal generation
Music information retrieval
Interactive composition & creative support systems

Methods / Research Toolkit
Symbolic music processing · MusicXML · Representation learning · Transformer / attention · Graph-based modelling · MIR evaluation · Human-in-the-loop systems

MUSIC STRUCTURE / ONGOING RESEARCH

Musicology-Guided Structural Attention (MGSA)

Measure-Level Boundary Detection in Symbolic Music

Independent research · Manuscript in preparation · Preliminary validation

RESEARCH QUESTION

Can explicit musicological priors improve measure-level boundary detection in symbolic scores?

MUSICOLOGICAL PRIORS

Metric + Motif + Harmony + Cadence + Repetition − Boundary

A = softmax(QKᵀ / √d + ∑ λᵣ Bᵣ − λᵦ Bᵦ); H = AV

The sum covers Metric, Motif, Harmony, Cadence and Repetition. Nonnegative weights control each bias; Boundary enters as a separate subtractive term.

View architecture ↗

MusicXML→Measure Encoder→MGSA→Measure-Level Boundary Prediction

Current prototype · Boundary / phrase-ending pilot on Mozart symbolic scores, with baseline and prior-ablation experiments.

Research targets · Independently supervised section relations and global form decoding remain to be completed.

Data & annotation pipeline. The pilot uses sonata-grouped splits from the DCML Mozart piano-sonata corpus, with phrase-boundary projections and cadence supervision from its annotations. Score-derived input features are separated from annotation supervision. Preprocessing records grouped splits, annotation audits, and data fingerprints. These are derived research assets, not a newly authored corpus.

Real-score pilot · Mozart K.283, movement I

Exploratory case study / preliminary visualisations

Transformer and MGSA attention matrices on Mozart K.283 movement I — Last-layer attention averaged over heads, using the same colour scale. Visual structure alone does not establish improved boundary prediction.

Transformer and MGSA learned embedding self-similarity matrices for K.283-1 — Learned embedding cosine similarity; lines show projected phrase annotations. Pairwise similarity ground truth is not available.

Research method & evaluation

Updated 25 October 2026 · existing saved pilot runs · no new training

Data. DCML Mozart: 18 sonatas / 54 movements. Fold 0 is grouped by sonata: 10 / 4 / 4 sonatas, or 30 / 12 / 12 movements, for train / validation / test.

Metrics. Test event micro F1 with one-to-one boundary matching: exact gap or ±1 measure. Reference phrase-ending onsets are projected to inter-measure gaps. Checkpoint and threshold are selected on validation. F1 is not per-gap accuracy.

Baseline comparison

MGSA and matched control boundary F1. Exact: 0.3332 and 0.3297. Within one measure: 0.4569 and 0.4588. — Five-seed means and sample standard deviations on Mozart Fold 0.

Learned models use five seeds and five epochs. The matched control keeps learned pooling, the encoder, decoder and auxiliary cadence supervision; it disables attention priors.

Mozart Fold 0 · mean ± seed sample SD; novelty is a single run
Method / configuration	Exact F1	±1-measure F1
Novelty (single deterministic run)	0.2639 (single run)	0.3624 (single run)
Mean-pooling Transformer	0.3133 ± 0.0164	0.4324 ± 0.0109
Matched pooling / no priors	0.3297 ± 0.0135	0.4588 ± 0.0136
Full MGSA	0.3332 ± 0.0156	0.4569 ± 0.0317

Baseline comparison on exact and tolerant boundary F1 — Mean ± sample SD across five seeds; not confidence intervals. Zoomed F1 axes.

MGSA minus matched control: exact F1 +0.35 percentage points; ±1-measure F1 -0.19 percentage points.

Ablation study

Each leave-one-prior-out variant is retrained with the same fold, seeds, pooling and five-epoch budget. Removing cadence bias does not remove auxiliary cadence supervision.

Retrained ablations · five seeds · mean ± seed sample SD
Method / configuration	Exact F1	±1-measure F1
Full MGSA	0.3332 ± 0.0156	0.4569 ± 0.0317
− Metric	0.3293 ± 0.0053	0.4462 ± 0.0254
− Motif	0.3319 ± 0.0096	0.4538 ± 0.0188
− Harmony	0.3346 ± 0.0085	0.4528 ± 0.0199
− Cadence	0.3353 ± 0.0199	0.4477 ± 0.0155
− Repetition	0.3365 ± 0.0148	0.4501 ± 0.0140
− Boundary	0.3346 ± 0.0111	0.4514 ± 0.0214

Retrained leave-one-prior-out MGSA ablations — Dashed line: Full MGSA mean. Error bars: sample seed SD. No significance claim.

All removals lower mean tolerant F1 in this pilot; exact F1 changes are mixed. These observations do not prove a reliable contribution from every prior.

Multi-seed stability

Exact and tolerant F1 for seeds 42, 123, 2026, 3407 and 8888 — Same held-out fold. Seeds are categorical; connecting lines are not training curves.

MGSA exceeds matched pooling on exact F1 in 4/5 seeds, but on ±1-measure F1 in only 1/5 seeds. Tolerant seed SD is larger for MGSA (0.0317) than for the control (0.0136).

Held-out example · Mozart K.283, movement I

This retained pilot figure shows the Transformer and MGSA gap probabilities, DCML-derived reference positions and the model’s cadence probabilities. Gap indices use the original figure’s zero-based convention.

Mozart K.283 first movement, predicted boundary probabilities and reference gaps — Fixed held-out example from the saved MGSA pilot. The phrase-ending target belongs to this research model. See the separate Xunpu Studio form boundary test ↗

Per-seed results

Exact F1 · original saved runs
Seed	Mean pooling	Matched pooling	Full MGSA
42	0.2913	0.3327	0.3235
123	0.3035	0.3124	0.3222
2026	0.3237	0.3300	0.3355
3407	0.3154	0.3239	0.3255
8888	0.3328	0.3495	0.3595

±1-measure F1 · original saved runs
Seed	Mean pooling	Matched pooling	Full MGSA
42	0.4134	0.4448	0.4338
123	0.4351	0.4533	0.4402
2026	0.4388	0.4620	0.4485
3407	0.4346	0.4534	0.4497
8888	0.4402	0.4805	0.5124

Current conclusion: no consistent advantage over the matched control has been established. One composer, one fold and short training limit the interpretation. Seed SD is not a cross-work confidence interval; no statistical-significance claim is made. Full form and independent section-relation metrics are unavailable. OMR is evaluated separately.

Next: longer training, additional folds and a frozen comparison protocol.

Full evaluation & public records Aggregate CSV Per-seed CSV Plain-text report

Working paper / research roadmap · PDF ↗Architecture ↗

Future Directions

Creative practice and long-term research

Creative Practice & Professional ToolsInteractive music prototypes demonstrated through original musical works

Visual, Symbolic & Auditory UnderstandingScore recognition / musical structure / audio-boundary review

Real-Time Performance UnderstandingScore following / gesture / phrasing / timing

Adaptive Musical AccompanimentLong-term exploration of human–AI co-performance

Visual and symbolic score understanding → auditory understanding → interactive musical practice → performance understanding → adaptive accompaniment and human–AI co-creation

Xunpu GraphOMR architecture: score perception, typed symbol graph, relation reasoning and music-aware decoding

Xunpu GraphOMR: Explainable Optical Music Recognition

Research prototype · Typed symbol graphs + rule-based decoding

Score images become typed symbol graphs; geometric relations and musical consistency checks support inspectable reconstruction into MusicXML, MIDI, and diagnostics.

The validated baseline covers printed single-staff melodies in treble clef, C major and 4/4. Current evaluation uses 16 variants of two synthetic fixtures.

[GitHub Repo][Architecture]

From printed notation to a symbolic score

I implemented typed symbol graphs, geometric reconstruction and MusicXML export. The baseline detects and relates printed noteheads, stems and staff geometry, then applies score rules to decode pitches and durations. The overlay makes the recovered symbols available for inspection.

Clean synthetic single-staff duration fixture used by the GraphOMR test — Input · clean synthetic duration fixture.

GraphOMR overlay of the clean duration fixture — Output · symbol overlay for the clean duration fixture.

Regression results and retained failures

Two generated fixtures were tested under eight image conditions each. Fourteen of sixteen sequences matched exactly. The duration fixture at 0.75× scale retained correct pitch matches but only 33.3% duration match. Under the recorded noise condition it retained 50.0% pitch match and 75.0% duration match. These cases expose failure modes within the synthetic baseline.

The melody fixture matched under all eight conditions. The durations fixture had errors at 0.75× scale and noise 12. — Pitch and duration matches for all 16 synthetic cases. The two non-exact cases are highlighted.

Recognition overlay on the synthetic melody score — GraphOMR · synthetic score recognition overlay

Read the per-case evaluation and reproduction command ↗

Xunpu Studio · Harmony annotation

Two systems of a piano score with Roman-numeral annotations — Additional harmony annotation example. The 12-event review concerns a separate F-major score.

Read the separate review ↗Original F-major review score ↗

ReCoP & Shaanbei folk-song annotation

Reranking with Consensus-Gated Profiles for Shaanbei Folk-Song Lyrics Retrieval is a research submission to Transactions of the International Society for Music Information Retrieval (TISMIR). The editor decision recorded on 8 September 2026 was “Send to Review”.

Third author (Zhao, J., Guan, Q., Dai, X., Li, S., Ban, Y., & Zhang, Y.). I was responsible for the AI audio algorithm component and built database, annotation, calibration and review interfaces supporting the wider folk-song workflow.

Visit the folk-song database ↗

Folk-song audio segmentation

This related macOS research tool explores audio structure and supports human boundary review. It analyses waveform, energy, spectral, timbral, rhythmic and harmonic changes; lyric timing can assist boundary calibration. The related implementation uses acoustic features and lyric-assisted boundary refinement. These figures document that workflow and are not results of the lyrics-retrieval experiment.

The figures cover different samples and analyses. Counts describe their stated review or processing scope, not general accuracy. The reference in one comparison comes from a screenshot; its authorship and whether annotation was blind are unconfirmed.

Recordings and batch review

Waveform, novelty and segmentation in a Shaanbei folk-song recording. Open the original figure for its scope and notes.

Nineteen acoustic features from the same recording. Open the original figure for its scope and notes.

Review scale and boundary counts across 420 songs. Open the original figure for its scope and notes.

Per-song segment counts and boundary changes. Open the original figure for its scope and notes.

Annotation and reference methods

Valence–arousal scale and automatic initial estimates. Open the original figure for its scope and notes.

One recording: screenshot reference compared with local AI boundaries. Open the original figure for its scope and notes.

Methods and reference projects. Open the original figure for its scope and notes.

Platform review across 35 songs and reference sensitivity. Open the original figure for its scope and notes.

Agreement, ablations and failure cases

Within-song boundary agreement across six recordings. Open the original figure for its scope and notes.

Feature ablation across six recordings. Open the original figure for its scope and notes.

Parameter sensitivity across six recordings. Open the original figure for its scope and notes.

A competing-peak failure case in one recording. Open the original figure for its scope and notes.

Runtime and boundary decisions

Local processing time for recordings. Open the original figure for its scope and notes.

Personal annotations and AI timelines across six recordings. Open the original figure for its scope and notes.

Peak selection and maximum-segment-length insertion in one recording. Open the original figure for its scope and notes.

The original figures preserve their own methods, sample sizes, limitations and failure cases. Select any image to view it at full resolution.

Selected Research Outputs

MGSAOngoing research · Manuscript in preparation
ReCoPSubmitted to TISMIR · Editor decision: Send to Review · Co-author · 2026Reranking with Consensus-Gated Profiles for Shaanbei Folk-Song Lyrics Retrieval
Xunpu GraphOMRSource-available research prototype
Patent / Software IPApplication and software records listed below