Sonilo benchmarks: music, sound effects & video-to-audio

At a glance

Benchmark findings at a glance

Sources: genre evaluation report and version-pinned audio_eval results (details). Numerical leads are descriptive; statistical significance was not established in the supplied summaries.

Musicality

How does Sonilo compare with Suno and Lyria on musicality?

Sonilo 1.2 ranked first or tied for first on musicality in 111 of 239 valid prompt groups (46.4%). Suno V6 did so in 97 groups (40.6%), and Lyria 3.5 in 78 groups (32.6%). These are outcomes from a three-system comparison, not head-to-head win rates. Genre evaluation source

Relative mean scores use a 1–3 scale; higher is better. Bold marks the highest value in each row. Model names follow the naming clarification supplied for this page.

Mixing and rhythm/groove showed the largest musicality leads

Sonilo 1.2 achieved the highest mean in melody, harmony, rhythm/groove, arrangement, mixing, and vocals. Its lead over the next-highest system was 0.280 relative-score points for mixing and 0.213 for rhythm/groove. The melody difference was much smaller, at 0.021 points.

Melody: 238 valid groups; other dimensions: 239. “Sonilo lead” is the gap to the next-highest system. Overall musicality was evaluated separately, not calculated as the average of these six dimensions. Relative-score differences are not percentage improvements in audio quality.

Genres

Which music genres were strongest for Sonilo?

Electronic and Pop were Sonilo 1.2’s strongest genres in this evaluation. It led Electronic with a mean musicality score of 2.467 and Pop with 2.450. Each category contained 30 prompt groups. Sonilo 1.2 also shared the highest displayed mean in Rock and Punk. Suno V6 led Metal and Hip-Hop, and Lyria 3.5 led Jazz.

Mean relative musicality score by genre. Each bar spans the full 1–3 scale.

Higher is better. Bold indicates the highest displayed mean, including ties.

The genre results support choosing Electronic and Pop when exploring Sonilo’s demonstrated strengths.

MusicCaps

What do automated music benchmarks show?

Sonilo Music v1.1 scored higher than Suno v5.5 on CLAP and lower on all four reported Fréchet-distance variants on MusicCaps. These automated results concern text–audio correspondence and similarity to reference audio in learned feature spaces; they are not direct listener-preference scores. MusicCaps results

Selected metrics from the repository’s MusicCaps table. Evaluated sample count is not specified in that table.

This MusicCaps experiment uses different model versions and a different evaluation method from the genre study. Its CLAP result does not override the genre study’s instruction-matching result.

Video-to-audio

How well does Sonilo generate sound aligned with video?

Sonilo Sound Effects 1.0 obtained lower DeSync scores and higher ImageBind similarity than Mirelo v1.6 on Movie Gen Audio Bench and internal CineBench. The results favor Sonilo on the reported temporal and audiovisual alignment measures across both test sets. Video-to-audio results

Selected alignment metrics. The repository’s tables do not specify evaluated sample counts.

Sonilo also led all four AudioBox outputs on both datasets and every reported metric row on internal CineBench. On Movie Gen Audio Bench, Mirelo led PANNs Inception Score and the PANNs and PaSST Fréchet-distance variants.

Sound effects

How does Sonilo compare on text-to-sound effects?

Sonilo Sound Effects 1.0 achieved the highest reported CLAP score among Sonilo, ElevenLabs SFX v2, and Mirelo v1.6 on both Clotho and AudioCaps. CLAP measures text–audio correspondence using a learned model. Text-to-audio results

Higher CLAP scores are better under this evaluation. These scores are not percentages or prompt-success rates.

Sonilo also had the lowest reported KL values for both backends and the lowest VGGish Fréchet distance on both datasets. ElevenLabs led OpenL3 and PaSST Fréchet distance on both datasets, while Mirelo led AudioBox CU and PQ on AudioCaps. Sonilo’s AudioCaps CLAP lead over ElevenLabs was 0.0050.

Metrics

What do the benchmark metrics mean?

Relative musicality score

A 1–3 comparative score within the genre evaluation. Higher is better; it is not an absolute quality rating.

First or tied first

The share of prompt groups in which a system received the highest musicality rank, including ties. Not a head-to-head win rate.

CLAP

Model-based text–audio correspondence. Higher is better in these tables.

DeSync

Model-based audiovisual synchronization measure. Lower is better in these tables.

ImageBind

Video–audio embedding similarity. Higher is better in these tables.

KL

Divergence based on generated and reference classifier outputs. Lower is better in these tables.

Fréchet distance

Distance between generated and reference audio distributions in a specified feature space. Lower is better; values from different backends are not interchangeable.

Source: the genre report and audio_eval metric documentation.

Sources

Sources and methodology