Illustration from https://source-separation.github.io/tutorial/intro/src_sep_101.html

Using source separation to improve audience recordings

As bootleg collectors know well, audience recordings of concerts have existed ever since portable recording devices became available. With the advent of cell phones, the practice has become widespread. Most recordings are uploaded directly to YouTube or similar platforms, but their audio quality is generally poor, and it is common to come across “remasters”, i.e. attempts at audio restoration.

In this post, I illustrate the use of AI-based source separation for this type of restoration. I use the Open-Unmix algorithm, which produces six outputs: bass, drums, guitar, “other”, piano, and vocals. Source separation makes it possible to process these outputs individually, which is much more practical than working on the full mix.

With audience recordings of live performances, source separation may only be partially successful. In the Alt-J, Ekkstacy and The Cure examples below, for instance, the guitar alternates between the “guitar” and “other” outputs, where it may be mixed with other elements.

I show four examples, using the following live recordings:

• Alt-J, “Bane”, Copenhagen, November 5, 2022
• Ekkstacy, “Stain”, Tulips, Fort Worth, July 19, 2025
• The Cure, “Pornography”, Strasbourg, May 27, 1982
• Fred Avril, “Eve +++++”, Montreux Jazz Festival, 2002

In these examples, I avoid processes such as denoising and dereverberation. I find denoising often destructive, while dereverberation does not yet seem sufficiently mature. The processed files keep the original’s character, while improving their intelligibility and, where possible, their expressivity.

Method

The aim is to use source separation to process the extracted tracks individually, using equalization (EQ), dynamics processing (DYN), amp simulation (AMP), reverberation (REV), and noise reduction (NR), mainly through Avid (Pro Tools) and Waves plug-ins. The processing also includes volume automation on individual tracks, both to optimize the separated-track content and to add expressivity to the final output. The image below summarizes the process.

Model of process.

A specific class of processing concerns stereo imaging correction — often critical in audience recording. In stereo, source position is perceived mainly through three cues: intensity difference (∆I), time difference (∆T), and head-related transfer functions (HRTFs). On the separated tracks, I make extensive use of frequency-selective ∆I and ∆T correction, using the following non-commercial software developed at Sony CSL Paris.

XSpecMatch is a real-time spectral-matching plug-in that balances energy between the left and right channels for all frequency bands.

Sony CSL’s XSpecMatch, used for ∆I stereo correction.

MultiBand Phase Matching is an offline process that estimates and corrects time offsets between the channels (EP3673671A1).

Sony CSL’s MultiBand Phase Matching, used for ∆T stereo correction.

Alt-J, “Bane”, Copenhagen, Nov. 5th, 2022

Alt-J are a British band formed in 2007. Their defining trait is their singularity: some listeners love it, others do not, but it is certainly not music one hears every day. “Bane”, the opening track of the band’s fourth album (2021), is an ironic piece about cola addiction, built from a series of contrasting sections that eventually culminate in an evocation of God’s absence.

Alt-J’s songs are often complex, sometimes fragile, and difficult to perform live. A particularly articulate version of “Bane” can be found online, from a 2022 concert in Copenhagen. Unfortunately, the audio quality is poor, with much of the song coming through the left channel. It therefore seemed worth creating a more balanced and listenable version.

“Bane”, original audio (blue) and separated tracks (black). The black rectangle shows the L/R unbalance.

Simply boosting the right channel would have been insufficient: in the sections where its energy drops, its timbre also changes. The chosen solution was therefore to apply frequency-selective ∆I stereo correction using XSpecMatch, giving the two channels a reasonably similar timbre.

∆I correction did not solve all imaging problems: the perceived sources still consistently appeared to come from the right, suggesting a phase issue, as confirmed by the image below. To address this, ∆T stereo correction was applied using offline Band Phase Matching.

Phase analysis of the drum and guitar tracks. The right channel is late, leading to the perception of the source coming from the left channel.

Even after the ∆T and ∆I corrections, some subjective left–right directionality remained in the stereo image. I therefore narrowed the drum panning and reversed the piano panning.

Additional processing was more conventional, including slight compression on the separated drum track. The separated bass track was left alone, as processing only seemed to introduce additional audible artifacts.

“Bane”, process summary.

Volume automation was also applied, as in a standard mixing process, to improve intelligibility and expressivity.

“Bane”, volume automations.

Here is the result, along with the original video. The video quality is low and would need enhancing on its own, but it’s not the object of this post 🙂

[embed]Alt-J, “Bane”, processed version.

Ekkstacy, “Stain”, Tulips, Fort Worth, July 19th, 2025

Khyree Zienty, better known as Ekkstacy, is a young Canadian musician who came to public attention in 2021 with “I Walk This Earth All by Myself”. His third album, Forever, was released in 2025. Heavier than his previous records, it features several tracks driven by dark, dense guitars, including “Stain”, with its continuous, slightly oppressive riffs.

A live performance of the track is available online. Although the crowd is small, it remains intensely enthusiastic throughout the show, leaving the singer somewhat bemused and unsure how to react. Unfortunately, the recording is poor: the guitars are barely audible and the vocals muffled. It therefore seemed worth producing a version in which, at the very least, all instruments are audible.

“Stain”, process summary.

Since much of the guitar ends up in the “other” source-separation output, the “guitar” and “other” outputs were merged. I then tried to revitalize all tracks using a range of standard mixing processes, including volume automation on the vocals.

One specific process involved using XSpecMatch on the master track, not for ∆I correction but in side-chain mode for EQ purposes. In this configuration, the plug-in tends to match the spectrum of the master to that of the original version. The wet/dry setting was set to approximately 50%, providing a compromise between the two spectra. For a reason I do not quite understand, in this case, the overall result sounds subjectively better when it remains closer to the original tonal color.

Here is the result, along with the original video.

[embed]Ekkstacy, “Stain”, processed version.

The Cure, “Pornography”, Strasbourg, May 27th, 1982

Following the release of the album of the same name, The Cure’s 1982 Pornography tour has been described as a difficult one. Too many dates over two years, extensive drug and alcohol use, internal conflicts… After the Strasbourg show, frontman Robert Smith and bassist Simon Gallup got into a fight and returned separately to England, leading to the cancellation of several subsequent concerts. During the Strasbourg performance itself, Smith sounds both dejected and angry.

“Pornography”, the title track of the album, is a troubled piece. Among the available bootlegs, the Strasbourg version stands out as one of the most intense. The second half of the song contains some of the most deranged music I know. It seemed worthwhile to make this performance more clearly accessible. While the concert is available online, the source I used is a lossless bootleg-trade recording.

“Pornography”, process summary.

In addition to ∆I and ∆T processing for stereo-image correction, the workflow included stabilizing and enhancing the lead vocals. On the master, while a parametric EQ addressed a resonance near 5 kHz, the Sony CSL Profile EQ (EP3688756B1) was used to conform the low-frequency balance toward a predefined target. The use of such a target is particularly useful when addressing low frequencies, which are difficult to hear and depend heavily on the monitoring system.

Automatic bass frequency correction using Profile EQ.

Overall, the process was difficult to implement, with the guitars constantly switching between “guitar” and “others” and poorly defined drums. Here is the result, along with the original fan-made slideshow from YouTube.

[embed]The Cure, “Pornography”, processed version.

Fred Avril, “Eve +++++”, Montreux Jazz Festival, 2002

Frédéric Magnon, better known as Fred Avril, released his debut album, That Horse Must Be Starving, on the French label F Communications in 2002. He followed it in 2004 with Members Only. Although the second album did not achieve the same recognition, it contains a number of finely crafted, at times deeply personal tracks. One striking example is “Eve +++++”, a delicate yet exuberant love song.

After this second album, the worldwide record-industry crisis hit hard, and, like many musicians of his generation, Fred Avril turned increasingly toward film music. I cannot help wondering what later Avril albums might have sounded like had he continued his own project. A live version of “Eve +++++” is available online, and it seemed worth enhancing it, in order to preserve a sharper memory of that period.

“Eve +++++” can be divided into three main sections: (A), with vocals; (B), a central instrumental section; and (C), again with vocals. The main issues identified in the live version were excessive low-frequency content in the lead vocals during sections A and C, poor drum intelligibility in section B, and a missing synth lead in section B. Secondary issues included an overly resonant bass in sections A and C, a lack of bass intelligibility in section B, a weak guitar part in section B, and poorly defined keyboards throughout.

Because section B has its own specific characteristics, it was processed separately. Since the studio version has the same tempo as the live version, I also applied source separation to it and used some of the extracted elements as patches.

“Eve +++++”, processes used on separated sources.

The studio drums in section C required extensive manual synchronization, as the live part differs from the studio one. The “other” — mainly synth — patches from the studio version required stereo reimaging to better fit the live elements.

“Eve +++++”, capture of the ProTools session, drum tracks.

Here is the result, along with the original video.

[embed]Fred Avril, “Eve +++++”, processed version.

Conclusion

Having “remastered” live audience recordings for years, I find source separation to be a considerable step forward. Even when the separation process works less well than it does with studio recordings — with substantial leakage between tracks — being able to process the audio by groups or stems makes the work much more satisfying.

Our team’s page: Sony CSL Music — expanding creativity with A.I.