Sony Computer Science (CSL) Researchers Design a Machine Learning based Model that automatically acquires samples that match specific Music Track from Large Archives
The process of modern digital music creation is quite different from what was obtainable in the past. There are lots of acoustic aspects to be considered which drives the musical composition of the day. For instance drum samples can determine the features of the percussive aspects of a musical composition.
Ascertaining the suitability of specific drum samples for certain musical contexts depends on the aesthetic judgement of an artist. Nonetheless, ascertaining this suitability is a painstakingly exerting process because the artist has to most likely browse through an extensive library. This process may easily asphyxiate the creativity of the artist. Sony Computer Science Laboratory (CSL) from France are working on Machine Learning algorithms that will help music producers to identify and recover specific audio samples from a large database. Samples are quite revealing.
The samples the library of an artist contains can be graded according to its suitability for different musical contexts at different levels of the production process. Sony has now taken the initiative by unveiling SampleMatch a machine learning based algorithm which automatically picks out drum samples from a large archive that matches a particular music track. The model will be officially unveiled in December during the revered ISMIR 2022 conference, which is dedicated to the retrieval of music information.
The application of AI in music creation so far has barely scratched the surface as it has infinite potentials waiting to be harnessed. The benefits are quite enormous. What Sony is prospecting is the development of AI applications that would make music production easier for music producers. Any one with even the faintest idea of music production can tell how tiresome it is for creatives to select drum samples. Producers have to carefully search for the most suitable drum sample to balance a music track without drums.
Searching, discovering and settling for a drum sample is quite stressful because drum sample libraries contain quite a lot. Till date, simple computational tools which help music producers choose percussion samples are too few and limiting. The process also involves filtering huge datasets with keywords or tags.
Previously, the team at Sony had made unsuccessful attempts to design a system to easily and quickly aggregate drum samples. Their attempts failed due to technological limitations and the esoteric nature of the whole system. On this previous limitations however, progress has been recorded due to contrastive learning coupled with the success achieved in neural network encoders. This development has made it easier to easily and accurately estimate the suitability of two data points. What this implies is that it is now easier to implement the system which was hitherto considered too esoteric. With this design, artists simply transmit their track into their system at any point during the creative stage with SampleMatch. Once the system is fed, it automatically uses the algorithm to identify what sample is best for the track.
In designing SampleMatch, the team at Sony used 4,830 electronic music songs and 885 popular pop and rock songs to shape the system. While there may be existing designs using extracted musical features to link audio samples, the nature of such samples and the prearranged features shapes the retrieval quality
During the computational process of matching score for drum samples, determining what features to include is a a grey area without a clear cut definition. Although SampleMatch was developed to identify percussion samples that suit particular tracks, it is unique in that it is applicable to various types of audio matching. There is liberty to configure the model to extract bass, guitar, or other instrumental sounds that complement a track through training sample pairings.
In the nearest future, Sony CSL will expand the audio retrieval model to facilitate the identification of the most suitable drum or instrumental samples to support music producers in their compositional process. A lot can also be achieved by studying how the data on the system was organized. For example this can create updated hypotheses to shape the art of music production.
Through reverse engineering conducted by researchers into the system a number of foundational guidelines can be enacted to guide how musicians use the system to blend their music. The Sony CSL team will now use the SampleMatch approach with its DrumGAN technology to create drum samples that accurately balance a particular tune.
As originally reported in (https://www.marktechpost.com/2022/10/11/researchers-at-sony-computer-science-lab-csl-have-developed-a-machine-learning-based-model-called-samplematch-that-can-automatically-retrieve-drum-samples-that-match-a-specific-music/)
