Synesthesizer: Physical Modelling and Machine Learning for a Color-Based Synthesizer in Virtual Reality

Giovanni Santini*

*Corresponding author for this work

Research output: Chapter in Book or Report/Conference proceedingConference Proceedingpeer-review

7 Citations (Scopus)

Abstract

The Synesthesizer is a software synthesizer inspired to chromesthesia, that kind of synesthesia that connects sounds and colors. While chromesthesia usually produces color perception in response to sound stimulation, this synthesizer does the opposite: sound is generated according to color detection. More precisely, RGB (Red Green Blue) values are detected (one pixel at a time) and used to determine the behaviour of five physical models for virtual instruments. The motivation for creating such a synthesizer arose from the will to generate a timbral continuum out of the color continuum, allowing to explore the relation between color spectrum and sound spectrum. The Synesthesizer has two additional possible applications: A picture can become a sort of score; graphic scores can have a different source of interpretation;Given its intuitiveness, it might allow even non-experts to explore the possibilities of sound synthesis. The current version has been developed in a Virtual Reality (VR) environment.

Original languageEnglish
Title of host publicationMathematics and Computation in Music - 7th International Conference, MCM 2019, Proceedings
EditorsMariana Montiel, Francisco Gomez-Martin, Octavio A. Agustín-Aquino
PublisherSpringer Verlag
Pages229-235
Number of pages7
ISBN (Print)9783030213916
DOIs
Publication statusPublished - 2019
Externally publishedYes
Event7th International Conference on Mathematics and Computation in Music, MCM 2019 - Madrid, Spain
Duration: 18 Jun 201921 Jun 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11502 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference7th International Conference on Mathematics and Computation in Music, MCM 2019
Country/TerritorySpain
CityMadrid
Period18/06/1921/06/19

Keywords

  • Machine Learning
  • Physical modeling
  • Synesthesia
  • Synthesizer
  • Virtual reality

Fingerprint

Dive into the research topics of 'Synesthesizer: Physical Modelling and Machine Learning for a Color-Based Synthesizer in Virtual Reality'. Together they form a unique fingerprint.

Cite this