Text
| Site: | INTERactive environment for MUSIC learning and practicing |
| Course: | Network Music and Shared Network Music Techniques |
| Book: | Text |
| Printed by: | Guest user |
| Date: | Friday, 6 March 2026, 7:33 AM |
Description
This dissertation has as target the study of the state of the art in computer networks applications on musical instruments and sound design and musical composition environments.
The method used is the analysis of the modern technologies used and experimentation of some of them.
Results are diverse, a major focus has been given to gathering as many as possible technologies and up to date practices and to development of a software that could allow anybody to join this experience without any previous technical experience.
Being a subject in continued development, these result shouldn’t be intended as a completed work but as a work in day-to-day update.
1. Introduction
The interest in the subject of networks and interaction in a computer and people network in real time is an argument widely discussed nowadays. Applying it to the audio and music field, thanks to the adoption of network technologies, it has been possible to solve problems, propose new ideas and artistic forms. We will see how thanks to this process the creation of works including the “space” of networks was allowed, with a new form of acoustics and acoustics components, to the point of becoming an element to keep in mind during the composition process.
Networks are used in various field of the musical and sound design field, as Álvaro Barbosa, associate professor and coordinator of the Creative Industries Faculty at the Saint Joseph (USJ) in Macau, described in his PhD. dissertation entitled “Displaced Soundscapes”, in which he provides a detailed picture of the state of the art of networks in music by historically analyzing from the first tests in this subject to our days, explaining through experiments and works during his PhD period, how networks can add value to works and projects, exposing theories regarding interaction and principles of interaction through them.
The purpose of this dissertation is experimenting these technologies, having has previous knowledge and experience common concepts in the field of networks and programming basics.
Being this a very broad territory to experiment in, the practical part of the dissertation was focused mostly on two sides: real time interaction in a Laptop Orchestra and distance learning and performing with a split formation between multiple locations distant many hundreds of kilometers.
2. Overview
We can look at how in the field of music they are well established from a long time.
In terms of musical and performance education are present systems and infrastructures well tried for distance learning, actual classrooms and halls equipped with audio-video systems able to host a proper frontal class in which the lecturer is hundreds of kilometers distant from the student, with a little perception difference from a traditional situation.
In an experimental situation we have already attended performances with the ensemble split in multiple locations, reducing the distances between locations through powerful computer systems able to offer a time gap imperceptible to the human hear.
These technologies are still in development, but commercial solutions are close to a potential “standard” are constantly hitting the market.
Networks are, in one way or another, used to improve arts, starting from the first Personal Computer ensembles, to internet portals for sharing audio samples as freesound.org, to the incredible installations places in different locations of the world, connected between them in order to obtain one big installation to ”pass through the earth” like the one made by Atau Tanaka, Global String.
Álvaro Barbosa, one of the most important scholars in network arts and the application of networks in music, explains in his publication which are the key points for performers connected in an interactive real time to be comparable to frontal interaction.
Thanks to technological innovation, real time systems are in continued growth meanwhile deferred systems are mostly part of everyday life, with millions of daily access between websites and sharing applications.
The difference between similar technologies results from the obtained performance, first and foremost the capacity to have the shortest possible latency, even in few milliseconds, highlights a key role in the selection progress of the technology to adopt, together with the audio - video stream quality, for example. Multi-platform compatibility is an important element that the majority of real time systems still can’t implement up to today because of the requirements hardware and software to take in account during the development of these kind of products.
Important steps are being done in the direction and sharing of music production project by the means of DAW (Digital Audio Workstation) thanks to the introduction of Plug-ins or whole softwares able to connect multiple computers on the same project, like development repositories, allowing producers to work simultaneously on co-written projects, providing among other things, solutions regarding undo/redo controls on edits from each user and installation management for external plug-ins installed client side on user’s computers.
3. Distance Learning
One of the biggest interest regarding new technologies in the field of networks for learning environments is made up by Distance Learning. Various institutions have already set up and configured permanently classrooms equipped with audio-video systems allowing students to have frontal lectures in real time together with lecturers of other institutions hundreds of kilometers distant. Thanks to Milan’s conservatory and to the interest of Professor Giovanni Cospito, we managed to join one of the most important conferences in this sector, in which the most advanced technologies and the most recent areas in research and discussion have been showed: the “Network Performing Arts Production Workshop 2015” (NPAP) in London.
3.1. Network Performing Arts Production Workshop
The Network Performing Arts Production Workshop 2015 has been held in London in the Royal Academy of Music, between representatives of commercial and experimental systems like Polycom, Ultragrid and LOLA (Consortium GARR) together with artists and people in charge of the distance learning department of some ot the most important academies between Europe and the United States of America.
The three workshop days have been a sequence of demonstrations of the capability and theories on the showed technologies and discussions on arguments to address in the development of these technologies.
3.2. Why we chose LOLA
As it’s already been stated before, differences between the various systems, albeit small, may allow completely different projects to be realized, between the various, the one that drew our attention was LOLA (LOw LAtenty audio visual streaming system) developed from the G. Tartini Conservatoire in Trieste in collaboration with Consortium GARR.
The difference with other systems reside in the significant minor latency obtained thanks to the use of a dedicated Gigabit connection, which helps avoiding bottleneck effect on common networks and a better audio quality thanks to a less aggressive compression algorithm.
The LOLA systems needs to be executed on a powerful machine in terms of processor and graphics, meanwhile the Polycom systems, thanks to their proprietary cameras and software/hardware with user-friendly options.
Other systems such as Polycom showed to be able to reach, on average, a superior video quality at the cost of a worse audio quality and more latency, mainly because of the use of common networks and a design which was not focused, at least at the beginning, on musical interaction but on dialogue and video conferences.
Nevertheless in the last updates some major improvements have been brought in terms of quality, like the new “Music Mode”, reducing the quality gap with competitors.
3.3. LOLA Workstation
Once confirmed that the system from GARR would have been used for this first experience, we started, with their experts, a period of research and discussion on the hardware specs on which run LOLA. The system requirements specifically request a fairly high processing power, both on processor and graphics, some storage capability in order to store audio and video recordings of the sessions and compatible video hardware with their software.
In order to ensure optimal performance we decided to use high quality hardware, such as gaming hardware for motherboard and video card, an Intel i7 5960X 8 Core processor with hyperthreading (16 thread) on a liquid cooled h100i cooler from Corsair, a SSD for the Operative System (Windows 7, software requirement), two Black hard drives from Western Digital, one Terabyte each, for storing recordings, and an Intel Gigabit NIC (software requirement).
The video interface is a Nvidia GTX 980 4GB.
The audio interface, key element in the system, is a RME HDSPe-AIO, this is one of the internal PCI-Express from RME which ensures a input/out latency of 0.7 milliseconds while using 32 bit sample buffer, optimal in order to reduce latency time on the workstation side.
The camera is a Ximea (LOLA compatible) USB 3.0 camera.
The monitor is a gaming monitor, features 2k resolution at 144hz refresh rate and a latency of less than 1 millisecond.
The speakers are studio 5” speaker from Dynaudio, directly connected to the RME audio interface in the workstation through its own breakout cable.
Once the workstation is acquired and after installing all needed pieces of software, we started internet connection tests with the Royal Danish Academy of Music (RDAM) helped by Professor Thomas Solak and by the LOLA support team from Conservatorio di Musica “G.Tartini” in Trieste.
The system is expected to work on a properly set up network, like the one that GARR provides (GARR is a National Research and Education Network (NREN), each country has an equivalent to it) avoiding passing through a complex routers or switches. Tests highlighted the need of reconfiguring some parts of the Conservatory local network: available classrooms able to host the LOLA workstation were provided with CAT6 ethernet cables going directly to the main switch.
Once installed and tested the cables, the need of changing the switch unit with one able to support the actual traffic load generated by the connection with LOLa, in fact the previous switch wasn’t able to support the stream of data at the chosen video resolution and compression.
Once prepared the network and finished the preliminary tests we tested the whole system with a string quartet, in preparation for the concert shared between Milan’s conservatory and RDAM.
The tests happened between the Verdi Hall in Milan’s Conservatory and the Distance Learning classroom in RDAM
4. Concert
The organization of the concert, chaired by the internationl offices of the two institutions agreed for an evening split in four different performances, showing the repertoires shared between the two schools and the technological possibilities achieved.
4.1. Organization
The second part showed how a Vocal Mastercal can work through distance learning, with Professor Kirsten Buhl Møller from Copenhagen and a Singer from Milan, together with a piano accompanist.
The third part has seen a improvisation performance between chinese traditional instruments from Copenhagen and live electronics from Milan, with 4 components of the Laptop Orchestra 1h20nein in the Live Electronics section.
Fourth part included choirs from the two schools in a classic repertoire, the direction of both choirs was entrusted to Professors Martin Nagashima Toft and Luca Colombo, with a repertoire of Heinrich Schutz and Johann Sebastian Bach.
Being the evening split in four different parts, it was decided to prepare on the stage a mobile setup with the workstation inside a wheeled rack. Another table of the height of 50cm supported the LOLA screen and the two audio monitors.
Microphones were four DPA 4099 on the string quartet, one per instrument. On the choir two DPA 2011 in NOS technique (30cm distance in a 90° angle).
On the masterclass have been used three Atm350 from Audio Technica, one on the singer and two on the piano.
The impressions on the evening have been very positive, some technical inconveniences caused communication issues in the video quality but overall the concert was successful.
At the end of this experience a questionnaire was sent to all musicians, teachers and technicians in Milan and Copenhagen who took part to the event in order to gather opinions and ideas regarding the environment and the concert (See appendix C).
Thanks to the questionnaire and to the personal experience on stage it was possible to formulate some feedback to LOLA developers:
create the chance to include a black frame of an image file as video filling/mute to avoid disconnecting the videoconference in case of stage changes
a bug regarding last gen NVidia video cards was found and reported to the team
4.2. Ensemble Performance Threshold (EPT)
Musicians from both sides reported that the ability to follow and play music with others through a videoconference online system implies particular attention to timing and compensating delays, especially in case of latency higher, even if of few milliseconds, over a certain threshold.
This threshold is called EPT - Ensemble Performance Threshold (Schuett, 2002) and it defines the maximum temporal distance possible between performers to allow a symmetrical and synchronic performance between the two parts of the videoconference.
“In a vocal conversation it is possible to tolerate as much as one-way delays of 500 ms (Holub, Kastner & Tomiska, 2007). But, for the Human ear to accurately perceive two simultaneous sounds, they should not be displaced in time over 20 ms (Hirsh, 1959), which leads to notion that for mutual awareness to be supported in a bilateral performance, this threshold should be between 20 and 40ms (the time period that it would take one performer to perceive the other performer’s reaction to his action).”
It was described by Alvaro Barbosa in Displaced Soundscapes how EPT is a fundamental aspect in order to perform music in a networked environment:
“[…] the ability to perform music synchronously is strongly dependent on many factors, beyond the musicians experience, such as: the music expressive qualities (Dynamics and Articulation); the music style (rhythm, melody, harmony); the music timbre qualities; the music structure/form; the musician’s practicing strategies; the listening conditions.“
“In addition, when performing remotely, musicians face the inverse effect of the real world visual feedback, since real time visual cues accompanying the sound are not possible as in the physical space. Video requires transmission of heavier data and over the Internet the sped of transmitting sound is actually faster than transmitting images (the opposite of the atmosphere). Therefore, in Networked Music Performance, musicians tend not to rely on gestural cues from videoconference and use primarily the sound feedback to perform […]“
During tests in preparation for the concert it has been clear how EPT it’s a fundamental part in deciding repertoire.
In fact, it has been necessary to reduce the length of the Beethoven piece for the string quartet, since latency during rehearsals was higher than the performers EPT.
Also, listening the rehearsals it is clear that the performance was played in a slower tempo than normal, further confirming the exceeding of EPT.
5. Appendix - Lola Workstation specs
- Intel i7 5960X Octacore 3.00 Ghz Haswell-E Extreme 20MB 5GT/s 140W sk2011-3
- MSI X99S Gaming 7 Intel X99 8*DDR4 SATAIII SLI/Crossfire sk2011-3 ATX
- Samsung MZ-7KE256BW SSD 256GB 850 PRO SataIII 550/520MB/s 100K IOPS
OEM
- Western Digital HDD 2TB Black 3.5'' 64mb SATA III
- Crucial CT4K4G4DFS8213 16GB Kit 4*4GB DDR4-2133 PC4-17000 CAS15 1.2v
- Gigabyte NVIDIA GeForce GTX980 WindForce 3 OC 4GB GDDR5 256bit 2*DVI/HDMI/3*DP Pci-Ex 3.0
- Cooler Master Serie V 1000Watt PFC Attivo Modulare Nero 80+ Gold
- Cooler Master RL-N24M-24PK-R1 Nepton 240M CPU Cooler Intel/AMD
- Cooler Master Cosmos SE Full Tower No Power ATX Nero 18*2.5''
- Logitech MK120 Kit Tastiera/Mouse Usb black
- Microsoft Windows 7 Professional SP1 64bit OEM UK
- Intel EXPI9301CTBLK Intel 82574L Gigabit Ethernet PCi-Express x1 Bulk
- ASUS PG278Q 27'' Monitor Wide 2560x1440 1ms Pivot Multimedia DP 3D NVIDIA G-SYNC Nero
6. Appendix - Questionnaire
- What was your role in the performance?
(Qual è stato il tuo ruolo nella performance?)
- Do you think that the service and instruments provided during the rehearsals and the performance where appropriate? Why?
(Pensi che il servizio e gli strumenti forniti dal conservatorio durante le prove e il concerto fossero adatti? Perché?)
- Do you feel like something was missing during your performance? (Acoustically, visually)
(Pensi che mancasse qualcosa durante la tua performance? (Acusticamente, visivamente))
- What do you think it's different from a traditional performance? Did you have problems with it?
(Cosa pensi che sia diverso da una performance tradizionale? Ti ha creato dei problemi?)
- Would you be interested in partecipating again in some distance learning/performing rehearsals and tests?
(Saresti interessato nel partecipare di nuovo in un alcune prove e test di distance learning/performing?)
- Did you have to make up for the latency? (Latency is the delay between the two sites of the performance). If yes, did you synced with video or audio to keep time? Both? How?
(Hai dovuto compensare la latenza? (La latenza è il ritardo tra i due siti della performance). Se sì, ti sei sincronizzato con il video o con l'audio per tenere il tempo? Entrambi? Come?)
- Do you have any suggestions?
(Hai dei suggerimenti?)
- Other
(Altro)
7. Bibliography
• Á. Barbosa, Displaced Soundscapes, VDM Verlag Dr. Mueller e.K. , 2008
• Á. Barbosa, Internet-based Sound Art - USJ, University of Saint Joseph, Macao, S.A.R. , China UCP-CITAR, Research Center for Science and Technology of the Arts, Portugal, MIT journal Article
• F. Halsall, Computer Networking and the Internet, Addison Wesley, 2005
• G. Cospito, Riflessioni sul medium elettroacustico nell'era delle tecnologie dell'informazione e della “Galassia Internet”: modelli sociali, tassonomia delle pratiche creative e questioni di senso, Journal Article,
• D. Tanzi, Tecnologie dell'informazione e comunicazione dell'esperienza musicale, tesi finale, Conservatorio di Musica “G. Verdi” di Como, Scuola di musica elettronica e tecnologie del suono, a.a. 2006-07, rel. G. Cospito
8. Sitography
• http://msp.ucsd.edu/techniques/latest/book-html/
• https://en.flossmanuals.net/pure-data/
• https://cycling74.com/products/max/#.V1Wz3Md6SRs
• http://www.conservatorio.trieste.it/art/ricerca/progetto-lola-low-latency
• https://www.jisc.ac.uk/janet
• http://www.geant.org/Projects/GEANT_Project_GN4
•
•