Audio by the Meter

-

Figure 1: Pinot Gallizio, 'Industrial Painting'

Introduction

After meeting with Will and Philip, I thought it would be a good idea to start trying to run some machine learning on RPPtv web servers, at least as a proof of concept. As an starting point, I thought it would be nice if people could go to a web page, enter how much audio they want to generate, and then be served a wav file. So I made the simplest page possible, just to try to get the pipeline flowing. It generates lakeside sounds from a model trained by wavenet. The user can only specify the length of the recording. In my tests it takes about a minute and a half to generate 1 second of audio.

Installation

The code for this page lives in the ambisynth private repository. The main page is just simple html. On submit, the audio is generated by a CGI script written in python. The python script expects there to be a virtual environment where TensorFlow and librosa are installed. The repository contains an install script that creates the virtual environment. After running that, the website can be served with 
python -m CGIHTTPServer 8000
So far, I have only run it locally. I don't have the resources to run this live on the web.

Screen Shots



Figure 2: Screenshots of the main page, and the audio file page, after it has been generated




Figure 3: Server log while serving the index.html, generating the audio file, and serving it

Future Work

First, this needs to be set up on RPPtv servers. After that we can start trying to streamline it the user experience. For example, there should be a progress meter, it should save checkpoints so if a user kills it halfway through they can still get half of their audio. One person using it should not create a DoS for other users. Also, we need to add more audio classes that the user can select, and eventually other features we have discussed like multichannel, and realtime audio generation.

Comments

  1. UnknownSeptember 19, 2018 at 9:28 AM

    Excellent Michael.
    We will get Sam and Julian to have a look and assess how we can move this forwards.
    Thank you, this is all very interesting.
    Russ

    ReplyDelete

Post a Comment

Popular posts from this blog

Ambisonic Rendering in the Story Bubble

-
Image
Figure 1: Sound Sphere in the Center for Vision, Speech and Signal Processing (CVSSP) at the University of Surrey. The person is shown standing in the intended orientation, facing the front of the sphere. Introduction I recently needed some audio content that I could quickly render for playback in the sound sphere shewn in Figure 1. There are a reasonable amount of free ambisonic recordings available online, e.g. Angelo Farina Youtube So I spent some time putting together tools to render ambisonics in the sphere, and here I wanted to share what I learned in case anybody else at CVSSP wants to try this out in the future. Speaker Arrangement The sphere has 24 speakers (plus a subwoofer which I will henceforth ignore), which at the time of writing are arranged as shown in Figure 2 and Figure 3. Figure 2: Speaker layout. The center of the diagram represents the top of the sphere. The top of the diagram represents the front of the sphere. There is one spe
Read more

How I calibrated my contact microphone

-
Image
Introduction I used the Hsu-Nielson method to find the frequency response of my contact microphones. This method involves cracking a mechanical pencil graphite on the microphone transducer. The microphone in question is the one shown above, and described in detail here: http://michaelkrzyzaniak.com/marshmallow.php I specifically used the white, low-gain variant. Experimental Setup and Results Figure 1: The mechanical pencil used in this experiment, with 0.5mm number-2 graphite Figure 2: Three millimeters of graphite were extended from the pencil Figure 3: Audio interface used for experiment (Scarlett 2i4), showing the settings used. Figure 4: Video showing the graphite being cracked. A small amount of putty was used to prevent the tip of the pencil tapping the transducer after the graphite breaks. The graphite is placed on the transducer and constant pressure is applied. When the graphite breaks, the pressure is released in a very short time (shorter than th
Read more

WaveRNN

-
Image
Figure 1: CVSSP's Pet Wyvern. Introduction I've been thinking about how we can make audio synthesis faster. In particular, I would be interested in realtime soundscape synthesis, partially because I think it would be good for the project, and partly because it would be well aligned with my own personal research goals. I found two options: Parallel Wavenet For a while, I was looking at parallel wavenet . This can supposedly generate samples in realtime, and is now used in the Google Assistant. However, I have been unable to find a good, vanilla implementation of it, and the paper is really sparse on details. There are some projects on GitHub where people have started implementing it more than 6 months ago and have not finished, so given the time constraints of this project, implementing it myself doesn't seem feasible. Moreover, the training process is very intricate and involves training a couple of networks separately and then together -- which makes it really har
Read more