Listening Tests

-


I started making a series of listening tests. These might eventually help us figure out e.g. the minimum acceptable audio sample-rate, what words people would use to search for sounds, or more generally if people find certain synthesis algorithms to be better than others, or as good as real recordings. They contain placeholder audio files right now. But I'm putting them here a) for feedback, and b) to facilitate focusing the discussion surrounding project outcomes.

The code for the listening tests has its own repository: https://github.com/michaelkrzyzaniak/Listening_Tests/tree/master

Classification

Take this test here


Figure 1: Screenshots of the classification test and its results.

Discrimination

Take this test here


Figure 2: Screenshots of the discrimination test and its results.

Ranking

Take this test here


Figure 3: Screenshots of the ranking test.

Word Cloud

Take this test here


Figure 4: Screenshots of the word cloud test and its results.

Video Coherence

Take this test here


Figure 5: Screenshot of the video coherence test.

Future Work

In light of our discussion with Russ and Will last week, we should also design tests that include audio and video together. Are the audio and video coherent? This might be an a/b test with two videos side by side (same video different audio) and listeners would select their preference.Update: This has been implemented, see Figure 5.

Comments

  1. UnknownAugust 9, 2018 at 10:07 AM

    This comment has been removed by the author.

    ReplyDelete
  2. UnknownAugust 9, 2018 at 10:09 AM

    Michael, this is very interesting.
    A lot of our people are on holiday at the moment but I will get some of the sound designers to try out these tests as soon as we can.
    Keep up the good work.
    Many thanks

    ReplyDelete
    Replies
    1. MikeAugust 10, 2018 at 3:05 AM

      Thanks! No rush on having sound designers look at this; it actually isn't ready yet. I just put in fake sound files so I could design the interface, so let's wait until we have more synthesized sounds that we are happy with.

      Delete
    2. UnknownAugust 10, 2018 at 8:34 AM

      Mike, thanks for that and please let us know when you are happy for this to be tested?

      Delete
  3. AnonymousAugust 17, 2018 at 2:23 PM

    These mock ups are very helpful for knocking up a quick experiment once you have some test files. Although simple, I believe we can get meaningful insights with even a small number of tests. For me, the ranking one seems likely to be most useful for evaluating engineering refinements, where the others can provide validation, benchmarking and summarisation.

    ReplyDelete

Post a Comment

Popular posts from this blog

WaveRNN

-
Image
Figure 1: CVSSP's Pet Wyvern. Introduction I've been thinking about how we can make audio synthesis faster. In particular, I would be interested in realtime soundscape synthesis, partially because I think it would be good for the project, and partly because it would be well aligned with my own personal research goals. I found two options: Parallel Wavenet For a while, I was looking at parallel wavenet . This can supposedly generate samples in realtime, and is now used in the Google Assistant. However, I have been unable to find a good, vanilla implementation of it, and the paper is really sparse on details. There are some projects on GitHub where people have started implementing it more than 6 months ago and have not finished, so given the time constraints of this project, implementing it myself doesn't seem feasible. Moreover, the training process is very intricate and involves training a couple of networks separately and then together -- which makes it really har
Read more

How I calibrated my contact microphone

-
Image
Introduction I used the Hsu-Nielson method to find the frequency response of my contact microphones. This method involves cracking a mechanical pencil graphite on the microphone transducer. The microphone in question is the one shown above, and described in detail here: http://michaelkrzyzaniak.com/marshmallow.php I specifically used the white, low-gain variant. Experimental Setup and Results Figure 1: The mechanical pencil used in this experiment, with 0.5mm number-2 graphite Figure 2: Three millimeters of graphite were extended from the pencil Figure 3: Audio interface used for experiment (Scarlett 2i4), showing the settings used. Figure 4: Video showing the graphite being cracked. A small amount of putty was used to prevent the tip of the pencil tapping the transducer after the graphite breaks. The graphite is placed on the transducer and constant pressure is applied. When the graphite breaks, the pressure is released in a very short time (shorter than th
Read more

Ambisonic Rendering in the Story Bubble

-
Image
Figure 1: Sound Sphere in the Center for Vision, Speech and Signal Processing (CVSSP) at the University of Surrey. The person is shown standing in the intended orientation, facing the front of the sphere. Introduction I recently needed some audio content that I could quickly render for playback in the sound sphere shewn in Figure 1. There are a reasonable amount of free ambisonic recordings available online, e.g. Angelo Farina Youtube So I spent some time putting together tools to render ambisonics in the sphere, and here I wanted to share what I learned in case anybody else at CVSSP wants to try this out in the future. Speaker Arrangement The sphere has 24 speakers (plus a subwoofer which I will henceforth ignore), which at the time of writing are arranged as shown in Figure 2 and Figure 3. Figure 2: Speaker layout. The center of the diagram represents the top of the sphere. The top of the diagram represents the front of the sphere. There is one spe
Read more