Open Collections

UBC Library and Archives

The science of soundscapes : systems creating expressions of mood and text Thorogood, Miles; Macklem, Larissa 2017-08-15

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.

Item Metadata


67246-Thorogood_M_et_al_Science_soundscapes.mp3 [ 27.02MB ]
67246-Thorogood_M_et_al_Science_soundscapes_transcript.pdf [ 140.48kB ]
JSON: 67246-1.0355663.json
JSON-LD: 67246-1.0355663-ld.json
RDF/XML (Pretty): 67246-1.0355663-rdf.xml
RDF/JSON: 67246-1.0355663-rdf.json
Turtle: 67246-1.0355663-turtle.txt
N-Triples: 67246-1.0355663-rdf-ntriples.txt
Original Record: 67246-1.0355663-source.json
Full Text

Full Text

            TITLE: The science of soundscapes: systems creating expressions of mood and text PODCAST SERIES: Frequencies DATE RECORDED: August 15, 2017       CREATORS: Miles Thorogood and Larissa Macklem CONTRIBUTORS: Karin Haug, Sajni Lacey, Arielle Lomness, Trevor Neill, Michelle Tinling, and Mathew Vis-Dunbar AFFILIATIONS: Library, UBC ; Creative and Critical Studies, Faculty of (Okanagan) -- Creative Studies, Department of (Okanagan) ; Irving K. Barber School of Arts and Sciences (Okanagan) -- Computer Science, Department of (Okanagan) SCHOLARLY LEVEL: Faculty ; Other NUMBER OF SPEAKERS: 2 Speakers TRANSCRIPT STYLE: Smooth/Intelligent Verbatim FILE DURATION: 24 minutes 11 seconds     SPEAKERS:  [LM]: Larissa Macklem [MT]: Miles Thorogood   [0:00]  [Music Intro] [Music fades out]   [LM]  This is Larissa Macklem and you are listening to Frequencies, a podcast from the Library at UBC Okanagan.  [Music fades in] [Music fades out]  [LM]  Today I will be talking with Miles Thorogood, from UBC Okanagan’s Faculty of Creative and Critical Studies.  [Music ends]  [MT]  My name is Miles Thorogood. I’m an instructor here at UBC.  [LM]  We are here today as part of our Science Literacy week podcasts. Thank you so much for coming in. Perhaps to start you could tell us a little bit about your background?  [MT]  So I’m a product of the research agenda of the new media art of the 1990s where a number of institutions around the world and creative practitioners [throat clearing] were asking the question about what happens if we start to intersect the arts and the sciences, you know specifically computer science but then also mixing other branches of science into that as well with the idea of having some artistic or creative endeavor that’s attached to that. Uh and so you have you know artists such as you know, Stelarc who was working with engineers and roboticists and his body in creating this quite wild performance pieces. And to uh I guess to elevate myself into that kind of field of working in the arts and sciences I would be taking um courses in interactive multimedia which includes like computer science as well as formal music training and then other kinds of artistic practice and scientific courses as well.  [LM]  Quite the blend and um, is that a trajectory that you had anticipated for yourself as a younger person, um, seeing…a not strictly arts or strictly kind of science path. Is that something that you always knew that you wanted, was kind of an interwoven perspective?  [MT]  It was just something that made sense for me going forward. Ah one of the difficulties with um taking this kind of blended form of learning of like you know arts and sciences was that you know, talking with colleagues or fellow students that were in the scientific discipline, they would be like, well you know we don’t really understand exactly what you’re trying to do because it’s you know not completely science, what you’re doing from what our understanding of what science is. Uh and then talking with traditional artistic practitioners uh they’re like well you know you’re creating stuff, that’s kind of cool but I’m not too sure I understand why you might be working with technology because that’s not what I do. So there’s a point where it’s like you know an identity crisis almost of you know which camp do I belong into as far as my academic and practice goes. And so, you know just through the idea of that worrying me out the door and just kept on doing it, the things that made sense to me, putting together this learning trajectory.  [LM]  So could you tell us a little bit about your current projects or areas of research?  [MT]  Uh yes so current project is called Audiometaphor. Audiometaphor is a system that turns texts into sound designs. So a user will uh type in a sentence into the computer such as “I’m on the beach in Mexico, the sun is shining and the gulls are making noise in the background” and uh the program will then do a text analysis by finding semantic identifiers in the text as well as taking out some sentiment attributes as well, things such as mood. Like so we can expect a sunny day to be you know positively valenced or have a pleasurable sort of sound to it. Uh the system will then be able to go through a series of information retrieval steps of searching large databases of audio files to find parts of those audio files which match the search criteria that is being extracted from the sentence. Then use a number of artificial intelligence algorithms including machine learning to be able to select parts of those sounds and to combine them together to create the final sound design which is a representation of the text input.  [LM]  Hm that sounds very interesting. Do you have a favorite output of Audiometaphor so far?  [MT]  Um, at the moment I’m liking outputs which tend to be positively valenced but have a low amount of arousal to them. So things such as quietly walking in the forest. So um I enjoy that kind of motif. [laughter]  [5:10]  [LM]  Well let’s take a listen to a sample soundscape, here’s a clip of ‘a sunny day by the sea side’. [Soundscape]   [LM]  Furthering kind of the questions around science literacy specifically, so um.. what courses are you teaching this upcoming term and how do you hope to blend in opportunities for your perspective on science literacy into those courses?  [MT]  Um hm, ah so the classes that I’m going to be teaching this coming term are Computer Science 150, that is Physical Computing and also media theory. I’m teaching a Visual Arts course also which is Sound Art and in these courses I take a design thinking perspective to the way that students are able to you know formulate problems and come up with alternative solutions and to carry out you know the unfolding of one of those solutions and then evaluate that. So the Physical Computing course I’m getting them to make Internet of things devices as well as design systems where that can be meaningful as well. So taking a human centered approach to the design of Internet of things enabled devices and objects. The Sound Art course I’m going to be teaching which is a second year VISA course will include a performance aspect to it, an instrument building as well as a data sonification project too. And so in undertaking those projects students will be doing not only design but also the development of the systems that are required to be able to create sound with the computer uh and maybe marry that will acoustic instruments also, and then also be able to make sense of data using sound.  [LM]  Mm. So the scientific principles, like just the um trial and error kind of experimental design, those are kind of the key practices that you essentially are drawing from the realm of science at a very basic level?  [MT]  Um hm. Cognitive science also plays a big part especially within the sound art field. Um of looking at how you know human creative processes can be represented by a computer and the machine. Then uh perceptual psychology will also play a big part in that. So we having a look at some of the research of early acoustic researchers like Claude Risset, Max Mathews, uh and some of the psychoacoustic kinds of effects that we can then synthesize using the computer.   [LM]  Are there any areas um that you have found yourself needing to do some research in that you hadn’t anticipated?  [MT]  Um in regards to the classes that I’m going to be teaching?  [LM]  Or in just in your own research itself?  [MT]  Well it’s often the case that you start researching something and then you realize that it fits into a research field of its own. Uh so for example um when I was doing some research regarding the soundscape and the sound design composition I was having a look at it from a human centred perspective of how people perceive sound, I was doing it initially from you know sound design practice and design field. After exploring that somewhat it came across that really what we’re doing was looking at the field of affective computing and some early research in that domain by Rosalind Picard and then also UBC um researcher were looking at how we can represent mood within a particular um cartesian space and so by mapping sounds out into this sort of affective perceptual space we then use that as our model for being able to have a computer make human like decisions based on mood attributes of sound in the sound design process.  [9:50]  [LM]  You’ve also had I think in your career a number of public installations of pieces is that correct?  [MT]  Uh yes, so my moonlighting is as a new media artist. So I work within the domain of interactive artworks and installations. And these have been um, displayed at you know Vancouver Olympics, as well as galleries and public spaces in Canada and Australia and elsewhere.  [LM]  And any time that you’re putting out an installations, do you have a best case like desired outcome for the impact of that installation?  [MT]  Um hm.  Ah so always public engagement is central. And so typically we’ll have a design brief about what it is that we are wanting to create. So it might be there’s going to be some narrative or story that underlies the purpose of the installation. Always. But then it’s like well, you know do people just understand what it is you're trying to relate to them as far as the narrative goes or do they get engaged you know at a deeper level. And so what we have to look at doing is going through the design process of thinking about who the user is or who the viewer is in this case. Uh so that goes to the idea of different kinds of viewer types that you might experience in a museum for example where you will have you know, school children coming through, you’ll have adults coming through, you know over a weekend, maybe taking their grandparents out and they will all want different things, get different things from what it is that they are going to be interacting with. So for example, you’ll have some users or viewers which are you know just interested in taking a very surface level reading of the installations. Other ones which want to be able to explore it more deeply. And other ones which are interested in showing other people what it is that they have learned from it as well. So taking these visitor types into consideration is important.  [LM]  And how do you, how do you go about getting the feedback on who is using and how they are using the installation? Do you sit there, as a you know like a bystander and just observe people or how do you go about getting that information?  [MT]  Ya so evaluation of public art is a tricky thing and an open? problem. So typically when we go to evaluate systems that we are designing in our research what I’ll have a look at ah is um, saying you know, listen to this soundscape that my system has developed or generated and then listen to something that a human has generated and tell me which one you prefer. And if the statistics come up and show that there is no significant difference between how people rate soundscapes or sounds designs which are generated from my system compared to ones that humans have generated I can say that it produces human like sound designs. But within the public art context what you can look for is, you sit and have a look and take an observation about how people are playing and interacting with the system. So you know maybe they’re taking more time than you would expect for somebody that would not be interested in it. So let’s say that somebody who is not interested or doesn’t get involved in the interactive artwork might spend you know 30 seconds because they are working past it, maybe walking slowly, ah but they are not really involved in it. Whereas somebody who is getting involved in it and you designed the unfolding of the experience to take like 10 minutes maybe they are spending 7 minutes there you can say it’s more or less effective because they are spending 7 minutes in front of it instead of 30 sec. Now 7 mins is still probably a long time for, you know people in public art context to be engaged with something but you know if you can, the longer you can get somebody to be involved with your story and your message you can consider that to be more of a success. Now the way that you know we can take those metrics is by sitting down and observing but that can obviously take a lot of time because well these installations they don’t normally turn off while the venue is open. Now because we’re dealing with computation technologies what we can also do is to observe the metadata of the users behavior and glean some information from that and be able to evaluate the exhibition based on those.  [14:50]  [LM]  In general do you think the typical public is very computer science literate?  [MT]  Uh it’s going to be changing more and more. We’re finding that children which are working their way through the K-12 program are becoming they call them digital natives or you know various other common nomenclature which is associated with people that have been using technology since infancy almost now.  Uh so I mean there’s a difference between you know using technology and being able to create something with technology. So more and more we’re seeing technology you know pervasive in our cultures but at the same time we’re seeing it become more and more user friendly so you know a lot of the time people aren’t actually creating or understanding what is happening in the technology that they’re using. So for example programming computers, creating things with digital technology is becoming more and more necessary for students to make it through the K through 12 program. I mean in Canada here we have more computer science literacy coming into the foreground of those programs. So when I see people that are working with you know the installations that I create often it comes down to button mashing. They’ll come and you know just see how much it works and if it doesn’t seem to work for them they’ll like just sort of pass it back because it isn’t as user friendly as something they might find that is given to them otherwise. Ah so there is this bit of play between creating something that is user friendly and then something that requires work to be able to uncover what is inside of it. And you know being able to create an experience that works for both of those types of viewers, that’s where some of the strengths are. So for example like creating something that is immediately accessible, like you know has some moving graphics on it, and it has a button to make the moving graphics move more. So you know in the example I was giving of the simulation of birds, you might press a button and it scares the birds and they flutter away or they fly around a little more quickly. But maybe there’s a sort of deeper interaction there as well where um, you know if you just take time and stay still instead of like you know mashing the button and just going for that instantaneous reward of some effect, but if you just take time and spend it there with the system maybe it starts to evolve further, the birds come closer to you in the screen and you start to get more of the story, perhaps they even start speaking pieces of poetry in between each other that you get to hear that you might not otherwise hear if you didn’t take that time.  [LM]  I can see how that could foster some scientific depth of thinking about an installation like that where like you said it’s not just going for instant gratification which is very prevalent in a lot of our interactions with technology these days. But encouraging that waiting attitude and the problem solving of well what happens if I wait? What happens if I don’t push the button? So that seems a really engaging approach for encouraging that in a really, um…uh non-stressful like, or not stressful, but in a way that doesn’t make people necessarily think that they’re learning about scientific principles in a sense. So I just find that a really interesting approach.  [MT]  Or if they start pushing the button in different ways. What happens if you stroke the button?  [LM]  Right ya. All kind of things like that  [MT]  Stroke all the buttons there you go [laughter]  [LM]  Along the lines of general public and scientific or computer science specific literacy, do you have any suggestions that you would make to people listening of things that they could do to improve their own computer science literacy at kind of like a deeper level, because I think a lot of people have a very surface level interaction ability to use and new products and devices but not necessarily to think deeper about what’s going on behind the scenes.  [19:15]  [MT]  The first step is always like you know, start to speak the language of computers. And the language of computers is code. And so looking at creating a simple program even like you know a webpage from scratch uh is an empowering um vehicle for being able to start to work within the domain of computer science. So you know we find that to create a webpage right now, it doesn’t take much. You just go to some template in some content management system and say you know here give me a webpage, here’s my name, here’s my content and it basically develops that for you. Ah but cutting your teeth on the tools of web design using HTML 5 will give somebody more depth of knowledge then just doing something from a template. So one of the nice things about computer science and computer literacy is that you know all the good stuff is free. It’s based on open standards so if we’re talking about ah web design then it’s based on HTML 5 which is open web standards. There’s a plethora of good resources out there online that someone can start to look into for creating their own web space. The Mozilla developer network is one such vehicle. That’s the work from the Mozilla Foundation which has a look, which looks very much at computer literacy and extending the knowledge of being able to work in computers to a wider audience in an open and free way. And you know, similarly moving on from there if you were to develop apps for phones or other you know kinds of software, then again there is many free tools and free programming languages that people can start to tackle. So it’s always like what is interesting for somebody and then once they can articulate what the things they are interested is, there is a solution out there within the domain of like you know free open source software that they can start to play around. One such tool that can also extend beyond building web pages is And this is um an implementation of a Java like language that allows you to create um you know dynamic and interactive graphics very easily and so that’s one of the tools that we would teach students that are taking design and computer literacy course.  [LM]   Hm that’s a great bunch of resources that you’ve mentioned. Thank you. When you are communicating your research out do you find that you communicate it differently to different audiences?   [MT]  Um that’s something that I certainly try to do. So for example you know speaking with you know colleagues in the field of computer science we’ll you know talk about the algorithms that we develop and that we implement into our research in a way that we expect that they will understand, that they will have knowledge of that set of algorithms. Whereas speaking with somebody who is within a different field, let’s say the artistic community I’ll speak much more about the higher level sorts of design decisions and processes that we are looking at modelling not necessarily the implementation of the modelling itself. Uh so being about to communicate to different audiences is certainly something that is very important because science touches us all and we all have different ways of accessing it.   [LM]  That seems like a perfect wrap up statement [laughter].  So I’d just like thank you Miles for coming in and talking with us today. It’s be a really engaging conversation and I hope our audience gets as much enjoyment out of it as we’ve had today. Thanks very much.  [MT]  Thank you very much. It’s been a pleasure.  [LM]  The soundscapes you heard as part of today’s podcast were created with the website a tool created by Miles Thorogood.   [Music fades in] [Music fades out]  [LM]  You have been listening to Frequencies a podcast from the Library at UBC Okanagan. I was your host today, Larissa Macklem. Editing by Karin Haug, Larissa Macklem, Anna-Marie Krahn, and Mathew Vis-Dunbar. Music by Trevor Neill. Artwork by Alison Ward. Additional support provided by Michelle Tinling, Sajni Lacey and Arielle Lomness. Thank you for listening.  [Music fades in] [End] [24:11] 


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items