3d sound application for game environments - IDt

3D SOUND
APPLICATION
FOR GAME
ENVIRONMENTS
14/10/2014 – ALFRED TÅNG
Supervisor: Daniel Kade, Mälardalen University
Examinator: Thomas Larsson, Mälardalen University
MDH
BachelorThesisinComputerScience
Abstract
Thisreportcoversthecreationandimplementationofa3DaudioapplicationusingFMODExAPI.
Thereportwillalsocoverawalkthroughofthebasicprinciplesof3Dandsurroundaudio,
examplesofotherusesof3Daudio,acomparisonbetweenavailabletechnologiestoday,both
softwareandhardwareandfinallytheresultoftheimplementationofthe3Dsoundenvironment
software,bothserverandclient.Theapplicationwascreatedtoexploretheuseof3Daudioin
immersiveenvironments.Therewasnoapplicationlikethisavailablewhenthisprojectwas
conducted.Aninductiveapproachalongwithaformofrapidapplicationdevelopmentand
scenariocreationwasusedtoachievetheresultspresentedinthisreport.Theimplementation
resultedinaworkingclientandserversoftwarewhichisabletocreatea3Dsoundenvironment.
Basedonauserevaluationthesoftwareprovedtobequitesuccessful.Withthehelpofthe
implementationtheuser,oroperator,cannowcreateasoundenvironmentforanotheruser,ora
listener.Theenvironmentiscreatedanddesignedbytheoperatorusingtheclientsideofthe
implementationandlaterplayedthroughtheserversidewhichisconnectedtoa4.1speaker
system.Theoperatorcansteerandaddsoundsfromtheclienttoanactiveenvironmentandthe
listenercanexperiencethechangeinrealtime.Thisprojectwasconductedasabachelorthesisin
computerscienceatMä lardalensUniversityinVä sterå s,Sweden.
TABLE OF CONTENTS
TableofFigures__________________________________________________________________________________________ 1
Introduction______________________________________________________________________________________________ 2
3dAudio,SurroundAudioandtheCrossTalk________________________________________________________3
3DAudio _____________________________________________________________________________________________3
SurroundAudio______________________________________________________________________________________3
Crosstalk_____________________________________________________________________________________________4
Summary_____________________________________________________________________________________________5
State-of-the-Art __________________________________________________________________________________________ 6
SimilarProjects_________________________________________________________________________________________6
AudioChile ___________________________________________________________________________________________7
Cocktail–AMMAEImplementation_______________________________________________________________7
Summary_____________________________________________________________________________________________8
RecentStudies__________________________________________________________________________________________8
ProblemDefinition_______________________________________________________________________________________ 9
Challenge _______________________________________________________________________________________________9
AvailableHardware__________________________________________________________________________________9
AvailableSoftware_________________________________________________________________________________10
Method________________________________________________________________________________________________11
Approach___________________________________________________________________________________________11
ObtainingInformation____________________________________________________________________________12
Scenarios___________________________________________________________________________________________12
ComparisonofTechnologies___________________________________________________________________________13
AudioSoftware_______________________________________________________________________________________13
Summary___________________________________________________________________________________________14
AudioHardware______________________________________________________________________________________14
Headphones________________________________________________________________________________________14
Loudspeakers______________________________________________________________________________________15
Multipleusers-problem___________________________________________________________________________15
TABLE OF CONTENTS
Summary___________________________________________________________________________________________16
SystemDesign___________________________________________________________________________________________17
DesignProcess _______________________________________________________________________________________17
Scenarios___________________________________________________________________________________________17
Lookandfeel_______________________________________________________________________________________18
SystemImplementation________________________________________________________________________________22
GettingtoknowFMOD_______________________________________________________________________________22
Prototype_____________________________________________________________________________________________22
Server_______________________________________________________________________________________________22
Client_______________________________________________________________________________________________23
Network____________________________________________________________________________________________23
ProblemsandObstacles_____________________________________________________________________________24
UserEvaluation_________________________________________________________________________________________25
Evaluation ____________________________________________________________________________________________27
Discussion_______________________________________________________________________________________________29
Conclusion____________________________________________________________________________________________29
FutureDevelopement________________________________________________________________________________31
References_______________________________________________________________________________________________32
TABLE OF FIGURES
TableofFigures
Figure1-Attheleftabirdsoundisplayedthroughanormalsurroundsystem.Thesamebird
soundisplayedintherightpicturebutknowfroma3Dsoundsystem.Theusercannow
determinethepositionofthesoundmoreaccurately..............................................................................................4
Figure2-ThecrosstalkistheaudiowavesrepresentedbyRxandLx............................................................5
Figure3-ScreenshotofthegameHalf-Life,released8thNovember,1998.(ValveCorporation,n.d.)
................................................................................................................................................................................................................6
Figure4–ScreenshotsofdifferentsituationsfromAudioChile.(Sá nchez&Sá enz,3DSound
InteractiveEnviromentsforBlindChildrenPorblemSolvingSkills)................................................................7
Figure5-Picturerepresentingthetestroom.A)Speakers,B)TheaudiointerfaceandC)The
server...............................................................................................................................................................................................16
Figure6-SketchshowingscenarioA.............................................................................................................................17
Figure7-SketchshowingscenarioB.............................................................................................................................18
Figure8-Screenshotoftheserversoftware..............................................................................................................19
Figure9-ScreenshotfromWindowsMetrosimulatorrunningtheclientsoftware.............................20
Figure10-Conceptofoneofthediscussedfinaldesigns...................................................................................21
Figure11-ScreenshotofthefirsttestapplicationusingFMOD......................................................................22
Figure12-1.Usersendsaconnectionrequesttospecifiedip-adressandport.2.Theserverthen
answertheclientwithanack-message.........................................................................................................................23
Figure13-Pictureshowingthetestsetup.A-Theaudiointerface,B-Theserver,C-Theclient25
Figure14-Pictureshowingatestpersonplacedatthesweetspotinthetestingenvironment.A–
Speakersconnectedtotheaudiointerface..................................................................................................................26
Page1
INTRODUCTION
Introduction
Thisreportdescribesthecreationandimplementationofathesistosupportaresearchin
computerscienceconductedbyDanielKadeatMä lardalensHö gskola(MDH)inVä sterå s.The
basicideabehindDaniel’sresearchistofindeasywaystocreateahighlyimmersiveenvironment
fortheuser,wherethisthesiscontributestothetaskofexploringandimplementingthebasisof
thesoundexperienceresearch.Thereportwillgothroughthebasicprinciplesofthree
dimensionalaudio(3Daudio),softwarelibrariesandtheimplementationofacontrolpanel
softwarecompletewithserverandclient.Thegoalofthisprojectistodevelopthecontrolpanel
softwaresotheusercancreate,controlandrecordathreedimensionalsoundenvironmentfor
anotherpersonplacedinthecenterofthenewlycreatedsoundenvironment.
Theclientsideofthecontrolpanelwillbeusedasatooltocontroltheserversideofthesoftware.
Theservertranslatesmessagessentfromtheclienttosoundinstructionswhichlaterbeing
renderedwiththeuseofthesoftwarelibrarycalledFMODExtoa4-wayloudspeaker-system
throughanexternalsoundcard.Thethreedimensionalsoundenvironment,whichisgenerated
fromthesoundinstructions,isthenexperiencedbyanotherpersonstandinginthefocalpoint
createdbythespeakers.
Page2
INTRODUCTION
3D AUDIO, SURROUND AUDIO AND THE CROSSTALK
3Daudioandsurroundaudioareactuallytwodifferentthingsbutoftenmistakenasthesame.
Muchbecausethecurrenttrendinhomeentertainmentwhereeverythingshouldbe“High
Resolution3DSuperQuality”,wherethe3Dattributeisusedasasellingpointandhavenothing
orverylittletodowiththesoundimageoftheproduct.
3D Audio
3Daudioisoftenusedingamesorotherinteractiveapplicationsandhelpstocreateamore
realisticenvironment.The3Daudiotechnologywerecreatedtomimicthereallifeaudio.
Comparedtosurroundaudiothe3Daudioisamorecomplexandmoreimmersiveaudiosystem.
3Daudioisusedtocreateamorevividprojectionofthesurroundingenvironmentforthe
listener.Muchlikethesurroundaudio,thelistenerisabletosaywhichdirection,amoreprecise
direction,thesoundiscomingfrom.Besidesthedirection,thelistenercancomprehendthe
distanceaswell,muchlikewecaninournormallives.Allinall,thesimplecharacteristicsof3D
audiocomesdowntothis(Thorn,2013);



Volumeandattenuation
o Togivethelisteneranideaofthedistance
Panninganddirection
o Togivethelisteneranideaofthedirection
Reverberationandacoustics
o Togivethelisteneranideaofthesurroundingenvironment
Evenifa3Dsoundsystemconsistofonlyastereosetup,likegamingheadsets,thelistenercan
experiencethesethingsreallywellduetosoftwareenhancementslike“RazerSurround”(Inc.,
n.d.)or“EAX”.Thesoftwareaimstosimulateasurroundsoundsystemintheheadphonesusing
techniqueslikeHRTF(GardnerG.W.,1997)tocalculatehowthesoundshouldbeplayed.
Surround Audio
Thesurroundaudiosystemisoftenusedforhomeentertainmentorbycinemas.Thebasicidea
behindthesurroundsystemistomakethelistenerunderstandthebasicdirectionofthesoundto
enhancetheimmersiveexperience.Forexample;fromtheback,thefrontoroneofthesides.The
listenercannotreallyunderstandhowfarorhownearthesoundisbecauseitwillalwaysseemas
ifthesoundisbythespeakersorspeaker,seeFigure1.
Page3
INTRODUCTION
Figure1-Attheleftabirdsoundisplayedthroughanormalsurroundsystem.Thesamebirdsoundisplayed
intherightpicturebutnowthrougha3Dsoundsystem.Theusercannowdeterminethepositionofthesound
moreaccurately.
Crosstalk
Backintheearlydaysof3Daudio,roughlyatthetime3Dgamesbecamepopular,manypeople
saidthat3Daudiowasbestexperiencedthroughheadphones.Thismaybebecauseitismuch
easiertoisolatethe3Daudiosourcesanddeliverthemtoeachearwithoutanyinterferingaudio
signals(Song,Zhang,Florencio,&Kang,2010),alsoknownascrosstalk.Forexample,let’ssay
thatonesoundisplayedintheleftloudspeaker.Thissoundisonlymeantfortheleftearbutas
youknowyouwillhearitintherightaswell.Thesamegoesfortherightloudspeaker.When
thesethingshappensthe3Daudioeffectfailsandthesoundyouwillhearwillsoundasanormal
stereosoundwherethetwodifferentsources,leftandright,aremixedtogether.SeeFigure2for
anillustratedexplanationoftheproblem.
PerdefinitionintheCollinsEnglishDictionary(Dictionary.com),crosstalkis;unwantedsignalsin
onechannelofacommunicationssystemasaresultofatransferofenergyfromoneormore
otherchannels.
Page4
INTRODUCTION
Figure2-ThecrosstalkistheaudiowavesrepresentedbyRxandLx.
Todayitispossibletoeliminatecrosstalkbyconstructingspecialaudiofilters.(GardnerW.G.,
1999)Thesetypeoffilterscreatesabarrierbetweenthetwospeakersalmostlikeavirtualsetof
headphones.Thistechnologymakesiteasierforeveryonetoexperience3Dsoundinamore
comfortableandcheaperwaywithouttheuseofheadphonesorcomplexsurroundsetups.
Summary
Dependingonthesituationthetwotypesofaudioimagerysystemspresentedearlierisused
differently.Inamovie,TVorotherstaticentertainmentwheretheimmersionalreadyislimited
thesurroundaudiosystemismostused.Thisismaybebecausethesurroundaudiomodelis
fairlyoldandreliableandthereisnoneedofyou,theviewerorlistener,tointeractwiththe
entertainment.Indynamicentertainmenthowever,suchasgamesorvirtualreality,where
immersionandinteractivityplaysahugeroleoneshouldthinkaboutusinga3Daudiosystem.
The3Daudiosystemhasthebestabilitytorecreateaudioaswephysicallyhearitinour
everydaylife,makingtheillusionofrealitymuchmorerealthanthesurroundaudiooption.
Page5
STATE-OF-THE-ART
State-of-the-Art
SIMILAR PROJECTS
3Daudiohasbeenwellusedinthe
gamingindustryforyears.Half-Life
(Figure3)wasoneofthefirstgamesin
theindustrythathadsupportfor4
channel3Daudio.TodayFMOD
(FirelightTechnologiesPTYLTD.,n.d.),
thesoftwarelibrarythatisusedto
achievetheresultsinthisthesis,is
usedinavarietyofgames.Including
AAA-titlessuchasBioshockandCrysis.
(FirelightTechnologiesPTYLTD.,
2013)Ithasalsobeenintegratedin
Figure3-ScreenshotofthegameHalf-Life,released8th
thefamousgameengineUnity(Unity November,1998.(ValveCorporation,n.d.)
Technologies,n.d.).
FMODisjustoneamongmanyotherenginesavailableforthedevelopers.3Daudiodonotexist
onlyinentertainment.3Daudioisareallygoodtooltouseiftheuserisunabletoexperience
visualfeedback,forexamplewhentheuserisdriving.Alotoftheresearchin3Daudiohasbeen
donetocreateaneasierlivingforpeoplewithseeingdisabilities(Sá nchez,Baloian,Hassler,&
Hoppe,2003).Withthehelpof3Daudioseeingimpairedindividualscanmoreeasilyvisualize
theirsurroundingsorevenusecomputers(Frauenberger&Noisternig,2003).Therearealso
otherusesfor3Daudio,includingeducationalandworkrelated,whicharefurtherexplained
below.
Page6
STATE-OF-THE-ART
AudioChile
“3DSoundInteractiveEnvironmentsforBlind
ChildrenProblemSolvingSkills”(Sá nchez&
Sá enz,3DSoundInteractiveEnviromentsfor
BlindChildrenPorblemSolvingSkills),isaproject
thatuses3Daudiotohelpblindorseeing
impairedchildrenincreasetheircognitiveand
learningskills.Thesoftwaretheycreated,
AudioChile,isagamewheretheusertravelsto
differentcitiesacrossChile.Theuserthensolves
mazesandpuzzlesusingimpressionsand
feedbackfrombothstereoand3Daudiotogether
withsimpleonscreengraphics(Figure4).
Themainpurposeoftheirstudywastoseeif
youngchildrenwithvisualdisabilitiescould
easierdevelopcertainproblem-solvingskillsand
alsoseeifthistypeofvirtualenvironmentwould
beanyusefulingeneral.The3Dsoundwasused
asahelptodirecttheuser,localizedifferent
objectsorcharactersandnavigatethegame
world.
Figure4–Screenshotsofdifferentsituationsfrom
AudioChile.(Sánchez&Sáenz,3DSound
InteractiveEnviromentsforBlindChildren
PorblemSolvingSkills)
Cocktail – A MMAE Implementa on
Aprojectcalled“Cocktail”(Edlund,Gustafson,&Beskow,2010),whichwasbasedatKTHin
Stockholm,used3Daudiotocreateaflexible3Daudioenvironmentthatcouldbechangedin
real-time,muchlikethegoalofthisthesis.Cocktailreliesonthesocalled“cocktailpartyeffect”
(Benyon,2010),therebythename.Theeffectoccurswhenthelistenerexperiencealotofsounds
atthesametimeandfocusesononlyone,thenmakesajudgmentifitisimportantornotand
finallymovesontothenextsound.
Page7
STATE-OF-THE-ART
Thesoftwarecreatesa3Dsoundscapeusingarandomsetofsoundsthatisbeingplayedallat
oncewhentheuserstarts.Thesoftwareselectsrandomsoundsfromarepositorycontainingover
thousandsofsoundsthathasbeenconfiguredinadvance.Probabilitiesforacertainpositionofa
sound,orforacertainsoundtobeplayed,canbeconfiguredatruntime.Cocktailthenplaysthese
sounds,whichcanbefromahundredtoathousandormore,allatonce.Thisismadeavailable
throughtheirsoundengineMMAE.MMAE,comparedtoothernormalsoundengines,can
computethousandsof3Dsoundobjectswithverylowcomputationalcost.Thesoftwareuses
SnackSoundlibrary(Sjö lander,2006),writtenbyKå reSjö landeratKTH,asitsbackbone.
Summary
Therearedifferentaspectsfrombothoftheseprojectsthatisinterestingforthisthesis.Onething
istheareaofapplicationof3Dsound.“3DSoundInteractiveEnvironmentsforBlindChildren
ProblemSolvingSkills”explainsthe3Dsoundasatooltowidenthecognitivefunctionsforyoung
children.Thishopefullymeansthattheproductofthisthesiscanserveaseducationaltool,in
additiontoitsentertainmentapplications.Anotherinterestingabilityfortheproductofthis
thesiswouldbeforittobeabletoplaysoundsatthecapacityofCocktail.
RECENT STUDIES
Inrecentyearsthemainfocusin3Daudioresearchhasbeentoenhancethequalityofthe3D
soundandmakeitmoreavailabletoconsumers,oneofthesestudiesiscalledS3A: Future Spatial
Audio for an Immersive Listener Experience at Home (ResearchCouncilsUK,2014).Thisresearch
isacollaborationbetweenseveralcompaniesintheentertainmentindustryandresearchersat
theUniversityofSurreyintheUnitedKingdom.Theentertainmentindustriesconsistof
companiesfromdifferentbranchesofentertainmentincludingTV,filmandgames.
Themainfocusoftheresearchistomake3Daudiomorepracticalandprovidenewplatformsfor
UK’sentertainmentindustriestodeliverhighqualityimmersivesoundtotheirusers.Today,a
highqualityexperienceof3Daudiorequiresanadvancedloudspeakersetupandthemajorityof
listenersdonothavethisathomeortogetherwiththeirmobiledevice.BythetimeS3Aisover
theresearchteamhashopefullyfoundawaywhere3Daudiocanbeexperiencedatthesame
qualityregardlessaudiosetup.
Theprojectisfullyfundedandasoftodaystillactivebutduetoitsearlystatenoresultshasyet
beenpublished.Dr.TimBrookes,supervisorfortheS3Aproject,haslistedthattheresearchis
plannedtocontinueuntil2018(UniversityofSurrey,2014).
Page8
PROBLEM DEFINITION
ProblemDefinition
Thetimebeforethisthesiswasconductedsimilartoolswasnon-existentornotabletocreatethe
immersivesoundenvironmentthatwasrequiredforthegoalofthisthesis.Withthehelpofthe
solutionprovidedinthisthesistheusercancreateandmovesoundsina3D-envirnomentfroma
clientsoftwarewhichisthenhandledbyaserversoftware.
Thisthesiswillserveastheaudiopartofalargerresearchprojectinimmersiveenvironments
andhopefullyprovideagoodfoundationtofurtherresearchineducationalandentertainment
purposesofimmersiveenvironments.
CHALLENGE
Createaclientsoftwarewhichcontrolsthe3Dsoundenvironmentrenderedbyaserverthat
receivessignalsfromtheclientsoftware.Theserversoftwareshouldsupportscalablesurround
systemsrangingfromsimpleheadphonesto7.1setupsormore(currentsoundinterface
supportsonlyupto7.1,seehardwarespecificationsbelow).Theclientshouldbeabletoselect
soundsfromalistofavailablesoundsontheserverandthenaddadditionaleffectssuchas
reverb,max/minvolumeetc.
Available Hardware
Audio Processing
Therearetwodifferentsystemsusedwhenprocessingaudio.Theloudspeakersystemconsistsof
fourGenelek8020CspeakersconnectedtoaSteinbergUR824audiointerfacewhichisconnected
totheserver.Theothersystemconsistsofjustonesetofstandardheadphonesconnectedtothe
standardsoundcardoutputoftheserver.
Genelek8020CSpeaker(Oy,n.d.)





Page9
Activespeaker
Frequencyrange:66Hz-20kHz
Dimensions: HxWxD,242x151x142mm
Lowfrequencypower:20W
Highfrequencypower:20W
PROBLEM DEFINITION
SteinbergUR824AudioInterface(Technologies,n.d.)






ConnectsviaUSB(24bit/192kHz)
8linechannels(output)
6XLRchannels(input)
OpticalI/O
Supportssurroundsetupsupto7.1(MAC)
SupportsiPad
Server
IfthepurposeistorunthesoftwarewiththeGenelekspeakersandtheSteinberginterfacethe
serverisaMacmachinerunningOSX10.7orlaterwithnetworksupport.Thisisbecausethe
Steinberginterfacedriversdon’tsupportsurroundsetupsinaMicrosoftWindowsenvironment.
ButforabasicsetupwithheadphonesaWindowslaptoprunningWindows8.1hasbeenused.
Client
TheclientsoftwareisrunningonatabletPCrunningWindows8.1withanIntelAtomCPUand
standardWi-Fiandtouchsupport.
Available So ware
TocontroltheoutputmixingfromtheSteinberginterfacetheaccompanyingsoftwarecalled
dspMixFxisused.Withthissoftwaretheusercanmonitorandcontrolhowtheinterfaceshould
handlethevariousbuilt-inchannels.
Razer’ssoftwareRazerSurroundisusedattheserversidetoenhancethesurroundfeelingwhen
theuserusesheadphonesinsteadoftheloudspeakersystem.RazerSurroundisavirtual7.1
surroundenginethatsimulatesafully-fledgedsurroundspeakersysteminapairofregular
headphones.(Inc.,n.d.)
Page10
PROBLEM DEFINITION
METHOD
Atthestartofthethesisthreemainpartswerediscussedanddecided.Thefirstpartconsistedof
informationgatheringandliteraturereview.Inthispartthemainpurposewastofindoutif
similarworkhadbeendoneanywhereelseandifso,howcouldtheirapproachbebeneficialfor
ourideas.Hardwareandsoftwareoptionswerealsoinvestigatedatthisstage.
Afteradecisionwasmadeaboutwhathardwareandwhichsoftwarelibrarywasmostsuitedfor
thisthesisthesecondpartbegan.Thispartconsistedmostlyinmeetingsanddiscussions
betweenstudentandsupervisoraboutpossiblescenariosandhowthesoftwarewasgoingtobe
used.Foramorein-depthlookoftheproductofthispartseeScenarios,page15.
Thelastpartwastoimplementaworkingdemo,aprototypeofthecombinedknowledgefrom
thetwoearlierparts.Theprototypewascreatedtoexplorethecapabilitiesoftheideasandthe
earlierdevelopedscenarios.
Besidesthesethreepartsthestudentmetwiththesupervisoreachweekthroughoutthethesisto
discusstheprogressandtogethelpincaseofproblems.Itwasalsoatthesetimestheweekly
planningwasdecided.Theplanningconsistedofsmalltasksthatwassupposedtobedoneuntil
thenextmeetingalmostlikescrumsprints(Schwaber&Sutherland,2013).
Severaldifferentmethodswereusedthroughoutthethesis.Inthetwofirstpartsofthethesis
standardmethodsforinformationgatheringandliteraturereviewtogetherwithmethodsfrom
interactiondesignwerechosen,includingbrainstormingsessionsandscenarios.Duringthe
implementationoftheprototypethisthesisincorporatedtwomethodsfromcomputerscience,
RapidApplicationDevelopment(orRAD)and,asmentionedearlier,abitofscrum.Aninductive
approachwasusedtocreateknowledgeandconducttheresearch.Theresearchwasbasedonthe
thoughttoexplorehowa3Dsoundenvironmentcanbeusedtocreateimmersiveexperiencesfor
users.Toprovethisreasoningandexplorethecapabilitiesoftheimplementationaquantitative
usertestwasconducted(seeUserEvaluation,page25).
Approach
Therearealotofdifferentwaystoapproachathesisandtoknowwhichwaywouldbebestfor
thisthesisthequestionthatismeanttobeansweredmustfirstberecognized.Withthehelpofan
inductiveapproachtogetherwithaquantitativeusertestthisthesiswilltrytogiveananswerifit
ispossibletocreateanimmersivesoundenvironmentinascenariowheretheusereasilysteers
thesoundinreal-timeforanotheruserthatisplacedinthemiddleofthecreatedsound
environment.Theresultwillbebasedontheusertest(page25)oftheprototypeexplainedin
“SystemImplementation”,page22.
Page11
PROBLEM DEFINITION
Obtaining Informa on
Atthisstagethemaingoalwastoachieveagreaterunderstandingaboutthelimitationsandthe
requirementswhenworkingwith3Daudio.Alotofinformationaboutearlierprojectsand3D
audioingeneralwasobtained,theinformationwasthensortedinrelevancytothethesisand
unwantedinformationwasdisposed.
Researchandobtaininginformationlikethisisacrucialpartinaproject.Itisfromherea
researcherplanthenextcourseofactionandpossiblyanticipatesunwantedpitfallsinthefuture
oftheproject.
Scenarios
Scenarioscreatedinthesecondpartofthethesisservedasabackgroundorasketchof
instructionsonhowthefinalsoftwarewasmeanttobeused.Noconsiderationswastakentoreal
worldlimitationswhenthefirstscenarioswascreated.Asthelastpartprogressed,the
implementationpart,thescenariosbecamemoreofafinalgoalthanasketchandmore
customizedtofitcertainlimitations.
Thismethodcombinedwithanopenmind,withoutthethoughtoflimitations,canbeausefultool
tocreateuniqueandwell-functioningsoftware.
Page12
COMPARISON OF TECHNOLOGIES
ComparisonofTechnologies
AUDIO SOFTWARE
Thereareseveraldifferentaudiolibrariesthatsupport3Daudio.Dependingonwhatplatform
andwhichfeaturesareneededoneisbetterthantheother.Togetabetterunderstandingof
whichlibrarywasthemostfittedforthethesisaninvestigationwasmade.
Thelibrariesthatweremostinterestingforthisthesiswere:




IrrKlang(Ambiera,n.d.)
FMODEx(FirelightTechnologiesPTYLTD.,n.d.)
OpenALSoft(Strangesoft,n.d.)
XAudio2(Microsoft,n.d.)
Thegoaloftheinitialinvestigationwastofindoutwhichoftheselibrariesthatsupportedthe
followingkeyfeaturesandcheckpointsthebest:






Howeasyitwastolocatethesound
Ifithasagoodqualityoutput
Ifitsupportsindividualspeakerassignment
Ifitispossibletopositionmultiplelistenersinonesoundenvironment
Ifitisscalable,fromstereo(twospeakers)toe.g.32speakers
Ifitisfree
Easytolocatethe
sound
IrrKlang
FMOD Ex
OpenAL Soft
XAudio2
5
4
5
4
4
5
3
4
NO
YES
N/A
YES
Scale1-5
SoundQuality
Scale1-5
Individual
speaker
assignment
Page13
COMPARISON OF TECHNOLOGIES
Multiple
Listeners
NO
YES
Scalable
N/A
YES
NO
YES
YES
YES
(Onlyupto7.1)
YES
YES
YES
Free
YES
(Non-commercial) (Non-commercial)
Table1–Anoverviewofthecomparisonofthesoftwarelibraries
Summary
Investigationshowedthatsomeoftheselibrarieswasnotinterestinganymore.Someduetolack
offurtherdevelopmentandothersduetomissingkeyfeatureslike,3Dsoundsupportor
scalability,seeTable1.Togetadeeperunderstandingaboutthepossibilities,differencesand
limitationsregardingthelibrariesthatstillwasinthepictureasimpletestsoftwarewascreated.
Thissoftwaretestedhoweasyitwastogetthewantedresultsandthequalityofsoundthe
libraryprovided.
AfteranextensivelookintotheremaininglibrariesonlyFMODandXAudio2wereconsideredan
option.TheoverallwinnerofthetestwereFMODduetoitseasyimplementationandgood
documentation.Thereisonethingtoconsiderthough,becauseofthebarebonestructureofthe
XAudio2APIonecouldwritemoreefficientandoptimizedcode.Duetoitsbarebone
characteristics,theAPIrequiremoretimetogetsimplefunctionalityworkingandtheXAudio2is
notcrossplatformcompatible.TheXAudio2APIisonlysupportedinWindowsandXbox,when
FMODissupportedonalmosteverywellusedplatformtoday.
AUDIO HARDWARE
Thereareavarietyofhardwareoptions,rangingallthewayfromheadphonesto9.2systemsand
soontochoosefromtowhencreatingafunctional3D-soundenvironment.Followingparagraph
containssomegeneralprosandconsregardingheadphonesandloudspeakers.
Headphones
Pros:



Page14
Availability
Cost
Easytoachieveeffect
COMPARISON OF TECHNOLOGIES




Compatibility
Easytostore
Mobility
Abilitytocloseoutsurroundings(Isolation)
Cons:


Lackthefeelingofspace(spacious-effect)
Longuse=uncomfortable
Loudspeakers
Pros:





Availability
Soundquality
Canbecosteffective,dependingonsituation(outputeffectetc.)
Spacious-effect
Largereffectiveareaor“sweetspot”
Cons:



Oftenbig
Moreperipherals=morecost
Hardtocloseoutsurroundings(soundfromotherhardwareetc.)
Mul ple users-problem
Whenusingheadphonesonlyoneusercanparticipateatatimeunlessmultipleheadphonesare
connectedtothesamesystem.Thiscanbeagreatdisadvantageiflet’ssaytwoplayersplaya
gameinthesameroom.Atthesametimeassoundisplayinginplayerone’sheadphonesplayer
twotriestocommunicatewithplayeronebutplayeronecannothearplayertwo’svoicebecause
ofthesoundthegameismaking.Thisproblemcanhoweverbeeliminatedifeachplayerwould
usesometypeofvoiceinputtothesystem.
Ifaloudspeakersetupwouldbeusedinstead,bothplayerswouldexperiencethesame3Deffect
andtheywouldprobablyheareachother.Takenotethatwhenyouareusingaloudspeakersetup
theonlysoundimagethatcanbeexperiencedisfromoneplaceinthegame,thismeansthatthe
twoplayersmustbeintheexactsamelocationatthesametimeiftheyplaythegametogetherin
thesameroomatthesamemachine.Thisscenariocouldforexampleexistinahelicopter
simulationwherethetwopilotsalmostsitsoneachother.Meaningthatthetwopilotswouldin
realityalmosthavethesameaudioenvironment.
Page15
COMPARISON OF TECHNOLOGIES
Summary
Dependingonsituationandpurposeofthesoundenvironmentoneismoresuitablethanthe
other.Forexample,ifthereisn’tsomuchspaceandthepurposeistogetthelistenerfully
immersedinagameorsimulationoneshouldconsidergoingforheadphones.Thisbecause
headphonesdon’ttakethatmuchofspaceanditcapturesthelistenerinamoreintensewaythan
regularspeakers.Headphoneshastheabilityto“shoot”thesoundmorepreciseintothelistener’s
earsandisolatethelistenerfromtheambientnoise.
Thereisnoreallyscientificanswerifheadphonesarebetterthanloudspeakersortheotherway
around.Thisdebatehasbeengoingonfordecadesandwillprobablycontinuetodoso.Asof
2013,atrendshowingthatheadphoneshavebecomemoreofafashioniteminsteadofa
functionalaccessory.Sotoday,ifyoulookatitthroughtheeyesofthegeneraluser,itshouldbe
headphonesbeforeloudspeakers.Notjustanyheadphones,theyshouldratherlookgoodthan
soundgood.(Arthur&Gibbs,2013)
Withthatbeingsaidtastesdifferfrompersontoperson,thisisreallyuptothelistenerandwhat
heorsheprefers.
Forthisthesishoweverthefocusissetonfourloudspeakersplacedinasquareform.Withits
currentmeasurements(fromonespeakertoanother)thissetupcreatesasweetspotofroughly
0.09squaremeters.
Figure5-Picturerepresentingthetestroom.A)Speakers,B)TheaudiointerfaceandC)Theserver
Page16
SYSTEM DESIGN
SystemDesign
DESIGN PROCESS
Scenarios
Fromdiscussionswiththesupervisortwopossiblescenarioswerecreatedwherethissoftware
couldbeused.
Scenario A
Figure6-SketchshowingscenarioA.
ScenarioArevolvesaroundoneuserstandinginthesweetspotandanotheruser,whichcanbe
seenasatypeofadministrator,standsoutsideofthesoundenvironmentandcontrolsthesound
usingthetablet.TheadministratororuserBcreatesasoundscapeforUserAwiththehelpofthe
controlpanelsoftwarerunningonthetablet.ThesoundisthenroutedtotheSteinberginterface
andfromthereouttothespeakerssurroundinguserA.
Page17
SYSTEM DESIGN
Scenario B
Figure7-SketchshowingscenarioB.
InthisscenariouserA,CandDhasamobiledevice,preferablyaphoneorsomethinginthatsize.
UserBisstillhavingthesameroleasinscenarioAbutinsteadoftheserversendingtheaudio
signalsthroughtheexternalaudiointerfaceitstreamssounddirectlytouserA,CandD’smobile
devices.UserA,CandDthenreceivesthesoundinapairofheadphonesconnectedtotheirown
mobiledevice.
Summary
ThescenariothatwaschosenasthegoalforthisthesiswasscenarioA(Figure6).ScenarioB
(Figure7)wasignoredduetoitsaddedcomplexity,wheretheserverstreamedthesoundto
multipleusersinsteadofplayingtheoutputdirectlytoasetofspeakers.Thisrequiredthe
developmentofanapplicationforamobiledeviceinadditiontothealreadydiscussedserverand
client,whichwouldbeatoogreattasktofinishinthegiventime.
Look and feel
Duringthewholethesistheprototypechangeditsdesignseveraltimes.Becauseofitspermanent
prototypestageappearancescamesecondhand.Thefocuswasmoreonthefunctionalityand
Page18
SYSTEM DESIGN
debuggingonbothsidesofthewholeimplementation(clientandserver).Evenifthefocuswas
setonfunctionalitytheideasofthefinallookwaskepttoallowforfuturerefinementoftheuser
interfaceandexperience.
Server
Figure8-Screenshotoftheserversoftware.
Whenlookingatthecurrentdesignoftheserver(Figure8)onecanquicklyseethatsomeofthe
featuresarereallyunnecessaryforafinalproduct.Forthefinalproducttheserverprobably
wouldhavebeenconvertedintoaserviceapplication,aconsoleapplicationorasimplerForm
application.Thenalmosteverycurrentfeaturecouldhavebeenremovedandreplacedwithsome
simplecommandlineoptionsorsomethingsimilar.
Page19
SYSTEM DESIGN
Client
Figure9-ScreenshotfromWindowsMetrosimulatorrunningtheclientsoftware.
Astheserver,muchofthecurrentcontrolsareimplementedforeasierdebuggingandevaluation.
Forexamplethewholeupperareaoftheapplicationconsistofalog(seeFigure9).The
applicationlogseverythingthatishappening,includingnetworkeventsandsoundevents,this
featureisoneofmanythatwouldnotbeconsumerfriendly.Thegeneralusershouldnotreally
seeorcareofwhatishappeninginthebackground.
Muchlikethescenariosafinallookwerediscussedduringthetimethatthethesiswere
conducted.Thediscussionsweremainlyabouttheclientsoftwarebecauseitwasthebiggestpart
whichwasmeanttobeseenbygeneralusers.
Page20
SYSTEM DESIGN
Future looks
Figure10-Conceptofoneofthediscussedfinaldesigns.
Oneofthebiggestdifferencesbetweentheconcept(Figure10)andtheactualsketch(Figure9)is
thatthepanelrepresentingthesoundarea,theareawiththereddotinthelowerleft,hasbeen
centeredandthewholelogcontrolhasbeenremoved.Thecoloreddotsrepresentsdifferent
soundsinthesoundareaandthebigblackdotrepresentsthelistener.Alloftheactivesoundsis
thenrepresentedinalistwiththeircurrentstatusandcoordinates.Itwasalsodiscussedthatthe
usershouldbeabletosaveandloaddifferentsoundscenariosandhavetheabilityto“paint”a
roadorsetwaypointsthataselectedsoundcouldfollow.
Atthefinalstagesoftheclientsoftwarethehopeswastohaveafinaldesignliketheone
describedabove.Asalways,whendevelopinguserinterfacesforthegeneralconsumer,one
shouldputagreatweightonuserfriendlyplacingofcontrolsandtousewellknowndesign
guidelines.Thismakesiteasierfornewuserstofeelfamiliarwiththeinterface(Benyon,2010)
andfurthermoreitmakesthemutilizethesoftwarethewayit’smeantto.
Page21
SYSTEM IMPLEMENTATION
SystemImplementation
GETTING TO KNOW FMOD
WiththehelpofsourcecodefromdifferentFMODexamples
thatcamewiththeAPIasimpleapplicationwascreatedin
VisualStudio2013usingC#.Theapplicationwasableto
playaselectedsoundatgivencoordinates.Thissoftware
becamethebaseoftheprototype’sserverpartbutitwas
initiallymeantasasimple“gettoknow”-applicationto
bettergrasptheunderstandingoftheFMODAPI.(Figure
11)
Inthistestapplicationtheusercouldplayasoundfromthe
harddiscdriveandthencontrolthecoordinatesofthe
soundusingthekeyboard.Whentheuserstartedthe
applicationFMODfetchedsysteminformationlikethename
ofthedriverandthespeakersetupthatwasconfiguredon
themachineitranon.AstheknowledgegrewaboutFMOD
underthetimeofthethesis,morepartswereimplemented
likesoundeffectsandsupportofmultipleaudiosources.
Figure11-Screenshotofthefirsttest
applicationusingFMOD.
PROTOTYPE
Duringthewholeperiodofthethesistheprototypeevolvedtoamorecomplexapplication
almosteachweek.Fromthestartitcouldbeconsideredasasimpleaudioplayerwithvery
limitedoptions.Thissimpleaudioplayerfinallybecameamorecomplex3Daudiorendererwith
thesupportofdifferenteffects,audioformatsandoutputs.
Theprototypeconsistsoftwomainsoftwareparts.Aclientwhichcansendinstructionstothe
otherpart,theserver.Theimplementationofthesetwopartsweredoneinparalleltoalwaysbe
abletomaintainfunctionalitywitheachotherwhentestingoccurred.Amoredetailed
explanationoftheimplementationprocessofthesepartscanbefoundbelow.
Server
TheserverwasimplementedusingC#andWindowsForms.InsteadofaWindowsForms
implementation,theservercouldhavebeenimplementedasaserviceoraconsoleapplication.
Butbecauseitevolvedfromthefirsttestapplicationtheformpartwaskept.Oneofthemain
Page22
SYSTEM IMPLEMENTATION
reasonstokeeptheexistingtestapplicationwasbecauseitwouldberathertimeconsumingto
copy-pasteandspecialfitthecodeintoanewproject.
Client
BecausetheclientsoftwarewereplannedtorunonaWindows8tablet,theclientwascreatedas
aMetrostyleapplication.TheMetrointerfacewasfirstintroducedinWindows8andismeantto
workasaseamlesstouchinterface.ThismeansthatmostoftheuserinterfaceofWindows8is
bestsuitedfortouchscreens.TheapplicationswrittenforWindows8Metroaremorelikemobile
applicationsinsteadofanormalcomputerapplications.
Withthisinmindtheworkbeganwithcreatingthedesignthatwasdiscussedwiththethesis
supervisor(seeDesignProcess).ThedesignwasimplementedusingXAMLinMicrosoftBlend
andafterallthecontrolsandbuttonswereinplacethecodewasimplementedinVisualStudio.
Network
Theconnectionbetweentheserverandclientdiffersalittlebitfromeachother.Becausethe
clientisaWindowsMetroapplicationthenetworkAPIisnotthesameasastandardForm
application.WhenMicrosoftreleasedtheSDKforWindowsMetroapplications(WindowsStore
app)theychangedthenetworkingpartintheAPItobeabletocommunicatewiththenetworkin
thebackgroundwhentheappisrunning.
Inastandardformapplication,liketheserver,the
System.Net.SocketsnamespaceisusedtocreateaTCPconnection.Thisnamespaceisnotavailableinthe
WindowsStoreAPIsotheclientisusinganamespace
calledWindows.Networking.Sockets.
Inamoregeneralsense,theconnectionbetweenthe
clientandserverisaTCPbasedconnectionwith
acknowledge-messages(ack-messages).Forexample,
whentheclientconnectstotheserveranack-message
Figure12-1.Usersendsaconnection
issentfromtheservertolettheclientknowthatitis
requesttospecifiedip-adressandport.2.
Theserverthenanswertheclientwithan
listeningforinstructions.Thistypeofmessagingis
ack-message.
persistentthroughoutthewholesession,e.g.
sending/receivingsoundinstructions,stopallsoundsandsoon.Theack-messagesmakesit
easiertokeeptheclientandserversynchronizedwitheachother.SeeFigure12.
Page23
SYSTEM IMPLEMENTATION
Themessagesthatissentfromtheclientconsistsofsimplestrings.Thesestringsarethen
decodedbytheserverandtherightmethodiscalled.Forexample,iftheclientwanttoplaya
certainsoundatagivenpositionthemessagecanlooklikethis:
“SEDINGSOUND^4^12^20^nameOfSound.wav^”
Eachparameterisseparatedbya“^”.Thefirstparametertellstheserverwhichmethodshouldbe
called,thethreeafterthattellsatwhichcoordinatesthesoundshouldbeplacedandthelasttells
theserverwhichsoundshouldbeplayed.
PROBLEMS AND OBSTACLES
Afterthesoftwarewasupandrunningitwastimetoconnectittotheaudiohardware.The
softwareworkedasexpectedwhensimpleheadphoneswasconnecteddirectlyintotheserver’s
built-insoundcard.
TheproblemcamelaterwhentheSteinberginterfacewasconnectedtoaPCrunningWindows
8.1,thesoundinterfacewasunabletooutputsurroundwhenrunningunderWindows.Aftera
briefconversationwiththeSteinbergsupportitbecameclearthattheydonotsupportsurround
setupsinWindowsbutinOSXitshouldrunfine,eveniftheystatesupportforbothoperating
systemsintheirproductdescription.Withthisset-backtheserversideoftheapplicationhadto
berununderOSXandbecauseithadbeendevelopedusingWindowsformsandC#ithadtobe
runwithsomesortofWindowsemulationlikeWine.(WineHQ,n.d.)
AftertheserverapplicationfinallywasinstalledontheOSXmachinerunningWineanew
problememerged.BecauseofsomelimitationsintheWineenginetheworkingsurroundsetupin
OSXcouldnotberecognizedinthe“emulated”serversoftware.
AsanextsteptogettheserversoftwarerunningontheOSXmachinewastotrytoportthe
wholeapplicationwiththehelpofMonoAPI.Monoisacross-platformimplementationof
Microsoft’s.NETframeworkwhich,ifdoneright,wouldworkonallmajorplatformswithout
changingmuchofthecode(Mono-Project,n.d.).Monowasagreattooltousebecausetheserver
softwarewasalreadywritteninC#andaftertheportalmosteverypartoftheserverwaskept
andrannativelyunderOSX.
EvenifitwasrathereasytoporttheserverfromWindowsformstoMonoalloftheseproblems
couldhavebeeneliminatedfromthestartifSteinberghadgiventhecorrectinformationabout
theirproducttobeginwith.TheserverapplicationwouldthenbewritteninCorC++and
compiledforOSXtorunassmoothaspossibleiftheSteinberginterfacewerestillconsideredas
thesoundhardware.
Page24
USER EVALUATION
UserEvaluation
Toachieveagreater
understandingofhowclose
thesoftwarewastotheinitial
ideaandhowwellauser
experiencedthesound
environmentcreatedbythe
softwareusertestswere
conducted.Thereareseveral
differentmethodsavailableto
evaluateuserexperience,such
astheGenevaEmotionWheel
(Scherer,2005),PANAS
Figure13-Pictureshowingthetestsetup.A-Theaudiointerface,B-The
(Watson,Clark,&Tellegen,
server,C-Theclient
1988)and3E(Tä hti&
Arhippainen,2004).Amorein-depthwalkthroughof
user-centeredevaluationmethodologiescanbefound
inHandbookofHuman-ComputerInteraction(Karat,
1997).Afterdiscussionswiththesupervisorthemost
suitableevaluationmethodwastoletoneuserata
timeexperiencethesoundenvironmentand
afterwardsparticipateinasemi-structured
experienceinterview(Longhurst,2003)(alsocalled
softinterview).Whenconductingasoftinterviewitis
importanttokeeptheinterviewquestionsopen.The
softinterviewismeanttobemoreinaconversational
mannerthananormalinterviewsoinsteadofasking
“Didyouexperiencethesoundtotheleft?”the
questionswasmore“how”-based,like“Howdidyou
experiencethesoundtotheleft?”.Thismadethe
interviewmorelikeadiscussioninsteadofanormal
interview.
Page25
USER EVALUATION
Sevenrandomlyselectedpeople,bothfemale(2)and Figure14-Pictureshowingatestperson
placedatthesweetspotinthetesting
male(5)intheagesof22-28testedthesoftware.
environment.A–Speakersconnectedtothe
Noneofthepeoplethattestedthesoftwarehadany
audiointerface.
experienceofsimilarprojectsor3Daudio.Thetest
consistedofthreedifferentscenariosandaftereveryscenario,eachoftheparticipantsfilledouta
questionnaire.Aftertheparticipantreadthequestionnaireheorshewasplacedatthesweetspot
(Figure14)inthesamesetupasshowninFigure5,page11.Thequestionnairewasthen
discussedinashortsoftinterview.
Thefollowingtextwasplacedinthebeginningofthequestionnairetothanktheparticipantand
toensurehisorhersconfidentialitywhentheresultswillbepublished:
“The purpose of this questionnaire is to gather information about the current state of the prototype.
The information gathered with this questionnaire is vital and very valuable for us. Therefore, we
assure you that we treat your information confidentially and thank you already in advance for your
participation. Results from the study will be academically published, but no individual or opinions
will be identifiable.”
Eachparticipantwasalsotoldfromthestartthatthistestwasvoluntaryandthattheycould
cancelthetestanytimeiftheyfeltuncomfortable.Theseinstructionsalongwiththetextinthe
questionnaireisverysimplebutveryimportantthingsonemustincludewhenconductingtests
likethis.(Burnmeister,2000)
Figure13showsthetestsetup.Theserver(B)wasanoldermacbookrunningOSXMaverick.The
tablet(C)wasconnectedtotheserverandtheblacklaptopwasusedonlytodeploytheclient
softwaretothetablet.Theaudiointerface(A)inputwasconnectedtotheserveranditsoutput
connectedtofourspeakersthatcanbeviewedinFigure14(A),page25.
Inthefirstscenariothetestpersonwastoldtolocateandunderstandthedirectionofthesound
placedintheenvironment.Inthesecondscenariothetestpersonwassupposedtoestimatehow
farthesoundwasfromthetestperson’spositioninrelationtothespeakerspositionandinthe
lastscenariofivedifferentsoundswereplayedandthetestpersonwastoldtodescribeeach
sound,theirpositionandhoweasyitwastoseparatethesoundsfromeachother.
Page26
USER EVALUATION
EVALUATION
Beforethetestbegantheparticipants
Onascalefrom1(nearly
showedgreatinterestandseemedtobe
impossible)to5(veryeasy),how
excitedtoexperiencethesoftware,mostly
easywasittounderstandthe
becausetheyneverhadanyprevious
distanceofthesound?
experiencewith3Daudiobefore.The
timebetweenthedifferentscenariosof
1
thetestwasshortenoughtokeepthe
2
participantsinterestedandfocused.The
4
overalltimingofeverythingworkedout
quitewellandeachparticipanthad
enoughtimetofilloutthecurrentpart’s
questionsinthequestionnairebeforethe Chart1-Chartdisplayingresultsfromtestscenario1.
nextpartbegan.
Thetestsshowedthatsixofseven
personsthoughtasinglesoundwasfairly
easytopointoutwhereitcamefrom
(Chart1)andhowfaritwasfromthe
listener(Chart2).Theproblemcame
laterwhentheuserwastoldtocountthe
numberofdifferentsoundsinone
environment.
2
3
4
5
Onascalefrom1(nearly
impossible)to5(veryeasy),how
easywasittopointoutthesource
ofthesound?
1
1
2
3
3
3
4
5
Chart2-Chartdisplayingresultsfromtestscenario2.
Page27
1
USER EVALUATION
Fiveofseventesterswereonesound
Howmanysoundsdidyouhear
awayfromtherightnumber(Chart3).
(guessedrightorwrong)?
Thishowevercanbebecauseofsounds
thatwerechosenforthisscenariowhich
werenoteasytoheartobeginwithor
2
Right
duetothefactthattheyalmostmerged
Wrong
togetherwhentheywereplayedinthe
5
sameenvironment.Inretrospect,a
selectionofmoredistinctsoundsshould
havebeenchosentocreatemoreaccurate
Chart3-Chartdisplayingresultsfromthefirstquestionin
results,seeChart4and5.
testscenario3.
Inconclusion,theoverallexperienceofthetestprovedthatthissoftwareisheadingintheright
direction.Theimplementationprovidedinthisthesisprovesthatitispossibletocreatean
immersivesoundenvironmentbasedononeofthescenarios(Figure6)thatweredeveloped
beforetheimplementationprocessbegan.Throughoutthetesttheparticipantsseemedtoenjoy
thesoftwareandtheyseemedtobecomfortableinthetestingenvironment.Besidesthepoor
soundchoicesinthelastpartofthetest,thetesting,viewedinabiggerpicture,couldnothave
wentbetter.
Onascalefrom1(nearly
impossible)to5(veryeasy),
howeasywasittoseparatethe
differentsounds?
Onascalefrom1(nearly
impossible)to5(veryeasy),
howeasywasittodetermine
thepositionofthedifferent
sounds?
1
2
2
1
2
1
1
3
4
3
Chart4-Chartdisplayingresultsofthesecond
questionintestscenario3.
Page28
2
2
3
3
5
4
5
Chart5-Chartdisplayingresultsofthethrid
questionintestscenario3.
DISCUSSION
Discussion
CONCLUSION
Atthebeginningofthisthesisthemainquestionwastofindoutifitwaspossibletocreate
immersivesoundexperiencesforauserinthesoundenvironmentandastheresearchwent
furtherthisthesisprovedtobeunique.Thismeansthattherewasnoexactroadtofollow,
becausethishadnotbeendonebefore.Theresultoftheusertestsprovesthatitwentthewayit
wassupposedto.Inotherwords,itispossibletocreateimmersivesoundexperienceswiththe
helpof3Dsound.
Thefinalproductofthethesisprovedtobeuniqueincomparisontothestateoftheartpresented
earlierinthisreport.Evenintheinformationgatheringpartofthethesisnoinformationwas
foundthatcameclosetotheresultofthisthesis.Becauseoftheuniqueway3Dsoundisbeing
usedinthisthesistheresultcouldserveasagoodsteppingstoneinfurtherresearchineasy
controlledsoundenvironments.
TheonlythingthatissimilartotheimplementationpresentedinthisthesisistheCocktailproject
(seepage7).ThemostuniquethingaboutthisimplementationcomparedtoCocktailisthat
Cocktailisfairlylimitedwhenitcomestocustomizingthesoundspace.Asstatedinthestateof
theart,inCocktailtheclienthashardcodedpresetstochoosefromwhencreatingthesound
spacebutinthisimplementationtheclientcanchoosewhateversoundshelikes.Otherthan
Cocktailnoneoftheotherprojectsthatwasfoundcameclosetothesameuseorimplementation
asthisthesis.
Asstatedearlierinthispaper,theserversoftwarewouldhavebeenwritteninCorC++insteadof
C#iftheinformationthattheSteinberginterfacedidnotsupportsurroundsoundinaWindows
environmenthadbeenknowfromthestart.Tosolvethisproblemtheserverwasimplemented
usingMonoandiftheserverapplicationwerewritteninCorC++itwouldhaverunnatively
underOSX,withoutanyspecialtreatmentlikethis.
OnethingthatworkedreallywellwaswhentheserverwasrunningunderWindows,likeitwas
meantto,withheadphonesconnectedasoutput.Whenusingitthiswayalong-sidetheRazer
Surroundsoftwaretheresultisreallygood,the3Dfeelisreallythere.
Duringthetestingoftheimplementationthetestpersonswerereallycuriousonhowitwould
feeltoexperiencethesoundenvironmentcreatedbytheprototypeandthefourspeakers(see
page25).Besidesthetestpersonsotherpeoplethatheardaboutthisthesiswerereally
interested.Ifthissoftwarewascombinedwithsomesortofvisualfeedbackitcouldbecome
Page29
DISCUSSION
reallyinterestingandenjoyablebutpresentedonitsownitreallyfeelslikethereissomething
missing.
Althoughthedesignandsomepartsoftheimplementationdidnotliveuptotherequirementsof
thedecidedscenario,thisthesishavecontributedtoagoodstartupplatformforfuture
developmentandresearchinimmersivegameenvironments.
Workingwithsoundingeneralhasalwaysbeenaninterestofmineandthethoughtofthis
project,toworkwith3Dsound,wasthrilling.AtthestartIdidnotknowhowsoundin3Dspace
worked.Ihadonlyexperienceditbeforewhenplayinggamesandonlyworkedwithitinasimple
waythroughUnity3DbutnoexperiencewhatsoevertocodewithaudioAPI’s.
WhenIstartedtointegrateFMODintheprototypeitwasreallyeasytounderstandthewayitwas
meanttobeused.FMODhasareallyextensivedocumentationandthereisalotofdevelopers
thatuseFMODwhichresultsinawiderangeofforums(GameDev.NetLLC,n.d.)and3rd-party
guides(Katy,2012-2013)thatisavailableontheInternet.Allthismadeiteasierformeto
implementFMOD,besidesthesoundqualitythiswasoneofthemainreasonswhyIchoose
FMODasthefeaturedsoftwarelibraryinthisthesis.
OneofthebiggestdisappointmentswastheaudiointerfacefromSteinberg.Itwasrather
displeasingthatitdidnotsupportsurroundcapabilitiesinWindows.AfterthisitwasclearthatI
hadtogetmysoftwareworkingonOSX.IttookawhilebutIlearnedtouseOSXandgotmy
softwareworking,firstthroughasimple“emulation”andlaterIlearnedtoportit.Evenifitwas
ratherfrustratingandtimeconsumingIfeellikeIlearnedsomething.NowIhaveexperienceto
workinOSXandIhaveexperienceinusingcross-platformtoolslikeWineorMono.
Page30
DISCUSSION
FUTURE DEVELOPEMENT
Itwouldbeinterestingtoseethisprojectasafinalproduct.Oneofthemajorimprovementsthat
shouldbefixedisthesupportforOSX.TheserverneedstoberewritteninCorC++torun
nativelyunderOSX.Rightnowtheclienthassomeperformanceissueswhencreatingand
steeringasoundintheenvironment.Oneotherthingtofixforafinalversionisthedesignofthe
clientsoftware,accordingtoFigure10.
IfIgettheopportunityandtimeIwouldhelpDanielfurtherinhisresearch.Ithasbeenagreat
learningexperienceandhopefully,inanotsofarfuture,weseeproductslikethisinourown
homes.
Page31
REFERENCES
References
Ambiera.(n.d.).irrKlang.RetrievedMarch20,2014,fromhttp://www.ambiera.com/irrklang/
Arthur,C.,&Gibbs,S.(2013,December6).News: Technology: The Guardian.RetrievedMarch19,
2014,fromhttp://www.theguardian.com/technology/2013/dec/06/headphonesmarket-beats-by-dre
Benyon,D.(2010).Designing Interactive Systems.Harlow:PearsonEducationLimited.
Burnmeister,O.K.(2000).Usabilitytesting:revisitinginformedconsentproceduresfortesting
internetsites.Selected papers from the second Australian Institute conference on Computer
ethics.(pp.3-9).AustralianComputerSociety,Inc.
Dictionary.com.(n.d.).Collins English Dictionary - Complete & Unabridged 10th Edition.Retrieved
March20,2014,fromCrosstalk:http://dictionary.reference.com/browse/crosstalk
Edlund,J.,Gustafson,J.,&Beskow,J.(2010).Cocktail–ademonstrationofmassivelymulticomponentaudioenvironmentsforillustrationandanalysis.SLTC 2010,23.
FirelightTechnologiesPTYLTD.(2013).About Us: Fmod.RetrievedMarch19,2014,from
http://www.fmod.org/about-us/
FirelightTechnologiesPTYLTD.(n.d.).FMOD Ex: FMOD.RetrievedMarch20,2014,from
http://www.fmod.org/fmod-ex/
Frauenberger,C.,&Noisternig,M.(2003,July).3Daudiointerfacesfortheblind.InWorkshop on
Nomadic Data Services and Mobility, Graz, Austria, March(pp.11-12).
GameDev.NetLLC.(n.d.).Forum: Gamedev.net.RetrievedMarch23,2014,from
http://www.gamedev.net/page/resources/_/technical/game-programming/a-quickguide-to-fmod-r2098
Gardner,W.G.(1998).3-D audio using loudspeakers.Springer.
Gardner,W.G.(1999).3Daudioandacousticenvironmentmodeling.Wave Arts, Inc,99.
Inc.,R.(n.d.).Razer Surround Personalized.RetrievedMarch12,2014,from
https://www.razerzone.com/surround
Karat,J.(1997).HandbookofHuman-ComputerInteraction.InM.G.Helander,T.K.Landauer,&P.
V.Prabhu(Eds.),Handbook of Human-Computer Interaction(pp.689-705).Amsterdam:
ElsevierScienceB.V.
Page32
REFERENCES
Katy.(2012-2013).Katy's Code: FMOD.RetrievedMarch23,2014,from
http://katyscode.wordpress.com/category/programming/audio/fmod/
Longhurst,R.(2003).Semi-structuredinterviewsandfocusgroups.InR.Longhurst,Key methods
in geography(pp.117-132).
Microsoft.(n.d.).XAudio2 Introduction: MSDN Microsoft.RetrievedMarch20,2014,from
http://msdn.microsoft.com/en-us/library/windows/desktop/ee415813(v=vs.85).aspx
Mono-Project.(n.d.).What is Mono: Mono-Project.RetrievedMarch20,2014,fromhttp://monoproject.com/What_is_Mono
Oy,G.(n.d.).Genelec 8020C.RetrievedMarch12,2014,from
http://www.genelec.com/products/8020c/
ResearchCouncilsUK.(2014,March10).Gateway to Research: S3A.RetrievedAugust25,2014,
fromGatewaytoResearch:http://gtr.rcuk.ac.uk/project/3C158598-11E5-4178-976A27790E395282
Sá nchez,J.,&Sá enz,M.(2006).3Dsoundinteractiveenvironmentsforblindchildrenproblem
solvingskills.Behaviour & Information Technology,25(4),367-378.
Sá nchez,J.,Baloian,N.,Hassler,T.,&Hoppe,U.(2003,April).Audiobattleship:BlindLearners
CollaborationthroughSound.InCHI'03 Extended Abstracts on Human Factors in
Computing Systems(pp.798-799).ACM.
Scherer,K.R.(2005).Whatareemotions?Andhowcantheybemeasured?Social science
information 44.4,695-729.
Schwaber,K.,&Sutherland,J.(2011).Thescrumguide.Scrum.org, October.
Sjö lander,K.(2006,January23).The Snack Sound Toolkit.RetrievedMarch19,2014,from
http://www.speech.kth.se/snack/
Song,M.S.,Zhang,C.,Florencio,D.,&Kang,H.G.(2010,July).Personal3Daudiosystemwith
loudspeakers.InMultimedia and Expo (ICME), 2010 IEEE International Conference on(pp.
1600-1605).IEEE.
Strangesoft.(n.d.).OpenAl Soft: Kcat Strangesoft.RetrievedMarch20,2014,from
http://kcat.strangesoft.net/openal.html
Technologies,S.M.(n.d.).UR824: Steinberg.net.RetrievedMarch12,2014,from
http://www.steinberg.net/en/products/audio_interfaces/ur_series/models/ur824.html
Page33
REFERENCES
Thorn,A.(2013).Game Developement Principles(1sted.).Boston:CengageLearning.
Tä hti,M.,&Arhippainen,L.(2004).AProposalofcollectingEmotionsandExperiences.
Interactive Experiences in HCI, 2,195-198.
UnityTechnologies.(n.d.).Unity: Unity3d - Game engine, tools and multiplatform.RetrievedMarch
23,2014,fromhttp://unity3d.com/unity
UniversityofSurrey.(2014,May22).University of Surrey: Tim Brookes.RetrievedAugust25,
2014,fromUnivesrityofSurrey:
http://www.surrey.ac.uk/schoolofarts/people/complete_staff_list/tim_brookes/index.ht
m
ValveCorporation.(n.d.).Half-Life: Steam Store.RetrievedMarch20,2014,from
http://store.steampowered.com/app/70/
Watson,D.,Clark,L.A.,&Tellegen,A.(1988).DevelopmentandValidationofBriefMeasuresof
PositiveandNegativeAffect:ThePANASScales.Journal of Personality and Social
Psychology.
WineHQ.(n.d.).About: WineHQ.org.RetrievedMarch20,2014,from
http://www.winehq.org/about/
Page34