Making High-Performance Embedded Instruments with Bela and

MakingHigh-PerformanceEmbeddedInstruments
withBelaandPureData
1
2
3
4
5
GiulioMoro ,AstridBin ,RobertH.Jack ,ChristianHeinrichs ,AndrewP.McPherson CentreforDigitalMusic,
QueenMaryUniversityofLondon,UK
1
2
3
[email protected], [email protected], [email protected],
4
5
[email protected], [email protected]
Abstract.Belaisanembeddedplatformforultra-lowlatencyaudioandsensorprocessing.Wepresentherethehardware
andsoftwarefeaturesofBelawithparticularfocusonitsintegrationwithPureData.SensorinputsonBelaaresampledat
audiorate,whichopenstothepossibilityofdoingsignalprocessingusingPureData’saudio-rateobjects.
Keywords:embeddedaudio,sonicinteractiondesign,sensors,lowlatency,musicalinstrumentdesign.
Introduction
Theincreasingpowerandavailabilityofmicrocontrollersandsingle-boardcomputershasgivenrisetomanynewplatforms
forcreatingmusicalinstrumentsandplatformsforinteractiveaudio.Choosingasuitableplatformcanbeachallenge,
involvingtradeoffsbetweencomputingpower,hardwareconnectivity,easeofprogrammingandprice.
Manycurrentapproachestodesigningsoundingobjectscombinetwoormoredevicestogether,forinstanceanArduino
whichhandlesanaloganddigitalsensorinputcommunicatingviaUSB-serialwithaacomputerrunningtheaudio
processing.Usingaself-containedembeddedplatforminthecreationofDMIsandinteractiveaudiosystemshasseveral
advantagesoversuchasetup.
•
•
•
•
ReliabilityUsingasingledeviceislesspronetocommunicationerrorsanditiseasiertoprovideabackupsolution
forasimplersystem.
PerformanceWithacompositesetupastheonedescribedabove,theserialconnectionisslowandthethroughput
islimited.Assuch,thelatency,samplingrateandjitteroftheacquireddataareallaffectednegatively,whichmay
inturnaffecttheexpressivenessoftheperformance.MIDIdevicestypicallyperformbetterthanserialones,while
wirelesslinksmaybeaffectedbypacketlossorchannelcongestion(McPherson,Jack,andMoro2016).
ReproducibilityItiseasierforotherpeopletorecreateadeviceifitdoesnotrelyonmultiplepiecesofsoftware
andhardwaredevicesandspecificrevisionsofeachofthem.
SustainabilitySimilarly,thedevelopersthemselveswillfinditeasiertomaintainanddevelopasystemthatdoes
nothavemultipledependencies,alsototheadvantageofmakingsoftwareversioncontroleasier.
RecentEmbeddedPlatformsforDigitalMusicalInstrumentCreation
Arduinoandsimilarboardsareanaccessiblewayofprovidinglow-levelconnectivitytoanaloganddigitalsensors,butthe
low-poweredAVRmicrocontrollerdoesnotallowaudioon-boardaudioprocessing.Thex-OSCboardprovidesanalogand
digitalI/Osoverawirelesslink(MadgwickandMitchell2013).
Twoaudio-oriented,self-containedplatformsbasedona168MHzCortexM4microcontrollerhitthemarketinthepastfew
1
2
years:theOwl programmabledigitaleffect(Webster,LeNost,andKlang2014),whichsurfacedin2013,andAxoloti ,which
1
http://hoxtonowl.com/
2
http://www.axoloti.com/
cameoutearly2015.TheformercanbeprogrammedthroughaC++APIorcanrunPureDatapatchesusingtheHeavy
3
AudioToolsfromEnzienAudio, whilethelatterprovidesacustomgraphicalpatcherwhichincludesDSPmodulesandcan
beexpandedwithC++.
RaspberryPiisarguablythemostpopularsingle-board-computerintheworldanditslatestrevision3featuresaquad-core
4
1.2GHz64bitCPU.TheCCRMASatellitedistribution (BerdahlandJu2011)wasdevelopedtoprovideanefficientaudioorientedenvironmentfortheRaspberryPi.
5
CoalaisanaudioprocessingplatformbasedontheBeagleBoneBlack whichwaspresentedin(Piéchaud2014).The
6
softwareandhardwarearchitectureofCoalaweredevelopedforthespecifictaskofmodalcontrol ,whichrequiresavery
tightfeedbackloop.Theplatformisthereforeoptimizedforfastsample-by-sampleprocessinginordertominimizeroundtriplatency.
Bela:anembeddedplatformforaudioandsensorprocessing
7
Bela (formerlyknownasBeagleRT)isacombinedhardwareandsoftwareenvironmentthatconsistsofaBeagleBoneBlack
withanexpansion“cape”(McPhersonandZappi2015a).ItwasoriginallydevelopedfortheD-BoxHackableDigital
Instrument(ZappiandMcPherson2014)whichrequiredmultiplelow-latencyhybridanalog-digitalfeedbackloops
(McPhersonandZappi2015b).Belacombinestheconnectivityofamicrocontrollerwiththeprocessingcapabilityofasingleboardcomputer.ThecapeprovidesstereoaudioI/Oincluding1Wspeakeramplifiers,8channelseachof16-bitanalogI/O,
and16digitalGPIOpins.Belaisopen-sourcehardwareandsoftware.Sourcecodeanddesignmaterialsarepublicly
8
available. 9
TheBelasoftwareusesaDebianLinuxdistributionwiththeXenomai real-timekernelextensions.TheProgrammable
RealtimeUnit(PRU),a200MHzmicrocontrolleronthesamechipastheBeagleBoneBlackCPU,transfersaudioandsensor
datadirectlytothehardware,bypassingthekerneldrivers.Theuser’sBelacodethereforerunsatthehighestpriorityofany
taskontheboard,includingtheLinuxkernelitself.Thisallowsaudioblocksizesaslowas2samples,resultinginround-trip
audiolatencyof1ms(orevendownto100usifusingtheanaloginputsandoutputsratherthantheaudioconverters)
(McPherson,Jack,andMoro2016).
OnBela,everyanaloganddigitalchannelisautomaticallysampledataudiorates,synchronouslywiththeaudioclock.The
highsamplingrateoftheanaloganddigitalchannelsareuniquetoBelaandtheirjitter-freealignmentwiththeaudiomakes
itidealforinteractive,intuitive,responsiveaudioapplications.
ComparedtoAxolotiandOwl,Belahasmoreprocessingpower,whilestillprovidinghardreal-timeperformances,withthe
addedconvenienceofafullLinuxOSandwhilebeingminimallyaffectedbysystemload.Itismoregeneral-purposethan
Coalawhichaddressesthespecificfieldofreal-timecontrol,thoughCoalaiscapableofevenlowerlatenciesthanBela.Ona
RaspberryPirunningCCRMASatellite,despitethehighprocessingpoweravailableonboard,audiodependsonthestandard
Linuxaudiodrivers,sothatlow-latencyprocessingisdifficultbecauseofthepresenceofotherprocessesontheboard,
whichmaycauseunderrunsatsmallaudioblocksizesevenwhentheCPUloadislowonaverage.TheBeagleBoneBlackCPU
islesspowerfuloverallbuttheXenomaiextensionsusedintheBelasoftwareallowreliableandconsistentperformance
withsub-millisecondlatency.Additionally,thenumberofI/OsavailableinBelaisgreaterthanthoseoncommonlyavailable
RaspberryPihats.
3
http://enzienaudio.com/
4
https://ccrma.stanford.edu/~eberdahl/Satellite/
5
http://beagleboard.org/black
6
http://instrum.ircam.fr/smartinstruments/
7
http://bela.io
8
http://bela.io/code/
9
http://xenomai.org/
Bela,providingalargenumberofI/Osforaudioandsensors,poweroutputforloudspeakers,andprovidingenough
processingpowertosatisfymostneeds,entirelyfulfillstherequirementsofaself-containeddevice,whichcanbe
embeddedinastand-aloneDigitalMusicalInstrumentorsoundingobject.
PureDataonBela
10
PureData (Pd)isapopularopensourcegraphicalprogramminglanguagewidelyusedbymusiciansandsounddesigners
alike,whichallowsforquickprototypingofsoundandsensormappings.PdpatchesareusuallyrunwithinPditself,orusing
11
thesharedlibrarylibpd .ThemessagingarchitectureandtheaudioengineofPdwasnotdesignedtobefastand
computationallyefficientwhichcanleadperformancepenaltiesonplatformswithlimitedcomputationalpower.
Figure1:TheBelacape
Figure2:UsingaforcesensitiveresistorwithBela
HeavyAudioToolsTheHeavyAudioToolsfromEnzienAudiousePdasafront-endtogenerateoptimisedCcode.By
analyzingthegraphofconnectionsbetweenobjectsinthePdcode,Heavyiscapableofproducinghigh-performance
vectorizedCcodewhichcanoutperformlibpd,makingitparticularlywellsuitedforembeddeddevicesand,moregenerally,
hardwarewithlimitedcomputationalpower.Heavyisaproprietary,cloud-basedserviceandthegeneratedcodeislicensed
undertheMITnon-commerciallicense.
TheCcodeproducedbyHeavyiswell-suitedtobeintegratedinaXenomaienvironment,asmemoryisallocatedonthe
stack,thusavoidingsystemcallsduringexecution.AnautomatedscripttakescareofuploadingthePdpatchtoHeavy’s
server,collectthegeneratedCcodeandcompileitontheBelaboard.Theentireprocessgenerallytakeslessthanone
minuteandmostofthetimeisspentcompilingtheCcodeontheBeagleBoneBlack.
libpdMinimalmodificationswererequiredtoportlibpdforBela,theseincludedallowingblocksizesassmallas8samples
perblockandremovingsocketanddiskI/Ofromtheaudiothread.Additionally,thecallstothepthreadfunctionswere
wrappedintoXenomaifunctions.TheresultingsharedlibrarycanbelinkedtoaBelaprogramand
libpd_process_float()istheninvokedfromwithinBela’saudiocallback.
DeployingaPdpatchusinglibpdisvirtuallyinstantaneousasitdoesnotrequirecompiling.Assoonasthepatchissavedon
theBeagleBone’sfilesystem,theBelaprogramcanberestartedanditwillloadtheupdatedpatch.Anaddedadvantageof
usinglibpdisthatitiseasiertoportPdexternalswhentheirsourcecodeisavailable.Thesameprecautionslistedabove
shouldbetakenfornewexternalsinordertomakesurethatnewobjectsdonotintroduceXenomaimodeswitchesinthe
12
audiothread .
10
http://puredata.info
11
http://libpd.cc/
12
https://xenomai.org/2014/08/porting-a-linux-application-to-xenomai-dual-kernel/
PerformancecomparisonRunninganexamplepatchcontainingagenerativeaudiocomposition,Heavycodecompiledwith
13
14
theclang compileruses26%oftheCPU.ThesameHeavycode,compiledwithgcc ,occupies43%oftheCPUcycles.
Runningthepatchusinglibpduses53%oftheCPU.
Traditionally,thehighest-performanceplatformshavealsoplacedthemosttechnicaldemandsontheprogrammer.For
manyyears,customDSPboardsofferedthebestbalanceofhardreal-timeperformanceandhighprocessingpower,but
theyweregenerallyprogrammedinlow-levellanguagesusingcustomdevelopmentenvironments.High-levelmusic
programminglanguagesoftencomewithsignificantprocessingoverhead.RunningPureDataonBela,especiallythroughthe
HeavyAudioTools,providesaconvenientgraphicalenvironmentwithminimalsacrificeinperformancecomparedto
programminginC++.
(a)Smoothing(b)Re-centering
(c)Differentiating
(d)Thresholding
Figure3:UsingPureDataobjectstoprocesssensordata
SensorprocessinginPureData
Manyinteractivesystemstakeapproacheswheresensorsaresampledatlowandnon-constantratesandthemostrecent
frameofsensordataisusedtomodulateaparticularsonicparameter.Butinactualfact,themeaningofsensordataisoften
deeper,initsbehaviourovertimeoritsfrequencycontent.Ahighsamplingrateyieldsaveryhighbandwidthofinteraction
whichcapturessubtledetailsthatmightbelostatlowersamplerates.Thoughallthesametechniquescouldbe
implementedatcontrolrate,audio-ratesensordatacanhelpreorientthedesigner’sthinkingtobecomemoreawareof
thesepossibilities.WhenusingBelawithPd,thisallowstoconvenientlyprocesssensorsignalsusingaudio-rateobjects.
Someexamplesinclude:
•
SmoothingSomesensorsareinherentlynoisy,forinstanceapotentiometermaygeneratehigh-frequencynoise
whenitisactuated,oraninfra-redopticalsensormaybesubjecttotransientperturbationsfromotheremitting
sources.Thenoiseinthesensorreadingsmayleakintotheaudiosignal,dependingonthesignalflow.Aneasy
approachtoremovehigh-frequencynoiseistoapplyalow-passfilterwithanappropriatecut-offfrequency,asin
Figure3a.
13
http://clang.llvm.org/
14
https://gcc.gnu.org/
•
•
•
Re-centeringReadingsfromaccelerometersandothersensorshaveinherentDC-offsetswhichmaybeundesirable
forcertainapplications.Aquickwayofremovingthemwhichdoesnotrequirecalibrationisusinganhigh-pass
filterwithanappropriatecut-offfrequency,asinFigure3b.
DifferentiatingSomesound-generatorparametersarebettercontrolledusingthevelocityofasensorreading,
ratherthanwiththerawreading.Ahigh-passfilterwithacut-offfrequencyof0,properlyrescaledcanusedforthis
purpose,asinFigure3c.
ThresholdingAmorecomplicatedexampleinFigure3dshowshowtocombinefull-waverectification,smoothing,
DCshiftandconstraintothresholdasignal.
Conclusion
Thereareseveraltradeoffsinvolvedindifferentdigitalmusicalinstrumentdesigntools:processingpower,latency,
connectivity,sensorbandwidth,easeofprogrammingandaccessibility.Withanyoftheprogrammingenvironments,Bela
bringstogethertheconnectivityandCPUpowerofanembeddedLinuxcomputerwiththelowlatencyandprecise
synchronisationofamicrocontrollerandbringsahigh-bandwidthdimensiontosensorprocessing.UsingeithertheHeavyor
thelibpdenvironments,Belaisalsosuitableforrapidprototypingusingthewidely-usedPureDatagraphicalprogramming
language,withfullaccesstobothaudioandsensors.
Acknowledgements.ThisworkissupportedbygrantsEP/K032046/1,EP/K009559/1andEP/L019981/1fromtheUK
EngineeringandPhysicalSciencesResearchCouncilandtheQueenMaryCentreforPublicEngagement.
References
Berdahl,Edgar,andWendyJu.2011.“SatelliteCCRMA:AMusicalInteractionandSoundSynthesisPlatform.”Proceedingsof
theInternationalConferenceonNewInterfacesforMusicalExpression.Oslo,Norway,173–178.
Madgwick,Sebastian,andThomasJMitchell.2013.“x-OSC:AversatilewirelessI/Odeviceforcreative/musicapplications.”
SMCSoundandMusicComputingConference.Stockholm,Sweden:KTHRoyalInstituteofTechnology.
McPherson,A.P.,andV.Zappi.2015a.“Anenvironmentforsubmillisecond-latencyaudioandsensorprocessingon
BeagleBoneBlack.”AudioEngineeringSocietyConvention138.AudioEngineeringSociety.
McPherson,Andrew,andVictorZappi.2015b.“ExposingtheScaffoldingofDigitalInstrumentswithHardware-Software
FeedbackLoops.”ProceedingsoftheInternationalConferenceonNewInterfacesforMusicalExpression.BatonRouge,
Louisiana,USA:LouisianaStateUniversity,162–167.
McPherson,AndrewP.,RobertH.Jack,andGiulioMoro.2016.“Action-SoundLatency:AreOurToolsFastEnough?”
ProceedingsoftheInternationalConferenceonNewInterfacesforMusicalExpression.Brisbane,Australis.
Piéchaud,Robert.2014.“ALightweightC++Real-TimeActiveControlFramework.”16thRealTimeLinuxWorkshop,October
12to13,2014attheCCDCongressCenterDusseldorfcollocatedwithLinuxConEuropeinDusseldorf,Germany.
Webster,Thomas,GuillaumeLeNost,andMartinKlang.2014.“TheOWLprogrammablestageeffectspedal:Revisingthe
conceptoftheon-stagecomputerforlivemusicperformance.”ProceedingsoftheInternationalConferenceonNew
InterfacesforMusicalExpression.London,UnitedKingdom:Goldsmiths,UniversityofLondon,621–624.
Zappi,V.,andA.McPherson.2014.“DesignandUseofaHackableDigitalInstrument.”ProceedingsoftheInternational
ConferenceonLiveInterfaces.Lisbon,Portugal,208–219.