ArangoDB v3.0.8 Documentation

Transcription

ArangoDBv3.0.10Documentation
TableofContents
Introduction
0
GettingStarted
1
Installing
1.1
Linux
1.1.1
M acOSX
1.1.2
Windows
1.1.3
Compiling
1.1.4
Authentication
1.2
AccessingtheWebInterface
1.3
ComingfromSQL
1.4
Scalability
2
Architecture
2.1
Datamodels
2.2
Limitations
2.3
Datamodels&modeling
3
Concepts
3.1
Databases
3.2
WorkingwithDatabases
3.2.1
NotesaboutDatabases
3.2.2
Collections
3.3
CollectionM ethods
3.3.1
DatabaseM ethods
3.3.2
Documents
3.4
BasicsandTerminology
3.4.1
CollectionM ethods
3.4.2
DatabaseM ethods
3.4.3
Graphs,Vertices&Edges
3.5
NamingConventions
3.6
DatabaseNames
3.6.1
CollectionNames
3.6.2
DocumentKeys
3.6.3
AttributeNames
3.6.4
Indexing
4
IndexBasics
4.1
Whichindextousewhen
4.2
IndexUtilization
4.3
WorkingwithIndexes
4.4
HashIndexes
4.4.1
Skiplists
4.4.2
Persistent
4.4.3
FulltextIndexes
4.4.4
1
GeoIndexes
Graphs
4.4.5
5
GeneralGraphs
5.1
GraphM anagement
5.1.1
GraphFunctions
5.1.2
Traversals
5.2
UsingTraversalObjects
5.2.1
ExampleData
5.2.2
WorkingwithEdges
FoxxM icroservices
5.3
6
Ataglance
6.1
Gettingstarted
6.2
Servicemanifest
6.3
Servicecontext
6.4
Configuration
6.5
Dependencies
6.6
Routers
6.7
Endpoints
6.7.1
M iddleware
6.7.2
Request
6.7.3
Response
6.7.4
Sessionsmiddleware
6.8
Sessionstorages
6.8.1
Collectionstorage
6.8.1.1
JWTstorage
6.8.1.2
Sessiontransports
6.8.2
Cookietransport
6.8.2.1
Headertransport
6.8.2.2
Servingfiles
6.9
Scriptsandqueuedjobs
6.10
M igrating2.xservices
6.11
M igratingfrompre-2.8
6.11.1
manifest.json
6.11.2
applicationContext
6.11.3
RepositoriesandM odels
6.11.4
Controllers
6.11.5
Requestcontext
6.11.5.1
Errorhandling
6.11.5.2
Before/After/Around
6.11.5.3
Requestobject
6.11.5.4
Responseobject
6.11.5.5
DependencyInjection
6.11.5.6
Sessions
6.11.6
AuthandOAuth2
6.11.7
2
FoxxQueries
6.11.8
Legacycompatibilitymode
6.12
Usermanagement
6.13
Relatedmodules
6.14
Authentication
Transactions
6.14.1
7
Transactioninvocation
7.1
Passingparameters
7.2
Lockingandisolation
7.3
Durability
7.4
Limitations
7.5
Deployment
8
Singleinstance
8.1
Cluster:M esos,DC/OS
8.2
Cluster:Localtest
8.3
Cluster:Processes
8.4
Cluster:Docker
8.5
Administration
WebInterface
9
9.1
Queries
9.1.1
Collections
9.1.2
Cluster
9.1.3
Dashboard
9.1.4
Document
9.1.5
Logs
9.1.6
Services
9.1.7
Graphs
9.1.8
ArangoDBShell
9.2
ShellOutput
9.2.1
Configuration
9.2.2
Details
9.2.3
Arangoimp
9.3
Arangodump
9.4
Arangorestore
9.5
M anagingUsers
9.6
ServerConfiguration
9.7
Arangodoptions
9.7.1
Write-aheadlogoptions
9.7.2
M anagingEndpoints
9.7.3
Clusteroptions
9.7.4
Loggingoptions
9.7.5
Communicationoptions
9.7.6
Durability
9.8
Replication
9.9
3
AsynchronousReplication
9.9.1
Components
9.9.1.1
ExampleSetup
9.9.1.2
SyncingCollections
9.9.1.3
ReplicationLimitations
9.9.1.4
SynchronousReplication
9.9.2
Implementation
9.9.2.1
Configuration
9.9.2.2
Sharding
9.10
Upgrading
9.11
Upgradingto3.0
9.11.1
Upgradingto2.8
9.11.2
Upgradingto2.6
9.11.3
Upgradingto2.5
9.11.4
Upgradingto2.4
9.11.5
Upgradingto2.3
9.11.6
Upgradingto2.2
9.11.7
Troubleshooting
10
arangod
10.1
EmergencyConsole
10.2
DatafileDebugger
10.3
Arangobench
10.4
Architecture
Write-aheadlog
Releasenotes
11
11.1
12
WhatsNewin3.0
12.1
WhatsNewin2.8
12.2
WhatsNewin2.7
12.3
WhatsNewin2.6
12.4
WhatsNewin2.5
12.5
WhatsNewin2.4
12.6
WhatsNewin2.3
12.7
WhatsNewin2.2
12.8
WhatsNewin2.1
12.9
Incompatiblechangesin3.0
12.10
12.11
12.12
12.13
12.14
12.15
12.16
Appendix
References
db
13
13.1
13.1.1
4
collection
JavaScriptM odules
13.1.2
13.2
console
13.2.1
crypto
13.2.2
fs
13.2.3
request
13.2.4
actions
13.2.5
queries
13.2.6
Write-aheadlog
13.2.7
TaskM anagement
13.2.8
Deprecated
SimpleQueries
13.3
13.3.1
SequentialAccess
13.3.1.1
Pagination
13.3.1.2
M odificationQueries
13.3.1.3
GeoQueries
13.3.1.4
FulltextQueries
13.3.1.5
Actions
13.3.2
DeliveringHTM LPages
13.3.2.1
JsonObjects
13.3.2.2
M odifying
13.3.2.3
Errorcodesandmeanings
13.4
Glossary
13.5
5
WelcometotheArangoDBdocumentation!
NewandeagertotryoutArangoDB?Startrightawaywithourbeginner'sguide:GettingStarted
Thedocumentationisorganizedinfourhandbooks:
ThismanualdescribesArangoDBanditsfeaturesindetailforyouasauser,developerandadministrator.
TheAQLhandbookexplainsArangoDB'squerylanguageAQL.
TheHTTPhandbookdescribestheinternalAPIofArangoDBthatisusedtocommunicatewithclients.Ingeneral,theHTTP
handbookwillbeofinteresttodriverdevelopers.Ifyouuseanyoftheexistingdriversforthelanguageofyourchoice,youcanskip
thishandbook.
Ourcookbookwithrecipesforspecificproblemsandsolutions.
Featuresareillustratedwithinteractiveusageexamples;youcancut'n'pastethemintoarangoshtotrythemout.TheHTTPREST-API
fordriverdevelopersisdemonstratedwithcut'n'pasterecepiesintendedtobeusedwiththecURL.Driversmayprovidetheirown
examplesbasedonthese.jsbasedexamplestoimproveunderstandeabilityfortheirrespectiveusers,i.e.forthejavadriversomeofthe
samplesarere-implemented.
Overview
ArangoDBisamulti-model,open-sourcedatabasewithflexibledatamodelsfordocuments,graphs,andkey-values.Buildhigh
performanceapplicationsusingaconvenientSQL-likequerylanguageorJavaScriptextensions.UseACIDtransactionsifyourequire
them.Scalehorizontallyandverticallywithafewmouseclicks.
Keyfeaturesinclude:
installingArangoDBonaclusterisaseasyasinstallinganapponyourmobile
Flexibledatamodeling:modelyourdataascombinationofkey-valuepairs,documentsorgraphs-perfectforsocialrelations
Powerfulquerylanguage(AQL)toretrieveandmodifydata
UseArangoDBasanapplicationserverandfuseyourapplicationanddatabasetogetherformaximalthroughput
Transactions:runqueriesonmultipledocumentsorcollectionswithoptionaltransactionalconsistencyandisolation
ReplicationandS harding:setupthedatabaseinamaster-slaveconfigurationorspreadbiggerdatasetsacrossmultipleservers
Configurabledurability:lettheapplicationdecideifitneedsmoredurabilityormoreperformance
No-nonsensestorage:ArangoDBusesallofthepowerofmodernstoragehardware,likeSSDandlargecaches
JavaScriptforall:nolanguagezoo,youcanuseonelanguagefromyourbrowsertoyourback-end
Itisopensource(ApacheLicense2.0)
Community
IfyouhavequestionsregardingArangoDB,Foxx,drivers,orthisdocumentationdon'thesitatetocontactuson:
GitHubforissuesandmisbehaviororpullrequests
GoogleGroupsfordiscussionsaboutArangoDBingeneralortoannounceyournewFoxxApp
StackOverflowforquestionsaboutAQL,usagescenariosetc.
Slack,ourcommunitychat
Whenreportingissues,pleasedescribe:
theenvironmentyourunArangoDBin
theArangoDBversionyouuse
whetheryou'reusingFoxx
theclientyou'reusing
whichpartsofthedocumentationyou'reworkingwith(link)
whatyouexpecttohappen
whatisactuallyhappening
Wewillrespondassoonaspossible.
Introduction
6
Gettingstarted
Overview
Thisbeginner'sguidewillmakeyoufamiliarwithArangoDB.Wewillcoverhowto
installandrunalocalArangoDBserver
usethewebinterfacetointeractwithit
storeexampledatainthedatabase
querythedatabasetoretrievethedataagain
editandremoveexistingdata
Installation
Headtoarangodb.com/download,selectyouroperatingsystemanddownloadArangoDB.Youmayalsofollowtheinstructionsonhow
toinstallwithapackagemanager,ifavailable.
IfyouinstalledabinarypackageunderLinux,theserverisautomaticallystarted.
IfyouinstalledArangoDBusinghomebrewunderM acOSX,starttheserverbyrunning /usr/local/sbin/arangod.
IfyouinstalledArangoDBunderWindowsasaservice,theserverisautomaticallystarted.Otherwise,runthe arangod.exelocatedin
theinstallationfolder's bindirectory.Youmayhavetorunitasadministratortograntitwritepermissionsto C:\ProgramFiles.
Formorein-depthinformationonhowtoinstallArangoDB,aswellasavailablestartupparameters,installationinaclusterandsoon,see
Installing.
Securingtheinstallation
Thedefaultinstallationcontainsonedatabase_systemandausernamedroot.
DebianbasedpackagesandtheWindowsinstallerwillaskforapasswordduringtheinstallationprocess.Red-Hatbasedpackageswill
setarandompassword.Forallotherinstallationpackagesyouneedtoexecute
shell>arango-secure-installation
Thiswillaskedforarootpasswordandsetsthispassword.
Webinterface
Theserveritself(arangod)speaksHTTP/REST,butyoucanusethegraphicalwebinterfacetokeepitsimple.There'salsoarangosh,a
synchronousshellforinteractionwiththeserver.Ifyou'readeveloper,youmightprefertheshellovertheGUI.Itdoesnotprovide
featureslikesyntaxhighlightinghowever.
WhenyoustartusingArangoDBinyourproject,youwilllikelyuseanofficialorcommunity-madedriverwritteninthesamelanguageas
yourproject.Driversimplementaprogramminginterfacethatshouldfeelnaturalforthatprogramminglanguage,anddoallthetalkingto
theserver.Therefore,youcanmostcertainlyignoretheHTTPAPIunlessyouwanttowriteadriveryourselforexplicitlywanttouse
therawinterface.
Togetfamiliarwiththedatabasesystemyoucanevenputdriversasideandusethewebinterface(codenameAardvark)forbasic
interaction.Thewebinterfacewillbecomeavailableshortlyafteryoustarted arangod.Youcanaccessitinyourbrowserat
http://localhost:8529-ifnot,pleaseseeTroubleshooting.
Bydefault,authenticationisenabled.Thedefaultuseris root.Dependingontheinstallationmethodused,theinstallationprocess
eitherpromptedfortherootpasswordorthedefaultrootpasswordisempty(seeabove).
GettingStarted
7
Nextyouwillbeaskedwhichdatabasetouse.Everyserverinstancecomeswitha _systemdatabase.Selectthisdatabasetocontinue.
Youshouldthenbepresentedthedashboardwithserverstatisticslikethis:
GettingStarted
8
Foramoredetaileddescriptionoftheinterface,seeWebInterface.
Databases,collectionsanddocuments
Databasesaresetsofcollections.Collectionsstorerecords,whicharereferredtoasdocuments.Collectionsaretheequivalentoftablesin
RDBM S,anddocumentscanbethoughtofasrowsinatable.Thedifferenceisthatyoudon'tdefinewhatcolumns(orratherattributes)
therewillbeinadvance.Everydocumentinanycollectioncanhavearbitraryattributekeysandvalues.Documentsinasinglecollection
willlikelyhaveasimilarstructureinpracticehowever,butthedatabasesystemitselfdoesnotimposeitandwilloperatestableandfast
nomatterhowyourdatalookslike.
Readmoreinthedata-modelconceptschapter.
Fornow,youcanstickwiththedefault _systemdatabaseandusethewebinterfacetocreatecollectionsanddocuments.Startby
clickingtheCOLLECTIONSmenuentry,thentheAddCollectiontile.Giveitaname,e.g.users,leavetheothersettingsunchanged(we
wantittobeadocumentcollection)andSaveit.Anewtilelabeledusersshouldshowup,whichyoucanclicktoopen.
TherewillbeNodocumentsyet.Clickthegreencirclewiththewhiteplusontheright-handsidetocreateafirstdocumentinthis
collection.Adialogwillaskyoufora _key.YoucanleavethefieldblankandclickCreatetoletthedatabasesystemassignan
automaticallygenerated(unique)key.Notethatthe _keypropertyisimmutable,whichmeansyoucannotchangeitoncethedocument
iscreated.Whatyoucanuseasdocumentkeyisdescribedinthenamingconventions.
Anautomaticallygeneratedkeycouldbe "9883"( _keyisalwaysastring!),andthedocument _idwouldbe "users/9883"inthat
case.Asidefromafewsystemattributes,thereisnothinginthisdocumentyet.Let'saddacustomattributebyclickingtheicontothe
leftof(emptyobject),thenAppend.Twoinputfieldswillbecomeavailable,FIELD(attributekey)andVALUE(attributevalue).Type
nameaskeyandyournameasvalue.Appendanotherattribute,nameit ageandsetittoyourage.ClickSavetopersistthechanges.If
youclickonCollection:usersatthetopontheright-handsideoftheArangoDBlogo,thedocumentbrowserwillshowthedocumentsin
theuserscollectionandyouwillseethedocumentyoujustcreatedinthelist.
Queryingthedatabase
GettingStarted
9
TimetoretrieveourdocumentusingAQL,ArangoDB'squerylanguage.Wecandirectlylookupthedocumentwecreatedviathe _id,
buttherearealsootheroptions.ClicktheQUERIESmenuentrytobringupthequeryeditorandtypethefollowing(adjustthe
documentIDtomatchyourdocument):
RETURNDOCUMENT("users/9883")
ThenclickExecutetorunthequery.Theresultappearsbelowthequeryeditor:
[
{
"_key":"9883",
"_id":"users/9883",
"_rev":"9883",
"age":32,
"name":"JohnSmith"
}
]
Asyoucansee,theentiredocumentincludingthesystemattributesisreturned.DOCUM ENT()isafunctiontoretrieveasingle
documentoralistofdocumentsofwhichyouknowthe _keysor _ids.Wereturntheresultofthefunctioncallasourqueryresult,
whichisourdocumentinsideoftheresultarray(wecouldhavereturnedmorethanoneresultwithadifferentquery,butevenforasingle
documentasresult,westillgetanarrayatthetoplevel).
Thistypeofqueryiscalleddataaccessquery.Nodataiscreated,changedordeleted.Thereisanothertypeofquerycalleddata
modificationquery.Let'sinsertaseconddocumentusingamodificationquery:
INSERT{name:"KatieFoster",age:27}INTOusers
Thequeryisprettyself-explanatory:the INSERTkeywordtellsArangoDBthatwewanttoinsertsomething.Whattoinsert,a
documentwithtwoattributesinthiscase,followsnext.Thecurlybraces {}signifydocuments,orobjects.Whentalkingabout
recordsinacollection,wecallthemdocuments.EncodedasJSON,wecallthemobjects.Objectscanalsobenested.Here'sanexample:
{
"name":{
"first":"Katie",
"last":"Foster"
}
}
INTOisamandatorypartofevery INSERToperationandisfollowedbythecollectionnamethatwewanttostorethedocumentin.
Notethattherearenoquotemarksaroundthecollectionname.
Ifyourunabovequery,therewillbeanemptyarrayasresultbecausewedidnotspecifywhattoreturnusinga RETURNkeyword.Itis
optionalinmodificationqueries,butmandatoryindataaccessqueries.Evenwith RESULT,thereturnvaluecanstillbeanemptyarray,
e.g.ifthespecifieddocumentwasnotfound.Despitetheemptyresult,theabovequerystillcreatedanewuserdocument.Youcanverify
thiswiththedocumentbrowser.
Let'saddanotheruser,butreturnthenewlycreateddocumentthistime:
INSERT{name:"JamesHendrix",age:69}INTOusers
RETURNNEW
NEWisapseudo-variable,whichreferstothedocumentcreatedby INSERT.Theresultofthequerywilllooklikethis:
GettingStarted
10
[
{
"_key":"10074",
"_id":"users/10074",
"_rev":"10074",
"age":69,
"name":"JamesHendrix"
}
]
Nowthatwehave3usersinourcollection,howtoretrievethemallwithasinglequery?Thefollowingdoesnotwork:
Therecanonlybeasingle RETURNstatementhereandasyntaxerrorisraisedifyoutrytoexecuteit.The DOCUMENT()functionoffersa
secondarysignaturetospecifymultipledocumenthandles,sowecoulddo:
RETURNDOCUMENT(["users/9883","users/9915","users/10074"])
Anarraywiththe _idsofall3documentsispassedtothefunction.Arraysaredenotedbysquarebrackets []andtheirelements
areseparatedbycommas.
Butwhatifweaddmoreusers?Wewouldhavetochangethequerytoretrievethenewlyaddedusersaswell.Allwewanttosaywith
ourqueryis:"Foreveryuserinthecollectionusers,returntheuserdocument".Wecanformulatethiswitha FORloop:
FORuserINusers
RETURNuser
Itexpressestoiterateovereverydocumentin usersandtouse userasvariablename,whichwecanusetorefertothecurrentuser
document.Itcouldalsobecalled doc, uor ahuacatlguacamole,thisisuptoyou.Itisadvisabletouseashortandself-descriptive
namehowever.
Theloopbodytellsthesystemtoreturnthevalueofthevariable user,whichisasingleuserdocument.Alluserdocumentsare
returnedthisway:
[
{
"_key":"9915",
"_id":"users/9915",
"_rev":"9915",
"age":27,
"name":"KatieFoster"
},
{
"_key":"9883",
"_id":"users/9883",
"_rev":"9883",
"age":32,
"name":"JohnSmith"
},
{
"_key":"10074",
"_id":"users/10074",
"_rev":"10074",
"age":69,
"name":"JamesHendrix"
}
]
Youmayhavenoticedthattheorderofthereturneddocumentsisnotnecessarilythesameastheywereinserted.Thereisnoorder
guaranteedunlessyouexplicitlysortthem.Wecanadda SORToperationveryeasily:
GettingStarted
11
FORuserINusers
SORTuser._key
RETURNuser
Thisdoesstillnotreturnthedesiredresult:James(10074)isreturnedbeforeJohn(9883)andKatie(9915).Thereasonisthatthe _key
attributeisastringinArangoDB,andnotanumber.Theindividualcharactersofthestringsarecompared. 1islowerthan 9andthe
resultistherefore"correct".Ifwewantedtousethenumericalvalueofthe _keyattributesinstead,wecouldconvertthestringtoa
numberanduseitforsorting.Therearesomeimplicationshowever.Wearebetteroffsortingsomethingelse.Howabouttheage,in
descendingorder?
FORuserINusers
SORTuser.ageDESC
RETURNuser
Theuserswillbereturnedinthefollowingorder:James(69),John(32),Katie(27).Insteadof DESCfordescendingorder, ASCcanbe
usedforascendingorder. ASCisthedefaultthoughandcanbeomitted.
Wemightwanttolimittheresultsettoasubsetofusers,basedontheageattributeforexample.Let'sreturnusersolderthan30only:
FORuserINusers
FILTERuser.age>30
SORTuser.age
RETURNuser
ThiswillreturnJohnandJames(inthisorder).Katie'sageattributedoesnotfulfillthecriterion(greaterthan30),sheisonly27and
thereforenotpartoftheresultset.Wecanmakeheragetoreturnheruserdocumentagain,usingamodificationquery:
UPDATE"9915"WITH{age:40}INusers
RETURNNEW
UPDATEallowstopartiallyeditanexistingdocument.Thereisalso REPLACE,whichwouldremoveallattributes(exceptfor _keyand
_id,whichremainthesame)andonlyaddthespecifiedones. UPDATEontheotherhandonlyreplacesthespecifiedattributesand
keepseverythingelseas-is.
The UPDATEkeywordisfollowedbythedocumentkey(oradocument/objectwitha _keyattribute)toidentifywhattomodify.The
attributestoupdatearewrittenasobjectafterthe WITHkeyword. INdenotesinwhichcollectiontoperformthisoperationin,just
like INTO(bothkeywordsareactuallyinterchangablehere).Thefulldocumentwiththechangesappliedisreturnedifweusethe NEW
pseudo-variable:
[
{
"_key":"9915",
"_id":"users/9915",
"_rev":"12864",
"age":40,
"name":"KatieFoster"
}
Ifweused REPLACEinstead,thenameattributewouldbegone.With UPDATE,theattributeiskept(thesamewouldapplytoadditional
attributesifwehadthem).
Letusrunour FILTERqueryagain,butonlyreturntheusernamesthistime:
FORuserINusers
FILTERuser.age>30
SORTuser.age
RETURNuser.name
Thiswillreturnthenamesofall3users:
GettingStarted
12
[
"JohnSmith",
"KatieFoster",
"JamesHendrix"
]
Itiscalledaprojectionifonlyasubsetofattributesisreturned.Anotherkindofprojectionistochangethestructureoftheresults:
FORuserINusers
RETURN{userName:user.name,age:user.age}
Thequerydefinestheoutputformatforeveryuserdocument.Theusernameisreturnedas userNameinsteadof name,theagekeeps
theattributekeyinthisexample:
[
{
"userName":"JamesHendrix",
"age":69
},
{
"userName":"JohnSmith",
"age":32
},
{
"userName":"KatieFoster",
"age":40
}
]
Itisalsopossibletocomputenewvalues:
FORuserINusers
RETURNCONCAT(user.name,"'sageis",user.age)
CONCAT()isafunctionthatcanjoinelementstogethertoastring.Weuseitheretoreturnastatementforeveryuser.Asyoucansee,the
resultsetdoesnotalwayshavetobeanarrayofobjects:
[
"JamesHendrix'sageis69",
"JohnSmith'sageis32",
"KatieFoster'sageis40"
]
Nowlet'sdosomethingcrazy:foreverydocumentintheuserscollection,iterateoveralluserdocumentsagainandreturnuserpairs,e.g.
JohnandKatie.Wecanusealoopinsidealoopforthistogetthecrossproduct(everypossiblecombinationofalluserrecords,33=9).
Wedon'twantpairingslikeJohn+John*however,solet'seliminatethemwithafiltercondition:
FORuser1INusers
FORuser2INusers
FILTERuser1!=user2
RETURN[user1.name,user2.name]
Weget6pairings.PairslikeJames+JohnandJohn+Jamesarebasicallyredundant,butfairenough:
[
["JamesHendrix","JohnSmith"],
["JamesHendrix","KatieFoster"],
["JohnSmith","JamesHendrix"],
["JohnSmith","KatieFoster"],
["KatieFoster","JamesHendrix"],
["KatieFoster","JohnSmith"]
]
Wecouldcalculatethesumofbothagesandcomputesomethingnewthisway:
GettingStarted
13
FORuser1INusers
FORuser2INusers
FILTERuser1!=user2
RETURN{
pair:[user1.name,user2.name],
sumOfAges:user1.age+user2.age
}
Weintroduceanewattribute sumOfAgesandaddupbothagesforthevalue:
[
{
"pair":["JamesHendrix","JohnSmith"],
"sumOfAges":101
},
{
"pair":["JamesHendrix","KatieFoster"],
"sumOfAges":109
},
{
"pair":["JohnSmith","JamesHendrix"],
"sumOfAges":101
},
{
"pair":["JohnSmith","KatieFoster"],
"sumOfAges":72
},
{
"pair":["KatieFoster","JamesHendrix"],
"sumOfAges":109
},
{
"pair":["KatieFoster","JohnSmith"],
"sumOfAges":72
}
]
Ifwewantedtopost-filteronthenewattributetoonlyreturnpairswithasumlessthan100,weshoulddefineavariabletotemporarily
storethesum,sothatwecanuseitina FILTERstatementaswellasinthe RETURNstatement:
FORuser1INusers
FORuser2INusers
FILTERuser1!=user2
LETsumOfAges=user1.age+user2.age
FILTERsumOfAges<100
RETURN{
pair:[user1.name,user2.name],
sumOfAges:sumOfAges
}
The LETkeywordisfollowedbythedesignatedvariablename( sumOfAges),thenthere'sa =symbolandthevalueoranexpression
todefinewhatvaluethevariableissupposedtohave.Were-useourexpressiontocalculatethesumhere.Wethenhaveanother FILTER
toskiptheunwantedpairingsandmakeuseofthevariablewedeclaredbefore.Wereturnaprojectionwithanarrayoftheusernamesand
thecalculatedage,forwhichweusethevariableagain:
[
{
"pair":["JohnSmith","KatieFoster"],
"sumOfAges":72
},
{
"pair":["KatieFoster","JohnSmith"],
"sumOfAges":72
}
]
Protip:whendefiningobjects,ifthedesiredattributekeyandthevariabletousefortheattributevaluearethesame,youcanusea
shorthandnotation: {sumOfAges}insteadof {sumOfAges:sumOfAges}.
GettingStarted
14
Finally,let'sdeleteoneoftheuserdocuments:
REMOVE"9883"INusers
ItdeletestheuserJohn( _key:"9883").Wecouldalsoremovedocumentsinaloop(samegoesfor INSERT, UPDATEand REPLACE):
FORuserINusers
FILTERuser.age>=30
REMOVEuserINusers
Thequerydeletesalluserswhoseageisgreaterthanorequalto30.
Howtocontinue
ThereisalotmoretodiscoverinAQLandmuchmorefunctionalitythatArangoDBoffers.Continuereadingtheotherchaptersand
experimentwithatestdatabasetofosteryourknowledge.
IfyouwanttowritemoreAQLqueriesrightnow,havealookhere:
DataQueries:dataaccessandmodificationqueries
High-leveloperations:detaileddescriptionsof FOR, FILTERandmoreoperationsnotshowninthisintroduction
Functions:areferenceofallprovidedfunctions
ArangoDBprograms
TheArangoDBpackagecomeswiththefollowingprograms:
arangod:TheArangoDBdatabasedaemon.Thisserverprogramisintendedtorunasadaemonprocessandtoservethevarious
clientsconnectiontotheserverviaTCP/HTTP.
arangosh:TheArangoDBshell.Aclientthatimplementsaread-eval-printloop(REPL)andprovidesfunctionstoaccessand
administratetheArangoDBserver.
arangoimp:AbulkimporterfortheArangoDBserver.ItsupportsJSONandCSV.
arangodump:AtooltocreatebackupsofanArangoDBdatabaseinJSONformat.
arangorestore:AtooltoloaddataofabackupbackintoanArangoDBdatabase.
arango-dfdb:AdatafiledebuggerforArangoDB.ItisprimarilyintendedtobeusedduringdevelopmentofArangoDB.
arangobench:Abenchmarkandtesttool.Itcanbeusedforperformanceandserverfunctiontesting.
GettingStarted
15
Installing
Firstofall,downloadandinstallthecorrespondingRPM orDebianpackageorusehomebrewonM acOSX.Youcanfindpackagesfor
variousoperationsystemsatourinstallsection,includinginstallersforWindows.
Howtodothatindetailisdescribedinthesubchaptersofthissection.
Onhowtosetupacluster,checkouttheDeploymentchapter.
Installing
16
Linux
VisittheofficialArangoDBinstallpageanddownloadthecorrectpackageforyourLinuxdistribution.Youcanfindbinarypackages
forthemostcommondistributionsthere.
Followtheinstructionstouseyourfavoritepackagemanagerforthemajordistributions.AftersettinguptheArangoDBrepository
youcaneasilyinstallArangoDBusingyum,aptitude,urpmiorzypper.
Debianbasedpackageswillaskforapasswordduringinstallation.ForanunattendedinstallationforDebian,seebelow.Red-Hat
basedpackageswillsetarandompasswordduringinstallation.Forotherdistributionsortochangethepassword,run arangosecure-installationtosetarootpassword.
Alternatively,seeCompilingifyouwanttobuildArangoDByourself.
Startupthedatabaseserver.
Normally,thisisdonebyexecutingthefollowingcommand:
unix>/etc/init.d/arangodstart
Itwillstarttheserver,anddothataswellatsystemboottime.
Tostoptheserveryoucanusethefollowingcommand:
unix>/etc/init.d/arangodstop
TheexactcommandsdependonyourLinuxdistribution.Youmayrequirerootprivilegestoexecutethesecommands.
LinuxMint
PleaseusethecorrespondingUbuntuorDebianpackages.
UnattendedInstallation
Debianbasedpackagewillaskforapasswordduringinstallation.Forunattendedinstallation,youcansetthepasswordusingthedebconf
helpers.
echoarangodb3arangodb3/passwordpasswordNEWPASSWORD|debconf-set-selections
echoarangodb3arangodb3/password_againpasswordNEWPASSWORD|debconf-set-selections
Thecommandsshouldbeexecutedpriortotheinstallation.
Red-Hatbasedpackageswillsetarandompasswordduringinstallation.Ifyouwanttoforceapassword,execute
ARANGODB_DEFAULT_ROOT_PASSWORD=NEWPASSWORDarango-secure-installation
Thecommandshouldbeexecutedaftertheinstallation.
Non-StandardInstallation
IfyoucompiledArangoDBfromsourceanddidnotuseanyinstallationpackage–orusingnon-defaultlocationsand/ormultiple
ArangoDBinstancesonthesamehost–youmaywanttostarttheserverprocessmanually.Youcandosobyinvokingthearangod
binaryfromthecommandlineasshownbelow:
unix>/usr/local/sbin/arangod/tmp/vocbase
20ZZ-XX-YYT12:37:08Z[8145]INFOusingbuilt-inJavaScriptstartupfiles
20ZZ-XX-YYT12:37:08Z[8145]INFOArangoDB(version1.x.y)isreadyforbusiness
20ZZ-XX-YYT12:37:08Z[8145]INFOHaveFun!
Linux
17
Tostopthedatabaseservergracefully,youcaneitherpressCTRL-CorbysendtheSIGINTsignaltotheserverprocess.Onmany
systemsthiscanbeachievedwiththefollowingcommand:
unix>kill-2`pidofarangod`
Onceyoustartedtheserver,thereshouldbearunninginstanceofarangod-theArangoDBdatabaseserver.
unix>psauxw|fgreparangod
arangodb145360.10.6530726423464s002S1:21pm0:00.18/usr/local/sbin/arangod
Ifthereisnosuchprocess,checkthelogfile/var/log/arangodb/arangod.logforerrors.Ifyouseealogmessagelike
2012-12-03T11:35:29Z[12882]ERRORDatabasedirectoryversion(1)islowerthanserverversion(1.2).
2012-12-03T11:35:29Z[12882]ERRORItseemslikeyouhaveupgradedtheArangoDBbinary.Ifthisiswhatyouwantedtodo,pleaserestartwiththe2012-12-03T11:35:29Z[12882]FATALDatabaseversioncheckfailed.Pleasestarttheserverwiththe--database.auto-upgradeoption
makesuretostarttheserveroncewiththe--database.auto-upgradeoption.
Notethatyoumayhavetoenableloggingfirst.Ifyoustarttheserverinashell,youshouldseeerrorsloggedthereaswell.
Linux
18
MacOSX
ThepreferredmethodforinstallingArangoDBunderM acOSXishomebrew.However,incaseyouarenotusinghomebrew,weprovide
acommand-lineapporgraphicalappwhichcontainsalltheexecutables.
Homebrew
Ifyouareusinghomebrew,thenyoucaninstalltheArangoDBusingbrewasfollows:
brewinstallarangodb
ThiswillinstallthecurrentstableversionofArangoDBandalldependencieswithinyourHomebrewtree.Notethattheserverwillbe
installedas:
/usr/local/sbin/arangod
Youcanstarttheserverbyrunningthecommand /usr/local/sbin/arangod&.
TheArangoDBshellwillbeinstalledas:
/usr/local/bin/arangosh
YoucanuninstallArangoDBusing:
brewuninstallarangodb
However,incaseyoustartedArangoDBusingthelaunchctl,youneedtounloaditbeforeuninstallingtheserver:
launchctlunload~/Library/LaunchAgents/homebrew.mxcl.arangodb.plist
ThenremovetheLaunchAgent:
rm~/Library/LaunchAgents/homebrew.mxcl.arangodb.plist
Note:IfthelatestArangoDBVersionisnotshowninhomebrew,youalsoneedtoupdatehomebrew:
brewupdate
Knownissues
Performance-theLLVM deliveredasofM acOSXElCapitanbuildsslowbinaries.UseGCCinstead,untilthisissuehasbeenfixed
byApple.
theCommandlineargumentparsingdoesn'tacceptblanksinfilenames;theCLIversionbelowdoes.
GraphicalApp
Incaseyouarenotusinghomebrew,wealsoprovideagraphicalapp.Youcandownloaditfromhere.
ChooseMacOSX.DownloadandinstalltheapplicationArangoDBinyourapplicationfolder.
Command-LineApp
M acOSX
19
Incaseyouarenotusinghomebrew,wealsoprovideacommand-lineapp.Youcandownloaditfromhere.
ChooseMacOSX.DownloadandinstalltheapplicationArangoDB-CLIinyourapplicationfolder.
Startingtheapplicationwillstarttheserverandopenaterminalwindowshowingyouthelog-file.
ArangoDBserverhasbeenstarted
Thedatabasedirectoryislocatedat
'/Applications/ArangoDB-CLI.app/Contents/MacOS/opt/arangodb/var/lib/arangodb'
Thelogfileislocatedat
'/Applications/ArangoDB-CLI.app/Contents/MacOS/opt/arangodb/var/log/arangodb/arangod.log'
Youcanaccesstheserverusingabrowserat'http://127.0.0.1:8529/'
orstarttheArangoDBshell
'/Applications/ArangoDB-CLI.app/Contents/MacOS/arangosh'
Switchingtolog-filenow,killingthiswindowswillNOTstoptheserver.
2013-10-27T19:42:04Z[23840]INFOArangoDB(version1.4.devel[darwin])isreadyforbusiness.Havefun!
Notethatitispossibletoinstallboth,thehomebrewversionandthecommand-lineapp.Youshould,however,edittheconfigurationfiles
ofoneversionandchangetheportused.
M acOSX
20
Windows
ThedefaultinstallationdirectoryisC:\ProgramFiles\ArangoDB-3.x.x.Duringtheinstallationprocessyoumaychangethis.Inthe
followingdescriptionwewillassumethatArangoDBhasbeeninstalledinthelocation<ROOTDIR>.
Youhavetobecarefulwhenchoosinganinstallationdirectory.Youneedeitherwritepermissiontothisdirectoryoryouneedtomodify
theconfigfilefortheserverprocess.InthelattercasethedatabasedirectoryandtheFoxxdirectoryhavetobewritablebytheuser.
SingleUserInstallation
Selectadifferentdirectoryduringinstallation.ForexampleC:\Users\<Username>\ArangoDBorC:\ArangoDB.
MultipleUsersInstallation
Keepthedefaultdirectory.Aftertheinstallationeditthefile<ROOTDIR>\etc\ArangoDB\arangod.conf.Adjustthedirectoryandapppathsothatthesepathspointintoyourhomedirectory.
[database]
directory=@HOMEDRIVE@\@HOMEPATH@\arangodb\databases
[javascript]
app-path=@HOMEDRIVE@\@HOMEPATH@\arangodb\apps
CreatethedirectoriesforeachuserthatwantstouseArangoDB.
ServiceInstallation
Keepthedefaultdirectory.Aftertheinstallationopenacommandlineasadministrator(searchforcmdandrightclickrunas
administrator).
cmd>arangod--install-service
INFO:addingservice'ArangoDB-themulti-modeldatabase'(internal'ArangoDB')
INFO:addedservicewithcommandline'"C:\ProgramFiles(x86)\ArangoDB3.x.x\bin\arangod.exe"--start-service'
OpentheservicemanagerandstartArangoDB.Inordertoenableloggingeditthefile"\etc\arangodb\arangod.conf"anduncommentthe
fileoption.
[log]
file=@ROOTDIR@\var\log\arangodb\arangod.log
Starting
IfyouinstalledArangoDBasaserviceitisautomaticallystarted.
Otherwise,usetheexecutablearangod.exelocatedin<ROOTDIR>\bin.Thiswillusetheconfigurationfilearangod.conflocatedin
<ROOTDIR>\etc\arangodb,whichyoucanadjusttoyourneedsandusethedatadirectory<ROOTDIR>\var\lib\arangodb.Thisisthe
placewhereallyourdata(databasesandcollections)willbestoredbydefault.
Pleasechecktheoutputofthearangod.exeexecutablebeforegoingon.Iftheserverstartedsuccessfully,youshouldseealine ArangoDB
isreadyforbusiness.Havefun!attheendofitsoutput.
Wenowwishtocheckthattheinstallationisworkingcorrectlyandtodothiswewillbeusingtheadministrationwebinterface.Execute
arangod.exeifyouhavenotalreadydoneso,thenopenupyourwebbrowserandpointittothepage:
http://127.0.0.1:8529/
Windows
21
AdvancedStarting
Ifyouwanttoprovideourownstartscripts,youcansettheenvironmentvariableARANGODB_CONFIG_PATH.Thisvariableshould
pointtoadirectorycontainingtheconfigurationfiles.
UsingtheClient
ToconnecttoanalreadyrunningArangoDBserverinstance,thereisashellarangosh.exelocatedin<ROOTDIR>\bin.Thisstartsashell
whichcanbeused–amongstotherthings–toadministerandqueryalocalorremoteArangoDBserver.
Notethatarangosh.exedoesNOTstartaseparateserver,itonlystartstheshell.Touseityoumusthaveaserverrunningsomewhere,
e.g.byusingthearangod.exeexecutable.
arangosh.exeusesconfigurationfromthefilearangosh.conflocatedin<ROOTDIR>\etc\arangodb\.Pleaseadjustthistoyourneedsif
youwanttousedifferentconnectionsettingsetc.
Uninstalling
TouninstalltheArangoserverapplicationyoucanusethewindowscontrolpanel(asyouwouldnormallyuninstallanapplication).Note
however,thatanydatafilescreatedbytheArangoserverwillremainaswellasthe<ROOTDIR>directory.Tocompletethe
uninstallationprocess,removethedatafilesandthe<ROOTDIR>directorymanually.
LimitationsforCygwin
PleasenotesomeimportantlimitationswhenrunningArangoDBunderCygwin:StartingArangoDBcanbestartedfromoutofaCygwin
terminal,butpressingCTRL-Cwillforcefullykilltheserverprocesswithoutgivingitachancetohandlethekillsignal.Inthiscase,a
regularservershutdownisnotpossible,whichmayleaveafileLOCKaroundintheserver'sdatadirectory.Thisfileneedstoberemoved
manuallytomakeArangoDBstartagain.Additionally,asArangoDBdoesnothaveachancetohandlethekillsignal,theservercannot
forcefullyflushanydatatodiskonshutdown,leadingtopotentialdataloss.WhenstartingArangoDBfromaCygwinterminalitmight
alsohappenthatnoerrorsareprintedintheterminaloutput.StartingArangoDBfromanM S-DOScommandpromptdoesnotimpose
theselimitationsandisthusthepreferredmethod.
PleasenotethatArangoDBusesUTF-8asitsinternalencodingandthatthesystemconsolemustsupportaUTF-8codepage(65001)
andfont.ItmaybenecessarytomanuallyswitchtheconsolefonttoafontthatsupportsUTF-8.
Windows
22
CompilingArangoDBfromscratch
ThefollowingsectionsdescribehowtocompileandbuildtheArangoDBfromscratch.ArangoDBwillcompileonmostLinuxandM ac
OSXsystems.WeassumethatyouusetheGNUC/C++compilerorclang/clang++tocompilethesource.ArangoDBhasbeentested
withthesecompilers,butshouldbeabletocompilewithanyPosix-compliant,C++11-enabledcompiler.Pleaseletusknowwhetheryou
successfullycompileditwithanotherC/C++compiler.
Bydefault,cloningthegithubrepositorywillcheckoutdevel.ThisversioncontainsthedevelopmentversionoftheArangoDB.Usethis
branchifyouwanttomakechangestotheArangoDBsource.
PleasecheckoutthecookbookonhowtocompileArangoDB.
Compiling
23
Authentication
ArangoDBallowstorestrictaccesstodatabasestocertainusers.Allusersofthesystemdatabaseareconsideredadministrators.During
installationadefaultuserrootiscreated,whichhasaccesstoalldatabases.
Youshouldcreateadatabaseforyourapplicationtogetherwithauserthathasaccessrightstothisdatabase.SeeM anagingUsers.
Usethearangoshtocreateanewdatabaseanduser.
arangosh>db._createDatabase("example");
arangosh>varusers=require("@arangodb/users");
arangosh>users.save("root@example","password");
arangosh>users.grantDatabase("root@example","example");
Youcannowconnecttothenewdatabaseusingtheuserroot@example.
shell>arangosh--server.username"root@example"--server.databaseexample
Authentication
24
ArangoDBcomeswithabuilt-inwebinterfaceforadministration.ThewebinterfacecanbeaccessedviatheURL:
http://127.0.0.1:8529
Ifeverythingworksasexpected,youshouldseetheloginview:
FormoreinformationontheArangoDBwebinterface,seeWebInterface
25
ComingfromSQL
Ifyouworkedwitharelationaldatabasemanagementsystem(RDBM S)suchasM ySQL,M ariaDBorPostgreSQL,youwillbefamiliar
withitsquerylanguage,adialectofSQL(StructuredQueryLanguage).
ArangoDB'squerylanguageiscalledAQL.Therearesomesimilaritiesbetweenbothlanguagesdespitethedifferentdatamodelsofthe
databasesystems.ThemostnotabledifferenceisprobablytheconceptofloopsinAQL,whichmakesitfeelmorelikeaprogramming
language.Itsuitestheschema-lessmodelmorenaturalandmakesthequerylanguageverypowerfulwhileremainingeasytoreadand
write.
TogetstartedwithAQL,havealookatourdetailedcomparisonofSQLandAQL.ItwillalsohelpyoutotranslateSQLqueriestoAQL
whenmigratingtoArangoDB.
Howdobrowsevectorstranslateintodocumentqueries?
IntraditionalSQLyoumayeitherfetchallcolumnsofatablerowbyrow,using SELECT*FROMtable,orselectasubsetofthecolumns.
Thelistoftablecolumnstofetchiscommonlycalledcolumnlistorbrowsevector:
SELECTcolumnA,columnB,columnZFROMtable
Sincedocumentsaren'ttwo-dimensional,andneitherdoyouwanttobelimitedtoreturningtwo-dimensionallists,therequirementsfora
querylanguagearehigher.AQListhusalittlebitmorecomplexthanplainSQLatfirst,butoffersmuchmoreflexibilityinthelongrun.It
letsyouhandlearbitrarilystructureddocumentsinconvenientways,mostlyleanedonthesyntaxusedinJavaScript.
Composingthedocumentstobereturned
TheAQL RETURNstatementreturnsoneitemperdocumentitishanded.Youcanreturnthewholedocument,orjustpartsofit.Given
thatoneDocumentisadocument(retrievedlike LEToneDocument=DOCUMENT("myusers/3456789")forinstance),itcanbereturnedas-is
likethis:
RETURNoneDocument
[
{
"_id":"myusers/3456789",
"_key":"3456789"
"_rev":"14253647",
"firstName":"John",
"lastName":"Doe",
"address":{
"city":"Gotham",
"street":"RoadToNowhere1"
},
"hobbies":[
{name:"swimming",howFavorite:10},
{name:"biking",howFavorite:6},
{name:"programming",howFavorite:4}
]
}
]
Returnthehobbiessub-structureonly:
RETURNoneDocument.hobbies
ComingfromSQL
26
[
[
]
]
Returnthehobbiesandtheaddress:
RETURN{
hobbies:oneDocument.hobbies,
address:oneDocument.address
}
[
{
hobbies:[
],
address:{
"city":"Gotham",
"street":"RoadToNowhere1"
}
}
]
Returnthefirsthobbyonly:
RETURNoneDocument.hobbies[0].name
[
"swimming"
]
Returnalistofallhobbystrings:
RETURN{hobbies:oneDocument.hobbies[*].name}
[
{hobbies:["swimming","biking","porgramming"]}
]
M orecomplexarrayandobjectmanipulationscanbedoneusingAQLfunctionsandoperators.
ComingfromSQL
27
Scalability
ArangoDBisadistributeddatabasesupportingmultipledatamodels,andcanthusbescaledhorizontally,thatis,byusingmanyservers,
typicallybasedoncommodityhardware.Thisapproachnotonlydeliversperformanceaswellascapacityimprovements,butalso
achievesresiliencebymeansofreplicationandautomaticfail-over.Furthermore,onecanbuildsystemsthatscaletheircapacity
dynamicallyupanddownautomaticallyaccordingtodemand.
OnecanalsoscaleArangoDBvertically,thatis,byusingeverlargerservers.ThereisnobuiltinlimitationinArangoDB,forexample,the
serverwillautomaticallyusemorethreadsifmoreCPUsarepresent.
However,scalingverticallyhasthedisadvantagethatthecostsgrowfasterthanlinearwiththesizeoftheserver,andnoneofthe
resilienceanddynamicalcapabilitiescanbeachievedinthisway.
InthischapterweexplainthedistributedarchitectureofArangoDBanddiscussitsscalabilityfeaturesandlimitations:
ArangoDB'sdistributedarchitecture
Differentdatamodelsandscalability
Limitations
Scalability
28
Architecture
TheclusterarchitectureofArangoDBisaCPmaster/mastermodelwithnosinglepointoffailure.With"CP"wemeanthatinthe
presenceofanetworkpartition,thedatabaseprefersinternalconsistencyoveravailability.With"master/master"wemeanthatclients
cansendtheirrequeststoanarbitrarynode,andexperiencethesameviewonthedatabaseregardless."Nosinglepointoffailure"means
thattheclustercancontinuetoserverequests,evenifonemachinefailscompletely.
Inthisway,ArangoDBhasbeendesignedasadistributedmulti-modeldatabase.Thissectiongivesashortoutlineonthecluster
architectureandhowtheabovefeaturesandcapabilitiesareachieved.
StructureofanArangoDBcluster
AnArangoDBclusterconsistsofanumberofArangoDBinstanceswhichtalktoeachotheroverthenetwork.Theyplaydifferentroles,
whichwillbeexplainedindetailbelow.Thecurrentconfigurationoftheclusterisheldinthe"Agency",whichisahighly-available
resilientkey/valuestorebasedonanoddnumberofArangoDBinstancesrunningRaftConsensusProtocol.
ForthevariousinstancesinanArangoDBclusterthereare4distinctroles:Agents,Coordinators,PrimaryandSecondaryDBservers.In
thefollowingsectionswewillshedlightoneachofthem.NotethatthetasksforallrolesrunthesamebinaryfromthesameDocker
image.
Agents
OneormultipleAgentsformtheAgencyinanArangoDBcluster.TheAgencyisthecentralplacetostoretheconfigurationinacluster.
Itperformsleaderelectionsandprovidesothersynchronizationservicesforthewholecluster.WithouttheAgencynoneoftheother
componentscanoperate.
Whilegenerallyinvisibletotheoutsideitistheheartofthecluster.Assuch,faulttoleranceisofcourseamusthavefortheAgency.To
achievethattheAgentsareusingtheRaftConsensusAlgorithm.Thealgorithmformallyguaranteesconflictfreeconfiguration
managementwithintheArangoDBcluster.
AtitscoretheAgencymanagesabigconfigurationtree.Itsupportstransactionalreadandwriteoperationsonthistree,andotherservers
cansubscribetoHTTPcallbacksforallchangestothetree.
Coordinators
Coordinatorsshouldbeaccessiblefromtheoutside.Thesearetheonestheclientstalkto.Theywillcoordinateclustertaskslike
executingqueriesandrunningFoxxservices.Theyknowwherethedataisstoredandwilloptimizewheretorunusersuppliedqueriesor
partsthereof.Coordinatorsarestatelessandcanthuseasilybeshutdownandrestartedasneeded.
PrimaryDBservers
PrimaryDBserversaretheoneswherethedataisactuallyhosted.Theyhostshardsofdataandusingsynchronousreplicationaprimary
mayeitherbeleaderorfollowerforashard.
Theyshouldnotbeaccessedfromtheoutsidebutindirectlythroughthecoordinators.Theymayalsoexecutequeriesinpartorasa
wholewhenaskedbyacoordinator.
Secondaries
SecondaryDBserversareasynchronousreplicasofprimaries.Ifoneisusingonlysynchronousreplication,onedoesnotneedsecondaries
atall.Foreachprimary,therecanbeoneormoresecondaries.Sincethereplicationworksasynchronously(eventualconsistency),the
replicationdoesnotimpedetheperformanceoftheprimaries.Ontheotherhand,theirreplicaofthedatacanbeslightlyoutofdate.The
secondariesareperfectlysuitableforbackupsastheydon'tinterferewiththenormalclusteroperation.
ClusterID
Everynon-AgencyArangoDBinstanceinaclusterisassignedauniqueIDduringitsstartup.UsingitsIDanodeisidentifiable
throughoutthecluster.AllclusteroperationswillcommunicateviathisID.
Architecture
29
Sharding
UsingtherolesoutlinedaboveanArangoDBclusterisabletodistributedatainsocalledshardsacrossmultipleprimaries.Fromthe
outsidethisprocessisfullytransparentandassuchweachievethegoalsofwhatothersystemscall"master-masterreplication".Inan
ArangoDBclusteryoutalktoanycoordinatorandwheneveryoureadorwritedataitwillautomaticallyfigureoutwherethedatais
stored(read)ortobestored(write).TheinformationabouttheshardsissharedacrossthecoordinatorsusingtheAgency.
AlsoseeShardingintheAdministrationchapter.
Manysensibleconfigurations
Thisarchitectureisveryflexibleandthusallowsmanyconfigurations,whicharesuitablefordifferentusagescenarios:
1. ThedefaultconfigurationistorunexactlyonecoordinatorandoneprimaryDBserveroneachmachine.Thisachievestheclassical
master/mastersetup,sincethereisaperfectsymmetrybetweenthedifferentnodes,clientscanequallywelltalktoanyoneofthe
coordinatorsandallexposethesameviewtothedatastore.
2. OnecandeploymorecoordinatorsthanDBservers.ThisisasensibleapproachifoneneedsalotofCPUpowerfortheFoxx
services,becausetheyrunonthecoordinators.
3. OnecandeploymoreDBserversthancoordinatorsifmoredatacapacityisneededandthequeryperformanceisthelesser
bottleneck
4. Onecandeployacoordinatoroneachmachinewhereanapplicationserver(e.g.anode.jsserver)runs,andtheAgentsand
DBserversonaseparatesetofmachineselsewhere.Thisavoidsanetworkhopbetweentheapplicationserverandthedatabaseand
thusdecreaseslatency.Essentially,thismovessomeofthedatabasedistributionlogictothemachinewheretheclientruns.
Theseforshallsufficefornow.Theimportantpieceofinformationhereisthatthecoordinatorlayercanbescaledanddeployed
independentlyfromtheDBserverlayer.
Replication
ArangoDBofferstwowaysofdatareplicationwithinacluster,synchronousandasynchronous.Inthissectionweexplainsomedetails
andhighlighttheadvantagesanddisadvantagesrespectively.
Synchronousreplicationwithautomaticfail-over
Synchronousreplicationworksonaper-shardbasis.Oneconfiguresforeachcollection,howmanycopiesofeachshardarekeptinthe
cluster.Atanygiventime,oneofthecopiesisdeclaredtobethe"leader"andallotherreplicasare"followers".Writeoperationsforthis
shardarealwayssenttotheDBserverwhichhappenstoholdtheleadercopy,whichinturnreplicatesthechangestoallfollowersbefore
theoperationisconsideredtobedoneandreportedbacktothecoordinator.Readoperationsareallservedbytheserverholdingthe
leadercopy,thisallowstoprovidesnapshotsemanticsforcomplextransactions.
IfaDBserverfailsthatholdsafollowercopyofashard,thentheleadercannolongersynchronizeitschangestothatfollower.Aftera
shorttimeout(3seconds),theleadergivesuponthefollower,declaresittobeoutofsync,andcontinuesservicewithoutthefollower.
Whentheserverwiththefollowercopycomesback,itautomaticallyresynchronizesitsdatawiththeleaderandsynchronousreplication
isrestored.
IfaDBserverfailsthatholdsaleadercopyofashard,thentheleadercannolongerserveanyrequests.Itwillnolongersendaheartbeat
totheAgency.Therefore,asupervisionprocessrunningintheRaftleaderoftheAgency,cantakethenecessaryaction(after15seconds
ofmissingheartbeats),namelytopromoteoneoftheserversthatholdin-syncreplicasoftheshardtoleaderforthatshard.Thisinvolves
areconfigurationintheAgencyandleadstothefactthatcoordinatorsnowcontactadifferentDBserverforrequeststothisshard.Service
resumes.Theothersurvivingreplicasautomaticallyresynchronizetheirdatawiththenewleader.WhentheDBserverwiththeoriginal
leadercopycomesback,itnoticesthatitnowholdsafollowerreplica,resynchronizesitsdatawiththenewleaderandorderisrestored.
Allsharddatasynchronizationsaredoneinanincrementalway,suchthatresynchronizationsarequick.Thistechnologyallowstomove
shards(followerandleaderones)betweenDBserverswithoutserviceinterruptions.Therefore,anArangoDBclustercanmoveallthe
dataonaspecificDBservertootherDBserversandthenshutdownthatserverinacontrolledway.Thisallowstoscaledownan
ArangoDBclusterwithoutserviceinterruption,lossoffaulttoleranceordataloss.Furthermore,onecanre-balancethedistributionofthe
shards,eithermanuallyorautomatically.
AlltheseoperationscanbetriggeredviaaREST/JSONAPIorviathegraphicalwebUI.Allfail-overoperationsarecompletelyhandled
withintheArangoDBcluster.
Architecture
30
Obviously,synchronousreplicationinvolvesacertainincreasedlatencyforwriteoperations,simplybecausethereisonemorenetwork
hopwithintheclusterforeveryrequest.Thereforetheusercansetthereplicationfactorto1,whichmeansthatonlyonecopyofeach
shardiskept,therebyswitchingoffsynchronousreplication.Thisisasuitablesettingforlessimportantoreasilyrecoverabledatafor
whichlowlatencywriteoperationsmatter.
Asynchronousreplicationwithautomaticfail-over
Asynchronousreplicationworksdifferently,inthatitisorganizedusingprimaryandsecondaryDBservers.Eachsecondaryserver
replicatesallthedataheldonaprimarybypollinginanasynchronousway.Thisprocesshasverylittleimpactontheperformanceofthe
primary.Thedisadvantageisthatthereisadelaybetweentheconfirmationofawriteoperationthatissenttotheclientandtheactual
replicationofthedata.Ifthemasterserverfailsduringthisdelay,thencommittedandconfirmeddatacanbelost.
Nevertheless,wealsoofferautomaticfail-overwiththissetup.Contrarytothesynchronouscase,herethefail-overmanagementisdone
fromoutsidetheArangoDBcluster.InafutureversionwemightmovethismanagementintothesupervisionprocessintheAgency,but
asofnow,themanagementisdoneviatheM esosframeworkschedulerforArangoDB(seebelow).
ThegranularityofthereplicationisawholeArangoDBinstancewithalldatathatresidesonthatinstance,whichmeansthatyouneed
twiceasmanyinstancesaswithoutasynchronousreplication.Synchronousreplicationismoreflexibleinthatrespect,youcanhave
smallerandlargerinstances,andifonefails,thedatacanberebalancedacrosstheremainingones.
Microservicesandzeroadministation
ThedesignandcapabilitiesofArangoDBaregearedtowardsusageinmodernmicroservicearchitecturesofapplications.WiththeFoxx
servicesitisveryeasytodeployadatacentricmicroservicewithinanArangoDBcluster.
Inaddition,onecandeploymultipleinstancesofArangoDBwithinthesameproject.Onepartoftheprojectmightneedascalable
documentstore,anothermightneedagraphdatabase,andyetanothermightneedthefullpowerofamulti-modeldatabaseactually
mixingthevariousdatamodels.Thereareenormousefficiencybenefitstobereapedbybeingabletouseasingletechnologyforvarious
rolesinaproject.
TosimplifyliveofthedevopsinsuchascenariowetryasmuchaspossibletouseazeroadministrationapproachforArangoDB.A
runningArangoDBclusterisresilientagainstfailuresandessentiallyrepairsitselfincaseoftemporaryfailures.Seethenextsectionfor
furthercapabilitiesinthisdirection.
ApacheMesosintegration
Forthedistributedsetup,weusetheApacheM esosinfrastructurebydefault.ArangoDBisafullycertifiedpackageforDC/OSandcan
thusbedeployedessentiallywithafewmouseclicksorasinglecommand,onceyouhaveanexistingDC/OScluster.Butevenonaplain
ApacheM esosclusteronecandeployArangoDBviaM arathonwithasingleAPIcallandsomeJSONconfiguration.
Theadvantageofthisapproachisthatwecannotonlyimplementtheinitialdeployment,butalsothelatermanagementofautomatic
replacementoffailedinstancesandthescalingoftheArangoDBcluster(triggeredmanuallyorevenautomatically).Sinceall
manipulationsareeitherviathegraphicalwebUIorviaJSON/RESTcalls,onecanevenimplementauto-scalingveryeasily.
ADC/OSclusterisaverynaturalenvironmenttodeploymicroservicearchitectures,sinceitissoconvenienttodeployvariousservices,
includingpotentiallymultipleArangoDBclusterinstanceswithinthesameDC/OScluster.Thebuilt-inservicediscoverymakesit
extremelysimpletoconnectthevariousmicroservicesandM esosautomaticallytakescareofthedistributionanddeploymentofthe
varioustasks.
AsofJune2016,weofferApacheM esosintegration,latertherewillbeintegrationwithotherclustermanagementinfrastructures.See
theDeploymentchapteranditssubsectionsforinstructions.
ItispossibletodeployanArangoDBclusterbysimplylaunchingabunchofDockercontainerswiththerightcommandlineoptionsto
linkthemup,orevenonasinglemachinestartingmultipleArangoDBprocesses.Inthatcase,synchronousreplicationwillworkwithin
thedeployedArangoDBcluster,andautomaticfail-overinthesensethatthedutiesofafailedserverwillautomaticallybeassignedto
another,survivingone.However,sincetheArangoDBclustercannotwithinitselflaunchadditionalinstances,replacementoffailednodes
isnotautomaticandscalingupanddownhastobemanagedmanually.Thisiswhywedonotrecommendthissetupforproduction
deployment.
Authentication
Architecture
31
Asofversion3.0ArangoDBauthenticationisNOTsupportedwithinacluster.YouHAVEtoproperlysecureyourclustertothe
outside.M ostsetupswillhaveasecureddatacenteranywayandArangoDBwillbeaccessedfromtheoutsideviaanapplication.Tothis
applicationonlythecoordinatorsneedtobemadeavailable.Ifyouwanttoisolateevenfurtheryoucaninstallareverseproxylike
haproxyornginxinfrontofthecoordinators(thatwillalsoalloweasyaccessfromtheapplication).
Authenticationintheclusterwillbeaddedsoonaftertheinitial3.0release.
Architecture
32
Differentdatamodelsandscalability
InthissectionwediscussscalabilityinthecontextofthedifferentdatamodelssupportedbyArangoDB.
Key/valuepairs
Thekey/valuestoredatamodelistheeasiesttoscale.InArangoDB,thisisimplementedinthesensethatadocumentcollectionalways
hasaprimarykey _keyattributeandintheabsenceoffurthersecondaryindexesthedocumentcollectionbehaveslikeasimple
key/valuestore.
Theonlyoperationsthatarepossibleinthiscontextaresinglekeylookupsandkey/valuepairinsertionsandupdates.If _keyisthe
onlyshardingattributethentheshardingisdonewithrespecttotheprimarykeyandalltheseoperationsscalelinearly.Iftheshardingis
doneusingdifferentshardkeys,thenalookupofasinglekeyinvolvesaskingallshardsandthusdoesnotscalelinearly.
Documentstore
Forthedocumentstorecaseeveninthepresenceofsecondaryindexesessentiallythesameargumentsapply,sinceanindexforasharded
collectionissimplythesameasalocalindexforeachshard.Therefore,singledocumentoperationsstillscalelinearlywiththesizeofthe
cluster,unlessaspecialshardingconfigurationmakeslookupsorwriteoperationsmoreexpensive.
ForadeeperanalysisofthistopicseethisblogpostinwhichgoodlinearscalabilityofArangoDBforsingledocumentoperationsis
demonstrated.
Complexqueriesandjoins
TheAQLquerylanguageallowscomplexqueries,usingmultiplecollections,secondaryindexesaswellasjoins.Inparticularwiththe
latter,scalingcanbeachallenge,sinceifthedatatobejoinedresidesondifferentmachines,alotofcommunicationhastohappen.The
AQLqueryexecutionengineorganizesadatapipelineacrosstheclustertoputtogethertheresultsinthemostefficientway.Thequery
optimizerisawareoftheclusterstructureandknowswhatdataiswhereandhowitisindexed.Therefore,itcanarriveataninformed
decisionaboutwhatpartsofthequeryoughttorunwhereinthecluster.
Nevertheless,forcertaincomplicatedjoins,therearelimitsastowhatcanbeachieved.
Graphdatabase
Graphdatabasesareparticularlygoodatqueriesongraphsthatinvolvepathsinthegraphofanaprioriunknownlength.Forexample,
findingtheshortestpathbetweentwoverticesinagraph,orfindingallpathsthatmatchacertainpatternstartingatagivenvertexare
suchexamples.
However,iftheverticesandedgesalongtheoccurringpathsaredistributedacrossthecluster,thenalotofcommunicationisnecessary
betweennodes,andperformancesuffers.Toachievegoodperformanceatscale,itisthereforenecessarytogetthedistributionofthe
graphdataacrosstheshardsintheclusterright.M ostofthetime,theapplicationdevelopersandusersofArangoDBknowbest,how
theirgraphsarastructured.Therefore,ArangoDBallowsuserstospecify,accordingtowhichattributesthegraphdataissharded.A
usefulfirststepisusuallytomakesurethattheedgesoriginatingatavertexresideonthesameclusternodeasthevertex.
Datamodels
33
Limitations
ArangoDBhasnobuilt-inlimitationstohorizontalscalability.ThecentralresilientAgencywilleasilysustainhundredsofDBserversand
coordinators,andtheusualdatabaseoperationsworkcompletelydecentrallyanddonotrequireassistanceoftheAgency.
Likewise,thesupervisionprocessintheAgencycaneasilydealwithlotsofservers,sinceallitsactivitiesarenotperformancecritical.
Obviously,anArangoDBclusterislimitedbytheavailableresourcesofCPU,memory,diskandnetworkbandwidthandlatency.
Limitations
34
Datamodels&modeling
ThischapterintroducesArangoDB'scoreconceptsandcovers
itsdatamodel(ordatamodelsrespectively),
theterminologyusedthroughoutthedatabasesystemandinthisdocumentation,aswellas
aspectstoconsiderwhenmodelingyourdatatostrikeabalancebetweennaturaldatastructuresandgreatperformance
Youwillalsofindusageexamplesonhowtointeractwiththedatabasesystemusingarangosh,e.g.howtocreateanddropdatabases/
collections,orhowtosave,update,replaceandremovedocuments.Youcandoallthisusingthewebinterfaceaswellandmaytherefore
skipthesesectionsasbeginner.
Datamodels&modeling
35
Concepts
DatabaseInteraction
ArangoDBisadatabasethatservesdocumentstoclients.ThesedocumentsaretransportedusingJSONviaaTCPconnection,usingthe
HTTPprotocol.ARESTAPIisprovidedtointeractwiththedatabasesystem.
ThewebinterfacethatcomeswithArangoDB,calledAardvark,providesgraphicaluserinterfacethatiseasytouse.Aninteractiveshell,
calledArangosh,isalsoshipped.Inaddition,therearesocalleddriversthatmakeiteasytousethedatabasesysteminvarious
environmentsandprogramminglanguages.AllthesetoolsusetheHTTPinterfaceoftheserverandremovethenecessitytorollownlowlevelcodeforbasiccommunicationinmostcases.
Datamodel
ThedocumentsyoucanstoreinArangoDBcloselyfollowtheJSONformat,althoughtheyarestoredinabinaryformatcalled
VelocyPack.Adocumentcontainszeroormoreattributes,eachoftheseattributeshavingavalue.Avaluecaneitherbeanatomictype,i.
e.number,string,booleanornull,oracompoundtype,i.e.anarrayorembeddeddocument/object.Arraysandsub-objectscancontain
allofthesetypes,whichmeansthatarbitrarilynesteddatastructurescanberepresentedinasingledocument.
Documentsaregroupedintocollections.Acollectioncontainszeroormoredocuments.Ifyouarefamiliarwithrelationaldatabase
managementsystems(RDBM S)thenitissafetocomparecollectionstotablesanddocumentstorows.Thedifferenceisthatina
traditionalRDBM S,youhavetodefinecolumnsbeforeyoucanstorerecordsinatable.Suchdefinitionsarealsoknownasschemas.
ArangoDBisschema-less,whichmeansthatthereisnoneedtodefinewhatattributesadocumentcanhave.Everysingledocumentcan
haveacompletelydifferentstructureandstillbestoredtogetherwithotherdocumentsinasinglecollection.Inpractice,therewillbe
commondenominatorsamongthedocumentsinacollection,butthedatabasesystemitselfdoesn'tforceyoutolimityourselftoacertain
datastructure.
Therearetwotypesofcollections:documentcollection(alsoreferedtoasvertexcollectionsinthecontextofgraphs)aswellasedge
collections.Edgecollectionsstoredocumentsaswell,buttheyincludetwospecialattributes,_fromand_to,whichareusedtocreate
relationsbetweendocuments.Usually,twodocuments(vertices)storedindocumentcollectionsarelinkedbyadocument(edge)stored
inanedgecollection.ThisisArangoDB'sgraphdatamodel.Itfollowsthemathematicalconceptofadirected,labeledgraph,exceptthat
edgesdon'tjusthavelabels,butarefull-blowndocuments.
Collectionsexistinsideofdatabases.Therecanbeoneormanydatabases.Differentdatabasesareusuallyusedformultitenantsetups,
asthedatainsidethem(collections,documentsetc.)isisolatedfromoneanother.Thedefaultdatabase_systemisspecial,becauseit
cannotberemoved.Databaseusersaremanagedinthisdatabase,andtheircredentialsarevalidforalldatabasesofaserverinstance.
DataRetrieval
Queriesareusedtofilterdocumentsbasedoncertaincriteria,tocomputenewdata,aswellastomanipulateordeleteexisting
documents.Queriescanbeassimpleasa"querybyexample"orascomplexas"joins"usingmanycollectionsortraversinggraph
structures.TheyarewrittenintheArangoDBQueryLanguage(AQL).
Cursorsareusedtoiterateovertheresultofqueries,sothatyougeteasilyprocessablebatchesinsteadofonebighunk.
Indexesareusedtospeedupsearches.Therearevarioustypesofindexes,suchashashindexesandgeoindexes.
Concepts
36
HandlingDatabases
ThisisanintroductiontomanagingdatabasesinArangoDBfromwithinJavaScript.
WhenyouhaveanestablishedconnectiontoArangoDB,thecurrentdatabasecanbechangedexplicitlyusingthedb._useDatabase()
method.Thiswillswitchtothespecifieddatabase(provideditexistsandtheusercanconnecttoit).Fromthispointon,anyfollowing
actioninthesameshellorconnectionwillusethespecifieddatabase,unlessotherwisespecified.
Note:Ifthedatabaseischanged,clientdriversneedtostorethecurrentdatabasenameontheirside,too.Thisisbecauseconnectionsin
ArangoDBdonotcontainanystateinformation.AllstateinformationiscontainedintheHTTPrequest/responsedata.
Toconnecttoaspecificdatabaseafterarangoshhasstartedusethecommanddescribedabove.Itisalsopossibletospecifyadatabase
namewheninvokingarangosh.Forthispurpose,usethecommand-lineparameter--server.database,e.g.
>arangosh--server.databasetest
Pleasenotethatcommands,actions,scriptsorAQLqueriesshouldneveraccessmultipledatabases,eveniftheyexist.Theonlyintended
andsupportedwayinArangoDBistouseonedatabaseatatimeforacommand,anaction,ascriptoraquery.Operationsstartedinone
databasemustnotswitchthedatabaselaterandcontinueoperatinginanother.
Databases
37
DatabaseMethods
ThefollowingmethodsareavailabletomanagedatabasesviaJavaScript.Pleasenotethatseveralofthesemethodscanbeusedfromthe
_systemdatabaseonly.
Name
returnthedatabasename db._name()
Returnsthenameofthecurrentdatabaseasastring.
Examples
arangosh>require("@arangodb").db._name();
_system
ID
returnthedatabaseid db._id()
Returnstheidofthecurrentdatabaseasastring.
Examples
arangosh>require("@arangodb").db._id();
1
Path
returnthepathtodatabasefiles db._path()
Returnsthefilesystempathofthecurrentdatabaseasastring.
Examples
arangosh>require("@arangodb").db._path();
/tmp/arangosh_ehCCsu/tmp-28823-2533987175/data/databases/database-1
isSystem
returnthedatabasetype db._isSystem()
Returnswhetherthecurrentlyuseddatabaseisthe_systemdatabase.Thesystemdatabasehassomespecialprivilegesandproperties,for
example,databasemanagementoperationssuchascreateordropcanonlybeexecutedfromwithinthisdatabase.Additionally,the
_systemdatabaseitselfcannotbedropped.
UseDatabase
changethecurrentdatabase db._useDatabase(name)
Changesthecurrentdatabasetothedatabasespecifiedbyname.Notethatthedatabasespecifiedbynamemustalreadyexist.
Changingthedatabasemightbedisallowedinsomecontexts,forexampleserver-sideactions(includingFoxx).
38
Whenperformingthiscommandfromarangosh,thecurrentcredentials(usernameandpassword)willbere-used.Thesecredentialsmight
notbevalidtoconnecttothedatabasespecifiedbyname.Additionally,thedatabaseonlybeaccessedfromcertainendpointsonly.In
thiscase,switchingthedatabasemightnotwork,andtheconnection/sessionshouldbeclosedandrestartedwithdifferentusernameand
passwordcredentialsand/orendpointdata.
ListDatabases
returnthelistofallexistingdatabases db._databases()
Returnsthelistofalldatabases.Thismethodcanonlybeusedfromwithinthe_systemdatabase.
CreateDatabase
createanewdatabase db._createDatabase(name,options,users)
Createsanewdatabasewiththenamespecifiedbyname.Therearerestrictionsfordatabasenames(seeDatabaseNames).
Notethatevenifthedatabaseiscreatedsuccessfully,therewillbenochangeintothecurrentdatabasetothenewdatabase.Changingthe
currentdatabasemustexplicitlyberequestedbyusingthedb._useDatabasemethod.
Theoptionsattributecurrentlyhasnomeaningandisreservedforfutureuse.
Theoptionalusersattributecanbeusedtocreateinitialusersforthenewdatabase.Ifspecified,itmustbealistofuserobjects.Each
userobjectcancontainthefollowingattributes:
username:theusernameasastring.Thisattributeismandatory.
passwd:theuserpasswordasastring.Ifnotspecified,thenitdefaultstoanemptystring.
active:abooleanflagindicatingwhethertheuseraccountshouldbeactiveornot.Thedefaultvalueistrue.
extra:anoptionalJSONobjectwithextrauserinformation.Thedatacontainedinextrawillbestoredfortheuserbutnotbe
interpretedfurtherbyArangoDB.
Ifnoinitialusersarespecified,adefaultuserrootwillbecreatedwithanemptystringpassword.Thisensuresthatthenewdatabasewill
beaccessibleviaHTTPafteritiscreated.
Youcancreateusersinadatabaseifnoinitialuserisspecified.Switchintothenewdatabase(usernameandpasswordmustbeidentical
tothecurrentsession)andaddormodifyuserswiththefollowingcommands.
require("@arangodb/users").save(username,password,true);
require("@arangodb/users").update(username,password,true);
require("@arangodb/users").remove(username);
Alternatively,youcanspecifyuserdatadirectly.Forexample:
db._createDatabase("newDB",[],[{username:"newUser",passwd:"123456",active:true}])
Thosemethodscanonlybeusedfromwithinthe_systemdatabase.
DropDatabase
dropanexistingdatabase db._dropDatabase(name)
Dropsthedatabasespecifiedbyname.Thedatabasespecifiedbynamemustexist.
Note:Droppingdatabasesisonlypossiblefromwithinthe_systemdatabase.The_systemdatabaseitselfcannotbedropped.
Databasesaredroppedasynchronously,andwillbephysicallyremovedifallclientshavedisconnectedandreferenceshavebeengarbagecollected.
39
NotesaboutDatabases
Pleasekeepinmindthateachdatabasecontainsitsownsystemcollections,whichneedtobesetupwhenadatabaseiscreated.Thiswill
makethecreationofadatabasetakeawhile.
Replicationisconfiguredonaper-databaselevel,meaningthatanyreplicationloggingorapplyingforanewdatabasemustbeconfigured
explicitlyafteranewdatabasehasbeencreated.
Foxxapplicationsarealsoavailableonlyinthecontextofthedatabasetheyhavebeeninstalledin.Anewdatabasewillonlyprovide
accesstothesystemapplicationsshippedwithArangoDB(thatisthewebinterfaceatthemoment)andnootherFoxxapplicationsuntil
theyareexplicitlyinstalledfortheparticulardatabase.
NotesaboutDatabases
40
JavaScriptInterfacetoCollections
ThisisanintroductiontoArangoDB'sinterfaceforcollectionsandhowtohandlecollectionsfromtheJavaScriptshellarangosh.For
otherlanguagesseethecorrespondinglanguageAPI.
Themostimportantcallisthecalltocreateanewcollection.
AddressofaCollection
AllcollectionsinArangoDBhaveauniqueidentifierandauniquename.ArangoDBinternallyusesthecollection'suniqueidentifierto
lookupcollections.Thisidentifier,however,ismanagedbyArangoDBandtheuserhasnocontroloverit.Inordertoallowuserstouse
theirownnames,eachcollectionalsohasauniquenamewhichisspecifiedbytheuser.Toaccessacollectionfromtheuserperspective,
thecollectionnameshouldbeused,i.e.:
Collection
db._collection(collection-name)
Acollectioniscreatedbya"db._create"call.
Forexample:Assumethatthecollectionidentifieris7254820andthenameisdemo,thenthecollectioncanbeaccessedas:
db._collection("demo")
Ifnocollectionwithsuchanameexists,thennullisreturned.
Thereisashort-cutthatcanbeusedfornon-systemcollections:
Collectionname
db.collection-name
Thiscallwilleitherreturnthecollectionnameddb.collection-nameorcreateanewonewiththatnameandasetofdefaultproperties.
Note:Creatingacollectionontheflyusingdb.collection-nameisnotrecommendanddoesnotworkinarangosh.Tocreateanew
collection,pleaseuse
Create
db._create(collection-name)
Thiscallwillcreateanewcollectioncalledcollection-name.ThismethodisadatabasemethodandisdocumentedindetailatDatabase
M ethods
Synchronousreplication
StartinginArangoDB3.0,thedistributedversionofferssynchronousreplication,whichmeansthatthereistheoptiontoreplicateall
dataautomaticallywithintheArangoDBcluster.Thisisconfiguredforshardedcollectionsonapercollectionbasisbyspecifyinga
"replicationfactor"whenthecollectioniscreated.Areplicationfactorofkmeansthataltogetherkcopiesofeachshardarekeptinthe
clusteronkdifferentservers,andarekeptinsync.Thatis,everywriteoperationisautomaticallyreplicatedonallcopies.
Thisisorganisedusingaleader/followermodel.Atalltimes,oneoftheserversholdingreplicasforashardis"theleader"andallothers
are"followers",thisconfigurationisheldintheAgency(seeScalabilityfordetailsoftheArangoDBclusterarchitecture).Everywrite
operationissenttotheleaderbyoneofthecoordinators,andthenreplicatedtoallfollowersbeforetheoperationisreportedtohave
succeeded.Theleaderkeepsarecordofwhichfollowersarecurrentlyinsync.Incaseofnetworkproblemsorafailureofafollower,a
leadercanandwilldropafollowertemporarilyafter3seconds,suchthatservicecanresume.Induecourse,thefollowerwill
automaticallyresynchronizewiththeleadertorestoreresilience.
Collections
41
Ifaleaderfails,theclusterAgencyautomaticallyinitiatesafailoverroutineafteraround15seconds,promotingoneofthefollowersto
leader.Theotherfollowers(andtheformerleader,whenitcomesback),automaticallyresynchronizewiththenewleadertorestore
resilience.Usually,thiswholefailoverprocedurecanbehandledtransparentlyforthecoordinator,suchthattheusercodedoesnoteven
seeanerrormessage.
Obviously,thisfaulttolerancecomesatacostofincreasedlatency.Eachwriteoperationneedsanadditionalnetworkroundtripforthe
synchronousreplicationofthefollowers,butallreplicationoperationstoallfollowershappenconcurrently.Thisis,whythedefault
replicationfactoris1,whichmeansnoreplication.
Fordetailsonhowtoswitchonsynchronousreplicationforacollection,seethedatabasemethod db._create(collection-name)inthe
sectionaboutDatabaseM ethods.
Collections
42
CollectionMethods
Drop
dropsacollection collection.drop()
Dropsacollectionandallitsindexes.
Examples
arangosh>col=db.example;
[ArangoCollection15249,"example"(typedocument,statusloaded)]
arangosh>col.drop();
arangosh>col;
[ArangoCollection15249,"example"(typedocument,statusdeleted)]
Truncate
truncatesacollection collection.truncate()
Truncatesacollection,removingalldocumentsbutkeepingallitsindexes.
Examples
Truncatesacollection:
arangosh>col.save({"Hello":"World"});
arangosh>col.count();
arangosh>col.truncate();
showexecutionresults
Properties
getsorsetsthepropertiesofacollection collection.properties()Returnsanobjectcontainingallcollectionproperties.
waitForSync:Iftruecreatingadocumentwillonlyreturnafterthedatawassyncedtodisk.
journalSize:Thesizeofthejournalinbytes.
isVolatile:IftruethenthecollectiondatawillbekeptinmemoryonlyandArangoDBwillnotwriteorsyncthedatatodisk.
keyOptions(optional)additionaloptionsforkeygeneration.ThisisaJSONarraycontainingthefollowingattributes(note:someof
theattributesareoptional):
type:thetypeofthekeygeneratorusedforthecollection.
allowUserKeys:ifsettotrue,thenitisallowedtosupplyownkeyvaluesinthe_keyattributeofadocument.Ifsettofalse,
thenthekeygeneratorwillsolelyberesponsibleforgeneratingkeysandsupplyingownkeyvaluesinthe_keyattributeof
documentsisconsideredanerror.
increment:incrementvalueforautoincrementkeygenerator.Notusedforotherkeygeneratortypes.
offset:initialoffsetvalueforautoincrementkeygenerator.Notusedforotherkeygeneratortypes.
indexBuckets:numberofbucketsintowhichindexesusingahashtablearesplit.Thedefaultis16andthisnumberhastobeapower
of2andlessthanorequalto1024.Forverylargecollectionsoneshouldincreasethistoavoidlongpauseswhenthehashtablehas
tobeinitiallybuiltorresized,sincebucketsareresizedindividuallyandcanbeinitiallybuiltinparallel.Forexample,64mightbea
sensiblevalueforacollectionwith100000000documents.Currently,onlytheedgeindexrespectsthisvalue,butotherindextypes
mightfollowinfutureArangoDBversions.Changes(seebelow)areappliedwhenthecollectionisloadedthenexttime.Inacluster
setup,theresultwillalsocontainthefollowingattributes:
numberOfShards:thenumberofshardsofthecollection.
CollectionM ethods
43
shardKeys:containsthenamesofdocumentattributesthatareusedtodeterminethetargetshardfordocuments.
collection.properties(properties)Changesthecollectionproperties.propertiesmustbeaobjectwithoneormoreofthe
followingattribute(s):
indexBuckets:Seeabove,changesareonlyappliedwhenthecollectionisloadedthenexttime.Note:itisnotpossibletochangethe
journalsizeafterthejournalordatafilehasbeencreated.Changingthisparameterwillonlyeffectnewlycreatedjournals.Alsonote
thatyoucannotlowerthejournalsizetolessthensizeofthelargestdocumentalreadystoredinthecollection.Note:someother
collectionproperties,suchastype,isVolatile,orkeyOptionscannotbechangedoncethecollectioniscreated.
Examples
Readallproperties
arangosh>db.example.properties();
Changeaproperty
arangosh>db.example.properties({waitForSync:true});
Figures
returnsthefiguresofacollection collection.figures()
Returnsanobjectcontainingstatisticsaboutthecollection.Note:Retrievingthefigureswillalwaysloadthecollectionintomemory.
alive.count:Thenumberofcurrentlyactivedocumentsinalldatafilesandjournalsofthecollection.Documentsthatarecontainedin
thewrite-aheadlogonlyarenotreportedinthisfigure.
alive.size:Thetotalsizeinbytesusedbyallactivedocumentsofthecollection.Documentsthatarecontainedinthewrite-aheadlog
onlyarenotreportedinthisfigure.
dead.count:Thenumberofdeaddocuments.Thisincludesdocumentversionsthathavebeendeletedorreplacedbyanewerversion.
Documentsdeletedorreplacedthatarecontainedinthewrite-aheadlogonlyarenotreportedinthisfigure.
dead.size:Thetotalsizeinbytesusedbyalldeaddocuments.
dead.deletion:Thetotalnumberofdeletionmarkers.Deletionmarkersonlycontainedinthewrite-aheadlogarenotreportinginthis
figure.
datafiles.count:Thenumberofdatafiles.
datafiles.fileSize:Thetotalfilesizeofdatafiles(inbytes).
journals.count:Thenumberofjournalfiles.
journals.fileSize:Thetotalfilesizeofthejournalfiles(inbytes).
compactors.count:Thenumberofcompactorfiles.
compactors.fileSize:Thetotalfilesizeofthecompactorfiles(inbytes).
shapefiles.count:Thenumberofshapefiles.Thisvalueisdeprecatedandkeptforcompatibilityreasonsonly.Thevaluewillalways
be0sinceArangoDB2.0andhigher.
shapefiles.fileSize:Thetotalfilesizeoftheshapefiles.Thisvalueisdeprecatedandkeptforcompatibilityreasonsonly.Thevalue
willalwaysbe0inArangoDB2.0andhigher.
shapes.count:Thetotalnumberofshapesusedinthecollection.Thisincludesshapesthatarenotinuseanymore.Shapesthatare
containedinthewrite-aheadlogonlyarenotreportedinthisfigure.
shapes.size:Thetotalsizeofallshapes(inbytes).Thisincludesshapesthatarenotinuseanymore.Shapesthatarecontainedin
thewrite-aheadlogonlyarenotreportedinthisfigure.
attributes.count:Thetotalnumberofattributesusedinthecollection.Note:thevalueincludesdataofattributesthatarenotinuse
anymore.Attributesthatarecontainedinthewrite-aheadlogonlyarenotreportedinthisfigure.
attributes.size:Thetotalsizeoftheattributedata(inbytes).Note:thevalueincludesdataofattributesthatarenotinuseanymore.
Attributesthatarecontainedinthewrite-aheadlogonlyarenotreportedinthisfigure.
indexes.count:Thetotalnumberofindexesdefinedforthecollection,includingthepre-definedindexes(e.g.primaryindex).
CollectionM ethods
44
indexes.size:Thetotalmemoryallocatedforindexesinbytes.
maxTick:Thetickofthelastmarkerthatwasstoredinajournalofthecollection.Thismightbe0ifthecollectiondoesnotyethave
ajournal.
uncollectedLogfileEntries:Thenumberofmarkersinthewrite-aheadlogforthiscollectionthathavenotbeentransferredtojournals
ordatafiles.
documentReferences:ThenumberofreferencestodocumentsindatafilesthatJavaScriptcodecurrentlyholds.Thisinformationcan
beusedfordebuggingcompactionandunloadissues.
waitingFor:Anoptionalstringvaluethatcontainsinformationaboutwhichobjecttypeisattheheadofthecollection'scleanup
queue.Thisinformationcanbeusedfordebuggingcompactionandunloadissues.
compactionStatus.time:Thepointintimethecompactionforthecollectionwaslastexecuted.Thisinformationcanbeusedfor
debuggingcompactionissues.
compactionStatus.message:Theactionthatwasperformedwhenthecompactionwaslastrunforthecollection.Thisinformation
canbeusedfordebuggingcompactionissues.
Note:collectiondatathatarestoredinthewrite-aheadlogonlyarenotreportedintheresults.Whenthewrite-aheadlogiscollected,
documentsmightbeaddedtojournalsanddatafilesofthecollection,whichmaymodifythefiguresofthecollection.Alsonotethat
waitingForand compactionStatusmaybeemptywhencalledonacoordinatorinacluster.
Additionally,thefilesizesofcollectionandindexparameterJSONfilesarenotreported.Thesefilesshouldnormallyhaveasizeofafew
byteseach.PleasealsonotethatthefileSizevaluesarereportedinbytesandreflectthelogicalfilesizes.Somefilesystemsmayuse
optimisations(e.g.sparsefiles)sothattheactualphysicalfilesizeissomewhatdifferent.Directoriesandsub-directoriesmayalso
requirespaceinthefilesystem,butthisspaceisnotreportedinthefileSizeresults.
Thatmeansthatthefiguresreporteddonotreflecttheactualdiskusageofthecollectionwith100%accuracy.Theactualdiskusageofa
collectionisnormallyslightlyhigherthanthesumofthereportedfileSizevalues.StillthesumofthefileSizevaluescanstillbeusedasa
lowerboundapproximationofthediskusage.
Examples
arangosh>db.demo.figures()
Load
loadsacollection collection.load()
Loadsacollectionintomemory.
Examples
arangosh>col.load();
arangosh>col;
Reserve
collection.reserve(number)
Sendsaresizehinttotheindexesinthecollection.Theresizehintallowsindexestoreservespaceforadditionaldocuments(specifiedby
number)inonego.
Thereservehintcanbesentbeforeamassinsertionintothecollectionisstarted.Itallowsindexestoallocatetherequiredmemoryat
onceandavoidsre-allocationsandpossiblere-locations.
Notallindexesimplementthereservefunctionatthemoment.Theindexesthatdon'timplementitwillsimplyignoretherequest.returns
therevisionidofacollection
CollectionM ethods
45
Revision
returnstherevisionidofacollection collection.revision()
Returnstherevisionidofthecollection
Therevisionidisupdatedwhenthedocumentdataismodified,eitherbyinserting,deleting,updatingorreplacingdocumentsinit.
Therevisionidofacollectioncanbeusedbyclientstocheckwhetherdatainacollectionhaschangedorifitisstillunmodifiedsincea
previousfetchoftherevisionid.
Therevisionidreturnedisastringvalue.Clientsshouldtreatthisvalueasanopaquestring,andonlyuseitforequality/non-equality
comparisons.
Checksum
calculatesachecksumforthedatainacollection collection.checksum(withRevisions,withData)
Thechecksumoperationcalculatesanaggregatehashvalueforalldocumentkeyscontainedincollectioncollection.
IftheoptionalargumentwithRevisionsissettotrue,thentherevisionidsofthedocumentsarealsoincludedinthehashcalculation.
IftheoptionalargumentwithDataissettotrue,thenalluser-defineddocumentattributesarealsochecksummed.Includingthedocument
datainchecksummingwillmakethecalculationslower,butismoreaccurate.
ThechecksumcalculationalgorithmchangedinArangoDB3.0,sochecksumsfrom3.0andearlierversionsforthesamedatawilldiffer.
Note:thismethodisnotavailableinacluster.
Unload
unloadsacollection collection.unload()
Startsunloadingacollectionfrommemory.Notethatunloadingisdeferreduntilallqueryhavefinished.
Examples
arangosh>col.unload();
arangosh>col;
[ArangoCollection7661,"example"(typedocument,statusunloaded)]
Rename
renamesacollection collection.rename(new-name)
Renamesacollectionusingthenew-name.Thenew-namemustnotalreadybeusedforadifferentcollection.new-namemustalsobea
validcollectionname.Formoreinformationonvalidcollectionnamespleaserefertothenamingconventions.
Ifrenamingfailsforanyreason,anerroristhrown.Ifrenamingthecollectionsucceeds,thenthecollectionisalsorenamedinallgraph
definitionsinsidethe _graphscollectioninthecurrentdatabase.
Examples
arangosh>c=db.example;
arangosh>c.rename("better-example");
arangosh>c;
[ArangoCollection15450,"better-example"(typedocument,statusloaded)]
CollectionM ethods
46
Rotate
rotatesthecurrentjournalofacollection collection.rotate()
Rotatesthecurrentjournalofacollection.Thisoperationmakesthecurrentjournalofthecollectionaread-onlydatafilesoitmay
becomeacandidateforgarbagecollection.Ifthereiscurrentlynojournalavailableforthecollection,theoperationwillfailwithanerror.
CollectionM ethods
47
DatabaseMethods
Collection
returnsasinglecollectionornull db._collection(collection-name)
Returnsthecollectionwiththegivennameornullifnosuchcollectionexists.
db._collection(collection-identifier)
Returnsthecollectionwiththegivenidentifierornullifnosuchcollectionexists.Accessingcollectionsbyidentifierisdiscouragedfor
endusers.Endusersshouldaccesscollectionsusingthecollectionname.
Examples
Getacollectionbyname:
arangosh>db._collection("demo");
[ArangoCollection89,"demo"(typedocument,statusloaded)]
Getacollectionbyid:
arangosh>db._collection(123456);
[ArangoCollection123456,"demo"(typedocument,statusloaded)]
Unknowncollection:
arangosh>db._collection("unknown");
null
Create
createsanewdocumentoredgecollection db._create(collection-name)
Createsanewdocumentcollectionnamedcollection-name.Ifthecollectionnamealreadyexistsorifthenameformatisinvalid,anerroris
thrown.Formoreinformationonvalidcollectionnamespleaserefertothenamingconventions.
db._create(collection-name,properties)
propertiesmustbeanobjectwiththefollowingattributes:
waitForSync(optional,defaultfalse):Iftruecreatingadocumentwillonlyreturnafterthedatawassyncedtodisk.
journalSize(optional,defaultisaconfigurationparameter:Themaximalsizeofajournalordatafile.Notethatthisalsolimitsthe
maximalsizeofasingleobject.M ustbeatleast1M B.
isSystem(optional,defaultisfalse):Iftrue,createasystemcollection.Inthiscasecollection-nameshouldstartwithanunderscore.
Endusersshouldnormallycreatenon-systemcollectionsonly.APIimplementorsmayberequiredtocreatesystemcollectionsin
veryspecialoccasions,butnormallyaregularcollectionwilldo.
isVolatile(optional,defaultisfalse):Iftruethenthecollectiondataiskeptin-memoryonlyandnotmadepersistent.Unloadingthe
collectionwillcausethecollectiondatatobediscarded.Stoppingorre-startingtheserverwillalsocausefulllossofdatainthe
collection.Thecollectionitselfwillremainhowever(onlythedataisvolatile).Settingthisoptionwillmaketheresultingcollection
beslightlyfasterthanregularcollectionsbecauseArangoDBdoesnotenforceanysynchronizationtodiskanddoesnotcalculate
anyCRCchecksumsfordatafiles(astherearenodatafiles).
keyOptions(optional):additionaloptionsforkeygeneration.Ifspecified,thenkeyOptionsshouldbeaJSONarraycontainingthe
followingattributes(note:someofthemareoptional):
type:specifiesthetypeofthekeygenerator.Thecurrentlyavailablegeneratorsaretraditionalandautoincrement.
DatabaseM ethods
48
numberOfShards(optional,defaultis1):inacluster,thisvaluedeterminesthenumberofshardstocreateforthecollection.Ina
singleserversetup,thisoptionismeaningless.
shardKeys(optional,defaultis ["_key"]):inacluster,thisattributedetermineswhichdocumentattributesareusedtodetermine
thetargetshardfordocuments.Documentsaresenttoshardsbasedonthevaluestheyhaveintheirshardkeyattributes.Thevalues
ofallshardkeyattributesinadocumentarehashed,andthehashvalueisusedtodeterminethetargetshard.Notethatvaluesof
shardkeyattributescannotbechangedonceset.Thisoptionismeaninglessinasingleserversetup.
Whenchoosingtheshardkeys,onemustbeawareofthefollowingrulesandlimitations:Inashardedcollectionwithmorethanone
sharditisnotpossibletosetupauniqueconstraintonanattributethatisnottheoneandonlyshardkeygiveninshardKeys.This
isbecauseenforcingauniqueconstraintwouldotherwisemakeaglobalindexnecessaryorneedextensivecommunicationforevery
singlewriteoperation.Furthermore,if_keyisnottheoneandonlyshardkey,thenitisnotpossibletosetthe_keyattributewhen
insertingadocument,providedthecollectionhasmorethanoneshard.Again,thisisbecausethedatabasehastoenforcetheunique
constraintonthe_keyattributeandthiscanonlybedoneefficientlyifthisistheonlyshardkeybydelegatingtotheindividual
shards.
replicationFactor(optional,defaultis1):inacluster,thisattributedetermineshowmanycopiesofeachshardarekeptondifferent
DBServers.Thevalue1meansthatonlyonecopy(nosynchronousreplication)iskept.Avalueofkmeansthatk-1replicasare
kept.AnytwocopiesresideondifferentDBServers.Replicationbetweenthemissynchronous,thatis,everywriteoperationtothe
"leader"copywillbereplicatedtoall"follower"replicas,beforethewriteoperationisreportedsuccessful.
Ifaserverfails,thisisdetectedautomaticallyandoneoftheserversholdingcopiestakeover,usuallywithoutanerrorbeing
reported.
db._create(collection-name,properties,type)
Specifiestheoptionaltypeofthecollection,itcaneitherbedocumentoredge.Ondefaultitisdocument.Insteadofgivingatypeyoucan
alsousedb._createEdgeCollectionordb._createDocumentCollection.
Examples
Withdefaults:
arangosh>c=db._create("users");
arangosh>c.properties();
Withproperties:
arangosh>c=db._create("users",{waitForSync:true,
........>journalSize:1024*1204});
arangosh>c.properties();
Withakeygenerator:
arangosh>db._create("users",
........>{keyOptions:{type:"autoincrement",offset:10,increment:5}});
arangosh>db.users.save({name:"user1"});
Withaspecialkeyoption:
DatabaseM ethods
49
arangosh>db._create("users",{keyOptions:{allowUserKeys:false}});
arangosh>db.users.save({name:"user2",_key:"myuser"});
createsanewedgecollection db._createEdgeCollection(collection-name)
Createsanewedgecollectionnamedcollection-name.Ifthecollectionnamealreadyexistsanerroristhrown.Thedefaultvaluefor
waitForSyncisfalse.
db._createEdgeCollection(collection-name,properties)
propertiesmustbeanobjectwiththefollowingattributes:
waitForSync(optional,defaultfalse):Iftruecreatingadocumentwillonlyreturnafterthedatawassyncedtodisk.
journalSize(optional,defaultis"configurationparameter"):Themaximalsizeofajournalordatafile.Notethatthisalsolimitsthe
maximalsizeofasingleobjectandmustbeatleast1M B.
createsanewdocumentcollection db._createDocumentCollection(collection-name)
Createsanewdocumentcollectionnamedcollection-name.Ifthedocumentnamealreadyexistsanderroristhrown.
AllCollections
returnsallcollections db._collections()
Returnsallcollectionsofthegivendatabase.
Examples
arangosh>db._collections();
CollectionName
selectsacollectionfromthevocbase db.collection-name
Returnsthecollectionwiththegivencollection-name.Ifnosuchcollectionexists,createacollectionnamedcollection-namewiththe
defaultproperties.
Examples
arangosh>db.example;
Drop
dropsacollection db._drop(collection)
Dropsacollectionandallitsindexes.
db._drop(collection-identifier)
Dropsacollectionidentifiedbycollection-identifierandallitsindexes.Noerroristhrownifthereisnosuchcollection.
db._drop(collection-name)
Dropsacollectionnamedcollection-nameandallitsindexes.Noerroristhrownifthereisnosuchcollection.
Examples
Dropsacollection:
DatabaseM ethods
50
arangosh>db._drop(col);
arangosh>col;
Dropsacollectionidentifiedbyname:
arangosh>db._drop("example");
arangosh>col;
[ArangoCollection15201,"example"(typedocument,statusdeleted)]
Truncate
truncatesacollection db._truncate(collection)
Truncatesacollection,removingalldocumentsbutkeepingallitsindexes.
db._truncate(collection-identifier)
Truncatesacollectionidentifiedbycollection-identified.Noerroristhrownifthereisnosuchcollection.
db._truncate(collection-name)
Truncatesacollectionnamedcollection-name.Noerroristhrownifthereisnosuchcollection.
Examples
Truncatesacollection:
arangosh>db._truncate(col);
Truncatesacollectionidentifiedbyname:
arangosh>db._truncate("example");
DatabaseM ethods
51
Documents
ThisisanintroductiontoArangoDB'sinterfaceforworkingwithdocumentsfromtheJavaScriptshellarangoshorinJavaScriptcodein
theserver.ForotherlanguagesseethecorrespondinglanguageAPI.
BasicsandTerminology:sectiononthebasicapproach
CollectionM ethods:detailedAPIdescriptionforcollectionobjects
DatabaseM ethods:detailedAPIdescriptionfordatabaseobjects
Documents
52
DocumentsinArangoDBareJSONobjects.Theseobjectscanbenested(toanydepth)andmaycontainlists.Eachdocumenthasa
uniqueprimarykeywhichidentifiesitwithinitscollection.Furthermore,eachdocumentisuniquelyidentifiedbyitsdocumenthandle
acrossallcollectionsinthesamedatabase.Differentrevisionsofthesamedocument(identifiedbyitshandle)canbedistinguishedby
theirdocumentrevision.Anytransactiononlyeverseesasinglerevisionofadocument.Forexample:
{
"_id":"myusers/3456789",
"_key":"3456789",
"_rev":"14253647",
"firstName":"John",
"lastName":"Doe",
"address":{
"street":"RoadToNowhere1",
"city":"Gotham"
},
"hobbies":[
]
}
Alldocumentscontainspecialattributes:thedocumenthandleisstoredasastringin _id,thedocument'sprimarykeyin _keyand
thedocumentrevisionin _rev.Thevalueofthe _keyattributecanbespecifiedbytheuserwhencreatingadocument. _idand
_keyvaluesareimmutableoncethedocumenthasbeencreated.The _revvalueismaintainedbyArangoDBautomatically.
DocumentHandle
Adocumenthandleuniquelyidentifiesadocumentinthedatabase.Itisastringandconsistsofthecollection'snameandthedocument
key( _keyattribute)separatedby /.
DocumentKey
Adocumentkeyuniquelyidentifiesadocumentinthecollectionitisstoredin.Itcanandshouldbeusedbyclientswhenspecific
documentsarequeried.Thedocumentkeyisstoredinthe _keyattributeofeachdocument.Thekeyvaluesareautomaticallyindexed
byArangoDBinacollection'sprimaryindex.Thuslookingupadocumentbyitskeyisafastoperation.The_keyvalueofadocumentis
immutableoncethedocumenthasbeencreated.Bydefault,ArangoDBwillauto-generateadocumentkeyifno_keyattributeis
specified,andusetheuser-specified_keyotherwise.
Thisbehaviorcanbechangedonaper-collectionlevelbycreatingcollectionswiththe keyOptionsattribute.
Using keyOptionsitispossibletodisallowuser-specifiedkeyscompletely,ortoforceaspecificregimeforauto-generatingthe _key
values.
DocumentRevision
AsArangoDBsupportsM VCC(M ultipleVersionConcurrencyControl),documentscanexistinmorethanonerevision.Thedocument
revisionistheM VCCtokenusedtospecifyaparticularrevisionofadocument(identifiedbyits _id).Itisastringvaluecurrently
containinganintegernumberandisuniquewithinthelistofdocumentrevisionsforasingledocument.Documentrevisionscanbeusedto
conditionallyquery,update,replaceordeletedocumentsinthedatabase.Inordertofindaparticularrevisionofadocument,youneed
thedocumenthandleorkey,andthedocumentrevision.
ArangoDBuses64bitunsignedintegervaluestomaintaindocumentrevisionsinternally.Whenreturningdocumentrevisionstoclients,
ArangoDBwillputthemintoastringtoensuretherevisionisnotclippedbyclientsthatdonotsupportbigintegers.Clientsshouldtreat
therevisionreturnedbyArangoDBasanopaquestringwhentheystoreoruseitlocally.ThiswillallowArangoDBtochangetheformat
ofrevisionslaterifthisshouldberequired.Clientscanuserevisionstoperformsimpleequality/non-equalitycomparisons(e.g.tocheck
whetheradocumenthaschangedornot),buttheyshouldnotuserevisionidstoperformgreater/lessthancomparisonswiththemto
checkifadocumentrevisionisolderthanoneanother,evenifthismightworkforsomecases.
53
MultipleDocumentsinasingleCommand
BeginningwithArangoDB3.0thebasicdocumentAPIhasbeenextendedtohandlenotonlysingledocumentsbutmultipledocumentsin
asinglecommand.Thisiscrucialforperformance,inparticularintheclustersituation,inwhichasinglerequestcaninvolvemultiple
networkhopswithinthecluster.Anotheradvantageisthatitreducestheoverheadofindividualnetworkroundtripsbetweentheclient
andtheserver.ThegeneralideatoperformmultipledocumentoperationsinasinglecommandistouseJSONarraysofobjectsinthe
placeofasingledocument.Asaconsequence,documentkeys,handlesandrevisionsforpreconditionshavetobesuppliedembeddedin
theindividualdocumentsgiven.M ultipledocumentoperationsarerestrictedtoasingledocumentoredgecollection.SeetheAPI
descriptionsforcollectionobjectsfordetails.NotethattheAPIfordatabaseobjectsdonotoffertheseoperations.
54
CollectionMethods
All
collection.all()
Fetchesalldocumentsfromacollectionandreturnsacursor.YoucanusetoArray,next,orhasNexttoaccesstheresult.Theresultcanbe
limitedusingtheskipandlimitoperator.
Examples
UsetoArraytogetalldocumentsatonce:
arangosh>db.five.save({name:"one"});
arangosh>db.five.save({name:"two"});
arangosh>db.five.save({name:"three"});
arangosh>db.five.save({name:"four"});
arangosh>db.five.save({name:"five"});
arangosh>db.five.all().toArray();
Uselimittorestrictthedocuments:
arangosh>db.five.save({name:"one"});
arangosh>db.five.save({name:"two"});
arangosh>db.five.save({name:"three"});
arangosh>db.five.save({name:"four"});
arangosh>db.five.save({name:"five"});
arangosh>db.five.all().limit(2).toArray();
Querybyexample
collection.byExample(example)
Fetchesalldocumentsfromacollectionthatmatchthespecifiedexampleandreturnsacursor.
YoucanusetoArray,next,orhasNexttoaccesstheresult.Theresultcanbelimitedusingtheskipandlimitoperator.
Anattributenameoftheforma.bisinterpretedasattributepath,notasattribute.Ifyouuse
{"a":{"c":1}}
asexample,thenyouwillfindalldocuments,suchthattheattributeacontainsadocumentoftheform{c:1}.Forexamplethe
document
{"a":{"c":1},"b":1}
willmatch,butthedocument
{"a":{"c":1,"b":1}}
willnot.
However,ifyouuse
CollectionM ethods
55
{"a.c":1}
thenyouwillfindalldocuments,whichcontainasub-documentinathathasanattributecofvalue1.Boththefollowingdocuments
{"a":{"c":1},"b":1}
and
{"a":{"c":1,"b":1}}
willmatch.
collection.byExample(path1,value1,...)
Asalternativeyoucansupplyanarrayofpathsandvalues.
Examples
arangosh>db.users.save({name:"Gerhard"});
arangosh>db.users.save({name:"Helmut"});
arangosh>db.users.save({name:"Angela"});
arangosh>db.users.all().toArray();
arangosh>db.users.byExample({"_id":"users/20"}).toArray();
arangosh>db.users.byExample({"name":"Gerhard"}).toArray();
arangosh>db.users.byExample({"name":"Helmut","_id":"users/15"}).toArray();
Usenexttoloopoveralldocuments:
arangosh>db.users.save({name:"Gerhard"});
arangosh>db.users.save({name:"Helmut"});
arangosh>db.users.save({name:"Angela"});
arangosh>vara=db.users.byExample({"name":"Angela"});
arangosh>while(a.hasNext())print(a.next());
FirstExample
collection.firstExample(example)
Returnssomedocumentofacollectionthatmatchesthespecifiedexample.Ifnosuchdocumentexists,nullwillbereturned.Theexample
hastobespecifiedaspathsandvalues.SeebyExamplefordetails.
collection.firstExample(path1,value1,...)
Asalternativeyoucansupplyanarrayofpathsandvalues.
Examples
arangosh>db.users.firstExample("name","Angela");
Range
CollectionM ethods
56
collection.range(attribute,left,right)
Returnsalldocumentsfromacollectionsuchthattheattributeisgreaterorequalthanleftandstrictlylessthanright.
Anattributenameoftheforma.bisinterpretedasattributepath,notasattribute.
Note:therangesimplequeryfunctionisdeprecatedasofArangoDB2.6.Thefunctionmayberemovedinfutureversionsof
ArangoDB.ThepreferredwayforretrievingdocumentsfromacollectionwithinaspecificrangeistouseanAQLqueryasfollows:
FORdocIN@@collection
FILTERdoc.value>=@left&&doc.value<@right
LIMIT@skip,@limit
RETURNdoc
Examples
arangosh>db.old.ensureIndex({type:"skiplist",fields:["age"]});
arangosh>db.old.save({age:15});
arangosh>db.old.range("age",10,30).toArray();
Closedrange
collection.closedRange(attribute,left,right)
Returnsalldocumentsofacollectionsuchthattheattributeisgreaterorequalthanleftandlessorequalthanright.
Anattributenameoftheforma.bisinterpretedasattributepath,notasattribute.
Note:theclosedRangesimplequeryfunctionisdeprecatedasofArangoDB2.6.Thefunctionmayberemovedinfutureversionsof
ArangoDB.ThepreferredwayforretrievingdocumentsfromacollectionwithinaspecificrangeistouseanAQLqueryasfollows:
FILTERdoc.value>=@left&&doc.value<=@right
LIMIT@skip,@limit
RETURNdoc
Examples
arangosh>db.old.ensureIndex({type:"skiplist",fields:["age"]});
arangosh>db.old.closedRange("age",10,30).toArray();
Any
collection.any()
Returnsarandomdocumentfromthecollectionornullifnoneexists.
CollectionM ethods
57
Count
collection.count()
Returnsthenumberoflivingdocumentsinthecollection.
Examples
arangosh>db.users.count();
0
toArray
collection.toArray()
Convertsthecollectionintoanarrayofdocuments.Neverusethiscallinaproductionenvironmentasitwillbasicallycreateacopyof
yourcollectioninRAM whichwilluseresourcesdependingonthenumberandsizeofthedocumentsinyourcollecion.
Document
collection.document(object)
Thedocumentmethodfindsadocumentgivenanobjectobjectcontainingthe_idor_keyattribute.Themethodreturnsthedocumentifit
canbefound.Ifbothattributesaregiven,the_idtakesprecedence,itisanerror,ifthecollectionpartofthe_iddoesnotmatchthe
collection.
Anerroristhrownif_revisspecifiedbutthedocumentfoundhasadifferentrevisionalready.Anerrorisalsothrownifnodocument
existswiththegiven_idor_keyvalue.
Pleasenotethatifthemethodisexecutedonthearangodserver(e.g.frominsideaFoxxapplication),animmutabledocumentobjectwill
bereturnedforperformancereasons.Itisnotpossibletochangeattributesofthisimmutableobject.Toupdateorpatchthereturned
document,itneedstobecloned/copiedintoaregularJavaScriptobjectfirst.Thisisnotnecessaryifthedocumentmethodiscalledfrom
outofarangoshorfromanyotherclient.
collection.document(document-handle)
Asbefore.Insteadofobjectadocument-handlecanbepassedasfirstargument.Norevisioncanbespecifiedinthiscase.
collection.document(document-key)
Asbefore.Insteadofobjectadocument-keycanbepassedasfirstargument.
collection.document(array)
Thisvariantallowstoperformtheoperationonawholearrayofarguments.Thebehaviorisexactlyasifdocumentwouldhavebeen
calledonallmembersofthearrayandallresultsarereturnedinanarray.Ifanerroroccurswithanyofthedocuments,theoperation
stopsimmediatelyreturningonlyanerrorobject.
Examples
Returnsthedocumentforadocument-handle:
arangosh>db.example.document("example/2873916");
Returnsthedocumentforadocument-key:
arangosh>db.example.document("2873916");
Returnsthedocumentforanobject:
arangosh>db.example.document({_id:"example/2873916"});
CollectionM ethods
58
Returnsthedocumentforanarrayoftwokeys:
arangosh>db.example.document(["2873916","2873917"]);
Anerrorisraisedifthedocumentisunknown:
[ArangoError1202:documentnotfound]
Anerrorisraisedifthehandleisinvalid:
arangosh>db.example.document("");
[ArangoError1205:illegaldocumenthandle]
Changesin3.0from2.8:
documentcannowquerymultipledocumentswithonecall.
Exists
checkswhetheradocumentexists collection.exists(object)
Theexistsmethoddetermineswhetheradocumentexistsgivenanobject objectcontainingthe_idor_keyattribute.Ifbothattributes
aregiven,the_idtakesprecedence,itisanerror,ifthecollectionpartofthe_iddoesnotmatchthecollection.
Anerroristhrownif_revisspecifiedbutthedocumentfoundhasadifferentrevisionalready.
Insteadofreturningthefounddocumentoranerror,thismethodwillonlyreturnanobjectwiththeattributes_id,_keyand_rev,orfalse
ifnodocumentwiththegiven_idor_keyexists.Itcanthusbeusedforeasyexistencechecks.
Thismethodwillthrowanerrorifusedimproperly,e.g.whencalledwithanon-documenthandle,anon-document,orwhenacrosscollectionrequestisperformed.
collection.exists(document-handle)
Asbefore.Insteadofobjectadocument-handlecanbepassedasfirstargument.
collection.exists(document-key)
Asbefore.Insteadofobjectadocument-keycanbepassedasfirstargument.
collection.exists(array)
Thisvariantallowstoperformtheoperationonawholearrayofarguments.Thebehaviorisexactlyasifexistswouldhavebeencalledon
allmembersofthearrayandallresultsarereturnedinanarray.Ifanerroroccurswithanyofthedocuments,theoperationstops
immediatelyreturningonlyanerrorobject.
Inthecaseofarevisionmismatchexistsnowthrowsanerrorinsteadofsimplyreturningfalse.Thisistomakeitpossibletotellthe
differencebetweenarevisionmismatchandanon-existingdocument.
existscannowquerymultipledocumentswithonecall.
LookupByKeys
collection.documents(keys)
CollectionM ethods
59
Looksupthedocumentsinthespecifiedcollectionusingthearrayofkeysprovided.Alldocumentsforwhichamatchingkeywas
specifiedinthekeysarrayandthatexistinthecollectionwillbereturned.Keysforwhichnodocumentcanbefoundintheunderlying
collectionareignored,andnoexceptionwillbethrownforthem.
Thismethodisdeprecatedinfavourofthearrayvariantofdocument.
Examples
arangosh>keys=[];
arangosh>for(vari=0;i<10;++i){
........>db.example.insert({_key:"test"+i,value:i});
........>keys.push("test"+i);
........>}
arangosh>db.example.documents(keys);
Insert
collection.insert(data)
Createsanewdocumentinthecollectionfromthegivendata.Thedatamustbeanobject.Theattributes_idand_revareignoredandare
automaticallygenerated.Auniquevaluefortheattribute_keywillbeautomaticallygeneratedifnotspecified.Ifspecified,theremustnot
beadocumentwiththegiven_keyinthecollection.
Themethodreturnsadocumentwiththeattributes_id,_keyand_rev.Theattribute_idcontainsthedocumenthandleofthenewly
createddocument,theattribute_keythedocumentkeyandtheattribute_revcontainsthedocumentrevision.
collection.insert(data,options)
Createsanewdocumentinthecollectionfromthegivendataasabove.Theoptionaloptionsparametermustbeanobjectandcanbeused
tospecifythefollowingoptions:
waitForSync:OnecanforcesynchronizationofthedocumentcreationoperationtodiskevenincasethatthewaitForSyncflagis
beendisabledfortheentirecollection.Thus,thewaitForSyncoptioncanbeusedtoforcesynchronizationofjustspecific
operations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthe
collection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationfor
collectionsthathaveadefaultwaitForSyncvalueoftrue.
silent:Ifthisflagissettotrue,themethoddoesnotreturnanyoutput.
returnNew:Ifthisflagissettotrue,thecompletenewdocumentisreturnedintheoutputundertheattributenew.
Note:sinceArangoDB2.2,insertisanaliasforsave.
collection.insert(array)
collection.insert(array,options)
Thesetwovariantsallowtoperformtheoperationonawholearrayofarguments.Thebehaviorisexactlyasifinsertwouldhavebeen
stopsimmediatelyreturningonlyanerrorobject.Theoptionsbehaveexactlyasbefore.
TheoptionssilentandreturnNewarenew.Themethodcannowinsertmultipledocumentswithonecall.
Examples
arangosh>db.example.insert({Hello:"World"});
arangosh>db.example.insert({Hello:"World"},{waitForSync:true});
CollectionM ethods
60
arangosh>db.example.insert([{Hello:"World"},{Hello:"there"}])
arangosh>db.example.insert([{Hello:"World"},{}],{waitForSync:true});
Replace
collection.replace(selector,data)
Replacesanexistingdocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idor_keyattribute.Theremustbea
documentwiththat_idor_keyinthecurrentcollection.Thisdocumentisthenreplacedwiththedatagivenassecondargument.Any
attribute_id,_keyor_revindataisignored.
Themethodreturnsadocumentwiththeattributes_id,_key,_revand_oldRev.Theattribute_idcontainsthedocumenthandleofthe
updateddocument,theattribute_revcontainsthedocumentrevisionoftheupdateddocument,theattribute_oldRevcontainsthe
revisionoftheold(nowreplaced)document.
Iftheselectorcontainsa_revattribute,themethodfirstchecksthatthespecifiedrevisionisthecurrentrevisionofthatdocument.Ifnot,
thereisaconflict,andanerroristhrown.
collection.replace(selector,data,options)
Asbefore,butoptionsmustbeanobjectthatcancontainthefollowingbooleanattributes:
overwrite:Ifthisflagissettotrue,a_revattributeintheselectorisignored.
returnOld:Ifthisflagissettotrue,thecompletepreviousrevisionofthedocumentisreturnedintheoutputundertheattributeold.
silent:Ifthisflagissettotrue,nooutputisreturned.
collection.replace(document-handle,data)
collection.replace(document-handle,data,options)
Asbefore.Insteadofselectoradocument-handlecanbepassedasfirstargument.Norevisionpreconditionistested.
collection.replace(document-key,data)
collection.replace(document-key,data,options)
Asbefore.Insteadofselectoradocument-keycanbepassedasfirstargument.Norevisionpreconditionistested.
collection.replace(selectorarray,dataarray)
collection.replace(selectorarray,dataarray,options)
Thesetwovariantsallowtoperformtheoperationonawholearrayofselector/datapairs.Thetwoarraysgivenasselectorarrayand
dataarraymusthavethesamelength.Thebehaviorisexactlyasifreplacewouldhavebeencalledonallrespectivemembersofthetwo
arraysandallresultsarereturnedinanarray.Ifanerroroccurswithanyofthedocuments,theoperationstopsimmediatelyreturning
onlyanerrorobject.Theoptionsbehaveexactlyasbefore.
Examples
Createandupdateadocument:
arangosh>a1=db.example.insert({a:1});
arangosh>a2=db.example.replace(a1,{a:2});
Useadocumenthandle:
CollectionM ethods
61
arangosh>a2=db.example.replace("example/3903044",{a:2});
Theoptionssilent,returnNewandreturnOldarenew.Themethodcannowreplacemultipledocumentswithonecall.
Update
collection.update(selector,data)
Updatesanexistingdocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idor_keyattribute.Theremustbea
documentwiththat_idor_keyinthecurrentcollection.Thisdocumentisthenpatchedwiththedatagivenassecondargument.Any
attribute_id,_keyor_revindataisignored.
revisionoftheold(nowupdated)document.
collection.update(selector,data,options)
keepNull:TheoptionalkeepNullparametercanbeusedtomodifythebehaviorwhenhandlingnullvalues.Normally,nullvaluesare
storedinthedatabase.BysettingthekeepNullparametertofalse,thisbehaviorcanbechangedsothatallattributesindatawithnull
valueswillberemovedfromthetargetdocument.
mergeObjects:Controlswhetherobjects(notarrays)willbemergedifpresentinboththeexistingandthepatchdocument.Ifsetto
false,thevalueinthepatchdocumentwilloverwritetheexistingdocument'svalue.Ifsettotrue,objectswillbemerged.Thedefault
istrue.
collection.update(document-handle,data)
collection.update(document-handle,data,options)
collection.update(document-key,data)
collection.update(document-key,data,options)
Asbefore.Insteadofselectoradocument-keycanbepassedasfirstargument.Norevisionpreconditionistested.
collection.update(selectorarray,dataarray)
collection.update(selectorarray,dataarray,options)
Thesetwovariantsallowtoperformtheoperationonawholearrayofselector/datapairs.Thetwoarraysgivenasselectorarrayand
dataarraymusthavethesamelength.Thebehaviorisexactlyasifupdatewouldhavebeencalledonallrespectivemembersofthetwo
arraysandallresultsarereturnedinanarray.Ifanerroroccurswithanyofthedocuments,theoperationstopsimmediatelyreturning
onlyanerrorobject.Theoptionsbehaveexactlyasbefore.
CollectionM ethods
62
Examples
arangosh>a1=db.example.insert({"a":1});
arangosh>a2=db.example.update(a1,{"b":2,"c":3});
arangosh>a3=db.example.update(a1,{"d":4});
arangosh>a4=db.example.update(a2,{"e":5,"f":6});
arangosh>db.example.document(a4);
arangosh>a5=db.example.update(a4,{"a":1,c:9,e:42});
Useadocumenthandle:
arangosh>a1=db.example.insert({"a":1});
arangosh>a2=db.example.update("example/18612115",{"x":1,"y":2});
UsethekeepNullparametertoremoveattributeswithnullvalues:
arangosh>db.example.insert({"a":1});
arangosh>db.example.update("example/19988371",
........>{"b":null,"c":null,"d":3});
arangosh>db.example.update("example/19988371",{"a":null},false,false);
arangosh>db.example.update("example/19988371",
........>{"b":null,"c":null,"d":null},false,false);
Patchingarrayvalues:
arangosh>db.example.insert({"a":{"one":1,"two":2,"three":3},
........>"b":{}});
arangosh>db.example.update("example/20774803",{"a":{"four":4},
........>"b":{"b1":1}});
arangosh>db.example.update("example/20774803",{"a":{"one":null},
........>"b":null},
........>false,false);
Theoptionssilent,returnNewandreturnOldarenew.Themethodcannowupdatemultipledocumentswithonecall.
Remove
collection.remove(selector)
Removesadocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idor_keyattribute.Theremustbeadocument
withthat_idor_keyinthecurrentcollection.Thisdocumentisthenremoved.
CollectionM ethods
63
Themethodreturnsadocumentwiththeattributes_id,_keyand_rev.Theattribute_idcontainsthedocumenthandleoftheremoved
document,theattribute_revcontainsthedocumentrevisionoftheremoveddocument.
collection.remove(selector,options)
collection.remove(document-handle)
collection.remove(document-handle,options)
Asbefore.Insteadofselectoradocument-handlecanbepassedasfirstargument.Norevisioncheckisperformed.
collection.remove(document-key)
collection.remove(document-handle,options)
collection.remove(selectorarray)
collection.remove(selectorarray,options)
Thesetwovariantsallowtoperformtheoperationonawholearrayofselectors.Thebehaviorisexactlyasifremovewouldhavebeen
stopsimmediatelyreturningonlyanerrorobject.Theoptionsbehaveexactlyasbefore.
Examples
Removeadocument:
arangosh>db.example.remove(a1);
Removeadocumentwithaconflict:
arangosh>db.example.remove(a1);
arangosh>db.example.remove(a1,true);
Themethodnowreturnsnotonlytruebutinformationabouttheremoveddocument(s).TheoptionssilentandreturnOldarenew.The
methodcannowremovemultipledocumentswithonecall.
RemoveByKeys
CollectionM ethods
64
collection.removeByKeys(keys)
Looksupthedocumentsinthespecifiedcollectionusingthearrayofkeysprovided,andremovesalldocumentsfromthecollection
whosekeysarecontainedinthekeysarray.Keysforwhichnodocumentcanbefoundintheunderlyingcollectionareignored,andno
exceptionwillbethrownforthem.
Themethodwillreturnanobjectcontainingthenumberofremoveddocumentsintheremovedsub-attribute,andthenumberofnotremoved/ignoreddocumentsintheignoredsub-attribute.
Thismethodisdeprecatedinfavourofthearrayvariantofremove.
Examples
arangosh>keys=[];
arangosh>for(vari=0;i<10;++i){
........>db.example.insert({_key:"test"+i,value:i});
........>keys.push("test"+i);
........>}
arangosh>db.example.removeByKeys(keys);
RemoveByExample
collection.removeByExample(example)
Removesalldocumentsmatchinganexample.
collection.removeByExample(document,waitForSync)
TheoptionalwaitForSyncparametercanbeusedtoforcesynchronizationofthedocumentdeletionoperationtodiskevenincasethat
thewaitForSyncflaghadbeendisabledfortheentirecollection.Thus,thewaitForSyncparametercanbeusedtoforcesynchronizationof
justspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,
thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationfor
collection.removeByExample(document,waitForSync,limit)
Theoptionallimitparametercanbeusedtorestrictthenumberofremovalstothespecifiedvalue.Iflimitisspecifiedbutlessthanthe
numberofdocumentsinthecollection,itisundefinedwhichdocumentsareremoved.
Examples
arangosh>db.example.removeByExample({Hello:"world"});
1
ReplaceByExample
collection.replaceByExample(example,newValue)
Replacesalldocumentsmatchinganexamplewithanewdocumentbody.Theentiredocumentbodyofeachdocumentmatchingthe
examplewillbereplacedwithnewValue.Thedocumentmeta-attributes_id,_keyand_revwillnotbereplaced.
collection.replaceByExample(document,newValue,waitForSync)
TheoptionalwaitForSyncparametercanbeusedtoforcesynchronizationofthedocumentreplacementoperationtodiskevenincase
thatthewaitForSyncflaghadbeendisabledfortheentirecollection.Thus,thewaitForSyncparametercanbeusedtoforce
synchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnot
specifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedto
disablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.
collection.replaceByExample(document,newValue,waitForSync,limit)
CollectionM ethods
65
Theoptionallimitparametercanbeusedtorestrictthenumberofreplacementstothespecifiedvalue.Iflimitisspecifiedbutlessthan
thenumberofdocumentsinthecollection,itisundefinedwhichdocumentsarereplaced.
Examples
arangosh>db.example.save({Hello:"world"});
arangosh>db.example.replaceByExample({Hello:"world"},{Hello:"mars"},false,5);
UpdateByExample
collection.updateByExample(example,newValue)
Partiallyupdatesalldocumentsmatchinganexamplewithanewdocumentbody.Specificattributesinthedocumentbodyofeach
documentmatchingtheexamplewillbeupdatedwiththevaluesfromnewValue.Thedocumentmeta-attributes_id,_keyand_revcannot
beupdated.
Partialupdatecouldalsobeusedtoappendnewfields,iftherewerenooldfieldwithsamename.
collection.updateByExample(document,newValue,keepNull,waitForSync)
TheoptionalkeepNullparametercanbeusedtomodifythebehaviorwhenhandlingnullvalues.Normally,nullvaluesarestoredinthe
database.BysettingthekeepNullparametertofalse,thisbehaviorcanbechangedsothatallattributesindatawithnullvalueswillbe
removedfromthetargetdocument.
TheoptionalwaitForSyncparametercanbeusedtoforcesynchronizationofthedocumentreplacementoperationtodiskevenincase
thatthewaitForSyncflaghadbeendisabledfortheentirecollection.Thus,thewaitForSyncparametercanbeusedtoforce
synchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnot
specifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedto
disablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.
collection.updateByExample(document,newValue,keepNull,waitForSync,limit)
Theoptionallimitparametercanbeusedtorestrictthenumberofupdatestothespecifiedvalue.Iflimitisspecifiedbutlessthanthe
numberofdocumentsinthecollection,itisundefinedwhichdocumentsareupdated.
collection.updateByExample(document,newValue,options)
Usingthisvariant,theoptionsfortheoperationcanbepassedusinganobjectwiththefollowingsub-attributes:
keepNull
waitForSync
limit
mergeObjects
Examples
arangosh>db.example.save({Hello:"world",foo:"bar"});
arangosh>db.example.updateByExample({Hello:"world"},{Hello:"foo",World:"bar"},false
arangosh>db.example.byExample({Hello:"foo"}).toArray()
Collectiontype
collection.type()
Returnsthetypeofacollection.Possiblevaluesare:
2:documentcollection
3:edgecollection
CollectionM ethods
66
GettheVersionofArangoDB
db._version()
Returnstheserverversionstring.Notethatthisisnottheversionofthedatabase.
Examples
arangosh>require("@arangodb").db._version();
3.0.10
Edges
Edgesarenormaldocumentsthatalwayscontaina _fromanda _toattribute.Therefore,youcanusethedocumentmethodsto
operateonedges.Thefollowingmethods,however,arespecifictoedges.
edge-collection.edges(vertex)
Theedgesoperatorfindsalledgesstartingfrom(outbound)orendingin(inbound)vertex.
edge-collection.edges(vertices)
Theedgesoperatorfindsalledgesstartingfrom(outbound)orendingin(inbound)adocumentfromvertices,whichmustalistof
documentsordocumenthandles.
arangosh>db._create("vertex");
arangosh>db._createEdgeCollection("relation");
arangosh>myGraph.v1=db.vertex.insert({name:"vertex1"});
arangosh>myGraph.e1=db.relation.insert(myGraph.v1,myGraph.v2,
........>{label:"knows"});
arangosh>db._document(myGraph.e1);
arangosh>db.relation.edges(myGraph.e1._id);
edge-collection.inEdges(vertex)
Theedgesoperatorfindsalledgesendingin(inbound)vertex.
edge-collection.inEdges(vertices)
Theedgesoperatorfindsalledgesendingin(inbound)adocumentfromvertices,whichmustalistofdocumentsordocumenthandles.
Examples
........>{label:"knows"});
arangosh>db.relation.inEdges(myGraph.v1._id);
arangosh>db.relation.inEdges(myGraph.v2._id);
edge-collection.outEdges(vertex)
Theedgesoperatorfindsalledgesstartingfrom(outbound)vertices.
edge-collection.outEdges(vertices)
CollectionM ethods
67
Theedgesoperatorfindsalledgesstartingfrom(outbound)adocumentfromvertices,whichmustalistofdocumentsordocument
handles.
Examples
........>{label:"knows"});
arangosh>db.relation.outEdges(myGraph.v1._id);
arangosh>db.relation.outEdges(myGraph.v2._id);
Misc
collection.iterate(iterator,options)
Iteratesoversomeelementsofthecollectionandapplythefunctioniteratortotheelements.Thefunctionwillbecalledwiththe
documentasfirstargumentandthecurrentnumber(startingwith0)assecondargument.
optionsmustbeanobjectwiththefollowingattributes:
limit(optional,defaultnone):useatmostlimitdocuments.
probability(optional,defaultall):anumberbetween0and1.Documentsarechosenwiththisprobability.
Examples
arangosh>for(i=-90;i<=90;i+=10){
........>for(j=-180;j<=180;j+=10){
........>db.example.save({name:"Name/"+i+"/"+j,
........>home:[i,j],
........>work:[-i,-j]});
........>}
........>}
........>
arangosh>db.example.ensureIndex({type:"geo",fields:["home"]});
arangosh>items=db.example.getIndexes().map(function(x){returnx.id;});
........>db.example.index(items[1]);
CollectionM ethods
68
DatabaseMethods
Document
db._document(object)
The_documentmethodfindsadocumentgivenanobjectobjectcontainingthe_idattribute.Themethodreturnsthedocumentifitcanbe
found.
Anerroristhrownif_revisspecifiedbutthedocumentfoundhasadifferentrevisionalready.Anerrorisalsothrownifnodocument
existswiththegiven_id.
Pleasenotethatifthemethodisexecutedonthearangodserver(e.g.frominsideaFoxxapplication),animmutabledocumentobjectwill
bereturnedforperformancereasons.Itisnotpossibletochangeattributesofthisimmutableobject.Toupdateorpatchthereturned
document,itneedstobecloned/copiedintoaregularJavaScriptobjectfirst.Thisisnotnecessaryifthe_documentmethodiscalledfrom
outofarangoshorfromanyotherclient.
db._document(document-handle)
Asbefore.Insteadofobjectadocument-handlecanbepassedasfirstargument.Norevisioncanbespecifiedinthiscase.
Examples
Returnsthedocument:
arangosh>db._document("example/12345");
Exists
db._exists(object)
The_existsmethoddetermineswhetheradocumentexistsgivenanobject objectcontainingthe_idattribute.
Anerroristhrownif_revisspecifiedbutthedocumentfoundhasadifferentrevisionalready.
Insteadofreturningthefounddocumentoranerror,thismethodwillonlyreturnanobjectwiththeattributes_id,_keyand_rev,orfalse
ifnodocumentwiththegiven_idor_keyexists.Itcanthusbeusedforeasyexistencechecks.
Thismethodwillthrowanerrorifusedimproperly,e.g.whencalledwithanon-documenthandle,anon-document,orwhenacrosscollectionrequestisperformed.
db._exists(document-handle)
Asbefore.Insteadofobjectadocument-handlecanbepassedasfirstargument.
Inthecaseofarevisionmismatch_existsnowthrowsanerrorinsteadofsimplyreturningfalse.Thisistomakeitpossibletotellthe
differencebetweenarevisionmismatchandanon-existingdocument.
Replace
db._replace(selector,data)
Replacesanexistingdocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idattribute.Theremustbeadocument
withthat_idinthecurrentdatabase.Thisdocumentisthenreplacedwiththedatagivenassecondargument.Anyattribute_id,_keyor
_revindataisignored.
revisionoftheold(nowreplaced)document.
DatabaseM ethods
69
collection.replace(selector,data,options)
db._replace(document-handle,data)
db._replace(document-handle,data,options)
Examples
Createandreplaceadocument:
arangosh>a2=db._replace(a1,{a:2});
Theoptionssilent,returnNewandreturnOldarenew.
Update
db._update(selector,data)
Updatesanexistingdocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idattribute.Theremustbeadocument
withthat_idinthecurrentdatabase.Thisdocumentisthenpatchedwiththedatagivenassecondargument.Anyattribute_id,_keyor
_revindataisignored.
revisionoftheold(nowupdated)document.
db._update(selector,data,options)
DatabaseM ethods
70
keepNull:TheoptionalkeepNullparametercanbeusedtomodifythebehaviorwhenhandlingnullvalues.Normally,nullvaluesare
storedinthedatabase.BysettingthekeepNullparametertofalse,thisbehaviorcanbechangedsothatallattributesindatawithnull
valueswillberemovedfromthetargetdocument.
mergeObjects:Controlswhetherobjects(notarrays)willbemergedifpresentinboththeexistingandthepatchdocument.Ifsetto
false,thevalueinthepatchdocumentwilloverwritetheexistingdocument'svalue.Ifsettotrue,objectswillbemerged.Thedefault
istrue.
db._update(document-handle,data)
db._update(document-handle,data,options)
Examples
arangosh>a2=db._update(a1,{b:2});
arangosh>a3=db._update(a1,{c:3});
Theoptionssilent,returnNewandreturnOldarenew.
Remove
db._remove(selector)
Removesadocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idattribute.Theremustbeadocumentwiththat
_idinthecurrentdatabase.Thisdocumentisthenremoved.
Themethodreturnsadocumentwiththeattributes_id,_keyand_rev.Theattribute_idcontainsthedocumenthandleoftheremoved
document,theattribute_revcontainsthedocumentrevisionoftheremovedeocument.
db._remove(selector,options)
db._remove(document-handle)
db._remove(document-handle,options)
Examples
Removeadocument:
DatabaseM ethods
71
arangosh>db._remove(a1);
arangosh>db._remove(a1,{overwrite:true});
Removethedocumentintherevision a1withaconflict:
arangosh>db._remove(a1,{overwrite:true});
arangosh>db._document(a1);
Removeadocumentusingnewsignature:
arangosh>db.example.insert({_key:"11265325374",a:1});
arangosh>db.example.remove("example/11265325374",
........>{overwrite:true,waitForSync:false})
Themethodnowreturnsnotonlytruebutinformationabouttheremoveddocument(s).TheoptionssilentandreturnOldarenew.
DatabaseM ethods
72
Graphs,vertices&edgesaredefinedintheGraphschapterindetails.
Relatedblogposts:
Graphsindatamodeling-istheemperornaked?
IndexFreeAdjacencyorHybridIndexesforGraphDatabases
73
NamingConventionsinArangoDB
Thefollowingnamingconventionsshouldbefollowedbyuserswhencreatingdatabases,collectionsanddocumentsinArangoDB.
NamingConventions
74
DatabaseNames
ArangoDBwillalwaysstartupwithadefaultdatabase,named_system.UserscancreateadditionaldatabasesinArangoDB,providedthe
databasenamesconformtothefollowingconstraints:
Databasenamesmustonlyconsistofthelettersatoz(bothloweranduppercaseallowed),thenumbers0to9,andtheunderscore
(_)ordash(-)symbolsThisalsomeansthatanynon-ASCIIdatabasenamesarenotallowed
Databasenamesmustalwaysstartwithaletter.Databasenamesstartingwithanunderscoreareconsideredtobesystemdatabases,
andusersshouldnotcreateordeletethose
Themaximumallowedlengthofadatabasenameis64bytes
Databasenamesarecase-sensitive
DatabaseNames
75
CollectionNames
Userscanpicknamesfortheircollectionsasdesired,providedthefollowingnamingconstraintsarenotviolated:
Collectionnamesmustonlyconsistofthelettersatoz(bothinloweranduppercase),thenumbers0to9,andtheunderscore(_)
ordash(-)symbols.Thisalsomeansthatanynon-ASCIIcollectionnamesarenotallowed
User-definedcollectionnamesmustalwaysstartwithaletter.Systemcollectionnamesmuststartwithanunderscore.Allcollection
namesstartingwithanunderscoreareconsideredtobesystemcollectionsthatareforArangoDB'sinternaluseonly.System
collectionnamesshouldnotbeusedbyendusersfortheirowncollections
Themaximumallowedlengthofacollectionnameis64bytes
Collectionnamesarecase-sensitive
CollectionNames
76
DocumentKeys
Userscandefinetheirownkeysfordocumentstheysave.Thedocumentkeywillbesavedalongwithadocumentinthe_keyattribute.
Userscanpickkeyvaluesasrequired,providedthatthevaluesconformtothefollowingrestrictions:
Thekeymustbeastringvalue.Numerickeysarenotallowed,butanynumericvaluecanbeputintoastringandcanthenbeusedas
documentkey.
Thekeymustbeatleast1byteandatmost254byteslong.Emptykeysaredisallowedwhenspecified(thoughitmaybevalidto
completelyomitthe_keyattributefromadocument)
Itmustconsistofthelettersa-z(loweroruppercase),thedigits0-9oranyofthefollowingpunctuationcharacters: _ - : .
@ ( ) + , = ; $ ! * ' %
Anyothercharacters,especiallymulti-byteUTF-8sequences,whitespaceorpunctuationcharacterscannotbeusedinsidekey
values
Thekeymustbeuniquewithinthecollectionitisused
Keysarecase-sensitive,i.e.myKeyandMyKEYareconsideredtobedifferentkeys.
Specifyingadocumentkeyisoptionalwhencreatingnewdocuments.Ifnodocumentkeyisspecifiedbytheuser,ArangoDBwillcreate
thedocumentkeyitselfaseachdocumentisrequiredtohaveakey.
Therearenoguaranteesabouttheformatandpatternofauto-generateddocumentkeysotherthantheaboverestrictions.Clientsshould
thereforetreatauto-generateddocumentkeysasopaquevaluesandnotrelyontheirformat.
DocumentKeys
77
AttributeNames
Userscanpickattributenamesfordocumentattributesasdesired,providedthefollowingattributenamingconstraintsarenotviolated:
AttributenamesstartingwithanunderscoreareconsideredtobesystemattributesforArangoDB'sinternaluse.Suchattribute
namesarealreadyusedbyArangoDBforspecialpurposes:
_idisusedtocontainadocument'shandle
_keyisusedtocontainadocument'suser-definedkey
_revisusedtocontainthedocument'srevisionnumber
Inedgecollections,the
_from
_to
attributesareusedtoreferenceotherdocuments.
M oresystemattributesmaybeaddedinthefuturewithoutfurthernoticesoendusersshouldtrytoavoidusingtheirownattribute
namesstartingwithunderscores.
Theoretically,attributenamescanincludepunctuationandspecialcharactersasdesired,providedthenameisavalidUTF-8string.
Formaximumportability,specialcharactersshouldbeavoidedthough.Forexample,attributenamesmaycontainthedotsymbol,
butthedothasaspecialmeaninginJavaScriptandalsoinAQL,sowhenusingsuchattributenamesinoneoftheselanguages,the
attributenameneedstobequotedbytheenduser.Overallitmightbebettertouseattributenameswhichdon'trequireany
quoting/escapinginalllanguagesused.Thisincludeslanguagesusedbytheclient(e.g.Ruby,PHP)iftheattributesaremappedto
objectmembersthere.
Attributenamesstartingwithanat-mark(@)willneedtobeenclosedinbacktickswhenusedinanAQLquerytotellthemapart
frombindvariables.Thereforewedonotencouragetheuseofattributesstartingwithat-marks,thoughtheywillworkwhenused
properly.
ArangoDBdoesnotenforcealengthlimitforattributenames.However,longattributenamesmayusemorememoryinresultsets
etc.Thereforetheuseoflongattributenamesisdiscouraged.
Attributenamesarecase-sensitive.
Attributeswithemptynames(anemptystring)aredisallowed.
AttributeNames
78
HandlingIndexes
ThisisanintroductiontoArangoDB'sinterfaceforindexesingeneral.
Therearespecialsectionsfor
IndexBasics:Introductiontoallindextypes
Whichindextousewhen:Indextypeandoptionsadviser
IndexUtilization:HowArangoDBusesindexes
WorkingwithIndexes:Howtohandleindexesprogrammaticallyusingthe dbobject
HashIndexes
Skiplists
PersistentIndexes
FulltextIndexes
Geo-spatialIndexes
Indexing
79
Indexbasics
Indexesallowfastaccesstodocuments,providedtheindexedattribute(s)areusedinaquery.WhileArangoDBautomaticallyindexes
somesystemattributes,usersarefreetocreateextraindexesonnon-systemattributesofdocuments.
User-definedindexescanbecreatedoncollectionlevel.M ostuser-definedindexescanbecreatedbyspecifyingthenamesoftheindex
attributes.Someindextypesallowindexingjustoneattribute(e.g.fulltextindex)whereasotherindextypesallowindexingmultiple
attributesatthesametime.
Thesystemattributes _id, _key, _fromand _toareautomaticallyindexedbyArangoDB,withouttheuserbeingrequiredtocreate
extraindexesforthem. _idand _keyarecoveredbyacollection'sprimarykey,and _fromand _toarecoveredbyanedge
collection'sedgeindexautomatically.
Usingthesystemattribute _idinuser-definedindexesisnotpossible,butindexing _key, _rev, _from,and _tois.
ArangoDBprovidesthefollowingindextypes:
PrimaryIndex
Foreachcollectiontherewillalwaysbeaprimaryindexwhichisahashindexforthedocumentkeys( _keyattribute)ofalldocuments
inthecollection.Theprimaryindexallowsquickselectionofdocumentsinthecollectionusingeitherthe _keyor _idattributes.It
willbeusedfromwithinAQLqueriesautomaticallywhenperformingequalitylookupson _keyor _id.
Therearealsodedicatedfunctionstofindadocumentgivenits _keyor _idthatwillalwaysmakeuseoftheprimaryindex:
db.collection.document("<document-key>");
db._document("<document-id>");
Astheprimaryindexisanunsortedhashindex,itcannotbeusedfornon-equalityrangequeriesorforsorting.
Theprimaryindexofacollectioncannotbedroppedorchanged,andthereisnomechanismtocreateuser-definedprimaryindexes.
EdgeIndex
Everyedgecollectionalsohasanautomaticallycreatededgeindex.Theedgeindexprovidesquickaccesstodocumentsbyeithertheir
_fromor _toattributes.Itcanthereforebeusedtoquicklyfindconnectionsbetweenvertexdocumentsandisinvokedwhenthe
connectingedgesofavertexarequeried.
EdgeindexesareusedfromwithinAQLwhenperformingequalitylookupson _fromor _tovaluesinanedgecollections.Thereare
alsodedicatedfunctionstofindedgesgiventheir _fromor _tovaluesthatwillalwaysmakeuseoftheedgeindex:
db.collection.edges("<from-value>");
db.collection.edges("<to-value>");
db.collection.outEdges("<from-value>");
db.collection.outEdges("<to-value>");
db.collection.inEdges("<from-value>");
db.collection.inEdges("<to-value>");
Internally,theedgeindexisimplementedasahashindex,whichstorestheunionofall _fromand _toattributes.Itcanbeusedfor
equalitylookups,butnotforrangequeriesorforsorting.Edgeindexesareautomaticallycreatedforedgecollections.Itisnotpossibleto
createuser-definededgeindexes.However,itispossibletofreelyusethe _fromand _toattributesinuser-definedindexes.
Anedgeindexcannotbedroppedorchanged.
HashIndex
Ahashindexcanbeusedtoquicklyfinddocumentswithspecificattributevalues.Thehashindexisunsorted,soitsupportsequality
lookupsbutnorangequeriesorsorting.
IndexBasics
80
Ahashindexcanbecreatedononeormultipledocumentattributes.Ahashindexwillonlybeusedbyaqueryifallindexattributesare
presentinthesearchcondition,andifallattributesarecomparedusingtheequality( ==)operator.Hashindexesareusedfromwithin
AQLandseveralqueryfunctions,e.g. byExample, firstExampleetc.
Hashindexescanoptionallybedeclaredunique,thendisallowingsavingthesamevalue(s)intheindexedattribute(s).Hashindexescan
optionallybesparse.
Thedifferenttypesofhashindexeshavethefollowingcharacteristics:
uniquehashindex:alldocumentsinthecollectionmusthavedifferentvaluesfortheattributescoveredbytheuniqueindex.
Tryingtoinsertadocumentwiththesamekeyvalueasanalreadyexistingdocumentwillleadtoauniqueconstraintviolation.
Thistypeofindexisnotsparse.Documentsthatdonotcontaintheindexattributesorthathaveavalueof nullintheindex
attribute(s)willstillbeindexed.Akeyvalueof nullmayonlyoccuronceintheindex,sothistypeofindexcannotbeusedfor
optionalattributes.
Theuniqueoptioncanalsobeusedtoensurethatnoduplicateedgesarecreated,byaddingacombinedindexforthefields _from
and _totoanedgecollection.
unique,sparsehashindex:alldocumentsinthecollectionmusthavedifferentvaluesfortheattributescoveredbytheunique
index.Documentsinwhichatleastoneoftheindexattributesisnotsetorhasavalueof nullarenotincludedintheindex.This
typeofindexcanbeusedtoensurethattherearenoduplicatekeysinthecollectionfordocumentswhichhavetheindexed
attributesset.Astheindexwillexcludedocumentsforwhichtheindexedattributesare nullornotset,itcanbeusedforoptional
attributes.
non-uniquehashindex:alldocumentsinthecollectionwillbeindexed.Thistypeofindexisnotsparse.Documentsthatdonot
containtheindexattributesorthathaveavalueof nullintheindexattribute(s)willstillbeindexed.Duplicatekeyvaluescan
occuranddonotleadtouniqueconstraintviolations.
non-unique,sparsehashindex:onlythosedocumentswillbeindexedthathavealltheindexedattributessettoavalueotherthan
null.Itcanbeusedforoptionalattributes.
Theamortizedcomplexityoflookup,insert,update,andremovaloperationsinuniquehashindexesisO(1).
Non-uniquehashindexeshaveanamortizedcomplexityofO(1)forinsert,update,andremovaloperations.Thatmeansnon-uniquehash
indexescanbeusedonattributeswithlowcardinality.
Ifahashindexiscreatedonanattributethatismissinginallormanyofthedocuments,thebehaviorisasfollows:
iftheindexissparse,thedocumentsmissingtheattributewillnotbeindexedandnotuseindexmemory.Thesedocumentswillnot
influencetheupdateorremovalperformancefortheindex.
iftheindexisnon-sparse,thedocumentsmissingtheattributewillbecontainedintheindexwithakeyvalueof null.
Hashindexessupportindexingarrayvaluesiftheindexattributenameisextendedwitha[*].
SkiplistIndex
Askiplistisasortedindexstructure.Itcanbeusedtoquicklyfinddocumentswithspecificattributevalues,forrangequeriesandfor
returningdocumentsfromtheindexinsortedorder.SkiplistswillbeusedfromwithinAQLandseveralqueryfunctions,e.g. byExample,
firstExampleetc.
Skiplistindexeswillbeusedforlookups,rangequeriesandsortingonlyifeitherallindexattributesareprovidedinaquery,orifa
leftmostprefixoftheindexattributesisspecified.
Forexample,ifaskiplistindexiscreatedonattributes value1and value2,thefollowingfilterconditionscanusetheindex(note:the
<=and >=operatorsareintentionallyomittedhereforthesakeofbrevity):
IndexBasics
81
FILTERdoc.value1==...
FILTERdoc.value1<...
FILTERdoc.value1>...
FILTERdoc.value1>...&&doc.value1<...
FILTERdoc.value1==...&&doc.value2==...
FILTERdoc.value1==...&&doc.value2>...
FILTERdoc.value1==...&&doc.value2>...&&doc.value2<...
Inordertouseaskiplistindexforsorting,theindexattributesmustbespecifiedinthe SORTclauseofthequeryinthesameorderas
theyappearintheindexdefinition.Skiplistindexesarealwayscreatedinascendingorder,buttheycanbeusedtoaccesstheindexed
elementsinbothascendingordescendingorder.However,foracombinedindex(anindexonmultipleattributes)thisrequiresthatthesort
ordersinasinglequeryasspecifiedinthe SORTclausemustbeeitherallascending(optionallyommittedasascendingisthedefault)or
alldescending.
Forexample,iftheskiplistindexiscreatedonattributes value1and value2(inthisorder),thenthefollowingsortsclausescanuse
theindexforsorting:
SORTvalue1ASC,value2ASC(anditsequivalent SORTvalue1,value2)
SORTvalue1DESC,value2DESC
SORTvalue1ASC(anditsequivalent SORTvalue1)
SORTvalue1DESC
Thefollowingsortclausescannotmakeuseoftheindexorder,andrequireanextrasortstep:
SORTvalue1ASC,value2DESC
SORTvalue1DESC,value2ASC
SORTvalue2(anditsequivalent SORTvalue2ASC)
SORTvalue2DESC(becausefirstindexedattribute value1isnotusedinsortclause)
Note:thelattertwosortclausescannotusetheindexbecausethesortclausedoesnotrefertoaleftmostprefixoftheindexattributes.
Skiplistscanoptionallybedeclaredunique,disallowingsavingthesamevalueintheindexedattribute.Theycanbesparseornon-sparse.
Thedifferenttypesofskiplistindexeshavethefollowingcharacteristics:
uniqueskiplistindex:alldocumentsinthecollectionmusthavedifferentvaluesfortheattributescoveredbytheuniqueindex.
Tryingtoinsertadocumentwiththesamekeyvalueasanalreadyexistingdocumentwillleadtoauniqueconstraintviolation.
Thistypeofindexisnotsparse.Documentsthatdonotcontaintheindexattributesorthathaveavalueof nullintheindex
attribute(s)willstillbeindexed.Akeyvalueof nullmayonlyoccuronceintheindex,sothistypeofindexcannotbeusedfor
optionalattributes.
unique,sparseskiplistindex:alldocumentsinthecollectionmusthavedifferentvaluesfortheattributescoveredbytheunique
index.Documentsinwhichatleastoneoftheindexattributesisnotsetorhasavalueof nullarenotincludedintheindex.This
typeofindexcanbeusedtoensurethattherearenoduplicatekeysinthecollectionfordocumentswhichhavetheindexed
attributesset.Astheindexwillexcludedocumentsforwhichtheindexedattributesare nullornotset,itcanbeusedforoptional
attributes.
non-uniqueskiplistindex:alldocumentsinthecollectionwillbeindexed.Thistypeofindexisnotsparse.Documentsthatdonot
containtheindexattributesorthathaveavalueof nullintheindexattribute(s)willstillbeindexed.Duplicatekeyvaluescan
occuranddonotleadtouniqueconstraintviolations.
non-unique,sparseskiplistindex:onlythosedocumentswillbeindexedthathavealltheindexedattributessettoavalueother
than null.Itcanbeusedforoptionalattributes.
Theoperationalamortizedcomplexityforskiplistindexesislogarithmicallycorrelatedwiththenumberofdocumentsintheindex.
Skiplistindexessupportindexingarrayvaluesiftheindexattributenameisextendedwitha[*]`.
PersistentIndex
IndexBasics
82
Thepersistentindexisasortedindexwithpersistence.Theindexentriesarewrittentodiskwhendocumentsarestoredorupdated.That
meanstheindexentriesdonotneedtoberebuiltfromthecollectiondatawhentheserverisrestartedortheindexedcollectionisinitially
loaded.Thususingpersistentindexesmayreducecollectionloadingtimes.
Thepersistentindextypecanbeusedforsecondaryindexesatthemoment.Thatmeansthepersistentindexcurrentlycannotbemade
theonlyindexforacollection,becausetherewillalwaysbethein-memoryprimaryindexforthecollectioninaddition,andpotentially
moreindexes(suchastheedgesindexforanedgecollection).
TheindeximplementationisusingtheRocksDBengine,anditprovideslogarithmiccomplexityforinsert,update,andremoveoperations.
Asthepersistentindexisnotanin-memoryindex,itdoesnotstorepointersintotheprimaryindexasallthein-memoryindexesdo,but
insteaditstoresadocument'sprimarykey.Toretrieveadocumentviaapersistentindexviaanindexvaluelookup,therewillthereforebe
anadditionalO(1)lookupintotheprimaryindextofetchtheactualdocument.
Asthepersistentindexissorted,itcanbeusedforpointlookups,rangequeriesandsortingoperations,butonlyifeitherallindex
attributesareprovidedinaquery,orifaleftmostprefixoftheindexattributesisspecified.
GeoIndex
Userscancreateadditionalgeoindexesononeormultipleattributesincollections.Ageoindexisusedtofindplacesonthesurfaceofthe
earthfast.
Thegeoindexstorestwo-dimensionalcoordinates.Itcanbecreatedoneithertwoseparatedocumentattributes(latitudeandlongitude)or
asinglearrayattributethatcontainsbothlatitudeandlongitude.Latitudeandlongitudemustbenumericvalues.
Thgeoindexprovidesoperationstofinddocumentswithcoordinatesnearesttoagivencomparisoncoordinate,andtofinddocuments
withcoordinatesthatarewithinaspecifiableradiusaroundacomparisoncoordinate.
ThegeoindexisusedviadedicatedfunctionsinAQLorthesimplequeriesfunctions,butwillnotbeusedforothertypesofqueriesor
conditions.
FulltextIndex
Afulltextindexcanbeusedtofindwords,orprefixesofwordsinsidedocuments.Afulltextindexcanbecreatedonasingleattribute
only,andwillindexallwordscontainedindocumentsthathaveatextualvalueinthatattribute.Onlywordswitha(specifiable)minimum
lengthareindexed.Wordtokenizationisdoneusingthewordboundaryanalysisprovidedbylibicu,whichistakingintoaccountthe
selectedlanguageprovidedatserverstart.Wordsareindexedintheirlower-casedform.Theindexsupportscompletematchqueries(full
words)andprefixqueries,plusbasiclogicaloperationssuchas and, orand notforcombiningpartialresults.
Thefulltextindexissparse,meaningitwillonlyindexdocumentsforwhichtheindexattributeissetandcontainsastringvalue.
Additionally,onlywordswithaconfigurableminimumlengthwillbeincludedintheindex.
ThefulltextindexisusedviadedicatedfunctionsinAQLorthesimplequeries,butwillnotbeenabledforothertypesofqueriesor
conditions.
Indexingattributesandsub-attributes
Top-levelaswellasnestedattributescanbeindexed.Forattributesatthetoplevel,theattributenamesalonearerequired.Toindexa
singlefield,passanarraywithasingleelement(stringoftheattributekey)tothefieldsparameteroftheensureIndex()method.Tocreate
acombinedindexovermultiplefields,simplyaddmorememberstothefieldsarray:
//{name:"Smith",age:35}
db.posts.ensureIndex({type:"hash",fields:["name"]})
db.posts.ensureIndex({type:"hash",fields:["name","age"]})
Toindexsub-attributes,specifytheattributepathusingthedotnotation:
//{name:{last:"Smith",first:"John"}}
db.posts.ensureIndex({type:"hash",fields:["name.last"]})
db.posts.ensureIndex({type:"hash",fields:["name.last","name.first"]})
Indexingarrayvalues
IndexBasics
83
Ifanindexattributecontainsanarray,ArangoDBwillstoretheentirearrayastheindexvaluebydefault.Accessingindividualmembers
ofthearrayviatheindexisnotpossiblethisway.
Tomakeanindexinserttheindividualarraymembersintotheindexinsteadoftheentirearrayvalue,aspecialarrayindexneedstobe
createdfortheattribute.Arrayindexescanbesetuplikeregularhashorskiplistindexesusingthe collection.ensureIndex()function.
Tomakeahashorskiplistindexanarrayindex,theindexattributenameneedstobeextendedwith[*]whencreatingtheindexandwhen
filteringinanAQLqueryusingthe INoperator.
Thefollowingexamplecreatesanarrayhashindexonthe tagsattributeinacollectionnamed posts:
db.posts.ensureIndex({type:"hash",fields:["tags[*]"]});
db.posts.insert({tags:["foobar","baz","quux"]});
Thisarrayindexcanthenbeusedforlookingupindividual tagsvaluesfromAQLqueriesviathe INoperator:
FORdocINposts
FILTER'foobar'INdoc.tags
RETURNdoc
Itispossibletoaddthearrayexpansionoperator[*],butitisnotmandatory.Youmayuseittoindicatethatanarrayindexisused,itis
purelycosmetichowever:
FORdocINposts
FILTER'foobar'INdoc.tags[*]
RETURNdoc
ThefollowingFILTERconditionswillnotusethearrayindex:
FILTERdoc.tagsANY=='foobar'
FILTERdoc.tagsANYIN'foobar'
FILTERdoc.tagsIN'foobar'
FILTERdoc.tags=='foobar'
FILTER'foobar'==doc.tags
Itisalsopossibletocreateanindexonsubattributesofarrayvalues.Thismakessenseiftheindexattributeisanarrayofobjects,e.g.
db.posts.ensureIndex({type:"hash",fields:["tags[*].name"]});
db.posts.insert({tags:[{name:"foobar"},{name:"baz"},{name:"quux"}]});
Thefollowingquerywillthenusethearrayindex(thisdoesrequirethearrayexpansionoperator):
FORdocINposts
FILTER'foobar'INdoc.tags[*].name
RETURNdoc
Ifyoustoreadocumenthavingthearraywhichdoescontainelementsnothavingthesubattributesthisdocumentwillalsobeindexed
withthevalue null,whichinArangoDBisequaltoattributenotexisting.
ArangoDBsupportscreatingarrayindexeswithasingle[*]operatorperindexattribute.Forexample,creatinganindexasfollowsisnot
supported:
db.posts.ensureIndex({type:"hash",fields:["tags[*].name[*].value"]});
Arrayvalueswillautomaticallybede-duplicatedbeforebeinginsertedintoanarrayindex.Forexample,ifthefollowingdocumentis
insertedintothecollection,theduplicatearrayvalue barwillbeinsertedonlyonce:
db.posts.insert({tags:["foobar","bar","bar"]});
IndexBasics
84
Ifanarrayindexisdeclaredunique,thede-duplicationofarrayvalueswillhappenbeforeinsertingthevaluesintotheindex,sotheabove
insertoperationwillnotnecessarilyfail.Itwillfailiftheindexalreadycontainsaninstanceofthe barvalue,butwillsucceedifthe
value barisnotalreadypresentintheindex.
Ifanarrayindexisdeclaredandyoustoredocumentsthatdonothaveanarrayatthespecifiedattributethisdocumentwillnotbe
insertedintheindex.Hencethefollowingobjectswillnotbeindexed:
db.posts.insert({something:"else"});
db.posts.insert({tags:null});
db.posts.insert({tags:"thisisnoarray"});
db.posts.insert({tags:{content:[1,2,3]}});
Anarrayindexisabletoindexanexplicit nullvalueandwhenqueriedforit,itwillonlyreturnthosedocumentshavingexplicitly
nullstoredinthearray,itwillnotreturnanydocumentsthatdonothavethearrayatall.
db.posts.insert({tags:null})//Willnotbeindexed
db.posts.insert({tags:[]})//Willnotbeindexed
db.posts.insert({tags:[null]});//Willbeindexedfornull
db.posts.insert({tags:[null,1,2]});//Willbeindexedfornull,1and2
Declaringanarrayindexassparsedoesnothaveaneffectonthearraypartoftheindex,thisinparticularmeansthatexplicit null
valuesarealsoindexedinthesparseversion.Ifanindexiscombinedfromanarrayandanormalattributethesparsitywillapplyforthe
attributee.g.:
db.posts.ensureIndex({type:"hash",fields:["tags[*]","name"],sparse:true});
db.posts.insert({tags:null,name:"alice"})//Willnotbeindexed
db.posts.insert({tags:[],name:"alice"})//Willnotbeindexed
db.posts.insert({tags:[1,2,3]})//Willnotbeindexed
db.posts.insert({tags:[1,2,3],name:null})//Willnotbeindexed
db.posts.insert({tags:[1,2,3],name:"alice"})
//Willbeindexedfor[1,"alice"],[2,"alice"],[3,"alice"]
db.posts.insert({tags:[null],name:"bob"})
//Willbeindexedfor[null,"bob"]
PleasenotethatfilteringusingarrayindexesonlyworksfromwithinAQLqueriesandonlyifthequeryfiltersontheindexedattribute
usingthe INoperator.Theothercomparisonoperators( ==, !=, >, >=, <, <=, ANY, ALL, NONE)currentlycannotuse
arrayindexes.
IndexBasics
85
WhichIndextousewhen
ArangoDBautomaticallyindexesthe _keyattributeineachcollection.Thereisnoneedtoindexthisattributeseparately.Pleasenote
thatadocument's _idattributeisderivedfromthe _keyattribute,andisthusimplicitlyindexed,too.
ArangoDBwillalsoautomaticallycreateanindexon _fromand _toinanyedgecollection,meaningincomingandoutgoing
connectionscanbedeterminedefficiently.
Indextypes
Userscandefineadditionalindexesononeormultipledocumentattributes.SeveraldifferentindextypesareprovidedbyArangoDB.
Theseindexeshavedifferentusagescenarios:
hashindex:providesquickaccesstoindividualdocumentsif(andonlyif)allindexedattributesareprovidedinthesearchquery.The
indexwillonlybeusedforequalitycomparisons.Itdoesnotsupportrangequeriesandcannotbeusedforsorting.
Thehashindexisagoodcandidateifallormostqueriesontheindexedattribute(s)areequalitycomparisons.Theuniquehashindex
providesanamortizedcomplexityofO(1)forinsert,update,removeandlookupoperations.Thenon-uniquehashindexprovides
O(1)inserts,updatesandremoves,andwillallowlookingupdocumentsbyindexvaluewithamortizedO(n)complexity,withn
beingthenumberofdocumentswiththatindexvalue.
Anon-uniquehashindexonanoptionaldocumentattributeshouldbedeclaredsparsesothatitwillnotindexdocumentsforwhich
theindexattributeisnotset.
skiplistindex:skiplistskeeptheindexedvaluesinanorder,sotheycanbeusedforequalitylookups,rangequeriesandforsorting.
Forhighselectivityattributes,skiplistindexeswillhaveahigheroverheadthanhashindexes.Forlowselectivityattributes,skiplist
indexeswillbemoreefficientthannon-uniquehashindexes.
Additionally,skiplistindexesallowmoreusecases(e.g.rangequeries,sorting)thanhashindexes.Furthermore,theycanbeusedfor
lookupsbasedonaleftmostprefixoftheindexattributes.
persistentindex:apersistentindexbehavesmuchlikethesortedskiplistindex,exceptthatallindexvaluesarepersistedondiskand
donotneedtoberebuiltinmemorywhentheserverisrestartedortheindexedcollectionisreloaded.Theoperationsinapersistent
indexhavelogarithmiccomplexity,butoperationshavemayhaveahigherconstantfactorthantheoperationsinaskiplistindex,
becausethepersistentindexmayneedtomakeextraroundtripstotheprimaryindextofetchtheactualdocuments.
Apersistentindexcanbeusedforequalitylookups,rangequeriesandforsorting.Forhighselectivityattributes,persistentindexes
willhaveahigheroverheadthanskiplistorhashindexes.
Persistentindexesallowmoreusecases(e.g.rangequeries,sorting)thanhashindexes.Furthermore,theycanbeusedforlookups
basedonaleftmostprefixoftheindexattributes.Incontrasttothein-memoryskiplistindexes,persistentindexesdonotneedtobe
rebuiltin-memorysotheydon'tinfluencetheloadingtimeofcollectionsasotherin-memoryindexesdo.
geoindex:thegeoindexprovidedbyArangoDBallowssearchingfordocumentswithinaradiusaroundatwo-dimensionalearth
coordinate(point),ortofinddocumentswithareclosesttoapoint.Documentcoordinatescaneitherbespecifiedintwodifferent
documentattributesorinasingleattribute,e.g.
{"latitude":50.9406645,"longitude":6.9599115}
or
{"coords":[50.9406645,6.9599115]}
Geoindexeswillonlybeinvokedviaspecialfunctions.
fulltextindex:afulltextindexcanbeusedtoindexallwordscontainedinaspecificattributeofalldocumentsinacollection.Only
wordswitha(specifiable)minimumlengthareindexed.Wordtokenizationisdoneusingthewordboundaryanalysisprovidedby
libicu,whichistakingintoaccounttheselectedlanguageprovidedatserverstart.
Whichindextousewhen
86
Theindexsupportscompletematchqueries(fullwords)andprefixqueries.Fulltextindexeswillonlybeinvokedviaspecial
functions.
Sparsevs.non-sparseindexes
Hashindexesandskiplistindexescanoptionallybecreatedsparse.Asparseindexdoesnotcontaindocumentsforwhichatleastoneof
theindexattributeisnotsetorcontainsavalueof null.
Assuchdocumentsareexcludedfromsparseindexes,theymaycontainfewerdocumentsthantheirnon-sparsecounterparts.This
enablesfasterindexingandcanleadtoreducedmemoryusageincasetheindexedattributedoesoccuronlyinsome,butnotalldocuments
ofthecollection.Sparseindexeswillalsoreducethenumberofcollisionsinnon-uniquehashindexesincasenon-existingoroptional
attributesareindexed.
Inordertocreateasparseindex,anobjectwiththeattribute sparsecanbeaddedtotheindexcreationcommands:
db.collection.ensureIndex({type:"hash",fields:["attributeName"],sparse:true});
db.collection.ensureIndex({type:"hash",fields:["attributeName1","attributeName2"],sparse:true});
db.collection.ensureIndex({type:"hash",fields:["attributeName"],unique:true,sparse:true});
db.collection.ensureIndex({type:"hash",fields:["attributeName1","attributeName2"],unique:true,sparse:true});
db.collection.ensureIndex({type:"skiplist",fields:["attributeName"],sparse:true});
db.collection.ensureIndex({type:"skiplist",fields:["attributeName1","attributeName2"],sparse:true});
db.collection.ensureIndex({type:"skiplist",fields:["attributeName"],unique:true,sparse:true});
db.collection.ensureIndex({type:"skiplist",fields:["attributeName1","attributeName2"],unique:true,sparse:true});
Whennotexplicitlyset,the sparseattributedefaultsto falsefornewindexes.Otherindexesthanhashandskiplistdonotsupport
sparsity.
Assparseindexesmayexcludesomedocumentsfromthecollection,theycannotbeusedforalltypesofqueries.Sparsehashindexes
cannotbeusedtofinddocumentsforwhichatleastoneoftheindexedattributeshasavalueof null.Forexample,thefollowingAQL
querycannotuseasparseindex,evenifonewascreatedonattribute attr:
FORdocIncollection
FILTERdoc.attr==null
RETURNdoc
Ifthelookupvalueisnon-constant,asparseindexmayormaynotbeused,dependingontheothertypesofconditionsinthequery.If
theoptimizercansafelydeterminethatthelookupvaluecannotbe null,asparseindexmaybeused.Whenuncertain,theoptimizer
willnotmakeuseofasparseindexinaqueryinordertoproducecorrectresults.
Forexample,thefollowingqueriescannotuseasparseindexon attrbecausetheoptimizerwillnotknowbeforehandwhetherthe
valueswhicharecomparedto doc.attrwillinclude null:
FORdocIncollection
FILTERdoc.attr==SOME_FUNCTION(...)
RETURNdoc
FORotherINotherCollection
FORdocIncollection
FILTERdoc.attr==other.attr
RETURNdoc
Sparseskiplistindexescanbeusedforsortingiftheoptimizercansafelydetectthattheindexrangedoesnotinclude nullforanyof
theindexattributes.
Notethatifyouintendtousejoinsitmaybeclevertousenon-sparsityandmaybeevenuniquenessforthatattribute,elseallitems
containingthe nullvaluewillmatchagainsteachotherandthusproducelargeresults.
Whichindextousewhen
87
IndexUtilization
InmostcasesArangoDBwilluseasingleindexpercollectioninagivenquery.AQLqueriescanusemorethanoneindexpercollection
whenmultipleFILTERconditionsarecombinedwithalogical ORandthesecanbecoveredbyindexes.AQLquerieswilluseasingle
indexpercollectionwhenFILTERconditionsarecombinedwithlogical AND.
Creatingmultipleindexesondifferentattributesofthesamecollectionmaygivethequeryoptimizermorechoiceswhenpickinganindex.
Creatingmultipleindexesondifferentattributescanalsohelpinspeedingupdifferentqueries,withFILTERconditionsondifferent
attributes.
Itisoftenbeneficialtocreateanindexonmorethanjustoneattribute.Byaddingmoreattributestoanindex,anindexcanbecomemore
selectiveandthusreducethenumberofdocumentsthatqueriesneedtoprocess.
ArangoDB'sprimaryindexes,edgesindexesandhashindexeswillautomaticallyprovideselectivityestimates.Indexselectivityestimates
areprovidedinthewebinterface,the getIndexes()returnvalueandinthe explain()outputforagivenquery.
Themoreselectiveanindexis,themoredocumentsitwillfilteronaverage.Theindexselectivityestimatesarethereforeusedbythe
optimizerwhencreatingqueryexecutionplanswhentherearemultipleindexestheoptimizercanchoosefrom.Theoptimizerwillthen
selectacombinationofindexeswiththelowestestimatedtotalcost.Ingeneral,theoptimizerwillpicktheindexeswiththehighest
estimatedselectivity.
Sparseindexesmayormaynotbepickedbytheoptimizerinaquery.Assparseindexesdonotcontain nullvalues,theywillnotbe
usedforqueriesiftheoptimizercannotsafelydeterminewhetheraFILTERconditionincludes nullvaluesfortheindexattributes.The
optimizerpolicyistoproducecorrectresults,regardlessofwhetherorwhichindexisusedtosatisfyFILTERconditions.Ifitisunsure
aboutwhetherusinganindexwillviolatethepolicy,itwillnotmakeuseoftheindex.
Troubleshooting
WhenindoubtaboutwhetherandwhichindexeswillbeusedforexecutingagivenAQLquery,usethe explain()methodforthe
statementasfollows(fromtheArangoShell):
varquery="FORdocINcollectionFILTERdoc.value>42RETURNdoc";
varstmt=db._createStatement(query);
stmt.explain();
The explain()commandwillreturnadetailedJSONrepresentationofthequery'sexecutionplan.TheJSONexplainoutputisintended
tobeusedbycode.Togetahuman-readableandmuchmorecompactexplanationofthequery,thereisanexplainertool:
varquery="FORdocINcollectionFILTERdoc.value>42RETURNdoc";
require("@arangodb/aql/explainer").explain(query);
Ifanyoftheexplainmethodsshowsthataqueryisnotusingindexes,thefollowingstepsmayhelp:
checkiftheattributenamesinthequeryarecorrectlyspelled.Inaschema-freedatabase,documentsinthesamecollectioncanhave
varyingstructures.Thereisnosuchthingasanon-existingattributeerror.Aquerythatreferstoattributenamesnotpresentinany
ofthedocumentswillnotreturnanerror,andobviouslywillnotbenefitfromindexes.
checkthereturnvalueofthe getIndexes()methodforthecollectionsusedinthequeryandvalidatethatindexesareactually
presentontheattributesusedinthequery'sfilterconditions.
ifindexesarepresentbutnotusedbythequery,theindexesmayhavethewrongtype.Forexample,ahashindexwillonlybeused
forequalitycomparisons(i.e. ==)butnotforothercomparisontypessuchas <, <=, >, >=.Additionallyhashindexeswill
onlybeusedifalloftheindexattributesareusedinthequery'sFILTERconditions.Askiplistindexwillonlybeusedifatleastits
firstattributeisusedinaFILTERcondition.Ifadditionallyoftheskiplistindexattributesarespecifiedinthequery(fromleft-toright),theymayalsobeusedandallowtofiltermoredocuments.
usingindexedattributesasfunctionparametersorinarbitraryexpressionswilllikelyleadtotheindexontheattributenotbeing
used.Forexample,thefollowingquerieswillnotuseanindexon value:
IndexUtilization
88
FORdocINcollectionFILTERTO_NUMBER(doc.value)==42RETURNdoc
FORdocINcollectionFILTERdoc.value-1==42RETURNdoc
Inthesecasesthequeriesshouldberewrittensothatonlytheindexattributeispresentononesideoftheoperator,oradditional
filtersandindexesshouldbeusedtorestricttheamountofdocumentsotherwise.
thequeryoptimizerwillingeneralpickoneindexpercollectioninaquery.Itcanpickmorethanoneindexpercollectionifthe
FILTERconditioncontainsmultiplebranchescombinedwithlogical OR.Forexample,thefollowingqueriescanuseindexes:
FORdocINcollectionFILTERdoc.value1==42||doc.value1==23RETURNdoc
FORdocINcollectionFILTERdoc.value1<42||doc.value2>23RETURNdoc
Thetwo ORsinthefirstquerywillbeconvertedtoan INlist,andifthereisasuitableindexon value1,itwillbeused.The
secondqueryrequirestwoseparateindexeson value1and value2andwillusethemifpresent.Thethirdquerycanusethe
indexeson value1and value2whentheyaresorted.
IndexUtilization
89
WorkingwithIndexes
IndexIdentifiersandHandles
Anindexhandleuniquelyidentifiesanindexinthedatabase.Itisastringandconsistsofthecollectionnameandanindexidentifier
separatedbya /.Theindexidentifierpartisanumericvaluethatisauto-generatedbyArangoDB.
Aspecificindexofacollectioncanbeaccessedusingitsindexhandleorindexidentifierasfollows:
db.collection.index("<index-handle>");
db.collection.index("<index-identifier>");
db._index("<index-handle>");
Forexample:Assumethattheindexhandle,whichisstoredinthe _idattributeoftheindex,is demo/362549736andtheindexwas
createdinacollectionnamed demo.Thenthisindexcanbeaccessedas:
db.demo.index("demo/362549736");
Becausetheindexhandleisuniquewithinthedatabase,youcanleaveoutthecollectionandusetheshortcut:
db._index("demo/362549736");
CollectionMethods
Listingallindexesofacollection
returnsinformationabouttheindexes getIndexes()
Returnsanarrayofallindexesdefinedforthecollection.
Notethat _keyimplicitlyhasanindexassignedtoit.
arangosh>db.test.ensureHashIndex("hashListAttribute",
........>"hashListSecondAttribute.subAttribute");
arangosh>db.test.getIndexes();
Creatinganindex
IndexesshouldbecreatedusingthegeneralmethodensureIndex.Thismethodobsoletesthespecializedindex-specificmethods
ensureHashIndex,ensureSkiplist,ensureUniqueConstraintetc.
ensuresthatanindexexists collection.ensureIndex(index-description)
Ensuresthatanindexaccordingtotheindex-descriptionexists.Anewindexwillbecreatedifnoneexistswiththegivendescription.
Theindex-descriptionmustcontainatleastatypeattribute.Otherattributesmaybenecessary,dependingontheindextype.
typecanbeoneofthefollowingvalues:
hash:hashindex
skiplist:skiplistindex
fulltext:fulltextindex
geo1:geoindex,withoneattribute
geo2:geoindex,withtwoattributes
WorkingwithIndexes
90
sparsecanbetrueorfalse.
Forhash,andskiplistthesparsitycanbecontrolled,fulltextandgeoaresparsebydefinition.
uniquecanbetrueorfalseandissupportedbyhashorskiplist
Callingthismethodreturnsanindexobject.Whetherornottheindexobjectexistedbeforethecallisindicatedinthereturnattribute
isNewlyCreated.
Examples
arangosh>db.test.ensureIndex({type:"hash",fields:["a"],sparse:true});
arangosh>db.test.ensureIndex({type:"hash",fields:["a","b"],unique:true});
Droppinganindex
dropsanindex collection.dropIndex(index)
Dropstheindex.Iftheindexdoesnotexist,thenfalseisreturned.Iftheindexexistedandwasdropped,thentrueisreturned.Notethat
youcannotdropsomespecialindexes(e.g.theprimaryindexofacollectionortheedgeindexofanedgecollection).
collection.dropIndex(index-handle)
Sameasabove.Insteadofanindexanindexhandlecanbegiven.
arangosh>db.example.ensureSkiplist("a","b");
arangosh>varindexInfo=db.example.getIndexes();
arangosh>indexInfo;
arangosh>db.example.dropIndex(indexInfo[0])
arangosh>db.example.dropIndex(indexInfo[1].id)
arangosh>indexInfo=db.example.getIndexes();
DatabaseMethods
Fetchinganindexbyhandle
findsanindex db._index(index-handle)
Returnstheindexwithindex-handleornullifnosuchindexexists.
arangosh>db.example.ensureIndex({type:"skiplist",fields:["a","b"]});
arangosh>varindexInfo=db.example.getIndexes().map(function(x){returnx.id;});
arangosh>indexInfo;
arangosh>db._index(indexInfo[0])
arangosh>db._index(indexInfo[1])
Droppinganindex
dropsanindex db._dropIndex(index)
Dropstheindex.Iftheindexdoesnotexist,thenfalseisreturned.Iftheindexexistedandwasdropped,thentrueisreturned.
db._dropIndex(index-handle)
Dropstheindexwithindex-handle.
WorkingwithIndexes
91
arangosh>varindexInfo=db.example.getIndexes();
arangosh>indexInfo;
arangosh>db._dropIndex(indexInfo[0])
arangosh>db._dropIndex(indexInfo[1].id)
arangosh>indexInfo=db.example.getIndexes();
Revalidatingwhetheranindexisused
findsanindex
Soyou'vecreatedanindex,andsinceitsmaintainanceisn'tforfree,youdefinitelywanttoknowwhetheryourquerycanutilizeit.
Youcanuseexplaintoverifywhetherskiplistsorhashindexesareused(ifyouomit colors:falseyouwillgetnicecolorsin
ArangoShell):
arangosh>varexplain=require("@arangodb/aql/explainer").explain;
arangosh>explain("FORdocINexampleFILTERdoc.a<23RETURNdoc",{colors:false});
WorkingwithIndexes
92
HashIndexes
IntroductiontoHashIndexes
Itispossibletodefineahashindexononeormoreattributes(orpaths)ofadocument.Thishashindexisthenusedinqueriestolocate
documentsinO(1)operations.Ifthehashindexisunique,thennotwodocumentsareallowedtohavethesamesetofattributevalues.
Creatinganewdocumentorupdatingadocumentwillfailiftheuniquenessisviolated.Iftheindexisdeclaredsparse,adocumentwillbe
excludedfromtheindexandnouniquenesscheckswillbeperformedifanyindexattributevalueisnotsetorhasavalueof null.
AccessingHashIndexesfromtheShell
UniqueHashIndexes
Ensuresthatauniqueconstraintexists: collection.ensureIndex({type:"hash",fields:["field1",...,"fieldn"],unique:true})
Createsauniquehashindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobegiven.The
indexwillbenon-sparsebydefault.
Alldocumentsinthecollectionmustdifferintermsoftheindexedattributes.Creatinganewdocumentorupdatinganexistingdocument
willwillfailiftheattributeuniquenessisviolated.
Tocreateasparseuniqueindex,setthesparseattributeto true:
collection.ensureIndex({type:"hash",fields:["field1",...,"fieldn"],unique:true,sparse:true})
Incasethattheindexwassuccessfullycreated,theindexidentifierisreturned.
Non-existingattributeswilldefaultto null.Inasparseindexalldocumentswillbeexcludedfromtheindexforwhichallspecifiedindex
attributesare null.Suchdocumentswillnotbetakenintoaccountforuniquenesschecks.
Inanon-sparseindex,alldocumentsregardlessof null-attributeswillbeindexedandwillbetakenintoaccountforuniqueness
checks.
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetails,includingtheindex-identifier,isreturned.
arangosh>db.test.ensureIndex({type:"hash",fields:["a","b.c"],unique:true});
arangosh>db.test.save({a:1,b:{c:1}});
arangosh>db.test.save({a:1,b:{c:1}});
arangosh>db.test.save({a:1,b:{c:null}});
arangosh>db.test.save({a:1});
Non-uniqueHashIndexes
Ensuresthatanon-uniquehashindexexists: collection.ensureIndex({type:"hash",fields:["field1",...,"fieldn"]})
Createsanon-uniquehashindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobegiven.The
collection.ensureIndex({type:"hash",fields:["field1",...,"fieldn"],sparse:true})
HashIndexes
93
arangosh>db.test.ensureIndex({type:"hash",fields:["a"]});
arangosh>db.test.save({a:null});
HashArrayIndexes
Ensuresthatahasharrayindexexists(non-unique): collection.ensureIndex({type:"hash",fields:["field1[*]",...,"fieldn[*]"]
})
Createsanon-uniquehasharrayindexfortheindividualelementsofthearrayattributesfield1[*],...fieldn[*]foundinthedocuments.At
leastoneattributepathhastobegiven.Theindexalwaystreatstheindexedarraysassparse.
Itispossibletocombinearrayindexingwithstandardindexing: collection.ensureIndex({type:"hash",fields:["field1[*]",
"field2"]})
arangosh>db.test.ensureIndex({type:"hash",fields:["a[*]"]});
arangosh>db.test.save({a:[1,2]});
Ensureuniquenessofrelationsinedgecollections
Itispossibletocreatesecondaryindexesusingtheedgeattributes _fromand _to,startingwithArangoDB3.0.Acombinedindexover
bothfieldstogetherwiththeuniqueoptionenabledcanbeusedtopreventduplicaterelationsfrombeingcreated.
Forexample,adocumentcollectionvertsmightcontainverticeswiththedocumenthandles verts/A, verts/Band verts/C.Relations
betweenthesedocumentscanbestoredinanedgecollectionedgesforinstance.Now,youmaywanttomakesurethatthevertex
verts/Aisneverlinkedto verts/Bbyanedgemorethanonce.Thiscanbeachievedbyaddingaunique,non-sparsehashindexforthe
fields _fromand _to:
db.edges.ensureIndex({type:"hash",fields:["_from","_to"],unique:true});
Creatinganedge {_from:"verts/A",_to:"verts/B"}inedgeswillbeaccepted,butonlyonce.Anotherattempttostoreanedgewith
therelationA→Bwillberejectedbytheserverwithauniqueconstraintviolatederror.Thisincludesupdatestothe _fromand _to
fields.
NotethataddingarelationB→Aisstillpossible,soisA→AandB→B,becausetheyarealldifferentrelationsinadirectedgraph.
Eachonecanonlyoccuroncehowever.
HashIndexes
94
Skiplists
IntroductiontoSkiplistIndexes
ThisisanintroductiontoArangoDB'sskiplists.
Itispossibletodefineaskiplistindexononeormoreattributes(orpaths)ofdocuments.Thisskiplististhenusedinqueriestolocate
documentswithinagivenrange.Iftheskiplistisdeclaredunique,thennotwodocumentsareallowedtohavethesamesetofattribute
values.
Creatinganewdocumentorupdatingadocumentwillfailiftheuniquenessisviolated.Iftheskiplistindexisdeclaredsparse,adocument
willbeexcludedfromtheindexandnouniquenesscheckswillbeperformedifanyindexattributevalueisnotsetorhasavalueof null.
AccessingSkiplistIndexesfromtheShell
UniqueSkiplistIndex
Ensuresthatauniqueskiplistindexexists: collection.ensureIndex({type:"skiplist",fields:["field1",...,"fieldn"],unique:
true})
Createsauniqueskiplistindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobegiven.The
collection.ensureIndex({type:"skiplist",fields:["field1",...,"fieldn"],unique:true,sparse:true})
Inasparseindexalldocumentswillbeexcludedfromtheindexthatdonotcontainatleastoneofthespecifiedindexattributesorthat
haveavalueof nullinanyofthespecifiedindexattributes.Suchdocumentswillnotbeindexed,andnotbetakenintoaccountfor
uniquenesschecks.
Inanon-sparseindex,thesedocumentswillbeindexed(fornon-presentindexedattributes,avalueof nullwillbeused)andwillbe
takenintoaccountforuniquenesschecks.
arangosh>db.ids.ensureIndex({type:"skiplist",fields:["myId"],unique:true});
arangosh>db.ids.save({"myId":123});
arangosh>db.ids.ensureIndex({type:"skiplist",fields:["name.first","name.last"],unique:
arangosh>db.ids.save({"name":{"first":"hans","last":"hansen"}});
arangosh>db.ids.save({"name":{"first":"jens","last":"jensen"}});
arangosh>db.ids.save({"name":{"first":"hans","last":"jensen"}});
Non-uniqueSkiplistIndex
Skiplists
95
Ensuresthatanon-uniqueskiplistindexexists: collection.ensureIndex({type:"skiplist",fields:["field1",...,"fieldn"]})
Createsanon-uniqueskiplistindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobegiven.
Theindexwillbenon-sparsebydefault.
Tocreateasparseuniqueindex,setthesparseattributeto true.
arangosh>db.names.ensureIndex({type:"skiplist",fields:["first"]});
arangosh>db.names.save({"first":"Tim"});
arangosh>db.names.save({"first":"Tom"});
arangosh>db.names.save({"first":"John"});
SkiplistArrayIndex
Ensuresthataskiplistarrayindexexists(non-unique): collection.ensureIndex({type:"skiplist",fields:["field1[*]",...,
"fieldn[*]"]})
Createsanon-uniqueskiplistarrayindexfortheindividualelementsofthearrayattributesfield1[*],...fieldn[*]foundinthedocuments.
Atleastoneattributepathhastobegiven.Theindexalwaystreatstheindexedarraysassparse.
Itispossibletocombinearrayindexingwithstandardindexing: collection.ensureIndex({type:"skiplist",fields:["field1[*]",
"field2"]})
arangosh>db.test.ensureIndex({type:"skiplist",fields:["a[*]"]});
Querybyexampleusingaskiplistindex
Constructsaquery-by-exampleusingaskiplistindex: collection.byExample(example)
Selectsalldocumentsfromthecollectionthatmatchthespecifiedexampleandreturnsacursor.Askiplistindexwillbeusedifpresent.
{"a":{"c":1}}
document
{"a":{"c":1},"b":1}
{"a":{"c":1,"b":1}}
willnot.
Skiplists
96
However,ifyouuse
{"a.c":1},
{"a":{"c":1},"b":1}
and
{"a":{"c":1,"b":1}}
willmatch.
Skiplists
97
Persistentindexes
IntroductiontoPersistentIndexes
ThisisanintroductiontoArangoDB'spersistentindexes.
Itispossibletodefineapersistentindexononeormoreattributes(orpaths)ofdocuments.Theindexisthenusedinqueriestolocate
documentswithinagivenrange.Iftheindexisdeclaredunique,thennotwodocumentsareallowedtohavethesamesetofattribute
values.
Creatinganewdocumentorupdatingadocumentwillfailiftheuniquenessisviolated.Iftheindexisdeclaredsparse,adocumentwillbe
excludedfromtheindexandnouniquenesscheckswillbeperformedifanyindexattributevalueisnotsetorhasavalueof null.
AccessingPersistentIndexesfromtheShell
ensuresthatauniquepersistentindexexists collection.ensureIndex({type:"persistent",fields:["field1",...,"fieldn"],
unique:true})
Createsauniquepersistentindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobegiven.
Theindexwillbenon-sparsebydefault.
collection.ensureIndex({type:"persistent",fields:["field1",...,"fieldn"],unique:true,sparse:true})
Inasparseindexalldocumentswillbeexcludedfromtheindexthatdonotcontainatleastoneofthespecifiedindexattributesorthat
haveavalueof nullinanyofthespecifiedindexattributes.Suchdocumentswillnotbeindexed,andnotbetakenintoaccountfor
uniquenesschecks.
Inanon-sparseindex,thesedocumentswillbeindexed(fornon-presentindexedattributes,avalueof nullwillbeused)andwillbe
takenintoaccountforuniquenesschecks.
arangosh>db.ids.ensureIndex({type:"persistent",fields:["myId"],unique:true});
arangosh>db.ids.ensureIndex({type:"persistent",fields:["name.first","name.last"],unique:
arangosh>db.ids.save({"name":{"first":"jens","last":"jensen"}});
arangosh>db.ids.save({"name":{"first":"hans","last":"jensen"}});
ensuresthatanon-uniquepersistentindexexists collection.ensureIndex({type:"persistent",fields:["field1",...,"fieldn"]})
Createsanon-uniquepersistentindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobe
given.Theindexwillbenon-sparsebydefault.
Persistent
98
Tocreateasparseuniqueindex,setthesparseattributeto true.
arangosh>db.names.ensureIndex({type:"persistent",fields:["first"]});
arangosh>db.names.save({"first":"John"});
Querybyexampleusingapersistentindex
constructsaquery-by-exampleusingapersistentindex collection.byExample(example)
Selectsalldocumentsfromthecollectionthatmatchthespecifiedexampleandreturnsacursor.Apersistentindexwillbeusedifpresent.
{"a":{"c":1}}
document
{"a":{"c":1},"b":1}
{"a":{"c":1,"b":1}}
willnot.
However,ifyouuse
{"a.c":1},
{"a":{"c":1},"b":1}
and
{"a":{"c":1,"b":1}}
willmatch.
PersistentIndexesandServerLanguage
Theorderofindexentriesinpersistentindexesadherestotheconfiguredserverlanguage.If,however,theserverisrestartedwitha
differentlanguagesettingaswhenthepersistentindexwascreated,notalldocumentsmaybereturnedanymoreandthesortorderof
thosewhicharereturnedcanbewrong(wheneverthepersistentindexisconsulted).
Tofixpersistentindexesafteralanguagechange,deleteandre-createthem.Skiplistindexesarenotaffected,becausetheyarenot
persistedandautomaticallyrebuiltoneveryserverstart.
Persistent
99
Persistent
100
Fulltextindexes
ThisisanintroductiontoArangoDB'sfulltextindexes.
IntroductiontoFulltextIndexes
Afulltextindexcanbeusedtofindwords,orprefixesofwordsinsidedocuments.
Afulltextindexcanbedefinedononeattributeonly,andwillincludeallwordscontainedindocumentsthathaveatextualvalueinthe
indexattribute.SinceArangoDB2.6theindexwillalsoincludewordsfromtheindexattributeiftheindexattributeisanarrayofstrings,
oranobjectwithstringvaluemembers.
Forexample,givenafulltextindexonthe translationsattributeandthefollowingdocuments,thensearchingfor лисаusingthe
fulltextindexwouldreturnonlythefirstdocument.Searchingfortheindexfortheexactstring Foxwouldreturnthefirsttwo
documents,andsearchingfor prefix:Foxwouldreturnallthreedocuments:
{translations:{en:"fox",de:"Fuchs",fr:"renard",ru:"лиса"}}
{translations:"FoxistheEnglishtranslationoftheGermanwordFuchs"}
{translations:["ArangoDB","document","database","Foxx"]}
Notethatdeepernestedobjectsareignored.Forexample,afulltextindexontranslationswouldindexFuchs,butnotfox,giventhe
followingdocumentstructure:
{translations:{en:{US:"fox"},de:"Fuchs"}
Ifyouneedtosearchacrossmultiplefieldsand/ornestedobjects,youmaywriteallthestringsintoaspecialattribute,whichyouthen
createtheindexon(itmightbenecessarytocleanthestringsfirst,e.g.removelinebreaksandstripcertainwords).
Iftheindexattributeisneitherastring,anobjectoranarray,itscontentswillnotbeindexed.Whenindexingthecontentsofanarray
attribute,anarraymemberwillonlybeincludedintheindexifitisastring.Whenindexingthecontentsofanobjectattribute,anobject
membervaluewillonlybeincludedintheindexifitisastring.Otherdatatypesareignoredandnotindexed.
AccessingFulltextIndexesfromtheShell
Ensuresthatafulltextindexexists:
collection.ensureIndex({type:"fulltext",fields:["field"],minLength:minLength})
Createsafulltextindexonalldocumentsonattributefield.
Fulltextindexesareimplicitlysparse:alldocumentswhichdonothavethespecifiedfieldattributeorthathaveanon-qualifyingvaluein
theirfieldattributewillbeignoredforindexing.
Onlyasingleattributecanbeindexed.Specifyingmultipleattributesisunsupported.
TheminimumlengthofwordsthatareindexedcanbespecifiedviatheminLengthparameter.WordsshorterthanminLengthcharacters
willnotbeindexed.minLengthhasadefaultvalueof2,butthisvaluemightbechangedinfutureversionsofArangoDB.Itisthus
recommendedtoexplicitlyspecifythisvalue.
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetailsisreturned.
FulltextIndexes
101
arangosh>db.example.ensureIndex({type:"fulltext",fields:["text"],minLength:3});
arangosh>db.example.save({text:"thequickbrown",b:{c:1}});
arangosh>db.example.save({text:"quickbrownfox",b:{c:2}});
arangosh>db.example.save({text:"brownfoxjums",b:{c:3}});
arangosh>db.example.save({text:"foxjumpsover",b:{c:4}});
arangosh>db.example.save({text:"jumpsoverthe",b:{c:5}});
arangosh>db.example.save({text:"overthelazy",b:{c:6}});
arangosh>db.example.save({text:"thelazydog",b:{c:7}});
arangosh>db._query("FORdocumentINFULLTEXT(example,'text','the')RETURNdocument");
Looksupafulltextindex:
collection.lookupFulltextIndex(attribute,minLength)
Checkswhetherafulltextindexonthegivenattributeattributeexists.
FulltextAQLFunctions
FulltextAQLfunctionsaredetailedinFulltextfunctions.
FulltextIndexes
102
GeoIndexes
IntroductiontoGeoIndexes
ThisisanintroductiontoArangoDB'sgeoindexes.
AQL'sgeographicfeaturesaredescribedinGeofunctions.
ArangoDBusesHilbertcurvestoimplementgeo-spatialindexes.Seethisblogfordetails.
Ageo-spatialindexassumesthatthelatitudeisbetween-90and90degreeandthelongitudeisbetween-180and180degree.Ageoindex
willignorealldocumentswhichdonotfulfilltheserequirements.
AccessingGeoIndexesfromtheShell
ensuresthatageoindexexists collection.ensureIndex({type:"geo",fields:["location"]})
Createsageo-spatialindexonalldocumentsusinglocationaspathtothecoordinates.Thevalueoftheattributehastobeanarraywithat
leasttwonumericvalues.Thearraymustcontainthelatitude(firstvalue)andthelongitude(secondvalue).
Alldocuments,whichdonothavetheattributepathorhaveanon-conformingvalueinitareexcludedfromtheindex.
Ageoindexisimplicitlysparse,andthereisnowaytocontrolitssparsity.
Tocreateageoonanarrayattributethatcontainslongitudefirst,setthegeoJsonattributeto true.Thiscorrespondstotheformat
describedinpositions
collection.ensureIndex({type:"geo",fields:["location"],geoJson:true})
Tocreateageo-spatialindexonalldocumentsusinglatitudeandlongitudeasseparateattributepaths,twopathsneedtobespecifiedin
thefieldsarray:
collection.ensureIndex({type:"geo",fields:["latitude","longitude"]})
Examples
Createageoindexforanarrayattribute:
arangosh>db.geo.ensureIndex({type:"geo",fields:["loc"]});
........>for(j=-180;j<=180;j+=10){
........>db.geo.save({name:"Name/"+i+"/"+j,loc:[i,j]});
........>}
........>}
arangosh>db.geo.count();
arangosh>db.geo.near(0,0).limit(3).toArray();
arangosh>db.geo.near(0,0).count();
Createageoindexforahasharrayattribute:
GeoIndexes
103
arangosh>db.geo2.ensureIndex({type:"geo",fields:["location.latitude","location.longitude"
........>for(j=-180;j<=180;j+=10){
........>db.geo2.save({name:"Name/"+i+"/"+j,location:{latitude:i,longitude:j}
........>}
........>}
arangosh>db.geo2.near(0,0).limit(3).toArray();
constructsageoindexselection collection.geo(location-attribute)Looksupageoindexdefinedonattributelocation_attribute.
Returnsageoindexobjectifanindexwasfound.The nearor withinoperatorscanthenbeusedtoexecuteageo-spatialqueryonthis
particularindex.Thisisusefulforcollectionswithmultipledefinedgeoindexes. collection.geo(location_attribute,true)Looksupa
geoindexonacompoundattributelocation_attribute.Returnsageoindexobjectifanindexwasfound.The nearor withinoperators
canthenbeusedtoexecuteageo-spatialqueryonthisparticularindex. collection.geo(latitude_attribute,longitude_attribute)
Looksupageoindexdefinedonthetwoattributeslatitude_attributeandlongitude-attribute.Returnsageoindexobjectifanindexwas
found.The nearor withinoperatorscanthenbeusedtoexecuteageo-spatialqueryonthisparticularindex.Note:thegeosimple
queryhelperfunctionisdeprecatedasofArangoDB2.6.ThefunctionmayberemovedinfutureversionsofArangoDB.Thepreferred
wayforrunninggeoqueriesistousetheirAQLequivalents.
Examples
Assumeyouhavealocationstoredaslistintheattributehomeandadestinationstoredintheattributework.Thenyoucanusethe geo
operatortoselectwhichgeo-spatialattributes(andthuswhichindex)touseina nearquery.
........>for(j=-180;j<=180;j+=10){
........>db.complex.save({name:"Name/"+i+"/"+j,
........>home:[i,j],
........>work:[-i,-j]});
........>}
........>}
........>
arangosh>db.complex.near(0,170).limit(5);
arangosh>db.complex.ensureIndex({type:"geo",fields:["home"]});
arangosh>db.complex.near(0,170).limit(5).toArray();
arangosh>db.complex.geo("work").near(0,170).limit(5);
arangosh>db.complex.ensureIndex({type:"geo",fields:["work"]});
arangosh>db.complex.geo("work").near(0,170).limit(5).toArray();
constructsanearqueryforacollection collection.near(latitude,longitude)Thereturnedlistissortedaccordingtothedistance,with
thenearestdocumenttothecoordinate(latitude,longitude)comingfirst.Ifthereareneardocumentsofequaldistance,documentsare
chosenrandomlyfromthissetuntilthelimitisreached.Itispossibletochangethelimitusingthelimitoperator.Inordertousethenear
operator,ageoindexmustbedefinedforthecollection.Thisindexalsodefineswhichattributeholdsthecoordinatesforthedocument.If
youhavemorethenonegeo-spatialindex,youcanusethegeooperatortoselectaparticularindex.Note: neardoesnotsupport
negativeskips.//However,youcanstilluse limitfollowedtoskip. collection.near(latitude,longitude).limit(limit)Limitsthe
resulttolimitdocumentsinsteadofthedefault100.Note:Unlikewithmultipleexplicitlimits, limitwillraisetheimplicitdefaultlimit
imposedby within. collection.near(latitude,longitude).distance()Thiswilladdanattribute distancetoalldocuments
returned,whichcontainsthedistancebetweenthegivenpointandthedocumentinmeters. collection.near(latitude,
longitude).distance(name)Thiswilladdanattributenametoalldocumentsreturned,whichcontainsthedistancebetweenthegiven
pointandthedocumentinmeters.Note:thenearsimplequeryfunctionisdeprecatedasofArangoDB2.6.Thefunctionmaybe
removedinfutureversionsofArangoDB.Thepreferredwayforretrievingdocumentsfromacollectionusingthenearoperatoristouse
theAQLNEARfunctioninanAQLqueryasfollows:
GeoIndexes
104
FORdocINNEAR(@@collection,@latitude,@longitude,@limit)
RETURNdoc
Examples
Togetthenearesttwolocations:
arangosh>for(vari=-90;i<=90;i+=10){
........>for(varj=-180;j<=180;j+=10){
........>db.geo.save({
........>name:"Name/"+i+"/"+j,
........>loc:[i,j]});
........>}}
arangosh>db.geo.near(0,0).limit(2).toArray();
Ifyouneedthedistanceaswell,thenyoucanusethe distanceoperator:
........>for(varj=-180;j<=180;j+=10){
........>db.geo.save({
........>name:"Name/"+i+"/"+j,
........>loc:[i,j]});
........>}}
arangosh>db.geo.near(0,0).distance().limit(2).toArray();
constructsawithinqueryforacollection collection.within(latitude,longitude,radius)Thiswillfindalldocumentswithinagiven
radiusaroundthecoordinate(latitude,longitude).Thereturnedarrayissortedbydistance,beginningwiththenearestdocument.Inorder
tousethewithinoperator,ageoindexmustbedefinedforthecollection.Thisindexalsodefineswhichattributeholdsthecoordinatesfor
thedocument.Ifyouhavemorethenonegeo-spatialindex,youcanusethe geooperatortoselectaparticularindex.
collection.within(latitude,longitude,radius).distance()Thiswilladdanattribute _distancetoalldocumentsreturned,which
containsthedistancebetweenthegivenpointandthedocumentinmeters. collection.within(latitude,longitude,
radius).distance(name)Thiswilladdanattributenametoalldocumentsreturned,whichcontainsthedistancebetweenthegivenpoint
andthedocumentinmeters.Note:thewithinsimplequeryfunctionisdeprecatedasofArangoDB2.6.Thefunctionmayberemovedin
futureversionsofArangoDB.ThepreferredwayforretrievingdocumentsfromacollectionusingthewithinoperatoristousetheAQL
WITHINfunctioninanAQLqueryasfollows:
FORdocINWITHIN(@@collection,@latitude,@longitude,@radius,@distanceAttributeName)
RETURNdoc
Examples
Tofindalldocumentswithinaradiusof2000kmuse:
........>for(varj=-180;j<=180;j+=10){
........>db.geo.save({name:"Name/"+i+"/"+j,loc:[i,j]});}}
arangosh>db.geo.within(0,0,2000*1000).distance().toArray();
ensuresthatageoindexexists collection.ensureIndex({type:"geo",fields:["location"]})
GeoIndexes
105
SinceArangoDB2.5,thismethodisanaliasforensureGeoIndexsincegeoindexesarealwayssparse,meaningthatdocumentsthatdonot
containtheindexattributesorhavenon-numericvaluesintheindexattributeswillnotbeindexed.ensureGeoConstraintisdeprecatedand
ensureGeoIndexshouldbeusedinstead.
Theindexdoesnotprovidea uniqueoptionbecauseofitslimitedusability.Itwouldpreventidenticalcoordinatesfrombeinginserted
only,butevenaslightlydifferentlocation(like1inchor1cmoff)wouldbeuniqueagainandnotconsideredaduplicate,althoughit
probablyshould.Thedesiredthresholdfordetectingduplicatesmayvaryforeveryproject(includinghowtocalculatethedistanceeven)
andneedstobeimplementedontheapplicationlayerasneeded.YoucanwriteaFoxxserviceforthispurposeandmakeuseoftheAQL
geofunctionstofindnearbycoordinatessupportedbyageoindex.
GeoIndexes
106
ArangoDBGraphs
FirstStepswithGraphs
AGraphconsistsofverticesandedges.Edgesarestoredasdocumentsinedgecollections.Avertexcanbeadocumentofadocument
collectionorofanedgecollection(soedgescanbeusedasvertices).Whichcollectionsareusedwithinanamedgraphisdefinedviaedge
definitions.Anamedgraphcancontainmorethanoneedgedefinition,atleastoneisneeded.Graphsallowyoutostructureyourmodels
inlinewithyourdomainandgroupthemlogicallyincollectionsandgivingyouthepowertoquerytheminthesamegraphqueries.
Comingfromarelationalbackground-what'sagraph?
InSQLyoucommonlyhavetheconstructofarelationtabletostoren:mrelationsbetweentwodatatables.Anedgecollectionis
somewhatsimilartotheserelationtables;vertexcollectionsresemblethedatatableswiththeobjectstoconnect.Whilesimplegraph
querieswithfixednumberofhopsviatherelationtablemaybedoableinSQLwithseveralnestedjoins,graphdatabasescanhandlean
arbitrarynumberofthesehopsoveredgecollections-thisiscalledtraversal.Alsoedgesinoneedgecollectionmaypointtoseveral
vertexcollections.Itscommontohaveattributesattachedtoedges,i.e.alabelnamingthisinterconnection.Edgeshaveadirection,with
theirrelations _fromand _topointingfromonedocumenttoanotherdocumentstoredinvertexcollections.Inqueriesyoucandefine
inwhichdirectionstheedgerelationsmaybefollowed( OUTBOUND: _from→ _to, INBOUND: _from← _to, ANY: _from↔
_to).
NamedGraphs
Namedgraphsarecompletelymanagedbyarangodb,andthusalsovisibleinthewebinterface.TheyusethefullspectrumofArangoDBs
graphfeatures.Youmayaccessthemviaseveralinterfaces.
AQLGraphOperationswithseveralflavors:
AQLTraversalsonbothnamedandanonymousgraphs
AQLShortestPathonbothnamedandanonymousgraph
JavaScriptGeneralGraphimplementation,asyoumayuseitinFoxxServices
GraphM anagement;creating&manipualatinggraphdefinitions;inserting,updatinganddeletingverticesandedgesintographs
GraphFunctionsforworkingwithedgesandvertices,toanalyzethemandtheirrelations
RESTfulGeneralGraphinterfaceusedtoimplementgraphmanagementinclientdrivers
Manipulatingcollectionsofnamedgraphswithregulardocumentfunctions
Theunderlyingcollectionsofthenamedgraphsarestillaccessibleusingthestandardmethodsforcollections.Howeverthegraphmodule
addsanadditionallayerontopofthesecollectionsgivingyouthefollowingguarantees:
Allmodificationsareexecutedtransactional
Ifyoudeleteavertexalledgeswillbedeleted,youwillneverhavelooseends
Ifyouinsertanedgeitischeckediftheedgematchestheedgedefinitions,youredgecollectionswillonlycontainvalidedges
TheseguaranteesarelostifyouaccessthecollectionsinanyotherwaythanthegraphmoduleorAQL,soifyoudeletedocumentsfrom
yourvertexcollectionsdirectly,theedgespointingtothemwillberemaininplace.
Anonymousgraphs
Sometimesyoumaynotneedallthepowersofnamedgraphs,butsomeofitsbitsmaybevaluabletoyou.Youmayuseanonymous
graphsinthetraversalsandintheWorkingwithEdgeschapter.Anonymousgraphsdon'thaveedgedefinitionsdescribingwhichvertex
collectionisconnectedbywhichedgecollection.Thegraphmodelhastobemaintainedintheclientsidecode.Thisgivesyoumore
freedomthanthestrictnamedgraphs.
Whentochooseanonymousornamedgraphs?
Asnotedabove,namedgraphsensuregraphintegrity,bothwheninsertingorremovingedgesorvertices.Soyouwon'tencounterdangling
edges,evenifyouusethesamevertexcollectioninseveralnamedgraphs.Thisinvolvesmoreoperationsinsidethedatabasewhichdon't
comeforfree.Thereforeanonymousgraphsmaybefasterinmanyoperations.Sothisquestionboilsdownto'CanIaffordtheadditional
Graphs
107
effortordoIneedthewarrantyforintegrity?'.
Multipleedgecollectionsvs. FILTERsonedgedocumentattributes
Ifyouwanttoonlytraverseedgesofaspecifictype,therearetwowaystoachievethis.Thefirstwouldbeanattributeintheedge
document-i.e. type,whereyouspecifyadifferentiatorfortheedge-i.e. "friends", "family", "married"or "workmates",soyou
canlater FILTERe.type="friends"ifyouonlywanttofollowthefriendedges.
Anotherway,whichmaybemoreefficientinsomecases,istousedifferentedgecollectionsfordifferenttypesofedges,soyouhave
friend_eges, family_edges, married_edgesand workmate_edgesascollectionnames.Youcanthenconfigureseveralnamedgraphs
includingasubsetoftheavailableedgeandvertexcollections-oryouuseanonymousgraphqueries,whereyouspecifyalistofedge
collectionstotakeintoaccountinthatquery.Toonlyfollowfriendedges,youwouldspecify friend_edgesassoleedgecollection.
Bothapproacheshaveadvantagesanddisadvantages. FILTERoperationsonedeattributeswilldocomparisonsoneachtraversededge,
whichmaybecomeCPU-intense.Whennotfindingtheedgesinthefirstplacebecauseofthecollectioncontainingthemisnottraversed
atall,therewillneverbeareasontoactualycheckfortheir typeattributewith FILTER.
Themultipleedgecollectionsapproachislimitedbythenumberofcollectionsthatcanbeusedsimultaneouslyinonequery.Every
collectionusedinaqueryrequiressomeresourcesinsideofArangoDBandthenumberisthereforelimitedtocaptheresource
requirements.Youmayalsohaveconstraintsonotheredgeattributes,suchasahashindexwithauniqueconstraint,whichrequiresthe
documentstobeinasinglecollectionfortheuniquenessguarantee,anditmaythusnotbepossibletostorethedifferenttypesofedges
inmultipleedegcollections.
So,ifyouredgeshaveaboutadozendifferenttypes,it'sokaytochoosethecollectionapproach,otherwisethe FILTERapproachis
preferred.Youcanstilluse FILTERoperationsonedgesofcourse.Youcangetridofa FILTERonthe typewiththeformerapproach,
everythingelsecanstaythesame.
WhichpartofmydataisanEdgeandwhichaVertex?
Themainobjectsinyourdatamodel,suchasusers,groupsorarticles,areusuallyconsideredtobevertices.Foreachtypeofobject,a
documentcollection(alsocalledvertexcollection)shouldstoretheindividualentities.Entitiescanbeconnectedbyedgestoexpressand
classifyrelationsbetweenvertices.Itoftenmakessensetohaveanedgecollectionperrelationtype.
ArangoDBdoesnotrequireyoutostoreyourdataingraphstructureswithedgesandvertices,youcanalsodecidetoembedattributes
suchaswhichgroupsauserispartof,or _idsofdocumentsinanotherdocumentinsteadofconnectingthedocumentswithedges.It
canbeameaningfulperformanceoptimizationfor1:nrelationships,ifyourdataisnotveryfocusedonrelationsandyoudon'tneed
graphtraversalwithvaryingdepth.Itusuallymeanstointroducesomeredundancyandpossiblyinconsistenciesifyouembeddata,butit
canbeanacceptabletradeoff.
Vertices
Let'ssaywehavetwovertexcollections, Usersand Groups.Documentsinthe GroupscollectioncontaintheattributesoftheGroup,
i.e.whenitwasfounded,itssubject,aniconURLandsoon. Usersdocumentscontainthedataspecifictoauser-likeallnames,
birthdays,AvatarURLs,hobbies...
Edges
Wecanuseanedgecollectiontostorerelationsbetweenusersandgroups.Sincemultipleusersmaybeinanarbitrarynumberofgroups,
thisisanm:nrelation.Theedgecollectioncanbecalled UsersInGroupswithi.e.oneedgewith _frompointingto Users/Johnand
_topointingto Groups/BowlingGroupHappyPin.ThismakestheuserJohnamemberofthegroupBowlingGroupHappyPin.
Attributesofthisrelationmaycontainqualifierstothisrelation,likethepermissionsofJohninthisgroup,thedatewhenhejoinedthe
groupetc.
Graphs
108
Advantagesofthisapproach
Graphsgiveyoutheadvantageofnotjustbeingabletohaveafixednumberofm:nrelationsinarow,butanarbitrarynumber.Edgescan
betraversedinbothdirections,soit'sveryeasytodetermineallgroupsauserisin,butalsotofindoutwhichmembersacertaingroup
has.Userscouldalsobeinterconnectedtocreateasocialnetwork.
Usingthegraphdatamodel,dealingwithdatathathaslotsofrelationsstaysmanageableandcanbequeriedinveryflexibleways,
whereasitwouldcauseheadachetohandleitinarelationaldatabasesystem.
Backupandrestore
Forsureyouwanttohavebackupsofyourgraphdata,youcanuseArangodumptocreatethebackup,andArangorestoretorestorea
backupintoanewArangoDB.Youshouldhowevernotethat:
youneedthesystemcollection _graphsifyoubackupnamedgraphs.
youneedtobackupthecompletesetofalledgeandvertexcollectionsyourgraphconsistsof.Partialdump/restoremaynotwork.
ExampleGraphs
ArangoDBcomeswithasetofeasilygraspablegraphsthatareusedtodemonstratetheAPIs.Youcanusethe addsamplestabinthe
creategraphwindowinthewebinterface,orloadthemodule @arangodb/graph-examples/example-graphinarangoshanduseittocreate
instancesofthesegraphsinyourArangoDB.Onceyou'vecreatedthem,youcaninspecttheminthewebinterface-whichwasusedto
createthepicturesbelow.
Youcaneasilylookintotheinnardsofthisscriptforreferenceabouthowtomanagegraphsprogramatically.
TheKnows_Graph
Asetofpersonsknowingeachother:
Graphs
109
Theknowsgraphconsistsofonevertexcollection personsconnectedviaoneedgecollection knows.ItwillcontainfivepersonsAlice,
Bob,Charlie,DaveandEve.Wewillhavethefollowingdirectedrelations:
AliceknowsBob
BobknowsCharlie
BobknowsDave
EveknowsAlice
EveknowsBob
Thisishowwecreateit,inspectitsverticesandedges,anddropitagain:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>varg=examples.loadGraph("knows_graph");
arangosh>db.persons.toArray()
arangosh>db.knows.toArray();
arangosh>examples.dropGraph("knows_graph");
TheSocialGraph
Asetofpersonsandtheirrelations:
Thisexamplehasfemaleandmalepersonsasverticesintwovertexcollections- femaleand male.Theedgesaretheirconnectionsin
the relationedgecollection.Thisishowwecreateit,inspectitsverticesandedges,anddropitagain:
arangosh>vargraph=examples.loadGraph("social");
arangosh>db.female.toArray()
arangosh>db.male.toArray()
arangosh>db.relation.toArray()
arangosh>examples.dropGraph("social");
TheCityGraph
Graphs
110
Asetofeuropeancities,andtheirfictionaltravelingdistancesasconnections:
Theexamplehasthecitiesasverticesinseveralvertexcollections- germanCityand frenchCity.Theedgesaretheirinterconnectionsin
severaledgecollections french/german/internationalHighway.Thisishowwecreateit,inspectitsedgesandvertices,anddropit
again:
arangosh>varg=examples.loadGraph("routeplanner");
arangosh>db.frenchCity.toArray();
arangosh>db.germanCity.toArray();
arangosh>db.germanHighway.toArray();
arangosh>db.frenchHighway.toArray();
arangosh>db.internationalHighway.toArray();
arangosh>examples.dropGraph("routeplanner");
TheTraversalGraph
Thisgraphwasdesignedtodemonstratefiltersintraversals.Ithassomelabelstofilteronit.
Graphs
111
Theexamplehasallitsverticesinthecirclescollection,andanedgesedgecollectiontoconnectthem.Circleshaveuniquenumericlabels.
Edgeshavetwobooleanattributes(theFalsealwaysbeingfalse,theTruthalwaysbeingtrue)andalabelsortingB-Dtotheleftside,GKtotherightside.LeftandrightsidesplitintoPaths-atBandGwhichareeachdirectneighboursoftheroot-nodeA.StartingfromA
thegraphhasadepthof3onallitspaths.
arangosh>varg=examples.loadGraph("traversalGraph");
arangosh>db.circles.toArray();
arangosh>db.edges.toArray();
arangosh>examples.dropGraph("traversalGraph");
TheWorldGraph
Theworldcountrygraphstructuresitsnodeslikethat:world→continent→country→capital.Insomecasesedgedirectionsaren't
forward(thereforeitwillbedisplayeddisjunctinthegraphviewer).Ithastwowaysofcreatingit.Oneusingthenamedgraphutilities
(worldCountry),onewithout(worldCountryUnManaged).Itisusedtodemonstraterawtraversaloperations.
arangosh>varg=examples.loadGraph("worldCountry");
arangosh>db.worldVertices.toArray();
arangosh>db.worldEdges.toArray();
arangosh>examples.dropGraph("worldCountry");
arangosh>varg=examples.loadGraph("worldCountryUnManaged");
arangosh>examples.dropGraph("worldCountryUnManaged");
Graphs
112
Cookbookexamples
TheabovereferencedchaptersdescribethevariousAPIsofArangoDBsgraphenginewithsmallexamples.Ourcookbookhassomemore
reallifeexamples:
Traversingagraphinfulldepth
Usinganexamplevertexwiththejavadriver
RetrievingdocumentsfromArangoDBwithoutknowingthestructure
Usingacustomvisitorfromnode.js
AQLExampleQueriesonanActorsandM oviesDatabase
Highervolumegraphexamples
Alloftheaboveexamplesarerathersmallsotheyareeasiertocomprehendandcandemonstratethewaythefunctionalityworks.There
arehoweverseveraldatasetsfreelyavailableonthewebthatarealotbigger.Wecollectedsomeofthemwithimportscriptssoyoumay
playaroundwiththem.AnotherhugegraphisthePokecsocialnetworkfromSlovakiathatweusedforperformancetestingonseveral
databases;Youwillfindimportingscriptsetc.inthisblogpost.
Graphs
113
Graphs
Thischapterdescribesthegeneral-graphmodule.Itallowsyoutodefineagraphthatisspreadacrossseveraledgeanddocument
collections.Thisallowsyoutostructureyourmodelsinlinewithyourdomainandgroupthemlogicallyincollectionsgivingyouthe
powertoquerytheminthesamegraphqueries.Thereisnoneedtoincludethereferencedcollectionswithinthequery,thismodulewill
handleitforyou.
ThreeStepstocreateagraph
Createagraph
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>vargraph=graph_module._create("myGraph");
arangosh>graph;
[GraphmyGraphEdgeDefinitions:[]VertexCollections:[]]
Addsomevertexcollections
arangosh>graph._addVertexCollection("shop");
arangosh>graph._addVertexCollection("customer");
arangosh>graph._addVertexCollection("pet");
arangosh>graph;
Definerelationsonthe
arangosh>varrel=graph_module._relation("isCustomer",["shop"],["customer"]);
arangosh>graph._extendEdgeDefinitions(rel);
arangosh>graph;
[GraphmyGraphEdgeDefinitions:[
"isCustomer:[shop]->[customer]"
]VertexCollections:[]]
GeneralGraphs
114
GraphManagement
Thischapterdescribesthejavascriptinterfaceforcreatingandmodifyingnamedgraphs.Inordertocreateanonemptygraphthe
functionalitytocreateedgedefinitionshastobeintroducedfirst:
EdgeDefinitions
Anedgedefinitionisalwaysadirectedrelationofagraph.Eachgraphcanhavearbitrarymanyrelationsdefinedwithintheedge
definitionsarray.
Initializethelist
Createalistofedgedefinitionstoconstructagraph.
graph_module._edgeDefinitions(relation1,relation2,...,relationN)
Thelistofedgedefinitionsofagraphcanbemanagedbythegraphmoduleitself.Thisfunctionistheentrypointforthemanagementand
willreturnthecorrectlist.
Parameters
relationX(optional)Anobjectrepresentingadefinitionofonerelationinthegraph
Examples
arangosh>directed_relation=graph_module._relation("lives_in","user","city");
arangosh>undirected_relation=graph_module._relation("knows","user","user");
arangosh>edgedefinitions=graph_module._edgeDefinitions(directed_relation,undirected_relation);
Extendthelist
Extendthelistofedgedefinitionstoconstructagraph.
graph_module._extendEdgeDefinitions(edgeDefinitions,relation1,relation2,...,relationN)
Inordertoaddmoreedgedefinitionstothegraphbeforecreatingthisfunctioncanbeusedtoaddmoredefinitionstotheinitiallist.
Parameters
edgeDefinitions(required)Alistofrelationdefinitionobjects.
relationX(required)Anobjectrepresentingadefinitionofonerelationinthegraph
Examples
arangosh>directed_relation=graph_module._relation("lives_in","user","city");
arangosh>undirected_relation=graph_module._relation("knows","user","user");
arangosh>edgedefinitions=graph_module._edgeDefinitions(directed_relation);
arangosh>edgedefinitions=graph_module._extendEdgeDefinitions(undirected_relation);
Relation
Defineadirectedrelation.
GraphM anagement
115
graph_module._relation(relationName,fromVertexCollections,toVertexCollections)
TherelationNamedefinesthenameofthisrelationandreferencestotheunderlyingedgecollection.ThefromVertexCollectionsisan
Arrayofdocumentcollectionsholdingthestartvertices.ThetoVertexCollectionsisanArrayofdocumentcollectionsholdingthetarget
vertices.RelationsareonlyallowedinthedirectionfromanycollectioninfromVertexCollectionstoanycollectionintoVertexCollections.
Parameters
relationName(required)Thenameoftheedgecollectionwheretheedgesshouldbestored.Willbecreatedifitdoesnotyetexist.
fromVertexCollections(required)Oneoralistofcollectionnames.Sourceverticesfortheedgeshavetobestoredinthese
collections.Collectionswillbecreatediftheydonotexist.
toVertexCollections(required)Oneoralistofcollectionnames.Targetverticesfortheedgeshavetobestoredinthesecollections.
Collectionswillbecreatediftheydonotexist.
Examples
arangosh>graph_module._relation("has_bought",["Customer","Company"],["Groceries","Electronics"
arangosh>graph_module._relation("has_bought","Customer","Product");
Createagraph
Afterhavingintroducededgedefinitionsagraphcanbecreated.
Createagraph
graph_module._create(graphName,edgeDefinitions,orphanCollections)
Thecreationofagraphrequiresthenameofthegraphandadefinitionofitsedges.
Foreverytypeofedgedefinitionaconveniencemethodexiststhatcanbeusedtocreateagraph.Optionallyalistofvertexcollections
canbeadded,whicharenotusedinanyedgedefinition.Thesecollectionsarereferredtoasorphancollectionswithinthischapter.All
collectionsusedwithinthecreationprocessarecreatediftheydonotexist.
Parameters
graphName(required)Uniqueidentifierofthegraph
edgeDefinitions(optional)Listofrelationdefinitionobjects
orphanCollections(optional)Listofadditionalvertexcollectionnames
Examples
Createanemptygraph,edgedefinitionscanbeaddedatruntime:
arangosh>graph=graph_module._create("myGraph");
[GraphmyGraphEdgeDefinitions:[]VertexCollections:[]]
Createagraphusinganedgecollection edgesandasinglevertexcollection vertices
GraphM anagement
116
arangosh>varedgeDefinitions=[{collection:"edges","from":["vertices"],"to":["vertices"
arangosh>graph=graph_module._create("myGraph",edgeDefinitions);
[GraphmyGraphEdgeDefinitions:[
"edges:[vertices]->[vertices]"
Createagraphwithedgedefinitionsandorphancollections:
arangosh>graph=graph_module._create("myGraph",
........>[graph_module._relation("myRelation",["male","female"],["male","female"])],["sessions"
CompleteExampletocreateagraph
ExampleCall:
arangosh>varedgeDefinitions=graph_module._edgeDefinitions();
arangosh>graph_module._extendEdgeDefinitions(edgeDefinitions,graph_module._relation("friend_of"
arangosh>graph_module._extendEdgeDefinitions(
........>edgeDefinitions,graph_module._relation(
........>"has_bought",["Customer","Company"],["Groceries","Electronics"]));
arangosh>graph_module._create("myStore",edgeDefinitions);
[GraphmyStoreEdgeDefinitions:[
"friend_of:[Customer]->[Customer]",
"has_bought:[Company,Customer]->[Electronics,Groceries]"
alternativecall:
arangosh>varedgeDefinitions=graph_module._edgeDefinitions(
........>graph_module._relation("friend_of",["Customer"],["Customer"]),graph_module._relation(
........>"has_bought",["Customer","Company"],["Groceries","Electronics"]));
arangosh>graph_module._create("myStore",edgeDefinitions);
[GraphmyStoreEdgeDefinitions:[
"friend_of:[Customer]->[Customer]",
"has_bought:[Company,Customer]->[Electronics,Groceries]"
Listavailablegraphs
Listallgraphs.
graph_module._list()
Listsallgraphnamesstoredinthisdatabase.
Examples
GraphM anagement
117
arangosh>graph_module._list();
[]
Loadagraph
Getagraph
graph_module._graph(graphName)
Agraphcanberetrievedbyitsname.
Parameters
Examples
Getagraph:
arangosh>graph=graph_module._graph("social");
[GraphsocialEdgeDefinitions:[
"relation:[female,male]->[female,male]"
Removeagraph
Removeagraph
graph_module._drop(graphName,dropCollections)
Agraphcanbedroppedbyitsname.Thiscandropallcollectionscontainedinthegraphaslongastheyarenotusedwithinothergraphs.
Todropthecollectionsonlybelongingtothisgraph,theoptionalparameterdrop-collectionshastobesettotrue.
Parameters
dropCollections(optional)Defineifcollectionsshouldbedropped(default:false)
Examples
Dropagraphandkeepcollections:
arangosh>graph_module._drop("social");
true
arangosh>db._collection("female");
[ArangoCollection16455,"female"(typedocument,statusloaded)]
arangosh>db._collection("male");
[ArangoCollection16457,"male"(typedocument,statusloaded)]
arangosh>db._collection("relation");
[ArangoCollection16459,"relation"(typeedge,statusloaded)]
GraphM anagement
118
arangosh>graph_module._drop("social",true);
true
arangosh>db._collection("female");
null
arangosh>db._collection("male");
null
arangosh>db._collection("relation");
null
Modifyagraphdefinitionduringruntime
Afteryouhavecreatedangraphitsdefinitionisnotimmutable.Youcanstilladd,deleteormodifyedgedefinitionsandvertexcollections.
Extendtheedgedefinitions
Addanotheredgedefinitiontothegraph
graph._extendEdgeDefinitions(edgeDefinition)
Extendstheedgedefinitionsofagraph.Ifanorphancollectionisusedinthisedgedefinition,itwillberemovedfromtheorphanage.If
theedgecollectionoftheedgedefinitiontoaddisalreadyusedinthegraphorusedinadifferentgraphwithdifferentfromand/orto
collectionsanerroristhrown.
Parameters
edgeDefinition(required)Therelationdefinitiontoextendthegraph
Examples
arangosh>vargraph_module=require("@arangodb/general-graph")
arangosh>vared1=graph_module._relation("myEC1",["myVC1"],["myVC2"]);
arangosh>vargraph=graph_module._create("myGraph",[ed1]);
arangosh>graph._extendEdgeDefinitions(ed2);
Modifyanedgedefinition
M odifyanrelationdefinition
graph_module._editEdgeDefinition(edgeDefinition)
Editsonerelationdefinitionofagraph.Theedgedefinitionusedasargumentwillreplacetheexistingedgedefinitionofthegraphwhich
hasthesamecollection.VertexCollectionsofthereplacededgedefinitionthatarenotusedinthenewdefinitionwilltransformtoan
orphan.Orphansthatareusedinthisnewedgedefinitionwillbedeletedfromthelistoforphans.Othergraphswiththesameedge
definitionwillbemodified,too.
Parameters
edgeDefinition(required)Theedgedefinitiontoreplacetheexistingedgedefinitionwiththesameattributecollection.
Examples
arangosh>varoriginal=graph_module._relation("myEC1",["myVC1"],["myVC2"]);
arangosh>varmodified=graph_module._relation("myEC1",["myVC2"],["myVC3"]);
arangosh>vargraph=graph_module._create("myGraph",[original]);
arangosh>graph._editEdgeDefinitions(modified);
GraphM anagement
119
Deleteanedgedefinition
Deleteonerelationdefinition
graph_module._deleteEdgeDefinition(edgeCollectionName,dropCollection)
Deletesarelationdefinitiondefinedbytheedgecollectionofagraph.Ifthecollectionsdefinedintheedgedefinition(collection,from,to)
arenotusedinanotheredgedefinitionofthegraph,theywillbemovedtotheorphanage.
Parameters
edgeCollectionName(required)Nameofedgecollectionintherelationdefinition.
dropCollection(optional)Defineiftheedgecollectionshouldbedropped.Defaultfalse.
Examples
Removeanedgedefinitionbutkeeptheedgecollection:
arangosh>vargraph=graph_module._create("myGraph",[ed1,ed2]);
arangosh>graph._deleteEdgeDefinition("myEC1");
arangosh>db._collection("myEC1");
[ArangoCollection20839,"myEC1"(typeedge,statusloaded)]
Removeanedgedefinitionanddroptheedgecollection:
arangosh>vargraph=graph_module._create("myGraph",[ed1,ed2]);
arangosh>graph._deleteEdgeDefinition("myEC1",true);
arangosh>db._collection("myEC1");
null
ExtendvertexCollections
Eachgraphcanhaveanarbitraryamountofvertexcollections,whicharenotpartofanyedgedefinitionofthegraph.Thesecollections
arecalledorphancollections.Ifthegraphisextendedwithanedgedefinitionusingoneoftheorphans,itwillberemovedfromthesetof
orphancollectionautomatically.
Addavertexcollection
Addavertexcollectiontothegraph
graph._addVertexCollection(vertexCollectionName,createCollection)
Addsavertexcollectiontothesetoforphancollectionsofthegraph.Ifthecollectiondoesnotexist,itwillbecreated.Ifitisalready
usedbyanyedgedefinitionofthegraph,anerrorwillbethrown.
Parameters
vertexCollectionName(required)Nameofvertexcollection.
createCollection(optional)Iftruethecollectionwillbecreatedifitdoesnotexist.Default:true.
Examples
GraphM anagement
120
arangosh>graph._addVertexCollection("myVC3",true);
Gettheorphanedcollections
Getallorphancollections
graph._orphanCollections()
Returnsallvertexcollectionsofthegraphthatarenotusedinanyedgedefinition.
Examples
arangosh>graph._orphanCollections();
[
"myVC3"
]
Removeavertexcollection
Removeavertexcollectionfromthegraph
graph._removeVertexCollection(vertexCollectionName,dropCollection)
Removesavertexcollectionfromthegraph.Onlycollectionsnotusedinanyrelationdefinitioncanberemoved.Optionallythecollection
canbedeleted,ifitisnotusedinanyothergraph.
Parameters
vertexCollectionName(required)Nameofvertexcollection.
dropCollection(optional)Iftruethecollectionwillbedroppedifitisnotusedinanyothergraph.Default:false.
Examples
arangosh>graph._removeVertexCollection("myVC3");
ManiuplatingVertices
Saveavertex
CreateanewvertexinvertexCollectionName
graph.vertexCollectionName.save(data)
GraphM anagement
121
Parameters
data(required)JSONdataofvertex.
Examples
arangosh>graph.male.save({name:"Floyd",_key:"floyd"});
Replaceavertex
ReplacesthedataofavertexincollectionvertexCollectionName
graph.vertexCollectionName.replace(vertexId,data,options)
Parameters
vertexId(required)_idattributeofthevertex
options(optional)Seecollectiondocumentation
Examples
arangosh>graph.male.save({neym:"Jon",_key:"john"});
arangosh>graph.male.replace("male/john",{name:"John"});
Updateavertex
UpdatesthedataofavertexincollectionvertexCollectionName
graph.vertexCollectionName.update(vertexId,data,options)
Parameters
Examples
arangosh>graph.female.save({name:"Lynda",_key:"linda"});
arangosh>graph.female.update("female/linda",{name:"Linda",_key:"linda"});
Removeavertex
RemovesavertexincollectionvertexCollectionName
graph.vertexCollectionName.remove(vertexId,options)
Additionallyremovesallingoingandoutgoingedgesofthevertexrecursively(seeedgeremove).
Parameters
GraphM anagement
122
Examples
arangosh>graph.male.save({name:"Kermit",_key:"kermit"});
arangosh>db._exists("male/kermit")
arangosh>graph.male.remove("male/kermit")
arangosh>db._exists("male/kermit")
ManipulatingEdges
Saveanewedge
CreatesanedgefromvertexfromtovertextoincollectionedgeCollectionName
graph.edgeCollectionName.save(from,to,data,options)
Parameters
from(required)_idattributeofthesourcevertex
to(required)_idattributeofthetargetvertex
data(required)JSONdataoftheedge
Examples
arangosh>graph.relation.save("male/bob","female/alice",{type:"married",_key:"bobAndAlice"
Ifthecollectionsoffromandtoarenotdefinedinanedgedefinitionofthegraph,theedgewillnotbestored.
arangosh>graph.relation.save(
........>"relation/aliceAndBob",
........>"female/alice",
........>{type:"married",_key:"bobAndAlice"});
[ArangoError1906:invalidedgebetweenrelation/aliceAndBobandfemale/alice.Doesn'tconformtoanyedg
Replaceanedge
ReplacesthedataofanedgeincollectionedgeCollectionName.Notethat _fromand _toaremandatory.
graph.edgeCollectionName.replace(edgeId,data,options)
Parameters
edgeId(required)_idattributeoftheedge
GraphM anagement
123
Examples
arangosh>graph.relation.save("female/alice","female/diana",{typo:"nose",_key:"aliceAndDiana"
arangosh>graph.relation.replace("relation/aliceAndDiana",{type:"knows",_from:"female/alice"
Updateanedge
UpdatesthedataofanedgeincollectionedgeCollectionName
graph.edgeCollectionName.update(edgeId,data,options)
Parameters
Examples
arangosh>graph.relation.save("female/alice","female/diana",{type:"knows",_key:"aliceAndDiana"
arangosh>graph.relation.update("relation/aliceAndDiana",{type:"quarreled",_key:"aliceAndDiana"
Removeanedge
RemovesanedgeincollectionedgeCollectionName
graph.edgeCollectionName.remove(edgeId,options)
Ifthisedgeisusedasavertexbyanotheredge,theotheredgewillberemoved(recursively).
Parameters
Examples
arangosh>graph.relation.save("female/alice","female/diana",{_key:"aliceAndDiana"});
arangosh>db._exists("relation/aliceAndDiana")
arangosh>graph.relation.remove("relation/aliceAndDiana")
arangosh>db._exists("relation/aliceAndDiana")
Connectedges
Getallconnectingedgesbetween2groupsofverticesdefinedbytheexamples
graph._connectingEdges(vertexExample,vertexExample2,options)
GraphM anagement
124
Thefunctionacceptsanid,anexample,alistofexamplesorevenanemptyexampleasparameterforvertexExample.
Parameters
vertexExample1(optional)SeeDefinitionofexamples
vertexExample2(optional)SeeDefinitionofexamples
options(optional)Anobjectdefiningfurtheroptions.Canhavethefollowingvalues:
edgeExamples:Filtertheedges,seeDefinitionofexamples
edgeCollectionRestriction:Oneoralistofedge-collectionnamesthatshouldbeconsideredtobeonthepath.
vertex1CollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredontheintermediatevertex
steps.
vertex2CollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredontheintermediatevertex
steps.
Examples
Arouteplannerexample,allconnectingedgesbetweencapitals.
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._getConnectingEdges({isCapital:true},{isCapital:true});
[]
GraphM anagement
125
GraphFunctions
Thischapterdescribesvariousfunctionsonagraph.Alotoftheseacceptavertex(oredge)exampleasparameterasdefinedinthenext
section.
ExampleswillexplaintheAPIonthethecitygraph:
Definitionofexamples
Formanyofthefollowingfunctionsexamplescanbepassedinasaparameter.Examplesareusedtofiltertheresultsetforobjectsthat
matchtheconditions.Theseexamplescanhavethefollowingvalues:
null,thereisnomatchingexecutedallfoundresultsarevalid.
Astring,onlyresultsarereturned,which_idequalthevalueofthestring
Anexampleobject,definingasetofattributes.Onlyresultshavingtheseattributesarematched.
Alistcontainingexampleobjectsand/orstrings.Allresultsmatchingatleastoneoftheelementsinthelistarereturned.
Getverticesfromedges.
Getvertexfromofanedge
Getthesourcevertexofanedge
graph._fromVertex(edgeId)
Returnsthevertexdefinedwiththeattribute_fromoftheedgewithedgeIdasits_id.
Parameters
Examples
GraphFunctions
126
arangosh>graph._fromVertex("relation/aliceAndBob")
Getvertextoofanedge
Getthetargetvertexofanedge
graph._toVertex(edgeId)
Returnsthevertexdefinedwiththeattribute_tooftheedgewithedgeIdasits_id.
Parameters
Examples
arangosh>graph._toVertex("relation/aliceAndBob")
_neighbors
Getallneighborsoftheverticesdefinedbytheexample
graph._neighbors(vertexExample,options)
Thefunctionacceptsanid,anexample,alistofexamplesorevenanemptyexampleasparameterforvertexExample.Thecomplexityof
thismethodisO(n*m^x)withnbeingtheverticesdefinedbytheparametervertexExamplex,mtheaverageamountofneighborsandx
themaximaldepths.HencethedefaultcallwouldhaveacomplexityofO(n*m);
Parameters
vertexExample(optional)SeeDefinitionofexamples
direction:Thedirectionoftheedges.Possiblevaluesareoutbound,inboundandany(default).
edgeExamples:Filtertheedges,seeDefinitionofexamples
neighborExamples:Filtertheneighborvertices,seeDefinitionofexamples
vertexCollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredontheintermediatevertexsteps.
minDepth:Definestheminimalnumberofintermediatestepstoneighbors(defaultis1).
maxDepth:Definesthemaximalnumberofintermediatestepstoneighbors(defaultis1).
Examples
Arouteplannerexample,allneighborsofcapitals.
arangosh>graph._neighbors({isCapital:true});
Arouteplannerexample,alloutboundneighborsofHamburg.
GraphFunctions
127
arangosh>graph._neighbors('germanCity/Hamburg',{direction:'outbound',maxDepth:2});
_commonNeighbors
Getallcommonneighborsoftheverticesdefinedbytheexamples.
graph._commonNeighbors(vertex1Example,vertex2Examples,optionsVertex1,optionsVertex2)
Thisfunctionreturnstheintersectionofgraph_module._neighbors(vertex1Example,optionsVertex1)and
graph_module._neighbors(vertex2Example,optionsVertex2).Forparameterdocumentationsee_neighbors.
ThecomplexityofthismethodisO(n*m^x)withnbeingthemaximalamountofverticesdefinedbytheparametersvertexExamples,m
theaverageamountofneighborsandxthemaximaldepths.HencethedefaultcallwouldhaveacomplexityofO(n*m);
Examples
Arouteplannerexample,allcommonneighborsofcapitals.
arangosh>graph._commonNeighbors({isCapital:true},{isCapital:true});
Arouteplannerexample,allcommonoutboundneighborsofHamburgwithanyotherlocationwhichhaveamaximaldepthof2:
arangosh>graph._commonNeighbors(
........>'germanCity/Hamburg',
........>{},
........>{direction:'outbound',maxDepth:2},
........>{direction:'outbound',maxDepth:2});
_countCommonNeighbors
Gettheamountofcommonneighborsoftheverticesdefinedbytheexamples.
graph._countCommonNeighbors(vertex1Example,vertex2Examples,optionsVertex1,optionsVertex2)
Similarto_commonNeighborsbutreturnscountinsteadoftheelements.
Examples
Arouteplannerexample,allcommonneighborsofcapitals.
arangosh>varexample={isCapital:true};
arangosh>varoptions={includeData:true};
arangosh>graph._countCommonNeighbors(example,example,options,options);
GraphFunctions
128
Arouteplannerexample,allcommonoutboundneighborsofHamburgwithanyotherlocationwhichhaveamaximaldepthof2:
arangosh>varoptions={direction:'outbound',maxDepth:2,includeData:true};
arangosh>graph._countCommonNeighbors('germanCity/Hamburg',{},options,options);
_commonProperties
Gettheverticesofthegraphthatsharecommonproperties.
graph._commonProperties(vertex1Example,vertex2Examples,options)
Thefunctionacceptsanid,anexample,alistofexamplesorevenanemptyexampleasparameterforvertex1Exampleand
vertex2Example.
ThecomplexityofthismethodisO(n)withnbeingthemaximalamountofverticesdefinedbytheparametersvertexExamples.
Parameters
vertex1Examples(optional)Filterthesetofsourcevertices,seeDefinitionofexamples
vertex2Examples(optional)Filterthesetofverticescomparedto,seeDefinitionofexamples
vertex1CollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbesearchedforsourcevertices.
vertex2CollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbesearchedforcomparevertices.
ignoreProperties:Oneoralistofattributenamesofadocumentthatshouldbeignored.
Examples
Arouteplannerexample,alllocationswiththesameproperties:
arangosh>graph._commonProperties({},{});
Arouteplannerexample,allcitieswhichsharesamepropertiesexceptforpopulation.
arangosh>graph._commonProperties({},{},{ignoreProperties:'population'});
_countCommonProperties
Gettheamountofverticesofthegraphthatsharecommonproperties.
graph._countCommonProperties(vertex1Example,vertex2Examples,options)
Similarto_commonPropertiesbutreturnscountinsteadoftheobjects.
Examples
Arouteplannerexample,alllocationswiththesameproperties:
GraphFunctions
129
arangosh>graph._countCommonProperties({},{});
Arouteplannerexample,allGermancitieswhichsharesamepropertiesexceptforpopulation.
arangosh>graph._countCommonProperties({},{},{vertex1CollectionRestriction:'germanCity',
........>vertex2CollectionRestriction:'germanCity',ignoreProperties:'population'});
_paths
The_pathsfunctionreturnsallpathsofagraph.
graph._paths(options)
Thisfunctiondeterminesallavailablepathsinagraph.
ThecomplexityofthismethodisO(n*n*m)withnbeingtheamountofverticesinthegraphandmtheaverageamountofconnected
edges;
Parameters
options(optional)Anobjectcontainingoptions,seebelow:
direction:Thedirectionoftheedges.Possiblevaluesareany,inboundandoutbound(default).
followCycles(optional):Ifsettotruethequeryfollowscyclesinthegraph,defaultisfalse.
minLength(optional):Definestheminimallengthapathmusthavetobereturned(defaultis0).
maxLength(optional):Definesthemaximallengthapathmusthavetobereturned(defaultis10).
Examples
Returnallpathsofthegraph"social":
arangosh>varg=examples.loadGraph("social");
arangosh>g._paths();
Returnallinboundpathsofthegraph"social"withamaximallengthof1andaminimallengthof2:
arangosh>varg=examples.loadGraph("social");
arangosh>g._paths({direction:'inbound',minLength:1,maxLength:2});
_shortestPath
The_shortestPathfunctionreturnsallshortestpathsofagraph.
graph._shortestPath(startVertexExample,endVertexExample,options)
GraphFunctions
130
Thisfunctiondeterminesallshortestpathsinagraph.Thefunctionacceptsanid,anexample,alistofexamplesorevenanempty
exampleasparameterforstartandendvertex.Ifonewantstocallthisfunctiontoreceivenearlyallshortestpathsforagraphtheoption
algorithmshouldbesettoFloyd-Warshalltoincreaseperformance.Ifnoalgorithmisprovidedintheoptionsthefunctionchoosesthe
appropriateone(eitherFloyd-WarshallorDijkstra)accordingtoitsparameters.Thelengthofapathisbydefaulttheamountofedges
fromonestartvertextoanendvertex.Theoptionweightallowstheusertodefineanedgeattributerepresentingthelength.
Parameters
startVertexExample(optional)AnexampleforthedesiredstartVertices(seeDefinitionofexamples).
endVertexExample(optional)AnexampleforthedesiredendVertices(seeDefinitionofexamples).
options(optional)Anobjectcontainingoptions,seebelow:
direction:Thedirectionoftheedgesasastring.Possiblevaluesareoutbound,inboundandany(default).
edgeCollectionRestriction:Oneormultipleedgecollectionnames.Onlyedgesfromthesecollectionswillbeconsideredforthe
path.
startVertexCollectionRestriction:Oneormultiplevertexcollectionnames.Onlyverticesfromthesecollectionswillbe
consideredasstartvertexofapath.
endVertexCollectionRestriction:Oneormultiplevertexcollectionnames.Onlyverticesfromthesecollectionswillbeconsidered
asendvertexofapath.
edgeExamples:Afilterexamplefortheedgesintheshortestpaths(seeexample).
algorithm:Thealgorithmtocalculatetheshortestpaths.IfbothstartandendvertexexamplesareemptyFloyd-Warshallis
used,otherwisethedefaultisDijkstra
weight:Thenameoftheattributeoftheedgescontainingthelengthasastring.
defaultWeight:Onlyusedwiththeoptionweight.Ifanedgedoesnothavetheattributenamedasdefinedinoptionweightthis
defaultisusedaslength.IfnodefaultissuppliedthedefaultwouldbepositiveInfinitysothepathcouldnotbecalculated.
Examples
Arouteplannerexample,shortestpathfromallgermantoallfrenchcities:
arangosh>g._shortestPath({},{},{weight:'distance',endVertexCollectionRestriction:'frenchCity'
........>startVertexCollectionRestriction:'germanCity'});
Arouteplannerexample,shortestpathfromHamburgandColognetoLyon:
arangosh>g._shortestPath([{_id:'germanCity/Cologne'},{_id:'germanCity/Munich'}],'frenchCity/Lyon'
........>{weight:'distance'});
_distanceTo
The_distanceTofunctionreturnsallpathsandtheredistancewithinagraph.
graph._distanceTo(startVertexExample,endVertexExample,options)
Thisfunctionisawrapperofgraph._shortestPath.Itdoesnotreturntheactualpathbutonlythedistancebetweentwovertices.
Examples
Arouteplannerexample,shortestdistancefromallgermantoallfrenchcities:
GraphFunctions
131
arangosh>g._distanceTo({},{},{weight:'distance',endVertexCollectionRestriction:'frenchCity'
........>startVertexCollectionRestriction:'germanCity'});
Arouteplannerexample,shortestdistancefromHamburgandColognetoLyon:
arangosh>g._distanceTo([{_id:'germanCity/Cologne'},{_id:'germanCity/Munich'}],'frenchCity/Lyon'
........>{weight:'distance'});
_absoluteEccentricity
Gettheeccentricityoftheverticesdefinedbytheexamples.
graph._absoluteEccentricity(vertexExample,options)
Parameters
vertexExample(optional)Filterthevertices,seeDefinitionofexamples
startVertexCollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredforsourcevertices.
endVertexCollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredfortargetvertices.
edgeExamples:Filtertheedgestobefollowed,seeDefinitionofexamples
algorithm:Thealgorithmtocalculatetheshortestpaths,possiblevaluesareFloyd-WarshallandDijkstra.
weight:Thenameoftheattributeoftheedgescontainingtheweight.
defaultisusedasweight.Ifnodefaultissuppliedthedefaultwouldbepositiveinfinitysothepathandhencetheeccentricity
cannotbecalculated.
Examples
Arouteplannerexample,theabsoluteeccentricityofalllocations.
arangosh>graph._absoluteEccentricity({});
Arouteplannerexample,theabsoluteeccentricityofalllocations.Thisconsiderstheactualdistances.
arangosh>graph._absoluteEccentricity({},{weight:'distance'});
Arouteplannerexample,theabsoluteeccentricityofallcitiesregardingonlyoutboundpaths.
GraphFunctions
132
arangosh>graph._absoluteEccentricity({},{startVertexCollectionRestriction:'germanCity',
........>direction:'outbound',weight:'distance'});
_eccentricity
Getthenormalizedeccentricityoftheverticesdefinedbytheexamples.
graph._eccentricity(vertexExample,options)
Similarto_absoluteEccentricitybutreturnsanormalizedresult.
Examples
Arouteplannerexample,theeccentricityofalllocations.
arangosh>graph._eccentricity();
Arouteplannerexample,theweightedeccentricity.
arangosh>graph._eccentricity({weight:'distance'});
_absoluteCloseness
Gettheclosenessoftheverticesdefinedbytheexamples.
graph._absoluteCloseness(vertexExample,options)
Parameters
startVertexCollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredforsourcevertices.
endVertexCollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredfortargetvertices.
edgeExamples:Filtertheedgestobefollowed,seeDefinitionofexamples
defaultisusedasweight.Ifnodefaultissuppliedthedefaultwouldbepositiveinfinitysothepathandhencetheclosenesscan
notbecalculated.
Examples
Arouteplannerexample,theabsoluteclosenessofalllocations.
GraphFunctions
133
arangosh>graph._absoluteCloseness({});
Arouteplannerexample,theabsoluteclosenessofalllocations.Thisconsiderstheactualdistances.
arangosh>graph._absoluteCloseness({},{weight:'distance'});
Arouteplannerexample,theabsoluteclosenessofallGermanCitiesregardingonlyoutboundpaths.
arangosh>graph._absoluteCloseness({},{startVertexCollectionRestriction:'germanCity',
........>direction:'outbound',weight:'distance'});
_closeness
Getthenormalizedclosenessofgraphsvertices.
graph._closeness(options)
Similarto_absoluteClosenessbutreturnsanormalizedvalue.
Examples
Arouteplannerexample,thenormalizedclosenessofalllocations.
arangosh>graph._closeness();
Arouteplannerexample,theclosenessofalllocations.Thisconsiderstheactualdistances.
arangosh>graph._closeness({weight:'distance'});
Arouteplannerexample,theclosenessofallcitiesregardingonlyoutboundpaths.
arangosh>graph._closeness({direction:'outbound',weight:'distance'});
_absoluteBetweenness
GraphFunctions
134
Getthebetweennessofallverticesinthegraph.
graph._absoluteBetweenness(vertexExample,options)
Parameters
defaultisusedasweight.Ifnodefaultissuppliedthedefaultwouldbepositiveinfinitysothepathandhencethebetweeness
cannotbecalculated.
Examples
Arouteplannerexample,theabsolutebetweennessofalllocations.
arangosh>graph._absoluteBetweenness({});
Arouteplannerexample,theabsolutebetweennessofalllocations.Thisconsiderstheactualdistances.
arangosh>graph._absoluteBetweenness({weight:'distance'});
{
}
Arouteplannerexample,theabsolutebetweennessofallcitiesregardingonlyoutboundpaths.
arangosh>graph._absoluteBetweenness({direction:'outbound',weight:'distance'});
{
}
_betweenness
Getthenormalizedbetweennessofgraphsvertices.
graph_module._betweenness(options)
Similarto_absoluteBetweenessbutreturnsnormalizedvalues.
Examples
Arouteplannerexample,thebetweennessofalllocations.
arangosh>graph._betweenness();
Arouteplannerexample,thebetweennessofalllocations.Thisconsiderstheactualdistances.
GraphFunctions
135
arangosh>graph._betweenness({weight:'distance'});
Arouteplannerexample,thebetweennessofallcitiesregardingonlyoutboundpaths.
arangosh>graph._betweenness({direction:'outbound',weight:'distance'});
_radius
Gettheradiusofagraph.
`
Parameters
defaultisusedasweight.Ifnodefaultissuppliedthedefaultwouldbepositiveinfinitysothepathandhencetheradiuscan
notbecalculated.
Examples
Arouteplannerexample,theradiusofthegraph.
arangosh>graph._radius();
1
Arouteplannerexample,theradiusofthegraph.Thisconsiderstheactualdistances.
arangosh>graph._radius({weight:'distance'});
1
Arouteplannerexample,theradiusofthegraphregardingonlyoutboundpaths.
arangosh>graph._radius({direction:'outbound',weight:'distance'});
1
_diameter
Getthediameterofagraph.
GraphFunctions
136
graph._diameter(graphName,options)
Parameters
defaultisusedasweight.Ifnodefaultissuppliedthedefaultwouldbepositiveinfinitysothepathandhencetheradiuscan
notbecalculated.
Examples
Arouteplannerexample,thediameterofthegraph.
arangosh>graph._diameter();
1
Arouteplannerexample,thediameterofthegraph.Thisconsiderstheactualdistances.
arangosh>graph._diameter({weight:'distance'});
1
Arouteplannerexample,thediameterofthegraphregardingonlyoutboundpaths.
arangosh>graph._diameter({direction:'outbound',weight:'distance'});
1
GraphFunctions
137
Traversals
ArangoDBprovidesseveralwaystoquerygraphdata.Verysimpleoperationscanbecomposedwiththelow-leveledgemethodsedges,
inEdges,andoutEdgesforedgecollections.Theseworkonnamedandanonymousgraphs.Formorecomplexoperations,ArangoDB
providespredefinedtraversalobjects.
AlsoTraversalshavebeenaddedtoAQL.PleasereadthechapteraboutAQLtraversersalsbeforeyoucontinuereadinghere.M ostofthe
traversalcasesarecoveredbyAQLandwillbeexecutedinanoptimizedway.Onlyifthelogicforyouristoocomplextobedefined
usingAQLfiltersyoucanusethetraversalobjectdefinedherewhichgivesyoucompleteprogrammaticaccesstothedata.
Foranyofthefollowingexamples,we'llbeusingtheexamplecollectionsvande,populatedwithcontinents,countriesandcapitalsdata
listedbelow(seeExampleData).
StartingfromScratch
ArangoDBprovidestheedges,inEdges,andoutEdgesmethodsforedgecollections.Thesemethodscanbeusedtoquicklydetermineifa
vertexisconnectedtoothervertices,andwhich.ThisfunctionalitycanbeexploitedtowriteverysimplegraphqueriesinJavaScript.
Forexample,todeterminewhichedgesarelinkedtotheworldvertex,wecanuseinEdges:
db.e.inEdges('v/world').forEach(function(edge){
require("@arangodb").print(edge._from,"->",edge.type,"->",edge._to);
});
inEdgeswillgiveusallingoingedgesforthespecifiedvertexv/world.TheresultisaJavaScriptarraythatwecaniterateoverandprint
theresults:
v/continent-africa->is-in->v/world
v/continent-south-america->is-in->v/world
v/continent-asia->is-in->v/world
v/continent-australia->is-in->v/world
v/continent-europe->is-in->v/world
v/continent-north-america->is-in->v/world
Note:edges,inEdges,andoutEdgesreturnanarrayofedges.Ifwewanttoretrievethelinkedvertices,wecanuseeachedges'_fromand
_toattributesasfollows:
db.e.inEdges('v/world').forEach(function(edge){
require("@arangodb").print(db._document(edge._from).name,"->",edge.type,"->",db._document(edge._to).name);
});
Weareusingthedocumentmethodfromthedbobjecttoretrievetheconnectedverticesnow.
Whilethismaybesufficientforone-levelgraphoperations,writingatraversalbyhandmaybecometoocomplexformulti-level
traversals.
Traversals
138
Gettingstarted
Touseatraversalobject,wefirstneedtorequirethetraversalmodule:
vartraversal=require("@arangodb/graph/traversal");
varexamples=require("@arangodb/graph-examples/example-graph.js");
examples.loadGraph("worldCountry");
Wethenneedtosetupaconfigurationforthetraversalanddetermineatwhichvertextostartthetraversal:
varconfig={
datasource:traversal.generalGraphDatasourceFactory("worldCountry"),
strategy:"depthfirst",
order:"preorder",
filter:traversal.visitAllFilter,
expander:traversal.inboundExpander,
maxDepth:1
};
varstartVertex=db._document("v/world");
Note:ThestartVertexneedstobeadocument,notonlyadocumentid.
Wecanthencreateatraverserandstartthetraversalbycallingitstraversemethod.Notethattraverseneedsaresultobject,whichitcan
modifyinplace:
varresult={
visited:{
vertices:[],
paths:[]
}
};
vartraverser=newtraversal.Traverser(config);
traverser.traverse(result,startVertex);
Finally,wecanprintthecontentsoftheresultsobject,limitedtothevisitedvertices.Wewillonlyprintthenameandtypeofeach
visitedvertexforbrevity:
require("@arangodb").print(result.visited.vertices.map(function(vertex){
returnvertex.name+"("+vertex.type+")";
}));
Thefullscript,whichincludesallstepscarriedoutsofaristhus:
139
varconfig={
datasource:traversal.generalGraphDatasourceFactory("worldCountry"),
order:"preorder",
maxDepth:1
};
varstartVertex=db._document("v/world");
varresult={
visited:{
vertices:[],
paths:[]
}
};
}));
Theresultisanarrayofverticesthatwerevisitedduringthetraversal,startingatthestartvertex(i.e.v/worldinourexample):
[
"World(root)",
"Africa(continent)",
"Asia(continent)",
"Australia(continent)",
"Europe(continent)",
"NorthAmerica(continent)",
"SouthAmerica(continent)"
]
Note:Theresultislimitedtoverticesdirectlyconnectedtothestartvertex.WeachievedthisbysettingthemaxDepthattributeto1.Not
settingitwouldreturnthefullarrayofvertices.
TraversalDirection
Fortheexamplescontainedinthismanual,we'llbestartingthetraversalsatvertexv/world.Verticesinourgraphareconnectedlikethis:
v/world<-is-in<-continent(Africa)<-is-in<-country(Algeria)<-is-in<-capital(Algiers)
Togetanymeaningfulresults,wemusttraversethegraphininboundorder.Thismeans,we'llbefollowingallincomingedgesoftoa
vertex.Inthetraversalconfiguration,wehavespecifiedthisviatheexpanderattribute:
varconfig={
...
expander:traversal.inboundExpander
};
Forothergraphs,wemightwanttotraverseviatheoutgoingedges.Forthis,wecanusetheoutboundExpander.Thereisalsoan
anyExpander,whichwillfollowbothoutgoingandincomingedges.Thisshouldbeusedwithcareandthetraversalshouldalwaysbe
limitedtoamaximumnumberofiterations(e.g.usingthemaxIterationsattribute)inordertoterminateatsomepoint.
Toinvokethedefaultoutboundexpanderforagraph,simplyusethepredefinedfunction:
varconfig={
...
expander:traversal.outboundExpander
};
140
Pleasenotetheoutboundexpanderwillnotproduceanyoutputfortheexamplesifwestillstartthetraversalatthev/worldvertex.
Still,wecanusetheoutboundexpanderifwestartsomewhereelseinthegraph,e.g.
varconfig={
datasource:traversal.generalGraphDatasourceFactory("world_graph"),
order:"preorder",
expander:traversal.outboundExpander
};
varstartVertex=db._document("v/capital-algiers");
varresult={
visited:{
vertices:[],
paths:[]
}
};
}));
Theresultis:
[
"Algiers(capital)",
"Algeria(country)",
"World(root)"
]
whichconfirmsthatnowwe'regoingoutbound.
TraversalStrategy
Depth-firsttraversals
Thevisitationorderofverticesisdeterminedbythestrategyandorderattributessetintheconfiguration.Wechosedepthfirstand
preorder,meaningthetraverserwillvisiteachvertexbeforehandlingconnectededges(pre-order),anddescendintoanyconnectededges
beforeprocessingotherverticesonthesamelevel(depth-first).
Let'sremovethemaxDepthattributenow.We'llnowbegettingallvertices(directlyandindirectlyconnectedtothestartvertex):
141
varconfig={
order:"preorder",
expander:traversal.inboundExpander
};
varresult={
visited:{
vertices:[],
paths:[]
}
};
}));
Theresultwillbealongerarray,assembledindepth-first,pre-orderorder.Foreachcontinentfound,thetraverserwilldescendintolinked
countries,andthenintothelinkedcapital:
[
"World(root)",
"Algeria(country)",
"Algiers(capital)",
"Angola(country)",
"Luanda(capital)",
"Botswana(country)",
"Gaborone(capital)",
"BurkinaFaso(country)",
"Ouagadougou(capital)",
...
]
Let'sswitchtheorderattributefrompreordertopostorder.Thiswillmakethetraverservisitverticesafterallconnectedverticeswere
visited(i.e.mostdistantverticeswillbeemittedfirst):
[
"Algiers(capital)",
"Algeria(country)",
"Luanda(capital)",
"Angola(country)",
"Gaborone(capital)",
"Botswana(country)",
"Ouagadougou(capital)",
"Bujumbura(capital)",
"Burundi(country)",
"Yaounde(capital)",
"Cameroon(country)",
"N'Djamena(capital)",
"Chad(country)",
"Yamoussoukro(capital)",
"Coted'Ivoire(country)",
"Cairo(capital)",
"Egypt(country)",
"Asmara(capital)",
"Eritrea(country)",
...
]
Breadth-firsttraversals
142
Ifwegobacktopreorder,butchangethestrategytobreadth-firstandre-runthetraversal,we'llseethatthereturnorderchanges,and
itemsonthesamelevelwillbereturnedadjacently:
[
"World(root)",
"Asia(continent)",
"SouthAmerica(continent)",
"Burundi(country)",
"Cameroon(country)",
"Chad(country)",
"Algeria(country)",
"Angola(country)",
...
]
Note:Theorderofitemsreturnedforthesamelevelisundefined.Thisisbecausethereisnonaturalorderofedgesforavertexwith
multipleconnectededges.Toexplicitlysettheorderforedgesonthesamelevel,youcanspecifyanedgecomparatorfunctionwiththe
sortattribute:
varconfig={
...
sort:function(l,r){returnl._key<r._key?1:-1;}
...
};
Theargumentslandrareedgedocuments.Thiswilltraverseedgesofthesamevertexinbackward_keyorder:
[
"World(root)",
"SouthAmerica(continent)",
"Asia(continent)",
"Ecuador(country)",
"Colombia(country)",
"Chile(country)",
"Brazil(country)",
"Bolivia(country)",
"Argentina(country)",
...
]
Note:Thisattributeonlyworksfortheusualexpanderstraversal.inboundExpander,traversal.outboundExpander,
traversal.anyExpanderandtheircorresponding"WithLabels"variants.Ifyouareusingcustomexpandersyouhavetoorganizethe
sortingwithinthespecifiedexpander.
WritingCustomVisitors
Sofarwehaveusedmuchofthetraverser'sdefaultfunctions.Thetraverserisveryconfigurableandmanyofthedefaultfunctionscanbe
overriddenwithcustomfunctionality.
Forexample,wehavebeenusingthedefaultvisitorfunction(whichisalwaysusediftheconfigurationdoesnotcontainthevisitor
attribute).Thedefaultvisitorfunctioniscalledforeachvertexinatraversal,andwillpushitintotheresult.Thisisthereasonwhythe
resultvariablelookeddifferentafterthetraversal,andneededtobeinitializedbeforethetraversalwasstarted.
Notethatthedefaultvisitor(named trackingVisitor)willaddeveryvisitedvertexintotheresult,includingthefullpathsfromthestart
vertex.Thisisusefulforlearninganddebuggingpurposes,butshouldbeavoidedinproductionbecauseitmightproduce(andcopy)huge
amountsofdata.Instead,onlythosedatashouldbecopiedintotheresultthatareactuallynecessary.
143
Thetraversercomeswiththefollowingpredefinedvisitors:
trackingVisitor:thisisthedefaultvisitor.Itwillcopyalldataofeachvisitedvertexplusthefullpathinformationintotheresult.
Thiscanbeslowiftheresultsetishugeorverticescontainalotofdata.
countingVisitor:thisisaverylightweightvisitor:allitdoesisincreaseacounterintheresultforeachvertexvisited.Vertexdataand
pathswillnotbecopiedintotheresult.
doNothingVisitor:ifnoactionshallbecarriedoutwhenavertexisvisited,thisvisitorcanbeemployed.Itwillnotdoanythingand
willthusbefast.Itcanbeusedforperformancecomparisonswithothervisitors.
Wecanalsowriteourownvisitorfunctionifwewantto.Thegeneralfunctionsignatureforvisitorfunctionsisasfollows:
varconfig={
...
visitor:function(config,result,vertex,path,connected){...}
};
Note:theconnectedparametervaluewillonlybesetifthetraversalorderissettopreorder-expander.Otherwise,thisparameterwon't
besetbythetraverser.
Visitorfunctionsarenotexpectedtoreturnanyvalues.Instead,theycanmodifytheresultvariable(e.g.bypushingthecurrentvertex
intoit),ordoanythingelse.Forexample,wecancreateasimplevisitorfunctionthatonlyprintsinformationaboutthecurrentvertexas
wetraverse:
varconfig={
order:"preorder",
visitor:function(config,result,vertex,path){
require("@arangodb").print("visitingvertex",vertex.name);
}
};
traverser.traverse(undefined,startVertex);
Towriteavisitorthatincrementsacountereachtimeavertexisvisited,wecouldwritethefollowingcustomvisitor:
config.visitor=function(config,result,vertex,path,connected){
if(!result){
result={};
}
if(!result.hasOwnProperty('count')){
result.count=0;
}
++result.count;
}
Notethatsuchvisitorisalreadypredefined(it'sthecountingVisitordescribedabove).Itcanbeusedasfollows:
config.visitor=traversal.countingVisitor;
Anotherexampleofavisitorisonethatcollectsthe _idvaluesofallverticesvisited:
144
if(!result){
result={};
}
if(!result.hasOwnProperty("visited")){
result.visited={vertices:[]};
}
result.visited.vertices.push(vertex._id);
}
Whenthetraversalorderissettopreorder-expander,thetraverserwillpassafifthparametervalueintothevisitorfunction.This
parametercontainstheconnectededgesofthevisitedvertexasanarray.Thiscanbehandybecauseinthiscasethevisitorwillgetall
informationaboutthevertexandtheconnectededgestogether.
Forexample,thefollowingvisitorcanbeusedtoprintonlyleafnodes(thatdonothaveanyfurtherconnectededges):
if(connected&&connected.length===0){
require("@arangodb").print("foundaleaf-node:",vertex);
}
}
Notethatforthisvisitortowork,thetraversalorderattributeneedstobesettothevaluepreorder-expander.
FilteringVerticesandEdges
FilteringVertices
Sofarwehaveprintedorreturnedallverticesthatwerevisitedduringthetraversal.Thisisnotalwaysrequired.Iftheresultshallbe
restricttojustspecificvertices,wecanuseafilterfunctionforvertices.Itcanbedefinedbysettingthefilterattributeofatraversal
configuration,e.g.:
varconfig={
filter:function(config,vertex,path){
if(vertex.type!=='capital'){
return'exclude';
}
}
}
Theabovefilterfunctionwillexcludeallverticesthatdonothaveatypevalueofcapital.Thefilterfunctionwillbecalledforeachvertex
foundduringthetraversal.Itwillreceivethetraversalconfiguration,thecurrentvertex,andthefullpathfromthetraversalstartvertexto
thecurrentvertex.Thepathconsistsofanarrayofedges,andanarrayofvertices.Wecouldalsofiltereverythingbutcapitalsby
checkingthelengthofthepathfromthestartvertextothecurrentvertex.Capitalswillhaveadistanceof3fromthev/worldstartvertex
(capital→is-in→country→is-in→continent→is-in→world):
varconfig={
...
if(path.edges.length<3){
return'exclude';
}
}
}
Note:Ifafilterfunctionreturnsnothing(orundefined),thecurrentvertexwillbeincluded,andallconnectededgeswillbefollowed.Ifa
filterfunctionreturnsexcludethecurrentvertexwillbeexcludedfromtheresult,andallstillallconnectededgeswillbefollowed.Ifa
filterfunctionreturnsprune,thecurrentvertexwillbeincluded,butnoconnectededgeswillbefollowed.
Forexample,thefollowingfilterfunctionwillnotdescendintoconnectededgesofcontinents,limitingthedepthofthetraversal.Still,
continentverticeswillbeincludedintheresult:
145
varconfig={
...
if(vertex.type==='continent'){
return'prune';
}
}
}
Itisalsopossibletocombineexcludeandprunebyreturninganarraywithbothvalues:
return['exclude','prune'];
FilteringEdges
Itispossibletoexcludecertainedgesfromthetraversal.Tofilteronedges,afilterfunctioncanbedefinedviatheexpandFilterattribute.
TheexpandFilterisafunctionwhichiscalledforeachedgeduringatraversal.
Itwillreceivethecurrentedge(edgevariable)andthevertexwhichtheedgeconnectsto(inthedirectionofthetraversal).Italsoreceives
thecurrentpathfromthestartvertexuptothecurrentvertex(excludingthecurrentedgeandthevertextheedgepointsto).
Ifthefunctionreturnstrue,theedgewillbefollowed.Ifthefunctionreturnsfalse,theedgewillnotbefollowed.Hereisaverysimple
customedgefilterfunctionimplementation,whichsimplyincludesedgesifthe(edges)pathlengthislessthan1,andwillexcludeany
otheredges.Thiswilleffectivelyterminatethetraversalafterthefirstlevelofedges:
varconfig={
...
expandFilter:function(config,vertex,edge,path){
return(path.edges.length<1);
}
};
WritingCustomExpanders
Theedgesconnectedtoavertexaredeterminedbytheexpander.Sofarwehaveusedadefaultexpander(thedefaultinboundexpanderto
beprecise).Thedefaultinboundexpandersimplyenumeratesallconnectedingoingedgesforavertex,basedontheedgecollection
specifiedinthetraversalconfiguration.
Thereisalsoadefaultoutboundexpander,whichwillenumerateallconnectedoutgoingedges.Finally,thereisananyexpander,which
willfollowbothingoingandoutgoingedges.
Ifconnectededgesmustbedeterminedinsomedifferentfashionforwhateverreason,acustomexpandercanbewrittenandregisteredby
settingtheexpanderattributeoftheconfiguration.Theexpanderfunctionsignatureisasfollows:
varconfig={
...
expander:function(config,vertex,path){...}
}
Itistheexpander'sresponsibilitytoreturnalledgesandverticesdirectlyconnectedtothecurrentvertex(whichispassedviathevertex
variable).Thefullpathfromthestartvertexuptothecurrentvertexisalsosuppliedviathepathvariable.Anexpanderisexpectedto
returnanarrayofobjects,whichneedtohaveanedgeandavertexattributeeach.
Note:Ifyouwanttorelyonaparticularorderinwhichtheedgesaretraversed,youhavetosorttheedgesreturnedbyyourexpander
withinthecodeoftheexpander.Thefunctionstogetoutbound,inboundoranyedgesfromavertexdonotguaranteeanyparticularorder!
Acustomimplementationofaninboundexpandercouldlooklikethis(thisisanon-deterministicexpander,whichrandomlydecides
whetherornottoincludeconnectededges):
146
varconfig={
...
expander:function(config,vertex,path){
varconnected=[];
vardatasource=config.datasource;
datasource.getInEdges(vertex._id).forEach(function(edge){
if(Math.random()>=0.5){
connected.push({edge:edge,vertex:(edge._from)});
}
});
returnconnected;
}
};
Acustomexpandercanalsobeusedasanedgefilterbecauseithasfullcontroloverwhichedgeswillbereturned.
Followingaretwoexamplesofcustomexpandersthatpickedgesbasedonattributesoftheedgesandtheconnectedvertices.
Findingtheconnectededges/verticesbasedonanattributewhenintheconnectedvertices.Thegoalistofollowtheedgethatleadstothe
vertexwiththehighestvalueinthewhenattribute:
varconfig={
...
//determinealloutgoingedges
varoutEdges=datasource.getOutEdges(vertex);
if(outEdges.length===0){
return[];
}
vardata=[];
outEdges.forEach(function(edge){
data.push({edge:edge,vertex:datasource.getInVertex(edge)});
});
//sortoutgoingverticesaccordingto"when"attributevalue
data.sort(function(l,r){
if(l.vertex.when===r.vertex.when){
return0;
}
return(l.vertex.when<r.vertex.when?1:-1);
});
//pickfirstvertexfound(withhighest"when"attributevalue)
return[data[0]];
}
...
};
Findingtheconnectededges/verticesbasedonanattributewhenintheedgeitself.Thegoalistopicktheoneedge(outofpotentially
many)thathasthehighestwhenattributevalue:
147
varconfig={
...
//determinealloutgoingedges
varoutEdges=datasource.getOutEdges(vertex);
if(outEdges.length===0){
return[];//returnanemptyarray
}
//sortalloutgoingedgesaccordingto"when"attribute
outEdges.sort(function(l,r){
if(l.when===r.when){
return0;
}
return(l.when<r.when?-1:1);
});
//returnfirstedge(theonewithhighest"when"value)
varedge=outEdges[0];
try{
varv=datasource.getInVertex(edge);
return[{edge:edge,vertex:v}];
}
catch(e){}
return[];
}
...
};
HandlingUniqueness
Graphsmaycontaincycles.Tobeontopofwhathappenswhenatraversalencountersavertexoranedgeithasalreadyvisited,thereare
configurationoptions.
Thedefaultconfigurationistovisiteveryvertex,regardlessofwhetheritwasalreadyvisitedinthesametraversal.However,edgeswill
bydefaultonlybefollowediftheyarenotalreadypresentinthecurrentpath.
Imaginethefollowinggraphwhichcontainsacycle:
A->B->C->A
WhenthetraversalfindstheedgefromCtoA,itwillbydefaultfollowit.Thisisbecausewehavenotseenthisedgeyet.Itwillalsovisit
vertexAagain.Thisisbecausebydefaultallverticeswillbevisited,regardlessofwhetheralreadyvisitedornot.
However,thetraversalwillnotagainfollowingtheoutgoingedgefromAtoB.ThisisbecausewealreadyhavetheedgefromAtoBinour
currentpath.
Thesedefaultsettingswillpreventinfinitetraversals.
Toadjusttheuniquenessforvisitingvertices,therearethefollowingoptionsforuniqueness.vertices:
"none":alwaysvisitavertices,regardlessofwhetheritwasalreadyvisitedornot
"global":visitavertexonlyifitwasnotvisitedinthetraversal
"path":visitavertexifitisnotincludedinthecurrentpath
Toadjusttheuniquenessforfollowingedges,therearethefollowingoptionsforuniqueness.edges:
"none":alwaysfollowanedge,regardlessofwhetheritwasfollowedbefore
"global":followanedgeonlyifitwasn'tfollowedinthetraversal
"path":followanedgeifitisnotincludedinthecurrentpath
Notethatuniquenesscheckingwillhavesomeeffectonbothruntimeandmemoryusage.Forexample,whenuniquenesschecksaresetto
"global",arraysofvisitedverticesandedgesmustbekeptinmemorywhilethetraversalisexecuted.Globaluniquenessshouldthusonly
beusedwhenatraversalisexpectedtovisitfewnodes.
148
Intermsofruntime,turningoffuniquenesschecks(bysettingbothoptionsto"none")isthebestchoice,butitisonlysafeforgraphs
thatdonotcontaincycles.Whenuniquenesschecksaredeactivatedinagraphwithcycles,thetraversalmightnotabortinasensible
amountoftime.
Optimizations
Thereareafewoptionsformakingatraversalrunfaster.
Thebestoptionistomaketheamountofvisitedverticesandfollowededgesassmallaspossible.Thiscanbeachievedbywriting
customfilterandexpanderfunctions.Suchfunctionsshouldonlyincludeverticesofinterest,andonlyfollowedgesthatmightbe
interesting.
TraversaldepthcanalsobeboundedwiththeminDepthandmaxDepthoptions.
Anotherwaytospeeduptraversalsistowriteacustomvisitorfunction.Thedefaultvisitorfunction(trackingVisitor)willcopyevery
visitedvertexintotheresult.Ifverticescontainlotsofdata,thismightbeexpensive.Itisthereforerecommendedtoonlycopysuchdata
intotheresultthatisactuallyneeded.Thedefaultvisitorfunctionwillalsocopythefullpathtothevisiteddocumentintotheresult.
Thisisevenmoreexpensiveandshouldbeavoidedifpossible.
Ifthegoalofatraversalistoonlycountthenumberofvisitedvertices,theprefabcountingVisitor
willbemuchmoreefficientthanthedefaultvisitor.
Forgraphsthatareknowntonotcontainanycycles,uniquenesschecksshouldbeturnedoff.Thiscanachievedviatheuniqueness
configurationoptions.Notethatuniquenesschecksshouldnotbeturnedoffforgraphsthatareknowncontaincyclesorifthereisno
informationaboutthegraph'sstructure.
Bydefault,atraversalwillonlyprocessalimitednumberofvertices.Thisisprotecttheuserfromunintentionallyrunanever-ending
traversalonagraphwithcyclicdata.HowmanyverticeswillbeprocessedatmostisdeterminedbythemaxIterationsconfiguration
option.IfatraversalhitsthecapspecifiedbymaxIterations,itwillabortandthrowatoomanyiterationsexception.Ifthiserroris
encountered,themaxIterationsvalueshouldbeincreasedifitismadesurethattheothertraversalconfigurationparametersaresaneand
thetraversalwillabortnaturallyatsomepoint.
Finally,thebuildVerticesconfigurationoptioncanbesettofalsetoavoidlookingupandfullyconstructingvertexdata.Ifallthat'sneeded
fromverticesarethe_idor_keyattributes,thebuildverticesoptioncanbesettofalse.Ifvisitor,filterorexpandFilterfunctionsneedto
accessothervertexattributes,theoptionshouldnotbechanged.
ConfigurationOverview
Thissectionsummarizestheconfigurationattributesforthetraversalobject.Theconfigurationcanconsistofthefollowingattributes:
visitor:visitorfunctionforvertices.Itwillbecalledforallnon-excludedvertices.Thegeneralvisitorfunctionsignatureisfunction
(config,result,vertex,path).Ifthetraversalorderispreorder-expander,theconnectingedgesofthevisitedvertexwillbepassedas
thefifthparameter,extendingthefunctionsignatureto:function(config,result,vertex,path,edges).
Visitorfunctionsarenotexpectedtoreturnvalues,buttheymaymodifytheresultvariableasneeded(e.g.bypushingvertexdata
intotheresult).
expander:expanderfunctionthatisresponsibleforreturningedgesandverticesdirectlyconnectedtoavertex.Thefunction
signatureisfunction(config,vertex,path).Theexpanderfunctionisrequiredtoreturnanarrayofconnectionobjects,consistingof
anedgeandvertexattributeeach.Iftherearenoconnectingedges,theexpanderisexpectedtoreturnanemptyarray.
filter:vertexfilterfunction.Thefunctionsignatureisfunction(config,vertex,path).Itmayreturnoneofthefollowingvalues:
undefined:vertexwillbeincludedintheresultandconnectededgeswillbetraversed
"exclude":vertexwillnotbeincludedintheresultandconnectededgeswillbetraversed
"prune":vertexwillbeincludedintheresultbutconnectededgeswillnotbetraversed
["prune","exclude"]:vertexwillnotbeincludedintheresultandconnectededgeswillnotbereturned
expandFilter:filterfunctionappliedoneachedge/vertexcombinationdeterminedbytheexpander.Thefunctionsignatureisfunction
(config,vertex,edge,path).Thefunctionshouldreturntrueiftheedge/vertexcombinationshouldbeprocessed,andfalseifitshould
beignored.
sort:afilterfunctiontodeterminetheorderinwhichconnectededgesareprocessed.Thefunctionsignatureisfunction(l,r).The
functionisrequiredtoreturnoneofthefollowingvalues:
-1iflshouldhaveasortvaluelessthanr
149
1iflshouldhaveahighersortvaluethanr
0iflandrhavethesamesortvalue
strategy:determinesthevisitationstrategy.Possiblevaluesaredepthfirstandbreadthfirst.
order:determinesthevisitationorder.Possiblevaluesarepreorder,postorder,andpreorder-expander.preorder-expanderisthe
sameaspreorder,exceptthatthesignatureofthevisitorfunctionwillchangeasdescribedabove.
itemOrder:determinestheorderinwhichconnectionsreturnedbytheexpanderwillbeprocessed.Possiblevaluesareforwardand
backward.
maxDepth:ifsettoavaluegreaterthan0,thiswilllimitthetraversaltothismaximumdepth.
minDepth:ifsettoavaluegreaterthan0,allverticesfoundonalevelbelowtheminDepthlevelwillnotbeincludedintheresult.
maxIterations:themaximumnumberofiterationsthatthetraversalisallowedtoperform.Itissensibletosetthisnumberso
unboundedtraversalswillterminateatsomepoint.
uniqueness:anobjectthatdefineshowrepeatedvisitationsofverticesshouldbehandled.Theuniquenessobjectcanhaveasubattributevertices,andasub-attributeedges.Eachsub-attributecanhaveoneofthefollowingvalues:
"none":nouniquenessconstraints
"path":elementisexcludedifitisalreadycontainedinthecurrentpath.Thissettingmaybesensibleforgraphsthatcontain
cycles(e.g.A→B→C→A).
"global":elementisexcludedifitwasalreadyfound/visitedatanypointduringthetraversal.
buildVertices:thisattributecontrolswhetherverticesencounteredduringthetraversalwillbelookedupinthedatabaseandwillbe
madeavailabletovisitor,filter,andexpandFilterfunctions.Bydefault,verticeswillbelookedupandmadeavailable.However,
therearesomespecialusecaseswhenfullyconstructingvertexobjectsisnotnecessaryandcanbeavoided.Forexample,ifa
traversalismeanttoonlycountthenumberofvisitedverticesbutdonotreadanydatafromvertices,thisoptionmightbesetto
true.
150
ExampleData
Thefollowingexamplesalluseavertexcollectionvandanedgecollectione.Thevertexcollectionvcontainscontinents,countries,and
capitals.Theedgecollectionecontainsconnectionsbetweencontinentsandcountries,andbetweencountriesandcapitals.
Tosetupthecollectionsandpopulatethemwithinitialdata,thefollowingscriptwasused:
db._create("v");
db._createEdgeCollection("e");
//vertices:rootnode
db.v.save({_key:"world",name:"World",type:"root"});
//vertices:continents
db.v.save({_key:"continent-africa",name:"Africa",type:"continent"});
db.v.save({_key:"continent-asia",name:"Asia",type:"continent"});
db.v.save({_key:"continent-australia",name:"Australia",type:"continent"});
db.v.save({_key:"continent-europe",name:"Europe",type:"continent"});
db.v.save({_key:"continent-north-america",name:"NorthAmerica",type:"continent"});
db.v.save({_key:"continent-south-america",name:"SouthAmerica",type:"continent"});
//vertices:countries
db.v.save({_key:"country-afghanistan",name:"Afghanistan",type:"country",code:"AFG"});
db.v.save({_key:"country-albania",name:"Albania",type:"country",code:"ALB"});
db.v.save({_key:"country-algeria",name:"Algeria",type:"country",code:"DZA"});
db.v.save({_key:"country-andorra",name:"Andorra",type:"country",code:"AND"});
db.v.save({_key:"country-angola",name:"Angola",type:"country",code:"AGO"});
db.v.save({_key:"country-antigua-and-barbuda",name:"AntiguaandBarbuda",type:"country",code:"ATG"});
db.v.save({_key:"country-argentina",name:"Argentina",type:"country",code:"ARG"});
db.v.save({_key:"country-australia",name:"Australia",type:"country",code:"AUS"});
db.v.save({_key:"country-austria",name:"Austria",type:"country",code:"AUT"});
db.v.save({_key:"country-bahamas",name:"Bahamas",type:"country",code:"BHS"});
db.v.save({_key:"country-bahrain",name:"Bahrain",type:"country",code:"BHR"});
db.v.save({_key:"country-bangladesh",name:"Bangladesh",type:"country",code:"BGD"});
db.v.save({_key:"country-barbados",name:"Barbados",type:"country",code:"BRB"});
db.v.save({_key:"country-belgium",name:"Belgium",type:"country",code:"BEL"});
db.v.save({_key:"country-bhutan",name:"Bhutan",type:"country",code:"BTN"});
db.v.save({_key:"country-bolivia",name:"Bolivia",type:"country",code:"BOL"});
db.v.save({_key:"country-bosnia-and-herzegovina",name:"BosniaandHerzegovina",type:"country",code:"BIH"});
db.v.save({_key:"country-botswana",name:"Botswana",type:"country",code:"BWA"});
db.v.save({_key:"country-brazil",name:"Brazil",type:"country",code:"BRA"});
db.v.save({_key:"country-brunei",name:"Brunei",type:"country",code:"BRN"});
db.v.save({_key:"country-bulgaria",name:"Bulgaria",type:"country",code:"BGR"});
db.v.save({_key:"country-burkina-faso",name:"BurkinaFaso",type:"country",code:"BFA"});
db.v.save({_key:"country-burundi",name:"Burundi",type:"country",code:"BDI"});
db.v.save({_key:"country-cambodia",name:"Cambodia",type:"country",code:"KHM"});
db.v.save({_key:"country-cameroon",name:"Cameroon",type:"country",code:"CMR"});
db.v.save({_key:"country-canada",name:"Canada",type:"country",code:"CAN"});
db.v.save({_key:"country-chad",name:"Chad",type:"country",code:"TCD"});
db.v.save({_key:"country-chile",name:"Chile",type:"country",code:"CHL"});
db.v.save({_key:"country-colombia",name:"Colombia",type:"country",code:"COL"});
db.v.save({_key:"country-cote-d-ivoire",name:"Coted'Ivoire",type:"country",code:"CIV"});
db.v.save({_key:"country-croatia",name:"Croatia",type:"country",code:"HRV"});
db.v.save({_key:"country-czech-republic",name:"CzechRepublic",type:"country",code:"CZE"});
db.v.save({_key:"country-denmark",name:"Denmark",type:"country",code:"DNK"});
db.v.save({_key:"country-ecuador",name:"Ecuador",type:"country",code:"ECU"});
db.v.save({_key:"country-egypt",name:"Egypt",type:"country",code:"EGY"});
db.v.save({_key:"country-eritrea",name:"Eritrea",type:"country",code:"ERI"});
db.v.save({_key:"country-finland",name:"Finland",type:"country",code:"FIN"});
db.v.save({_key:"country-france",name:"France",type:"country",code:"FRA"});
db.v.save({_key:"country-germany",name:"Germany",type:"country",code:"DEU"});
db.v.save({_key:"country-people-s-republic-of-china",name:"People'sRepublicofChina",type:"country",code:"CHN"});
//vertices:capitals
db.v.save({_key:"capital-algiers",name:"Algiers",type:"capital"});
db.v.save({_key:"capital-andorra-la-vella",name:"AndorralaVella",type:"capital"});
db.v.save({_key:"capital-asmara",name:"Asmara",type:"capital"});
db.v.save({_key:"capital-bandar-seri-begawan",name:"BandarSeriBegawan",type:"capital"});
db.v.save({_key:"capital-beijing",name:"Beijing",type:"capital"});
db.v.save({_key:"capital-berlin",name:"Berlin",type:"capital"});
db.v.save({_key:"capital-bogota",name:"Bogota",type:"capital"});
ExampleData
151
db.v.save({_key:"capital-brasilia",name:"Brasilia",type:"capital"});
db.v.save({_key:"capital-bridgetown",name:"Bridgetown",type:"capital"});
db.v.save({_key:"capital-brussels",name:"Brussels",type:"capital"});
db.v.save({_key:"capital-buenos-aires",name:"BuenosAires",type:"capital"});
db.v.save({_key:"capital-bujumbura",name:"Bujumbura",type:"capital"});
db.v.save({_key:"capital-cairo",name:"Cairo",type:"capital"});
db.v.save({_key:"capital-canberra",name:"Canberra",type:"capital"});
db.v.save({_key:"capital-copenhagen",name:"Copenhagen",type:"capital"});
db.v.save({_key:"capital-dhaka",name:"Dhaka",type:"capital"});
db.v.save({_key:"capital-gaborone",name:"Gaborone",type:"capital"});
db.v.save({_key:"capital-helsinki",name:"Helsinki",type:"capital"});
db.v.save({_key:"capital-kabul",name:"Kabul",type:"capital"});
db.v.save({_key:"capital-la-paz",name:"LaPaz",type:"capital"});
db.v.save({_key:"capital-luanda",name:"Luanda",type:"capital"});
db.v.save({_key:"capital-manama",name:"Manama",type:"capital"});
db.v.save({_key:"capital-nassau",name:"Nassau",type:"capital"});
db.v.save({_key:"capital-n-djamena",name:"N'Djamena",type:"capital"});
db.v.save({_key:"capital-ottawa",name:"Ottawa",type:"capital"});
db.v.save({_key:"capital-ouagadougou",name:"Ouagadougou",type:"capital"});
db.v.save({_key:"capital-paris",name:"Paris",type:"capital"});
db.v.save({_key:"capital-phnom-penh",name:"PhnomPenh",type:"capital"});
db.v.save({_key:"capital-prague",name:"Prague",type:"capital"});
db.v.save({_key:"capital-quito",name:"Quito",type:"capital"});
db.v.save({_key:"capital-saint-john-s",name:"SaintJohn's",type:"capital"});
db.v.save({_key:"capital-santiago",name:"Santiago",type:"capital"});
db.v.save({_key:"capital-sarajevo",name:"Sarajevo",type:"capital"});
db.v.save({_key:"capital-sofia",name:"Sofia",type:"capital"});
db.v.save({_key:"capital-thimphu",name:"Thimphu",type:"capital"});
db.v.save({_key:"capital-tirana",name:"Tirana",type:"capital"});
db.v.save({_key:"capital-vienna",name:"Vienna",type:"capital"});
db.v.save({_key:"capital-yamoussoukro",name:"Yamoussoukro",type:"capital"});
db.v.save({_key:"capital-yaounde",name:"Yaounde",type:"capital"});
db.v.save({_key:"capital-zagreb",name:"Zagreb",type:"capital"});
//edges:continent->world
db.e.save("v/continent-africa","v/world",{type:"is-in"});
db.e.save("v/continent-asia","v/world",{type:"is-in"});
db.e.save("v/continent-australia","v/world",{type:"is-in"});
db.e.save("v/continent-europe","v/world",{type:"is-in"});
db.e.save("v/continent-north-america","v/world",{type:"is-in"});
db.e.save("v/continent-south-america","v/world",{type:"is-in"});
//edges:country->continent
db.e.save("v/country-afghanistan","v/continent-asia",{type:"is-in"});
db.e.save("v/country-albania","v/continent-europe",{type:"is-in"});
db.e.save("v/country-algeria","v/continent-africa",{type:"is-in"});
db.e.save("v/country-andorra","v/continent-europe",{type:"is-in"});
db.e.save("v/country-angola","v/continent-africa",{type:"is-in"});
db.e.save("v/country-antigua-and-barbuda","v/continent-north-america",{type:"is-in"});
db.e.save("v/country-argentina","v/continent-south-america",{type:"is-in"});
db.e.save("v/country-australia","v/continent-australia",{type:"is-in"});
db.e.save("v/country-austria","v/continent-europe",{type:"is-in"});
db.e.save("v/country-bahamas","v/continent-north-america",{type:"is-in"});
db.e.save("v/country-bahrain","v/continent-asia",{type:"is-in"});
db.e.save("v/country-bangladesh","v/continent-asia",{type:"is-in"});
db.e.save("v/country-barbados","v/continent-north-america",{type:"is-in"});
db.e.save("v/country-belgium","v/continent-europe",{type:"is-in"});
db.e.save("v/country-bhutan","v/continent-asia",{type:"is-in"});
db.e.save("v/country-bolivia","v/continent-south-america",{type:"is-in"});
db.e.save("v/country-bosnia-and-herzegovina","v/continent-europe",{type:"is-in"});
db.e.save("v/country-botswana","v/continent-africa",{type:"is-in"});
db.e.save("v/country-brazil","v/continent-south-america",{type:"is-in"});
db.e.save("v/country-brunei","v/continent-asia",{type:"is-in"});
db.e.save("v/country-bulgaria","v/continent-europe",{type:"is-in"});
db.e.save("v/country-burkina-faso","v/continent-africa",{type:"is-in"});
db.e.save("v/country-burundi","v/continent-africa",{type:"is-in"});
db.e.save("v/country-cambodia","v/continent-asia",{type:"is-in"});
db.e.save("v/country-cameroon","v/continent-africa",{type:"is-in"});
db.e.save("v/country-canada","v/continent-north-america",{type:"is-in"});
db.e.save("v/country-chad","v/continent-africa",{type:"is-in"});
db.e.save("v/country-chile","v/continent-south-america",{type:"is-in"});
db.e.save("v/country-colombia","v/continent-south-america",{type:"is-in"});
db.e.save("v/country-cote-d-ivoire","v/continent-africa",{type:"is-in"});
db.e.save("v/country-croatia","v/continent-europe",{type:"is-in"});
db.e.save("v/country-czech-republic","v/continent-europe",{type:"is-in"});
db.e.save("v/country-denmark","v/continent-europe",{type:"is-in"});
ExampleData
152
db.e.save("v/country-ecuador","v/continent-south-america",{type:"is-in"});
db.e.save("v/country-egypt","v/continent-africa",{type:"is-in"});
db.e.save("v/country-eritrea","v/continent-africa",{type:"is-in"});
db.e.save("v/country-finland","v/continent-europe",{type:"is-in"});
db.e.save("v/country-france","v/continent-europe",{type:"is-in"});
db.e.save("v/country-germany","v/continent-europe",{type:"is-in"});
db.e.save("v/country-people-s-republic-of-china","v/continent-asia",{type:"is-in"});
//edges:capital->country
db.e.save("v/capital-algiers","v/country-algeria",{type:"is-in"});
db.e.save("v/capital-andorra-la-vella","v/country-andorra",{type:"is-in"});
db.e.save("v/capital-asmara","v/country-eritrea",{type:"is-in"});
db.e.save("v/capital-bandar-seri-begawan","v/country-brunei",{type:"is-in"});
db.e.save("v/capital-beijing","v/country-people-s-republic-of-china",{type:"is-in"});
db.e.save("v/capital-berlin","v/country-germany",{type:"is-in"});
db.e.save("v/capital-bogota","v/country-colombia",{type:"is-in"});
db.e.save("v/capital-brasilia","v/country-brazil",{type:"is-in"});
db.e.save("v/capital-bridgetown","v/country-barbados",{type:"is-in"});
db.e.save("v/capital-brussels","v/country-belgium",{type:"is-in"});
db.e.save("v/capital-buenos-aires","v/country-argentina",{type:"is-in"});
db.e.save("v/capital-bujumbura","v/country-burundi",{type:"is-in"});
db.e.save("v/capital-cairo","v/country-egypt",{type:"is-in"});
db.e.save("v/capital-canberra","v/country-australia",{type:"is-in"});
db.e.save("v/capital-copenhagen","v/country-denmark",{type:"is-in"});
db.e.save("v/capital-dhaka","v/country-bangladesh",{type:"is-in"});
db.e.save("v/capital-gaborone","v/country-botswana",{type:"is-in"});
db.e.save("v/capital-helsinki","v/country-finland",{type:"is-in"});
db.e.save("v/capital-kabul","v/country-afghanistan",{type:"is-in"});
db.e.save("v/capital-la-paz","v/country-bolivia",{type:"is-in"});
db.e.save("v/capital-luanda","v/country-angola",{type:"is-in"});
db.e.save("v/capital-manama","v/country-bahrain",{type:"is-in"});
db.e.save("v/capital-nassau","v/country-bahamas",{type:"is-in"});
db.e.save("v/capital-n-djamena","v/country-chad",{type:"is-in"});
db.e.save("v/capital-ottawa","v/country-canada",{type:"is-in"});
db.e.save("v/capital-ouagadougou","v/country-burkina-faso",{type:"is-in"});
db.e.save("v/capital-paris","v/country-france",{type:"is-in"});
db.e.save("v/capital-phnom-penh","v/country-cambodia",{type:"is-in"});
db.e.save("v/capital-prague","v/country-czech-republic",{type:"is-in"});
db.e.save("v/capital-quito","v/country-ecuador",{type:"is-in"});
db.e.save("v/capital-saint-john-s","v/country-antigua-and-barbuda",{type:"is-in"});
db.e.save("v/capital-santiago","v/country-chile",{type:"is-in"});
db.e.save("v/capital-sarajevo","v/country-bosnia-and-herzegovina",{type:"is-in"});
db.e.save("v/capital-sofia","v/country-bulgaria",{type:"is-in"});
db.e.save("v/capital-thimphu","v/country-bhutan",{type:"is-in"});
db.e.save("v/capital-tirana","v/country-albania",{type:"is-in"});
db.e.save("v/capital-vienna","v/country-austria",{type:"is-in"});
db.e.save("v/capital-yamoussoukro","v/country-cote-d-ivoire",{type:"is-in"});
db.e.save("v/capital-yaounde","v/country-cameroon",{type:"is-in"});
db.e.save("v/capital-zagreb","v/country-croatia",{type:"is-in"});
ExampleData
153
Edges,Identifiers,Handles
ThisisanintroductiontoArangoDB'sinterfaceforedges.Edgesmaybeusedingraphs.HereweworkwithedgesfromtheJavaScript
shellarangosh.ForotherlanguagesseethecorrespondinglanguageAPI.
Agraphdatamodelalwaysconsistsofatleasttwocollections:therelationsbetweenthenodesinthegraphsarestoredinan"edges
collection",thenodesinthegrapharestoredindocumentsinregularcollections.
EdgesinArangoDBarespecialdocuments.Inadditiontothesystemattributes_key,_idand_rev,theyhavetheattributes_fromand
_to,whichcontaindocumenthandles,namelythestart-pointandtheend-pointoftheedge.
Example:
the"edge"collectionstorestheinformationthatacompany'sreceptionissub-unittotheservicesunitandtheservicesunitissubunittotheCEO.Youwouldexpressthisrelationshipwiththe_fromand_toattributes
the"normal"collectionstoresallthepropertiesaboutthereception,e.g.that20peopleareworkingthereandtheroomnumberetc
_fromisthedocumenthandleofthelinkedvertex(incomingrelation)
_toisthedocumenthandleofthelinkedvertex(outgoingrelation)
Edgecollectionsarespecialcollectionsthatstoreedgedocuments.Edgedocumentsareconnectiondocumentsthatreferenceother
documents.Thetypeofacollectionmustbespecifiedwhenacollectioniscreatedandcannotbechangedafterwards.
Tochangeedgeendpointsyouwouldneedtoremoveolddocument/edgeandinsertnewone.Otherfieldscanbeupdatedasindefault
collection.
WorkingwithEdges
Edgesarenormaldocumentsthatalwayscontaina _fromanda _toattribute.
WorkingwithEdges
154
Foxx
Traditionally,server-sideprojectshavebeendevelopedasstandaloneapplicationsthatguidethecommunicationbetweentheclient-side
frontendandthedatabasebackend.Thishasledtoapplicationsthatwereeitherdevelopedassinglemonolithsorthatduplicateddata
accessanddomainlogicacrossallservicesthathadtoaccessthedatabase.Additionally,toolstoabstractawaytheunderlyingdatabase
callscouldincuralotofnetworkoverheadwhenusingremotedatabaseswithoutcarefuloptimization.
ArangoDBallowsapplicationdeveloperstowritetheirdataaccessanddomainlogicasmicroservicesrunningdirectlywithinthedatabase
withnativeaccesstoin-memorydata.TheFoxxmicroserviceframeworkmakesiteasytoextendArangoDB'sownRESTAPIwith
customHTTPendpointsusingmodernJavaScriptrunningonthesameV8engineyouknowfromNode.jsandtheGoogleChromeweb
browser.
Unliketraditionalapproachestostoringlogicinthedatabase(likestoredprocedures),thesemicroservicescanbewrittenasregular
structuredJavaScriptapplicationsthatcanbeeasilydistributedandversioncontrolled.Dependingonyourproject'sneedsFoxxcanbe
usedtobuildanythingfromoptimizedRESTendpointsperformingcomplexdataaccesstoentirestandaloneapplicationsrunning
directlyinsidethedatabase.
FoxxM icroservices
155
Foxxataglance
EachFoxxserviceisdefinedbyaJSONmanifestspecifyingtheentrypoint,anyscriptsdefinedbytheservice,possibleconfiguration
optionsandFoxxdependencies,aswellasothermetadata.Withinaservice,theseoptionsareexposedastheservicecontext.
AttheheartoftheFoxxframeworkliestheFoxxRouterwhichisusedtodefineHTTPendpoints.Aservicecanaccessthedatabase
eitherdirectlyfromitscontextusingprefixedcollectionsortheArangoDBdatabaseAPI.
WhileFoxxisprimarilydesignedtobeusedtoaccessthedatabaseitself,ArangoDBalsoprovidesanAPItomakeHTTPrequeststo
externalservices.
Finally,scriptscanbeusedtoperformone-offtasks,whichcanalsobescheduledtobeperformedasynchronouslyusingthebuilt-injob
queue.
Howdoesitwork
FoxxservicesconsistofJavaScriptcoderunningintheV8JavaScriptruntimeembeddedinsideArangoDB.Eachserviceismountedin
eachavailableV8context(thenumberofcontextscanbeadjustedintheArangoDBconfiguration).Incomingrequestsaredistributed
accrossthesecontextsautomatically.
Ifyou'recomingfromanotherJavaScriptenvironmentlikeNode.jsthisissimilartorunningmultipleNode.jsprocessesbehindaload
balancer:youshouldnotrelyonserver-sidestate(otherthanthedatabaseitself)betweendifferentrequestsasthereisnowayofmaking
sureconsecutiverequestswillbehandledinthesamecontext.
BecausetheJavaScriptcodeisrunninginsidethedatabaseanotherdifferenceisthatallFoxxandArangoDBAPIsarepurelysynchronous
andshouldbeconsideredblocking.Thisisespeciallyimportantfortransactions,whichinArangoDBcanexecutearbitrarycodebutmay
havetolockentirecollections(effectivelypreventinganydatatobewritten)untilthecodehascompleted.
Forinformationonhowthisaffectsinteroperabilitywiththird-partyJavaScriptmoduleswrittenforotherJavaScriptenvironmentssee
thechapterondependencies.
Developmentmode
Developmentmodeallowsyoutomakechangestodeployedservicesin-placedirectlyonthedatabaseserver'sfilesystemwithout
downloadingandre-uploadingtheservicebundle.
Youcantoggledevelopmentmodeonandoffintheservicesettingstabofthewebinterface.Onceactivatedtheservice'sfilesystempath
willbeshownintheinfotab.
Onceenabledtheservice'ssourcefilesandmanifestwillbere-evaluatedeverytimearouteoftheserviceisaccessed,effectivelyredeployingtheserviceoneveryrequest.Asthenameindicatesthisisintendedtobeusedstrictlyduringdevelopmentandismost
definitelyabadideaonproductionservers.
Alsonotethatifyouareservingstaticfilesaspartofyourservice,accessingthesefilesfromabrowsermayalsotriggerare-deployment
oftheservice.Finally,makingHTTPrequeststoaservicerunningindevelopmentmodefromwithintheservice(i.e.usingthe
@arangodb/requestmoduletoaccesstheserviceitself)isprobablynotagoodideaeither.
Bewareofdeletingthedatabasetheserviceisdeployedon:itwillerasethesourcefilesoftheservicealongwiththecollections.You
shouldbackupthecodeyouworkedonindevelopmentbeforedoingthattoavoidlosingyourprogress.
Foxxstore
TheFoxxstoreprovidesaccesstoanumberofready-to-useofficialandcommunity-maintainedFoxxservicesyoucaninstallwithasingle
click,includingexampleservicesandwrappersforexternalSaaStoolsliketransactionale-mailservices,bugloggersoranalyticstrackers.
YoucanfindtheFoxxstoreinthewebinterfacebyusingtheAddServicebuttonintheservicelist.
Ataglance
156
Cluster-Foxx
WhenrunningArangoDBinaclustertheFoxxserviceswillrunoneachcoordinator.Installing,upgradinganduninstallingservicesonany
coordinatorwillautomaticallyaffecttheothercoordinators,makingdeploymentsaseasyasinsingle-servermode.However,thismeans
therearesomelimitations:
Youshouldavoidanykindoffilesystemstatebeyondthedeployedservicebundleitself.Don'twritedatatothefilesystemorencode
anyexpectationsofthefilesystemstateotherthanthefilesintheservicefolderthatwereinstalledaspartoftheservice(e.g.fileuploads
orcustomlogfiles).
Additionally,thedevelopmentmodeisnotsupportedinclustermode.Thedevelopmentmodeisintendedtoallowmodifyingthe
service'scodeandseeingtheeffectofthosechangesinrealtime.Theserviceisautomaticallydeployedtomultiplecoordinators,butwith
(temporarily)differentcopiesoftheservice,theinconsistentstatewouldleadtounpredictablebehavior.Itisrecommendedthatyou
eitherre-deployserviceswhenmakingchangestocoderunninginaclusterorusedevelopmentmodeonasingle-serverinstallation.
Ataglance
157
GettingStarted
We'regoingtostartwithanemptyfolder.Thiswillbetherootfolderofourservices.Youcannameitsomethingcleverbutforthecourse
ofthisguidewe'llassumeit'scalledthenameofyourservice: getting-started.
Firstweneedtocreateamanifest.Createanewfilecalled manifest.jsonandaddthefollowingcontent:
{
"engines":{
"arangodb":"^3.0.0"
}
}
ThisjusttellsArangoDBtheserviceiscompatiblewithversions3.0.0andlater(allthewayuptobutnotincluding4.0.0),allowingolder
versionsofArangoDBtounderstandthatthisservicelikelywon'tworkforthemandnewerversionswhatbehaviortoemulateshould
theystillsupportit.
Thelittlehattotheleftoftheversionnumberisnotatypo,it'scalleda"caret"andindicatestheversionrange.Foxxusessemantic
versioning(alsocalled"semver")formostofitsversionhandling.Youcanfindoutmoreabouthowsemverworksattheofficialsemver
website.
Nextwe'llneedtospecifyanentrypointtoourservice.ThisistheJavaScriptfilethatwillbeexecutedtodefineourservice'sHTTP
endpoints.Wecandothisbyaddinga"main"fieldtoourmanifest:
{
"engines":{
"arangodb":"^3.0.0"
},
"main":"index.js"
}
That'sallweneedinourmanifestfornow,solet'snextcreatethe index.jsfile:
'usestrict';
constcreateRouter=require('@arangodb/foxx/router');
constrouter=createRouter();
module.context.use(router);
Thefirstlinecausesourfiletobeinterpretedusingstrictmode.AllexamplesintheArangoDBdocumentationassumestrictmode,so
youmightwanttofamiliarizeyourselfwithitifyouhaven'tencountereditbefore.
Thesecondlineimportsthe @arangodb/foxx/routermodulewhichprovidesafunctionforcreatingnewFoxxrouters.We'reusingthis
functiontocreateanew routerobjectwhichwe'llbeusingforourservice.
The module.contextistheso-calledFoxxcontextorservicecontext.ThisvariableisavailableinallfilesthatarepartofyourFoxx
serviceandprovidesaccesstoFoxxAPIsspecifictothecurrentservice,likethe usemethod,whichtellsFoxxtomountthe routerin
thisservice(andtoexposeitsroutestoHTTP).
Nextlet'sdefinearoutethatprintsagenericgreeting:
//continued
router.get('/hello-world',function(req,res){
res.send('HelloWorld!');
})
.response(['text/plain'],'Agenericgreeting.')
.summary('Genericgreeting')
.description('Printsagenericgreeting.');
The routerprovidesthemethods get, post,etccorrespondingtoeachHTTPverbaswellasthecatch-all all.Thesemethods
indicatethatthegivenrouteshouldbeusedtohandleincomingrequestswiththegivenHTTPverb(oranymethodwhenusing all).
Gettingstarted
158
Thesemethodstakeanoptionalpath(ifomitted,itdefaultsto "/")aswellasarequesthandler,whichisafunctiontakingthe req
(request)and res(response)objectstohandletheincomingrequestandgeneratetheoutgoingresponse.Ifyouhaveusedtheexpress
frameworkinNode.js,youmayalreadybefamiliarwithhowthisworks,otherwisecheckoutthechapteronroutes.
Theobjectreturnedbytherouter'smethodsprovidesadditionalmethodstoattachmetadataandvalidationtotheroute.We'reusing
summaryand descriptiontodocumentwhattheroutedoes--thesearen'tstrictlynecessarybutgiveussomeniceauto-generated
documentation.The responsemethodletsusadditionallydocumenttheresponsecontenttypeandwhattheresponsebodywill
represent.
Tryitout
AtthispointyoucanuploadtheservicefolderasaziparchivefromthewebinterfaceusingtheServicestab.
ClickAddServicethenpicktheZipoptioninthedialog.Youwillneedtoprovideamountpath,whichistheURLprefixatwhichthe
servicewillbemounted(e.g. /getting-started).
Onceyouhavepickedtheziparchiveusingthefilepicker,theuploadshouldbeginimmediatelyandyourserviceshouldbeinstalled.
OtherwisepresstheInstallbuttonandwaitforthedialogtodisappearandtheservicetoshowupintheservicelist.
Clickanywhereonthecardwithyourmountpathonthelabeltoopentheservice'sdetails.
IntheAPIdocumentationyoushouldseetheroutewedefinedearlier( /hello-world)withtheword GETnexttoitindicatingthe
HTTPmethoditsupportsandthe summaryweprovidedontheright.Byclickingontheroute'spathyoucanopenthedocumentation
fortheroute.
Notethatthe descriptionweprovidedappearsinthegenerateddocumentationaswellasthedescriptionweaddedtothe response
(whichshouldcorrectlyindicatethecontenttype text/plain,i.e.plaintext).
ClicktheTryitout!buttontosendarequesttotherouteandyoushouldseeanexamplerequestwiththeservice'sresponse:"Hello
World!".
Congratulations!Youhavejustcreated,installedandusedyourfirstFoxxservice.
Parametervalidation
Let'saddanotherroutethatprovidesamorepersonalizedgreeting:
//continued
constjoi=require('joi');
router.get('/hello/:name',function(req,res){
res.send(`Hello${req.pathParams.name}`);
})
.pathParam('name',joi.string().required(),'Nametogreet.')
.response(['text/plain'],'Apersonalizedgreeting.')
.summary('Personalizedgreeting')
.description('Printsapersonalizedgreeting.');
Thefirstlineimportsthe joimodulefromnpmwhichcomesbundledwithArangoDB.Joiisavalidationlibrarythatisused
throughoutFoxxtodefineschemasandparametertypes.
Note:Youcanbundleyourownmodulesfromnpmbyinstallingtheminyourservicefolderandmakingsurethe node_modulesfolderis
includedinyourziparchive.Formoreinformationseethesectiononmoduledependenciesinthechapterondependencies.
The pathParammethodallowsustospecifyparametersweareexpectinginthepath.Thefirstargumentcorrespondstotheparameter
nameinthepath,thesecondargumentisajoischematheparameterisexpectedtomatchandthefinalargumentservestodescribethe
parameterintheAPIdocumentation.
Thepathparametersareaccessiblefromthe pathParamspropertyoftherequestobject.We'reusingatemplatestringtogeneratethe
server'sresponsecontainingtheparameter'svalue.
Gettingstarted
159
NotethatrouteswithpathparametersthatfailtovalidatefortherequestURLwillbeskippedasiftheywouldn'texist.Thisallowsyou
todefinemultipleroutesthatareonlydistinguishedbytheschemasoftheirpathparameters(e.g.aroutetakingonlynumericparameters
andonetakinganystringasafallback).
Let'stakethisfurtherandcreatearoutethattakesaJSONrequestbody:
//continued
router.post('/sum',function(req,res){
constvalues=req.body.values;
res.send({
result:values.reduce(function(a,b){
returna+b;
},0)
});
})
.body(joi.object({
values:joi.array().items(joi.number().required()).required()
}).required(),'Valuestoaddtogether.')
.response(joi.object({
result:joi.number().required()
}).required(),'Sumoftheinputvalues.')
.summary('Addupnumbers')
.description('Calculatesthesumofanarrayofnumbervalues.');
Notethatweused posttodefinethisrouteinsteadof get(whichdoesnotsupportrequestbodies).TryingtosendaGETrequestto
thisroute'sURL(intheabsenceofa getrouteforthesamepath)willresultinFoxxrespondingwithanappropriateerrorresponse,
indicatingthesupportedHTTPmethods.
AsthisroutenotonlyexpectsaJSONobjectasinputbutalsorespondswithaJSONobjectasoutputweneedtodefinetwoschemas.
Wedon'tstrictlyneedaresponseschemabutithelpsdocumentingwhattherouteshouldbeexpectedtorespondwithandwillshowup
intheAPIdocumentation.
Becausewe'repassingaschematothe responsemethodwedon'tneedtoexplicitlytellFoxxwearesendingaJSONresponse.The
presenceofaschemaintheabsenceofacontenttypealwaysimplieswewantJSON.Thoughwecouldjustadd ["application/json"]
asanadditionalargumentaftertheschemaifwewantedtomakethismoreexplicit.
The bodymethodworksthesamewayasthe responsemethodexcepttheschemawillbeusedtovalidatetherequestbody.Ifthe
requestbodycan'tbeparsedasJSONordoesn'tmatchtheschema,Foxxwillrejecttherequestwithanappropriateerrorresponse.
Creatingcollections
TherealpowerofFoxxcomesfrominteractingwiththedatabaseitself.Inordertobeabletouseacollectionfromwithinourservice,we
shouldfirstmakesurethatthecollectionactuallyexists.Therightplacetocreatecollectionsyourserviceisgoingtouseisinasetup
script,whichFoxxwillexecuteforyouwheninstallingorupdatingtheservice.
Firstcreateanewfoldercalled"scripts"intheservicefolder,whichwillbewhereourscriptsaregoingtolive.Forsimplicity'ssake,our
setupscriptwillliveinafilecalled setup.jsinsidethatfolder:
//continued
'usestrict';
constdb=require('@arangodb').db;
constcollectionName='myFoxxCollection';
if(!db._collection(collectionName)){
db._createDocumentCollection(collectionName);
}
Thescriptusesthe dbobjectfromthe @arangodbmodule,whichletsusinteractwiththedatabasetheFoxxservicewasinstalledin
andthecollectionsinsidethatdatabase.Becausethescriptmaybeexecutedmultipletimes(i.e.wheneverweupdatetheserviceorwhen
theserverisrestarted)weneedtomakesurewedon'taccidentallytrytocreatethesamecollectiontwice(whichwouldresultinan
exception);wedothatbyfirstcheckingwhetheritalreadyexistsbeforecreatingit.
Gettingstarted
160
The _collectionmethodlooksupacollectionbynameandreturns nullifnocollectionwiththatnamewasfound.The
_createDocumentCollectionmethodcreatesanewdocumentcollectionbyname( _createEdgeCollectionalsoexistsandworks
analogouslyforedgecollections).
Note:Becausewehavehardcodedthecollectionname,multiplecopiesoftheserviceinstalledalongsideeachotherinthesamedatabase
willsharethesamecollection.Becausethismaynotalwaysbewhatyouwant,theFoxxcontextalsoprovidesthe collectionName
methodwhichappliesamountpointspecificprefixtoanygivencollectionnametomakeituniquetotheservice.Italsoprovidesthe
collectionmethod,whichbehavesalmostexactlylike db._collectionexceptitalsoappliestheprefixbeforelookingthecollection
up.
Nextweneedtotellourserviceaboutthescriptbyaddingittothemanifestfile:
{
"engines":{
"arangodb":"^3.0.0"
},
"main":"index.js",
"scripts":{
"setup":"scripts/setup.js"
}
}
Theonlythingthathaschangedisthatweaddeda"scripts"fieldspecifyingthepathofthesetupscriptwejustwrote.
Gobacktothewebinterfaceandupdatetheservicewithournewcode,thenchecktheCollectionstab.Ifeverythingworkedright,you
shouldseeanewcollectioncalled"myFoxxCollection".
Accessingcollections
Let'sexpandourservicebyaddingafewmoreroutestoour index.js:
//continued
consterrors=require('@arangodb').errors;
constfoxxColl=db._collection('myFoxxCollection');
constDOC_NOT_FOUND=errors.ERROR_ARANGO_DOCUMENT_NOT_FOUND.code;
router.post('/entries',function(req,res){
constdata=req.body;
constmeta=foxxColl.save(req.body);
res.send(Object.assign(data,meta));
})
.body(joi.object().required(),'Entrytostoreinthecollection.')
.response(joi.object().required(),'Entrystoredinthecollection.')
.summary('Storeanentry')
.description('Storesanentryinthe"myFoxxCollection"collection.');
router.get('/entries/:key',function(req,res){
try{
constdata=foxxColl.document(req.pathParams.key);
res.send(data)
}catch(e){
if(!e.isArangoError||e.errorNum!==DOC_NOT_FOUND){
throwe;
}
res.throw(404,'Theentrydoesnotexist',e);
}
})
.pathParam('key',joi.string().required(),'Keyoftheentry.')
.response(joi.object().required(),'Entrystoredinthecollection.')
.summary('Retrieveanentry')
.description('Retrievesanentryfromthe"myFoxxCollection"collectionbykey.');
We'reusingthe saveand documentmethodsofthecollectionobjecttostoreandretrievedocumentsinthecollectionwecreatedinour
setupscript.Becausewedon'tcarewhatthedocumentslooklikeweallowanyattributesontherequestbodyandjustacceptanobject.
Gettingstarted
161
BecausethekeywillbeautomaticallygeneratedbyArangoDBwhenonewasn'tspecifiedintherequestbody,we'reusing
Object.assigntoapplytheattributesofthemetadataobjectreturnedbythe savemethodtothedocumentbeforereturningitfrom
ourfirstroute.
The documentmethodreturnsadocumentinacollectionbyits _keyor _id.Howeverwhennomatchingdocumentexistsitthrows
an ArangoErrorexception.BecausewewanttoprovideamoredescriptiveerrormessagethanArangoDBdoesoutofthebox,weneed
tohandlethaterrorexplicitly.
All ArangoErrorexceptionshaveatruthyattribute isArangoErrorthathelpsyourecognizingtheseerrorswithouthavingtoworry
about instanceofchecks.Theyalsoprovidean errorNumandan errorMessage.Ifyouwanttocheckforspecificerrorsyoucanjust
importthe errorsobjectfromthe @arangodbmoduleinsteadofhavingtomemorizenumericerrorcodes.
Insteadofdefiningourownresponselogicfortheerrorcasewejustuse res.throw,whichmakestheresponseobjectthrowan
exceptionFoxxcanrecognizeandconverttotheappropriateserverresponse.WealsopassalongtheexceptionitselfsoFoxxcanprovide
morediagnosticinformationwhenwewantitto.
Writingdatabasequeries
Storingandretrievingentriesisfine,butrightnowwehavetomemorizeeachkeywhenwecreateanentry.Let'saddaroutethatgivesus
alistofthekeysofallentriessowecanusethosetolookanentryupindetail.
Thenaïveapproachwouldbetousethe toArray()methodtoconverttheentirecollectiontoanarrayandjustreturnthat.Butwe're
onlyinterestedinthekeysandtheremightpotentiallybesomanyentriesthatfirstretrievingeverysingledocumentmightgetunwieldy.
Let'swriteashortAQLquerytodothisinstead:
//continued
constaql=require('@arangodb').aql;
router.get('/entries',function(req,res){
constkeys=db._query(aql`
FORentryIN${foxxColl}
RETURNentry._key
`);
res.send(keys);
})
.response(joi.array().items(
joi.string().required()
).required(),'Listofentrykeys.')
.summary('Listentrykeys')
.description('Assemblesalistofkeysofentriesinthecollection.');
Herewe'reusingtwonewthings:
The _querymethodexecutesanAQLqueryintheactivedatabase.
The aqltemplatestringhandlerallowsustowritemulti-lineAQLqueriesandalsohandlesqueryparametersandcollectionnames.
Insteadofhardcodingthenameofthecollectionwewanttouseinthequerywecansimplyreferencethe foxxCollvariablewedefined
earlier--itrecognizesthevalueasanArangoDBcollectionobjectandknowswearespecifyingacollectionratherthanaregularvalue
eventhoughAQLdistinguishesbetweenthetwo.
Note:Ifyouaren'tusedtoJavaScripttemplatestringsandtemplatestringhandlersjustthinkof aqlasafunctionthatreceivesthe
multilinestringsplitatevery ${}expressionaswellasanarrayofthevaluesofthoseexpressions--that'sactuallyallthereistoit.
Alternatively,here'saversionwithouttemplatestrings(noticehowmuchcleanerthe aqlversionwillbeincomparisonwhenyouhave
multiplevariables):
constkeys=db._query(
'FORentryIN@@collRETURNentry._key',
{'@coll':foxxColl}
);
Nextsteps
Gettingstarted
162
YounowknowhowtocreateaFoxxservicefromscratch,howtohandleuserinputandhowtoaccessthedatabasefromwithinyour
Foxxservicetostore,retrieveandquerydatayoustoreinsideArangoDB.ThisshouldallowyoutobuildmeaningfulAPIsforyourown
applicationsbuttherearemanymorethingsyoucandowithFoxx:
Needtogofaster?Turnondevelopmentmodeandhackonyourcoderightontheserver.
Concernedaboutsecurity?Youcouldaddauthenticationtoyourservicetoprotectaccesstothedatabeforeitevenleavesthe
database.
Writingasinglepageapp?YoucouldstoresomebasicassetsrightinsideyourFoxxservice.
Needtointegrateexternalservices?YoucanmakeHTTPrequestsfrominsideFoxxandusequeuedjobstoperformthatworkinthe
background.
Tiredofreinventingthewheel?Learnaboutdependencies.
Gettingstarted
163
Manifestfiles
Everyservicecomeswitha manifest.jsonfileprovidingmetadata.Thefollowingfieldsareallowedinmanifests:
configuration: Object(optional)
Anobjectdefiningtheconfigurationoptionsthisservicerequires.
defaultDocument: string(optional)
Ifspecified,the /(root)routeoftheservicewillautomaticallyredirecttothegivenrelativepath,e.g.:
"defaultDocument":"index.html"
ThiswouldhavethesameeffectascreatingthefollowingrouteinJavaScript:
constindexRouter=createRouter();
indexRouter.all('/',function(req,res){
res.redirect('index.html');
});
module.context.use(indexRouter);
Note:Asof3.0.0thisfieldcansafelybeomitted;thevaluenolongerdefaultsto "index.html".
dependencies: Object(optional)andprovides: Object(optional)
Objectsspecifyingotherservicesthisservicehasasdependenciesandwhatdependenciesitcanprovidetootherservices.
engines: Object(optional)
AnobjectindicatingthesemanticversionrangesofArangoDB(orcompatibleenvironments)theservicewillbecompatiblewith,
e.g.:
"engines":{
"arangodb":"^3.0.0"
}
ThisshouldcorrectlyindicatetheminimumversionofArangoDBtheservicehasbeentestedagainst.Foxxmaintainsastrict
semanticversioningpolicyasofArangoDB3.0.0soitisgenerallysafetousesemverranges(e.g. ^3.0.0tomatchanyversion
greaterorequalto 3.0.0andbelow 4.0.0)formaximumcompatibility.
files: Object(optional)
Anobjectdefiningfileassetsservedbythisservice.
lib: string(Default: ".")
TherelativepathtotheFoxxJavaScriptfilesintheservice,e.g.:
"lib":"lib"
Thiswouldresultinthemainentrypoint(seebelow)andotherJavaScriptpathsbeingresolvedasrelativetothe libfolderinside
theservicefolder.
main: string(optional)
Therelativepathtothemainentrypointofthisservice(relativetolib,seeabove),e.g.:
"main":"index.js"
ThiswouldresultinFoxxloadingandexecutingthefile index.jswhentheserviceismountedorstarted.
Servicemanifest
164
Note:whileitistechnicallypossibletoomitthisfield,youwilllikelywanttoprovideanentrypointtoyourserviceasthisisthe
onlywaytoexposeHTTProutesorexportaJavaScriptAPI.
scripts: Object(optional)
Anobjectdefiningnamedscriptsprovidedbythisservice,whichcaneitherbeuseddirectlyorasqueuedjobsbyotherservices.
tests: stringor Array<string>(optional)
ApathorlistofpathsofJavaScripttestsprovidedforthisservice.
Additionallymanifestscanprovidethefollowingmetadata:
author: string(optional)
Thefullnameoftheauthoroftheservice(i.e.you).Thiswillbeshowninthewebinterface.
contributors: Array<string>(optional)
Alistofnamesofpeoplethathavecontributedtothedevelopmentoftheserviceinsomeway.Thiswillbeshownintheweb
interface.
description: string(optional)
Ahuman-readabledescriptionoftheservice.Thiswillbeshowninthewebinterface.
keywords: Array<string>(optional)
Alistofkeywordsthathelpcategorizethisservice.ThisisusedbytheFoxxStoreinstallerstoorganizeservices.
license: string(optional)
Astringidentifyingthelicenseunderwhichtheserviceispublished,ideallyintheformofanSPDXlicenseidentifier.Thiswillbe
showninthewebinterface.
name: string(optional)
ThenameoftheFoxxservice.AllowedcharactersareA-Z,0-9,theASCIIhyphen( -)andunderscore( _)characters.Thename
mustnotstartwithanumber.Thiswillbeshowninthewebinterface.
thumbnail: string(optional)
Thefilenameofathumbnailthatwillbeusedalongsidetheserviceinthewebinterface.ThisshouldbeaJPEGorPNGimagethat
looksgoodatsizes50x50and160x160.
version: string(optional)
TheversionnumberoftheFoxxservice.Theversionnumbermustfollowthesemanticversioningformat.Thiswillbeshowninthe
webinterface.
Examples
Servicemanifest
165
{
"name":"example-foxx-service",
"version":"3.0.0-dev",
"license":"MIT",
"description":"Anexampleservicewitharelativelyfull-featuredmanifest.",
"thumbnail":"foxx-icon.png",
"keywords":["demo","service"],
"author":"ArangoDBGmbH",
"contributors":[
"AlanPlum<[email protected]>"
],
"lib":"dist",
"main":"entry.js",
"defaultDocument":"welcome.html",
"engines":{
"arangodb":"^3.0.0"
},
"files":{
"welcome.html":"assets/index.html",
"hello.jpg":"assets/hello.jpg"
"world.jpg":{
"path":"assets/world.jpg",
"type":"image/jpeg",
"gzip":false
}
},
"tests":"dist/**.spec.js"
}
Servicemanifest
166
Foxxservicecontext
Theservicecontextprovidesaccesstomethodsandattributesthatarespecifictoagivenservice.InaFoxxservicethecontextisgenerally
availableasthe module.contextvariable.Withinarouter'srequesthandlertherequestandresponseobjects' contextattributealso
provideaccesstothecontextoftheservicetheroutewasmountedin(whichmaybedifferentfromtheonetheroutehandlerwasdefined
in).
Examples
//inservice/my-foxx-1
//Seethechapterondependenciesformoreinfoon
//howexportsanddependenciesworkacrossservices
module.exports={routes:router};
router.get(function(req,res){
module.context.mount==='/my-foxx-1';
req.context.mount==='/my-foxx-2';
res.write('Hellofrommy-foxx-1');
});
//inservice/my-foxx-2
constrouter2=createRouter();
module.context.use(router2);
router2.post(function(req,res){
module.context.mount==='/my-foxx-2';
req.context.mount==='/my-foxx-2';
res.write('Hellofrommy-foxx-2');
});
constrouter1=module.context.dependencies.myFoxx1.routes;
module.context.use(router1);
Theservicecontextspecifiesthefollowingproperties:
argv: any
Anyargumentspassedinifthecurrentfilewasexecutedasascriptorqueuedjob.
basePath: string
Thefilesystempathoftheservice,i.e.thefolderinwhichtheservicewasinstalledtobyArangoDB.
baseUrl: string
ThebaseURLoftheservice,relativetotheArangoDBserver,e.g. /_db/_system/my-foxx.
collectionPrefix: string
TheprefixthatwillbeusedbycollectionandcollectionNametoderivethenamesofservice-specificcollections.Thisisderivedfrom
theservice'smountpoint,e.g. /my-foxxbecomes my_foxx.
configuration: Object
Configurationoptionsfortheservice.
dependencies: Object
Configureddependenciesfortheservice.
isDevelopment: boolean
Indicateswhethertheserviceisrunningindevelopmentmode.
Servicecontext
167
isProduction: boolean
TheinverseofisDevelopment.
manifest: Object
Theparsedmanifestfileoftheservice.
mount: string
Themountpointoftheservice,e.g. /my-foxx.
apiDocumentation
module.context.apiDocumentation([options]):Function
CreatesaroutehandlerthatservestheAPIdocumentation.
Note:Theroutehandlermustbemountedatawildcardroute(seeexamplesbelow).
Arguments
options: Object(optional)
Anobjectwithanyofthefollowingproperties:
mount: string(Default: module.context.mount)
Themountpathoftheservicetoservethedocumentationof.
indexFile: string(Default: "index.html")
FilenameoftheHTM LfileservingtheAPIdocumentation.
swaggerRoot: string(optional)
FullpathofthefoldercontainingtheSwaggerassetsandtheindexFile.DefaultstotheSwaggerassetsusedbytheweb
interface.
before: Function(optional)
Requesthandlertobeexecutedbeforearequestishandled.
Ifthefunctionreturns falsetherequestwillnotbeprocessedanyfurther.
Ifthefunctionreturnsanobject,itsattributeswillbeusedtooverridetheoptionsforthecurrentrequest.
Anyotherreturnvaluewillbeignored.
Ifoptionsisafunctionitwillbeusedasthebeforeoption.
IfoptionsisastringitwillbeusedastheswaggerRootoption.
Returnsarequesthandlerfunction.
Examples
Servicecontext
168
//ServetheAPIdocsforthecurrentservice
router.get('/docs/*',module.context.apiDocumentation());
//--or-//ServetheAPIdocsfortheservicetherouterismountedin
router.get('/docs/*',module.context.apiDocumentation(function(req){
return{mount:req.context.mount};
}));
//--or-//ServetheAPIdocsonlyforusersauthenticatedwithArangoDB
router.get('/docs/*',module.context.apiDocumentation(function(req,res){
if(req.suffix==='swagger.json'&&!req.arangoUser){
res.throw(401,'Notauthenticated');
}
}));
collection
module.context.collection(name):ArangoCollection|null
PassesthegivennametocollectionName,thenlooksupthecollectionwiththeprefixedname.
Arguments
name: string
Unprefixednameoftheservice-specificcollection.
Returnsacollectionor nullifnocollectionwiththeprefixednameexists.
collectionName
module.context.collectionName(name):string
PrefixesthegivennamewiththecollectionPrefixforthisservice.
Arguments
name: string
Unprefixednameoftheservice-specificcollection.
Returnstheprefixedname.
Examples
module.context.mount==='/my-foxx'
module.context.collectionName('doodads')==='my_foxx_doodads'
file
module.context.file(name,[encoding]):Buffer|string
PassesthegivennametofileName,thenloadsthefilewiththeresultingname.
Arguments
name: string
Nameofthefiletoload,relativetothecurrentservice.
encoding: string(optional)
Encodingofthefile,e.g. utf-8.Ifomittedthefilewillbeloadedasarawbufferinsteadofastring.
Servicecontext
169
Returnsthefile'scontents.
fileName
module.context.fileName(name):string
Resolvesthegivenfilenamerelativetothecurrentservice.
Arguments
name: string
Nameofthefile,relativetothecurrentservice.
Returnstheabsolutefilepath.
use
module.context.use([path],router):Endpoint
M ountsagivenrouterontheservicetoexposetherouter'sroutesontheservice'smountpoint.
Arguments
path: string(Default: "/")
Pathtomounttherouterat,relativetotheservice'smountpoint.
router: Router|Middleware
Arouterormiddlewaretomount.
ReturnsanEndpointforthegivenrouterormiddleware.
Note:M ountingservicesatruntime(e.g.withinrequesthandlersorqueuedjobs)isnotsupported.
Servicecontext
170
Foxxconfiguration
Foxxservicescandefineconfigurationparameterstomakethemmorere-usable.
The configurationobjectmapsnamestoconfigurationparameters:
Thekeyisthenameunderwhichttheparameterwillbeavailableontheservicecontext's configurationproperty.
Thevalueisaparameterdefinition.
Theparameterdefinitioncanhavethefollowingproperties:
description: string
Humanreadabledescriptionoftheparameter.
type: string(Default: "string")
Typeoftheconfigurationparameter.Supportedvaluesare:
"integer"or "int":anyfiniteintegernumber.
"boolean"or "bool":thevalues trueor false.
"number":anyfinitedecimalorintegernumber.
"string":anystringvalue.
"json":anywell-formedJSONvalue.
"password":likestringbutwillbedisplayedasamaskedinputfieldinthewebfrontend.
default: any
Defaultvalueoftheconfigurationparameter.
required:(Default: true)
Whethertheparameterisrequired.
Iftheconfigurationhasparametersthatdonotspecifyadefaultvalue,youneedtoconfiguretheservicebeforeitbecomesactive.Inthe
meantimeafallbackservicelicationwillbemountedthatrespondstoallrequestswithaHTTP500statuscodeindicatingaserver-side
error.
TheconfigurationparametersofamountedservicecanbeadjustedfromthewebinterfacebyclickingtheConfigurationbuttoninthe
servicedetails.
Examples
"configuration":{
"currency":{
"description":"Currencysymboltouseforpricesintheshop.",
"default":"$",
"type":"string"
},
"secretKey":{
"description":"Secretkeytouseforsigningsessiontokens.",
"type":"password"
}
}
Configuration
171
Dependencymanagement
Therearetwothingscommonlycalled"dependencies"inFoxx:
M oduledependencies,i.e.dependenciesonexternalJavaScriptmodules(e.g.fromthepublicnpmregistry)
Foxxdependencies,i.e.dependenciesbetweenFoxxservices
Let'slookattheminmoredetail:
Moduledependencies
Youcanusethe node_modulesfoldertobundlethird-partyFoxx-compatiblenpmandNode.jsmoduleswithyourFoxxservice.
Typicallythisisachievedbyaddinga package.jsonfiletoyourprojectspecifyingnpmdependenciesusingthe dependenciesattribute
andinstallingthemwiththenpmcommand-linetool.
M akesuretoincludetheactual node_modulesfolderinyourFoxxservicebundleasArangoDBwillnotdoanythingspecialtoinstall
thesedependencies.Alsokeepinmindthatbundlingextraneousmoduleslikedevelopmentdependenciesmaybloatthefilesizeofyour
Foxxservicebundle.
Compatibilitycaveats
UnlikeJavaScriptinbrowsersorNode.js,theJavaScriptenvironmentinArangoDBissynchronous.Thismeansanymodulesthat
dependonasynchronousbehaviourlikepromisesor setTimeoutwillnotbehavecorrectlyinArangoDBorFoxx.Additionallyunlike
Node.jsArangoDBdoesnotsupportnativeextensions.AllmoduleshavetobeimplementedinpureJavaScript.
WhileArangoDBprovidesalotofcompatibilitycodetosupportmoduleswrittenforNode.js,someNode.jsbuilt-inmodulescannotbe
providedbyArangoDB.ForacloserlookattheNode.jsmodulesArangoDBdoesordoesnotprovidecheckouttheappendixon
JavaScriptmodules.
Alsonotethattheserestrictionsnotonlyapplyonthemodulesyouwishtoinstallbutalsothedependenciesofthosemodules.Asarule
ofthumb:moduleswrittentoworkinNode.jsandthebrowserthatdonotrelyonasyncbehaviourshouldgenerallywork;modulesthat
relyonnetworkorfilesystemI/Oormakeheavyuseofasyncbehaviourmostlikelywillnot.
Foxxdependencies
Foxxdependenciescanbedeclaredinaservice'smanifestusingthe providesand dependenciesfields:
providesliststhedependenciesagivenserviceprovides,i.e.whichAPIsitclaimstobecompatiblewith
dependenciesliststhedependenciesagivenserviceuses,i.e.whichAPIsitsdependenciesneedtobecompatiblewith
Adependencynameshouldgenerallyusethesameformatasanamespaced(org-scoped)NPM module,e.g. @foxx/sessions.
DependencynamesrefertotheexternalJavaScriptAPIofaserviceratherthanspecificservicesimplementingthoseAPIs.Some
dependencynamesdefinedbyofficiallymaintainedservicesare:
@foxx/auth(version 1.0.0)
@foxx/api-keys(version 1.0.0)
@foxx/bugsnag(versions 1.0.0and 2.0.0)
@foxx/mailgun(versions 1.0.0and 2.0.0)
@foxx/postageapp(versions 1.0.0and 2.0.0)
@foxx/postmark(versions 1.0.0and 2.0.0)
@foxx/sendgrid(versions 1.0.0and 2.0.0)
@foxx/oauth2(versions 1.0.0and 2.0.0)
@foxx/segment-io(versions 1.0.0and 2.0.0)
@foxx/sessions(versions 1.0.0and 2.0.0)
@foxx/users(versions 1.0.0, 2.0.0and 3.0.0)
Dependencies
172
A providesdefinitionmapseachprovideddependency'snametotheprovidedversion:
"provides":{
"@foxx/auth":"1.0.0"
}
A dependenciesdefinitionmapsthelocalaliasofagivendependencyagainstitsnameandthesupportedversionrange(eitherasa
JSONobjectorashorthandstring):
"dependencies":{
"mySessions":"@foxx/sessions:^2.0.0",
"myAuth":{
"name":"@foxx/auth",
"version":"^1.0.0",
"required":false
}
}
Dependenciescanbeconfiguredfromthewebinterfaceinaservice'ssettingstabusingtheDependenciesbutton.
Thevalueforeachdependencyshouldbethedatabase-relativemountpathoftheservice(includingtheleadingslash).Inordertobe
usableasthedependencyofanotherservicebothservicesneedtobemountedinthesamedatabase.Aservicecanbeusedtoprovide
multipledependenciesforthesameservice(aslongastheexpectedJavaScriptAPIsdon'tconflict).
Aservicethathasunconfiguredrequireddependenciescannotbeuseduntilallofitsdependencieshavebeenconfigured.
Itispossibletospecifythemountpathofaservicethatdoesnotactuallydeclarethedependencyasprovided.Thereiscurrentlyno
validationbeyondthemanifestformats.
Whenaserviceusesanothermountedserviceasadependencythedependency's mainentryfile's exportsobjectbecomesavailablein
the module.context.dependenciesobjectoftheotherservice:
Examples
ServiceAandServiceBaremountedinthesamedatabase.ServiceBhasadependencywiththelocalalias "greeter".Thedependency
isconfiguredtousethemountpathofServiceB.
//EntryfileofServiceA
module.exports={
sayHi(){
return'Hello';
}
};
//SomewhereinServiceB
constgreeter=module.context.dependencies.greeter;
res.write(greeter.sayHi());
Dependencies
173
Routers
RoutersletyoudefineroutesthatextendArangoDB'sHTTPAPIwithcustomendpoints.
Routersneedtobemountedusingthe usemethodofaservicecontexttoexposetheirHTTProutesataservice'smountpath.
Youcanpassroutersbetweenservicesmountedinthesamedatabaseasdependencies.Youcanevennestrouterswithineachother.
Creatingarouter
createRouter():Router
Thisreturnsanew,cleanrouterobjectthathasnotyetbeenmountedintheserviceandcanbeexportedlikeanyotherobject.
Requesthandlers
router.get([path],handler,[name]):Endpoint
router.post([path],handler,[name]):Endpoint
router.put([path],handler,[name]):Endpoint
router.patch([path],handler,[name]):Endpoint
router.delete([path],handler,[name]):Endpoint
router.all([path],handler,[name]):Endpoint
Thesemethodsletyouspecifyroutesontherouter.The allmethoddefinesaroutethatwillmatchanysupportedHTTPverb,the
othermethodsdefineroutesthatonlymatchtheHTTPverbwiththesamename.
Arguments
path: string(Default: "/")
ThepathoftherequesthandlerrelativetothebasepaththeRouterismountedat.Ifomitted,therequesthandlerwillhandle
requeststothebasepathoftheRouter.Forinformationondefiningdynamicroutesseethesectiononpathparametersinthe
chapteronrouterendpoints.
handler: Function
Afunctionthattakesthefollowingarguments:
req: Request
Anincomingserverrequestobject.
res: Response
Anoutgoingserverresponse.
AnamethatcanbeusedtogenerateURLsfortheendpoint.Formoreinformationseethe reversemethodoftherequestobject.
ReturnsanEndpointfortheroute.
Mountingchildroutersandmiddleware
router.use([path],handler,[name]):Endpoint
The usemethodletsyoumountachildrouterormiddlewareatagivenpath.
Arguments
Routers
174
path: string(optional)
ThepathoftherequesthandlerrelativetothebasepaththeRouterismountedat.Ifomitted,therequesthandlerwillhandle
requeststothebasepathoftheRouter.Forinformationondefiningdynamicroutesseethesectiononpathparametersinthe
chapteronrouterendpoints.
handler: Router|Middleware
Anunmountedrouterobjectoramiddleware.
AnamethatcanbeusedtogenerateURLsforendpointsofthisrouter.Formoreinformationseethe reversemethodofthe
requestobject.HasnoeffectifhandlerisaM iddleware.
ReturnsanEndpointforthemiddlewareorchildrouter.
Routers
175
Endpoints
Endpointsarereturnedbythe use, allandHTTPverb(e.g. get, post)methodsofroutersaswellasthe usemethodofthe
servicecontext.Theycanbeusedtoattachmetadatatomountedroutes,middlewareandchildroutersthataffectshowrequestsand
responsesareprocessedorprovidesAPIdocumentation.
Endpointsshouldonlybeusedtoinvokethefollowingmethods.Endpointmethodscanbechainedtogether(eachmethodreturnsthe
endpointitself).
header
endpoint.header(name,[schema],[description]):this
Definesarequestheaderrecognizedbytheendpoint.Anyadditionalnon-definedheaderswillbetreatedasoptionalstringvalues.The
definitionswillalsobeshownintheroutedetailsintheAPIdocumentation.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethisheaderdefinitionunlessoverridden.
Arguments
name: string
Nameoftheheader.Thisshouldbeconsideredcaseinsensitiveasallheadernameswillbeconvertedtolowercase.
schema: Schema(optional)
Aschemadescribingtheformatoftheheadervalue.Thiscanbeajoischemaoranythingthathasacompatible validatemethod.
Thevalueofthisheaderwillbesettothe valuepropertyofthevalidationresult.Avalidationfailurewillresultinanautomatic
400(BadRequest)errorresponse.
AhumanreadablestringthatwillbeshownintheAPIdocumentation.
Returnstheendpoint.
Examples
router.get(/*...*/)
.header('arangoVersion',joi.number().min(30000).default(30000));
pathParam
endpoint.pathParam(name,[schema],[description]):this
Definesapathparameterrecognizedbytheendpoint.Pathparametersareexpectedtobefilledaspartoftheendpoint'smountpath.
Anyadditionalnon-definedpathparameterswillbetreatedasoptionalstringvalues.Thedefinitionswillalsobeshownintheroute
detailsintheAPIdocumentation.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethisparameterdefinitionunlessoverridden.
Arguments
name: string
Nameoftheparameter.
Aschemadescribingtheformatoftheparameter.Thiscanbeajoischemaoranythingthathasacompatible validatemethod.
Endpoints
176
Thevalueofthisparameterwillbesettothe valuepropertyofthevalidationresult.Avalidationfailurewillresultintheroute
failingtomatchandbeingignored(resultingina404(NotFound)errorresponseifnootherroutesmatch).
Returnstheendpoint.
Examples
router.get('/some/:num/here',/*...*/)
.pathParam('num',joi.number().required());
queryParam
endpoint.queryParam(name,[schema],[description]):this
Definesaqueryparameterrecognizedbytheendpoint.Anyadditionalnon-definedqueryparameterswillbetreatedasoptionalstring
values.ThedefinitionswillalsobeshownintheroutedetailsintheAPIdocumentation.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethisparameterdefinitionunlessoverridden.
Arguments
name: string
Nameoftheparameter.
Aschemadescribingtheformatoftheparameter.Thiscanbeajoischemaoranythingthathasacompatible validatemethod.
Thevalueofthisparameterwillbesettothe valuepropertyofthevalidationresult.Avalidationfailurewillresultinan
automatic400(BadRequest)errorresponse.
Returnstheendpoint.
Examples
router.get(/*...*/)
.queryParam('num',joi.number().required());
body
endpoint.body([model],[mimes],[description]):this
Definestherequestbodyrecognizedbytheendpoint.Therecanonlybeonerequestbodydefinitionperendpoint.Thedefinitionwill
alsobeshownintheroutedetailsintheAPIdocumentation.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethisbodydefinitionunlessoverridden.Iftheendpointisamiddleware,
therequestbodywillonlybeparsedonce(i.e.theM IM Etypesoftheroutematchingthesamerequestwillbeignoredbutthebodywill
stillbevalidatedagain).
Arguments
model: Model|Schema|null(optional)
Amodelorjoischemadescribingtherequestbody.Avalidationfailurewillresultinanautomatic400(BadRequest)errorresponse.
Ifthevalueisamodelwitha fromClientmethod,thatmethodwillbeappliedtotheparsedrequestbody.
Endpoints
177
Ifthevalueisaschemaoramodelwithaschema,theschemawillbeusedtovalidatetherequestbodyandthe valuepropertyof
thevalidationresultoftheparsedrequestbodywillbeusedinsteadoftheparsedrequestbodyitself.
IfthevalueisamodeloraschemaandtheM IM Etypehasbeenomitted,theM IM EtypewilldefaulttoJSONinstead.
Ifthevalueisexplicitlysetto null,norequestbodywillbeexpected.
Ifthevalueisanarraycontainingexactlyonemodelorschema,therequestbodywillbetreatedasanarrayofitemsmatchingthat
modelorschema.
mimes: Array<string>(optional)
AnarrayofM IM Etypestheroutesupports.
Commonnon-mimealiaseslike"json"or"html"arealsosupportedandwillbeexpandedtotheappropriateM IM Etype(e.g.
"application/json"and"text/html").
IftheM IM EtypeisrecognizedbyFoxxtherequestbodywillbeparsedintotheappropriatestructurebeforebeingvalidated.
CurrentlyonlyJSON, application/x-www-form-urlencodedandmultipartformatsaresupportedinthisway.
IftheM IM EtypeindicatedintherequestheadersdoesnotmatchanyofthesupportedM IM Etypes,thefirstM IM Etypeinthe
listwillbeusedinstead.
Failuretoparsetherequestbodywillresultinanautomatic400(BadRequest)errorresponse.
Returnstheendpoint.
Examples
router.post('/expects/some/json',/*...*/)
.body(
joi.object().required(),
'ThisimpliesJSON.'
);
router.post('/expects/nothing',/*...*/)
.body(null);//Nobodyallowed
router.post('/expects/some/plaintext',/*...*/)
.body(['text/plain'],'Thisbodywillbeastring.');
response
endpoint.response([status],[model],[mimes],[description]):this
Definesaresponsebodyfortheendpoint.Whenusingtheresponseobject's sendmethodintherequesthandlerofthisroute,the
definitionwiththematchingstatuscodewillbeusedtogeneratetheresponsebody.Thedefinitionswillalsobeshownintheroute
detailsintheAPIdocumentation.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethisresponsedefinitionunlessoverridden.Iftheendpointisa
middleware,thismethodhasnoeffect.
Arguments
status: number|string(Default: 200or 204)
HTTPstatuscodetheresponseappliesto.Ifastringisprovidedinsteadofanumericstatuscodeitwillbeusedtolookupa
numericstatuscodeusingthestatusesmodule.
model: Model|Schema|null(optional)
Amodelorjoischemadescribingtheresponsebody.
Endpoints
178
Ifthevalueisamodelwitha forClientmethod,thatmethodwillbeappliedtothedatapassedto response.sendwithinthe
routeiftheresponsestatuscodematches(butalsoifnostatuscodehasbeenset).
Ifthevalueisaschemaoramodelwithaschema,theactualschemawillnotbeusedtovalidatetheresponsebodyandonlyserves
todocumenttheresponseinmoredetailintheAPIdocumentation.
IfthevalueisamodeloraschemaandtheM IM Etypehasbeenomitted,theM IM EtypewilldefaulttoJSONinstead.
Ifthevalueisexplicitlysetto nullandthestatuscodehasbeenomitted,thestatuscodewilldefaultto 204("nocontent")
insteadof 200.
Ifthevalueisanarraycontainingexactlyonemodelorschema,theresponsebodywillbeanarrayofitemsmatchingthatmodelor
schema.
mimes: Array<string>(optional)
AnarrayofM IM Etypestheroutemightrespondwithforthisstatuscode.
Commonnon-mimealiaseslike"json"or"html"arealsosupportedandwillbeexpandedtotheappropriateM IM Etype(e.g.
"application/json"and"text/html").
Whenusingthe response.sendmethodtheresponsebodywillbeconvertedtotheappropriateM IM Etypeifpossible.
Ahuman-readablestringthatbrieflydescribestheresponseandwillbeshownintheendpoint'sdetaileddocumentation.
Returnstheendpoint.
Examples
//Thisexampleonlyprovidesdocumentation
//andimpliesagenericJSONresponsebody.
router.get(/*...*/)
.response(
joi.array().items(joi.string()),
'Alistofdoodadidentifiers.'
);
//Noresponsebodywillbeexpectedhere.
router.delete(/*...*/)
.response(null,'Thedoodadnolongerexists.');
//Anendpointcandefinemultipleresponsetypes
//fordifferentstatuscodes--butnevermorethan
//oneforeachstatuscode.
router.post(/*...*/)
.response('found','Thedoodadislocatedelsewhere.')
.response(201,['text/plain'],'Thedoodadwascreatedsohereisahaiku.');
//Heretheresponsebodywillbesetto
//thequerystring-encodedresultof
//FormModel.forClient({some:'data'})
//becausethestatuscodedefaultsto200.
router.patch(function(req,res){
//...
res.send({some:'data'});
})
.response(FormModel,['application/x-www-form-urlencoded'],'OMG.');
//Inthiscasetheresponsebodywillbesetto
//SomeModel.forClient({some:'data'})because
//thestatuscodehasbeensetto201before.
router.put(function(req,res){
//...
res.status(201);
res.send({some:'data'});
})
.response(201,SomeModel,'Somethingamazinghappened.');
Endpoints
179
error
endpoint.error(status,[description]):this
Documentsanerrorstatusfortheendpoint.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethiserrordescriptionunlessoverridden.Iftheendpointisamiddleware,
thismethodhasnoeffect.
ThismethodonlyaffectsthegeneratedAPIdocumentationandhasnotothereffectwithintheserviceitself.
Arguments
status: number|string
HTTPstatuscodefortheerror(e.g. 400for"badrequest").Ifastringisprovidedinsteadofanumericstatuscodeitwillbeused
tolookupanumericstatuscodeusingthestatusesmodule.
Ahuman-readablestringthatbrieflydescribestheerrorconditionandwillbeshownintheendpoint'sdetaileddocumentation.
Returnstheendpoint.
Examples
router.get(function(req,res){
//...
res.throw(403,'Validationerroratx.y.z');
})
.error(403,'Indicatesthatavalidationhasfailed.');
summary
endpoint.summary(summary):this
Addsashortdescriptiontotheendpoint'sAPIdocumentation.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethissummaryunlessoverridden.Iftheendpointisamiddleware,this
methodhasnoeffect.
Arguments
summary: string
Ahuman-readablestringthatbrieflydescribestheendpointandwillappearnexttotheendpoint'spathinthedocumentation.
Returnstheendpoint.
Examples
router.get(/*...*/)
.summary('Listalldiscombobulateddoodads')
description
endpoint.description(description):this
Addsalongdescriptiontotheendpoint'sAPIdocumentation.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethisdescriptionunlessoverridden.Iftheendpointisamiddleware,this
methodhasnoeffect.
Endpoints
180
Arguments
description: string
Ahuman-readablestringthatdescribestheendpointindetailandwillbeshownintheendpoint'sdetaileddocumentation.
Returnstheendpoint.
Examples
//The"dedent"libraryhelpsformatting
//multi-linestringsbyadjustingindentation
//andremovingleadingandtrailingblanklines
constdd=require('dedent');
router.post(/*...*/)
.description(dd`
Thisroutediscombobulatesthedoodadsby
frobnicatingthemoxieoftherequestbody.
`)
deprecated
endpoint.deprecated([deprecated]):this
M arkstheendpointasdeprecated.
Iftheendpointisachildrouter,allroutesofthatrouterwillalsobemarkedasdeprecated.Iftheendpointisamiddleware,thismethod
hasnoeffect.
Arguments
deprecated: boolean(Default: true)
Whethertheendpointshouldbemarkedasdeprecated.Ifsetto falsetheendpointwillbeexplicitlymarkedasnotdeprecated.
Returnstheendpoint.
Examples
router.get(/*...*/)
.deprecated();
Endpoints
181
Middleware
M iddlewareinFoxxreferstofunctionsthatcanbemountedlikeroutesandcanmanipulatetherequestandresponseobjectsbeforeand
aftertherouteitselfisinvoked.Theycanalsobeusedtocontrolaccessortoprovidecommonlogiclikeloggingetc.Unlikeroutes,
middlewareismountedwiththe usemethodlikearouter.
Insteadofafunctionthe usemethodcanalsoacceptanobjectwitha registerfunctionthatwillbepassedtheendpointthe
middlewarewillbemountedatandreturnstheactualmiddlewarefunction.Thisallowsmanipulatingtheendpointbeforecreatingthe
middleware(e.g.todocumentheaders,requestbodies,pathparametersorqueryparameters).
Examples
RestrictaccesstoArangoDB-authenticatedusers:
module.context.use(function(req,res,next){
if(!req.arangoUser){
res.throw(401,'NotauthenticatedwithArangoDB');
}
next();
});
Trivialloggingmiddleware:
module.context.use(function(req,res,next){
conststart=Date.now();
try{
next();
}finally{
console.log(`Handledrequestin${Date.now()-start}ms`);
}
});
M orecomplexexampleforheader-basedsessions:
constsessions=module.context.collection('sessions');
module.context.use({
register(endpoint){
endpoint.header('x-session-id',joi.string().optional(),'ThesessionID.');
returnfunction(req,res,next){
constsid=req.get('x-session-id');
if(sid){
try{
req.session=sessions.document(sid);
}catch(e){
deletereq.headers['x-session-id'];
}
}
next();
if(req.session){
if(req.session._rev){
sessions.replace(req.session,req.session);
res.set('x-session-id',req.session._key);
}else{
constmeta=sessions.save(req.session);
res.set('x-session-id',meta._key);
}
}
};
}
});
M iddleware
182
Requestobjects
Therequestobjectspecifiesthefollowingproperties:
arangoUser: string|null
TheauthenticatedArangoDBusernameusedtomaketherequest.ThisvalueisonlysetifauthenticationisenabledinArangoDB
andtherequestsetan authorizationheaderArangoDBwasabletoverify.Youarestronglyencouragedtoimplementyourown
authenticationlogicforyourownservicesbutthispropertycanbeusefulifyouneedtointegratewithArangoDB'sown
authenticationmechanisms.
arangoVersion: number
Thenumericvalueofthe x-arango-versionheaderorthenumericversionoftheArangoDBserver(e.g. 30102forversion3.1.2)if
novalidheaderwasprovided.
baseUrl: string
Root-relativebaseURLoftheservice,i.e.theprefix "/_db/"followedbythevalueofdatabase.
body: any
Theprocessedandvalidatedrequestbodyforthecurrentroute.
FordetailsonhowrequestbodiescanbeprocessedandvalidatedbyFoxxseethebodymethodoftheendpointobject.
context: Context
Theservicecontextinwhichtherouterwasmounted(ratherthanthecontextinwhichtheroutewasdefined).
database: string
Thenameofthedatabaseinwhichtherequestisbeinghandled,e.g. "_system".
headers: object
Therawheadersobject.
FordetailsonhowrequestheaderscanbevalidatedbyFoxxseetheheadermethodoftheendpointobject.
hostname: string
Thehostname(domainname)indicatedintherequestheaders.
Defaultstothehostnameportion(i.e.excludingtheport)ofthe Hostheaderandfallsbacktothelisteningaddressoftheserver.
method: string
TheHTTPverbusedtomaketherequest,e.g. "GET".
originalUrl: string
Root-relativeURLoftherequest,i.e.pathfollowedbytherawqueryparameters,ifany.
path: string
Database-relativepathoftherequestURL(notincludingthequeryparameters).
pathParams: object
Anobjectmappingthenamesofpathparametersofthecurrentroutetotheirvalidatedvalues.
FordetailsonhowpathparameterscanbevalidatedbyFoxxseethepathParammethodoftheendpointobject.
port: number
Theportindicatedintherequestheaders.
Request
183
Defaultstotheportportion(i.e.excludingthehostname)ofthe Hostheaderandfallsbacktothelisteningportortheappropriate
defaultport( 443forHTTPSor 80forHTTP,dependingonsecure)iftheheaderonlyindicatesahostname.
Iftherequestwasmadeusingatrustedproxy(seetrustProxy),thisissettotheportportionofthe X-Forwarded-Hostheader(or
approriatedefaultport)ifpresent.
protocol: string
Theprotocolusedfortherequest.
Defaultsto "https"or "http"dependingonwhetherArangoDBisconfiguredtouseSSLornot.
Iftherequestwasmadeusingatrustedproxy(seetrustProxy),thisissettothevalueofthe X-Forwarded-Protoheaderifpresent.
queryParams: object
Anobjectmappingthenamesofqueryparametersofthecurrentroutetotheirvalidatedvalues.
FordetailsonhowqueryparameterscanbevalidatedbyFoxxseethequeryParammethodoftheendpointobject.
rawBody: Buffer
Theraw,unparsed,unvalidatedrequestbodyasabuffer.
remoteAddress: string
TheIPoftheclientthatmadetherequest.
Iftherequestwasmadeusingatrustedproxy(seetrustProxy),thisissettothefirstIPlistedinthe X-Forwarded-Forheaderif
present.
remoteAddresses: Array<string>
AlistcontainingtheIPaddressesusedtomaketherequest.
DefaultstothevalueofremoteAddresswrappedinanarray.
Iftherequestwasmadeusingatrustedproxy(seetrustProxy),thisissettothelistofIPsspecifiedinthe X-Forwarded-Forheader
ifpresent.
remotePort: number
Thelisteningportoftheclientthatmadetherequest.
Iftherequestwasmadeusingatrustedproxy(seetrustProxy),thisissettotheportspecifiedinthe X-Forwarded-Portheaderif
present.
secure: boolean
Whethertherequestwasmadeoverasecureconnection(i.e.HTTPS).
Thisissetto falsewhenprotocolis "http"and truewhenprotocolis "https".
suffix: string
Thetrailingpathrelativetothecurrentrouteifthecurrentrouteendsinawildcard.
trustProxy: boolean
Indicateswhethertherequestwasmadeusingatrustedproxy.Iftheoriginserver'saddresswasspecifiedintheArangoDB
configurationusing --frontend.trusted-proxyortheservice's trustProxysettingisenabled,thiswillbe true,otherwiseitwill
be false.
url: string
TheURLoftherequest.
xhr: boolean
WhethertherequestindicatesitwasmadewithinabrowserusingAJAX.
Thisissetto trueifthe X-Requested-Withheaderispresentandisacase-insensitivematchforthevalue "xmlhttprequest".
Request
184
NotethatthisvaluedoesnotguaranteewhethertherequestwasmadefrominsideabrowserorwhetherAJAXwasusedandis
merelyaconventionestablishedbyJavaScriptframeworkslikejQuery.
accepts
req.accepts(types):string|false
req.accepts(...types):string|false
req.acceptsCharsets(charsets):string|false
req.acceptsCharsets(...charsets):string|false
req.acceptsEncodings(encodings):string|false
req.acceptsEncodings(...encodings):string|false
req.acceptsLanguages(languages):string|false
req.acceptsLanguages(...languages):string|false
Thesemethodswrapthecorrespondingcontentnegotiationmethodsoftheacceptsmoduleforthecurrentrequest.
Examples
if(req.accepts(['json','html'])==='html'){
//ClientexplicitlyprefersHTMLoverJSON
res.write('<h1>ClientprefersHTML</h1>');
}else{
//OtherwisejustsendJSON
res.json({success:true});
}
cookie
req.cookie(name,options):string|null
Getsthevalueofacookiebyname.
Arguments
name: string
Nameofthecookie.
options: object(optional)
secret: string(optional)
Secretthatwasusedtosignthecookie.
Ifasecretisspecified,thecookie'ssignatureisexpectedtobepresentinasecondcookiewiththesamenameandthesuffix
.sig.Otherwisethesignature(ifpresent)willbeignored.
algorithm: string(Default: "sha256")
Algorithmthatwasusedtosignthecookie.
Ifastringispassedinsteadofanoptionsobjectitwillbeinterpretedasthesecretoption.
Returnsthevalueofthecookieor nullifthecookieisnotsetoritssignatureisinvalid.
get/header
req.get(name):string
req.header(name):string
Request
185
Getsthevalueofaheaderbyname.Youcanvalidaterequestheadersusingtheheadermethodoftheendpoint.
Arguments
name: string
Nameoftheheader.
Returnstheheadervalue.
is
req.is(types):string
req.is(...types):string
Thismethodwrapsthe(requestbody)contenttypedetectionmethodofthetype-ismoduleforthecurrentrequest.
Examples
consttype=req.is('html','application/xml','application/*+xml');
if(type===false){//nomatch
handleDefault(req.rawBody);
}elseif(type==='html'){
handleHtml(req.rawBody);
}else{//isXML
handleXml(req.rawBody);
}
json
req.json():any
AttemptstoparsetherawrequestbodyasJSONandreturnstheresult.
Itisgenerallymoreusefultodefinearequestbodyontheendpointandusethe req.bodypropertyinstead.
Returns undefinediftherequestbodyisempty.M aythrowa SyntaxErrorifthebodycouldnotbeparsed.
makeAbsolute
req.makeAbsolute(path,[query]):string
Resolvesthegivenpathrelativetothe req.context.service'smountpathtoafullURL.
Arguments
path: string
Thepathtoresovle.
query: string|object
AstringorobjectwithqueryparameterstoaddtotheURL.
ReturnstheformattedabsoluteURL.
params
req.param(name):any
Arguments
Looksupaparameterbyname,preferring pathParamsover queryParams.
It'sprobablybetterstyletousethe req.pathParamsor req.queryParamsobjectsdirectly.
Request
186
name: string
Nameoftheparameter.
Returnsthe(validated)valueoftheparameter.
range
req.range([size]):Ranges|number
Thismethodwrapstherangeheaderparsingmethodoftherange-parsermoduleforthecurrentrequest.
Arguments
size: number(Default: Infinity)
Lengthofthesatisfiablerange(e.g.numberofbytesinthefullresponse).Ifpresent,rangesexceedingthesizewillbeconsidered
unsatisfiable.
Returns undefinedifthe Rangeheaderisabsent, -2iftheheaderispresentbutmalformed, -1iftherangeisinvalid(e.g.start
offsetislargerthanendoffset)orunsatisfiableforthegivensize.
Otherwisereturnsanarrayofobjectswiththepropertiesstartandendvaluesforeachrange.Thearrayhasanadditionalpropertytype
indicatingtherequestrangetype.
Examples
console.log(req.headers.range);//"bytes=40-80"
constranges=req.range(100);
console.log(ranges);//[{start:40,end:80}]
console.log(ranges.type);//"bytes"
reverse
req.reverse(name,[params]):string
LooksuptheURLofanamedrouteforthegivenparameters.
Arguments
name: string
Nameoftheroutetolookup.
params: object(optional)
Anobjectcontainingvaluesforthe(pathorquery)parametersoftheroute.
ReturnstheURLoftherouteforthegivenparameters.
Examples
router.get('/items/:id',function(req,res){
/*...*/
},'getItemById');
router.post('/items',function(req,res){
//...
consturl=req.reverse('getItemById',{id:createdItem._key});
res.set('location',req.makeAbsolute(url));
});
Request
187
Responseobjects
Theresponseobjectspecifiesthefollowingproperties:
body: Buffer|string
Responsebodyasastringorbuffer.Canbesetdirectlyorusingsomeoftheresponsemethods.
context: Context
Theservicecontextinwhichtherouterwasmounted(ratherthanthecontextinwhichtheroutewasdefined).
headers: object
Therawheadersobject.
statusCode: number
Statuscodeoftheresponse.Defaultsto 200(bodysetandnotanemptystringorbuffer)or 204(otherwise)ifnotchangedfrom
undefined.
attachment
res.attachment([filename]):this
Setsthe content-dispositionheadertoindicatetheresponseisadownloadablefilewiththegivenname.
Arguments
filename: string(optional)
Nameofthedownloadablefileintheresponsebody.
Ifpresent,theextensionofthefilenamewillbeusedtosettheresponse content-typeifithasnotyetbeenset.
Returnstheresponseobject.
cookie
res.cookie(name,value,[options]):this
Setsacookiewiththegivenname.
Arguments
name: string
Nameofthecookie.
value: string
Valueofthecookie.
Secretthatwillbeusedtosignthecookie.
Ifasecretisspecified,thecookie'ssignaturewillbestoredinasecondcookiewiththesameoptions,thesamenameandthe
suffix .sig.Otherwisenosignaturewillbeadded.
algorithm: string(Default: "sha256")
Response
188
Algorithmthatwillbeusedtosignthecookie.
ttl: number(optional)
Timetoliveofthecookie.
path: number(optional)
Pathofthecookie.
domain: number(optional)
Domainofthecookie.
secure: number(Default: false)
Whetherthecookieshouldbemarkedassecure(i.e.HTTPS/SSL-only).
httpOnly: boolean(Default: false)
WhetherthecookieshouldbemarkedasHTTP-only.
Ifanumberispassedinsteadofanoptionsobjectitwillbeinterpretedasthettloption.
download
res.download(path,[filename]):this
Theequivalentofcalling res.attachment(filename).sendFile(path).
Arguments
path: string
Pathtothefileonthelocalfilesystemtobesentastheresponsebody.
filename: string(optional)
Filenametoindicateinthe content-dispositionheader.
Ifomittedthepathwillbeusedinstead.
getHeader
res.getHeader(name):string
Getsthevalueoftheheaderwiththegivenname.
Arguments
name: string
Nameoftheheadertoget.
Returnsthevalueoftheheaderor undefined.
json
res.json(data):this
SetstheresponsebodytotheJSONstringvalueofthegivendata.
Response
189
Arguments
data: any
Thedatatobeusedastheresponsebody.
redirect
res.redirect([status],path):this
Redirectstheresponsebysettingtheresponse locationheaderandstatuscode.
Arguments
status: number|string(optional)
Responsestatuscodetoset.
Ifthestatuscodeisthestringvalue "permanent"itwillbetreatedasthevalue 301.
Ifthestatuscodeisastringitwillbeconvertedtoanumericstatuscodeusingthestatusesmodulefirst.
Ifthestatuscodeisomittedbuttheresponsestatushasnotalreadybeenset,theresponsestatuswillbesetto 302.
path: string
URLtosetthe locationheaderto.
removeHeader
res.removeHeader(name):this
Removestheheaderwiththegivennamefromtheresponse.
Arguments
name: string
Nameoftheheadertoremove.
send
res.send(data,[type]):this
Setstheresponsebodytothegivendatawithrespecttotheresponsedefinitionfortheresponse'scurrentstatuscode.
Arguments
data: any
Thedatatobeusedastheresponsebody.Willbeconvertedaccordingtheresponsedefinitionfortheresponse'scurrentstatuscode
(or 200)inthefollowingway:
IfthedataisanArangoDBresultset,itwillbeconvertedtoanarrayfirst.
Iftheresponsedefinitionspecifiesamodelwitha forClientmethod,thatmethodwillbeappliedtothedatafirst.Ifthedataisan
arrayandtheresponsedefinitionhasthe multipleflagset,themethodwillbeappliedtoeachentryindividuallyinstead.
Finallythedatawillbeprocessedbytheresponsetypehandlertoconvertheresponsebodytoastringorbuffer.
type: string(Default: "auto")
Response
190
Content-typeoftheresponsebody.
Ifsetto "auto"thefirstM IM Etypespecifiedintheresponsedefinitionfortheresponse'scurrentstatuscode(or 200)willbe
usedinstead.
Ifsetto "auto"andnoresponsedefinitionexists,theM IM Etypewillbedeterminedthefollowingway:
IfthedataisabuffertheM IM Etypewillbesettobinary( application/octet-stream).
IfthedataisanobjecttheM IM EtypewillbesettoJSONandthedatawillbeconvertedtoaJSONstring.
OtherwisetheM IM EtypewillbesettoHTM Landthedatawillbeconvertedtoastring.
sendFile
res.sendFile(path,[options]):this
Sendsafilefromthelocalfilesystemastheresponsebody.
Arguments
path: string
Pathtothefileonthelocalfilesystemtobesentastheresponsebody.
Ifno content-typeheaderhasbeensetyet,theextensionofthefilenamewillbeusedtosetthevalueofthatheader.
lastModified: boolean(optional)
Ifsetto trueorifno last-modifiedheaderhasbeensetyetandthevalueisnotsetto falsethe last-modifiedheader
willbesettothemodificationdateofthefileinmilliseconds.
Examples
//Sendthefile"favicon.ico"fromthisservice'sfolder
res.sendFile(module.context.fileName('favicon.ico'));
sendStatus
res.sendStatus(status):this
Sendsaplaintextresponseforthegivenstatuscode.Theresponsestatuswillbesettothegivenstatuscode,theresponsebodywillbe
settothestatusmessagecorrespondingtothatstatuscode.
Arguments
setHeader/set
res.setHeader(name,value):this
res.set(name,value):this
Response
191
res.set(headers):this
Setsthevalueoftheheaderwiththegivenname.
Arguments
name: string
Nameoftheheadertoset.
value: string
Valuetosettheheaderto.
headers: object
Headerobjectmappingheadernamestovalues.
status
res.status(status):this
Setstheresponsestatustothegivenstatuscode.
Arguments
throw
res.throw(status,[reason],[options]):void
ThrowsanHTTPexceptionforthegivenstatus,whichwillbehandledbyFoxxtoservetheappropriateJSONerrorresponse.
Arguments
Ifthestatuscodeisinthe500-range(500-599),itsstacktracewillalwaysbeloggedasifitwereanunhandledexception.
Ifdevelopmentmodeisenabled,theerror'sstacktracewillbeloggedasawarningifthestatuscodeisinthe400-range(400-499)or
asaregularmessageotherwise.
reason: string(optional)
M essagefortheexception.
Ifomitted,thestatusmessagecorrespondingtothestatuscodewillbeusedinstead.
cause: Error(optional)
Causeoftheexceptionthatwillbeloggedaspartoftheerror'sstacktrace(recursively,iftheexceptionalsohasa cause
propertyandsoon).
Response
192
extra: object(optional)
AdditionalpropertiesthatwillbeaddedtotheerrorresponsebodygeneratedbyFoxx.
Ifdevelopmentmodeisenabled,an exceptionpropertywillbeaddedtothisvaluecontainingtheerrormessageanda
stacktracepropertywillbeaddedcontaininganarraywitheachlineoftheerror'sstacktrace.
Ifanerrorispassedinsteadofanoptionsobjectitwillbeinterpretedasthecauseoption.Ifnoreasonwasprovidedtheerror's message
willbeusedasthereasoninstead.
Returnsnothing.
vary
res.vary(names):this
res.vary(...names):this
Thismethodwrapsthe varyheadermanipulationmethodofthevarymoduleforthecurrentresponse.
Thegivennameswillbeaddedtotheresponse's varyheaderifnotalreadypresent.
Examples
res.vary('user-agent');
res.vary('cookie');
res.vary('cookie');//duplicateswillbeignored
//--or-res.vary('user-agent','cookie');
//--or-res.vary(['user-agent','cookie']);
write
res.write(data):this
Appendsthegivendatatotheresponsebody.
Arguments
data: string|Buffer
Datatoappend.
Ifthedataisabuffertheresponsebodywillbeconvertedtoabufferfirst.
Iftheresponsebodyisabufferthedatawillbeconvertedtoabufferfirst.
IfthedataisanobjectitwillbeconvertedtoaJSONstringfirst.
Ifthedataisanyothernon-stringvalueitwillbeconvertedtoastringfirst.
Response
193
SessionMiddleware
constsessionMiddleware=require('@arangodb/foxx/sessions');
Thesessionmiddlewareaddsthe sessionand sessionStoragepropertiestotherequestobjectanddealswithserializingand
deserializingthesessionaswellasextractingsessionidentifiersfromincomingrequestsandinjectingthemintooutgoingresponses.
Examples
//Createasessionmiddleware
constsessions=sessionsMiddleware({
storage:module.context.collection('sessions'),
transport:['header','cookie']
});
//Firstenablethemiddlewareforthisservice
module.context.use(sessions);
//Nowmounttheroutersthatusethesession
router.get('/',function(req,res){
res.send(`Hello${req.session.uid||'anonymous'}!`);
},'hello');
router.post('/login',function(req,res){
req.session.uid=req.body;
req.sessionStorage.save(req.session);
res.redirect(req.reverse('hello'));
});
.body(['text/plain'],'Username');
Creatingasessionmiddleware
sessionMiddleware(options):Middleware
Createsasessionmiddleware.
Arguments
options: Object
Anobjectwiththefollowingproperties:
storage: Storage
Storagethatwillbeusedtopersistthesessions.
Thestorageisalsoexposedasthe sessionStorageonallrequestobjectsandasthe storagepropertyofthemiddleware.
IfastringorcollectionispassedinsteadofaStorage,itwillbeusedtocreateaCollectionStorage.
transport: Transport|Array<Transport>
Transportorarrayoftransportsthatwillbeusedtoextractthesessionidentifiersfromincomingrequestsandinjecttheminto
outgoingresponses.Whenattemptingtoextractasessionidentifier,thetransportswillbeusedintheorderspecifieduntila
matchisfound.Wheninjecting(orclearing)sessionidentifiers,alltransportswillbeinvoked.
Thetransportsarealsoexposedasthe transportpropertyofthemiddleware.
Ifthestring "cookie"ispassedinsteadofaTransport,theCookieTransportwillbeusedwiththedefaultsettingsinstead.
Ifthestring "header"ispassedinsteadofaTransport,theHeaderTransportwillbeusedwiththedefaultsettingsinstead.
autoCreate: boolean(Default: true)
Ifenabledthesessionstorage's newmethodwillbeinvokedtocreateanemptysessionwheneverthetransportfailedtoreturn
asessionfortheincomingrequest.Otherwisethesessionwillbeinitializedas null.
Sessionsmiddleware
194
Returnsthesessionmiddleware.
Sessionsmiddleware
195
SessionStorages
Sessionstoragesareusedbythesessionsmiddlewaretopersistsessionsacrossrequests.Sessionstoragesmustimplementthe
fromClientand forClientmethodsandcanoptionallyimplementthe newmethod.
Thebuilt-insessionstoragesgenerallyprovidethefollowingattributes:
uid: string(Default: null)
Auniqueidentifierindicatingtheactiveuser.
created: number(Default: Date.now())
Thenumerictimestampofwhenthesessionwascreated.
data: any(Default: null)
Arbitrarydatatopersistedinthesession.
new
storage.new():Session
Generatesanewsessionobjectrepresentinganemptysession.Theemptysessionobjectshouldnotbepersistedunlessnecessary.The
returnvaluewillbeexposedbythemiddlewareasthe sessionpropertyoftherequestobjectifnosessionidentifierwasreturnedby
thesessiontransportsandauto-creationisnotexplicitlydisabledinthesessionmiddleware.
Examples
new(){
return{
uid:null,
created:Date.now(),
data:null
};
}
fromClient
storage.fromClient(sid):Session|null
Resolvesordeserializesasessionidentifiertoasessionobject.
Arguments
sid: string
Sessionidentifiertoresolveordeserialize.
Returnsasessionobjectrepresentingthesessionwiththegivensessionidentifierthatwillbeexposedbythemiddlewareasthe
sessionpropertyoftherequestobject.Thismethodwillonlybecalledifanyofthesessiontransportsreturnedasessionidentifier.If
thesessionidentifierisinvalidorexpired,themethodshouldreturna nullvaluetoindicatenomatchingsession.
Examples
fromClient(sid){
returndb._collection('sessions').firstExample({_key:sid});
}
forClient
storage.forClient(session):string|null
Sessionstorages
196
Derivesasessionidentifierfromthegivensessionobject.
Arguments
session: Session
Sessiontoderiveasessionidentifierfrom.
Returnsasessionidentifierforthesessionrepresentedbythegivensessionobject.Thismethodwillbecalledwiththe session
propertyoftherequestobjectunlessthatpropertyisempty(e.g. null).
Examples
forClient(session){
if(!session._key){
constmeta=db._collection('sessions').save(session);
returnmeta._key;
}
db._collection('sessions').replace(session._key,session);
returnsession._key;
}
Sessionstorages
197
CollectionSessionStorage
constcollectionStorage=require('@arangodb/foxx/sessions/storages/collection');
Thecollectionsessionstoragepersistssessionstoacollectioninthedatabase.
Creatingastorage
collectionStorage(options):Storage
CreatesaStoragethatcanbeusedinthesessionsmiddleware.
Arguments
options: Object
collection: ArangoCollection
Thecollectionthatshouldbeusedtopersistthesessions.Ifastringispassedinsteadofacollectionitisassumedtobethe
fullyqualifiednameofacollectioninthecurrentdatabase.
ttl: number(Default: 60*60)
Thetimeinsecondssincethelastupdateuntilasessionwillbeconsideredexpired.
pruneExpired: boolean(Default: false)
Whetherexpiredsessionsshouldberemovedfromthecollectionwhentheyareaccessedinsteadofsimplybeingignored.
autoUpdate: boolean(Default: true)
Whethersessionsshouldbeupdatedinthecollectioneverytimetheyareaccessedtokeepthemfromexpiring.Disablingthis
optionwillimproveperformancebutmeansyouwillhavetotakecareofkeepingyoursessionsaliveyourself.
Ifastringorcollectionispassedinsteadofanoptionsobject,itwillbeinterpretedasthecollectionoption.
prune
storage.prune():Array<string>
Removesallexpiredsessionsfromthecollection.ThismethodshouldbecalledevenifthepruneExpiredoptionisenabledtocleanup
abandonedsessions.
Returnsanarrayofthekeysofallsessionsthatwereremoved.
save
storage.save(session):Session
Saves(replaces)thegivensessionobjectinthecollection.Thismethodneedstobeinvokedexplicitlyaftermakingchangestothesession
orthechangeswillnotbepersisted.Assignsanew _keytothesessionifitpreviouslydidnothaveone.
Arguments
session: Session
Asessionobject.
Returnsthemodifiedsession.
clear
Sessionstorages
198
storage.clear(session):boolean
Removesthesessionfromthecollection.Hasnoeffectifthesessionwasalreadyremovedorhasnotyetbeensavedtothecollection(i.e.
hasno _key).
Arguments
session: Session
Asessionobject.
Returns trueifthesessionwasremovedor falseifithadnoeffect.
Sessionstorages
199
JWTSessionStorage
constjwtStorage=require('@arangodb/foxx/sessions/storages/jwt');
TheJWTsessionstorageconvertssessionstoandfromJSONWebTokens.
Examples
//PassinasecuresecretfromtheFoxxconfiguration
constsecret=module.context.configuration.jwtSecret;
storage:jwtStorage(secret),
transport:'header'
});
Creatingastorage
jwtStorage(options):Storage
CreatesaStoragethatcanbeusedinthesessionsmiddleware.
Note:whilethe"none"algorithm(i.e.nosignature)issupportedthisdummyalgorithmprovidesnosecurityandallowsclientstomake
arbitrarymodificationstothepayloadandshouldnotbeusedunlessyouarecertainyouspecificallyneedit.
Arguments
options: Object
algorithm: string(Default: "HS512")
Thealgorithmtouseforsigningthetoken.
Supportedvalues:
"HS256"(HM AC-SHA256)
"none"(nosignature)
secret: string
Thesecrettouseforsigningthetoken.
Thisfieldisforbiddenwhenusingthe"none"algorithmbutrequiredotherwise.
ttl: number(Default: 3600)
Themaximumlifetimeofthetokeninseconds.Youmaywanttokeepthisshortasanewtokenisgeneratedoneveryrequest
allowingclientstorefreshtokensautomatically.
verify: boolean(Default: true)
Ifsetto falsethesignaturewillnotbeverifiedbutstillgenerated(unlessusingthe"none"algorithm).
Sessionstorages
200
SessionTransports
Sessiontransportsareusedbythesessionsmiddlewaretostoreandretrievesessionidentifiersinrequestsandresponses.Session
transportsmustimplementthe getand/or setmethodsandcanoptionallyimplementthe clearmethod.
get
transport.get(request):string|null
Retrievesasessionidentifierfromarequestobject.
Ifpresentthismethodwillautomaticallybeinvokedforeachtransportuntilatransportreturnsasessionidentifier.
Arguments
request: Request
Requestobjecttoextractasessionidentifierfrom.
Returnsthesessionidentifieror nullifthetransportcannotfindasessionidentifierintherequest.
Examples
get(req){
returnreq.get('x-session-id')||null;
}
set
transport.set(response,sid):void
Attachesasessionidentifiertoaresponseobject.
Ifpresentthismethodwillautomaticallybeinvokedattheendofarequestregardlessofwhetherthesessionwasmodifiedornot.
Arguments
response: Response
Responseobjecttoattachasessionidentifierto.
sid: string
Sessionidentifiertoattachtotheresponse.
Returnsnothing.
Examples
set(res){
res.set('x-session-id',value);
}
clear
transport.clear(response):void
Attachesapayloadindicatingthatthesessionhasbeenclearedtotheresponseobject.Thiscanbeusedtoclearasessioncookiewhenthe
sessionhasbeendestroyed(e.g.duringlogout).
Ifpresentthismethodwillautomaticallybeinvokedinsteadof setwhenthe req.sessionattributewasremovedbytheroutehandler.
Sessiontransports
201
Arguments
response: Response
Responseobjecttoremovethesessionidentifierfrom.
Returnsnothing.
Sessiontransports
202
CookieSessionTransport
constcookieTransport=require('@arangodb/foxx/sessions/transports/cookie');
Thecookietransportstoressessionidentifiersincookiesontherequestandresponseobject.
Examples
//PassinasecuresecretfromtheFoxxconfiguration
constsecret=module.context.configuration.cookieSecret;
transport:cookieTransport({
name:'FOXXSESSID',
ttl:60*60*24*7,//oneweekinseconds
algorithm:'sha256',
secret:secret
})
});
Creatingatransport
cookieTransport([options]):Transport
CreatesaTransportthatcanbeusedinthesessionsmiddleware.
Arguments
name: string(Default: "sid")
Thenameofthecookie.
ttl: number(optional)
Cookielifetimeinseconds.
algorithm: string(optional)
Thealgorithmusedtosignandverifythecookie.Ifnoalgorithmisspecified,thecookiewillnotbesignedorverified.Seethe
cookiemethodontheresponseobject.
Secrettouseforthesignedcookie.Willbeignoredifnoalgorithmisprovided.
Ifastringispassedinsteadofanoptionsobject,itwillbeinterpretedasthenameoption.
Sessiontransports
203
HeaderSessionTransport
constheaderTransport=require('@arangodb/foxx/sessions/transports/header');
Theheadertransportstoressessionidentifiersinheadersontherequestandresponseobjects.
Examples
transport:headerTransport('X-FOXXSESSID')
});
Creatingatransport
headerTransport([options]):Transport
CreatesaTransportthatcanbeusedinthesessionsmiddleware.
Arguments
name: string(Default: X-Session-Id)
Nameoftheheaderthatcontainsthesessionidentifier(notcasesensitive).
Ifastringispassedinsteadofanoptionsobject,itwillbeinterpretedasthenameoption.
Sessiontransports
204
Staticfileassets
ThemostflexiblewaytoservefilesinyourFoxxserviceistosimplypassthemthroughinyourrouterusingthecontextobject's
fileNamemethodandtheresponseobject's sendFilemethod:
router.get('/some/filename.png',function(req,res){
constfilePath=module.context.fileName('some-local-filename.png');
res.sendFile(filePath);
});
Whileallowingforgreatercontrolofhowthefileshouldbesenttotheclientandwhoshouldbeabletoaccessit,doingthisforallyour
staticassetscangettedious.
AlternativelyyoucanspecifyfileassetsthatshouldbeservedbyyourFoxxservicedirectlyintheservicemanifestusingthe files
attribute:
"files":{
"/some/filename.png":{
"path":"some-local-filename.png",
"type":"image/png",
"gzip":false
},
"/favicon.ico":"bookmark.ico",
"/static":"my-assets-folder"
}
Eachentryinthe filesattributecanrepresenteitherasinglefileoradirectory.Whenservingentiredirectories,thekeyactsasaprefix
andrequeststothatprefixwillberesolvedwithinthegivendirectory.
Options
path: string
Therelativepathofthefileorfolderwithintheservice.
type: string(optional)
TheM IM Econtenttypeofthefile.Defaultstoanintelligentguessbasedonthefilename'sextension.
gzip: boolean(Default: false)
Ifsetto truethefilewillbeservedwithgzip-encodingifsupportedbytheclient.Thiscanbeusefulwhenservingtextfileslike
client-sideJavaScript,CSSorHTM L.
Ifastringisprovidedinsteadofanobject,itwillbeinterpretedasthepathoption.
Servingfiles
205
Foxxscriptsandqueuedjobs
Foxxletsyoudefinescriptsthatcanbeexecutedaspartoftheinstallationandremovalprocess,invokedmanuallyorscheduledtorunat
alatertimeusingthejobqueue.
Toregisteryourscript,justadda scriptssectiontoyourservicemanifest:
{
...
"scripts":{
"setup":"scripts/setup.js",
"send-mail":"scripts/send-mail.js"
}
...
}
Thescriptsyoudefineinyourservicemanifestcanbeinvokedfromthewebinterfaceintheservice'ssettingspagewiththeScripts
dropdown.
Youcanalsousethescriptsasqueuedjobs:
'usestrict';
constqueues=require('@arangodb/foxx/queues');
queues.get('default').push(
{mount:'/my-service-mount-point',name:'send-mail'},
{to:'[email protected]',body:'Hello'}
);
Scriptargumentsandreturnvalues
Ifthescriptwasinvokedwithanyarguments,youcanaccessthemusingthe module.context.argvarray.
Toreturndatafromyourscript,youcanassignthedatato module.exportsasusual.Pleasenotethatthisdatawillbeconvertedto
JSON.
Anyerrorsraisedbythescriptwillbehandleddependingonhowthescriptwasinvoked:
ifthescriptwasinvokedfromtheHTTPAPI(e.g.usingthewebinterface),itwillreturnanerrorresponseusingtheexception's
statusCodepropertyifspecifiedor500.
ifthescriptwasinvokedfromaFoxxjobqueue,thejob'sfailurecounterwillbeincrementedandthejobwillberescheduledor
markedasfailedifnoattemptsremain.
Examples
Let'ssayyouwanttodefineascriptthattakestwonumericvaluesandreturnstheresultofmultiplyingthem:
'usestrict';
constassert=require('assert');
constargv=module.context.argv;
assert.equal(argv.length,2,'Expectedexactlytwoarguments');
assert.equal(typeofargv[0],'number','Expectedfirstargumenttobeanumber');
assert.equal(typeofargv[1],'number','Expectedsecondargumenttobeanumber');
module.exports=argv[0]*argv[1];
LifecycleScripts
Foxxrecognizeslifecyclescriptsiftheyaredefinedandwillinvokethemduringtheinstallation,updateandremovalprocessofthe
serviceifyouwant.
206
Thefollowingscriptsarecurrentlyrecognizedaslifecyclescriptsbytheirname: "setup"and "teardown".
SetupScript
ThesetupscriptwillbeexecutedwithoutargumentsduringtheinstallationofyourFoxxservice.
Thesetupscriptistypicallyusedtocreatecollectionsyourserviceneedsorinsertseeddatalikeinitialadministrativeuseraccountsand
soon.
Examples
'usestrict';
consttextsCollectionName=module.context.collectionName('texts');
//`textsCollectionNameìsnowtheprefixednameofthisservice's"texts"collection.
//e.g."example_texts"iftheservicehasbeenmountedat`/example`
if(db._collection(textsCollectionName)===null){
constcollection=db._create(textsCollectionName);
collection.save({text:'entry1fromcollectiontexts'});
}else{
console.log(`collection${texts}alreadyexists.Leavingituntouched.`);
}
TeardownScript
TheteardownscriptwillbeexecutedwithoutargumentsduringtheremovalofyourFoxxservice.
Itcanalsooptionallybeexecutedbeforeupgradinganservice.
Thisscripttypicallyremovesthecollectionsand/ordocumentscreatedbyyourservice'ssetupscript.
Examples
'usestrict';
consttextsCollection=module.context.collection('texts');
if(textsCollection){
textsCollection.drop();
}
Queues
Foxxallowsdefiningjobqueuesthatletyouperformsloworexpensiveactionsasynchronously.Thesequeuescanbeusedtosendemails,callexternalAPIsorperformotheractionsthatyoudonotwanttoperformdirectlyorwanttoretryonfailure.
enableordisabletheFoxxqueuesfeature --foxx.queuesflagIftrue,theFoxxqueueswillbeavailableandjobsinthequeueswillbe
executedasynchronously.Thedefaultistrue.Whensetto falsethequeuemanagerwillbedisabledandanyjobsarepreventedfrom
beingprocessed,whichmayreduceCPUloadabit.PleasenotethatFoxxjobqueuesaredatabase-specific.Queuesandjobsarealways
relativetothedatabaseinwhichtheyarecreatedoraccessed.
pollintervalforFoxxqueues --foxx.queues-poll-intervalvalueThepollintervalfortheFoxxqueuesmanager.Thevalueisspecifiedin
seconds.LowervalueswillmeanmoreimmediateandmorefrequentFoxxqueuejobexecution,butwillmakethequeuethreadwakeup
andquerythequeuesmoreoften.Whensettoalowvalue,thequeuethreadmightcauseCPUload.Thedefaultis1second.IfFoxx
queuesarenotusedmuch,thenthisvaluemaybeincreasedtomakethequeuesthreadwakeupless.Forthelow-levelfunctionalitysee
thechapteronthetaskmanagementmodule.
Creatingorupdatingaqueue
207
queues.create(name,[maxWorkers]):Queue
Returnsthequeueforthegivenname.Ifthequeuedoesnotexist,anewqueuewiththegivennamewillbecreated.Ifaqueuewiththe
givennamealreadyexistsandmaxWorkersisset,thequeue'smaximumnumberofworkerswillbeupdated.Thequeuewillbecreatedin
thecurrentdatabase.
Arguments
name: string
Nameofthequeuetocreate.
maxWorkers: number(Default: 1)
Themaximumnumberofworkers.
Examples
//Createaqueuewiththedefaultnumberofworkers(i.e.one)
constqueue1=queues.create("my-queue");
//Createaqueuewithagivennumberofworkers
constqueue2=queues.create("another-queue",2);
//Updatethenumberofworkersofanexistingqueue
constqueue3=queues.create("my-queue",10);
//queue1andqueue3refertothesamequeue
assertEqual(queue1,queue3);
Fetchinganexistingqueue
queues.get(name):Queue
Returnsthequeueforthegivenname.Ifthequeuedoesnotexistanexceptionisthrowninstead.
Thequeuewillbelookedupinthecurrentdatabase.
Arguments
name: string
Nameofthequeuetofetch.
Examples
Ifthequeuedoesnotyetexistanexceptionisthrown:
queues.get("some-queue");
//Error:Queuedoesnotexist:some-queue
//at...
Otherwisethequeuewillbereturned:
constqueue1=queues.create("some-queue");
constqueue2=queues.get("some-queue");
assertEqual(queue1,queue2);
Deletingaqueue
queues.delete(name):boolean
Returns trueifthequeuewasdeletedsuccessfully.Ifthequeuedidnotexist,itreturns falseinstead.Thequeuewillbelookedup
anddeletedinthecurrentdatabase.
Whenaqueueisdeleted,jobsonthatqueuewillnolongerbeexecuted.
Deletingaqueuewillnotdeleteanyjobsonthatqueue.
Arguments
name: string
208
Nameofthequeuetodelete.
Examples
constqueue=queues.create("my-queue");
queues.delete("my-queue");//true
queues.delete("my-queue");//false
Addingajobtoaqueue
queue.push(script,data,[opts]):string
Thejobwillbeaddedtothespecifiedqueueinthecurrentdatabase.
Returnsthejobid.
Arguments
script: object
Ajobtypedefinition,consistingofanobjectwiththefollowingproperties:
name: string
Nameofthescriptthatwillbeinvoked.
mount: string
M ountpathoftheservicethatdefinesthescript.
backOff: Function|number(Default: 1000)
Eitherafunctionthattakesthenumberoftimesthejobhasfailedbeforeasinputandreturnsthenumberofmillisecondsto
waitbeforetryingthejobagain,orthedelaytobeusedtocalculateanexponentialback-off,or 0fornodelay.
maxFailures: number|Infinity(Default: 0):
Numberoftimesasinglerunofajobwillbere-triedbeforeitismarkedas "failed".Anegativevalueor Infinitymeans
thatthejobwillbere-triedonfailureindefinitely.
Schematovalidateajob'sdataagainstbeforeenqueuingthejob.
preprocess: Function(optional)
Functiontopre-processajob's(validated)databeforeserializingitinthequeue.
repeatTimes: Function(Default: 0)
Ifsettoapositivenumber,thejobwillberepeatedthismanytimes(notcountingrecoverywhenusingmaxFailures).Ifsetto
anegativenumberor Infinity,thejobwillberepeatedindefinitely.Ifsetto 0thejobwillnotberepeated.
repeatUntil: number|Date(optional)
Ifthejobissettoautomaticallyrepeat,thiscanbesettoatimestampinmilliseconds(or Dateinstance)afterwhichthejob
willnolongerrepeat.Settingthisvaluetozero,anegativevalueor Infinityhasnoeffect.
repeatDelay: number(Default: 0)
Ifthejobissettoautomaticallyrepeat,thiscanbesettoanon-negativevaluetosetthenumberofmillisecondsforwhichthe
jobwillbedelayedbeforeitisstartedagain.
data: any
Jobdataofthejob;mustbeserializabletoJSON.
opts: object(optional)
Objectwithanyofthefollowingproperties:
209
success: Function(optional)
Functiontobecalledafterthejobhasbeencompletedsuccessfully.
failure: Function(optional)
Functiontobecalledafterthejobhasfailedtoomanytimes.
delayUntil: number|Date(Default: Date.now())
Timestampinmilliseconds(or Dateinstance)untilwhichtheexecutionofthejobshouldbedelayed.
backOff: Function|number(Default: 1000)
Seescript.backOff.
maxFailures: number|Infinity(Default: 0):
Seescript.maxFailures.
repeatTimes: Function(Default: 0)
Seescript.repeatTimes.
repeatUntil: number|Date(optional)
Seescript.repeatUntil.
repeatDelay: number(Default: 0)
Seescript.repeatDelay.
NotethatifyoupassafunctionforthebackOffcalculation,successcallbackorfailurecallbackoptionsthefunctionwillbeserializedto
thedatabaseasastringandthereforemustnotrelyonanyexternalscopeorexternalvariables.
Whenthejobissettoautomaticallyrepeat,thefailurecallbackwillonlybeexecutedwhenarunofthejobhasfailedmorethan
maxFailurestimes.NotethatifthejobfailsandmaxFailuresisset,itwillberescheduledaccordingtothebackOffuntilithaseither
failedtoomanytimesorcompletedsuccessfullybeforebeingscheduledaccordingtotherepeatDelayagain.Recoveryattemptsby
maxFailuresdonotcounttowardsrepeatTimes.
Thesuccessandfailurecallbacksreceivethefollowingarguments:
result: any
Thereturnvalueofthescriptforthecurrentrunofthejob.
jobData: any
Thedatapassedtothismethod.
job: object
ArangoDBdocumentrepresentingthejob'scurrentstate.
Examples
Let'ssaywehaveanservicemountedat /mailerthatprovidesascriptcalled send-mail:
'usestrict';
constqueue=queues.create('my-queue');
queue.push(
{mount:'/mailer',name:'send-mail'},
{to:'[email protected]',body:'Helloworld'}
);
Thiswillnotwork,because logwasdefinedoutsidethecallbackfunction(thecallbackmustbeserializabletoastring):
210
//WARNING:THISDOESNOTWORK!
'usestrict';
constlog=require('console').log;//outsidethecallback'sfunctionscope
queue.push(
{to:'[email protected]',body:'Helloworld'},
{success:function(){
log('Yay!');//throws'logisnotdefined'
}}
);
Here'sanexampleofajobthatwillbeexecutedevery5secondsuntiltomorrow:
'usestrict';
constqueues=require('@arangodb/foxx').queues;
queue.push(
{to:'[email protected]',body:'Helloworld'},
{
repeatTimes:Infinity,
repeatUntil:Date.now()+(24*60*60*1000),
repeatDelay:5*1000
}
);
Fetchingajobfromthequeue
queue.get(jobId):Job
Createsaproxyobjectrepresentingajobwiththegivenjobid.
Thejobwillbelookedupinthespecifiedqueueinthecurrentdatabase.
ReturnsthejobforthegivenjobId.Propertiesofthejobobjectwillbefetchedwhenevertheyarereferencedandcannotbemodified.
Arguments
jobId: string
Theidofthejobtocreateaproxyobjectfor.
Examples
constjobId=queue.push({mount:'/logger',name:'log'},'HelloWorld!');
constjob=queue.get(jobId);
assertEqual(job.id,jobId);
Deletingajobfromthequeue
queue.delete(jobId):boolean
Deletesajobwiththegivenjobid.Thejobwillbelookedupanddeletedinthespecifiedqueueinthecurrentdatabase.
Arguments
jobId: string
Theidofthejobtodelete.
Returns trueifthejobwasdeletedsuccessfully.Ifthejobdidnotexistitreturns falseinstead.
Fetchinganarrayofjobsinaqueue
Examples
211
constlogScript={mount:'/logger',name:'log'};
queue.push(logScript,'HelloWorld!',{delayUntil:Date.now()+50});
assertEqual(queue.pending(logScript).length,1);
//50mslater...
assertEqual(queue.pending(logScript).length,0);
assertEqual(queue.progress(logScript).length,1);
//evenlater...
assertEqual(queue.progress(logScript).length,0);
assertEqual(queue.complete(logScript).length,1);
Fetchinganarrayofpendingjobsinaqueue
queue.pending([script]):Array<string>
Returnsanarrayofjobidsofjobsinthegivenqueuewiththestatus "pending",optionallyfilteredbythegivenjobtype.Thejobswill
belookedupinthespecifiedqueueinthecurrentdatabase.
Arguments
script: object(optional)
name: string
Nameofthescript.
mount: string
M ountpathoftheservicedefiningthescript.
Fetchinganarrayofjobsthatarecurrentlyinprogress
queue.progress([script])
Returnsanarrayofjobidsofjobsinthegivenqueuewiththestatus "progress",optionallyfilteredbythegivenjobtype.Thejobs
willbelookedupinthespecifiedqueueinthecurrentdatabase.
Arguments
name: string
Nameofthescript.
mount: string
Fetchinganarrayofcompletedjobsinaqueue
queue.complete([script]):Array<string>
Returnsanarrayofjobidsofjobsinthegivenqueuewiththestatus "complete",optionallyfilteredbythegivenjobtype.Thejobs
willbelookedupinthespecifiedqueueinthecurrentdatabase.
Arguments
name: string
Nameofthescript.
mount: string
212
Fetchinganarrayoffailedjobsinaqueue
queue.failed([script]):Array<string>
Returnsanarrayofjobidsofjobsinthegivenqueuewiththestatus "failed",optionallyfilteredbythegivenjobtype.Thejobswill
belookedupinthespecifiedqueueinthecurrentdatabase.
Arguments
name: string
Nameofthescript.
mount: string
Fetchinganarrayofalljobsinaqueue
queue.all([script]):Array<string>
Returnsanarrayofjobidsofalljobsinthegivenqueue,optionallyfilteredbythegivenjobtype.Thejobswillbelookedupinthe
specifiedqueueinthecurrentdatabase.
Arguments
name: string
Nameofthescript.
mount: string
Abortingajob
job.abort():void
Abortsanon-completedjob.
Setsajob'sstatusto "failed"ifitisnotalready "complete",withoutcallingthejob'sonFailurecallback.
213
Migrating2.xservicesto3.0
WhenmigratingservicesfromolderversionsofArangoDBitisgenerallyrecommendedyoumakesuretheyworkinlegacycompatibility
mode,whichcanalsoserveasastop-gapsolution.
ThischapteroutlinesthemajordifferencesintheFoxxAPIbetweenArangoDB2.8andArangoDB3.0.
Generalchanges
The consoleobjectinlaterversionsofArangoDB2.ximplementedaspecialFoxxconsoleAPIandwouldoptionallylogmessagestoa
collection.ArangoDB3.0restorestheoriginalbehaviourwhere consoleisthesameobjectavailablefromtheconsolemodule.
M igrating2.xservices
214
Migratingfrompre-2.8
WhenmigratingfromaversionolderthanArangoDB2.8pleasenotethatstartingwithArangoDB2.8thebehaviourofthe require
functionmorecloselymimicsthebehaviourobservedinNode.jsandmodulebundlersforbrowsers,e.g.:
Inafile /routes/examples.js(relativetotherootfolderoftheservice):
require('./my-module')willbeattemptedtoberesolvedinthefollowingorder:
1.
/routes/my-module(relativetoserviceroot)
2.
/routes/my-module.js(relativetoserviceroot)
3.
/routes/my-module.json(relativetoserviceroot)
4.
/routes/my-module/index.js(relativetoserviceroot)
5.
/routes/my-module/index.json(relativetoserviceroot)
require('lodash')willbeattemptedtoberesolvedinthefollowingorder:
1.
/routes/node_modules/lodash(relativetoserviceroot)
2.
/node_modules/lodash(relativetoserviceroot)
3. ArangoDBmodule lodash
4. Nodecompatibilitymodule lodash
5. BundledNPM module lodash
require('/abs/path')willbeattemptedtoberesolvedinthefollowingorder:
1.
/abs/path(relativetofilesystemroot)
2.
/abs/path.js(relativetofilesystemroot)
3.
/abs/path.json(relativetofilesystemroot)
4.
/abs/path/index.js(relativetofilesystemroot)
5.
/abs/path/index.json(relativetofilesystemroot)
ThisbehaviourisincompatiblewiththesourcecodegeneratedbytheFoxxgeneratorinthewebinterfacebeforeArangoDB2.8.
Note:The org/arangodbmoduleisaliasedtothenewname @arangodbinArangoDB3.0.0andthe @arangodbmodulewasaliasedto
theoldname org/arangodbinArangoDB2.8.0.Eitheronewillworkin2.8and3.0butoutsideoflegacyservicesyoushoulduse
@arangodbgoingforward.
Foxxqueue
InArangoDB2.6FoxxintroducedanewwaytodefinequeuedjobsusingFoxxscriptstoreplacethefunction-basedjobtypedefinitions
whichwerecausingproblemswhenrestartingtheserver.Thefunction-basedjobshavebeenremovedin2.7andarenolongersupported
atall.
CoffeeScript
ArangoDB3.0nolongerprovidesbuilt-insupportforCoffeeScriptsourcefiles,eveninlegacycompatibilitymode.Ifyouwanttousean
alternativelanguagelikeCoffeeScript,makesuretopre-compiletherawsourcefilestoJavaScriptandusethecompiledJavaScriptfilesin
theservice.
Therequestmodule
The @arangodb/requestmodulewhenusedwiththe jsonoptionpreviouslyoverwrotethestringinthe bodypropertyofthe
responseobjectoftheresponsewiththeparsedJSONbody.In2.8thiswaschangedsotheparsedJSONbodyisaddedasthe json
propertyoftheresponseobjectinadditiontooverwritingthe bodyproperty.In3.0andlater(includinglegacycompatibilitymode)the
bodypropertyisnolongeroverwrittenandmustusethe jsonpropertyinstead.Notethatthisonlyaffectscodeusingthe json
optionwhenmakingtherequest.
215
BundledNPMmodules
ThebundledNPM moduleshavebeenupgradedandmayincludebackwards-incompatiblechanges,especiallytheAPIof joihas
changedseveraltimes.Ifindoubtyoushouldbundleyourownversionsofthesemodulestoensurespecificversionswillbeused.
Theutilitymodule lodashisnowavailableandshouldbeusedinsteadof underscore,butbothmoduleswillcontinuetobeprovided.
216
Manifest
M anyofthefieldsthatwererequiredinArangoDB2.xarenowoptionalandcanbesafelyomitted.
ToavoidcompatibilityproblemswithfutureversionsofArangoDByoushouldalwaysspecifythe enginesfield,e.g.:
{
"engines":{
"arangodb":"^3.0.0"
}
}
Controllers&exports
PreviouslyFoxxdistinguishedbetween exportsand controllers,eachofwhichcouldbespecifiedasanobject.InArangoDB3.0
thesehavebeenmergedintoasingle mainfieldspecifyinganentryfile.
Theeasiestwaytomigrateservicesusingmultipleexportsand/orcontrollersistocreateaseparateentryfilethatimportsthesefiles:
Old(manifest.json):
{
"exports":{
"doodads":"doodads.js",
"dingbats":"dingbats.js"
},
"controllers":{
"/doodads":"routes/doodads.js",
"/dingbats":"routes/dingbats.js",
"/":"routes/root.js"
}
}
New(manifest.json):
{
"main":"index.js"
}
New(index.js):
'usestrict';
module.context.use('/doodads',require('./routes/doodads'));
module.context.use('/dingbats',require('./routes/dingbats'));
module.context.use('/',require('./routes/root'));
module.exports={
doodads:require('./doodads'),
dingbats:require('./dingbats')
};
Indexredirect
Ifyoupreviouslydidnotdefinethe defaultDocumentfield,pleasenotethatinArangoDB3.0thefieldwillnolongerdefaulttothevalue
index.htmlwhenomitted:
Old:
{
//nodefaultDocument
}
manifest.json
217
New:
{
"defaultDocument":"index.html"
}
Thisalsomeansitisnolongernecessarytospecifythe defaultDocumentfieldwithanemptyvaluetopreventtheredirectandbeableto
serverequestsatthe /(root)pathofthemountpoint:
Old:
{
"defaultDocument":""
}
New:
{
//nodefaultDocument
}
Assets
The assetsfieldisnolongersupportedinArangoDB3.0outsideoflegacycompatibilitymode.
Ifyoupreviouslyusedthefieldtoserveindividualfilesas-isyoucansimplyusethe filesfieldinstead:
Old:
{
"assets":{
"client.js":{
"files":["assets/client.js"],
"contentType":"application/javascript"
}
}
}
New:
{
"files":{
"client.js":{
"path":"assets/client.js",
"type":"application/javascript"
}
}
}
Ifyoureliedonbeingabletospecifymultiplefilesthatshouldbeconcatenated,youwillhavetousebuildtoolsoutsideofArangoDBto
preparethesefilesaccordingly.
Rootelement
The rootElementfieldisnolongersupportedandhasbeenremovedentirely.
IfyourcontrollersreliedonthisfieldbeingavailableyouneedtoadjustyourschemasandroutestobeabletohandlethefullJSON
structureofincomingdocuments.
Systemservices
manifest.json
218
The isSystemfieldisnolongersupported.ThepresenceorabsenceofthefieldhadnoeffectinmostrecentversionsofArangoDB2.x
andhasnowbeenremovedentirely.
manifest.json
219
Theapplicationcontext
Theglobal applicationContextvariableavailableinFoxxmoduleshasbeenreplacedwiththe contextattributeofthe module
variable.Forconsistencyitisnowreferredtoastheservicecontextthroughoutthisdocumentation.
SomemethodsoftheservicecontexthavechangedinArangoDB3.0:
fileName()nowbehaveslike path()didinArangoDB2.x
path()hasbeenremoved(use fileName()instead)
foxxFileName()hasbeenremoved(use fileName()instead)
Additionallythe versionand nameattributeshavebeenremovedandcannowonlybeaccessedviathe manifestattribute(as
manifest.versionand manifest.name).Notethatthecorrespondingmanifestfieldsarenowoptionalandmaybeomitted.
The optionsattributehasalsobeenremovedasitshouldbeconsideredanimplementationdetail.Youshouldinsteadaccessthe
dependenciesand configurationattributesdirectly.
Theinternal _prefixattribute(whichwasanaliasfor basePath)andtheinternal commentand clearCommentsmethods(whichwere
usedbythemagicaldocumentationcommentsinArangoDB2.x)havealsobeenremoved.
Theinternal _serviceattribute(whichprovidesaccesstotheserviceitself)hasbeenrenamedto service.
applicationContext
220
Repositoriesandmodels
PreviouslyFoxxwasheavilybuiltaroundtheconceptofrepositoriesandmodels,whichprovidedcomplexbutrarelynecessary
abstractionsontopofArangoDBcollectionsanddocuments.InArangoDB3.0thesehavebeenremovedentirely.
Repositoriesvscollections
RepositoriesmostlywrappedmethodsthatalreadyexistedonArangoDBcollectionobjectsandprimarilydealtwithconvertingbetween
plainArangoDBdocumentsandFoxxmodelinstances.InArangoDB3.0youcansimplyusethesecollectionsdirectlyandtreat
documentsasplainJavaScriptobjects.
Old:
'usestrict';
constFoxx=require('org/arangodb/foxx');
constmyRepo=newFoxx.Repository(
applicationContext.collection('myCollection'),
{model:Foxx.Model}
);
//...
constmodels=myRepo.byExample({color:'green'});
res.json(models.map(function(model){
returnmodel.forClient();
}));
New:
'usestrict';
constmyDocs=module.context.collection('myCollection');
//...
constdocs=myDocs.byExample({color:'green'});
res.json(docs);
Schemavalidation
ThemainpurposeofmodelsinArangoDB2.xwastovalidateincomingdatausingjoischemas.InmorerecentversionsofArangoDB2.x
itwasalreadypossibletopasstheseschemasdirectlyinmostplaceswhereamodelwasexpectedasanargument.Theonlydifferenceis
thatschemasshouldnowbeconsideredthedefault.
IfyoupreviouslyreliedontheautomaticvalidationofFoxxmodelinstanceswhensettingattributesorinstantiatingmodelsfrom
untrusteddata,youcansimplyusetheschema's validatemethoddirectly.
Old:
221
'usestrict';
constmySchema={
name:joi.string().required(),
size:joi.number().required()
};
constMyModel=Foxx.Model.extend({schema:mySchema});
//...
constmodel=newMyModel(req.json());
if(!model.isValid){
res.status(400);
res.write('Badrequest');
return;
}
New:
'usestrict';
//Notethisisnowwrappedinajoi.object()
constmySchema=joi.object({
}).required();
//...
constresult=mySchema.validate(req.body);
if(result.errors){
res.status(400);
res.write('Badrequest');
return;
}
Migratingmodels
Whilemostusecasesformodelscannowbereplacedwithplainjoischemas,thereisstilltheconceptofa"model"inFoxxinArangoDB
3.0althoughitisquitedifferentfromFoxxmodelsinArangoDB2.x.
AmodelinFoxxnowreferstoaplainJavaScriptobjectwithanoptional schemaattributeandtheoptionalmethods forClientand
fromClient.M odelscanbeusedinsteadofplainjoischemastodefinerequestandresponsebodiesbuttherearenomodel"instances"in
ArangoDB3.0.
Old:
'usestrict';
const_=require('underscore');
constMyModel=Foxx.Model.extend({
schema:{
},
forClient(){
return_.omit(this.attributes,['_key','_id','_rev']);
}
});
//...
ctrl.get(/*...*/)
.bodyParam('body',{type:MyModel});
New:
222
'usestrict';
const_=require('lodash');
constMyModel={
schema:joi.object({
}).required(),
forClient(data){
return_.omit(data,['_key','_id','_rev']);
}
};
//...
router.get(/*...*/)
.body(MyModel);
Triggers
Whensaving,updating,replacingordeletingmodelsinArangoDB2.xusingtherepositorymethodstherepositoryandmodelwouldfire
eventsthatcouldbesubscribedtoinordertoperformside-effects.
Notethatevenin2.xtheseeventswouldnotfirewhenusingqueriesormanipulatingdocumentsinanyotherwaythanusingthespecific
repositorymethodsthatoperatedonindividualdocuments.
ThisbehaviourisnolongeravailableinArangoDB3.0butcanbeemulatedbyusingan EventEmitterdirectlyifitisnotpossibleto
solvetheproblemdifferently:
Old:
'usestrict';
constMyModel=Foxx.Model.extend({
//...
},{
afterRemove(){
console.log(this.get('name'),'wasremoved');
}
});
//...
constmodel=myRepo.firstExample({name:'myName'});
myRepo.remove(model);
//->"myNamewasremovedsuccessfully"
New:
'usestrict';
constEventEmitter=require('events');
constemitter=newEventEmitter();
emitter.on('afterRemove',function(doc){
console.log(doc.name,'wasremoved');
});
//...
constdoc=myDocs.firstExample({name:'myName'});
myDocs.remove(doc);
emitter.emit('afterRemove',doc);
Orsimply:
223
'usestrict';
functionafterRemove(doc){
console.log(doc.name,'wasremoved');
}
//...
constdoc=myDocs.firstExample({name:'myName'});
myDocs.remove(doc);
afterRemove(doc);
224
Controllersvsrouters
FoxxControllershavebeenreplacedwithrouters.Thisismorethanacosmeticchangeastherearesignificantdifferencesinbehaviour:
Controllerswereautomaticallymountedwhenthefiledefiningthemwasexecuted.Routersneedtobeexplicitlymountedusingthe
module.context.usemethod.Routerscanalsobeexported,importedandevennested.Thismakesiteasiertosplitupcomplexrouting
treesacrossmultiplefiles.
Old:
'usestrict';
constctrl=newFoxx.Controller(applicationContext);
ctrl.get('/hello',function(req,res){
//...
});
New:
'usestrict';
constcreateRouter=require('org/arangodb/foxx/router');
//Ifyouareimportingthisfilefromyourentryfile("main"):
module.exports=router;
//Otherwise:module.context.use(router);
router.get('/hello',function(req,res){
//...
});
Somegeneralchangesinbehaviourthatmighttripyouup:
WhenspecifyingpathparameterswithschemasFoxxwillnowignoretherouteiftheschemadoesnotmatch(i.e. /hello/foxxwill
nolongermatch /hello/:numif numspecifiesaschemathatdoesn'tmatchthevalue "foxx").Withcontrollersthiscould
previouslyresultinusersseeinga400(badrequest)errorwhentheyshouldinsteadbeserveda404(notfound)response.
WhenarequestismadewithanHTTPverbnotsupportedbyanendpoint,Foxxwillnowrespondwitha405(methodnotallowed)
errorwithanappropriate AllowedheaderlistingthesupportedHTTPverbsforthatendpoint.
FoxxwillnolongerparseyourJSDoccommentstogenerateroutedocumentation(usethe summaryand descriptionmethodsof
theendpointinstead).
The apiDocumentationmethodnowlivesontheservicecontextandbehavesslightlydifferently.
Thereisnorouterequivalentforthe activateAuthenticationand activateSessionsmethods.Insteadyoushouldusethesession
middleware(seethesectiononsessionsbelow).
Thereisno delaliasforthe deletemethodonrouters.IthasalwaysbeensafetousekeywordsasmethodnamesinFoxx,so
theuseofthisaliaswasalreadydiscouragedbefore.
The allRoutesproxyisnolotavailableonroutersbutcaneasilybereplacedwithmiddlewareorchildrouters.
Controllers
225
Therequestcontext
Whendefiningarouteonacontrollerthecontrollerwouldreturnanobjectcalledrequestcontext.Routersreturnasimilarobjectcalled
endpoint.Routersalsoreturnendpointswhenmountingchildroutersormiddleware,asdoesthe usemethodoftheservicecontext.
ThemaindifferencesbetweenthenewendpointsandtheobjectsreturnedbycontrollersinpreviousversionsofArangoDBare:
bodyParamisnowsimplycalled body;itisnolongerneccessaryorpossibletogivethebodyanameandtherequestbodywill
notshowupintherequestparameters.It'salsopossibletospecifyaM IM Etype
body, queryParamand pathParamnowtakepositionargumentsinsteadofanobject.Forspecificsseetheendpoint
documentation.
notesisnowcalled descriptionandtakesasinglestringargument.
onlyIfand onlyIfAuthenticatedarenolongeravailable;theycanbeemulatedwithmiddlewareifnecessary:
Old:
ctrl.get(/*...*/)
.onlyIf(function(req){
if(!req.user){
thrownewError('Notauthenticated!');
}
});
New:
router.use(function(req,res,next){
if(!req.arangoUser){
res.throw(403,'Notauthenticated!');
}
next();
});
router.get(/*...*/);
Controllers
226
Errorhandling
The errorResponsemethodprovidedbycontrollerrequestcontextshasnoequivalentinrouterendpoints.Ifyouwanttohandle
specificerrortypeswithspecificstatuscodesyouneedtocatchthemexplicitly,eitherintherouteorinamiddleware:
Old:
ctrl.get('/puppies',function(req,res){
//Exceptionisthrownhere
})
.errorResponse(TooManyPuppiesError,400,'Somethingwentwrong!');
New:
ctrl.get('/puppies',function(req,res){
try{
//Exceptionisthrownhere
}catch(e){
if(!(einstanceofTooManyPuppiesError)){
throwe;
}
res.throw(400,'Somethingwentwrong!');
}
})
//The"error"methodmerelydocumentsthemeaning
//ofthestatuscodeandhasnoothereffect.
.error(400,'Throwniftherearetoomanypuppies.');
Notethaterrorscreatedwith http-errorsarestillhandledbyFoxxintelligently.Infact res.throwisjustahelpermethodforcreating
andthrowingtheseerrors.
Controllers
227
Before,afterandaround
The before, afterand aroundmethodscaneasilybereplacedbymiddleware:
Old:
letstart;
ctrl.before(function(req,res){
start=Date.now();
});
ctrl.after(function(req,res){
console.log('Requesthandledin',(Date.now()-start),'ms');
});
New:
letstart=Date.now();
next();
console.log('Requesthandledin',(Date.now()-start),'ms');
});
Notethatunlike aroundmiddlewarereceivesthe nextfunctionasthethirdargument(the"opts"argumenthasnoequivalent).
Controllers
228
Requestobjects
Thenamesofsomeattributesoftherequestobjecthavebeenadjustedtomorecloselyalignwiththoseofthecorrespondingmethodson
theendpointobjectsandestablishedconventionsinotherJavaScriptframeworks:
req.urlParametersisnowcalled req.pathParams
req.parametersisnowcalled req.queryParams
req.params()isnowcalled req.param()
req.requestTypeisnowcalled req.method
req.compatibilityisnowcalled req.arangoVersion
req.userisnowcalled req.arangoUser
Someattributeshavebeenremovedorchanged:
req.cookieshasbeenremovedentirely(use req.cookie(name))
req.requestBodyhasbeenremovedentirely(seebelow)
req.suffixisnowastringratherthananarray
Additionallythe req.serverand req.clientattributesarenolongeravailable.Theinformationisnowexposedinawaythatcan
(optionally)transparentlyhandleproxyforwardingheaders:
req.hostnamedefaultsto req.server.address
req.portdefaultsto req.server.port
req.remoteAddressdefaultsto client.address
req.remotePortdefaultsto client.port
Finally,the req.cookiemethodnowtakesthe signedoptionsdirectly.
Old:
constsid=req.cookie('sid',{
signed:{
secret:'keyboardcat',
algorithm:'sha256'
}
});
New:
constsid=req.cookie('sid',{
algorithm:'sha256'
});
Requestbodies
The req.bodyisnolongeramethodandnolongerautomaticallyparsesJSONrequestbodiesunlessarequestbodywasdefined.The
req.rawBodynowcorrespondstothe req.rawBodyBufferofArangoDB2.xandisalsonolongeramethod.
Old:
Controllers
229
ctrl.post('/',function(req,res){
constdata=req.body();
//...
});
New:
router.post('/',function(req,res){
constdata=req.body;
//...
})
.body(['json']);
Orsimply:
constdata=req.body;
//...
})
.body(joi.object().optional());
Multipartrequests
The req.requestPartsmethodhasbeenremovedentirely.Ifyouneedtoacceptmultipartrequestbodies,youcansimplydefinethe
requestbodyusingamultipartM IM Etypelike multipart/form-data:
Old:
ctrl.post('/',function(req,res){
constparts=req.requestParts();
//...
});
New:
constparts=req.body;
//...
})
.body(['multipart/form-data']);
Controllers
230
Responseobjects
TheresponseobjecthasalotofnewmethodsinArangoDB3.0butotherwiseremainssimilartotheresponseobjectofprevious
versions:
The res.sendmethodbehavesverydifferentlyfromhowthemethodwiththesamenamebehavedinArangoDB2.x:theconversion
nowtakestheresponsebodydefinitionoftherouteintoaccount.Thereisanewmethod res.writethatimplementstheoldbehaviour.
Notethatconsecutivecallsto res.writewillappendtotheresponsebodyratherthanreplacingitlike res.send.
The res.contentTypepropertyisalsonolongeravailable.IfyouwanttosettheM IM Etypeoftheresponsebodytoanexplicitvalue
youshouldsetthe content-typeheaderinstead:
Old:
res.contentType='application/json';
res.body=JSON.stringify(results);
New:
res.set('content-type','application/json');
res.body=JSON.stringify(results);
Orsimply:
//setsthecontenttypetoJSON
//ifithasnotalreadybeenset
res.json(results);
The res.cookiemethodnowtakesthe signedoptionsaspartoftheregularoptionsobject.
Old:
res.cookie('sid','abcdef',{
ttl:60*60,
signed:{
algorithm:'sha256'
}
});
New:
res.cookie('sid','abcdef',{
ttl:60*60,
algorithm:'sha256'
});
Controllers
231
Dependencyinjection
Thereisnoequivalentofthe addInjectormethodavailableinArangoDB2.xcontrollers.M ostusecasescanbesolvedbysimplyusing
plainvariablesbutifyouneedsomethingmoreflexibleyoucanalsousemiddleware:
Old:
ctrl.addInjector('magicNumber',function(){
returnMath.random();
});
ctrl.get('/',function(req,res,injected){
res.json(injected.magicNumber);
});
New:
functionmagicMiddleware(name){
return{
register(){
letmagic;
returnfunction(req,res,next){
if(!magic){
magic=Math.random();
}
req[name]=magic;
next();
};
}
};
}
router.use(magicMiddleware('magicNumber'));
res.json(req.magicNumber);
});
Orsimply:
constmagicNumber=Math.random();
res.json(magicNumber);
});
Controllers
232
Sessions
The ctrl.activateSessionsmethodandtherelated util-sessions-localFoxxservicehavebeenreplacedwiththeFoxxsessions
middleware.Itisnolongerpossibletousethebuilt-insessionstoragebutyoucansimplypassinanydocumentcollectiondirectly.
Old:
constlocalSessions=applicationContext.dependencies.localSessions;
constsessionStorage=localSessions.sessionStorage;
ctrl.activateSessions({
sessionStorage:sessionStorage,
cookie:{secret:'keyboardcat'}
});
ctrl.destroySession('/logout',function(req,res){
res.json({message:'Goodbye!'});
});
New:
constsessionMiddleware=require('@arangodb/foxx/sessions');
constcookieTransport=require('@arangodb/foxx/sessions/transports/cookie');
router.use(sessionMiddleware({
transport:cookieTransport('keyboardcat')
}));
router.post('/logout',function(req,res){
req.sessionStorage.clear(req.session);
res.json({message:'Goodbye!'});
});
Sessions
233
AuthandOAuth2
The util-simple-authand util-oauth2FoxxserviceshavebeenreplacedwiththeFoxxauthandFoxxOAuth2modules.Itisno
longernecessarytoinstalltheseservicesasdependenciesinordertousethefunctionality.
Old:
'usestrict';
constauth=applicationContext.dependencies.simpleAuth;
//...
constvalid=auth.verifyPassword(authData,password);
New:
'usestrict';
constcreateAuth=require('@arangodb/foxx/auth');
constauth=createAuth();//Usedefaultconfiguration
//...
constvalid=auth.verifyPassword(authData,password);
AuthandOAuth2
234
Foxxqueries
The createQuerymethodhasbeenremoved.ItcanbetriviallyreplacedwithplainJavaScriptfunctionsanddirectcallstothe
db._querymethod:
Old:
'usestrict';
constquery=Foxx.createQuery({
query:'FORuIN_usersSORTu.userASCRETURNu[@propName]',
params:['propName'],
transform:function(results,uppercase){
return(
uppercase
?results[0].toUpperCase()
:results[0].toLowerCase()
);
}
});
query('user',true);
New:
'usestrict';
functionquery(propName,uppercase){
constresults=db._query(aql`
FORuIN_users
SORTu.userASC
RETURNu[${propName}]
`);
return(
uppercase
?results[0].toUpperCase()
:results[0].toLowerCase()
);
}
query('user',true);
FoxxQueries
235
Legacycompatibilitymodefor2.8services
ArangoDB3continuestosupportFoxxserviceswrittenforArangoDB2.8byrunningtheminaspeciallegacycompatibilitymodethat
providesaccesstosomeofthemodulesandAPIsnolongerprovidedin3.0andbeyond.
Note:Legacycompatibilitymodeisstrictlyintendedasatemporarystopgapsolutionforsupportingexistingserviceswhileupgrading
toArangoDB3.0andshouldnotbeconsideredapermanentfeatureofArangoDBorFoxx.
Inordertomarkanexistingserviceasalegacyservice,justmakesurethefollowingattributeisdefinedintheservicemanifest:
"engines":{
"arangodb":"^2.8.0"
}
ThissemanticversionrangedenotesthattheserviceisknowntoworkwithArangoDB2.8.0andsupportsallnewerversionsof
ArangoDBuptobutnotincluding3.0.0(noranydevelopmentversionof3.0.0andgreater).
Anysimilarversionrangethedoesnotinclude3.0.0orgreaterwillhavethesameeffect(e.g. ^2.5.0willalsotriggerthelegacy
compatibilitymode,aswill 1.2.3,but >=2.8.0willnotasitindicatescompatibilitywithallversionsgreaterorequal2.8.0,notjust
thosewithinthe2.xversionrange).
Featuressupportedinlegacycompatibilitymode
Legacycompatibilitymodesupportstheoldmanifestformat,specifically:
mainisignored
controllerswillbemountedasin2.8
exportswillbeexecutedasin2.8
Additionallythe isSystemattributewillbeignoredifpresentbutdoesnotresultinawarninginlegacycompatibilitymode.
TheFoxxconsoleisavailableasthe consolepseudo-globalvariable(shadowingtheglobalconsoleobject).
Theservicecontextisavailableasthe applicationContextpseudo-globalvariableinthe controllers, exports, scriptsand tests
asin2.8.Thefollowingadditionalpropertiesareavailableontheservicecontextinlegacycompatibilitymode:
path()isanaliasfor3.x fileName()(using path.jointobuildfilepaths)
fileName()behavesasin2.x(using fs.safeJointobuildfilepaths)
foxxFileName()isanaliasfor2.x fileName
versionexposestheservicemanifest's versionattribute
nameexposestheservicemanifest's nameattribute
optionsexposestheservice'srawoptions
Thefollowingmethodsareremovedontheservicecontextinlegacycompatibilitymode:
use()--use @arangodb/foxx/controllerinstead
apiDocumentation()--use controller.apiDocumentation()instead
registerType()--notsupportedinlegacycompatibilitymode
Thefollowingmodulesthathavebeenremovedorreplacedin3.0.0areavailableinlegacycompatibilitymode:
@arangodb/foxx/authentication
@arangodb/foxx/console
@arangodb/foxx/controller
@arangodb/foxx/model
@arangodb/foxx/query
@arangodb/foxx/repository
@arangodb/foxx/schema
@arangodb/foxx/sessions
@arangodb/foxx/template_middleware
236
The @arangodb/foxxmodulealsoprovidesthesameexportsasin2.8,namely:
Controllerfrom @arangodb/foxx/controller
createQueryfrom @arangodb/foxx/query
Modelfrom @arangodb/foxx/model
Repositoryfrom @arangodb/foxx/repository
toJSONSchemafrom @arangodb/foxx/schema
getExportsand requireAppfrom @arangodb/foxx/manager
queuesfrom @arangodb/foxx/queues
Anyfeaturenotsupportedin2.8willalsonotworkinlegacycompatibilitymode.WhenmigratingfromanolderversionofArangoDBit
isagoodideatomigratetoArangoDB2.8firstforaneasierupgradepath.
AdditionallypleasenotethedifferenceslaidoutinthechapterMigratingfrompre-2.8inthemigrationguide.
237
Usermanagement
Foxxdoesnotprovideanyusermanagementoutoftheboxbutitisveryeasytorollyourownsolution:
Thesessionmiddlewareprovidesmechanismsforaddingsessionlogictoyourservice,usinge.g.acollectionorJSONWebTokensto
storethesessionsbetweenrequests.
Theauthmoduleprovidesutilitiesforbasicpasswordverificationandhashing.
Thefollowingexampleservicedemonstrateshowusermanagementcanbeimplementedusingthesebasicbuildingblocks.
Settingupthecollections
Let'ssaywewanttostoresessionsandusersincollections.Wecanusethesetupscripttomakesurethesecollectionsarecreatedbefore
theserviceismounted.
Firstaddasetupscripttoyourmanifestifitisn'talreadydefined:
"scripts":{
"setup":"scripts/setup.js"
}
Thencreatethesetupscriptwiththefollowingcontent:
'usestrict';
constsessions=module.context.collectionName('sessions');
constusers=module.context.collectionName('users');
if(!db._collection(sessions)){
db._createDocumentCollection(sessions);
}
if(!db._collection(users)){
db._createDocumentCollection(users);
}
db._collection(users).ensureIndex({
type:'hash',
fields:['username'],
unique:true
});
Creatingtherouter
Thefollowingmainfiledemonstratesbasicusermanagement:
'usestrict';
constsessionsMiddleware=require('@arangodb/foxx/sessions');
constauth=createAuth();
constusers=module.context.collection('users');
transport:'cookie'
});
router.get('/whoami',function(req,res){
Usermanagement
238
try{
constuser=users.document(req.session.uid);
res.send({username:user.username});
}catch(e){
res.send({username:null});
}
})
.description('Returnsthecurrentlyactiveusername.');
router.post('/login',function(req,res){
//Thismayreturnauserobjectornull
constuser=users.firstExample({
username:req.body.username
});
constvalid=auth.verify(
//Pretendtovalidateevenifnouserwasfound
user?user.authData:{},
req.body.password
);
if(!valid)res.throw('unauthorized');
//Logtheuserin
req.session.uid=user._key;
res.send({sucess:true});
})
.body(joi.object({
username:joi.string().required(),
password:joi.string().required()
}).required(),'Credentials')
.description('Logsaregistereduserin.');
router.post('/logout',function(req,res){
if(req.session.uid){
req.session.uid=null;
}
res.send({success:true});
})
.description('Logsthecurrentuserout.');
router.post('/signup',function(req,res){
constuser=req.body;
try{
//Createanauthenticationhash
user.authData=auth.create(user.password);
deleteuser.password;
constmeta=users.save(user);
Object.assign(user,meta);
}catch(e){
//Failedtosavetheuser
//We'llassumetheUniqueConstrainthasbeenviolated
res.throw('badrequest','Usernamealreadytaken',e);
}
//Logtheuserin
req.session.uid=user._key;
res.send({success:true});
})
.body(joi.object({
username:joi.string().required(),
password:joi.string().required()
}).required(),'Credentials')
.description('Createsanewuserandlogsthemin.');
Usermanagement
239
Relatedmodules
ThesearesomeofthemodulesoutsideofFoxxyouwillfindusefulwhenwritingFoxxservices.
AdditionallytherearemodulesprovidingsomelevelofcompatibilitywithNode.jsaswellasanumberofbundledNPM modules(like
lodashandjoi).FormoreinformationonthesemodulesseetheJavaScriptmodulesappendix.
The @arangodbmodule
ThismoduleprovidesaccesstovariousArangoDBinternalsaswellasthreeofthemostimportantexportsnecessarytoworkwiththe
databaseinFoxx:
The dbobject
require('@arangodb').db
The dbobjectrepresentsthecurrentdatabaseandletsyouaccesscollectionsandrunqueries.Formoreinformationseethedbobject
reference.
Examples
constthirteen=db._query('RETURN5+8')[0];
The aqltemplatestringhandler
require('@arangodb').aql
The aqlfunctionisaJavaScripttemplatestringhandler.ItcanbeusedtowritecomplexAQLqueriesasmulti-linestringswithout
havingtoworryaboutbindVarsandthedistinctionbetweencollectionsandregularparameters.
TouseitjustprefixaJavaScripttemplatestring(theoneswithbackticksinsteadofquotes)withitsimportname(e.g. aql)andpassin
variableslikeyouwouldwitharegulartemplatestring.Thestringwillautomaticallybeconvertedintoanobjectwith queryand
bindVarsattributeswhichyoucanpassdirectlyto db._querytoexecute.Ifyoupassinacollectionitwillbeautomaticallyrecognized
asacollectionreferenceandhandledaccordingly.
TofindoutmoreaboutAQLseetheAQLdocumentation.
Examples
constfilterValue=23;
constmydata=db._collection('mydata');
constresult=db._query(aql`
FORdIN${mydata}
FILTERd.num>${filterValue}
RETURNd
`);
The errorsobject
require('@arangodb').errors
ThisobjectprovidesusefulobjectsforeacherrorcodeArangoDBmightusein ArangoErrorerrors.Thisishelpfulwhentryingtocatch
specificerrorsraisedbyArangoDB,e.g.whentryingtoaccessadocumentthatdoesnotexist.Eachobjecthasa codeproperty
correspondingtothe errorNumfoundon ArangoErrorerrors.
Foracompletelistoftheerrornamesandcodesyoumayencounterseetheappendixonerrorcodes.
Relatedmodules
240
Examples
consterrors=require('@arangodb').errors;
try{
mydata.document('does-not-exist');
}catch(e){
if(e.isArangoError&&e.errorNum===errors.ERROR_ARANGO_DOCUMENT_NOT_FOUND.code){
res.throw(404,'Documentdoesnotexist');
}
res.throw(500,'Somethingwentwrong',e);
}
The @arangodb/requestmodule
require('@arangodb/request')
ThismoduleprovidesafunctionformakingHTTPrequeststoexternalservices.NotethatwhilethisallowscommunicatingwiththirdpartyservicesitmayaffectdatabaseperformancebyblockingFoxxrequestsasArangoDBwaitsfortheremoteservicetorespond.If
youroutinelymakerequeststoslowexternalservicesandarenotdirectlyinterestedintheresponseitisprobablyabetterideato
delegatetheactualrequest/responsecycletoagatewayservicerunningoutsideArangoDB.
Youcanfindafulldescriptionofthismoduleintherequestmoduleappendix.
The @arangodb/general-graphmodule
require('@arangodb/general-graph')
ThismoduleprovidesaccesstoArangoDBgraphdefinitionsandvariouslow-levelgraphoperationsinJavaScript.Formorecomplex
queriesitisprobablybettertouseAQLbutthismodulecanbeusefulinyoursetupandteardownscriptstocreateanddestroygraph
definitions.
Formoreinformationseethechapteronthegeneralgraphmodule.
Relatedmodules
241
Authentication
Authenticatorsallowimplementingbasicpasswordmechanismusingsimplebuilt-inhashingfunctions.
ForafullexampleofsessionswithauthenticationandregistrationseetheexampleinthechapteronUserM anagement.
Creatinganauthenticator
auth([options]):Authenticator
Createsanauthenticator.
Arguments
method: string(Default: "sha256")
Thehashingalgorithmtousetocreatepasswordhashes.Theauthenticatorwillbeabletoverifypasswordsagainsthashes
usinganysupportedhashingalgorithm.Thisonlyaffectsnewhashescreatedbytheauthenticator.
Supportedvalues:
"md5"
"sha1"
"sha224"
"sha256"
"sha384"
"sha512"
saltLength: number(Default: 16)
Lengthofthesaltsthatwillbegeneratedforpasswordhashes.
Returnsanauthenticator.
Creatingauthenticationdataobjects
auth.create(password):AuthData
Createsanauthenticationdataobjectforthegivenpasswordwiththefollowingproperties:
method: string
Themethodusedtogeneratethehash.
salt: string
Arandomsaltusedtogeneratethishash.
hash: string
Thehashstringitself.
Arguments
password: string
Apasswordtohash.
Returnstheauthenticationdataobject.
Authentication
242
Validatingpasswordsagainstauthenticationdataobjects
auth.verify([hash,[password]]):boolean
Verifiesthegivenpasswordagainstthegivenhashusingaconstanttimestringcomparison.
Arguments
hash: AuthData(optional)
Aauthenticationdataobjectgeneratedwiththecreatemethod.
password: string(optional)
Apasswordtoverifyagainstthehash.
Returns trueifthehashmatchesthegivenpassword.Returns falseotherwise.
Authentication
243
Transactions
Startingwithversion1.3,ArangoDBprovidessupportforuser-definabletransactions.
TransactionsinArangoDBareatomic,consistent,isolated,anddurable(ACID).
TheseACIDpropertiesprovidethefollowingguarantees:
Theatomicityprinciplemakestransactionseithercompleteintheirentiretyorhavenoeffectatall.
Theconsistencyprincipleensuresthatnoconstraintsorotherinvariantswillbeviolatedduringorafteranytransaction.
Theisolationpropertywillhidethemodificationsofatransactionfromothertransactionsuntilthetransactioncommits.
Finally,thedurabilitypropositionmakessurethatoperationsfromtransactionsthathavecommittedwillbemadepersistent.The
amountoftransactiondurabilityisconfigurableinArangoDB,asisthedurabilityoncollectionlevel.
Transactions
244
ArangoDBtransactionsaredifferentfromtransactionsinSQL.
InSQL,transactionsarestartedwithexplicitBEGINorSTARTTRANSACTIONcommand.Followinganyseriesofdataretrievalor
modificationoperations,anSQLtransactionisfinishedwithaCOMMITcommand,orrolledbackwithaROLLBACKcommand.There
maybeclient/servercommunicationbetweenthestartandthecommit/rollbackofanSQLtransaction.
InArangoDB,atransactionisalwaysaserver-sideoperation,andisexecutedontheserverinonego,withoutanyclientinteraction.All
operationstobeexecutedinsideatransactionneedtobeknownbytheserverwhenthetransactionisstarted.
TherearenoindividualBEGIN,COMMITorROLLBACKtransactioncommandsinArangoDB.Instead,atransactioninArangoDBis
startedbyprovidingadescriptionofthetransactiontothedb._executeTransactionJavaScriptfunction:
db._executeTransaction(description);
Thisfunctionwillthenautomaticallystartatransaction,executeallrequireddataretrievaland/ormodificationoperations,andattheend
automaticallycommitthetransaction.Ifanerroroccursduringtransactionexecution,thetransactionisautomaticallyaborted,andall
changesarerolledback.
Executetransaction
executesatransaction db._executeTransaction(object)
Executesaserver-sidetransaction,asspecifiedbyobject.
objectmusthavethefollowingattributes:
collections:asub-objectthatdefineswhichcollectionswillbeusedinthetransaction.collectionscanhavetheseattributes:
read:asinglecollectionoralistofcollectionsthatwillbeusedinthetransactioninread-onlymode
write:asinglecollectionoralistofcollectionsthatwillbeusedinthetransactioninwriteorreadmode.
action:aJavascriptfunctionorastringwithJavascriptcodecontainingalltheinstructionstobeexecutedinsidethetransaction.If
thecoderunsthroughsuccessfully,thetransactionwillbecommittedattheend.Ifthecodethrowsanexception,thetransactionwill
berolledbackandalldatabaseoperationswillberolledback.
Additionally,objectcanhavethefollowingoptionalattributes:
waitForSync:booleanflagindicatingwhetherthetransactionisforcedtobesynchronous.
lockTimeout:anumericvaluethatcanbeusedtosetatimeoutforwaitingoncollectionlocks.Ifnotspecified,adefaultvaluewillbe
used.SettinglockTimeoutto0willmakeArangoDBnottimeoutwaitingforalock.
params:optionalargumentspassedtothefunctionspecifiedinaction.
Declarationofcollections
Allcollectionswhicharetoparticipateinatransactionneedtobedeclaredbeforehand.Thisisanecessitytoensureproperlockingand
isolation.
Collectionscanbeusedinatransactioninwritemodeorinread-onlymode.
Ifanydatamodificationoperationsaretobeexecuted,thecollectionmustbedeclaredforuseinwritemode.Thewritemodeallows
modifyingandreadingdatafromthecollectionduringthetransaction(i.e.thewritemodeincludesthereadmode).
Contrary,usingacollectioninread-onlymodewillonlyallowperformingreadoperationsonacollection.Anyattempttowriteintoa
collectionusedinread-onlymodewillmakethetransactionfail.
Collectionsforatransactionaredeclaredbyprovidingtheminthecollectionsattributeoftheobjectpassedtothe_executeTransaction
function.Thecollectionsattributehasthesub-attributesreadandwrite:
245
db._executeTransaction({
collections:{
write:["users","logins"],
read:["recommendations"]
}
});
readandwriteareoptionalattributes,andonlyneedtobespecifiediftheoperationsinsidethetransactionsdemandforit.
Thecontentsofreadorwritecaneachbelistsarrayscollectionnamesorasinglecollectionname(asastring):
collections:{
write:"users",
read:"recommendations"
}
});
Note:Itiscurrentlyoptionaltospecifycollectionsforread-onlyaccess.Evenwithoutspecifyingthem,itisstillpossibletoreadfrom
suchcollectionsfromwithinatransaction,butwithrelaxedisolation.PleaserefertoTransactionsLockingformoredetails.
Inordertomakeatransactionfailwhenanon-declaredcollectionisusedinsideforreading,theoptionalallowImplicitsub-attributeof
collectionscanbesettofalse:
collections:{
read:"recommendations",
allowImplicit:false/*thisdisallowsreadaccesstoothercollections
thanspecified*/
},
action:function(){
vardb=require("@arangodb").db;
returndb.foobar.toArray();/*willfailbecausedb.foobarmustnotbeaccessed
forreadinginsidethistransaction*/
}
});
ThedefaultvalueforallowImplicitistrue.Write-accessingcollectionsthathavenotbeendeclaredinthecollectionsarrayisnever
possible,regardlessofthevalueofallowImplicit.
Declarationofdatamodificationandretrievaloperations
AlldatamodificationandretrievaloperationsthataretobeexecutedinsidethetransactionneedtobespecifiedinaJavascriptfunction,
usingtheactionattribute:
collections:{
write:"users"
},
action:function(){
//alloperationsgohere
}
});
AnyvalidJavascriptcodeisallowedinsideactionbutthecodemayonlyaccessthecollectionsdeclaredincollections.actionmaybea
Javascriptfunctionasshownabove,orastringrepresentationofaJavascriptfunction:
collections:{
write:"users"
},
action:"function(){doSomething();}"
});
246
Pleasenotethatanyoperationsspecifiedinactionwillbeexecutedontheserver,inaseparatescope.Variableswillbeboundlate.
AccessinganyJavaScriptvariablesdefinedontheclient-sideorinsomeotherservercontextfrominsideatransactionmaynotwork.
Instead,anyvariablesusedinsideactionshouldbedefinedinsideactionitself:
collections:{
write:"users"
},
action:function(){
vardb=require(...).db;
db.users.save({...});
}
});
Whenthecodeinsidetheactionattributeisexecuted,thetransactionisalreadystartedandallrequiredlockshavebeenacquired.When
thecodeinsidetheactionattributefinishes,thetransactionwillautomaticallycommit.Thereisnoexplicitcommitcommand.
Tomakeatransactionabortandrollbackallchanges,anexceptionneedstobethrownandnotcaughtinsidethetransaction:
collections:{
write:"users"
},
action:function(){
db.users.save({_key:"hello"});
//willabortandrollbackthetransaction
throw"doh!";
}
});
Thereisnoexplicitabortorrollbackcommand.
Asmentionedearlier,atransactionwillcommitautomaticallywhentheendoftheactionfunctionisreachedandnoexceptionhasbeen
thrown.Inthiscase,theusercanreturnanylegalJavaScriptvaluefromthefunction:
collections:{
write:"users"
},
action:function(){
db.users.save({_key:"hello"});
//willcommitthetransactionandreturnthevalue"hello"
return"hello";
}
});
Examples
Thefirstexamplewillwrite3documentsintoacollectionnamedc1.Thec1collectionneedstobedeclaredinthewriteattributeofthe
collectionsattributepassedtotheexecuteTransactionfunction.
Theactionattributecontainstheactualtransactioncodetobeexecuted.Thiscodecontainsalldatamodificationoperations(3inthis
example).
247
//setup
db._create("c1");
collections:{
write:["c1"]
},
action:function(){
db.c1.save({_key:"key1"});
}
});
db.c1.count();//3
Abortingthetransactionbythrowinganexceptionintheactionfunctionwillrevertallchanges,soasifthetransactionneverhappened:
//setup
db._create("c1");
collections:{
write:["c1"]
},
action:function(){
db.c1.count();//1
db.c1.count();//2
throw"doh!";
}
});
db.c1.count();//0
Theautomaticrollbackisalsoexecutedwhenaninternalexceptionisthrownatsomepointduringtransactionexecution:
//setup
db._create("c1");
collections:{
write:["c1"]
},
action:function(){
//willthrowduplicateakeyerror,notexplicitlyrequestedbytheuser
//we'llnevergethere...
}
});
db.c1.count();//0
Asrequiredbytheconsistencyprinciple,abortingorrollingbackatransactionwillalsorestoresecondaryindexestothestateat
transactionstart.
Cross-collectiontransactions
There'salsothepossibilitytorunatransactionacrossmultiplecollections.Inthiscase,multiplecollectionsneedtobedeclaredinthe
collectionsattribute,e.g.:
248
//setup
db._create("c1");
db._create("c2");
collections:{
write:["c1","c2"]
},
action:function(){
}
});
db.c1.count();//1
db.c2.count();//1
Again,throwinganexceptionfrominsidetheactionfunctionwillmakethetransactionabortandrollbackallchangesinallcollections:
//setup
db._create("c1");
db._create("c2");
collections:{
write:["c1","c2"]
},
action:function(){
for(vari=0;i<100;++i){
db.c1.save({_key:"key"+i});
db.c2.save({_key:"key"+i});
}
db.c1.count();//100
db.c2.count();//100
//abort
throw"doh!"
}
});
db.c1.count();//0
db.c2.count();//0
249
Passingparameterstotransactions
Arbitraryparameterscanbepassedtotransactionsbysettingtheparamsattributewhendeclaringthetransaction.Thisfeatureishandy
tore-usethesametransactioncodeformultiplecallsbutwithdifferentparameters.
Abasicexample:
collections:{},
action:function(params){
returnparams[1];
},
params:[1,2,3]
});
Theaboveexamplewillreturn2.
Someexamplethatusescollections:
collections:{
write:"users",
read:["c1","c2"]
},
action:function(params){
vardb=require('@arangodb').db;
vardoc=db.c1.document(params['c1Key']);
db.users.save(doc);
doc=db.c2.document(params['c2Key']);
db.users.save(doc);
},
params:{
c1Key:"foo",
c2Key:"bar"
}
});
Passingparameters
250
LockingandIsolation
Allcollectionsspecifiedinthecollectionsattributearelockedintherequestedmode(readorwrite)attransactionstart.Lockingof
multiplecollectionsisperformedinalphabeticalorder.Whenatransactioncommitsorrollsback,alllocksarereleasedinreverseorder.
Thelockingorderisdeterministictoavoiddeadlocks.
Whilelocksareheld,modificationsbyothertransactionstothecollectionsparticipatinginthetransactionareprevented.Atransaction
willthusseeaconsistentviewoftheparticipatingcollections'data.
Additionally,atransactionwillnotbeinterruptedorinterleavedwithanyotherongoingoperationsonthesamecollection.Thismeans
eachtransactionwillruninisolation.Atransactionshouldneverseeuncommittedorrolledbackmodificationsbyothertransactions.
Additionally,readsinsideatransactionarerepeatable.
Notethattheaboveistrueonlyforallcollectionsthataredeclaredinthecollectionsattributeofthetransaction.
Lazilyaddingcollections
Theremightbesituationswhendeclaringallcollectionsaprioriisnotpossible,forexample,becausefurthercollectionsaredeterminedby
adynamicAQLqueryinsidethetransaction,forexampleaqueryusingAQLgraphfunctions.
Inthiscase,itwouldbeimpossibletoknowbeforehandwhichcollectiontolock,andthusitislegaltonotdeclarecollectionsthatwillbe
accessedinthetransactioninread-onlymode.Accessinganon-declaredcollectioninread-onlymodeduringatransactionwilladdthe
collectiontothetransactionlazily,andfetchdatafromthecollectionasusual.However,asthecollectionisaddedlazily,thereisno
isolationfromotherconcurrentoperationsortransactions.Readsfromsuchcollectionsarepotentiallynon-repeatable.
Examples:
collections:{
read:"users"
},
action:function(){
constdb=require("@arangodb").db;
/*ExecuteanAQLquerythattraversesagraphstartingata"users"vertex.
Itisyetunknownintowhichothercollectionsthequerymighttraverse*/
db._createStatement({
query:`FORvINANY"users/1234"connectionsRETURNv`
}).execute().toArray().forEach(function(d){
/*...*/
});
}
});
Thisautomaticlazyadditionofcollectionstoatransactionalsointroducesthepossibilityofdeadlocks.Deadlocksmayoccurifthereare
concurrenttransactionsthattrytoacquirelocksonthesamecollectionslazily.
Inordertomakeatransactionfailwhenanon-declaredcollectionisusedinsideatransactionforreading,theoptionalallowImplicitsubattributeofcollectionscanbesettofalse:
Lockingandisolation
251
collections:{
read:"users",
allowImplicit:false
},
action:function(){
/*Thebelowquerywillnowfailbecausethecollection"connections"hasnot
beenspecifiedinthelistofcollectionsusedbythetransaction*/
db._createStatement({
query:`FORvINANY"users/1234"connectionsRETURNv`
}).execute().toArray().forEach(function(d){
/*...*/
});
}
});
ThedefaultvalueforallowImplicitistrue.Write-accessingcollectionsthathavenotbeendeclaredinthecollectionsarrayisnever
possible,regardlessofthevalueofallowImplicit.
Ifusers/1234hasanedgeinconnections,linkingittoanotherdocumentintheuserscollection,thenthefollowingexplicitdeclarationwill
work:
collections:{
read:["users","connections"],
allowImplicit:false
},
/*...*/
Iftheedgepointstoadocumentinanothercollectionhowever,thenthequerywillfail,unlessthatothercollectionisaddedtothe
declarationaswell.
Notethatifadocumenthandleisusedasstartingpointforatraversal,e.g. FORvINANY"users/not_linked"...or FORvINANY{_id:
"users/not_linked"}...,thennoerrorisraisedinthecaseofthestartvertexnothavinganyedgestofollow,with allowImplicit:
falseandusersnotbeingdeclaredforreadaccess.AQLonlyseesastringanddoesnotconsideritareadaccess,unlessthereareedges
connectedtoit. FORvINANYDOCUMENT("users/not_linked")...willfailevenwithoutedges,asitisalwaysconsideredtobearead
accesstotheuserscollection.
DeadlocksandDeadlockdetection
Adeadlockisasituationinwhichtwoormoreconcurrentoperations(usertransactionsorAQLqueries)trytoaccessthesameresources
(collections,documents)andneedtowaitfortheotherstofinish,butnoneofthemcanmakeanyprogress.
AgoodexampleforadeadlockistwoconcurrentlyexecutingtransactionsT1andT2thattrytoaccessthesamecollectionsbutthatneed
towaitforeachother.Inthisexample,transactionT1willwritetocollection c1,butwillalsoreaddocumentsfromcollection c2
withoutannouncingit:
collections:{
write:"c1"
},
action:function(){
/*writeintoc1(announced)*/
db.c1.insert({foo:"bar"});
/*someoperationherethattakeslongtoexecute...*/
/*readfromc2(unannounced)*/
db.c2.toArray();
}
});
TransactionT2announcestowriteintocollection c2,butwillalsoreaddocumentsfromcollection c1withoutannouncingit:
Lockingandisolation
252
collections:{
write:"c2"
},
action:function(){
/*writeintoc2(announced)*/
db.c2.insert({bar:"baz"});
/*someoperationherethattakeslongtoexecute...*/
/*readfromc1(unannounced)*/
db.c1.toArray();
}
});
Intheaboveexample,adeadlockwilloccuriftransactionT1andT2havebothacquiredtheirwritelocks(T1forcollection c1andT2
forcollection c2)andarethentryingtoreadfromtheotherother(T1willreadfrom c2,T2willreadfrom c1).T1willthentryto
acquirethereadlockoncollection c2,whichispreventedbytransactionT2.T2howeverwillwaitforthereadlockoncollection c1,
whichispreventedbytransactionT1.
Incaseofsuchdeadlock,therewouldbenoprogressforanyoftheinvolvedtransactions,andnoneoftheinvolvedtransactionscould
evercomplete.Thisiscompletelyundesirable,sotheautomaticdeadlockdetectionmechanisminArangoDBwillautomaticallyabortone
ofthetransactionsinvolvedinsuchdeadlock.Abortingmeansthatallchangesdonebythetransactionwillberolledbackanderror29
( deadlockdetected)willbethrown.
Clientcode(AQLqueries,usertransactions)thataccessesmorethanonecollectionshouldbeawareofthepotentialofdeadlocksand
shouldhandletheerror29( deadlockdetected)properly,eitherbypassingtheexceptiontothecallerorretryingtheoperation.
Toavoidbothdeadlocksandnon-repeatablereads,allcollectionsusedinatransactionshouldbespecifiedinthe collectionsattribute
whenknowninadvance.Incasethisisnotpossiblebecausecollectionsareaddeddynamicallyinsidethetransaction,deadlocksmay
occurandthedeadlockdetectionmaykickinandabortthetransaction.
Lockingandisolation
253
Durability
Transactionsareexecutedinmainmemoryfirstuntilthereiseitherarollbackoracommit.Onrollback,nodatawillbewrittentodisk,
buttheoperationsfromthetransactionwillbereversedinmemory.
Oncommit,allmodificationsdoneinthetransactionwillbewrittentothecollectiondatafiles.Thesewriteswillbesynchronizedtodisk
ifanyofthemodifiedcollectionshasthewaitForSyncpropertysettotrue,orifanyindividualoperationinthetransactionwasexecuted
withthewaitForSyncattribute.Additionally,transactionsthatmodifydatainmorethanonecollectionareautomaticallysynchronizedto
disk.Thissynchronizationisdonetonotonlyensuredurability,buttoalsoensureconsistencyincaseofaservercrash.
Thatmeansifyouonlymodifydatainasinglecollection,andthatcollectionhasitswaitForSyncpropertysettofalse,thewhole
transactionwillnotbesynchronizedtodiskinstantly,butwithasmalldelay.
Thereisthusthepotentialriskoflosingdatabetweenthecommitofthetransactionandtheactual(delayed)disksynchronization.This
isthesameaswritingintocollectionsthathavethewaitForSyncpropertysettofalseoutsideofatransaction.Incaseofacrashwith
waitForSyncsettofalse,theoperationsperformedinthetransactionwilleitherbevisiblecompletelyornotatall,dependingonwhether
thedelayedsynchronizationhadkickedinornot.
ToensuredurabilityoftransactionsonacollectionthathavethewaitForSyncpropertysettofalse,youcansetthewaitForSyncattribute
oftheobjectthatispassedtoexecuteTransaction.Thiswillforceasynchronizationofthetransactiontodiskevenforcollectionsthat
havewaitForSyncsettofalse:
collections:{
write:"users"
},
waitForSync:true,
action:function(){...}
});
Analternativeistoperformanoperationwithanexplicitsyncrequestinatransaction,e.g.
db.users.save({_key:"1234"},true);
Inthiscase,thetruevaluewillmakethewholetransactionbesynchronizedtodiskatthecommit.
Inanycase,ArangoDBwillgiveusersthechoiceofwhetherornottheywantfulldurabilityforsinglecollectiontransactions.Usingthe
delayedsynchronization(i.e.waitForSyncwithavalueoffalse)willpotentiallyincreasethroughputandperformanceoftransactions,but
willintroducetheriskoflosingthelastcommittedtransactionsinthecaseofacrash.
Incontrast,transactionsthatmodifydatainmorethanonecollectionareautomaticallysynchronizedtodisk.Thiscomesatthecostof
severaldisksync.Foramulti-collectiontransaction,thecalltothe_executeTransactionfunctionwillonlyreturnafterthedataofall
modifiedcollectionshasbeensynchronizedtodiskandthetransactionhasbeenmadefullydurable.Thisnotonlyreducestheriskof
losingdataincaseofacrashbutalsoensuresconsistencyafterarestart.
Incaseofaservercrash,anymulti-collectiontransactionsthatwerenotyetcommittedorinpreparationtobecommittedwillberolled
backonserverrestart.
Formulti-collectiontransactions,therewillbeatleastonedisksyncoperationpermodifiedcollection.M ulti-collectiontransactionsthus
haveapotentiallyhighercostthansinglecollectiontransactions.ThereisnoconfigurationtoturnoffdisksynchronizationformulticollectiontransactionsinArangoDB.Thedisksyncspeedofthesystemwillthusbethemostimportantfactorfortheperformanceof
multi-collectiontransactions.
Durability
254
Limitations
InGeneral
TransactionsinArangoDBhavebeendesignedwithparticularusecasesinmind.Theywillbemainlyusefulforshortandsmalldata
retrievaland/ormodificationoperations.
Theimplementationisnotoptimizedforverylong-runningorveryvoluminousoperations,andmaynotbeusableforthesecases.
Onelimitationisthatatransactionoperationinformationmustfitintomainmemory.Thetransactioninformationconsistsofrecord
pointers,revisionnumbersandrollbackinformation.Theactualdatamodificationoperationsofatransactionarewrittentothewriteaheadloganddonotneedtofitentirelyintomainmemory.
Ongoingtransactionswillalsopreventthewrite-aheadlogsfrombeingfullygarbage-collected.Informationinthewrite-aheadlogfiles
cannotbewrittentocollectiondatafilesorbediscardedwhiletransactionsareongoing.
Toensureprogressofthewrite-aheadloggarbagecollection,transactionsshouldbekeptassmallaspossible,andbigtransactionsshould
besplitintomultiplesmallertransactions.
TransactionsinArangoDBcannotbenested,i.e.atransactionmustnotstartanothertransaction.Ifanattemptismadetocalla
transactionfrominsidearunningtransaction,theserverwillthrowerror1651(nestedtransactionsdetected).
ItisalsodisallowedtoexecuteusertransactiononsomeofArangoDB'sownsystemcollections.Thisshouldn'tbeaproblemforregular
usageassystemcollectionswillnotcontainuserdataandthereisnoneedtoaccessthemfromwithinausertransaction.
Someoperationsarenotallowedinsidetransactionsingeneral:
creationanddeletionofcollections( db._create(), db._drop(), db._rename())
creationanddeletionofindexes( db.ensure...Index(), db.dropIndex())
Ifanattemptismadetocarryoutanyoftheseoperationsduringatransaction,ArangoDBwillabortthetransactionwitherrorcode1653
(disallowedoperationinsidetransaction).
Finally,allcollectionsthatmaybemodifiedduringatransactionmustbedeclaredbeforehand,i.e.usingthecollectionsattributeofthe
objectpassedtothe_executeTransactionfunction.Ifanyattemptismadetocarryoutadatamodificationoperationonacollectionthat
wasnotdeclaredinthecollectionsattribute,thetransactionwillbeabortedandArangoDBwillthrowerror1652unregisteredcollection
usedintransaction.Itislegaltonotdeclareread-onlycollections,butthisshouldbeavoidedifpossibletoreducetheprobabilityof
deadlocksandnon-repeatablereads.
PleaserefertoLockingandIsolationformoredetails.
InClusters
UsingasingleinstanceofArangoDB,multi-document/multi-collectionqueriesareguaranteedtobefullyACID.Thisismorethanmany
otherNoSQLdatabasesystemssupport.Inclustermode,single-documentoperationsarealsofullyACID.M ulti-document/multicollectionqueriesinaclusterarenotACID,whichisequallythecasewithcompetingdatabasesystems.Transactionsinaclusterwillbe
supportedinafutureversionofArangoDBandmaketheseoperationsfullyACIDaswell.Notethatfornon-shardedcollectionsina
cluster,thetransactionalpropertiesofasingleserverapply(fullyACID).
Limitations
255
Deployment
InthischapterwedescribevariouspossibilitiestodeployArangoDB.Inparticularfortheclustermode,therearedifferentwaysandwe
wanttohighlighttheiradvantagesanddisadvantages.Weevendocumentindetail,howtosetupaclusterbysimplystartingvarious
ArangoDBprocessesondifferentmachines,eitherdirectlyorusingDockercontainers.
Singleinstance
Cluster:DC/OS,ApacheM esosandM arathon
Cluster:Testsetuponalocalmachine
Cluster:Startingprocessesondifferentmachines
Cluster:LaunchinganArangoDBclusterusingDockercontainers
Deployment
256
Singleinstancedeployment
ThelatestofficialbuildsofArangoDBforallsupportedoperatingsystemsmaybeobtainedfromhttps://www.arangodb.com/download/.
Linuxremarks
Besidestheofficialimageswhichareprovidedforthemostpopularlinuxdistributionstherearealsoavarietyofunofficialimages
providedbythecommunity.Wearetrackingmostofthecommunitycontributions(includingneworupdatedimages)inournewsletter:
https://www.arangodb.com/category/newsletter/
Windowsremarks
PleasenotethatArangoDBwillonlyworkon64bit.
Docker
ThesimplestwaytodeployArangoDBisusingDocker.TogetageneralunderstandingofDockerhavealookattheirexcellent
documentation.
Authentication
TostarttheofficialDockercontaineryouwillhavetodecideonanauthenticationmethod.Otherwisethecontainerwon'tstart.
ProvideoneoftheargumentstoDockerasanenvironmentvariable.
Therearethreeoptions:
1. ARANGO_NO_AUTH=1
Disableauthenticationcompletely.Usefulforlocaltestingorforoperatinginatrustednetwork(withoutapublicinterface).
2. ARANGO_ROOT_PASSWORD=password
StartArangoDBwiththegivenpasswordforroot
3. ARANGO_RANDOM _ROOT_PASSWORD=1
LetArangoDBgeneratearandomrootpassword
Togetgoingquickly:
dockerrun-eARANGO_RANDOM_ROOT_PASSWORD=1arangodb/arangodb
ForanindepthguideaboutDockerandArangoDBpleasechecktheofficialdocumentation:https://hub.docker.com/r/arangodb/arangodb/.
Notethatweareusingtheimage arangodb/arangodbherewhichisalwaysthemostcurrentone.Thereisalsothe"official"onecalled
arangodbwhosedocumentationishere:https://hub.docker.com/_/arangodb/
Singleinstance
257
DistributeddeploymentusingApacheMesos
ArangoDBhasasophisticatedandyeteasywaytouseclustermode.Toleveragethefullclusterfeatureset(monitoring,scaling,
automaticfailoverandautomaticreplacementoffailednodes)youhavetorunArangoDBonsomekindofclustermanagementsystem.
CurrentlyArangoDBreliesonApacheM esosinthatmatter.M esosisaclusteroperatingsystemwhichpowerssomeoftheworlds
biggestdatacentersrunningseveralthousandsofnodes.
DC/OS
DC/OSistherecommendedwaytoinstallaclusterasiteasesmuchoftheprocesstoinstallaM esoscluster.Youcandeployitvery
quicklyonavarietyofcloudhostersorsetupyourownDC/OSlocally.DC/OSisasetoftoolsbuiltontopofApacheM esos.Apache
M esosisasocalled"DistributedClusterOperationSystem"andthecoreofDC/OS.ApacheM esoshastheconceptofsocalled
persistentvolumeswhichmakeitperfectlysuitableforadatabase.
Installing
FirstprepareaDC/OSclusterbygoingtohttps://dcos.ioandfollowingtheinstructionsthere.
DC/OScomeswithitsownpackagemanagement.Packagescanbeinstalledfromthesocalled"Universe".AsanofficialDC/OSpartner
ArangoDBcanbeinstalledfromtherestraightaway.
1. InstallingviaDC/OSUI
i. OpenyourbrowserandgototheDC/OSadmininterface
ii. Openthe"Universe"tab
iii. Locatearangodbandhit"InstallPackage"
iv. Press"InstallPackage"
2. InstallingviatheDC/OScommandline
i. Installthedcoscli
ii. Openaterminalandissue dcosinstallarangodb
Bothoptionsareessentiallydoingthesameinthebackground.BotharestartingArangoDBwithitsdefaultoptionsset.
Toreviewthedefaultoptionsusingthewebinterfacesimplyclick"AdvancedInstallation"inthewebinterface.Thereyouwillfindalist
ofoptionsincludingsomeexplanation.
ToreviewthedefaultoptionsusingtheCLIfirsttype dcospackagedescribe--configarangodb.Thiswillgiveyouaflatlistofdefault
settings.
Togetanexplanationofthevariouscommandlineoptionspleasecheckthelatestoptionshere(choosethemostrecentnumberandhave
alookat config.json):
https://github.com/mesosphere/universe/tree/version-3.x/repo/packages/A/arangodb
AfterinstallationDC/OSwillstartdeployingtheArangoDBclusterontheDC/OScluster.YoucanwatchArangoDBstartingonthe
"Services"tabinthewebinterface.OnceitislistedashealthyclickthelinknexttoitandyoushouldseetheArangoDBwebinterface.
ArangoDBMesosframework
AssoonasArangoDBwasdeployedM esoswillkeepyourclusterrunning.Thewebinterfacehasmanymonitoringfacilitiessobesure
tomakeyourselffamiliarwiththeDC/OSwebinterface.AsafaulttolerantsystemM esoswilltakecareofmostfailurescenarios
automatically.M esosdoesthatbyrunningArangoDBasasocalled"framework".Thisframeworkhasbeenspecificallybuilttokeep
ArangoDBrunninginahealthyconditionontheM esoscluster.Fromtimetotimeataskmightfail.TheArangoDBframeworkwillthen
takecareofreschedulingthefailedtask.Asitknowsabouttheveryspecificsofeachclustertaskanditsroleitwillautomaticallytake
careofmostfailurescenarios.
Toinspectwhattheframeworkisdoinggoto http://web-interface-url/mesosinyourbrowser.Locatethetask"arangodb"andinspect
stderrinthe"Sandbox".Thiscanbeofinterestforexamplewhenaslavegotlostandtheframeworkisreschedulingthetask.
258
UsingArangoDB
TouseArangoDBasadatastoreinyourDC/OSclusteryoucanfacilitatetheservicediscoveryofDC/OS.Assumingyoudeployeda
standardArangoDBclusterthemesosdnswillknowabout arangodb.mesos.BydoingaSRVDNSrequest(checkthedocumentationof
mesosdns)youcanfindouttheportwheretheinternalHAProxyofArangoDBisrunning.Thiswillofferaroundrobinloadbalancerto
accessallArangoDBcoordinators.
ScalingArangoDB
TochangethesettingsofyourArangoDBClusteraccesstheArangoDBUIandhit"Nodes".Onthescaletabyouwillhavetheabilityto
scaleyourclusterupanddown.
AfterchangingthesettingstheArangoDBframeworkwilltakecareoftherest.Scalingyourclusterupisgenerallyastraightforward
operationasM esoswillsimplylaunchanothertaskandbedonewithit.Scalingdownisabitmorecomplicatedasthedatafirsthastobe
movedtosomeotherplacesothatwillnaturallytakesomewhatlonger.
Pleasenotethatscalingoperationsmightnotalwayswork.ForexampleiftheunderlyingM esosclusteriscompletelysaturatedwithits
runningtasksscalingupwillnotbepossible.Scalingdownmightalsofailduetotheclusternotbeingabletomoveallshardsofa
DBServertoanewdestinationbecauseofsizelimitations.BesuretochecktheoutputoftheArangoDBframework.
Deinstallation
DeinstallingArangoDBisabitmoredifficultasthereismuchstatebeingkeptintheM esosclusterwhichisnotautomaticallycleaned
up.Todeinstallfromthecommandlineusethefollowingoneliner:
dcosarangodbuninstall;dcospackageuninstallarangodb
Thiswillfirstcleanupthestateintheclusterandthenuninstallarangodb.
arangodb-cleanup-framework
Shouldyouforgettocleanupthestateyoucandosolaterbyusingthearangodb-cleanup-frameworkcontainer.Otherwiseyoumightnot
beabletodeployanewarangodbinstallation.
ThecleanupframeworkwillannounceitselfasanormalArangoDB.M esoswillrecognizethisandofferallpersistentvolumesitstillhas
forArangoDBtothisframework.Thecleanupframeworkwillthenproperlyfreethepersistentvolumes.Finallyitwillcleanupany
stateleftinzookeeper(thecentralconfigurationmanagerinaM esoscluster).
Todeploythecleanupframework,followtheinstructionsinthegithubrepository.Afterdeploymentwatchtheoutputinthesandboxof
theM esoswebinterface.Afterawhilethereshouldn'tbeanypersistentresourceoffersanymoreaseverythingwascleanedup.After
thatyoucandeletethecleanupframeworkagainviaM arathon.
ApacheMesosandMarathon
YoucanalsoinstallArangoDBonabareApacheM esosclusterprovidedthatM arathonisrunningonit.
Doingsohasthefollowingdownsides:
1. M anualM esosclustersetup
2. Youneedtoimplementyourownservicediscovery
3. Youaremissingthedcoscli
4. Installationanddeinstallationaretedious
5. YouneedtosetupsomekindofproxytunneltoaccessArangoDBfromtheoutside
6. Sparsemonitoringcapabilities
HoweverthesearethingswhichdonotinfluenceArangoDBitselfandoperatingyourclusterlikethisisfullysupported.
InstallingviaMarathon
ToinstallArangoDBviamarathonyouneedaproperconfigfile:
259
{
"id":"arangodb",
"cpus":0.25,
"mem":256.0,
"ports":[0,0,0],
"instances":1,
"args":[
"framework",
"--framework_name=arangodb",
"--master=zk://172.17.0.2:2181/mesos",
"--zk=zk://172.17.0.2:2181/arangodb",
"--user=",
"--principal=pri",
"--role=arangodb",
"--mode=cluster",
"--async_replication=true",
"--minimal_resources_agent=mem(*):512;cpus(*):0.25;disk(*):512",
"--minimal_resources_dbserver=mem(*):512;cpus(*):0.25;disk(*):1024",
"--minimal_resources_secondary=mem(*):512;cpus(*):0.25;disk(*):1024",
"--minimal_resources_coordinator=mem(*):512;cpus(*):0.25;disk(*):1024",
"--nr_agents=1",
"--nr_dbservers=2",
"--nr_coordinators=2",
"--failover_timeout=86400",
"--arangodb_image=arangodb/arangodb-mesos:3.0",
"--secondaries_with_dbservers=false",
"--coordinators_with_dbservers=false"
],
"container":{
"type":"DOCKER",
"docker":{
"image":"arangodb/arangodb-mesos-framework:3.0",
"network":"HOST"
}
},
"healthChecks":[
{
"protocol":"HTTP",
"path":"/framework/v1/health.json",
"gracePeriodSeconds":3,
"intervalSeconds":10,
"portIndex":0,
"timeoutSeconds":10,
"maxConsecutiveFailures":0
}
]
}
Carefullyreviewthesettings(especiallytheIPsandtheresources).ThenyoucandeploytoM arathon:
curl-XPOST-H"Content-Type:application/json"http://url-of-marathon/v2/[email protected]
AlternativelyusethewebinterfaceofM arathontodeployArangoDB.IthasaJSONmodeandyoucanusetheaboveconfigurationfile.
DeinstallationviaMarathon
AswithDC/OSyoufirstneedtoproperlycleanupanystateleftovers.
TheeasiestistosimplydeleteArangoDBandthendeploythecleanup-framework(seesectionarangodb-cleanup-framework).
Configurationoptions
TheArangodbM esosframeworkhasatonofdifferentoptionswhicharelistedanddescribedhere:
https://github.com/arangodb/arangodb-mesos-framework/tree/3.0
260
LaunchinganArangoDBclusterfortesting
AnArangoDBclusterconsistsofseveralrunningtaskswhichformthecluster.ArangoDBitselfwon'tstartormonitoranyofthesetasks.
Soitwillneedsomekindofsupervisorwhichismonitoringandstartingthesetasks.ForproductionusagewerecommendusingApache
M esosastheclustersupervisor.
HoweverstartingaclustermanuallyispossibleandisaveryeasymethodtogetafirstimpressionofwhatanArangoDBclusterlooks
like.
Theeasiestwaytostartalocalclusterfortestingpurposesistorun scripts/startLocalCluster.shfromacloneofthesource
repositoryaftercompilingArangoDBfromsource(seeinstructionsinthefile README_maintainers.mdintherepository.Thiswillstart1
Agency,2DBServersand1Coordinator.Tostoptheclusterissue scripts/stopLocalCluster.sh.
ThissectionwilldiscusstherequiredparametersforeveryroleinanArangoDBcluster.BesuretoreadtheArchitecturedocumentation
togetabasicunderstandingoftheunderlyingarchitectureandtheinvolvedrolesinanArangoDBcluster.
Inthefollowingsectionswewillgothroughtherelevantoptionsperrole.
Agency
Thebareminimumtostartanagentistoprovidetheid.Theidhastobe 0forasingleinstance.
Tostartuptheagencyinitsfaulttolerantmodesetthe --agency.sizeto 3andstarttheagentswithincreasingidsstartingfrom 0.
Furthermoreyoushouldprovidedifferent --server.endpointvaluestoeveryagent.Oneoftheagentsmustalsohave --agency.notify
truetobootstraptheagency.Thisagentmustalsogetalistofallendpointsvia --agency.endpoint.
Soinsummarythisiswhatyourstartupmightlooklike:
Cluster:Localtest
261
build/bin/arangod--server.endpointtcp://0.0.0.0:5001--server.authenticationfalse--agency.id0--agency.size3--agency.supervisiontrueagency
build/bin/arangod--server.endpointtcp://0.0.0.0:5003--server.authenticationfalse--agency.id2--agency.size3--agency.endpointtcp://127.0.0.
Noteinparticularthattheendpointdescriptionsgivenunder --agency.endpointmustnotusetheIPaddress 0.0.0.0becausethey
mustcontainanactualaddressthatcanberoutedtothecorrespondingserver.The 0.0.0.0in --server.endpointsimplymeansthat
theserverbindsitselftoallavailablenetworkdeviceswithallavailableIPaddresses.
Ifyouarehappywithasingleagent,thensimplyuseasinglecommandlikethis:
build/bin/arangod--server.endpointtcp://0.0.0.0:5001--server.authenticationfalse--agency.id0--agency.size1--agency.endpointtcp://127.0.0.
Furthermore,inthefollowingsectionswhen --cluster.agency-addressisusedmultipletimestospecifyallthreeagentaddresses,just
useasingleoption --cluster.agency.addresstcp://127.0.0.1:5001instead.
CoordinatorsandDBServers
Thesetworolesshareacommonsetofrelevantoptions.Firstyoushouldspecifytheroleusing --cluster.my-role.Thiscaneitherbe
PRIMARY(adatabaseserver)or COORDINATOR.Bothneedsomeuniqueinformationwithwhichtheywillregisterintheagency,too.This
couldforexamplebesomecombinationofhostnameandportorwhateveryouhaveathand.Howeveritmustbeuniqueforeach
instanceandbeprovidedasvaluefor --cluster.my-local-info.Furthermoreprovidetheexternalendpoint(IPandport)ofthetaskvia
--cluster.my-address.
Thefollowingisafull-exampleofwhatitmightlooklike:
build/bin/arangod--server.authentication=false--server.endpointtcp://0.0.0.0:8529--cluster.my-addresstcp://127.0.0.1:8529--cluster.my-local-i
Noteinparticularthattheendpointdescriptionsgivenunder --cluster.my-addressand --cluster.agency-endpointmustnotusethe
IPaddress 0.0.0.0becausetheymustcontainanactualaddressthatcanberoutedtothecorrespondingserver.The 0.0.0.0in -server.endpointsimplymeansthattheserverbindsitselftoallavailablenetworkdeviceswithallavailableIPaddresses.
UponregisteringwiththeagencyduringstartuptheclusterwillassignanIDtoeverytask.ThegeneratedIDwillbeprintedouttothe
logorcanbeaccessedviathehttpAPIbycalling http://server-address/_admin/server/id.Shouldyoueverhavetorestartatask,
simplyreusethesamevaluefor --cluster.my-local-infoandthesameIDwillbepicked.
YouhavenowlaunchedacompleteArangoDBclusterandcancontactitscoordinatorattheendpoint tcp://127.0.0.1:8531,which
meansthatyoucanreachthewebUIunder http://127.0.0.1:8531.
Secondaries
Secondariesneedabitmorework.Secondariesneedtohavesomeprimaryassigned.Todothatthereisaspecialroute.Toregistera
secondaryyoumustfirstfindouttheServer-IDoftheprimaryserver.ThengenerateyourownIDforthesecondaryyouareaboutto
startandcalloneofthecoordinatorslikethis(replacethevalueof"newSecondary"inthecommand):
curl-f-XPUT--data'{"primary":"DBServer001","oldSecondary":"none","newSecondary":"Secondary001"}'-H"Content-Type:application/json"http
Ifthatcallwassuccessfulyoucanstartthesecondary.Insteadofproviding --cluster.my-local-infoyoushouldnowprovidetheIdin
thecurlcallabovevia --cluster.my-id.Youcanomitthe --cluster.my-roleinthiscase.Thesecondarywillfindoutfromtheagency
aboutitsrole.
Tosumitup:
Cluster:Localtest
262
Cluster:Localtest
263
LaunchinganArangoDBclusteronmultiplemachines
Essentially,onecanusethemethodfromtheprevioussectiontostartanArangoDBclusteronmultiplemachinesaswell.Theonly
changesarethatonehastoreplacealllocaladdresses 127.0.0.1bytheactualIPaddressofthecorrespondingserver.
IfweassumethatyouwanttostartyouArangoDBclusteronthreedifferentmachineswithIPaddresses
192.168.1.1
192.168.1.2
192.168.1.3
thenthecommandsyouhavetouseare(youcanusehostnamesiftheycanberesolvedtoIPaddressesonallmachines):
On192.168.1.1:
On192.168.1.2:
On192.168.1.3:
build/bin/arangod--server.endpointtcp://0.0.0.0:5003--server.authenticationfalse--agency.id2--agency.size3--agency.endpointtcp://192.168.
On192.168.1.1:
build/bin/arangod--server.authentication=false--server.endpointtcp://0.0.0.0:8529--cluster.my-addresstcp://192.168.1.1:8529--cluster.my-local
On192.168.1.2:
On192.168.1.3:
Obviously,itwouldnolongerbenecessarytousedifferentportnumbersondifferentservers.Wehavechosentokeepallportnumbers
incomparisontothelocalsetuptominimizethenecessarychanges.
Ifyouwanttosetupsecondaries,thefollowingcommandswilldothejob:
On192.168.1.2:
On192.168.1.1:
Cluster:Processes
264
Notethatwehavestartedthe Secondary002onthesamemachineas DBServer001and Secondary001onthesamemachineas
DBServer002toavoidthatacompletepairislostwhenamachinefails.Furthermore,notethatArangoDBdoesnotyetperform
automaticfailovertothesecondary,ifaprimaryfails.ThisonlyworksintheApacheM esossetting.Forsynchronousreplication,
automaticfailoveralwaysworksandyoudonotneedtosetupsecondariesforthis.
Afterhavingswallowedtheselongishcommands,wehopethatyouappreciatethesimplicityofthesetupwithApacheM esosand
DC/OS.
Cluster:Processes
265
ArangoDBClusterandDocker
Networking
AbitofextracarehastobeinvestedduetothewayinwhichDockerisolatesitsnetwork.Bydefaultitfullyisolatesthenetworkandby
doingsoanendpointlike --server.endpointtcp://0.0.0.0:8529willonlybindtoallinterfacesinsidetheDockercontainerwhichdoes
notincludeanyexternalinterfaceonthehostmachine.Thismaybesufficientifyoujustwanttoaccessitlocallybutincaseyouwantto
exposeittotheoutsideyoumustfacilitateDockersportforwardingusingthe -pcommandlineoption.Besuretochecktheofficial
Dockerdocumentation.
Tosimplymakearangodbavailableonallhostinterfacesonport8529:
dockerrun-p8529:8529-eARANGO_NO_AUTH=1arangodb
AnotherpossibilityistostartDockervianetworkmode host.Thisispossiblebutgenerallynotrecommended.Todoitanywaycheck
theDockerdocumentationfordetails.
DockerandClustertasks
TostarttheclusterviaDockerisbasicallythesameasstartinglocallyoronmultiplemachines.Howeverjustlikewiththesingle
networkingimagewewillfacenetworkingissues.Youcansimplyusethe -pflagtomaketheindividualtaskavailableonthehost
machineoryoucoulduseDocker'slinkstoenabletaskintercommunication.
PleasenotethattherearesomeflagsthatspecifyhowArangoDBcanreachataskfromtheoutside.Theseareveryimportantandbuilt
forthisexactusecase.Anexampleconfigurationmightlooklikethis:
dockerrun-eARANGO_NO_AUTH=1-p192.168.1.1:10000:8529arangodb/arangodbarangod--server.endpointtcp://0.0.0.0:8529--cluster.my-addresstcp://
ThiswillstartaprimaryDBserverwithinaDockercontainerwithanisolatednetwork.WithintheDockercontaineritwillbindtoall
interfaces(thiswillbe127.0.0.1:8529andsomeinternalDockeriponport8529).Bysupplying -p192.168.1.1:10000:8529weare
establishingaportforwardingfromourlocalIP(192.168.1.1port10000inthisexample)toport8529insidethecontainer.Withinthe
commandwearetellingarangodhowitcanbereachedfromtheoutside --cluster.my-addresstcp://192.168.1.1:10000.This
informationwillbeforwardedtotheagencysothattheothertasksinyourclustercanseehowthisparticularDBServermaybereached.
Cluster:Docker
266
Administration
M ostadministrationcanbemanagedusingthearangosh.
Administration
267
WebInterface
ArangoDBcomeswithabuilt-inwebinterfaceforadministration.Theinterfacediffersforstandaloneinstancesandclustersetups.
Standalone:
Cluster:
WebInterface
268
WebInterface
269
QueryView
Thequeryviewoffersyouthreedifferentsubviews:
Editor
RunningQueries
SlowQueryHistory
AQLQueryEditor
ThewebinterfaceoffersaAQLQueryEditor:
Theeditorissplitintotwoparts,thequeryeditorpaneandthebindparameterpane.
Theleftpaneisyourregularqueryinputfield,whereyoucaneditandthenexecuteorexplainyourqueries.Bydefault,theenteredbind
parameterwillautomaticallyberecognizedandshowninthebindparametertableintherightpane,whereyoucaneasilyeditthem.
Theinputfieldsareequippedwithtypedetection.Thismeansyoudon'thavetousequotemarksaroundstring,justwritethemas-is.
Numberswillbetreatedasnumbers,trueandfalseasbooleans,nullasnull-typevalue.Squarebracketscanbeusedtodefinearrays,and
curlybracesforobjects(keysandvalueshavetobesurroundedbydoublequotes).Thiswillmostlybewhatyouwant.Butifyouwant
toforcesomethingtobetreatedasstring,usequotationmarksforthevalue:
123//interpretedasnumber
"123"//interpretedasstring
["foo","bar",123,true]//interpretedasarray
['foo','bar',123,true]//interpretedasstring
Queries
270
IfyouareusedtoworkwithJSON,youmaywanttoswitchthebindparametereditortoJSONmodebyclickingontheupperright
togglebutton.YoucantheneditthebindparametersinrawJSONformat.
CustomQueries
TosavethecurrentqueryusetheSavebuttoninthetopleftcorneroftheeditororusetheshortcut(seebelow).
BypressingtheQueriesbuttoninthetopleftcorneroftheeditoryouactivatethecustomqueriesview.Hereyoucanselectapreviously
storedcustomqueryoroneofourqueryexamples.
Clickonaquerytitletogetacodepreview.Inaddition,thereareactionbuttonsto:
Copytoeditor
Explainquery
Runquery
Deletequery
Forthebuilt-inexamplequeries,thereisonlyCopytoeditoravailable.
ToexportorimportqueriestoandfromJSONyoucanusethebuttonsontheright-handside.
Result
Queries
271
Eachqueryyouexecuteorexplainopensupanewresultbox,soyouareabletofireupmultiplequeriesandviewtheirresultsatthe
sametime.Everyqueryresultboxgivesyoudetailedqueryinformationandofcoursethequeryresultitself.Theresultboxescanbe
dismissedindividually,oraltogetherusingtheRemoveresultsbutton.Thetogglebuttoninthetoprightcornerofeachboxswitchesback
andforthbetweentheResultandAQLquerywithbindparameters.
Spotlight
Thespotlightfeatureopensupamodalview.ThereyoucanfindallAQLkeywords,AQLfunctionsandcollections(filteredbytheir
type)tohelpyoutobemoreproductiveinwritingyourqueries.Spotlightcanbeopenedbythemagicwandiconinthetoolbarorvia
shortcut(seebelow).
AQLEditorShortcuts
Queries
272
Ctrl/Cmd+Returntoexecuteaquery
Ctrl/Cmd+Shift+Returntoexplainaquery
Ctrl/Cmd+Shift+Stosavethecurrentquery
Ctrl/Cmd+Shift+Ctotogglecomments
Ctrl+Spacetoopenupthespotlightsearch
Ctrl+Cmd+Ztoundolastchange
Ctrl+Cmd+Shift+Ztoredolastchange
RunningQueries
TheRunningQueriestabgivesyouacompactoverviewofallrunningqueries.Byclickingtheredminusbutton,youcanabortthe
executionofarunningquery.
SlowQueryHistory
Queries
273
TheSlowQueryHistorytabgivesyouacompactoverviewofallpastslowqueries.
Queries
274
Collections
Thecollectionssectiondisplaysallavailablecollections.Fromhereyoucancreatenewcollectionsandjumpintoacollectionfordetails
(clickonacollectiontile).
Functions:
A:Togglefilterproperties
B:Searchcollectionbyname
D:Createcollection
C:Filterproperties
H:Showcollectiondetails(clicktile)
Information:
E:Collectiontype
F:Collectionstate(unloaded,loaded,...)
G:Collectionname
Collection
Collections
275
Therearefourviewcategories:
1. Content:
Createadocument
Deleteadocument
Filterdocuments
Downloaddocuments
Uploaddocuments
2. Indices:
Createindices
Deleteindices
3. Info:
Detailedcollectioninformationandstatistics
4. Settings:
Configurename,journalsize,indexbuckets,waitforsync
Deletecollection
Truncatecollection
Unload/Loadcollection
Savemodifedproperties(name,journalsize,indexbuckets,waitforsync)
Additionalinformation:
Uploadformat:
I.Line-wise
{"_key":"key1",...}
{"_key":"key2",...}
II.JSONdocumentsinalist
Collections
276
[
{"_key":"key1",...},
{"_key":"key2",...}
]
Collections
277
Cluster
Theclustersectiondisplaysstatisticsaboutthegeneralclusterperformance.
Statistics:
Availableandmissingcoordinators
Availableandmissingdatabaseservers
M emoryusage(percent)
Currentconnections
Data(bytes)
HTTP(bytes)
Averagerequesttime(seconds)
Nodes
Overview
Theoverviewshowsavailableandmissingcoordinatorsanddatabaseservers.
Cluster
278
Functions:
CoordinatorDashboard:ClickonaCoordinatorwillopenastatisticsdashboard.
Information(Coordinator/Databaseservers):
Name
Endpoint
LastHeartbeat
Status
Health
Shards
Theshardsectiondisplaysallavailableshardedcollections.
Cluster
279
Functions:
M oveShardLeader:Clickonaleaderdatabaseofashardserverwillopenamovesharddialog.Shardscanbetransferredtoall
availabledatabasservers,excepttheleadingdatabaseserveroranavailablefollower.
M oveShardFollower:Clickonafollowerdatabaseofashardwillopenamovesharddialog.Shardscanbetransferredtoall
availabledatabasservers,excepttheleadingdatabaseserveroranavailablefollower.
RebalanceShards:Anewdatabaseserverwillnothaveanyshards.Withtherebalancefunctionalitytheclusterwillstarttorebalance
shardsincludingemptydatabaseservers.
Information(collection):
Shard
Leader(greenstate:synciscomplete)
Followers
Cluster
280
Dashboard
TheDashboardtabprovidesstatisticswhicharepolledregularlyfromtheArangoDBserver.
RequestsStatistics:
Requestspersecond
Requesttypes
Numberofclientconnections
Transfersize
Transfersize(distribution)
Averagerequesttime
Averagerequesttime(distribution)
SystemResources:
Numberofthreads
M emory
Virtualsize
M ajorpagefaults
UsedCPUtime
Replication:
Replicationstate
Totals
Ticks
Progress
Dashboard
281
Document
Thedocumentsectionoffersaeditorwhichletyoueditdocumentsandedgesofacollection.
Functions:
Editdocument
Savedocument
Deletedocment
SwitchbetweenTree/Code-M ode
Createanewdocument
Information:
Displays:_id,_rev,_keyproperties
Document
282
Logs
Thelogssectiondisplaysallavailablelogentries.Logentriesarefilterablebytheirlogleveltypes.
Functions:
Filterlogentriesbyloglevel(all,info,error,warning,debug)
Information:
Loglevel
Date
M essage
Logs
283
Services
Theservicessectiondisplaysallinstalledfoxxapplications.Youcancreatenewservicesorgointoadetailedviewofachoosenservice.
CreateService
Therearefourdifferentpossibilitiestocreateanewservice:
1. Createserviceviazipfile
2. Createserviceviagithubrepository
3. CreateserviceviaofficialArangoDBstore
4. Createablankservicefromscratch
Services
284
ServiceView
Thissectionoffersseveralinformationaboutaspecificservice.
Therearefourviewcategories:
1. Info:
Displaysname,shortdescription,license,version,mode(production,development)
Offersabuttontogototheservicesinterface(ifavailable)
Services
285
2. Api:
DisplayAPIasSwaggerUI
DisplayAPIasRAWJSON
3. Readme:
Displaystheservicesmanual(ifavailable)
4. Settings:
Downloadserviceaszipfile
Runservicetests(ifavailable)
Runservicescripts(ifavailable)
Configuredependencies(ifavailable)
Changeserviceparameters(ifavailable)
Changemode(production,development)
Replacetheservice
Deletetheservice
Services
286
Graphs
TheGraphstabprovidesaviewerfacilityforgraphdatastoredinArangoDB.ItallowsbrowsingArangoDBgraphsstoredinthe
_graphssystemcollectionoragraphconsistingofanarbitraryvertexandedgecollection.
PleasenotethatthegraphviewerrequiresSVGsupportinyourbrowser.EspeciallyInternetExplorerbrowsersolderthanversion9are
likelytonotsupportthis.
Graphs
287
LeftToolbarFunctions:
Fold/Unfoldanode/draggraph
Draganode
Editanodeoredge
Draganddropanewedgebetweentwonodes
Addanode
Removeanode
TopToolbarFunctions:
Filtergraphbyattribute
Addnodebyattribute
Configurelabelsofnodesoredges
Groupnodesbyattribute
Limitcountofrenderednodes
Graphs
288
ArangoDBShellIntroduction
TheArangoDBshell(arangosh)isacommand-linetoolthatcanbeusedforadministrationofArangoDB,includingrunningad-hoc
queries.
ThearangoshbinaryisshippedwithArangoDB.ItoffersaJavaScriptshellenvironmentprovidingaccesstotheArangoDBserver.
Arangoshcanbeinvokedlikethis:
unix>arangosh
BydefaultarangoshwilltrytoconnecttoanArangoDBserverrunningonserverlocalhostonport8529.Itwillusetheusernameroot
andanemptypasswordbydefault.Additionallyitwillconnecttothedefaultdatabase(_system).Allthesedefaultscanbechangedusing
thefollowingcommand-lineoptions:
--server.database:nameofthedatabasetoconnectto
--server.endpoint:endpointtoconnectto
--server.username:databaseusername
--server.password:passwordtousewhenconnecting
--server.authentication:whetherornottouseauthentication
Forexample,toconnecttoanArangoDBserveronIP192.168.173.13onport8530withtheuserfooandusingthedatabasetest,use:
unix>arangosh\
--server.endpointtcp://192.168.173.13:8530\
--server.usernamefoo\
--server.databasetest\
--server.authenticationtrue
arangoshwillthendisplayapasswordpromptandtrytoconnecttotheserverafterthepasswordwasentered.
Tochangethecurrentdatabaseaftertheconnectionhasbeenmade,youcanusethe db._useDatabase()commandinarangosh:
arangosh>db._createDatabase("myapp");
true
arangosh>db._useDatabase("myapp");
true
arangosh>db._useDatabase("_system");
true
arangosh>db._dropDatabase("myapp");
true
Togetalistofavailablecommands,arangoshprovidesahelp()function.Callingitwilldisplayhelpfulinformation.
arangoshalsoprovidesauto-completion.Additionalinformationonavailablecommandsandmethodsisthusprovidedbytypingthe
firstfewlettersofavariableandthenpressingthetabkey.Itisrecommendtotrythiswithenteringdb.(withoutpressingreturn)and
thenpressingtab.
Bytheway,arangoshprovidesthedbobjectbydefault,andthisobjectcanbeusedforswitchingtoadifferentdatabaseandmanaging
collectionsinsidethecurrentdatabase.
Foralistofavailablemethodsforthedbobject,type
arangosh>db._help();
youcanpastemultiplelinesintoarangosh,giventhefirstlineendswithanopeningbrace:
ArangoDBShell
289
arangosh>for(vari=0;i<10;i++){
........>require("@arangodb").print("Helloworld"+i+"!\n");
........>}
ToloadyourownJavaScriptcodeintothecurrentJavaScriptinterpretercontext,usetheloadcommand:
require("internal").load("/tmp/test.js")//<-Linux/MacOS
require("internal").load("c:\\tmp\\test.js")//<-Windows
Exitingarangoshcanbedoneusingthekeycombination <CTRL>+Dorbytyping quit<CR>
ArangoDBShell
290
ArangoDBShellOutput
Bydefault,theArangoDBshellusesaprettyprinterwhenJSONdocumentsareprinted.Thisensuresdocumentsareprintedina
human-readableway:
arangosh>db._create("five")
arangosh>for(i=0;i<5;i++)db.five.save({value:i})
arangosh>db.five.toArray()
Whilethepretty-printerproducesnicelookingresults,itwillneedalotofscreenspaceforeachdocument.Sometimes,amoredense
outputmightbebetter.Inthiscase,theprettyprintercanbeturnedoffusingthecommandstop_pretty_print().
Toturnonprettyprintingagain,usethestart_pretty_print()command.
ShellOutput
291
ArangoDBShellConfiguration
arangoshwilllookforauser-definedstartupscriptnamed.arangosh.rcintheuser'shomedirectoryonstartup.Thehomedirectorywill
likelybe /home/<username>/onUnix/Linux,andisdeterminedonWindowsbypeekingintotheenvironmentvariables %HOMEDRIVE%and
%HOMEPATH%.
Ifthefile.arangosh.rcispresentinthehomedirectory,arangoshwillexecutethecontentsofthisfileinsidetheglobalscope.
Youcanusethistodefineyourownextravariablesandfunctionsthatyouneedoften.Forexample,youcouldputthefollowingintothe
.arangosh.rcfileinyourhomedirectory:
//"var"keywordavoidedintentionally...
//otherwise"timed"wouldnotsurvivethescopeofthisscript
global.timed=function(cb){
console.time("callback");
cb();
console.timeEnd("callback");
};
Thiswillmakeafunctionnamedtimedavailableinarangoshintheglobalscope.
Youcannowstartarangoshandinvokethefunctionlikethis:
timed(function(){
for(vari=0;i<1000;++i){
db.test.save({value:i});
}
});
Pleasekeepinmindthat,ifpresent,the.arangosh.rcfileneedstocontainvalidJavaScriptcode.Ifyouwantanyvariablesintheglobal
scopetosurviveyouneedtoomitthevarkeywordforthem.Otherwisethevariableswillonlybevisibleinsidethescriptitself,butnot
outside.
Configuration
292
DetailsabouttheArangoDBShell
Aftertheserverhasbeenstarted,youcanusetheArangoDBshell(arangosh)toadministratetheserver.Withoutanyarguments,the
ArangoDBshellwilltrytocontacttheserveronport8529onthelocalhost.FormoreinformationseetheArangoDBShell
documentation.Youmightneedtosetadditionaloptions(endpoint,usernameandpassword)whenconnecting:
unix>./arangosh--server.endpointtcp://127.0.0.1:8529--server.usernameroot
TheshellwillprintitsownversionnumberandifsuccessfullyconnectedtoaservertheversionnumberoftheArangoDBserver.
Command-LineOptions
Use --helptogetalistofcommand-lineoptions:
unix>./arangosh--help
STANDARDoptions:
--audit-log<string>auditlogfiletosavecommandsandresultsto
--configuration<string>readconfigurationfile
--helphelpmessage
--max-upload-size<uint64>maximumsizeofimportchunks(inbytes)(default:500000)
--no-auto-completedisableautocompletion
--no-colorsdeactivatecolorsupport
--pager<string>outputpager(default:"less-X-R-F-L")
--pretty-printprettyprintvalues
--quietnobanner
--temp.path<string>pathfortemporaryfiles(default:"/tmp/arangodb")
--use-pagerusepager
JAVASCRIPToptions:
--javascript.check<string>syntaxcheckcodeJavaScriptcodefromfile
--javascript.execute<string>executeJavaScriptcodefromfile
--javascript.execute-string<string>executeJavaScriptcodefromstring
--javascript.startup-directory<string>startuppathscontainingtheJavaScriptfiles
--javascript.unit-tests<string>donotstartasshell,rununittestsinstead
--jslint<string>donotstartasshell,runjslintinstead
LOGGINGoptions:
--log.level<string>loglevel(default:"info")
CLIENToptions:
--server.connect-timeout<double>connecttimeoutinseconds(default:3)
--server.authentication<bool>whetherornottouseauthentication(default:true)
--server.endpoint<string>endpointtoconnectto,use'none'tostartwithoutaserver(default:"tcp://127.0.0.1:8529")
--server.password<string>passwordtousewhenconnecting(leaveemptyforprompt)
--server.request-timeout<double>requesttimeoutinseconds(default:300)
--server.username<string>usernametousewhenconnecting(default:"root")
DatabaseWrappers
The dbobjectisavailableinarangoshaswellasonarangodi.e.ifyou'reusingFoxx.Whileitsinterfaceispersistantbetweenthe
arangoshandthearangodimplementations,itsunderpinningisnot.ThearangodimplementationareJavaScriptwrappersaround
ArangoDB'snativeC++implementation,whereasthearangoshimplementationwrapsHTTPaccessestoArangoDB'sRESTfullAPI.
Sowhilethiscodemayproducesimilarresultswhenexecutedinarangoshandarangod,thecpuusageandtimerequiredwillbereally
different:
for(i=0;i<100000;i++){
db.test.save({name:{first:"Jan"},count:i});
}
Details
293
Sincethearangoshversionwillbedoingaround100kHTTPrequests,andthearangodversionwilldirectlywritetothedatabase.
Using arangoshviaunixshebangmechanisms
Inunixoperatingsystemsyoucanstartscriptsbyspecifyingtheinterpreterinthefirstlineofthescript.Thisiscommonlycalled
shebangor hashbang.Youcanalsodothatwith arangosh,i.e.create ~/test.js:
#!/usr/bin/arangosh--javascript.execute
require("internal").print("helloworld")
db._query("FORxINtestRETURNx").toArray()
Notethatthefirstlinehastoendwithablankinordertomakeitwork.M arkitexecutabletotheOS:
#>chmoda+x~/test.js
andfinalytryitout:
#>~/test.js
Details
294
Arangoimp
ThismanualdescribestheArangoDBimporterarangoimp,whichcanbeusedforbulkimports.
ThemostconvenientmethodtoimportalotofdataintoArangoDBistousethearangoimpcommand-linetool.Itallowsyoutoimport
datarecordsfromafileintoanexistingdatabasecollection.
Itispossibletoimportdocumentkeyswiththedocumentsusingthe_keyattribute.Whenimportingintoanedgecollection,itis
mandatorythatallimporteddocumentshavethe_fromand_toattributes,andthattheycontainvalidreferences.
Let'sassumeforthefollowingexamplesyouwanttoimportuserdataintoanexistingcollectionnamed"users"ontheserver.
ImportingDataintoanArangoDBDatabase
ImportingJSON-encodedData
Let'sfurtherassumetheimportathandisencodedinJSON.We'llbeusingtheseexampleuserrecordstoimport:
{"name":{"first":"John","last":"Connor"},"active":true,"age":25,"likes":["swimming"]}
{"name":{"first":"Jim","last":"O'Brady"},"age":19,"likes":["hiking","singing"]}
{"name":{"first":"Lisa","last":"Jones"},"dob":"1981-04-09","likes":["running"]}
Toimporttheserecords,allyouneedtodoistoputthemintoafile(withonelineforeachrecordtoimport)andrunthefollowing
command:
>arangoimp--file"data.json"--typejson--collection"users"
Thiswilltransferthedatatotheserver,importtherecords,andprintastatussummary.Toshowtheintermediateprogressduringthe
importprocess,theoption--progresscanbeadded.Thisoptionwillshowthepercentageoftheinputfilethathasbeensenttothe
server.Thiswillonlybeusefulforbigimportfiles.
>arangoimp--file"data.json"--typejson--collectionusers--progresstrue
Itisalsopossibletousetheoutputofanothercommandasaninputforarangoimp.Forexample,thefollowingshellcommandcanbe
usedtopipedatafromthe catprocesstoarangoimp:
>catdata.json|arangoimp--file---typejson--collectionusers
Notethatyouhavetouse --file-ifyouwanttouseanothercommandasinputforarangoimp.Noprogresscanbereportedforsuch
importsasthesizeoftheinputwillbeunknowntoarangoimp.
Bydefault,theendpointtcp://127.0.0.1:8529willbeused.Ifyouwanttospecifyadifferentendpoint,youcanusethe--server.endpoint
option.Youprobablywanttospecifyadatabaseuserandpasswordaswell.Youcandosobyusingtheoptions--server.usernameand-server.password.Ifyoudonotspecifyapassword,youwillbepromptedforone.
>arangoimp--server.endpointtcp://127.0.0.1:8529--server.usernameroot--file"data.json"--typejson--collection"users"
Notethatthecollection(usersinthiscase)mustalreadyexistortheimportwillfail.Ifyouwanttocreateanewcollectionwiththe
importdata,youneedtospecifythe--create-collectionoption.Notethatbydefaultitwillcreateadocumentcollectionandnoede
collection.
>arangoimp--file"data.json"--typejson--collection"users"--create-collectiontrue
Tocreateanedgecollectioninstead,usethe--create-collection-typeoptionandsetittoedge:
Arangoimp
295
>arangoimp--file"data.json"--collection"myedges"--create-collectiontrue--create-collection-typeedge
Whenimportingdataintoanexistingcollectionitisoftenconvenienttofirstremovealldatafromthecollectionandthenstarttheimport.
Thiscanbeachievedbypassingthe--overwriteparametertoarangoimp.Ifitissettotrue,anyexistingdatainthecollectionwillbe
removedpriortotheimport.Notethatanyexistingindexdefinitionsforthecollectionwillbepreservedevenif--overwriteissettotrue.
>arangoimp--file"data.json"--typejson--collection"users"--overwritetrue
AstheimportfilealreadycontainsthedatainJSONformat,attributenamesanddatatypesarefullypreserved.Ascanbeseeninthe
exampledata,thereisnoneedforalldatarecordstohavethesameattributenamesortypes.Recordscanbeinhomogeneous.
Pleasenotethatbydefault,arangoimpwillimportdataintothespecifiedcollectioninthedefaultdatabase(_system).Tospecifya
differentdatabase,usethe--server.databaseoptionwheninvokingarangoimp.
JSONinputfileformats
Note:arangoimpsupportstwoformatswhenimportingJSONdatafromafile.Thefirstformat(alsousedabove)requirestheinputfile
tocontainonecompleteJSONdocumentineachline,e.g.
{"_key":"one","value":1}
{"_key":"two","value":2}
{"_key":"foo","value":"bar"}
...
Theaboveformatcanbeimportedsequentiallybyarangoimp.Itwillreaddatafromtheinputfileinchunksandsenditinbatchestothe
server.Eachbatchwillbeaboutasbigasspecifiedinthecommand-lineparameter--batch-size.
AnalternativeistoputonebigJSONdocumentintotheinputfilelikethis:
[
{"_key":"one","value":1},
{"_key":"two","value":2},
{"_key":"foo","value":"bar"},
...
]
Thisformatallowslinebreakswithintheinputfileasrequired.Thedownsideisthatthewholeinputfilewillneedtobereadby
arangoimpbeforeitcansendthefirstbatch.Thismightbeaproblemiftheinputfileisbig.Bydefault,arangoimpwillallowimporting
suchfilesuptoasizeofabout16M B.
Ifyouwanttoallowyourarangoimpinstancetousemorememory,youmaywanttoincreasethemaximumfilesizebyspecifyingthe
command-lineoption--batch-size.Forexample,tosetthebatchsizeto32M B,usethefollowingcommand:
>arangoimp--file"data.json"--typejson--collection"users"--batch-size33554432
Pleasealsonotethatyoumayneedtoincreasethevalueof--batch-sizeifasingledocumentinsidetheinputfileisbiggerthanthevalueof
--batch-size.
ImportingCSVData
arangoimpalsooffersthepossibilitytoimportdatafromCSVfiles.ThiscomeshandywhenthedataathandisinCSVformatalready
andyoudon'twanttospendtimeconvertingthemtoJSONfortheimport.
ToimportdatafromaCSVfile,makesureyourfilecontainstheattributenamesinthefirstrow.Allthefollowinglinesinthefilewillbe
interpretedasdatarecordsandwillbeimported.
TheCSVimportrequiresthedatatohaveahomogeneousstructure.Allrecordsmusthaveexactlythesameamountofcolumnsasthere
areheaders.
Arangoimp
296
Thecellvaluescanhavedifferentdatatypesthough.Ifacelldoesnothaveanyvalue,itcanbeleftemptyinthefile.Thesevalueswill
notbeimportedsotheattributeswillnot"bethere"indocumentcreated.Valuesenclosedinquoteswillbeimportedasstrings,soto
importnumericvalues,booleanvaluesorthenullvalue,don'tenclosethevalueinquotesinyourfile.
We'llbeusingthefollowingimportfortheCSVimport:
"first","last","age","active","dob"
"John","Connor",25,true,
"Jim","O'Brady",19,,
"Lisa","Jones",,,"1981-04-09"
Hans,dosSantos,0123,,
Wayne,Brewer,,false,
Thecommandlinetoexecutetheimportis:
>arangoimp--file"data.csv"--typecsv--collection"users"
Theabovedatawillbeimportedinto5documentswhichwilllookasfollows:
{"first":"John","last":"Connor","active":true,"age":25}
{"first":"Jim","last":"O'Brady","age":19}
{"first":"Lisa","last":"Jones","dob":"1981-04-09"}
{"first":"Hans","last":"dosSantos","age":123}
{"first":"Wayne","last":"Brewer","active":false}
Ascanbeseen,valuesleftcompletelyemptyintheinputfilewillbetreatedasabsent.Numericvaluesnotenclosedinquoteswillbe
treatedasnumbers.Notethatleadingzerosinnumericvalueswillberemoved.Toimportnumberswithleadingzeros,pleaseusestrings.
Theliteralstrueandfalsewillbetreatedasbooleansiftheyarenotenclosedinquotes.Othervaluesnotenclosedinquoteswillbetreated
asstrings.Anyvaluesenclosedinquoteswillbetreatedasstrings,too.
Stringvaluescontainingthequotecharacterortheseparatormustbeenclosedwithquotecharacters.Withinastring,thequotecharacter
itselfmustbeescapedwithanotherquotecharacter(orwithabackslashifthe--backslash-escapeoptionisused).
Notethatthequoteandseparatorcharacterscanbeadjustedviathe--quoteand--separatorargumentswheninvokingarangoimp.The
quotecharacterdefaultstothedoublequote(").Tousealiteralquoteinastring,youcanusetwoquotecharacters.Tousebackslashfor
escapingquotecharacters,pleasesettheoption--backslash-escapetotrue.
TheimportersupportsWindows(CRLF)andUnix(LF)linebreaks.Linebreaksmightalsooccurinsidevaluesthatareenclosedwiththe
quotecharacter.
Here'sanexampleforusingliteralquotesandnewlinesinsidevalues:
"name","password"
"Foo","r4ndom""123!"
"Bar","wow!
thisisa
multinepassword!"
"Bartholomew""Bart""Simpson","Milhouse"
Extrawhitespaceattheendofeachlinewillbeignored.Whitespaceatthestartoflinesorbetweenfieldvalueswillnotbeignored,so
pleasemakesurethatthereisnoextrawhitespaceinfrontofvaluesorbetweenthem.
ImportingTSVData
Youmayalsoimporttab-separatedvalues(TSV)fromafile.Thisformatisverysimple:everylineinthefilerepresentsadatarecord.
Thereisnoquotingorescaping.Thatalsomeansthattheseparatorcharacter(whichdefaultstothetabstopsymbol)mustnotbeused
anywhereintheactualdata.
AswithCSV,thefirstlineintheTSVfilemustcontaintheattributenames,andalllinesmusthaveanidenticalnumberofvalues.
Ifadifferentseparatorcharacterorstringshouldbeused,itcanbespecifiedwiththe--separatorargument.
AnexamplecommandlinetoexecutetheTSVimportis:
Arangoimp
297
>arangoimp--file"data.tsv"--typetsv--collection"users"
ImportingintoanEdgeCollection
arangoimpcanalsobeusedtoimportdataintoanexistingedgecollection.Theimportdatamust,foreachedgetoimport,containatleast
the_fromand_toattributes.Theseindicatewhichothertwodocumentstheedgeshouldconnect.Itisnecessarythattheseattributesare
setforallrecords,andpointtovaliddocumentidsinexistingcollections.
Examples
{"_from":"users/1234","_to":"users/4321","desc":"1234isconnectedto4321"}
Note:Theedgecollectionmustalreadyexistwhentheimportisstarted.Usingthe--create-collectionflagwillnotworkbecause
arangoimpwillalwaystrytocreatearegulardocumentcollectionifthetargetcollectiondoesnotexist.
Updatingexistingdocuments
Bydefault,arangoimpwilltrytoinsertalldocumentsfromtheimportfileintothespecifiedcollection.Incasetheimportfilecontains
documentsthatarealreadypresentinthetargetcollection(matchingisdoneviathe_keyattributes),thenadefaultarangoimprunwillnot
importthesedocumentsandcomplainaboutuniquekeyconstraintviolations.
However,arangoimpcanbeusedtoupdateorreplaceexistingdocumentsincasetheyalreadyexistinthetargetcollection.Itprovidesthe
command-lineoption--on-duplicatetocontrolthebehaviorincaseadocumentisalreadypresentinthedatabase.
Thedefaultvalueof--on-duplicateiserror.Thismeansthatwhentheimportfilecontainsadocumentthatispresentinthetarget
collectionalready,thentryingtore-insertadocumentwiththesame_keyvalueisconsideredanerror,andthedocumentinthedatabase
willnotbemodified.
Otherpossiblevaluesfor--on-duplicateare:
update:eachdocumentpresentintheimportfilethatisalsopresentinthetargetcollectionalreadywillbeupdatedbyarangoimp.
updatewillperformapartialupdateoftheexistingdocument,modifyingonlytheattributesthatarepresentintheimportfileand
leavingallotherattributesuntouched.
Thevaluesofsystemattributes_id,_key,_rev,_fromand_tocannotbeupdatedorreplacedinexistingdocuments.
replace:eachdocumentpresentintheimportfilethatisalsopresentinthetargetcollectionalreadywillbereplacebyarangoimp.
replacewillreplacetheexistingdocumententirely,resultinginadocumentwithonlytheattributesspecifiedintheimportfile.
Thevaluesofsystemattributes_id,_key,_rev,_fromand_tocannotbeupdatedorreplacedinexistingdocuments.
ignore:eachdocumentpresentintheimportfilethatisalsopresentinthetargetcollectionalreadywillbeignoredandnotmodified
inthetargetcollection.
When--on-duplicateissettoeitherupdateorreplace,arangoimpwillreturnthenumberofdocumentsupdated/replacedintheupdated
returnvalue.Whensettoanothervalue,thevalueofupdatedwillalwaysbezero.When--on-duplicateissettoignore,arangoimpwill
returnthenumberofignoreddocumentsintheignoredreturnvalue.Whensettoanothervalue,ignoredwillalwaysbezero.
Itispossibletoperformacombinationofinsertsandupdates/replaceswithasinglearangoimprun.When--on-duplicateissettoupdate
orreplace,alldocumentspresentintheimportfilewillbeinsertedintothetargetcollectionprovidedtheyarevalidanddonotalready
existwiththespecified_key.Documentsthatarealreadypresentinthetargetcollection(identifiedby_keyattribute)willinsteadbe
updated/replaced.
Arangoimpresultoutput
Anarangoimpimportrunwillprintoutthefinalresultsonthecommandline.Itwillshowthe
numberofdocumentscreated(created)
numberofdocumentsupdated/replaced(updated/replaced,onlynon-zeroif--on-duplicatewassettoupdateorreplace,seebelow)
numberofwarningsorerrorsthatoccurredontheserverside(warnings/errors)
numberofignoreddocuments(onlynon-zeroif--on-duplicatewassettoignore).
Arangoimp
298
Example
created:2
warnings/errors:0
updated/replaced:0
ignored:0
ForCSVandTSVimports,thetotalnumberofinputfilelinesreadwillalsobeprinted(linesread).
arangoimpwillalsoprintoutdetailsaboutwarningsanderrorsthathappenedontheserver-side(ifany).
AttributeNamingandSpecialAttributes
AttributeswhosenamesstartwithanunderscorearetreatedinaspecialwaybyArangoDB:
theoptional_keyattributecontainsthedocument'skey.Ifspecified,thevaluemustbeformallyvalid(e.g.mustbeastringand
conformtothenamingconventions).Additionally,thekeyvaluemustbeuniquewithinthecollectiontheimportisrunfor.
_from:whenimportingintoanedgecollection,thisattributecontainstheidofoneofthedocumentsconnectedbytheedge.The
valueof_frommustbeasyntacticallyvaliddocumentidandthereferredcollectionmustexist.
_to:whenimportingintoanedgecollection,thisattributecontainstheidoftheotherdocumentconnectedbytheedge.Thevalueof
_tomustbeasyntacticallyvaliddocumentidandthereferredcollectionmustexist.
_rev:thisattributecontainstherevisionnumberofadocument.However,therevisionnumbersaremanagedbyArangoDBand
cannotbespecifiedonimport.Thusanyvalueinthisattributeisignoredonimport.
Ifyouimportvaluesinto_key,youshouldmakesuretheyarevalidandunique.
Whenimportingdataintoanedgecollection,youshouldmakesurethatallimportdocumentscan_fromand_toandthattheirvalues
pointtoexistingdocuments.
Toavoidspecifyingcompletedocumentids(consistingofcollectionnamesanddocumentkeys)for_fromand_tovalues,therearethe
options--from-collection-prefixand--to-collection-prefix.Ifspecified,thesevalueswillbeautomaticallyprependedtoeachvaluein
_from(or_toresp.).Thisallowsspecifyingonlydocumentkeysinside_fromand/or_to.
Example
>arangoimp--from-collection-prefixusers--to-collection-prefixproducts...
Importingthefollowingdocumentwillthencreateanedgebetweenusers/1234andproducts/4321:
{"_from":"1234","_to":"4321","desc":"users/1234isconnectedtoproducts/4321"}
Arangoimp
299
DumpingDatafromanArangoDBdatabase
TodumpdatafromanArangoDBserverinstance,youwillneedtoinvokearangodump.Dumpscanbere-importedwitharangorestore.
arangodumpcanbeinvokedbyexecutingthefollowingcommand:
unix>arangodump--output-directory"dump"
ThiswillconnecttoanArangoDBserveranddumpallnon-systemcollectionsfromthedefaultdatabase(_system)intoanoutput
directorynameddump.Invokingarangodumpwillfailiftheoutputdirectoryalreadyexists.Thisisanintentionalsecuritymeasureto
preventyoufromaccidentallyoverwritingalreadydumpeddata.Ifyouarepositivethatyouwanttooverwritedataintheoutput
directory,youcanusetheparameter--overwritetruetoconfirmthis:
unix>arangodump--output-directory"dump"--overwritetrue
arangodumpwillbydefaultconnecttothe_systemdatabaseusingthedefaultendpoint.Ifyouwanttoconnecttoadifferentdatabaseor
adifferentendpoint,oruseauthentication,youcanusethefollowingcommand-lineoptions:
--server.username:username
--server.password:passwordtouse(omitthisandyou'llbepromptedforthepassword)
Here'sanexampleofdumpingdatafromanon-standardendpoint,usingadedicateddatabasename:
unix>arangodump--server.endpointtcp://192.168.173.13:8531--server.usernamebackup--server.databasemydb--output-directory"dump"
Whenfinished,arangodumpwillprintoutasummarylinewithsomeaggregatestatisticsaboutwhatitdid,e.g.:
Processed43collection(s),wrote408173500byte(s)intodatafiles,sent88batch(es)
Bydefault,arangodumpwilldumpbothstructuralinformationanddocumentsfromallnon-systemcollections.Toadjustthis,thereare
thefollowingcommand-linearguments:
--dump-data:settotruetoincludedocumentsinthedump.Settofalsetoexcludedocuments.Thedefaultvalueistrue.
--include-system-collections:whetherornottoincludesystemcollectionsinthedump.Thedefaultvalueisfalse.
Forexample,toonlydumpstructuralinformationofallcollections(includingsystemcollections),use:
unix>arangodump--dump-datafalse--include-system-collectionstrue--output-directory"dump"
Torestrictthedumptojustspecificcollections,thereisisthe--collectionoption.Itcanbespecifiedmultipletimesifrequired:
unix>arangodump--collectionmyusers--collectionmyvalues--output-directory"dump"
Structuralinformationforacollectionwillbesavedinfileswithnamepattern.structure.json.EachstructurefilewillcontainsaJSON
objectwiththeseattributes:
parameters:containsthecollectionproperties
indexes:containsthecollectionindexes
Documentdataforacollectionwillbesavedinfileswithnamepattern.data.json.Eachlineinadatafileisadocumentinsertion/update
ordeletionmarker,alongsidewithsomemetadata.
StartingwithVersion2.1ofArangoDB,thearangodumptoolalsosupportssharding.Simplypointittooneofthecoordinatorsandit
willbehaveexactlyasdescribedabove,workingonshardedcollectionsinthecluster.
Arangodump
300
However,asopposedtothesingleinstancesituation,thisoperationdoesnotguaranteetodumpaconsistentsnapshotifwrite
operationshappenduringthedumpoperation.Itisthereforerecommendednottoperformanydata-modifcationoperationsonthe
clusterwhilstarangodumpisrunning.
Asabove,theoutputwillbeonestructuredescriptionfileandonedatafilepershardedcollection.Notethatthedatainthedatafileis
sortedfirstbyshardsandwithineachshardbyascendingtimestamp.Thestructuralinformationofthecollectioncontainsthenumberof
shardsandtheshardkeys.
NotethattheversionofthearangodumpclienttoolneedstomatchtheversionoftheArangoDBserveritconnectsto.Bydefault,
arangodumpwillproduceadumpthatcanberestoredwiththearangorestoretoolofthesameversion.Anexceptionisarangodumpin
3.0,whichsupportsdumpingdatainaformatcompatiblewithArangoDB2.8.Inordertoproducea2.8-compatibledumpwitha3.0
ArangoDB,pleasespecifytheoption --compat28truewheninvokingarangodump.
unix>arangodump--compat28true--collectionmyvalues--output-directory"dump"
Arangodump
301
Arangorestore
Toreloaddatafromadumppreviouslycreatedwitharangodump,ArangoDBprovidesthearangorestoretool.
ReloadingDataintoanArangoDBdatabase
Invokingarangorestore
arangorestorecanbeinvokedfromthecommand-lineasfollows:
unix>arangorestore--input-directory"dump"
ThiswillconnecttoanArangoDBserverandreloadstructuralinformationanddocumentsfoundintheinputdirectorydump.Pleasenote
thattheinputdirectorymusthavebeencreatedbyrunningarangodumpbefore.
arangorestorewillbydefaultconnecttothe_systemdatabaseusingthedefaultendpoint.Ifyouwanttoconnecttoadifferentdatabase
oradifferentendpoint,oruseauthentication,youcanusethefollowingcommand-lineoptions:
--server.username:username
--server.password:passwordtouse(omitthisandyou'llbepromptedforthepassword)
Sinceversion2.6arangorestoreprovidestheoption--create-database.Settingthisoptiontotruewillcreatethetargetdatabaseifitdoes
notexist.Whencreatingthetargetdatabase,theusernameandpasswordspassedtoarangorestore(inoptions--server.usernameand-server.password)willbeusedtocreateaninitialuserforthenewdatabase.
Here'sanexampleofreloadingdatatoanon-standardendpoint,usingadedicateddatabasename:
unix>arangorestore--server.endpointtcp://192.168.173.13:8531--server.usernamebackup--server.databasemydb--input-directory"dump"
Tocreatethetargetdatabasewherestoring,useacommandlikethis:
unix>arangorestore--server.usernamebackup--server.databasenewdb--create-databasetrue--input-directory"dump"
arangorestorewillprintoutitsprogresswhilerunning,andwillendwithalineshowingsomeaggregatestatistics:
Processed2collection(s),read2256byte(s)fromdatafiles,sent2batch(es)
Bydefault,arangorestorewillre-createallnon-systemcollectionsfoundintheinputdirectoryandloaddataintothem.Ifthetarget
databasealreadycontainscollectionswhicharealsopresentintheinputdirectory,theexistingcollectionsinthedatabasewillbedropped
andre-createdwiththedatafoundintheinputdirectory.
Thefollowingparametersareavailabletoadjustthisbehavior:
--create-collection:settotruetocreatecollectionsinthetargetdatabase.Ifthetargetdatabasealreadycontainsacollectionwiththe
samename,itwillbedroppedfirstandthenre-createdwiththepropertiesfoundintheinputdirectory.Settofalsetokeepexisting
collectionsinthetargetdatabase.Ifsettofalseandarangorestoreencountersacollectionthatispresentinboththetargetdatabase
andtheinputdirectory,itwillabort.Thedefaultvalueistrue.
--import-data:settotruetoloaddocumentdataintothecollectionsinthetargetdatabase.Settofalsetonotloadanydocument
data.Thedefaultvalueistrue.
--include-system-collections:whetherornottoincludesystemcollectionswhenre-creatingcollectionsorreloadingdata.Thedefault
valueisfalse.
Forexample,to(re-)createallnon-systemcollectionsandloaddocumentdataintothem,use:
Arangorestore
302
unix>arangorestore--create-collectiontrue--import-datatrue--input-directory"dump"
Thiswilldroppotentiallyexistingcollectionsinthetargetdatabasethatarealsopresentintheinputdirectory.
Toincludesystemcollectionstoo,use--include-system-collectionstrue:
unix>arangorestore--create-collectiontrue--import-datatrue--include-system-collectionstrue--input-directory"dump"
To(re-)createallnon-systemcollectionswithoutloadingdocumentdata,use:
unix>arangorestore--create-collectiontrue--import-datafalse--input-directory"dump"
Thiswillalsodropexistingcollectionsinthetargetdatabasethatarealsopresentintheinputdirectory.
Tojustloaddocumentdataintoallnon-systemcollections,use:
unix>arangorestore--create-collectionfalse--import-datatrue--input-directory"dump"
Torestrictreloadingtojustspecificcollections,thereisisthe--collectionoption.Itcanbespecifiedmultipletimesifrequired:
unix>arangorestore--collectionmyusers--collectionmyvalues--input-directory"dump"
Collectionswillbeprocessedbyinalphabeticalorderbyarangorestore,withalldocumentcollectionsbeingprocessedbeforealledge
collections.Thisistoensurethatreloadingdataintoedgecollectionswillhavethedocumentcollectionslinkedinedges(_fromand_to
attributes)loaded.
RestoringRevisionIdsandCollectionIds
arangorestorewillreloaddocumentandedgesdatawiththeexactsame_key,_fromand_tovaluesfoundintheinputdirectory.
However,whenloadingdocumentdata,itwillassignitsownvaluesforthe_revattributeofthereloadeddocuments.Thoughthis
differenceisintentional(normally,everyservershouldcreateitsown_revvalues)theremightbesituationswhenitisrequiredtore-use
theexactsame_revvaluesforthereloadeddata.Thiscanbeachievedbysettingthe--recycle-idsparametertotrue:
unix>arangorestore--collectionmyusers--collectionmyvalues--recycle-idstrue--input-directory"dump"
Notethatsetting--recycle-idstotruewillalsocausecollectionstobe(re-)createdinthetargetdatabasewiththeexactsamecollectionid
asintheinputdirectory.Anypotentiallyexistingcollectioninthetargetdatabasewiththesamecollectionidwillthenbedropped.
Setting--recycle-idstofalseoromittingitwillonlyusethecollectionnamefromtheinputdirectoryandallowthetargetdatabaseto
createthecollectionwithadifferentid(thoughwiththesamename)thanintheinputdirectory.
ReloadingDataintoadifferentCollection
Withsomecreativityyoucanusearangodumpandarangorestoretotransferdatafromonecollectionintoanother(eitheronthesame
serverornot).Forexample,tocopydatafromacollectionmyvaluesindatabasemydbintoacollectionmycopyvaluesindatabasemycopy,
youcanstartwiththefollowingcommand:
unix>arangodump--collectionmyvalues--server.databasemydb--output-directory"dump"
Thiswillcreatetwofiles,myvalues.structure.jsonandmyvalues.data.json,intheoutputdirectory.Toloaddatafromthedatafileintoan
existingcollectionmycopyvaluesindatabasemycopy,renamethefilestomycopyvalues.structure.jsonandmycopyvalues.data.json.After
that,runthefollowingcommand:
unix>arangorestore--collectionmycopyvalues--server.databasemycopy--input-directory"dump"
Usingarangorestorewithsharding
Arangorestore
303
AsofVersion2.1thearangorestoretoolsupportssharding.Simplypointittooneofthecoordinatorsinyourclusteranditwillworkas
usualbutonshardedcollectionsinthecluster.
Ifarangorestoreisaskedtodropandre-createacollection,itwillusethesamenumberofshardsandthesameshardkeysaswhenthe
collectionwasdumped.Thedistributionoftheshardstotheserverswillalsobethesameasatthetimeofthedump.Thismeansin
particularthatDBserverswiththesameIDsasbeforemustbepresentintheclusterattimeoftherestore.
Ifacollectionwasdumpedfromasingleinstance,onecanmanuallyaddthestructuraldescriptionfortheshardkeysandthenumberand
distributionoftheshardsandthentherestoreintoaclusterwillwork.
IfyourestoreacollectionthatwasdumpedfromaclusterintoasingleArangoDBinstance,thenumberofshardsandtheshardkeyswill
silentlybeignored.
Notethatinacluster,everynewlycreatedcollectionwillhaveanewID,itisnotpossibletoreusetheIDfromtheoriginallydumped
collection.ThisisforsafetyreasonstoensureconsistencyofIDs.
Arangorestore
304
ManagingUsers
TheusermanagementinArangoDB3issimilartotheonefoundinM ySQL,Postgres,orotherdatabasesystems.
AnArangoDBservercontainsalistofusers.Eachusercanhaveaccesstooneormoredatabases(ornoneforthatmatter).
Inordertomanageusersusethewebinterface.Logintothe_systemdatabaseandgotothe"User"section.
UsingtheArangoDBshell
Alternatively,youcanusetheArangoDBshell.Fireuparangoshandrequiretheusersmodule.
arangosh>varusers=require("@arangodb/users");
arangosh>users.save("admin@testapp","mypassword");
[email protected].
arangosh>users.grantDatabase("admin@testapp","testdb");
Thisgrantstheuseraccesstothedatabasetestdb. revokeDatabasewillrevoketheright.
Save
users.save(user,passwd,active,extra)
ThiswillcreateanewArangoDBuser.Theusernamemustbespecifiedinuserandmustnotbeempty.
Thepasswordmustbegivenasastring,too,butcanbeleftemptyifrequired.Ifyoupassthespecialvalue
ARANGODB_DEFAULT_ROOT_PASSWORD,thepasswordwillbesetthevaluestoredintheenvironmentvariable
ARANGODB_DEFAULT_ROOT_PASSWORD.ThiscanbeusedtopassaninstancevariableintoArangoDB.Forexample,theinstanceidentifierfrom
Amazon.
Iftheactiveattributeisnotspecified,itdefaultstotrue.Theextraattributecanbeusedtosavecustomdatawiththeuser.
Thismethodwillfailifeithertheusernameorthepasswordsarenotspecifiedorgiveninawrongformat,ortherealreadyexistsauser
withthespecifiedname.
Note:theuserwillnothavepermissiontoaccessanydatabase.Youneedtogranttheaccessrightsforoneormoredatabasesusing
grantDatabase.
Examples
arangosh>require("@arangodb/users").save("my-user","my-secret-password");
GrantDatabase
users.grantDatabase(user,database)
Thisgrantsread/writeaccesstothedatabasefortheuser.
Ifauserhasaccessrightstothe_systemdatabase,heisconsideredsuperuser.
RevokeDatabase
users.revokeDatabase(user,database)
Thisrevokesread/writeaccesstothedatabasefortheuser.
M anagingUsers
305
Replace
users.replace(user,passwd,active,extra)
ThiswilllookupanexistingArangoDBuserandreplaceitsuserdata.
Theusernamemustbespecifiedinuser,andauserwiththespecifiednamemustalreadyexistinthedatabase.
Thepasswordmustbegivenasastring,too,butcanbeleftemptyifrequired.
Iftheactiveattributeisnotspecified,itdefaultstotrue.Theextraattributecanbeusedtosavecustomdatawiththeuser.
Thismethodwillfailifeithertheusernameorthepasswordsarenotspecifiedorgiveninawrongformat,orifthespecifiedusercannot
befoundinthedatabase.
Note:thisfunctionwillnotworkfromwithinthewebinterface
Examples
arangosh>require("@arangodb/users").replace("my-user","my-changed-password");
Update
users.update(user,passwd,active,extra)
ThiswillupdateanexistingArangoDBuserwithanewpasswordandotherdata.
Theusernamemustbespecifiedinuserandtheusermustalreadyexistinthedatabase.
Thepasswordmustbegivenasastring,too,butcanbeleftemptyifrequired.
Iftheactiveattributeisnotspecified,thecurrentvaluesavedfortheuserwillnotbechanged.Thesameistruefortheextraattribute.
Thismethodwillfailifeithertheusernameorthepasswordsarenotspecifiedorgiveninawrongformat,orifthespecifiedusercannot
befoundinthedatabase.
Examples
arangosh>require("@arangodb/users").update("my-user","my-secret-password");
isValid
users.isValid(user,password)
Checkswhetherthegivencombinationofusernameandpasswordisvalid.Thefunctionwillreturnabooleanvalueifthecombinationof
usernameandpasswordisvalid.
Eachcalltothisfunctionispenalizedbytheserversleepingarandomamountoftime.
Examples
arangosh>require("@arangodb/users").isValid("my-user","my-secret-password");
true
Remove
users.remove(user)
RemovesanexistingArangoDBuserfromthedatabase.
TheusernamemustbespecifiedinUserandthespecifiedusermustexistinthedatabase.
M anagingUsers
306
Thismethodwillfailiftheusercannotbefoundinthedatabase.
Examples
arangosh>require("@arangodb/users").remove("my-user");
Document
users.document(user)
FetchesanexistingArangoDBuserfromthedatabase.
Theusernamemustbespecifiedinuser.
Thismethodwillfailiftheusercannotbefoundinthedatabase.
Examples
arangosh>require("@arangodb/users").document("my-user");
all()
users.all()
FetchesallexistingArangoDBusersfromthedatabase.
Examples
arangosh>require("@arangodb/users").all();
Reload
users.reload()
Reloadstheuserauthenticationdataontheserver
Alluserauthenticationdataisloadedbytheserveronceonstartuponlyandiscachedafterthat.Whenusersgetaddedordeleted,acache
flushisdoneautomatically,andthiscanbeperformedbycalledthismethod.
Examples
arangosh>require("@arangodb/users").reload();
ComparisontoArangoDB2
ArangoDB2containedseparateusersperdatabase.Itwasnotpossibletogiveanuseraccesstotwoormoredatabases.Thisproved
impractical.ThereforeweswitchtoamorecommonusermodelinArangoDB3.
Command-LineOptionsfortheAuthenticationandAuthorization
--server.authenticationSettingthisoptiontofalsewillturnoffauthenticationontheserversidesoallclientscanexecuteanyaction
withoutauthorizationandprivilegechecks.Thedefaultvalueistrue.
--server.authentication-system-onlybooleanControlswhetherincomingrequestsneedauthenticationonlyiftheyaredirectedtothe
ArangoDB'sinternalAPIsandfeatures,locatedat/_api/,/_admin/etc.Iftheflagissettotrue,thenHTTPauthenticationisonlyrequired
forrequestsgoingtoURLsstartingwith/_,butnotforotherURLs.Theflagcanthusbeusedtoexposeauser-madeAPIwithoutHTTP
M anagingUsers
307
authenticationtotheoutsideworld,buttopreventtheoutsideworldfromusingtheArangoDBAPIandtheadmininterfacewithout
authentication.NotethatcheckingtheURLisperformedafteranydatabasenameprefixhasbeenremoved.Thatmeanswhentheactual
URLcalledis/_db/_system/myapp/myaction,theURL/myapp/myactionwillbeusedforauthentication-system-onlycheck.Thedefaultis
true.NotethatauthenticationstillneedstobeenabledfortheserverregularlyinorderforHTTPauthenticationtobeforcedforthe
ArangoDBAPIandthewebinterface.Settingonlythisflagisnotenough.YoucancontrolArangoDB'sgeneralauthenticationfeature
withthe--server.authenticationflag.
M anagingUsers
308
Command-lineoptions
ConfigurationFiles
Optionscanbespecifiedonthecommandlineorinconfigurationfiles.IfastringVariableoccursinthevalue,itisreplacedbythe
correspondingenvironmentvariable.
GeneralOptions
Generalhelp
programoptions --help
-h
Printsalistofthemostcommonoptionsavailableandthenexits.Inordertoseealloptionsuse--help-all.
Version
versionoftheapplication --version
-v
Printstheversionoftheserverandexits.
DatabaseUpgrade
--database.auto-upgrade
Specifyingthisoptionwillmaketheserverperformadatabaseupgradeatstart.Adatabaseupgradewillfirstcomparetheversionnumber
storedinthefileVERSIONinthedatabasedirectorywiththecurrentserverversion.
Ifthetwoversionnumbersmatch,theserverwillstartnormally.
Iftheversionnumberfoundinthedatabasedirectoryishigherthantheversionnumbertheserverisrunning,theserverexpectsthisisan
unintentionaldowngradeandwillwarnaboutthis.Itwillhoweverstartnormally.Usingtheserverintheseconditionsishowevernot
recommendednorsupported.
Iftheversionnumberfoundinthedatabasedirectoryislowerthantheversionnumbertheserverisrunning,theserverwillcheck
whetherthereareanyupgradetaskstoperform.Itwillthenexecuteallrequiredupgradetasksandprinttheirstatuses.Ifoneofthe
upgradetasksfails,theserverwillexitandrefusetostart.Re-startingtheserverwiththeupgradeoptionwillthenagaintriggerthe
upgradecheckandexecutionuntiltheproblemisfixed.Ifalltasksarefinished,theserverwillstartnormally.
Whetherornotthisoptionisspecified,theserverwillalwaysperformaversioncheckonstartup.RunningtheserverwithanonmatchingversionnumberintheVERSIONfilewillmaketheserverrefusetostart.
Configuration
configfile --configurationfilename
-cfilename
Specifiesthenameoftheconfigurationfiletouse.
Ifthiscommandisnotpassedtotheserver,thenbydefault,theserverwillattempttofirstlocateafilenamed~/.arango/arangod.confin
theuser'shomedirectory.
Ifnosuchfileisfound,theserverwillproceedtolookforafilearangod.confinthesystemconfigurationdirectory.Thesystem
configurationdirectoryisplatform-specific,andmaybechangedwhencompilingArangoDByourself.Itmaydefaultto/etc/arangodbor
/usr/local/etc/arangodb.Thisfileisinstalledwhenusingapackagemanagerlikerpmordpkg.Ifyoumodifythisfileandlaterupgradeto
ServerConfiguration
309
anewversionofArangoDB,thenthepackagemanagernormallywarnsyouabouttheconflict.Inordertoavoidthesewarningforsmall
adjustments,youcanputlocaloverridesintoafilearangod.conf.local.
Onlycommandlineoptionswithavalueshouldbesetwithintheconfigurationfile.Commandlineoptionswhichactasflagsshouldbe
enteredonthecommandlinewhenstartingtheserver.
Whitespaceintheconfigurationfileisignored.Eachoptionisspecifiedonaseparatelineintheform
key=value
Alternatively,aheadersectioncanbespecifiedandoptionspertainingtothatsectioncanbespecifiedinashorterform
[log]
level=trace
ratherthanspecifying
log.level=trace
Commentscanbeplacedintheconfigurationfile,onlyifthelinebeginswithoneormorehashsymbols(#).
Theremaybeoccasionswhereaconfigurationfileexistsandtheuserwishestooverrideconfigurationsettingsstoredinaconfiguration
file.Anysettingsspecifiedonthecommandlinewilloverwritethesamesettingwhenitappearsinaconfigurationfile.Iftheuserwishes
tocompletelyignoreconfigurationfileswithoutnecessarilydeletingthefile(orfiles),thenaddthecommandlineoption
-cnone
or
--configurationnone
Whenstartinguptheserver.Notethat,thewordnoneiscase-insensitive.
Daemon
--daemon
Runstheserverasadaemon(asabackgroundprocess).Thisparametercanonlybesetifthepid(processid)fileisspecified.Thatis,
unlessavaluetotheparameterpid-fileisgiven,thentheserverwillreportanerrorandexit.
DefaultLanguage
serverdefaultlanguageforsortingstrings --default-languagedefault-language
Thedefaultlanguageistusedforsortingandcomparingstrings.Thelanguagevalueisatwo-letterlanguagecode(ISO-639)oritis
composedbyatwo-letterlanguagecodewithandatwolettercountrycode(ISO-3166).Validlanguagesare"de","en","en_US"or
"en_UK".
Thedefaultdefault-languageissettobethesystemlocaleonthatplatform.
Supervisor
--supervisor
Executestheserverinsupervisormode.Intheeventthattheserverunexpectedlyterminatesduetoaninternalerror,thesupervisorwill
automaticallyrestarttheserver.Settingthisflagautomaticallyimpliesthattheserverwillrunasadaemon.Notethat,aswiththedaemon
flag,thisflagrequiresthatthepid-fileparameterwillset.
unix>./arangod--supervisor--pid-file/var/run/arangodb.pid/tmp/vocbase/
2012-06-27T15:58:28Z[10133]INFOstartingupinsupervisormode
ServerConfiguration
310
Ascanbeseen(e.g.byexecutingthepscommand),thiswillstartasupervisorprocessandtheactualdatabaseprocess:
unix>psfax|greparangod
10137?Ssl0:00./arangod--supervisor--pid-file/var/run/arangodb.pid/tmp/vocbase/
10142?Sl0:00\_./arangod--supervisor--pid-file/var/run/arangodb.pid/tmp/vocbase/
Whenthedatabaseprocessterminatesunexpectedly,thesupervisorprocesswillstartupanewdatabaseprocess:
>kill-SIGSEGV10142
>psfax|greparangod
10137?Ssl0:00./arangod--supervisor--pid-file/var/run/arangodb.pid/tmp/vocbase/
10168?Sl0:00\_./arangod--supervisor--pid-file/var/run/arangodb.pid/tmp/vocbase/
Useridentity
theuseridtousefortheprocess --uiduid
Thename(identity)oftheusertheserverwillrunas.Ifthisparameterisnotspecified,theserverwillnotattempttochangeitsUID,so
thattheUIDusedbytheserverwillbethesameastheUIDoftheuserwhostartedtheserver.Ifthisparameterisspecified,thenthe
serverwillchangeitsUIDafteropeningportsandreadingconfigurationfiles,butbeforeacceptingconnectionsoropeningotherfiles
(suchasrecoveryfiles).Thisisusefulwhentheservermustbestartedwithraisedprivileges(incertainenvironments)butsecurity
considerationsrequirethattheseprivilegesbedroppedoncetheserverhasstartedwork.
Observethatthisparametercannotbeusedtobypassoperatingsystemsecurity.Ingeneral,thisparameter(anditscorresponding
relativegid)canlowerprivilegesbutnotraisethem.
Groupidentity
thegroupidtousefortheprocess --gidgid
Thename(identity)ofthegrouptheserverwillrunas.Ifthisparameterisnotspecified,thentheserverwillnotattempttochangeits
GID,sothattheGIDtheserverrunsaswillbetheprimarygroupoftheuserwhostartedtheserver.Ifthisparameterisspecified,then
theserverwillchangeitsGIDafteropeningportsandreadingconfigurationfiles,butbeforeacceptingconnectionsoropeningotherfiles
(suchasrecoveryfiles).
Thisparameterisrelatedtotheparameteruid.
Processidentity
pidfile --pid-filefilename
ThenameoftheprocessIDfiletousewhenrunningtheserverasadaemon.Thisparametermustbespecifiedifeithertheflagdaemon
orsupervisorisset.
Console
--console
Runstheserverinanexclusiveemergencyconsolemode.Whenstartingtheserverwiththisoption,theserverisstartedwithan
interactiveJavaScriptemergencyconsole,withallnetworkingandHTTPinterfacesoftheserverdisabled.
Norequestscanbemadetotheserverinthismode,andtheonlywaytoworkwiththeserverinthismodeisbyusingtheemergency
console.Notethattheservercannotbestartedinthismodeifitisalreadyrunninginthisoranothermode.
RandomGenerator
randomnumbergeneratortouse --random.generatorarg
Theargumentisaninteger(1,2,3or4)whichsetsthemannerinwhichrandomnumbersaregenerated.Thedefaultmethod(3)istouse
theanon-blockingrandom(orpseudorandom)numbergeneratorsuppliedbytheoperatingsystem.
ServerConfiguration
311
Specifyinganargumentof2,usesablockingrandom(orpseudorandom)numbergenerator.Specifyinganargument1setsa
pseudorandomnumbergeneratorusinganimplicationoftheM ersenneTwisterM T19937algorithm.Algorithm4isacombinationofthe
blockingrandomnumbergeneratorandtheM ersenneTwister.
ServerConfiguration
312
Command-LineOptionsforarangod
Endpoint
endpointsforclientHTTPrequests --server.endpointendpointSpecifiesanendpointforHTTPrequestsbyclients.Endpointshave
thefollowingpattern:
tcp://ipv4-address:port-TCP/IPendpoint,usingIPv4
tcp://[ipv6-address]:port-TCP/IPendpoint,usingIPv6
ssl://ipv4-address:port-TCP/IPendpoint,usingIPv4,SSLencryption
ssl://[ipv6-address]:port-TCP/IPendpoint,usingIPv6,SSLencryption
unix:///path/to/socket-UnixdomainsocketendpointIfaTCP/IPendpointisspecifiedwithoutaportnumber,thenthedefaultport
(8529)willbeused.Ifmultipleendpointsneedtobeused,theoptioncanberepeatedmultipletimes.
Examples
unix>./arangod--server.endpointtcp://127.0.0.1:8529
--server.endpointssl://127.0.0.1:8530
--ssl.keyfileserver.pem/tmp/vocbase
2012-07-26T07:07:47Z[8161]INFOusingSSLprotocolversion'TLSv1'
2012-07-26T07:07:48Z[8161]INFOusingendpoint'ssl://127.0.0.1:8530'for
httpsslrequests
2012-07-26T07:07:48Z[8161]INFOusingendpoint'tcp://127.0.0.1:8529'for
httptcprequests
2012-07-26T07:07:49Z[8161]INFOArangoDB(version1.1.alpha)isreadyfor
business
2012-07-26T07:07:49Z[8161]INFOHaveFun!
Note:IfyouareusingSSL-encryptedendpoints,youmustalsosupplythepathtoaservercertificateusingthe --ssl.keyfileoption.
Endpointscanalsobechangedatruntime.PleaserefertoHTTPInterfaceforEndpointsformoredetails.
Reuseaddress
trytoreuseaddress --tcp.reuse-address
IfthisbooleanoptionissettotruethenthesocketoptionSO_REUSEADDRissetonallserverendpoints,whichisthedefault.Ifthis
optionissettofalseitispossiblethatittakesuptoaminuteafteraserverhasterminateduntilitispossibleforanewservertousethe
sameendpointagain.Thisiswhythisisactivatedbydefault.
Pleasenotehoweverthatundersomeoperatingsystemsthiscanbeasecurityriskbecauseitmightbepossibleforanotherprocessto
bindtothesameaddressandport,possiblyhijackingnetworktraffic.UnderWindows,ArangoDBadditionallysetstheflag
SO_EXCLUSIVEADDRUSEasameasuretoalleviatethisproblem.
Enable/disableauthentication
--server.authenticationSettingthisoptiontofalsewillturnoffauthenticationontheserversidesoallclientscanexecuteanyaction
withoutauthorizationandprivilegechecks.Thedefaultvalueistrue.
Enable/disableauthenticationforUNIXdomainsockets
disableauthenticationforrequestsviaUNIXdomainsockets --server.authentication-unix-socketsvalue
SettingvaluetotruewillturnoffauthenticationontheserversideforrequestscominginviaUNIXdomainsockets.Withthisflag
enabled,clientslocatedonthesamehostastheArangoDBservercanuseUNIXdomainsocketstoconnecttotheserverwithout
authentication.Requestscominginbyothermeans(e.g.TCP/IP)arenotaffectedbythisoption.
Thedefaultvalueisfalse.
Note:thisoptionisonlyavailableonplatformsthatsupportUNIXdomainsockets.
Arangodoptions
313
Enable/disableauthenticationforsystemAPIrequestsonly
--server.authentication-system-onlybooleanControlswhetherincomingrequestsneedauthenticationonlyiftheyaredirectedtothe
ArangoDB'sinternalAPIsandfeatures,locatedat/_api/,/_admin/etc.Iftheflagissettotrue,thenHTTPauthenticationisonlyrequired
forrequestsgoingtoURLsstartingwith/_,butnotforotherURLs.Theflagcanthusbeusedtoexposeauser-madeAPIwithoutHTTP
authenticationtotheoutsideworld,buttopreventtheoutsideworldfromusingtheArangoDBAPIandtheadmininterfacewithout
authentication.NotethatcheckingtheURLisperformedafteranydatabasenameprefixhasbeenremoved.Thatmeanswhentheactual
URLcalledis/_db/_system/myapp/myaction,theURL/myapp/myactionwillbeusedforauthentication-system-onlycheck.Thedefaultis
true.NotethatauthenticationstillneedstobeenabledfortheserverregularlyinorderforHTTPauthenticationtobeforcedforthe
ArangoDBAPIandthewebinterface.Settingonlythisflagisnotenough.YoucancontrolArangoDB'sgeneralauthenticationfeature
withthe--server.authenticationflag.
Enable/disablereplicationapplier
Enable/disablethereplicationapplieronserverstartup --database.replication-applierflag
Iffalsetheserverwillstartwithreplicationappliersturnedoff,evenifthereplicationappliersareconfiguredwiththeautoStartoption.
Usingthecommand-lineoptionwillnotchangethevalueoftheautoStartoptionintheapplierconfiguration,butwillsuppressautostartingthereplicationapplierjustonce.
Iftheoptionisnotused,ArangoDBwillreadtheapplierconfigurationfromthefileREPLICATION-APPLIER-CONFIGonstartup,and
usethevalueoftheautoStartattributefromthisfile.
Thedefaultistrue.
Keep-alivetimeout
timeoutforHTTPkeep-alive --http.keep-alive-timeoutAllowstospecifythetimeoutforHTTPkeep-aliveconnections.Thetimeout
valuemustbespecifiedinseconds.Idlekeep-aliveconnectionswillbeclosedbytheserverautomaticallywhenthetimeoutisreached.A
keep-alive-timeoutvalue0willdisablethekeepalivefeatureentirely.
HideProductheader
hidethe"Server:ArangoDB"headerinHTTPresponses --http.hide-product-header
Iftrue,theserverwillexcludetheHTTPheader"Server:ArangoDB"inHTTPresponses.Ifsettofalse,theserverwillsendtheheaderin
responses.
Thedefaultisfalse.
Allowmethodoverride
allowHTTPmethodoverrideviacustomheaders? --http.allow-method-override
Whenthisoptionissettotrue,theHTTPrequestmethodwilloptionallybefetchedfromoneofthefollowingHTTPrequestheadersif
presentintherequest:
x-http-method
x-http-method-override
x-method-override
Iftheoptionissettotrueandanyoftheseheadersisset,therequestmethodwillbeoverriddenbythevalueoftheheader.Forexample,
thisallowsissuinganHTTPDELETErequestwhichtotheoutsideworldwilllooklikeanHTTPGETrequest.Thisallowsbypassing
proxiesandtoolsthatwillonlyletcertainrequesttypespass.
Settingthisoptiontotruemayimposeasecurityrisksoitshouldonlybeusedincontrolledenvironments.
Thedefaultvalueforthisoptionisfalse.
Serverthreads
numberofdispatcherthreads --server.threadsnumber
Arangodoptions
314
SpecifiesthenumberofthreadsthatarespawnedtohandleHTTPRESTrequests.
Keyfile
keyfilecontainingservercertificate --ssl.keyfilefilename
IfSSLencryptionisused,thisoptionmustbeusedtospecifythefilenameoftheserverprivatekey.ThefilemustbePEM formatted
andcontainboththecertificateandtheserver'sprivatekey.
Thefilespecifiedbyfilenameshouldhavethefollowingstructure:
#createprivatekeyinfile"server.key"
opensslgenrsa-des3-outserver.key1024
#createcertificatesigningrequest(csr)infile"server.csr"
opensslreq-new-keyserver.key-outserver.csr
#copyawayoriginalprivatekeyto"server.key.org"
cpserver.keyserver.key.org
#removepassphrasefromtheprivatekey
opensslrsa-inserver.key.org-outserver.key
#signthecsrwiththekey,createscertificatePEMfile"server.crt"
opensslx509-req-days365-inserver.csr-signkeyserver.key-out
server.crt
#combinecertificateandkeyintosinglePEMfile"server.pem"
catserver.crtserver.key>server.pem
YoumayusecertificatesissuedbyaCertificateAuthorityorself-signedcertificates.Self-signedcertificatescanbecreatedbyatoolof
yourchoice.WhenusingOpenSSLforcreatingtheself-signedcertificate,thefollowingcommandsshouldcreateavalidkeyfile:
-----BEGINCERTIFICATE----(base64encodedcertificate)
-----ENDCERTIFICATE---------BEGINRSAPRIVATEKEY----(base64encodedprivatekey)
-----ENDRSAPRIVATEKEY-----
Forfurtherinformationpleasecheckthemanualsofthetoolsyouusetocreatethecertificate.
Note:the--ssl.keyfileoptionmustbesetiftheserverisstartedwithatleastoneSSLendpoint.
Cafile
CAfile --ssl.cafilefilename
ThisoptioncanbeusedtospecifyafilewithCAcertificatesthataresenttotheclientwhenevertheserverrequestsaclientcertificate.If
thefileisspecified,TheserverwillonlyacceptclientrequestswithcertificatesissuedbytheseCAs.Donotspecifythisoptionifyou
wantclientstobeabletoconnectwithoutspecificcertificates.
ThecertificatesinfilenamemustbePEM formatted.
Note:thisoptionisonlyrelevantifatleastoneSSLendpointisused.
SSLprotocol
SSLprotocoltypetouse --ssl.protocolvalue
Usethisoptiontospecifythedefaultencryptionprotocoltobeused.Thefollowingvariantsareavailable:
1:SSLv2
Arangodoptions
315
2:SSLv23
3:SSLv3
4:TLSv1
5:TLSv1.2(recommended)
Thedefaultvalueis4(i.e.TLSv1).Ifavailable,setitto5(i.e.TLSv1.2),becauselowerprotocolversionsareknowntobevulnerableto
POODLEattackvariants.
SSLcache
whetherornottouseSSLsessioncaching --ssl.session-cachevalue
SettotrueifSSLsessioncachingshouldbeused.
valuehasadefaultvalueoffalse(i.e.nocaching).
Note:thisoptionisonlyrelevantifatleastoneSSLendpointisused,andonlyiftheclientsupportssendingthesessionid.
SSLoptions
ssloptionstouse --ssl.optionsvalue
ThisoptioncanbeusedtosetvariousSSL-relatedoptions.IndividualoptionvaluesmustbecombinedusingbitwiseOR.
WhichoptionsareavailableonyourplatformisdeterminedbytheOpenSSLversionyouuse.Thelistofoptionsavailableonyour
platformmightberetrievedbythefollowingshellcommand:
>grep"#defineSSL_OP_.*"/usr/include/openssl/ssl.h
#defineSSL_OP_MICROSOFT_SESS_ID_BUG0x00000001L
#defineSSL_OP_NETSCAPE_CHALLENGE_BUG0x00000002L
#defineSSL_OP_LEGACY_SERVER_CONNECT0x00000004L
#defineSSL_OP_NETSCAPE_REUSE_CIPHER_CHANGE_BUG0x00000008L
#defineSSL_OP_SSLREF2_REUSE_CERT_TYPE_BUG0x00000010L
#defineSSL_OP_MICROSOFT_BIG_SSLV3_BUFFER0x00000020L
...
AdescriptionoftheoptionscanbefoundonlineintheOpenSSLdocumentation
SSLcipher
sslcipherlisttouse --ssl.cipher-listcipher-list
ThisoptioncanbeusedtorestricttheservertocertainSSLciphersonly,andtodefinetherelativeusagepreferenceofSSLciphers.
Theformatofcipher-listisdocumentedintheOpenSSLdocumentation.
Tocheckwhichciphersareavailableonyourplatform,youmayusethefollowingshellcommand:
>opensslciphers-v
ECDHE-RSA-AES256-SHASSLv3Kx=ECDHAu=RSAEnc=AES(256)Mac=SHA1
ECDHE-ECDSA-AES256-SHASSLv3Kx=ECDHAu=ECDSAEnc=AES(256)Mac=SHA1
DHE-RSA-AES256-SHASSLv3Kx=DHAu=RSAEnc=AES(256)Mac=SHA1
DHE-DSS-AES256-SHASSLv3Kx=DHAu=DSSEnc=AES(256)Mac=SHA1
DHE-RSA-CAMELLIA256-SHASSLv3Kx=DHAu=RSAEnc=Camellia(256)
Mac=SHA1
...
Thedefaultvalueforcipher-listis"ALL".
Arangodoptions
316
Backlogsize
listenbacklogsize --tcp.backlog-size
AllowstospecifythesizeofthebacklogforthelistensystemcallThedefaultvalueis10.Themaximumvalueisplatform-dependent.
Specifyingahighervaluethandefinedinthesystemheader'sSOM AXCONNmayresultinawarningonserverstart.Theactualvalue
usedbylistenmayalsobesilentlytruncatedonsomeplatforms(thishappensinsidethelistensystemcall).
Togglingserverstatistics
--server.statisticsvalue
Ifthisoptionisvalueisfalse,thenArangoDB'sstatisticsgatheringisturnedoff.StatisticsgatheringcausesregularCPUactivitysousing
thisoptiontoturnitoffmightrelieveheavy-loadedinstancesabit.
Sessiontimeout
timetoliveforserversessions --server.session-timeoutvalue
Thetimeoutforwebinterfacesessions,usingforauthenticatingrequeststothewebinterface(/_admin/aardvark)andrelatedareas.
Sessionsareonlyusedwhenauthenticationisturnedon.
Foxxqueues
enableordisabletheFoxxqueuesfeature --foxx.queuesflagIftrue,theFoxxqueueswillbeavailableandjobsinthequeueswillbe
executedasynchronously.Thedefaultistrue.Whensetto falsethequeuemanagerwillbedisabledandanyjobsarepreventedfrom
beingprocessed,whichmayreduceCPUloadabit.
Foxxqueuespollinterval
pollintervalforFoxxqueues --foxx.queues-poll-intervalvalueThepollintervalfortheFoxxqueuesmanager.Thevalueisspecifiedin
seconds.LowervalueswillmeanmoreimmediateandmorefrequentFoxxqueuejobexecution,butwillmakethequeuethreadwakeup
andquerythequeuesmoreoften.Whensettoalowvalue,thequeuethreadmightcauseCPUload.Thedefaultis1second.IfFoxx
queuesarenotusedmuch,thenthisvaluemaybeincreasedtomakethequeuesthreadwakeupless.
Directory
pathtothedatabase --database.directorydirectory
Thedirectorycontainingthecollectionsanddatafiles.Defaultsto/var/lib/arango.Whenspecifyingthedatabasedirectory,pleasemake
surethedirectoryisactuallywritablebythearangodprocess.
YoushouldfurthernotuseadatabasedirectorywhichisprovidedbyanetworkfilesystemsuchasNFS.Thereasonisthatnetworked
filesystemsmightcauseinconsistencieswhentherearemultipleparallelreadersorwritersortheylackfeaturesrequiredbyarangod(e.g.
flock()).
directory
Whenusingthecommandlineversion,youcansimplysupplythedatabasedirectoryasargument.
Examples
>./arangod--server.endpointtcp://127.0.0.1:8529--database.directory
/tmp/vocbase
Journalsize
--database.maximal-journal-sizesizeM aximalsizeofjournalinbytes.Canbeoverwrittenwhencreatinganewcollection.Notethat
thisalsolimitsthemaximalsizeofasingledocument.Thedefaultis32MB.
Waitforsync
Arangodoptions
317
defaultwaitforsyncbehavior --database.wait-for-syncbooleanDefaultwait-for-syncvalue.Canbeoverwrittenwhencreatinganew
collection.Thedefaultisfalse.
Forcesyncingofproperties
forcesyncingofcollectionpropertiestodisk --database.force-sync-propertiesbooleanForcesyncingofcollectionpropertiestodisk
aftercreatingacollectionorupdatingitsproperties.Ifturnedoff,nofsyncwillhappenforthecollectionanddatabasepropertiesstored
in parameter.jsonfilesinthefilesystem.Turningoffthisoptionwillspeedupworkloadsthatcreateanddropalotofcollections(e.g.
testsuites).Thedefaultistrue.
Enable/disableAQLquerytracking
disablethequerytrackingfeature --query.trackingflag
Iftrue,theserver'sAQLslowquerytrackingfeaturewillbeenabledbydefault.Trackingofqueriescanbedisabledbysettingtheoption
tofalse.
Thedefaultistrue.
ThresholdforslowAQLqueries
adjustthethresholdforslowAQLqueries --query.slow-thresholdvalue
BysettingvalueitcanbecontrolledafterwhatexecutiontimeanAQLqueryisconsidered"slow".Anyslowqueriesthatexceedthe
executiontimespecifiedinvaluewillbeloggedwhentheyarefinished.Thethresholdvalueisspecifiedinseconds.Trackingofslow
queriescanbeturnedoffentirelybysettingtheoption --query.trackingtofalse.
Thedefaultvalueis10.0.
Throwcollectionnotloadederror
throwcollectionnotloadederror --database.throw-collection-not-loaded-errorflag
Accessinganot-yetloadedcollectionwillautomaticallyloadacollectiononfirstaccess.Thisflagcontrolswhathappensincasean
operationwouldneedtowaitforanotherthreadtofinalizeloadingacollection.Ifsettotrue,thenthefirstoperationthataccessesan
unloadedcollectionwillloadit.Furtherthreadsthattrytoaccessthesamecollectionwhileitisstillloadingwillgetanerror(1238,
collectionnotloaded).Whentheinitialoperationhascompletedloadingthecollection,alloperationsonthecollectioncanbecarriedout
normally,anderror1238willnotbethrown.
Ifsettofalse,thefirstthreadthataccessesanot-yetloadedcollectionwillstillloadit.Otherthreadsthattrytoaccessthecollection
whileloadingwillnotfailwitherror1238butinsteadblockuntilthecollectionisfullyloaded.Thisconfigurationmightleadtoallserver
threadsbeingblockedbecausetheyareallwaitingforthesamecollectiontocompleteloading.Settingtheoptiontotruewillpreventthis
fromhappening,butrequiresclientstocatcherror1238andreactonit(maybebyschedulingaretryforlater).
Thedefaultvalueisfalse.
AQLQuerycachingmode
whetherornottoenabletheAQLqueryresultcache --query.cache-mode
TogglestheAQLquerycachebehavior.Possiblevaluesare:
off:donotusequerycache
on:alwaysusequerycache,exceptforqueriesthathavetheircacheattributesettofalse
demand:usequerycacheonlyforqueriesthathavetheircacheattributesettotrueset
AQLQuerycachesize
maximumnumberofelementsinthequerycacheperdatabase --query.cache-entries
M aximumnumberofqueryresultsthatcanbestoredperdatabase-specificquerycache.Ifaqueryiseligibleforcachingandthenumber
ofitemsinthedatabase'squerycacheisequaltothisthresholdvalue,anothercachedqueryresultwillberemovedfromthecache.
Arangodoptions
318
Thisoptiononlyhasaneffectifthequerycachemodeissettoeitheronordemand.
Indexthreads
numberofbackgroundthreadsforparallelindexcreation --database.index-threads
Specifiesthenumberofbackgroundthreadsforindexcreation.Whenacollectioncontainsextraindexesotherthantheprimaryindex,
theseotherindexescanbebuiltbymultiplethreadsinparallel.Theindexthreadsaresharedamongmultiplecollectionsanddatabases.
Specifyingavalueof0willturnoffparallelbuilding,meaningthatindexesforeachcollectionarebuiltsequentiallybythethreadthat
openedthecollection.Ifthenumberofindexthreadsisgreaterthan1,itwillalsobeusedtobuilttheedgeindexofacollectioninparallel
(thisalsorequirestheedgeindexinthecollectiontobesplitintomultiplebuckets).
V8contexts
numberofV8contextsforexecutingJavaScriptactions --javascript.v8-contextsnumber
SpecifiesthenumberofV8contextsthatarecreatedforexecutingJavaScriptcode.M orecontextsallowexecutemoreJavaScriptactions
inparallel,providedthattherearealsoenoughthreadsavailable.PleasenotethateachV8contextwilluseasubstantialamountof
memoryandrequiresperiodicCPUprocessingtimeforgarbagecollection.
Garbagecollectionfrequency(time-based)
JavaScriptgarbagecollectionfrequency(eachxseconds) --javascript.gc-frequencyfrequency
Specifiesthefrequency(inseconds)fortheautomaticgarbagecollectionofJavaScriptobjects.Thissettingisusefultohavethegarbage
collectionstillworkinperiodswithnoorlittlenumbersofrequests.
Garbagecollectioninterval(request-based)
JavaScriptgarbagecollectioninterval(eachxrequests) --javascript.gc-intervalinterval
Specifiestheinterval(approximatelyinnumberofrequests)thatthegarbagecollectionforJavaScriptobjectswillberunineachthread.
V8options
optionalargumentstopasstov8 --javascript.v8-optionsoptions
OptionalargumentstopasstotheV8Javascriptengine.TheV8enginewillrunwithdefaultsettingsunlessexplicitoptionsarespecified
usingthisoption.TheoptionspassedwillbeforwardedtotheV8enginewhichwillparsethemonitsown.Passinginvalidoptionsmay
resultinanerrorbeingprintedonstderrandtheoptionbeingignored.
Optionsneedtobepassedinonestring,withV8optionnamesbeingprefixedwithdoubledashes.M ultipleoptionsneedtobeseparated
bywhitespace.TogetalistofallavailableV8options,youcanusethevalue"--help"asfollows:
--javascript.v8-options"--help"
AnotherexampleofspecificV8optionsbeingsetatstartup:
--javascript.v8-options"--log"
NamesandfeaturesorusableoptionsdependontheversionofV8beingused,andmightchangeinthefutureifadifferentversionofV8
isbeingusedinArangoDB.NotalloptionsofferedbyV8mightbesensibletouseinthecontextofArangoDB.Usethespecificoptions
onlyifyouaresurethattheyarenotharmfulfortheregulardatabaseoperation.
Arangodoptions
319
SinceArangoDB2.2,theserverwillwritealldata-modificationoperationsintoitswrite-aheadlog.
Thewrite-aheadlogisasequenceoflogfilesthatarewritteninanappend-onlyfashion.Fulllogfileswilleventuallybegarbage-collected,
andtherelevantdatamightbetransferredintocollectionjournalsanddatafiles.Unneededandalreadygarbage-collectedlogfileswilleither
bedeletedorkeptforthepurposeofkeepingareplicationbacklog.
Directory
TheWALlogfilesdirectory: --wal.directory
Specifiesthedirectoryinwhichthewrite-aheadlogfilesshouldbestored.Ifthisoptionisnotspecified,itdefaultstothesubdirectory
journalsintheserver'sglobaldatabasedirectory.Ifthedirectoryisnotpresent,itwillbecreated.
Logfilesize
thesizeofeachWALlogfile --wal.logfile-sizeSpecifiesthefilesize(inbytes)foreachwrite-aheadlogfile.Thelogfilesizeshouldbe
chosensothateachlogfilecanstoreaconsiderableamountofdocuments.Thebiggerthelogfilesizeischosen,thelongeritwilltaketofill
upasinglelogfile,whichalsoinfluencesthedelayuntilthedatainalogfilewillbegarbage-collectedandwrittentocollectionjournalsand
datafiles.Italsoaffectshowlonglogfilerecoverywilltakeatserverstart.
Allowoversizeentries
whetherornotoversizeentriesareallowed --wal.allow-oversize-entriesWhetherornotitisallowedtostoreindividualdocuments
thatarebiggerthanwouldfitintoasinglelogfile.Settingtheoptiontofalsewillmakesuchoperationsfailwithanerror.Settingthe
optiontotruewillmakesuchoperationssucceed,butwithahighpotentialperformanceimpact.Thereasonisthatforeachoversize
operation,anindividualoversizelogfileneedstobecreatedwhichmayalsoblockotheroperations.Theoptionshouldbesettofalseifit
iscertainthatdocumentswillalwayshaveasizesmallerthanasinglelogfile.
Numberofreservelogfiles
maximumnumberofreservelogfiles --wal.reserve-logfilesThemaximumnumberofreservelogfilesthatArangoDBwillcreateina
backgroundprocess.Reservelogfilesareusefulinthesituationwhenanoperationneedstobewrittentoalogfilebutthereservespacein
thelogfileistoolowforstoringtheoperation.Inthiscase,anewlogfileneedstobecreatedtostoretheoperation.Creatingnewlogfiles
isnormallyslow,soArangoDBwilltrytopre-createlogfilesinabackgroundprocesssotherearealwaysreservelogfileswhentheactive
logfilegetsfull.ThenumberofreservelogfilesthatArangoDBkeepsinthebackgroundisconfigurablewiththisoption.
Numberofhistoriclogfiles
maximumnumberofhistoriclogfiles --wal.historic-logfilesThemaximumnumberofhistoriclogfilesthatArangoDBwillkeepafter
theyhavebeengarbage-collected.Ifnoreplicationisused,thereisnoneedtokeephistoriclogfilesexceptforhavingalocalchangelog.In
areplicationsetup,thenumberofhistoriclogfilesaffectstheamountofdataaslavecanfetchfromthemaster'slogs.Themorehistoric
logfiles,themorehistoricdataisavailableforaslave,whichisusefuliftheconnectionbetweenmasterandslaveisunstableorslow.Not
havingenoughhistoriclogfilesavailablemightleadtologfiledatabeingdeletedonthemasteralreadybeforeaslavehasfetchedit.
Syncinterval
intervalforautomatic,non-requesteddisksyncs --wal.sync-intervalTheinterval(inmilliseconds)thatArangoDBwilluseto
automaticallysynchronizedatainitswrite-aheadlogstodisk.Automaticsyncswillonlybeperformedfornot-yetsynchronizeddata,
andonlyforoperationsthathavebeenexecutedwithoutthewaitForSyncattribute.
Throttling
ThrottlewritestoWALwhenatleastsuchmanyoperationsarewaitingforgarbagecollection: --wal.throttle-when-pending
320
Themaximumvalueforthenumberofwrite-aheadloggarbage-collectionqueueelements.Ifsetto0,thequeuesizeisunbounded,andno
write-throttlingwilloccur.Ifsettoanon-zerovalue,write-throttlingwillautomaticallykickinwhenthegarbage-collectionqueue
containsatleastasmanyelementsasspecifiedbythisoption.Whilewrite-throttlingisactive,data-modificationoperationswill
intentionallybedelayedbyaconfigurableamountoftime.Thisistoensurethewrite-aheadloggarbagecollectorcancatchupwiththe
operationsexecuted.Write-throttlingwillstayactiveuntilthegarbage-collectionqueuesizegoesdownbelowthespecifiedvalue.Writethrottlingisturnedoffbydefault.
--wal.throttle-wait
Thisoptiondeterminesthemaximumwaittime(inmilliseconds)foroperationsthatarewrite-throttled.Ifwrite-throttlingisactiveanda
newwriteoperationistobeexecuted,itwillwaitforatmostthespecifiedamountoftimeforthewrite-aheadloggarbage-collection
queuesizetofallbelowthethrottlingthreshold.Ifthequeuesizedecreasesbeforethemaximumwaittimeisover,theoperationwillbe
executednormally.Ifthequeuesizedoesnotdecreasebeforethewaittimeisover,theoperationwillbeabortedwithanerror.This
optiononlyhasaneffectif --wal.throttle-when-pendinghasanon-zerovalue,whichisnotthedefault.
Numberofslots
M aximumnumberofslotstobeusedinparallel: --wal.slots
Configurestheamountofwriteslotsthewrite-aheadlogcangivetowriteoperationsinparallel.Anywriteoperationwillleaseaslotand
returnittothewrite-aheadlogwhenitisfinishedwritingthedata.Aslotwillremainblockeduntilthedatainitwassynchronizedto
disk.Afterthat,aslotbecomesreusablebyfollowingoperations.Therequirednumberofslotsisthusdeterminedbytheparallelityof
writeoperationsandthedisksynchronizationspeed.Slowdisksprobablyneedhighervalues,andfastdisksmayonlyrequireavalue
lowerthanthedefault.
Ignorelogfileerrors
Ignorelogfileerrorswhenopeninglogfiles: --wal.ignore-logfile-errors
Ignoresanyrecoveryerrorscausedbycorruptedlogfilesonstartup.Whensettofalse,therecoveryprocedureonstartupwillfailwithan
errorwheneveritencountersacorrupted(thatincludesonlyhalf-written)logfile.Thisisasecurityprecautiontopreventdatalossincase
ofdiskerrorsetc.Whentherecoveryprocedureabortsbecauseofcorruption,anycorruptedfilescanbeinspectedandfixed(orremoved)
manuallyandtheservercanberestartedafterwards.
Settingtheoptiontotruewillmaketheservercontinuewiththerecoveryprocedureevenincaseitdetectscorruptlogfileentries.Inthis
caseitwillstopatthefirstcorruptedlogfileentryandignoreallothers,whichmightcausedataloss.
Ignorerecoveryerrors
Ignorerecoveryerrors: --wal.ignore-recovery-errors
Ignoresanyrecoveryerrorsnotcausedbycorruptedlogfilesbutbylogicalerrors.Logicalerrorscanoccuriflogfilesoranyotherserver
datafileshavebeenmanuallyeditedortheserverissomehowmisconfigured.
Ignore(non-WAL)datafileerrors
Ignoredatafileerrorswhenloadingcollections: --database.ignore-datafile-errorsboolean
Ifsetto false,CRCmismatchandothererrorsincollectiondatafileswillleadtoacollectionnotbeingloadedatall.Thecollectionin
thiscasebecomesunavailable.IfsuchcollectionneedstobeloadedduringWALrecovery,theWALrecoverywillalsoabort(ifnotforced
withoption --wal.ignore-recovery-errorstrue).
Settingthisflagto falseprotectsusersfromunintentionallyusingacollectionwithcorrupteddatafiles,fromwhichonlyasubsetof
theoriginaldatacanberecovered.Workingwithsuchcollectioncouldleadtodatalossandfollowuperrors.Inordertoaccesssuch
collection,itisrequiredtoinspectandrepairthecollectiondatafilewiththedatafiledebugger(arango-dfdb).
Ifsetto true,CRCmismatchandothererrorsduringtheloadingofacollectionwillleadtothedatafilebeingpartiallyloaded,uptothe
positionofthefirsterror.Alldatauptountiltheinvalidpositionwillbeloaded.Thiswillenableuserstocontinuewithcollection
datafileseveniftheyarecorrupted,butthiswillresultinonlyapartialloadoftheoriginaldataandpotentialfollowuperrors.TheWAL
recoverywillstillabortwhenencounteringacollectionwithacorrupteddatafile,atleastif --wal.ignore-recovery-errorsisnotsetto
true.
321
Thedefaultvalueisfalse,socollectionswithcorrupteddatafileswillnotbeloadedatall,preventingpartialloadsandfollowuperrors.
However,ifsuchcollectionisrequiredatserverstartup,duringWALrecovery,theserverwillaborttherecoveryandrefusetostart.
322
JavaScriptInterfaceformanagingEndpoints
TheArangoDBservercanlistenforincomingrequestsonmultipleendpoints.
TheendpointsarenormallyspecifiedeitherinArangoDB'sconfigurationfileoronthecommand-line,usingthe"--server.endpoint"
option.ThedefaultendpointforArangoDBistcp://127.0.0.1:8529ortcp://localhost:8529.ArangoDBcanalsodoasocalledbroadcast
bindusingtcp://0.0.0.0:8529.Thiswayitwillbereachableonallinterfacesofthehost.Thismaybeusefulondevelopmentsystemsthat
frequentlychangetheirnetworksetuplikelaptops.
Eachendpointcanoptionallyberestrictedtoaspecificlistofdatabasesonly,thusallowingtheusageofdifferentportnumbersfor
differentdatabases.
Thismaybeusefulinmulti-tenantsetups.Amulti-endpointsetupmayalsobeusefultoturnonencryptedcommunicationforjust
specificdatabases.
EndpointsequalTCPportstobebound.Ononespecificethernetinterfaceeachportcanonlybeboundexactlyonce.Youcanlookup
youravailableinterfacesusingtheifconfigcommandonLinux/M acOSX-theWindowsequivalentisipconfig(SeeWikipediaformore
details).ThegeneralnamesoftheinterfacesdifferonOS'sandhardwarestheyrunon.However,typicallyeveryhosthasasocalled
loopbackinterface,whichisavirtualinterface.Byconventionitalwayshastheaddress127.0.0.1or::1(ipv6),andcanonlybereached
fromexactlytheverysamehost.Ethernetinterfacesusuallyhavenameslikeeth0,wlan0,eth1:17,le0oraplaintextnameinWindows.
Tofindoutwhichservicesalreadyuseports(soArangoDBcan'tbindthemanymore),youcanusethenetstatcommand(itbehavesa
littledifferentoneachplatform,runitwith-lnptonLinux,-ptcponM acOSXorwith-anonwindowsforvaluableinformation).
TheJavaScriptinterfaceforendpointsprovidesanoperationforretrievingthelistofendpoints.Thereisnofunctionalitytoadd,remove
orreconfigureanexistingendpointviatheAPIafterserverstart.
Pleasenotethattheoperationtoretrievethelistofendpointscanonlybecarriedoutinthedefaultdatabase(_system)andnoneofthe
otherdatabases.Whennotinthedefaultdatabase,youmustfirstswitchtoitusingthedb._useDatabasemethod.
List
Returnsalistofallendpointsandtheirmappeddatabases: db._endpoints()
Pleasenotethatmanagingendpointscanonlybeperformedfromoutofthe_systemdatabase.Whennotinthedefaultdatabase,you
mustfirstswitchtoitusingthe"db._useDatabase"method.
M anagingEndpoints
323
Command-LineOptionsforClusters
NodeID
Thisserver'sid: --cluster.my-local-infoinfo
Somelocalinformationabouttheserverinthecluster,thiscanforexamplebeanIPaddresswithaprocessIDoranystringuniquetothe
server.Specifyinginfoismandatoryonstartupiftheserverid(seebelow)isnotspecified.Eachserveroftheclustermusthaveaunique
localinfo.Thisisignoredifmy-idbelowisspecified.
Agencyendpoint
Listofagencyendpoints: --cluster.agency-endpointendpoint
Anagencyendpointtheservercanconnectto.Theoptioncanbespecifiedmultipletimes,sotheservercanuseaclusterofagency
servers.Endpointshavethefollowingpattern:
AtleastoneendpointmustbespecifiedorArangoDBwillrefusetostart.Itisrecommendedtospecifyatleasttwoendpointsso
ArangoDBhasanalternativeendpointifoneofthembecomesunavailable.
Examples
--cluster.agency-endpointtcp://192.168.1.1:4001--cluster.agency-endpoint
tcp://192.168.1.2:4002
Agencyprefix
Globalagencyprefix: --cluster.agency-prefixprefix
Theglobalkeyprefixusedinallrequeststotheagency.Thespecifiedprefixwillbecomepartofeachagencykey.Specifyingthekey
prefixallowsmanagingmultipleArangoDBclusterswiththesameagencyserver(s).
prefixmustconsistofthelettersa-z,A-Zandthedigits0-9only.Specifyingaprefixismandatory.
Examples
--cluster.prefixmycluster
MyId
Thisserver'sid: --cluster.my-idid
Thelocalserver'sidinthecluster.Specifyingidismandatoryonstartup.Eachserveroftheclustermusthaveauniqueid.
Specifyingtheidisveryimportantbecausetheserveridisusedfordeterminingtheserver'sroleandtasksinthecluster.
idmustbeastringconsistingofthelettersa-z,A-Zorthedigits0-9only.
MyAddress
Thisserver'saddress/endpoint: --cluster.my-addressendpoint
Theserver'sendpointforcluster-internalcommunication.Ifspecified,itmusthavethefollowingpattern:
Clusteroptions
324
Ifnoendpointisspecified,theserverwilllookupitsinternalendpointaddressintheagency.Ifnoendpointcanbefoundintheagency
fortheserver'sid,ArangoDBwillrefusetostart.
Examples
--cluster.my-addresstcp://192.168.1.1:8530
Username
Usernameusedforcluster-internalcommunication: --cluster.usernameusername
Theusernameusedforauthorizationofcluster-internalrequests.Thisusernamewillbeusedtoauthenticateallrequestsandresponsesin
cluster-internalcommunication,i.e.requestsexchangedbetweencoordinatorsandindividualdatabaseservers.
Thisoptionisusedforcluster-internalrequestsonly.Regularrequeststocoordinatorsareauthenticatednormallyusingthedatainthe
_userscollection.
Ifcoordinatorsanddatabaseserversarerunwithauthenticationturnedoff,(e.g.bysettingthe--server.authenticationoptiontofalse),the
cluster-internalcommunicationwillalsobeunauthenticated.
Password
Passwordusedforcluster-internalcommunication: --cluster.passwordpassword
Thepasswordusedforauthorizationofcluster-internalrequests.Thispasswordwillbeusedtoauthenticateallrequestsandresponses
incluster-internalcommunication,i.e.requestsexchangedbetweencoordinatorsandindividualdatabaseservers.
Thisoptionisusedforcluster-internalrequestsonly.Regularrequeststocoordinatorsareauthenticatednormallyusingthedatainthe
_userscollection.
Ifcoordinatorsanddatabaseserversarerunwithauthenticationturnedoff,(e.g.bysettingthe--server.authenticationoptiontofalse),the
cluster-internalcommunicationwillalsobeunauthenticated.
Clusteroptions
325
Command-LineOptionsforLogging
Loglevelsandtopics
ArangoDB'slogoutputisgroupedintotopics. --log.levelcanbespecifiedmultipletimesatstartup,forasmanytopicsasneeded.
Thelogverbosityandoutputfilescanbeadjustedperlogtopic.Forexample
--log.levelstartup=trace--log.levelqueries=trace--log.levelinfo
willlogmessagesconcerningstartupattracelevel,AQLqueriesattracelevelandeverythingelseatinfolevel.
Inaconfigurationfile,itiswrittenlikethis:
[log]
level=startup=trace
level=queries=trace
level=info
Notethattheremustnotbeanywhitespacearoundthesecond =.
Theavailableloglevelsare:
fatal:onlylogsfatalerrors
error:onlylogserrors
warning:onlylogswarningsanderrors
info:logsinformationmessages,warningsanderrors
debug:logsdebugandinformationmessages,warningsanderrors
trace:logstrace,debugandinformationmessages,warningsanderrors
Notethatlevels debugand tracewillbeveryverbose.
Somerelevantlogtopicsavailablein3.0are:
collector:informationabouttheWALcollector'sstate
compactor:informationaboutthecollectiondatafilecompactor
datafiles:datafile-relatedoperations
mmap:informationaboutmemory-mappingoperations(includingmsync)
performance:performance-releatedmessages
queries:executedAQLqueries,slowqueries
replication:replication-relatedinfo
requests:HTTPrequests
startup:informationaboutserverstartupandshutdown
threads:informationaboutthreads
Theoldoption --log.performanceisstillavailablein3.0.Itisnowashortcutforthemoregeneraloption --log.level
performance=trace.
Logoutputs
Thelogoption --log.output<definition>allowsdirectingtheglobalorper-topiclogoutputtodifferentoutputs.Theoutputdefinition
<definition>canbeoneof
-forstdin
+forstderr
syslog://<syslog-facility>
syslog://<syslog-facility>/<application-name>
file://<relative-path>
Loggingoptions
326
Theoptioncanbespecifiedmultipletimesinordertoconfiguretheoutputfordifferentlogtopics.Tosetupaper-topicoutput
configuration,use --log.output<topic>=<definition>,e.g.
queries=file://queries.txt
logsallqueriestothefile"queries.txt".
Theoldoption --log.fileisstillavailablein3.0forconveniencereasons.In3.0itisashortcutforthemoregeneraloption -log.outputfile://filename.
Theoldoption --log.requests-fileisstillavailablein3.0.Itisnowashortcutforthemoregeneraloption --log.output
requests=file://....
Using --log.outputalsoallowsdirectinglogoutputtodifferentfilesbasedontopics.Forexample,tologallAQLqueriestoafile
"queries.log"onecanusetheoptions:
--log.levelqueries=trace--log.outputqueries=file:///path/to/queries.log
ToadditionallylogHTTPrequesttoafilenamed"requests.log"addtheoptions:
--log.levelrequests=info--log.outputrequests=file:///path/to/requests.log
Forcingdirectoutput
Theoption --log.force-directcanbeusedtodisablelogginginanextraloggingthread.Ifsetto true,anylogmessagesare
immediatelyprintedinthethreadthattriggeredthelogmessage.Thisisnon-optimalforperformancebutcanaiddebugging.Ifsetto
false,logmessagesarehandedofftoanextraloggingthread,whichasynchronouslywritesthelogmessages.
Localtime
Logdatesandtimesinlocaltimezone: --log.use-local-time
Ifspecified,alldatesandtimesinlogmessageswillusetheserver'slocaltime-zone.Ifnotspecified,alldatesandtimesinlogmessages
willbeprintedinUTC/Zulutime.Thedateandtimeformatusedinlogsisalways YYYY-MM-DDHH:MM:SS,regardlessofthissetting.If
UTCtimeisused,a ZwillbeappendedtoindicateZulutime.
Linenumber
Loglinenumber: --log.line-number
Normally,ifanhumanreadablefatal,error,warningorinfomessageislogged,noinformationaboutthefileandlinenumberisprovided.
Thefileandlinenumberisonlyloggedfordebugandtracemessage.Thisoptioncanbeusetoalwayslogthesepiecesofinformation.
Prefix
Logprefix: --log.prefixprefix
Thisoptionisusedspecifyanprefixtologgedtext.
Thread
Logthreadidentifier: --log.thread
Wheneverlogoutputisgenerated,theprocessIDiswrittenaspartoftheloginformation.Settingthisoptionappendsthethreadidof
thecallingthreadtotheprocessid.Forexample,
2010-09-20T13:04:01Z[19355]INFOreadyforbusiness
whennothreadisloggedand
Loggingoptions
327
2010-09-20T13:04:17Z[19371-18446744072487317056]readyforbusiness
whenthiscommandlineoptionisset.
Loggingoptions
328
CommandLineOptionsforCommunication
Schedulerthreads
Numberofschedulerthreads: --scheduler.threadsarg
AnintegerargumentwhichsetsthenumberofthreadstouseintheIOscheduler.Thedefaultis1.
Schedulermaximalqueuesize
M aximumsizeofthedispatcherqueueforasynchronousrequests: --server.maximal-queue-sizesize
Specifiesthemaximumsizeofthedispatcherqueueforasynchronoustaskexecution.Ifthequeuealreadycontainssizetasks,newtasks
willberejecteduntilothertasksarepoppedfromthequeue.Settingthisvaluemayhelppreventingfromrunningoutofmemoryifthe
queueisfilledupfasterthantheservercanprocessrequests.
Schedulerbackend
Schedulerbackend: --scheduler.backendarg
TheI/Omethodusedbytheeventhandler.Thedefault(ifthisoptionisnotspecified)istotryallrecommendedbackends.Thisis
platformspecific.Seelibevforfurtherdetailsandthemeaningofselect,pollandepoll.
Iobackends
--scheduler.show-backends
Ifthisoptionisspecified,thentheserverwilllistavailablebackendsandexit.Thisoptionisusefulonlywhenusedinconjunctionwith
theoptionscheduler.backend.Anintegerisreturned(whichisplatformdependent)whichindicatesavailablebackendsonyourplatform.
Seelibevforfurtherdetailsandforthemeaningoftheintegerreturned.Thisdescribestheallowedintegersforscheduler.backend,seehere
fordetails.
Communicationoptions
329
DurabilityConfiguration
GlobalConfiguration
Thereareglobalconfigurationvaluesfordurability,whichcanbeadjustedbyspecifyingthefollowingconfigurationoptions:
defaultwaitforsyncbehavior --database.wait-for-syncbooleanDefaultwait-for-syncvalue.Canbeoverwrittenwhencreatinganew
collection.Thedefaultisfalse.
forcesyncingofcollectionpropertiestodisk --database.force-sync-propertiesbooleanForcesyncingofcollectionpropertiestodisk
aftercreatingacollectionorupdatingitsproperties.Ifturnedoff,nofsyncwillhappenforthecollectionanddatabasepropertiesstored
in parameter.jsonfilesinthefilesystem.Turningoffthisoptionwillspeedupworkloadsthatcreateanddropalotofcollections(e.g.
testsuites).Thedefaultistrue.
intervalforautomatic,non-requesteddisksyncs --wal.sync-intervalTheinterval(inmilliseconds)thatArangoDBwilluseto
automaticallysynchronizedatainitswrite-aheadlogstodisk.Automaticsyncswillonlybeperformedfornot-yetsynchronizeddata,
andonlyforoperationsthathavebeenexecutedwithoutthewaitForSyncattribute.
Per-collectionconfiguration
Youcanalsoconfigurethedurabilitybehavioronaper-collectionbasis.UsetheArangoDBshelltochangetheseproperties.
getsorsetsthepropertiesofacollection collection.properties()Returnsanobjectcontainingallcollectionproperties.
isVolatile:IftruethenthecollectiondatawillbekeptinmemoryonlyandArangoDBwillnotwriteorsyncthedatatodisk.
keyOptions(optional)additionaloptionsforkeygeneration.ThisisaJSONarraycontainingthefollowingattributes(note:someof
theattributesareoptional):
type:thetypeofthekeygeneratorusedforthecollection.
indexBuckets:numberofbucketsintowhichindexesusingahashtablearesplit.Thedefaultis16andthisnumberhastobeapower
of2andlessthanorequalto1024.Forverylargecollectionsoneshouldincreasethistoavoidlongpauseswhenthehashtablehas
tobeinitiallybuiltorresized,sincebucketsareresizedindividuallyandcanbeinitiallybuiltinparallel.Forexample,64mightbea
sensiblevalueforacollectionwith100000000documents.Currently,onlytheedgeindexrespectsthisvalue,butotherindextypes
mightfollowinfutureArangoDBversions.Changes(seebelow)areappliedwhenthecollectionisloadedthenexttime.Inacluster
setup,theresultwillalsocontainthefollowingattributes:
numberOfShards:thenumberofshardsofthecollection.
shardKeys:containsthenamesofdocumentattributesthatareusedtodeterminethetargetshardfordocuments.
collection.properties(properties)Changesthecollectionproperties.propertiesmustbeaobjectwithoneormoreofthe
followingattribute(s):
indexBuckets:Seeabove,changesareonlyappliedwhenthecollectionisloadedthenexttime.Note:itisnotpossibletochangethe
journalsizeafterthejournalordatafilehasbeencreated.Changingthisparameterwillonlyeffectnewlycreatedjournals.Alsonote
thatyoucannotlowerthejournalsizetolessthensizeofthelargestdocumentalreadystoredinthecollection.Note:someother
collectionproperties,suchastype,isVolatile,orkeyOptionscannotbechangedoncethecollectioniscreated.
Examples
Readallproperties
Durability
330
arangosh>db.example.properties();
Changeaproperty
arangosh>db.example.properties({waitForSync:true});
Per-operationconfiguration
M anydata-modificationoperationsandalsoArangoDB'stransactionsallowtospecifyawaitForSyncattribute,whichwhensetensures
theoperationdatahasbeensynchronizedtodiskwhentheoperationreturns.
Disk-UsageConfiguration
TheamountofdiskspaceusedbyArangoDBisdeterminedbyafewconfigurationoptions.
GlobalConfiguration
ThetotalamountofdiskstoragerequiredbyArangoDBisdeterminedbythesizeofthewrite-aheadlogfilesplusthesizesofthe
collectionjournalsanddatafiles.
Therearethefollowingoptionsforconfiguringthenumberandsizesofthewrite-aheadlogfiles:
maximumnumberofreservelogfiles --wal.reserve-logfilesThemaximumnumberofreservelogfilesthatArangoDBwillcreateina
backgroundprocess.Reservelogfilesareusefulinthesituationwhenanoperationneedstobewrittentoalogfilebutthereservespacein
thelogfileistoolowforstoringtheoperation.Inthiscase,anewlogfileneedstobecreatedtostoretheoperation.Creatingnewlogfiles
isnormallyslow,soArangoDBwilltrytopre-createlogfilesinabackgroundprocesssotherearealwaysreservelogfileswhentheactive
logfilegetsfull.ThenumberofreservelogfilesthatArangoDBkeepsinthebackgroundisconfigurablewiththisoption.
maximumnumberofhistoriclogfiles --wal.historic-logfilesThemaximumnumberofhistoriclogfilesthatArangoDBwillkeepafter
theyhavebeengarbage-collected.Ifnoreplicationisused,thereisnoneedtokeephistoriclogfilesexceptforhavingalocalchangelog.In
areplicationsetup,thenumberofhistoriclogfilesaffectstheamountofdataaslavecanfetchfromthemaster'slogs.Themorehistoric
logfiles,themorehistoricdataisavailableforaslave,whichisusefuliftheconnectionbetweenmasterandslaveisunstableorslow.Not
havingenoughhistoriclogfilesavailablemightleadtologfiledatabeingdeletedonthemasteralreadybeforeaslavehasfetchedit.
thesizeofeachWALlogfile --wal.logfile-sizeSpecifiesthefilesize(inbytes)foreachwrite-aheadlogfile.Thelogfilesizeshouldbe
chosensothateachlogfilecanstoreaconsiderableamountofdocuments.Thebiggerthelogfilesizeischosen,thelongeritwilltaketofill
upasinglelogfile,whichalsoinfluencesthedelayuntilthedatainalogfilewillbegarbage-collectedandwrittentocollectionjournalsand
datafiles.Italsoaffectshowlonglogfilerecoverywilltakeatserverstart.
whetherornotoversizeentriesareallowed --wal.allow-oversize-entriesWhetherornotitisallowedtostoreindividualdocuments
thatarebiggerthanwouldfitintoasinglelogfile.Settingtheoptiontofalsewillmakesuchoperationsfailwithanerror.Settingthe
optiontotruewillmakesuchoperationssucceed,butwithahighpotentialperformanceimpact.Thereasonisthatforeachoversize
operation,anindividualoversizelogfileneedstobecreatedwhichmayalsoblockotheroperations.Theoptionshouldbesettofalseifit
iscertainthatdocumentswillalwayshaveasizesmallerthanasinglelogfile.Whendatagetscopiedfromthewrite-aheadlogfilesintothe
journalsordatafilesofcollections,fileswillbecreatedonthecollectionlevel.Howbigthesefilesareisdeterminedbythefollowing
globalconfigurationvalue:
--database.maximal-journal-sizesizeM aximalsizeofjournalinbytes.Canbeoverwrittenwhencreatinganewcollection.Notethat
thisalsolimitsthemaximalsizeofasingledocument.Thedefaultis32MB.
Per-collectionconfiguration
Durability
331
Thejournalsizecanalsobeadjustedonaper-collectionlevelusingthecollection'spropertiesmethod.
Durability
332
IntroductiontoReplication
Replicationallowsyoutoreplicatedataontoanothermachine.ItformsthebaseofalldisasterrecoveryandfailoverfeaturesArangoDB
offers.
ArangoDBoffersasynchronousandsynchronousreplicationwhichbothhavetheirprosandcons.Bothmodesmayandshouldbe
combinedinarealworldscenarioandbeappliedintheusecasewheretheyexcelmost.
Wewilldescribeprosandconsofeachoftheminthefollowingsections.
Synchronousreplication
Synchronousreplicationonlyworksininaclusterandistypicallyusedformissioncriticaldatawhichmustbeaccessibleatalltimes.
Synchronousreplicationgenerallystoresacopyofthedataonanotherhostandkeepsitinsync.Essentiallywhenstoringdataafter
enablingsynchronousreplicationtheclusterwillwaitforallreplicastowriteallthedatabeforegreenlightingthewriteoperationtothe
client.Thiswillnaturallyincreasethelatencyabit,sinceonemorenetworkhopisneededforeachwrite.Howeveritwillenablethe
clustertoimmediatelyfailovertoareplicawheneveranoutagehasbeendetected,withoutlosinganycommitteddata,andmostly
withoutevensignalinganerrorconditiontotheclient.
Synchronousreplicationisorganizedinawaythateveryshardhasaleaderandr-1followers.Thenumberoffollowerscanbecontrolled
usingthe replicationFactorwheneveryoucreateacollection,the replicationFactoristhetotalnumberofcopiesbeingkept,thatis,
itisoneplusthenumberoffollowers.
Asynchronousreplication
InArangoDBanywriteoperationwillbeloggedtothewrite-aheadlog.WhenusingAsynchronousreplicationslaveswillconnecttoa
masterandapplyalltheeventsfromtheloginthesameorderlocally.Afterthat,theywillhavethesamestateofdataasthemaster
database.
Replication
333
Asynchronousreplication
Asynchronousreplicationworksbyloggingeverydatamodificationonamasterandreplayingtheseeventsonanumberofslaves.
Transactionsarehonoredinreplication,i.e.transactionalwriteoperationswillbecomevisibleonslavesatomically.
Asallwriteoperationswillbeloggedtoamasterdatabase'swrite-aheadlog,thereplicationinArangoDBcurrentlycannotbeusedfor
write-scaling.ThemainpurposesofthereplicationincurrentArangoDBaretoprovideread-scalabilityand"hotbackups"ofspecific
databases.
Itispossibletoconnectmultipleslavedatabasestothesamemasterdatabase.Slavedatabasesshouldbeusedasread-onlyinstances,and
nouser-initiatedwriteoperationsshouldbecarriedoutonthem.Otherwisedataconflictsmayoccurthatcannotbesolvedautomatically,
andthatwillmakethereplicationstop.
Inanasynchronousreplicationscenarioslaveswillpullchangesfromthemasterdatabase.Slavesneedtoknowtowhichmasterdatabase
theyshouldconnectto,butamasterdatabaseisnotawareoftheslavesthatreplicatefromit.Whenthenetworkconnectionbetweenthe
masterdatabaseandaslavegoesdown,writeoperationsonthemastercancontinuenormally.Whenthenetworkisupagain,slavescan
reconnecttothemasterdatabaseandtransfertheremainingchanges.Thiswillhappenautomaticallyprovidedslavesareconfigured
appropriately.
Replicationlag
Inthissetup,writeoperationsareappliedfirstinthemasterdatabase,andappliedintheslavedatabase(s)afterwards.
Forexample,let'sassumeawriteoperationisexecutedinthemasterdatabaseatpointintimet0.Tomakeaslavedatabaseapplythe
sameoperation,itmustfirstfetchthewriteoperation'sdatafrommasterdatabase'swrite-aheadlog,thenparseitandapplyitlocally.
Thiswillhappenatsomepointintimeaftert0,let'ssayt1.
Thedifferencebetweent1andt0iscalledthereplicationlag,anditisunavoidableinasynchronousreplication.Theamountofreplication
lagdependsonmanyfactors,afewofwhichare:
thenetworkcapacitybetweentheslavesandthemaster
theloadofthemasterandtheslaves
thefrequencyinwhichslavespollthemasterforupdates
Betweent0andt1,thestateofdataonthemasterisnewerthanthestateofdataontheslave(s).Atpointintimet1,thestateofdataon
themasterandslave(s)isconsistentagain(providednonewdatamodificationshappenedonthemasterinbetween).Thus,the
replicationwillleadtoaneventuallyconsistentstateofdata.
Replicationconfiguration
Thereplicationisturnedoffbydefault.Inordertocreateamaster-slavesetup,theso-calledreplicationapplierneedstobeenabledon
theslavedatabases.
Replicationisconfiguredonaper-databaselevel.Ifmultipledatabasearetobereplicated,thereplicationmustbesetupindividuallyper
database.
Thereplicationapplierontheslavecanbeusedtoperformaone-timesynchronizationwiththemaster(andthenstop),ortoperforman
ongoingreplicationofchanges.Toresumereplicationonslaverestart,theautoStartattributeofthereplicationappliermustbesettotrue.
Replicationoverhead
Asthemasterserversarelogginganywriteoperationinthewrite-ahead-loganywayreplicationdoesn'tcauseanyextraoverheadonthe
master.Howeveritwillofcoursecausesomeoverheadforthemastertoserveincomingreadrequestsoftheslaves.Returningthe
requesteddataishoweveratrivialtaskforthemasterandshouldnotresultinanotableperformancedegrationinproduction.
334
Components
ReplicationLogger
Purpose
Thereplicationloggerwillwritealldata-modificationoperationsintothewrite-aheadlog.Thislogmaythenbereadbyclientstoreplay
anydatamodificationonadifferentserver.
Checkingthestate
Toquerythecurrentstateofthelogger,usethestatecommand:
require("@arangodb/replication").logger.state();
Theresultmightlooklikethis:
{
"state":{
"running":true,
"lastLogTick":"133322013",
"totalEvents":16,
"time":"2014-07-06T12:58:11Z"
},
"server":{
"version":"2.2.0-devel",
"serverId":"40897075811372"
},
"clients":{
}
}
Therunningattributewillalwaysbetrue.InearlierversionsofArangoDBthereplicationwasoptionalandthiscouldhavebeenfalse.
ThetotalEventsattributeindicateshowmanylogeventshavebeenloggedsincethestartoftheArangoDBserver.Finally,thelastLogTick
valueindicatestheidofthelastoperationthatwaswrittentotheserver'swrite-aheadlog.Itcanbeusedtodeterminewhethernew
operationswerelogged,andisalsousedbythereplicationapplierforincrementalfetchingofdata.
Note:ThereplicationloggerstatecanalsobequeriedviatheHTTPAPI.
Toquerywhichdatarangesarestillavailableforreplicationclientstofetch,theloggerprovidesthefirstTickandtickRangesfunctions:
require("@arangodb/replication").logger.firstTick();
ThiswillreturntheminimumtickvaluethattheservercanprovidetoreplicationclientsviaitsreplicationAPIs.ThetickRangesfunction
returnstheminimumandmaximumtickvaluesperlogfile:
require("@arangodb/replication").logger.tickRanges();
ReplicationApplier
Purpose
Thepurposeofthereplicationapplieristoreaddatafromamasterdatabase'seventlog,andapplythemlocally.Theapplierwillcheck
themasterdatabasefornewoperationsperiodically.Itwillperformanincrementalsynchronization,i.e.onlyaskingthemasterfor
operationsthatoccurredafterthelastsynchronization.
335
Thereplicationapplierdoesnotgetnotifiedbythemasterdatabasewhenthereare"new"operationsavailable,butinsteadusesthepull
principle.Itmightthustakesometime(theso-calledreplicationlag)beforeanoperationfromthemasterdatabasegetsshippedtoand
appliedinaslavedatabase.
Thereplicationapplierofadatabaseisruninaseparatethread.Itmayencounterproblemswhenanoperationfromthemastercannotbe
appliedsafely,orwhentheconnectiontothemasterdatabasegoesdown(networkoutage,masterdatabaseisdownorunavailableetc.).
Inthiscase,thedatabase'sreplicationapplierthreadmightterminateitself.Itisthenuptotheadministratortofixtheproblemand
restartthedatabase'sreplicationapplier.
Ifthereplicationappliercannotconnecttothemasterdatabase,orthecommunicationfailsatsomepointduringthesynchronization,the
replicationapplierwilltrytoreconnecttothemasterdatabase.Itwillgiveupreconnectingonlyafteraconfigurableamountof
connectionattempts.
Thereplicationapplierstateisqueryableatanytimebyusingthestatecommandoftheapplier.Thiswillreturnthestateoftheapplier
ofthecurrentdatabase:
require("@arangodb/replication").applier.state();
{
"state":{
"running":true,
"lastAppliedContinuousTick":"152786205",
"lastProcessedContinuousTick":"152786205",
"lastAvailableContinuousTick":"152786205",
"progress":{
"time":"2014-07-06T13:04:57Z",
"message":"fetchingmasterlogfromoffset152786205",
"failedConnects":0
},
"totalRequests":38,
"totalFailedConnects":0,
"totalEvents":1,
"lastError":{
"errorNum":0
},
"time":"2014-07-06T13:04:57Z"
},
"server":{
"version":"2.2.0-devel",
"serverId":"210189384542896"
},
"endpoint":"tcp://master.example.org:8529",
"database":"_system"
}
Therunningattributeindicateswhetherthereplicationapplierofthecurrentdatabaseiscurrentlyrunningandpollingtheserverat
endpointfornewevents.
Theprogress.failedConnectsattributeshowshowmanyfailedconnectionattemptsthereplicationappliercurrentlyhasencounteredina
row.Incontrast,thetotalFailedConnectsattributeindicateshowmanyfailedconnectionattemptstheapplierhasmadeintotal.The
totalRequestsattributeshowshowmanyrequeststheapplierhassenttothemasterdatabaseintotal.ThetotalEventsattributeshows
howmanylogeventstheapplierhasreadfromthemaster.
Theprogress.messagesub-attributeprovidesabriefhintofwhattheappliercurrentlydoes(ifitisrunning).ThelastErrorattributealso
hasanoptionalerrorMessagesub-attribute,showingthelatesterrormessage.TheerrorNumsub-attributeofthelastErrorattributecan
beusedbyclientstoprogrammaticallycheckforerrors.Itshouldbe0ifthereisnoerror,anditshouldbenon-zeroiftheapplier
terminateditselfduetoaproblem.
Hereisanexampleofthestateafterthereplicationapplierterminateditselfdueto(repeated)connectionproblems:
336
{
"state":{
"running":false,
"progress":{
"time":"2014-07-06T13:14:37Z",
"message":"applierstopped",
"failedConnects":6
},
"totalRequests":79,
"totalFailedConnects":11,
"totalEvents":0,
"lastError":{
"time":"2014-07-06T13:09:41Z",
"errorMessage":"couldnotconnecttomasterattcp://master.example.org:8529:Couldnotconnectto'tcp:/...",
"errorNum":1400
},
...
}
}
Note:thestateofadatabase'sreplicationapplierisqueryableviatheHTTPAPI,too.PleaserefertoHTTPInterfaceforReplicationfor
moredetails.
All-in-onesetup
Tocopytheinitialdatafromtheslavetothemasterandstartthecontinuousreplication,thereisanall-in-onecommandsetupReplication:
require("@arangodb/replication").setupReplication(configuration);
Thefollowingexampledemonstrateshowtousethecommandforsettingupreplicationforthe_systemdatabase.Notethatitshouldbe
runontheslaveandnotthemaster:
db._useDatabase("_system");
require("@arangodb/replication").setupReplication({
endpoint:"tcp://master.domain.org:8529",
username:"myuser",
password:"mypasswd",
verbose:false,
includeSystem:false,
incremental:true,
autoResync:true
});
Thecommandwillreturnwhentheinitialsynchronizationisfinishedandthecontinuousreplicationisstarted,orincasetheinitial
synchronizationhasfailed.
Iftheinitialsynchronizationissuccessful,thecommandwillstorethegivenconfigurationontheslave.Italsoconfiguresthecontinuous
replicationtostartautomaticallyiftheslaveisrestarted,i.e.autoStartissettotrue.
Ifthecommandisrunwhiletheslave'sreplicationapplierisalreadyrunning,itwillfirststoptherunningapplier,dropitsconfiguration
anddoaresynchronizationofdatawiththemaster.Itwillthenusetheprovidedconfigration,overwritinganypreviouslyexisting
replicationconfigurationontheslave.
StartingandStopping
Tomanuallystartandstoptheapplierinthecurrentdatabase,thestartandstopcommandscanbeusedlikethis:
require("@arangodb/replication").applier.start(<tick>);
require("@arangodb/replication").applier.stop();
Note:Startingareplicationapplierwithoutsettingupaninitialconfigurationwillfail.Thereplicationapplierwilllookforits
configurationinafilenamedREPLICATION-APPLIER-CONFIGinthecurrentdatabase'sdirectory.Ifthefileisnotpresent,ArangoDB
willusesomedefaultconfiguration,butitcannotguesstheendpoint(theaddressofthemasterdatabase)theappliershouldconnectto.
Thusstartingtheapplierwithoutconfigurationwillfail.
337
Notethatatthefirsttimeyoustarttheapplier,youshouldpassthevaluereturnedinthelastLogTickattributeoftheinitialsync
operation.
Note:Startingadatabase'sreplicationapplierviathestartcommandwillnotnecessarilystarttheapplieronthenextandfollowing
ArangoDBserverrestarts.Additionally,stoppingadatabase'sreplicationappliermanuallywillnotnecessarilypreventtheapplierfrom
beingstartedagainonthenextserverstart.Allofthisisconfigurableseparately(hangonreading).
Note:whenstoppingandrestartingthereplicationapplierofdatabase,itwillresumewhereitlaststopped.Thisissensiblebecause
replicationlogeventsshouldbeappliedincrementally.Ifthereplicationapplierofadatabasehasneverbeenstartedbefore,itneedssome
tickvaluefromthemaster'slogfromwhichtostartfetchingevents.
Thereisonecaveattoconsiderwhenstoppingareplicationontheslave:iftherearestillongoingreplicatedtransactionsthatareneither
committedoraborted,stoppingthereplicationapplierwillcausetheseoperationstobelostfortheslave.Ifthesetransactionscommiton
themasterlaterandthereplicationisresumed,theslavewillnotbeabletocommitthesetransactions,too.Thusstoppingthereplication
applierontheslavemanuallyshouldonlybedoneifthereiscertaintythattherearenoongoingtransactionsonthemaster.
Configuration
Toconfigurethereplicationapplierofaspecificdatabase,usethepropertiescommand.Usingitwithoutanyargumentswillreturnthe
applier'scurrentconfiguration:
require("@arangodb/replication").applier.properties();
{
"requestTimeout":600,
"connectTimeout":10,
"ignoreErrors":0,
"maxConnectRetries":10,
"chunkSize":0,
"autoStart":false,
"adaptivePolling":true,
"includeSystem":true,
"requireFromPresent":false,
"autoResync":false,
"autoResyncRetries":2,
"verbose":false
}
Note:Thereisnoendpointattributeconfiguredyet.Theendpointattributeisrequiredforthereplicationappliertobestartable.Youmay
alsowanttoconfigureausernameandpasswordfortheconnectionviatheusernameandpasswordattributes.
require("@arangodb/replication").applier.properties({
username:"root",
password:"secret",
verbose:false
});
Thiswillre-configurethereplicationapplierforthecurrentdatabase.Theconfigurationwillbeusedfromthenextstartofthereplication
applier.Thereplicationappliercannotbere-configuredwhileitisrunning.Itmustbestoppedfirsttobere-configured.
TomakethereplicationapplierofthecurrentdatabasestartautomaticallywhentheArangoDBserverstarts,usetheautoStartattribute.
SettingtheadaptivePollingattributetotruewillmakethereplicationapplierpollthemasterdatabaseforchangeswithavariable
frequency.Thereplicationapplierwillthenlowerthefrequencywhenthemasterisidle,andincreaseitwhenthemastercanprovidenew
events).Otherwisethereplicationapplierwillpollthemasterdatabaseforchangeswithaconstantfrequency.
TheidleMinWaitTimeattributecontrolstheminimumwaittime(inseconds)thatthereplicationapplierwillintentionallyidlebefore
fetchingmorelogdatafromthemasterincasethemasterhasalreadysentallitslogdata.Thiswaittimecanbeusedtocontrolthe
frequencywithwhichthereplicationappliersendsHTTPlogfetchrequeststothemasterincasethereisnowriteactivityonthemaster.
338
TheidleMaxWaitTimeattributecontrolsthemaximumwaittime(inseconds)thatthereplicationapplierwillintentionallyidlebefore
fetchingmorelogdatafromthemasterincasethemasterhasalreadysentallitslogdataandtherehavebeenpreviouslogfetchattempts
thatresultedinnomorelogdata.Thiswaittimecanbeusedtocontrolthemaximumfrequencywithwhichthereplicationappliersends
HTTPlogfetchrequeststothemasterincasethereisnowriteactivityonthemasterforlongerperiods.Notethatthisconfiguration
valuewillonlybeusediftheoptionadaptivePollingissettotrue.
Tosetatimeoutforconnectionandfollowingrequestattempts,usetheconnectTimeoutandrequestTimeoutvalues.The
maxConnectRetriesattributeconfiguresafterhowmanyfailedconnectionattemptsinarowthereplicationapplierwillgiveupandturn
itselfoff.Youmaywanttosetthistoahighvaluesothattemporarynetworkoutagesdonotleadtothereplicationapplierstopping
itself.TheconnectRetryWaitTimeattributeconfigureshowlongthereplicationapplierwillwaitbeforeretryingtheconnectiontothe
masterincaseofconnectionproblems.
ThechunkSizeattributecanbeusedtocontroltheapproximatemaximumsizeofamaster'sresponse(inbytes).Settingittoalowvalue
maymakethemasterrespondfaster(lessdataisassembledbeforethemastersendstheresponse),butmayrequiremorerequestresponseroundtrips.Setitto0touseArangoDB'sbuilt-indefaultvalue.
TheincludeSystemattributecontrolswhetherchangestosystemcollections(suchas_graphsor_users)shouldbeapplied.Ifsettotrue,
changesinthesecollectionswillbereplicated,otherwise,theywillnotbereplicated.Itisoftennotnecessarytoreplicatedatafrom
systemcollections,especiallybecauseitmayleadtoconfusionontheslavebecausetheslaveneedstohaveitsownsystemcollectionsin
ordertostartandkeepoperational.
TherequireFromPresentattributecontrolswhethertheapplierwillstartsynchronizingincaseitdetectsthatthemastercannotprovide
datafortheinitialtickvalueprovidedbytheslave.Thismaybethecaseifthemasterdoesnothaveabigenoughbacklogofhistoric
WALlogfiles,andwhenthereplicationisre-startedafteralongerpause.WhenrequireFromPresentissettotrue,thenthereplication
applierwillcheckatstartwhetherthestarttickfromwhichitstartsorresumesreplicationisstillpresentonthemaster.Ifnot,thenthere
wouldbedataloss.IfrequireFromPresentistrue,thereplicationapplierwillabortwithanappropriateerrormessage.Ifsettofalse,
thenthereplicationapplierwillstillstart,andignorethedataloss.
TheautoResyncoptioncanbeusedinconjunctionwiththerequireFromPresentoptionasfollows:whenbothrequireFromPresentand
autoResyncaresettotrueandthemastercannotprovidethelogdatatheslaverequests,thereplicationapplierwillstopasusual.Butdue
tothefactthatautoResyncissettotrue,theslavewillautomaticallytriggerafullresyncofalldatawiththemaster.Afterthat,the
replicationapplierwillgointocontinuousreplicationmodeagain.Additionally,settingautoResynctotruewilltriggerafullresynchronizationofdatawhenthecontinuousreplicationisstartedanddetectsthatthereisnostarttickvalue.
Automaticre-synchronizationmaytransferalotofdatafromthemastertotheslaveandcanbeexpensive.Itisthereforeturnedoffby
default.Whenturnedoff,theslavewillneverperformanautomaticre-synchronizationwiththemaster.
TheautoResyncRetriesoptioncanbeusedtocontrolthenumberofresynchronizationretriesthatwillbeperformedinarowwhen
automaticresynchronizationisenabledandkicksin.Settingthisto0willeffectivelydisableautoResync.Settingittosomeothervalue
willlimitthenumberofretriesthatareperformed.Thishelpspreventingendlessretriesincaseresynchronizationsalwaysfail.
Theverboseattributecontrolstheverbosityofthereplicationlogger.Settingitto truewillmakethereplicationapplierwritealineto
thelogforeveryoperationitperforms.Thisshouldonlybeusedfordiagnosingreplicationproblems.
Thefollowingexamplewillsetmostofthediscussedpropertiesforthecurrentdatabase'sapplier:
username:"root",
password:"secret",
adaptivePolling:true,
connectTimeout:15,
maxConnectRetries:100,
chunkSize:262144,
autoStart:true,
includeSystem:true,
autoResync:true,
autoResyncRetries:2,
});
Aftertheapplierisnowfullyconfigured,itcouldtheoreticallybestarted.However,wemayfirstneedaninitialsynchronizationofall
collectionsandtheirdatafromthemasterbeforewestartthereplicationapplier.
Theonlysafemethodfordoingafullsynchronization(orre-synchronization)isthusto
339
stopthereplicationapplierontheslave(ifcurrentlyrunning)
performaninitialfullsyncwiththemasterdatabase
notethemasterdatabase'slastLogTickvalueand
startthecontinuousreplicationapplierontheslaveusingthistickvalue.
Theinitialsynchronizationforthecurrentdatabaseisexecutedwiththesynccommand:
require("@arangodb/replication").sync({
username:"root",
password:"secret,
includeSystem:true
});
TheincludeSystemoptioncontrolswhetherdatafromsystemcollections(suchas_graphsand_users)shallbesynchronized.
TheinitialsynchronizationcanoptionallybeconfiguredtoincludeorexcludespecificcollectionsusingtherestrictTypeand
restrictCollectionparameters.
Thefollowingcommandonlysynchronizescollectionfooandbar:
username:"root",
password:"secret,
restrictType:"include",
restrictCollections:["foo","bar"]
});
UsingarestrictTypeofexclude,allcollectionsbutthespecifiedwillbesynchronized.
Warning:syncwilldoafullsynchronizationofthecollectionsinthecurrentdatabasewithcollectionspresentinthemasterdatabase.
Anylocalinstancesofthecollectionsandalltheirdataareremoved!Onlyexecutethiscommandifyouaresureyouwanttoremovethe
localdata!
Assyncdoesafullsynchronization,itmighttakeawhiletoexecute.Whensynccompletessuccessfully,itreturnsanarrayofcollections
ithassynchronizedinitscollectionsattribute.Itwillalsoreturnthemasterdatabase'slastlogtickvalueatthetimethesyncwasstarted
onthemaster.ThetickvalueiscontainedinthelastLogTickattributeofthesynccommand:
{
"lastLogTick":"231848833079705",
"collections":[...]
}
Nowyoucanstartthecontinuoussynchronizationforthecurrentdatabaseontheslavewiththecommand
require("@arangodb/replication").applier.start("231848833079705");
Note:Thetickvaluesshouldbetreatedasstrings.Usingnumericdatatypesfortickvaluesisunsafebecausetheymightexceedthe32bit
valueandtheIEEE754doubleaccuracyranges.
340
ExampleSetup
Settingupaworkingmaster-slavereplicationrequirestwoArangoDBinstances:
master:thisistheinstancethatalldata-modificationoperationsshouldbedirectedto
slave:onthisinstance,we'llstartareplicationapplier,andthiswillfetchdatafromthemasterdatabase'swrite-aheadlogandapply
itsoperationslocally
Forthefollowingexamplesetup,we'llusetheinstancetcp://master.domain.org:8529asthemaster,andtheinstance
tcp://slave.domain.org:8530asaslave.
Thegoalistohavealldatafromthedatabase_systemonmastertcp://master.domain.org:8529bereplicatedtothedatabase_systemon
theslavetcp://slave.domain.org:8530.
Onthemaster,nothingspecialneedstobedone,asallwriteoperationswillautomaticallybeloggedinthemaster'swrite-aheadlog
(WAL).
All-in-onesetup
Tomakethereplicationcopytheinitialdatafromthemastertotheslaveandstartthecontinuousreplicationontheslave,thereisan
all-in-onecommand:
require("@arangodb/replication").setupReplication(configuration);
Thefollowingexampledemonstrateshowtousethecommandforsettingupreplicationforthe_systemdatabase.Notethatitshouldbe
runontheslaveandnotthemaster:
require("@arangodb/replication").setupReplication({
username:"myuser",
verbose:false,
incremental:true,
autoResync:true
});
Thecommandwillreturnwhentheinitialsynchronizationisfinishedandthecontinuousreplicationhasbeenstarted,orincasetheinitial
synchronizationhasfailed.
Iftheinitialsynchronizationissuccessful,thecommandwillstorethegivenconfigurationontheslave.Italsoconfiguresthecontinuous
replicationtostartautomaticallyiftheslaveisrestarted,i.e.autoStartissettotrue.
Ifthecommandisrunwhiletheslave'sreplicationapplierisalreadyrunning,itwillfirststoptherunningapplier,dropitsconfiguration
anddoaresynchronizationofdatawiththemaster.Itwillthenusetheprovidedconfigration,overwritinganypreviouslyexisting
replicationconfigurationontheslave.
Initialsynchronization
Theinitialsynchronizationandcontinuousreplicationappliercanalsobestartedseparately.Tostartreplicationontheslave,makesure
therecurrentlyisnoreplicationapplierrunning.
Thefollowingcommandsstoparunningapplierintheslave's_systemdatabase:
require("@arangodb/replication").applier.stop();
341
Thestopoperationwillterminateanyreplicationactivityinthe_systemdatabaseontheslave.
Afterthat,theinitialsynchronizationcanberun.Itwillcopythecollectionsfromthemastertotheslave,overwritingexistingdata.To
runtheinitialsynchronization,executethefollowingcommandsontheslave:
username:"myuser",
verbose:false
});
Usernameandpasswordonlyneedtobespecifiedwhenthemasterrequiresauthentication.Tocheckwhatthesynchronizationis
currentlydoing,supplysettheverboseoptiontotrue.Ifset,thesynchronizationwillcreatelogmessageswiththecurrent
synchronizationstatus.
Warning:Thesynccommandwillreplacedataintheslavedatabasewithdatafromthemasterdatabase!Onlyexecutethesecommands
ifyouhaveverifiedyouareonthecorrectserver,inthecorrectdatabase!
ThesyncoperationwillreturnanattributenamedlastLogTickwhichwe'llneedtonote.Thelastlogtickwillbeusedasthestartingpoint
forsubsequentreplicationactivity.Let'sassumewegotthefollowinglastlogtick:
{
"lastLogTick":"40694126",
...
}
InitialsynchronizationfromtheArangoShell
Theinitialsynchronizationviathesynccommandmaytakealongtimetocomplete.Theshellwillblockuntiltheslavehascompletedthe
initialsynchronizationoruntilanerroroccurs.Bydefault,thesynccommandintheArangoShellwillpolltheslaveforastatusupdate
every10seconds.
Optionallythesynccommandcanbemadenon-blockingbysettingitsasyncoptiontotrue.Inthiscase,thesynccommandwillreturn
instantlywithanidstring,andtheinitialsynchronizationwillrundetachedonthemaster.Tofetchthecurrentstatusofthesyncprogress
fromtheArangoShell,thegetSyncResultfunctioncanbeusedasfollows:
varreplication=require("@arangodb/replication");
/*runcommandinasyncmode*/
varid=replication.sync({
username:"myuser",
async:true
});
/*nowquerythestatusofouroperation*/
print(replication.getSyncResult(id));
getSyncResultwillreturnfalseaslongasthesynchronizationisnotcomplete,andreturnthesynchronizationresultotherwise.
Continuoussynchronization
Whentheinitialsynchronizationisfinished,thecontinuousreplicationappliercanbestartedusingthelastlogtickprovidedbythesync
command.Beforestartingit,thereisatleastoneconfigurationoptiontoconsider:replicationontheslavewillberunninguntiltheslave
getsshutdown.Whentheslaveservergetsrestarted,replicationwillbeturnedoffagain.Tochangethis,wefirstneedtoconfigurethe
slave'sreplicationapplierandsetitsautoStartattribute.
Here'sthecommandtoconfigurethereplicationapplierwithseveraloptions,includingtheautoStartattribute:
342
username:"myuser",
autoStart:true,
autoResync:true,
autoResyncRetries:2,
adaptivePolling:true,
requireFromPresent:false,
idleMinWaitTime:0.5,
idleMaxWaitTime:1.5,
verbose:false
});
Animportantconsiderationforreplicationiswhetherdatafromsystemcollections(suchas_graphsor_users)shouldbeapplied.The
includeSystemoptioncontrolsthat.Ifsettotrue,changesinsystemcollectionswillbereplicated.Otherwise,theywillnotbereplicated.
Itisoftennotnecessarytoreplicatedatafromsystemcollections,especiallybecauseitmayleadtoconfusionontheslavebecausethe
slaveneedstohaveitsownsystemcollectionsinordertostartandkeepoperational.
TherequireFromPresentattributecontrolswhethertheapplierwillstartsynchronizingincaseitdetectsthatthemastercannotprovide
datafortheinitialtickvalueprovidedbytheslave.Thismaybethecaseifthemasterdoesnothaveabigenoughbacklogofhistoric
WALlogfiles,andwhenthereplicationisre-startedafteralongerpause.WhenrequireFromPresentissettotrue,thenthereplication
applierwillcheckatstartwhetherthestarttickfromwhichitstartsorresumesreplicationisstillpresentonthemaster.Ifnot,thenthere
wouldbedataloss.IfrequireFromPresentistrue,thereplicationapplierwillabortwithanappropriateerrormessage.Ifsettofalse,
thenthereplicationapplierwillstillstart,andignorethedataloss.
TheautoResyncoptioncanbeusedinconjunctionwiththerequireFromPresentoptionasfollows:whenbothrequireFromPresentand
autoResyncaresettotrueandthemastercannotprovidethelogdatatheslavehadrequested,thereplicationapplierwillstopasusual.
ButduetothefactthatautoResyncissettotrue,theslavewillautomaticallytriggerafullresyncofalldatawiththemaster.Afterthat,
thereplicationapplierwillgointocontinuousreplicationmodeagain.Additionally,settingautoResynctotruewilltriggerafullresynchronizationofdatawhenthecontinuousreplicationisstartedanddetectsthatthereisnostarttickvalue.
Notethatautomaticre-synchronization(autoResyncoptionsettotrue)maytransferalotofdatafromthemastertotheslaveandcan
thereforebeexpensive.Stillit'sturnedonheresothere'slessneedformanualintervention.
TheautoResyncRetriesoptioncanbeusedtocontrolthenumberofresynchronizationretriesthatwillbeperformedinarowwhen
automaticresynchronizationisenabledandkicksin.Settingthisto0willeffectivelydisableautoResync.Settingittosomeothervalue
willlimitthenumberofretriesthatareperformed.Thishelpspreventingendlessretriesincaseresynchronizationsalwaysfail.
Nowit'stimetostartthereplicationapplierontheslaveusingthelastlogtickwegotbefore:
require("@arangodb/replication").applier.start("40694126");
Thiswillreplicatealloperationshappeninginthemaster'ssystemdatabaseandapplythemontheslave,too.
Afterthat,youshouldbeabletomonitorthestateandprogressofthereplicationapplierbyexecutingthestatecommandontheslave
server:
require("@arangodb/replication").applier.state();
Pleasenotethatstoppingthereplicationapplierontheslaveusingthestopcommandshouldbeavoided.Thereasonisthatcurrently
ongoingtransactions(thathavepartlybeenreplicatedtotheslave)willbeneedtoberestartedafterarestartofthereplicationapplier.
Stoppingandrestartingthereplicationapplierontheslaveshouldthusonlybeperformedifthereiscertaintythatthemasteriscurrently
fullyidleandalltransactionshavebeenreplicatedfully.
Notethatwhileaslavehasonlypartlyexecutedatransactionfromthemaster,itmightkeepawritelockonthecollectionsinvolvedin
thetransaction.
YoumayalsowanttocheckthemasterandslavestatesviatheHTTPAPIs(seeHTTPInterfaceforReplication).
343
SyncingCollections
Inordertosynchronizedataforasinglecollectionfromamastertoaslaveinstance,thereisthesyncCollectionfunction:
Itwillfetchalldocumentsofthespecifiedcollectionfromthemasterdatabaseandstoretheminthelocalinstance.Afterthe
synchronization,thecollectiondataontheslavewillbeidenticaltothedataonthemaster,providednofurtherdatachangeshappenon
themaster.Anydatachangesthatareperformedonthemasterafterthesynchronizationwasstartedwillnotbecapturedby
syncCollection,butneedtobereplicatedusingtheregularreplicationappliermechanism.
Forthefollowingexamplesetup,we'llusetheinstancetcp://master.domain.org:8529asthemaster,andtheinstance
tcp://slave.domain.org:8530asaslave.
Thegoalistohavealldatafromthecollectiontestindatabase_systemonmastertcp://master.domain.org:8529bereplicatedtothe
collectiontestindatabase_systemontheslavetcp://slave.domain.org:8530.
Onthemaster,thecollectiontestneedstobepresentinthe_systemdatabase,withanydatainit.
Totransferthiscollectiontotheslave,issuethefollowingcommandsthere:
require("@arangodb/replication").syncCollection("test",{
username:"myuser",
password:"mypasswd"
});
Warning:ThesyncCollectioncommandwillreplacethecollection'sdataintheslavedatabasewithdatafromthemasterdatabase!Only
executethesecommandsifyouhaveverifiedyouareonthecorrectserver,inthecorrectdatabase!
SettingtheoptionalincrementalattributeinthecalltosyncCollectionwillstartanincrementaltransferofdata.Thismaybeusefulincase
whentheslavealreadyhaspartsoralmostallofthedatainthecollectionandonlythedifferencesneedtobesynchronized.Notethatto
computethedifferencestheincrementaltransferwillbuildasortedlistofalldocumentkeysinthecollectiononboththeslaveandthe
master,whichmaystillbeexpensiveforhugecollectionsintermsofmemoryusageandruntime.Duringbuildingthelistofkeysthe
collectionwillberead-lockedonthemaster.
TheinitialSyncMaxWaitTimeattributeinthecalltosyncCollectioncontrolshowlongtheslavewillwaitforamaster'sresponse.Thiswait
timecanbeusedtocontrolafterwhattimethesynchronizationwillgiveupandfail.
ThesyncCollectioncommandmaytakealongtimetocompleteifthecollectionisbig.Theshellwillblockuntiltheslavehas
synchronizedtheentirecollectionfromthemasteroruntilanerroroccurs.Bydefault,thesyncCollectioncommandintheArangoShell
willpollforastatusupdateevery10seconds.
WhensyncCollectioniscalledfromtheArangoShell,theoptionalasyncattributecanbeusedtostartthesynchronizationasabackground
processontheslave.Ifasyncissettotrue,thecalltosyncCollectionwillreturnalmostinstantlywithanidstring.Usingthisidstring,
thestatusofthesyncjobontheslavecanbequeriedusingthegetSyncResultfunctionasfollows:
varreplication=require("@arangodb/replication");
/*runcommandinasyncmode*/
varid=replication.syncCollection("test",{
username:"myuser",
async:true
});
/*nowquerythestatusofouroperation*/
getSyncResultwillreturnfalseaslongasthesynchronizationisnotcomplete,andreturnthesynchronizationresultotherwise.
344
ReplicationLimitations
ThereplicationinArangoDBhasafewlimitations.SomeoftheselimitationsmayberemovedinlaterversionsofArangoDB:
thereisnofeedbackfromtheslavestothemaster.Ifaslavecannotapplyaneventitgotfromthemaster,themasterwillhavea
differentstateofdata.Inthiscase,thereplicationapplierontheslavewillstopandreportanerror.Administratorscantheneither
"fix"theproblemorre-syncthedatafromthemastertotheslaveandstarttheapplieragain.
atthemomentitisassumedthatonlythereplicationapplierexecuteswriteoperationsonaslave.ArangoDBcurrentlydoesnot
preventusersfromcarryingouttheirownwriteoperationsonslaves,thoughthismightleadtoundefinedbehaviorandthe
replicationapplierstopping.
whenareplicationslaveasksamasterforlogevents,thereplicationmasterwillreturnallwriteoperationsforuser-defined
collections,butitwillexcludewriteoperationsforcertainsystemcollections.Thefollowingcollectionsareexcludedintentionally
fromreplication:_apps,_trx,_replication,_configuration,_jobs,_queues,_sessions,_foxxlogandallstatisticscollections.Write
operationsforthefollowingsystemcollectionscanbequeriedfromamaster:_aqlfunctions,_graphs,_users.
Foxxapplicationsconsistofdatabaseentriesandapplicationscriptsinthefilesystem.ThefilesystempartsofFoxxapplications
arenottrackedanywhereandthusnotreplicatedincurrentversionsofArangoDB.ToreplicateaFoxxapplication,itisrequiredto
copytheapplicationtotheremoteserverandinstallitthereusingthefoxx-managerutility.
masterserversdonotknowwhichslavesareorwillbeconnectedtothem.Allserversinareplicationsetuparecurrentlyonly
looselycoupled.Therecurrentlyisnowayforaclienttoquerywhichserversarepresentinareplication.
whennotusingourmesosintegrationfailovermustbehandledbyclientsorclientAPIs.
therecurrentlyisonereplicationapplierperArangoDBdatabase.Itisthusnotpossibletohaveaslaveapplyoperationsfrom
multiplemastersintothesametargetdatabase.
replicationissetuponaper-databaselevel.WhenusingArangoDBwithmultipledatabases,replicationmustbeconfigured
individuallyforeachdatabase.
thereplicationapplierissingle-threaded,butwriteoperationsonthemastermaybeexecutedinparalleliftheyaffectdifferent
collections.Thusthereplicationappliermightnotbeabletocatchupwithaverypowerfulandloadedmaster.
replicationisonlysupportedbetweenthetwoArangoDBserversrunningthesameArangoDBversion.Itiscurrentlynotpossible
toreplicatebetweendifferentArangoDBversions.
areplicationappliercannotapplydatafromitself.
345
Atitscoresynchronousreplicationwillreplicatewriteoperationstomultiplehosts.Thisfeatureisonlyavailablewhenoperating
ArangoDBinacluster.Wheneveracoordinatorexecutesasychronouslyreplicatedwriteoperationitwillonlybereportedtobe
successfulifitwascarriedoutonallreplicas.IncontrasttomultimasterreplicationsetupsknownfromothersystemsArangoDB's
synchronousoperationguaranteesaconsistentstateacrossthecluster.
346
Implementation
Architectureinsidethecluster
SynchronousreplicationcanbeconfiguredpercollectionviathepropertyreplicationFactor.Synchronousreplicationrequiresaclusterto
operate.
WheneveryouspecifyareplicationFactorgreaterthan1whencreatingacollection,synchronousreplicationwillbeactivatedforthis
collection.Theclusterwilldeterminesuitableleadersandfollowersforeveryrequestedshard(numberOfShards)withinthecluster.
Whenrequestingdataofashardonlythecurrentleaderwillbeaskedwhereasfollowerswillonlykeeptheircopyinsync.Thisisdueto
thecurrentimplementationoftransactions.
Usingsynchronousreplicationalonewillguaranteeconsistencyandhighavailabiltyatthecostofreducedperformance:Writerequests
willhaveahigherlatency(duetoeverywrite-requesthavingtobeexecutedonthefollowers)andreadrequestswon'tscaleoutasonly
theleaderisbeingasked.
Inaclustersynchronousreplicationwillbemanagedbythecoordinatorsfortheclient.Thedatawillalwaysbestoredonprimaries.
ThefollowingexamplewillgiveyouanideaofhowsynchronousoperationhasbeenimplementedinArangoDB.
1. Connecttoacoordinatorviaarangosh
2. Createacollection
127.0.0.1:8530@_system>db._create("test",{"replicationFactor":2})
3. thecoordinatorwillfigureoutaleaderand1followerandcreate1shard(asthisisthedefault)
4. Insertdata
127.0.0.1:8530@_system>db.test.insert({"replication":"
"})
5. Thecoordinatorwillwritethedatatotheleader,whichinturnwillreplicateittothefollower.
6. Onlywhenbothweresuccessfultheresultisreportedtobesuccessful
{
"_id":"test/7987",
"_key":"7987",
"_rev":"7987"
}
Whenafollowerfails,theleaderwillgiveuponitafter3secondsandproceedwiththeoperation.Assoonasthefollower(orthe
networkconnectiontotheleader)isbackup,thetwowillresynchronizeandsynchronousreplicationisresumed.Thishappensall
transparentlytotheclient.
ThecurrentimplementationofArangoDBdoesnotallowchangingthereplicationFactorlater.Thisissubjecttochange.Inthemeantime
theonlywayistocreateanewcollectionhavingthedesired replicationFactor,copyalldataandswapnameswiththeoriginalone.
Automaticfailover
Whenevertheleaderofashardisfailingandthereisaquerytryingtoaccessdataofthatshardthecoordinatorwillcontinuetryingto
contacttheleaderuntilittimeouts.Theinternalclustersupervisionwillcheckclusterhealtheveryfewsecondsandwilltakeactionif
thereisnoheartbeatfromaserverfor15seconds.Iftheleaderdoesn'tcomebackintimethesupervisionwillreorganizetheclusterby
promotingforeachshardafollowerthatisinsyncwithitsleadertobethenewleader.Fromthenonthecoordinatorswillcontactthe
newleader.
Theprocessisbestoutlinedusinganexample:
1. Theleaderofashard(letsnameitDBServer001)isgoingdown.
2. AcoordinatorisaskedtoreturnadocumentofashardDBServer001ismanaging:
347
127.0.0.1:8530@_system>db.test.document("100069")
3. Thecoordinatortriestocontacttheleader(DBServer001)andtimeouts.
4. Thecoordinatorretriestocontacttheleader(DBServer001)andtimeouts.
5. ThesupervisiondetectsoutageofDBServer001.
6. Thesupervisionpromotesoneofthefollowers(sayDBServer002)thatisinsynctobeleaderandmakesDBServer001afollower.
7. Thecoordinatorretriestocontacttheleader(DBServer002)andreturnstheresult:
{
"_key":"100069",
"_id":"test/100069",
"_rev":"513",
"replication":" "
}
8. AfterawhilethesupervisiondeclaresDBServer001tobecompletelydead.
9. AnewfollowerisdeterminedfromthepoolofDBservers.
10. Thenewfollowersyncsitsdatafromtheleaderandorderisrestored.
Pleasenotethattheremaystillbetimeouts.Dependingonwhenexactlytherequesthasbeendone(inregardtothesupervision)and
dependingonthetimeneededtoreconfiguretheclusterthecoordinatormightfailwithatimeouterror!
348
Configuration
Requirements
SynchronousreplicationrequiresanoperationalArangoDBcluster.
Enablingsynchronousreplication
Synchronousreplicationcanbeenabledpercollection.WhencreatingyoucanspecifythenumberofreplicasusingreplicationFactor.The
defaultis 1whicheffectivelydisablessynchronousreplication.
Example:
127.0.0.1:8530@_system>db._create("test",{"replicationFactor":3})
Anywriteoperationwillrequire2replicastoreportsuccessfromnowon.
349
Sharding
ArangoDBisorganizingitscollectiondatainshards.ShardingallowstousemultiplemachinestorunaclusterofArangoDBinstances
thattogetherconstituteasingledatabase.Thisenablesyoutostoremuchmoredata,sinceArangoDBdistributesthedataautomatically
tothedifferentservers.Inmanysituationsonecanalsoreapabenefitindatathroughput,againbecausetheloadcanbedistributedto
multiplemachines.
Shardsareconfiguredpercollectionsomultipleshardsofdataformthecollectionasawhole.Todetermineinwhichshardthedataisto
bestoredArangoDBperformsahashacrossthevalues.Bydefaultthishashisbeingcreatedfrom_key.
Toconfigurethenumberofshards:
127.0.0.1:8529@_system>db._create("sharded_collection",{"numberOfShards":4});
Toconfigurethehashing:
127.0.0.1:8529@_system>db._create("sharded_collection",{"numberOfShards":4,"shardKeys":["country"]});
Thiswouldbeusefultokeepdataofeverycountryinoneshardwhichwouldresultinbetterperformanceforqueriesworkingonaper
countrybase.Youcanalsospecifymultiple shardKeys.Notehoweverthatifyouchangetheshardkeysfromtheirdefailt ["_key"],
thenfindingadocumentinthecollectionbyitsprimarykeyinvolvesarequesttoeverysingleshard.Furthermore,inthiscaseonecanno
longerprescribetheprimarykeyvalueofanewdocumentbutmustusetheautomaticallygeneratedone.Thislatterrestrictioncomes
fromthefactthatensuringuniquenessoftheprimarykeywouldbeveryinefficientiftheusercouldspecifytheprimarykey.
Sharding
350
GeneralUpgradeInformation
Recommendedmajorupgradeprocedure
ToupgradeanexistingArangoDB2.xto3.0pleaseusetheproceduredescribedhere.
Recommendedminorupgradeprocedure
ToupgradeanexistingArangoDBdatabasetoanewerversionofArangoDB(e.g.3.0to3.1,or3.3to3.4),thefollowingmethodis
recommended:
ChecktheCHANGELOGandthelistofincompatiblechangesforAPIorotherchangesinthenewversionofArangoDBandmake
sureyourapplicationscandealwiththem
Stopthe"old"arangodserviceorbinary
Copytheentire"old"datadirectorytoasafeplace(thatis,abackup)
InstallthenewversionofArangoDBandstarttheserverwiththe--database.auto-upgradeoptiononce.Thismightwritetothe
logfileofArangoDB,soyoumaywanttocheckthelogsforanyissuesbeforegoingon.
Startthe"new"arangodserviceorbinaryregularlyandcheckthelogsforanyissues.Whenyou'reconfidenteverythingwentwell,
youmaywanttocheckthedatabasedirectoryforanyfileswiththeending.old.ThesefilesarecreatedbyArangoDBduring
upgradesandcanbesafelyremovedmanuallylater.
Ifanythinggoeswrongduringorshortlyaftertheupgrade:
Stopthe"new"arangodserviceorbinary
Reverttothe"old"arangodbinaryandrestorethe"old"datadirectory
Startthe"old"versionagain
ItisnotsupportedtousedatafilescreatedormodifiedbyanewerversionofArangoDBwithanolderArangoDBversion.Forexample,it
isunsupportedandislikelytocauseproblemswhenusing2.3datafileswithanArangoDB2.2instance.
Upgrading
351
UpgradingtoArangoDB3.0
PleasereadthefollowingsectionsifyouupgradefromapreviousversiontoArangoDB3.0.Pleasebesurethatyouhavecheckedthelist
ofchangesin3.0beforeupgrading.
MigratingdatabasesandcollectionsfromArangoDB2.8to3.0
ArangoDB3.0doesnotprovideanautomaticupdatemechanismfordatabasedirectoriescreatedwiththe2.xbranchesofArangoDB.
InordertomigratedatafromArangoDB2.8(oranolder2.xversion)intoArangoDB3.0,itisnecessarytoexportthedatafrom2.8using
arangodump,andthenimportthedumpintoafreshArangoDB3.0with arangorestore.
Todothis,firstrunthe2.8versionof arangodumptoexportthedatabasedataintoadirectory. arangodumpwilldumpthe _system
databasebydefault.Inordertomakeitdumpmultipledatabases,itneedstobeinvokedoncepersourcedatabase,e.g.
#in2.8
arangodump--server.database_system--output-directorydump-system
arangodump--server.databasemydb--output-directorydump-mydb
...
Thatwillproduceadumpdirectoryforeachdatabasethat arangodumpiscalledfor.Iftheserverhasauthenticationturnedon,itmaybe
necessarytoprovidetherequiredcredentialswheninvoking arangodump,e.g.
arangodump--server.database_system--server.usernamemyuser--server.passwordmypasswd--output-directorydump-system
Thedumpsproducedby arangodumpcannowbeimportedintoArangoDB3.0usingthe3.0versionof arangodump:
#in3.0
arangorestore--server.database_system--input-directorydump-system
arangorestore--server.databasemydb--input-directorydump-mydb
...
arangorestorewillbydefaultfailifthetargetdatabasedoesnotexist.Itcanbetoldtocreateitautomaticallyusingtheoption --createdatabasetrue:
arangorestore--server.databasemydb--create-databasetrue--input-directorydump-mydb
Andagainitmayberequiredtoprovideaccesscredentialswheninvoking arangorestore:
arangorestore--server.databasemydb--create-databasetrue--server.usernamemyuser--server.passwordmypasswd--input-directorydump-system
Pleasenotethattheversionofdump/restoreshouldmatchtheserverversion,i.e.itisrequiredtodumptheoriginaldatawiththe2.8
versionof arangodumpandrestoreitwiththe3.0versionof arangorestore.
Afterthatthe3.0instanceofArangoDBwillcontainthedatabasesandcollectionsthatwerepresentinthe2.8instance.
Adjustingauthenticationinfo
AuthenticationinformationwasstoredperdatabaseinArangoDB2.8,meaningtherecouldbedifferentusersandaccesscredentialsper
database.In3.0,theusersarestoredinacentrallocationinthe _systemdatabase.Tousethesameusersetupasin2.8,itmaybe
requiredtocreateextrausersand/oradjusttheirpermissions.
Inordertodothat,pleaseconnecttothe3.0instancewithanArangoShell(thiswillconnecttothe _systemdatabasebydefault):
arangosh--server.usernamemyuser--server.passwordmypasswd
Upgradingto3.0
352
Usethefollowingcommandstocreateanewuserwithsomepasswordandgrantthemaccesstoaspecificdatabase
require("@arangodb/users").save(username,password,true);
require("@arangodb/users").grantDatabase(username,databaseName,"rw");
Forexample,tocreateauser myuserwithpassword mypasswdandgivethemaccesstodatabases mydb1and mydb2,thecommands
wouldlookasfollows:
require("@arangodb/users").save("myuser","mypasswd",true);
require("@arangodb/users").grantDatabase("myuser","mydb1","rw");
require("@arangodb/users").grantDatabase("myuser","mydb2","rw");
Existinguserscanalsobeupdated,removedorlistedusingthefollowingcommands:
/*updateusermyuserwithpasswordmypasswd*/
require("@arangodb/users").update("myuser","mypasswd",true);
/*removeusermyuser*/
require("@arangodb/users").remove("myuser");
/*listallusers*/
require("@arangodb/users").all();
Foxxapplications
Thedump/restoreproceduredescribedabovewillnotexportandre-importFoxxapplications.Inordertomovethesefrom2.8to3.0,
Foxxapplicationsshouldbeexportedaszipfilesviathe2.8webinterface.
Thezipfilescanthenbeuploadedinthe"Services"sectionintheArangoDB3.0webinterface.Applicationsmayneedtobeadjusted
manuallytorunin3.0.PleaseconsultthemigrationguideforFoxxapps.
AnalternativewayofmovingFoxxappsinto3.0istocopythesourcedirectoryofa2.8Foxxapplicationmanuallyintothe3.0Foxx
appsdirectoryforthetargetdatabase(whichisnormally /var/lib/arangodb3-apps/_db/<dbname>/buttheexactlocationisplatformspecific).
Upgradingto3.0
353
PleasenotefirstthatadatabasedirectoryusedwithArangoDB2.8cannotbeusedwithearlierversions(e.g.ArangoDB2.7)anymore.
Upgradingadatabasedirectorycannotbereverted.ThereforepleasemakesuretocreateafullbackupofyourexistingArangoDB
installationbeforeperforminganupgrade.
DatabaseDirectoryVersionCheckandUpgrade
ArangoDBwillperformadatabaseversioncheckatstartup.WhenArangoDB2.8encountersadatabasecreatedwithearlierversionsof
ArangoDB,itwillrefusetostart.Thisisintentional.
Theoutputwillthenlooklikethis:
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':Databasedirectoryversion(20702)islowerthancurrentversion(20800).
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':---------------------------------------------------------------------2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':ItseemslikeyouhaveupgradedtheArangoDBbinary.
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':Ifthisiswhatyouwantedtodo,pleaserestartwiththe
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':--upgrade
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':optiontoupgradethedatainthedatabasedirectory.
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':Normallyyoucanusethecontrolscripttoupgradeyourdatabase
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':/etc/init.d/arangodbstop
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':/etc/init.d/arangodbupgrade
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':/etc/init.d/arangodbstart
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':---------------------------------------------------------------------2015-12-04T17:11:17Z[31432]FATALDatabase'_system'needsupgrade.Pleasestarttheserverwiththe--upgradeoption
TomakeArangoDB2.8startwithadatabasedirectorycreatedwithanearlierArangoDBversion,youmayneedtoinvoketheupgrade
procedureonce.ThiscanbedonebyrunningArangoDBfromthecommandlineandsupplyingthe --upgradeoption.
Note:herethesamedatabaseshouldbespecifiedthatisalsospecifiedwhenarangodisstartedregularly.Pleasedonotrunthe -upgradecommandoneachindividualdatabasesubfolder(named database-<somenumber>).
Forexample,ifyouregularlystartyourArangoDBserverwith
unix>arangodmydatabasefolder
thenrunning
unix>arangodmydatabasefolder--upgrade
willperformtheupgradeforthewholeArangoDBinstance,includingallofitsdatabases.
Startingwith --upgradewillrunadatabaseversioncheckandperformanynecessarymigrations.Asusual,youshouldcreateabackup
ofyourdatabasedirectorybeforeperformingtheupgrade.
Thelastlineoftheoutputshouldlooklikethis:
2015-12-04T17:12:15Z[31558]INFOdatabaseupgradepassed
Pleasecheckthefulloutputthe --upgraderun.Upgradingmayproduceerrors,whichneedtobefixedbeforeArangoDBcanbeused
properly.Ifnoerrorsarepresentortheyhavebeenresolvedmanually,youcanstartArangoDB2.8regularly.
Upgradingaclusterplannedinthewebinterface
Upgradingto2.8
354
AclusterofArangoDBinstanceshastobeupgradedaswell.ThisinvolvesupgradingallArangoDBinstancesinthecluster,aswellas
runningtheversioncheckonthewholerunningclusterintheend.
Wehavetriedtomakethisprocedureaspainlessandconvenientforyou.Weassumethatyouplanned,launchedandadministrateda
clusterusingthegraphicalfrontendinyourbrowser.Theupgradeprocedureisthenasfollows:
1. Firstshutdownyourclusterusingthegraphicalfrontendasusual.
2. Thenupgradealldispatcherinstancesonallmachinesinyourclusterusingtheversioncheckasdescribedaboveandrestartthem.
3. Nowopentheclusterdashboardinyourbrowserbypointingittothesamedispatcherthatyouusedtoplanandlaunchthecluster
inthegraphicalfrontend.Inadditiontotheusualbuttons"Relaunch","Editclusterplan"and"Deleteclusterplan"youwillsee
anotherbuttonmarked"Upgradeandrelaunchcluster".
4. Hitthisbutton,yourclusterwillbeupgradedandlaunchedandallisdoneforyoubehindthescenes.Ifallgoeswell,youwillsee
theusualclusterdashboardafterafewseconds.Ifthereisanerror,youhavetoinspectthelogfilesofyourclusterArangoDB
instances.Pleaseletusknowifyourunintoproblems.
Thereisanalternativewayusingthe ArangoDBshell.Insteadofsteps3.and4.aboveyoucanlaunch arangosh,pointittothe
dispatcherthatyouhaveusedtoplanandlaunchtheclusterusingtheoption --server.endpoint,andexecute
arangosh>require("org/arangodb/cluster").Upgrade("root","");
Thisupgradestheclusterandlaunchesit,exactlyaswiththebuttonaboveinthegraphicalfrontend.Youhavetoreplace "root"witha
usernameand ""withapasswordthatisvalidforauthenticationwiththecluster.
UpgradingFoxxappsgeneratedbyArangoDB2.7andearlier
Theimplementationofthe requirefunctionusedtoimportmodulesinArangoDBandFoxxhaschangedinordertoimprove
compatibilitywithNode.jsmodules.
Givenanapp/servicewiththefollowinglayout:
manifest.json
controllers/
todos.js
models/
todo.js
repositories/
todos.js
node_modules/
models/
todo.js
Thefile controllers/todos.jswouldpreviouslycontainthefollowing requirecalls:
var_=require('underscore');
varjoi=require('joi');
varFoxx=require('org/arangodb/foxx');
varArangoError=require('org/arangodb').ArangoError;
varTodos=require('repositories/todos');//<--!
varTodo=require('models/todo');//<--!
Therequirepaths repositories/todosand models/todowerepreviouslyresolvedlocallyasrelativetotheapproot.
Startingwith2.8thesepathswouldinsteadberesolvedasrelativetothe node_modulesfolderortheglobalArangoDBmodulepaths
beforebeingresolvedlocallyasafallback.
Inthegivenexamplelayouttheappwouldbreakin2.8becausethemodulename models/todowouldalwaysresolveto
node_modules/models/todo.js(whichpreviouslywouldhavebeenignored)insteadofthelocal models/todo.js.
Inordertomakesuretheappstillworksin2.8,therequirecallsin controllers/todos.jswouldneedtobeadjustedtolooklikethis:
Upgradingto2.8
355
var_=require('underscore');
varjoi=require('joi');
varArangoError=require('org/arangodb').ArangoError;
varTodos=require('../repositories/todos');//<--!
varTodo=require('../models/todo');//<--!
Notethattheold"global"stylerequirecallsmaystillworkin2.8butmaybreakunexpectedlyifmoduleswithmatchingnamesare
installedglobally.
Upgradingto2.8
356
procedureonce.ThiscanbedonebyrunningArangoDBfromthecommandlineandsupplyingthe --upgradeoption.
thenrunning
Upgradingto2.6
357
Upgradingto2.6
358
In2.5wehavealsochangedthepathsforFoxxapplications.PleasealsomakesurethatyouhaveabackupofallFoxxappsinyour
javascript.app-pathand javascript.dev-app-path.ItissufficienttohavethesourcefilesforFoxxsomewhereelsesoyoucanreinstall
themonerror.Tocheckthateverythinghasworkedduringupgradeyoucouldusetheweb-interfaceApplicationstabor
unix>foxx-managerlist
forallyourdatabases.Thelistedappsshouldbeidenticalbeforeandaftertheupgrade.
procedureonce.ThiscanbedonebyrunningArangoDBfromthecommandlineandsupplyingthe --upgradeoption.Note:Wehave
changedFoxxfolderstructureandimplementedanupgradetasktomoveyourapplicationstothenewstructure.Inordertotellthis
upgradetasktoalsomoveyourdevelopmentFoxxappspleasemakesureyougivethedev-app-pathaswell.Ifyouhavenotused
developmentmodeforFoxxappsyoucandropthe --javascript.dev-app-path.Itisonlypossibletoupgradeonedev-app-path
togetherwithonedatafolder.
unix>arangoddata--upgrade--javascript.dev-app-pathdevapps
where dataisArangoDB'smaindatadirectoryand devappsisthedirectorywhereyoudevelopFoxxapps.
thenrunning
Upgradingto2.5
359
Upgradingto2.5
360
procedureonce.ThiscanbedonebyrunningArangoDBfromthecommandlineandsupplyingthe --upgradeoption:
unix>arangoddata--upgrade
where dataisArangoDB'smaindatadirectory.
thenrunning
Upgradingto2.4
361
Upgradingto2.4
362
2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':Databasedirectoryversion(2.2)islowerthancurrentversion(20300).
thenrunning
Theoutputshouldlooklikethis:
2014-11-03T15:48:47Z[2708]INFOIndatabase'_system':Found24definedtask(s),5task(s)torun
2014-11-03T15:48:47Z[2708]INFOIndatabase'_system':stateprod/standalone/upgrade,tasksupdateUserModel,createStatistics,upgradeClusterPlan,
2014-11-03T15:48:48Z[2708]INFOIndatabase'_system':upgradesuccessfullyfinished
Upgradingto2.3
363
Pleasechecktheoutputthe --upgraderun.Itmayproduceerrors,whichneedtobefixedbeforeArangoDBcanbeusedproperly.Ifno
errorsarepresentortheyhavebeenresolved,youcanstartArangoDB2.3regularly.
Upgradingto2.3
364
PleasereadthefollowingsectionsifyouupgradefromapreviousversiontoArangoDB2.2.
2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':Databasedirectoryversion(2.1)islowerthanserverversion(2.2).
2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':---------------------------------------------------------------------2014-07-07T22:04:53Z[18675]FATALDatabaseversioncheckfailedfor'_system'.Pleasestarttheserverwiththe--upgradeoption
thenrunning
Theoutputshouldlooklikethis:
2014-07-07T22:11:30Z[18867]INFOIndatabase'_system':startingupgradefromversion2.1to2.2.0
2014-07-07T22:11:30Z[18867]INFOIndatabase'_system':Found19definedtask(s),2task(s)torun
2014-07-07T22:11:30Z[18867]INFOIndatabase'_system':upgradesuccessfullyfinished
Upgradingto2.2
365
Pleasechecktheoutputthe --upgraderun.Itmayproduceerrors,whichneedtobefixedbeforeArangoDBcanbeusedproperly.Ifno
errorsarepresentortheyhavebeenresolved,youcanstartArangoDB2.2regularly.
Upgradingto2.2
366
Troubleshooting
Troubleshooting
367
Arangod
IftheArangoDBserverdoesnotstartorifyoucannotconnecttoitusingarangoshorotherclients,youcantrytofindtheproblem
causebyexecutingthefollowingsteps.Iftheserverstartsupwithoutproblemsyoucanskipthissection.
Checktheserverlogfile:Iftheserverhaswrittenalogfileyoushouldcheckitbecauseitmightcontainrelevanterrorcontext
information.
Checktheconfiguration:Theserverlooksforaconfigurationfilenamedarangod.confonstartup.Thecontentsofthisfilewillbe
usedasabaseconfigurationthatcanoptionallybeoverriddenwithcommand-lineconfigurationparameters.Youshouldcheckthe
configfileforthemostrelevantparameterssuchas:
server.endpoint:WhatIPaddressandporttobindto
logparameters:Ifandwheretolog
database.directory:Paththedatabasefilesarestoredin
Iftheconfigurationrevealsthatsomethingisnotconfiguredrighttheconfigfileshouldbeadjustedandtheserverberestarted.
Starttheservermanuallyandcheckitsoutput:Startingtheservermightfailevenbeforeloggingisactivatedsotheserverwillnot
producelogoutput.Thiscanhappeniftheserverisconfiguredtowritethelogstoafilethattheserverhasnopermissionson.In
thiscasetheservercannotloganerrortothespecifiedlogfilebutwillwriteastartuperroronstderrinstead.Startingtheserver
manuallywillalsoallowyoutooverridespecificconfigurationoptions,e.g.toturnon/offfileorscreenloggingetc.
ChecktheTCPport:Iftheserverstartsupbutdoesnotacceptanyincomingconnectionsthismightbeduetofirewallconfiguration
betweentheserverandanyclient(s).TheserverbydefaultwilllistenonTCPport8529.Pleasemakesurethisportisactually
accessiblebyotherclientsifyouplantouseArangoDBinanetworksetup.
Whenusinghostnamesintheconfigurationorwhenconnecting,pleasemakesurethehostnameisactuallyresolvable.Resolving
hostnamesmightinvokeDNS,whichcanbeasourceoferrorsonitsown.
ItisgenerallygoodadvicetonotuseDNSwhenspecifyingtheendpointsandconnectionaddresses.UsingIPaddressesinsteadwill
ruleoutDNSasasourceoferrors.Anotheralternativeistouseahostnamespecifiedinthelocal/etc/hostsfile,whichwillthen
bypassDNS.
Testifcurlcanconnect:Oncetheserverisstarted,youcanquicklyverifyifitrespondstorequestsatall.Thischeckallowsyouto
determinewhetherconnectionerrorsareclient-specificornot.Ifatleastoneclientcanconnect,itislikelythatconnectionproblems
ofotherclientsarenotduetoArangoDB'sconfigurationbutduetoclientorin-betweennetworkconfigurations.
Youcantestconnectivityusingasimplecommandsuchas:
curl--dump--XGEThttp://127.0.0.1:8529/_api/version&&echo
ThisshouldreturnaresponsewithanHTTP200statuscodewhentheserverisrunning.Ifitdoesitalsomeanstheserveris
generallyacceptingconnections.Alternativetoolstocheckconnectivityarelynxorab.
arangod
368
EmergencyConsole
TheArangoDBdatabaseserverhastwomodesofoperation:Asaserver,whereitwillanswertoclientrequestsandasanemergency
console,inwhichyoucanaccessthedatabasedirectly.Thelatter-asthenamesuggests-shouldonlybeusedincaseofanemergency,
forexample,acorruptedcollection.Usingtheemergencyconsoleallowsyoutoissueallcommandsnormallyavailableinactionsand
transactions.Whenstartingtheserverinemergencyconsolemode,theservercannothandleanyclientrequests.
Youshouldneverstartmorethanoneserverusingthesamedatabasedirectory,independentofthemodeofoperation.Normally,
ArangoDBwillpreventyoufromdoingthisbyplacingalockfileinthedatabasedirectoryandnotallowingasecondArangoDBinstance
tousethesamedatabasedirectoryifalockfileisalreadypresent.
InCaseOfDisaster
Thefollowingcommandstartsanemergencyconsole.
Note:Neverstarttheemergencyconsoleforadatabasewhichalsohasaserverattachedtoit.Ingeneral,theArangoDBshelliswhatyou
want.
>./arangod--console--logerror/tmp/vocbase
ArangoDBshell[V8version5.0.71.39,DBversion3.x.x]
arango>1+2;
3
arango>vardb=require("@arangodb").db;db.geo.count();
703
TheemergencyconsoleprovidesaJavaScriptconsoledirectlyrunninginthearangodserverprocess.Thisallowstodebugandexamine
collectionsanddocumentsaswiththenormalArangoDBshell,butwithoutclient/servercommunication.
However,itisverylikelythatyouwillneverneedtheemergencyconsoleunlessyouareanArangoDBdeveloper.
EmergencyConsole
369
DatafileDebugger
InCaseOfDisaster
AranagoDBusesappend-onlyjournals.Datacorruptionshouldonlyoccurwhenthedatabaseserveriskilled.Inthiscase,thecorruption
shouldonlyoccurinthelastobject(s)thathavebeingwrittentothejournal.
Ifacorruptionoccurswithinanormaldatafile,thenthiscanonlyhappenifahardwarefaultoccurred.
Ifajournalordatafileiscorrupt,shutdownthedatabaseserverandstarttheprogram
unix>arango-dfdb
inordertochecktheconsistencyofthedatafilesandjournals.Thisbringsup
_________________
/\___||____/_(_)|___/\/__\/_\
//\/_`|__/_`||_|||/_\//\/__\////_\/
//_//(_||||(_||_|||__///_//\/\//_\\
/___,'\__,_|\__\__,_|_||_|_|\___|/___,'\_____/\____/
Availablecollections:
0:_structures
1:_users
2:_routing
3:_modules
4:_graphs
5:products
6:prices
*:all
Collectiontocheck:
Youcannowselectwhichdatabaseandcollectionyouwanttocheck.Afteryouselectedoneorallofthecollections,aconsistencycheck
willbeperformed.
Checkingcollection#1:_users
Database
path:/usr/local/var/lib/arangodb
Collection
name:_users
identifier:82343
Datafiles
#ofjournals:1
#ofcompactors:1
#ofdatafiles:0
Datafile
path:/usr/local/var/lib/arangodb/collection-82343/journal-1065383.db
type:journal
currentsize:33554432
maximalsize:33554432
totalused:256
#ofentries:3
status:OK
Ifthereisaproblemwithoneofthedatafiles,thenthedatabasedebuggerwillprintitandpromptforwhethertoattempttofixit.
DatafileDebugger
370
WARNING:Thejournalwasnotclosedproperly,thelastentriesarecorrupted.
ThismighthappenArangoDBwaskilledandthelastentrieswerenot
fullywrittentodisk.
Wipethelastentries(Y/N)?
IfyouanswerY,thecorruptedentrywillberemoved.
Ifyouseeacorruptioninadatafile(andnotajournal),thensomethingisterriblywrong.Thesefilesareimmutableandneverchangedby
ArangoDB.Acorruptioninsuchfileisanindicationofahard-diskfailure.
DatafileDebugger
371
Arangobench
ArangobenchisArangoDB'sbenchmarkandtesttool.Itcanbeusedtoissuetestrequeststothedatabaseforperformanceandserver
functiontesting.Itsupportsparallelqueryingandbatchrequests.
Relatedblogposts:
M easuringArangoDBinsertperformance
Gainfactorof5usingbatchrequests
Startupoptions
--async:Sendasynchronousrequests.Thedefaultvalueisfalse.
--batch-size:Numberofoperationstosendperbatch.Use0todisablebatching(thisisthedefault).
--collection:Nameofcollectiontouseintest(onlyrelevantforteststhatinvokecollections).
--complexity:Complexityvaluefortestcase(default:1).M eaningdependsontestcase.
--concurrency:Numberofparallelthreadsthatwillissuerequests(default:1).
--configuration:Readconfigurationfromfile.
--delay:Useastartupdelay.Thisisonlynecessarywhenruninseries.Thedefaultvalueisfalse.
--keep-alive:UseHTTPkeep-alive(default:true).
--progress:Showprogressofbenchmark,onevery20threquest.Settofalsetodisableintermediatelogging.Thedefaultvalueis
true.
--requests:Totalnumberofrequeststoperform(default:1000).
--server.endpoint:Serverendpointtoconnectto,consistingofprotocol,IPaddressandport.Defaultstotcp://localhost:8529.
--server.database:Databasenametousewhenconnecting(default:"_system").
--server.username:Usernametousewhenconnecting(default:"root").
--server.password:Passwordtousewhenconnecting.Don'tspecifythisoptiontogetapasswordprompt.
--server.authentication:Wetherornottoshowthepasswordpromptanduseauthenticationwhenconnectingtotheserver(default:
true).
--test-case:Nameoftestcasetoperform(default:"version").Possiblevalues:
version:requests/_api/version
document:createsdocuments
collection:createscollections
import-document:createsdocumentsviatheimportAPI
hash:Create/Read/Update/Readdocumentsindexedbyahashindex
skiplist:Create/Read/Update/Readdocumentsindexedbyaskiplist
edge:Create/Read/Updateedgedocuments
shapes:Create&Deletedocumentswithheterogeneousattributenames
shapes-append:Createdocumentswithheterogeneousattributenames
random-shapes:Create/Read/Deleteheterogeneousdocumentswithrandomvalues
crud:Create/Read/Update/Delete
crud-append:Create/Read/Update/Readagain
crud-write-read:Create/ReadDocuments
aqltrx:AQLTransactionswithdeepnestedAQL FOR-loops
counttrx:usesJStransactionstocountthedocumentsandinserttheresultagain
multitrx:multipletransactionscombiningreads&writesfromjs
Arangobench
372
multi-collection:multipletransactionscombiningreads&writesfromjsonmultiplecollections
aqlinsert:insertdocumentsviaAQL
aqlv8:executeAQLwithV8functionstoinsertrandomdocuments
--verbose:PrintoutrepliesiftheHTTPheaderindicatesDBerrors.(default:false).
Examples
arangobench
StartsArangobenchwiththedefaultuserandserverendpoint.
--test-caseversion--requests1000--concurrency1
Runsthe'version'testcasewith1000requests,withoutconcurrency.
--test-casedocument--requests1000--concurrency2
Runsthe'document'testcasewith2000requests,withtwoconcurrentthreads.
--test-casedocument--requests1000--concurrency2--asynctrue
Runsthe'document'testcasewith2000requests,withconcurrency2,withasyncrequests.
--test-casedocument--requests1000--concurrency2--batch-size10
Runsthe'document'testcasewith2000requests,withconcurrency2,usingbatchrequests.
Arangobench
373
Architecture
AppendOnly/MVCC
Insteadofoverwritingexistingdocuments,ArangoDBwillcreateanewversionofmodifieddocuments.Thisiseventhecasewhena
documentgetsdeleted.Thetwobenefitsare:
Objectscanbestoredcoherentlyandcompactlyinthemainmemory.
Objectsarepreserved,isolatedwritingandreadingtransactionsallowaccessingtheseobjectsforparalleloperations.
Thesystemcollectsobsoleteversionsasgarbage,recognizingthemasforsaken.Garbagecollectionisasynchronousandrunsparallelto
otherprocesses.
MostlyMemory/Durability
Databasedocumentsarestoredinmemory-mappedfiles.Perdefault,thesememory-mappedfilesaresyncedregularlybutnotinstantly.
Thisisoftenagoodtradeoffbetweenstorageperformanceanddurability.Ifthislevelofdurabilityistoolowforanapplication,the
servercanalsosyncallmodificationstodiskinstantly.Thiswillgivefulldurabilitybutwillcomewithaperformancepenaltyaseach
datamodificationwilltriggerasyncI/Ooperation.
Architecture
374
Write-aheadlog
Startingwithversion2.2ArangoDBstoresalldata-modificationoperationinitswrite-aheadlog.Thewrite-aheadlogissequenceof
append-onlyfilescontainingallthewriteoperationsthatwereexecutedontheserver.
Itisusedtorundatarecoveryafteraservercrash,andcanalsobeusedinareplicationsetupwhenslavesneedtoreplaythesame
sequenceofoperationsasonthemaster.
Bydefault,eachwrite-aheadlogfileis32M Bbig.Thissizeisconfigurableviatheoption--wal.logfile-size.
Whenawrite-aheadlogfileisfull,itissettoread-only,andfollowingoperationswillbewrittenintothenextwrite-aheadlogfile.By
default,ArangoDBwillreservesomesparelogfilesinthebackgroundsoswitchinglogfilesshouldbefast.Howmanyreservelogfiles
ArangoDBwilltrytokeepavailableinthebackgroundcanbecontrolledbytheconfigurationoption--wal.reserve-logfiles.
Datacontainedinfulldatafileswilleventuallybetransferredintothejournalsordatafilesofcollections.Onlythe"surviving"documents
willbecopiedover.Whenallremainingoperationsfromawrite-aheadlogfilehavebeencopiedoverintothejournalsordatafilesofthe
collections,thewrite-aheadlogfilecansafelyberemovedifitisnotusedforreplication.
Long-runningtransactionspreventwrite-aheadlogfilesfrombeingfullygarbage-collectedbecauseitisunclearwhetheratransactionwill
commitorabort.Long-runningtransactionscanthusblockthegarbage-collectionprogressandshouldthereforebeavoidedatallcosts.
Onasystemthatactsasareplicationmaster,itisusefultokeepafewofthealreadycollectedwrite-aheadlogfilessoreplicationslaves
stillcanfetchdatafromthemifrequired.Howmanycollectedlogfileswillbekeptbeforetheygetdeletedisconfigurableviatheoption-wal.historic-logfiles.
Forallwrite-aheadlogconfigurationoptions,pleaserefertothepageWrite-aheadlogoptions.
Write-aheadlog
375
ReleaseNotes
WhatsNew
WhatsNewin3.0
WhatsNewin2.8
WhatsNewin2.7
WhatsNewin2.6
WhatsNewin2.5
WhatsNewin2.4
WhatsNewin2.3
WhatsNewin2.2
WhatsNewin2.1
Incompatiblechanges
AlsoseeUpgradingintheAdministrationchapter.
Releasenotes
376
FeaturesandImprovements
ThefollowinglistshowsindetailwhichfeatureshavebeenaddedorimprovedinArangoDB3.0.ArangoDB3.0alsocontainsseveral
bugfixesthatarenotlistedhere.
Internaldataformatchanges
ArangoDBnowusesVelocyPackforstoringdocuments,queryresultsandtemporarilycomputedvalues.Usingasingledataformat
removedtheneedforsomedataconversionsinthecorethatslowedoperationsdownpreviously.
TheVelocyPackformatisalsoquitecompact,andreducesstoragespacerequirementsfor"small"valuessuchasboolean,integers,short
strings.ThiscanspeedupseveraloperationsinsideAQLqueries.
VelocyPackdocumententriesstoredondiskarealsoself-contained,inthesensethateachstoreddocumentwillcontainallofitsdata
typeandattributenamedescriptions.Whilethismayrequireabitmorespaceforstoringthedocuments,itremovestheoverheadof
fetchingattributenamesanddocumentlayoutfromsharedstructuresasinpreviousversionsofArangoDB.Italsosimplifiesthecode
pathsforstoringandreadingdocuments.
AQLimprovements
Syntaximprovements
LIKEstring-comparisonoperator
AQLnowprovidesa LIKEoperatorandcanbeusedtocomparestringslikethis,forexampleinsidefilterconditions:
valueLIKEsearch
Thischangemakes LIKEanAQLkeyword.Using LIKEasanattributeorcollectionnameinAQLthusrequiresquotingthenamefrom
nowon.
The LIKEoperatoriscurrentlyimplementedbycallingthealreadyexistingAQLfunction LIKE,whichalsoremainsoperationalin3.0.
Usethe LIKEfunctionincaseyouwanttosearchcase-insensitive(optionalparameter),asthe LIKEoperatoralwayscomparescasesensitive.
AQLarraycomparisonoperators
AllAQLcomparisonoperatorsnowalsoexistinanarrayvariant.Inthearrayvariant,theoperatorisprecededwithoneofthekeywords
ALL,ANYorNONE.Usingoneofthesekeywordschangestheoperatorbehaviortoexecutethecomparisonoperationforall,any,or
noneofitslefthandargumentvalues.Itisthereforeexpectedthatthelefthandargumentofanarrayoperatorisanarray.
Examples:
WhatsNewin3.0
377
[1,2,3]ALLIN[2,3,4]//false
[1,2,3]ALLIN[1,2,3]//true
[1,2,3]NONEIN[3]//false
[1,2,3]NONEIN[23,42]//true
[1,2,3]ANYIN[4,5,6]//false
[1,2,3]ANYIN[1,42]//true
[1,2,3]ANY==2//true
[1,2,3]ANY==4//false
[1,2,3]ANY>0//true
[1,2,3]ANY<=1//true
[1,2,3]NONE<99//false
[1,2,3]NONE>10//true
[1,2,3]ALL>2//false
[1,2,3]ALL>0//true
[1,2,3]ALL>=3//false
["foo","bar"]ALL!="moo"//true
["foo","bar"]NONE=="bar"//false
["foo","bar"]ANY=="foo"//true
Regularexpressionstring-comparisonoperators
AQLnowsupportstheoperators=~and!~fortestingstringsagainstregularexpressions.=~testsifastringvaluematchesaregular
expression,and!~testsifastringvaluedoesnotmatcharegularexpression.
Thetwooperatorsexpecttheirleft-handoperandstobestrings,andtheirright-handoperandstobestringscontainingvalidregular
expressionsasspecifiedbelow.
Theregularexpressionsmayconsistofliteralcharactersandthefollowingcharactersandsequences:
.–thedotmatchesanysinglecharacterexceptlineterminators.Toincludelineterminators,use [\s\S]insteadtosimulate .
withDOTALLflag.
\d–matchesasingledigit,equivalentto [0-9]
\s–matchesasinglewhitespacecharacter
\S–matchesasinglenon-whitespacecharacter
\t–matchesatabcharacter
\r–matchesacarriagereturn
\n–matchesaline-feedcharacter
[xyz]–setofcharacters.matchesanyoftheenclosedcharacters(i.e.x,yorzinthiscase
[^xyz]–negatedsetofcharacters.matchesanyothercharacterthantheenclosedones(i.e.anythingbutx,yorzinthiscase)
[x-z]–rangeofcharacters.M atchesanyofthecharactersinthespecifiedrange,e.g. [0-9A-F]tomatchanycharacterin
0123456789ABCDEF
[^x-z]–negatedrangeofcharacters.M atchesanyothercharacterthantheonesspecifiedintherange
(xyz)–definesandmatchesapatterngroup
(x|y)–matcheseitherxory
^–matchesthebeginningofthestring(e.g. ^xyz)
![–matchestheendofthestring(e.g. xyz](http://quicklatex.com/cache3/13/ql_45973f7341701a4fbe493149c7a4dd13_l3.png"–
matchestheendofthestring(e.g. xyz"))
Notethatthecharacters ., *, ?, [, ], (, ), {, }, ^,and $haveaspecialmeaninginregularexpressionsandmay
needtobeescapedusingabackslash( \\).Aliteralbackslashshouldalsobeescapedusinganotherbackslash,i.e. \\\\.
Charactersandsequencesmayoptionallyberepeatedusingthefollowingquantifiers:
x*–matcheszeroormoreoccurrencesofx
x+–matchesoneormoreoccurrencesofx
x?–matchesoneorzerooccurrencesofx
x{y}–matchesexactlyyoccurrencesofx
x{y,z}–matchesbetweenyandzoccurrencesofx
x{y,}–matchesatleastyoccurencesofx
Enclosingidentifiersinforwardticks
WhatsNewin3.0
378
AQLidentifierscannowoptionallybeenclosedinforwardticksinadditiontousingbackwardticks.Thisallowsconvenientwritingof
AQLqueriesinJavaScripttemplatestrings(whicharedelimitedwithbackticksthemselves),e.g.
varq=`FORdocINćollection´RETURNdoc.ńame´`;
Functionsadded
ThefollowingAQLfunctionshavebeenaddedin3.0:
REGEX_TEST(value,regex):testswhetherthestringvaluematchestheregularexpressionspecifiedinregex.Returnstrueifit
matches,andfalseotherwise.
Thesyntaxforregularexpressionsisthesameasfortheregularexpressionoperators=~and!~.
HASH(value):Calculatesahashvalueforvalue.valueisnotrequiredtobeastring,butcanhaveanydatatype.Thecalculatedhash
valuewilltakethedatatypeofvalueintoaccount,soforexamplethenumber1andthestring"1"willhavedifferenthashvalues.
Forarraysthehashvalueswillbecrearedifthearrayscontainexactlythesamevalues(includingvaluetypes)inthesameorder.For
objectsthesamehashvalueswillbecreatediftheobjectshaveexactlythesameattributenamesandvalues(includingvaluetypes).
Theorderinwhichattributesappearinsideobjectsisnotimportantforhashing.Thehashvaluereturnedbythisfunctionisa
number.ThehashalgorithmisnotguaranteedtoremainthesameinfutureversionsofArangoDB.Thehashvaluesshouldtherefore
beusedonlyfortemporarycalculations,e.g.tocompareiftwodocumentsarethesame,orforgroupingvaluesinqueries.
TYPENAME(value):Returnsthedatatypenameofvalue.Thedatatypenamecanbeeithernull,bool,number,string,arrayor
object.
LOG(value):Returnsthenaturallogarithmofvalue.ThebaseisEuler'sconstant(2.71828...).
LOG2(value):Returnsthebase2logarithmofvalue.
LOG10(value):Returnsthebase10logarithmofvalue.
EXP(value):ReturnsEuler'sconstant(2.71828...)raisedtothepowerofvalue.
EXP2(value):Returns2raisedtothepowerofvalue.
SIN(value):Returnsthesineofvalue.
COS(value):Returnsthecosineofvalue.
TAN(value):Returnsthetangentofvalue.
ASIN(value):Returnsthearcsineofvalue.
ACOS(value):Returnsthearccosineofvalue.
ATAN(value):Returnsthearctangentofvalue.
ATAN2(y,x):Returnsthearctangentofthequotientofyandx.
RADIANS(value):Returnstheangleconvertedfromdegreestoradians.
DEGREES(value):Returnstheangleconvertedfromradianstodegrees.
Optimizerimprovements
"inline-subqueries"rule
TheAQLoptimizerrule"inline-subqueries"hasbeenadded.Thisrulecanpulloutcertainsubqueriesthatareusedasanoperandtoa
FORlooponelevelhigher,eliminatingthesubquerycompletely.Thisreducescomplexityofthequery'sexecutionplanandwilllikely
enablefurtheroptimizations.Forexample,thequery
WhatsNewin3.0
379
FORiIN(
FORjIN[1,2,3]
RETURNj
)
RETURNi
willbetransformedbytheruleto:
FORiIN[1,2,3]
RETURNi
Thequery
FORnameIN(
FORdocIN_users
FILTERdoc.status==1
RETURNdoc.name
)
LIMIT2
RETURNname
willbetransformedinto
FORtmpIN_users
FILTERtmp.status==1
LIMIT2
RETURNtmp.name
Therulewillonlyfirewhenthesubqueryisusedasanoperandtoa FORloop,andifthesubquerydoesnotcontaina COLLECTwithan
INTOvariable.
"remove-unnecessary-calculations"rule
TheAQLoptimizerrule"remove-unnecessary-calculations"nowfiresinmorecasesthaninpreviousversions.Thisruleremoves
calculationsfromexecutionplans,andbyhavinglesscalculationsdone,aquerymayexecutefasterorrequireslessmemory.
Therulewillnowremovecalculationsthatareusedexactlyonceinotherexpressions(e.g. LETa=docRETURNa.value)and
calculations,orcalculationsthatarejustreferencestoothervariables(e.g. LETa=b).
"optimize-traversals"rule
TheAQLoptimizerrule"merge-traversal-filter"wasrenamedto"optimize-traversals".Therulewillremoveunusededgeandpathresult
variablesfromthetraversalincasetheyarespecifiedinthe FORsectionofthetraversal,butnotreferencedlaterinthequery.Thissaves
constructingedgesandpathsresultsthatarenotusedlater.
AQLnowusesVelocyPackinternallyforstoringintermediatevalues.Formanyvaluetypesitcannowgetawaywithoutextramemory
allocationsandlessinternalconversions.ValuescanbepassedintointernalAQLfunctionswithoutcopyingthem.Thiscanleadto
reducedqueryexecutiontimesforqueriesthatuseC++-basedAQLfunctions.
"replace-or-with-in"and"use-index-for-sort"rules
Theserulesnowfireinsomeadditionalcases,whichallowssimplifyingindexlookupconditionsandremovingSortNodesfromexecution
plans.
Clusterstatemanagement
Thecluster'sinternalstateinformationisnowalsomanagedbyArangoDBinstances.Earlierversionsreliedonthirdpartysoftwarebeing
installedforthestoringtheclusterstate.ThestateismanagedbydedicatedArangoDBinstances,whichcanbestartedinaspecialagency
mode.Theseinstancescanoperateinadistributedfashion.Theywillautomaticallyelectoneofthemtobecometheirleader,being
WhatsNewin3.0
380
responsibileforstoringthestatechangessentfromserversinthecluster.Theotherinstanceswillautomaticallyfollowtheleaderandwill
transparentlystandinshoulditbecomeunavailable.Theagencyinstancesarealsoself-organizing:theywillcontinuouslyprobeeach
otherandre-electleaders.Thecommunicationbetweentheagencyinstancesusetheconsensus-basedRAFTprotocol.
TheoperationsforstoringandretrievingclusterstateinformationarenowmuchlessexpensivefromanArangoDBclusternode
perspective,whichinturnallowsforfasterclusteroperationsthatneedtofetchorupdatetheoverallclusterstate.
_fromand _toattributesofedgesareupdatableandusablein
indexes
InArangoDBpriorto3.0theattributes _fromand _toofedgesweretreatedspeciallywhenloadingorstoringedges.Thatspecial
handlingledtotheseattributesbeingnotasflexibleasregulardocumentattributes.Forexample,the _fromand _toattributevaluesof
anexistingedgecouldnotbeupdatedoncetheedgewascreated.Nowthisispossibleviathesingle-documentAPIsandviaAQL.
Additionally,the _fromand _toattributescouldnotbeindexedinuser-definedindexes,e.g.tomakeeachcombinationof _fromand
_tounique.Finally,as _fromand _toreferencedthelinkedcollectionsbycollectionidandnotbycollectionname,theirmeaning
becameunclearonceareferencedcollectionwasdropped.Thecollectionidstoredinedgesthenbecameunusable,andwhenaccessing
suchedgethecollectionnamepartofitwasalwaystranslatedto _undefined.
InArangoDB3.0,the _fromand _tovaluesofedgesaresavedasregularstrings.Thisallowsusing _fromand _toinuser-defined
indexes.Additionally,thisallowstoupdatethe _fromand _tovaluesofexistingedges.Furthermore,collectionsreferencedby _from
and _tovaluesmaybedroppedandre-createdlater.Any _fromand _tovaluesofedgespointingtosuchdroppedcollectionare
unaffectedbythedropoperationnow.
UnifiedAPIsforCRUDoperations
TheCRUDAPIsfordocumentsandedgehavebeenunified.EdgescannowbeinsertedandmodifiedviathesameAPIsasdocuments.
_fromand _toattributevaluescanbepassedasregulardocumentattributesnow:
db.myedges.insert({_from:"myvertices/some",_to:"myvertices/other",...});
Passing _fromand _toseparatelyasitwasrequiredinearlierversionsisnotnecessaryanymorebutwillstillwork:
db.myedges.insert("myvertices/some","myvertices/other",{...});
TheCRUDoperationsnowalsosupportbatchvariantsthatworksonarraysofdocuments/edges,e.g.
db.myedges.insert([
{_from:"myvertices/some",_to:"myvertices/other",...},
{_from:"myvertices/who",_to:"myvertices/friend",...},
{_from:"myvertices/one",_to:"myvertices/two",...},
]);
ThebatchvariantsarealsoavailableinArangoDB'sHTTPAPI.Theycanbeusedtomoreefficientlycarryoutoperationswithmultiple
documentsthantheirsingle-documentequivalents,whichrequiredoneHTTPrequestperoperation.Withthebatchoperations,the
HTTPrequest/responseoverheadcanbeamortizedacrossmultipleoperations.
Persistentindexes
ArangoDB3.0providesanexperimentalpersistentindexfeature.Persistentindexesstoretheindexvaluesondiskinsteadofin-memory
only.Thismeanstheindexesdonotneedtoberebuiltin-memorywhenacollectionisloadedorreloaded,whichshouldimprove
collectionloadingtimes.
ThepersistentindexesinArangoDBarebasedontheRocksDBengine.Tocreateapersistentindexforacollection,createanindexof
type"rocksdb"asfollows:
WhatsNewin3.0
381
db.mycollection.ensureIndex({type:"rocksdb",fields:["fieldname"]});
Thepersistentindexesaresorted,sotheyallowequalitylookupsandrangequeries.Notethatthefeatureisstillhighlyexperimentaland
hassomeknowndeficiencies.Itwillbefinalizeduntilthereleaseofthe3.0stableversion.
UpgradedV8version
TheV8enginethatisusedinsideArangoDBtoexecuteJavaScriptcodehasbeenupgradedfromversion4.3.61to5.0.71.39.Thenew
versionmakesseveralmoreES6featuresavailablebydefault,including
arrowfunctions
computedpropertynames
restparameters
arraydestructuring
numericandobjectliterals
WebAdminInterface
TheArangoDB3.0webinterfaceissignificantlyimproved.Itnowcomeswithamoreresponsivedesign,makingiteasiertouseon
differentdevices.Navigationandmenushavebeensimplified,andrelateditemshavebeenregroupedtostayclosertogetherandallow
tighterworkflows.
TheAQLqueryeditorisnowmucheasiertouse.M ultiplequeriescanbestartedandtrackedinparallel,whileresultsofearlierqueries
arestillpreserved.Queriesstillrunningcanbecanceleddirectlyfromtheeditor.TheAQLqueryeditornowallowstheusageofbind
parameterstoo,andprovidesahelperforfindingcollectionnames,AQLfunctionnamesandkeywordsquickly.
Thewebinterfacenowkeepstrackofwhethertheserverisofflineandofwhichserver-sideoperationshavebeenstartedandarestill
running.Itnowremainsusablewhilesuchlonger-runningoperationsareongoing.Italsokeepsmorestateaboutuser'schoices(e.g.
windowssizes,whetherthetreeorthecodeviewwaslastusedinthedocumenteditor).
Clusterstatisticsarenowintegratedintothewebinterfaceaswell.Additionally,amenuitem"Helpus"hasbeenaddedtoeasilyprovide
theArangoDBteamfeedbackabouttheproduct.
Thefrontendmaynowbemountedbehindareverseproxyonadifferentpath.ForthistoworktheproxyshouldsendaX-Script-Name
headercontainingthepath.
Abackendconfigurationforhaproxymightlooklikethis:
reqaddX-Script-Name:\/arangodb
Thefrontendwillrecognizethesubpathandproduceappropriatelinks.ArangoDBwillonlyacceptpathsfromtrustedfrontendproxies.
Trustedproxiesmaybeaddedonstartup:
--frontend.proxy-request-checktrue--frontend.trusted-proxy192.168.1.117
--frontend.trusted-proxymaybeanyaddressornetmask.
Todisablethecheckandblindlyacceptanyx-script-nameset--frontend.proxy-request-checktofalse.
Foxximprovements
TheFoxxframeworkhasbeencompletelyrewrittenfor3.0withanew,simplerandmorefamiliarAPI.Themostnotablechangesare:
Legacymodefor2.8services
Stuckwitholdcode?Youcancontinueusingyour2.8-compatibleFoxxserviceswith3.0byadding "engines":{"arangodb":
"^2.8.0"}(orsimilarversionrangesthatexclude3.0andup)totheservicemanifest.
WhatsNewin3.0
382
Nomoreglobalvariablesandmagicalcomments
The applicationContextisnow module.context.Insteadofmagicalcommentsjustusethe summaryand descriptionmethods
todocumentyourroutes.
RepositoryandM odelhavebeenremoved
InsteadofrepositoriesjustuseArangoDBcollectionsdirectly.Forvalidationsimplyusethejoischemas(butwrappedin
joi.object())thatpreviouslylivedinsidethemodel.CollectionsandqueriesreturnplainJavaScriptobjects.
Controllershavebeenreplacedwithnestablerouters
Createrouterswith require('@arangodb/foxx/router')(),attachthemtoyourservicewith module.context.use(router).Because
routersarenolongermountedautomagically,youcanexportandimportthemlikeanyotherobject.Use router.use('/path',
subRouter)tonestroutersasdeeplyasyouwant.
Routescanbenamedandreversed
NomorememorizingURLs:addanametoyourroutelike router.get('/hello/:name',function(){...},'hello')andredirectto
thefullURLwith res.redirect(req.resolve('hello',{name:'world'})).
Simplerexpress-likemiddleware
Ifyoualreadyknowexpress,thisshouldbefamiliar.Here'sarequestloggerinthreelinesofcode:
varstart=Date.now();
try{next();}
finally{console.log(`${req.method}${req.url}${res.statusCode}${Date.now()-start}ms`);}
});
Sessionsandauthwithoutdependencies
Tomakeiteasiertogetstarted,thefunctionalitypreviouslyprovidedbythe simple-auth, oauth2, sessions-localand
sessions-jwtserviceshavebeenmovedintoFoxxasthe @arangodb/foxx/auth, @arangodb/foxx/oauth2and
@arangodb/foxx/sessionsmodules.
Logging
ArangoDB'sloggingisnowgroupedintotopics.Thelogverbosityandoutputfilescanbeadjustedperlogtopic.Forexample
--log.levelstartup=trace--log.levelqueries=trace--log.levelinfo
willlogmessagesconcerningstartupattracelevel,AQLqueriesattracelevelandeverythingelseatinfolevel. --log.levelcanbe
specifiedmultipletimesatstartup,forasmanytopicsasneeded.
mmap:informationaboutmemory-mappingoperations(includingmsync)
queries:executedAQLqueries,slowqueries
Thisalsoallowsdirectinglogoutputtodifferentfilesbasedontopics.Forexample,tologallAQLqueriestoafile"queries.log"onecan
usetheoptions:
--log.levelqueries=trace--log.outputqueries=file:///path/to/queries.log
WhatsNewin3.0
383
ToadditionallylogHTTPrequesttoafilenamed"requests.log"addtheoptions:
--log.levelrequests=info--log.outputrequests=file:///path/to/requests.log
Buildsystem
ArangoDBnowusesthecross-platformbuildsystemCM akeforallitsbuilds.Previousversionsusedtwodifferentbuildsystems,
makingdevelopmentandcontributionsharderthannecessary.Nowthebuildsystemisunified,andalltargets(Linux,Windows,M acOS)
arebuiltfromthesamesetofbuildinstructions.
Documentation
Thedocumentationhasbeenenhancedandre-organizedtobemoreintuitive.
AnewintroductionforbeginnersshouldbringyouuptospeedwithArangoDBinlessthananhour.Additionaltopicshavebeen
introducedandwillbeextendedwithupcomingreleases.
ThetopicsAQLandHTTPAPIarenowseparatedfromthemanualforbettersearchabilityandlessconfusion.Aversionswitcher
makesiteasiertojumptotheversionofthedocsyouareinterestedin.
WhatsNewin3.0
384
bugfixesthatarenotlistedhere.Foralistofbugfixes,pleaseconsulttheCHANGELOG.
AQLimprovements
AQLGraphTraversals/PatternMatching
AQLoffersanewfeaturetotraverseoveragraphwithoutwritingJavaScriptfunctionsbutwithalltheotherfeaturesyouknowfrom
AQL.Forthispurpose,aspecialversionof FORvariableNameINexpressionhasbeenintroduced.
Thisspecialversionhasthefollowingformat: FORvertex-variable,edge-variable,path-variableINtraversal-expression,where
traversal-expressionhasthefollowingformat: [depth]directionstart-vertexgraph-definitionwiththefollowinginput
parameters:
depth(optional):defineshowmanystepsareexecuted.Thevaluecaneitherbeanintegervalue(e.g. 3)orarangeofintegervalues
(e.g. 1..5).Thedefaultis1.
direction:defineswhichedgedirectionsarefollowed.Canbeeither OUTBOUND, INBOUNDor ANY.
start-vertex:defineswherethetraversalisstarted.M ustbean _idvalueoradocument.
graph-definition:defineswhichedgecollectionsareusedforthetraversal.M ustbeeither GRAPHgraph-nameforgraphscreatedwith
thegraph-module,oralistofedgecollections edge-col1,edge-col2,..edge-colN.
Thethreeoutputvariableshavethefollowingsemantics:
vertex-variable:Thelastvisitedvertex.
edge-variable:Thelastvisitededge(optional).
path-variable:Thecompletepathfromstart-vertextovertex-variable(optional).
Thetraversalstatementcanbeusedinthesamewayastheoriginal FORvariableNameINexpression,andcanbecombinedwithfilters
andotherAQLconstructs.
AsanexampleonecannowfindthefriendsofafriendforacertainuserwiththisAQLstatement:
FORfoaf,e,pathIN2ANY@startUserGRAPH"relations"
FILTERpath.edges[0].type=="friend"
FILTERpath.edges[1].type=="friend"
FILTERfoaf._id!=@startUser
RETURNDISTINCTfoaf
Optimizerruleshavebeenimplementedtogainperformanceofthetraversalstatement.Theserulesmovefilterstatementsintothe
traversalstatements.t.pathswhichcanneverpassthefilterarenotemittedtothevariables.
Asanexampletakethequeryaboveandassumethereareedgesthatdonothave type=="friend".Ifinthefirstedgestepthereissuch
anon-friendedgethesecondstepswillneverbecomputedfortheseedgesastheycannotfulfillthefiltercondition.
ArrayIndexes
Hashindexesandskiplistindexescannowoptionallybedefinedforarrayvaluessothattheyindexindividualarraymembersinsteadof
theentirearrayvalue.
Todefineanindexforarrayvalues,theattributenameisextendedwiththeexpansionoperator [*]intheindexdefinition.
Example:
db._create("posts");
db.posts.ensureHashIndex("tags[*]");
Whengiventhefollowingdocument
WhatsNewin2.8
385
{
"tags":[
"AQL",
"ArangoDB",
"Index"
]
}
thisindexwillnowcontaintheindividualvalues "AQL", "ArangoDB"and "Index".
Nowtheindexcanbeusedforfindingalldocumentshaving "ArangoDB"somewhereintheir tagsarrayusingthefollowingAQL
query:
FORdocINposts
FILTER"ArangoDB"INdoc.tags[*]
RETURNdoc
Itisalsopossibletocreateanindexonsub-attributesofarrayvalues.Thismakessensewhentheindexattributeisanarrayofobjects,
e.g.
db._drop("posts");
db._create("posts");
db.posts.ensureIndex({type:"hash",fields:["tags[*].name"]});
db.posts.insert({tags:[{name:"AQL"},{name:"ArangoDB"},{name:"Index"}]});
db.posts.insert({tags:[{name:"AQL"},{name:"2.8"}]});
Thefollowingquerywillthenusethearrayindex:
FORdocINposts
FILTER'AQL'INdoc.tags[*].name
RETURNdoc
Arrayvalueswillautomaticallybede-duplicatedbeforebeinginsertedintoanarrayindex.
PleasenotethatfilteringusingarrayindexesonlyworksfromwithinAQLqueriesandonlyifthequeryfiltersontheindexedattribute
usingthe INoperator.Theothercomparisonoperators( ==, !=, >, >=, <, <=)currentlydonotusearrayindexes.
TheAQLqueryoptimizercannowuseindexesifmultiplefilterconditionsonattributesofthesamecollectionarecombinedwithlogical
ORs,andiftheusageofindexeswouldcompletelycovertheseconditions.
Forexample,thefollowingqueriescannowusetwoindependentindexeson value1and value2(thelatterqueryrequiresthatthe
indexesareskiplistindexesduetousageofthe <and >comparisonoperators):
FORdocINcollectionFILTERdoc.value1<42||doc.value2>23RETURNdoc
Thenewoptimizerrule"sort-in-values"cannowpre-sorttheright-handsideoperandof INand NOTINoperatorssotheoperation
canuseabinarysearchwithlogarithmiccomplexityinsteadofalinearsearch.Therulewillbeappliedwhentheright-handsideoperand
ofan INor NOTINoperatorinafilterconditionisavariablethatisdefinedinadifferentloop/scopethantheoperatoritself.
Additionally,thefilterconditionmustconsistofsolelythe INor NOTINoperationinordertoavoidanyside-effects.
Therulewillkickinforaqueriessuchasthefollowing:
LETvalues=/*someruntimeexpressionhere*/
FORdocINcollection
FILTERdoc.valueINvalues
RETURNdoc
Itwillnotbeappliedforthefollowigqueries,becausetheright-handsideoperandofthe INiseithernotavariable,orbecausethe
FILTERconditionmayhavesideeffects:
WhatsNewin2.8
386
FORdocINcollection
FILTERdoc.valueIN/*someruntimeexpressionhere*/
RETURNdoc
LETvalues=/*someruntimeexpressionhere*/
FORdocINcollection
FILTERFUNCTION(doc.values)==23&&doc.valueINvalues
RETURNdoc
AQLfunctionsadded
ThefollowingAQLfunctionshavebeenaddedin2.8:
POW(base,exponent):returnsthebasetotheexponentexp
UNSET_RECURSIVE(document,attributename,...):recursivelyremovestheattributesattributename(canbeoneormany)from
documentanditssub-documents.Allotherattributeswillbepreserved.M ultipleattributenamescanbespecifiedbyeitherpassing
multipleindividualstringargumentnames,orbypassinganarrayofattributenames:
UNSET_RECURSIVE(doc,'_id','_key','foo','bar')
UNSET_RECURSIVE(doc,['_id','_key','foo','bar'])
IS_DATESTRING(value):returnstrueifvalueisastringthatcanbeusedinadatefunction.Thisincludespartialdatessuchas2015
or2015-10andstringscontaininginvaliddatessuchas2015-02-31.Thefunctionwillreturnfalseforallnon-stringvalues,evenif
someofthemmaybeusableindatefunctions.
Miscellaneousimprovements
theArangoShellnowprovidestheconveniencefunction db._explain(query)forretrievingahuman-readableexplanationofAQL
queries.Thisfunctionisashorthandfor require("org/arangodb/aql/explainer").explain(query).
theAQLqueryoptimizernowautomaticallyconverts LENGTH(collection-name)toanoptimizedexpressionthatreturnsthe
numberofdocumentsinacollection.PreviousversionsofArangoDBreturnedawarningwhenusingthisexpressionandalso
enumeratedalldocumentsinthecollection,whichwasinefficient.
improvedperformanceofskippingovermanydocumentsinanAQLquerywhennoindexesandnofiltersareused,e.g.
FORdocINcollection
LIMIT1000000,10
RETURNdoc
addedclusterexecutionsiteinfoinexecutionplanexplainoutputforAQLqueries
for30+AQLfunctionsthereisnowanadditionalimplementationinC++thatremovestheneedforinternaldataconversionwhen
thefunctioniscalled
theAQLeditorinthewebinterfacenowsupportsusingbindparameters
Deadlockdetection
ArangoDB2.8nowhasanautomaticdeadlockdetectionfortransactions.
Adeadlockisasituationinwhichtwoormoreconcurrentoperations(usertransactionsorAQLqueries)trytoaccessthesameresources
(collections,documents)andneedtowaitfortheotherstofinish,butnoneofthemcanmakeanyprogress.
Incaseofsuchadeadlock,therewouldbenoprogressforanyoftheinvolvedtransactions,andnoneoftheinvolvedtransactionscould
evercomplete.Thisiscompletelyundesirable,sothenewautomaticdeadlockdetectionmechanisminArangoDBwillautomaticallykick
inandabortoneofthetransactionsinvolvedinsuchadeadlock.Abortingmeansthatallchangesdonebythetransactionwillberolled
backanderror29( deadlockdetected)willbethrown.
WhatsNewin2.8
387
Clientcode(AQLqueries,usertransactions)thataccessesmorethanonecollectionshouldbeawareofthepotentialofdeadlocksand
shouldhandletheerror29( deadlockdetected)properly,eitherbypassingtheexceptiontothecallerorretryingtheoperation.
Replication
Thefollowingimprovementsforreplicationhavebeenmadein2.8(note:mostofthemhavebeenbackportedtoArangoDB2.7aswell):
added autoResyncconfigurationparameterforcontinuousreplication.
Whensetto true,areplicationslavewillautomaticallytriggerafulldatare-synchronizationwiththemasterwhenthemaster
cannotprovidethelogdatatheslavehadaskedfor.Notethat autoResyncwillonlyworkwhentheoption requireFromPresentis
alsosetto trueforthecontinuousreplication,orwhenthecontinuoussyncerisstartedanddetectsthatnostarttickispresent.
Automaticre-synchronizationmaytransferalotofdatafromthemastertotheslaveandmaybeexpensive.Itisthereforeturnedoff
bydefault.Whenturnedoff,theslavewillneverperformanautomaticre-synchronizationwiththemaster.
added idleMinWaitTimeand idleMaxWaitTimeconfigurationparametersforcontinuousreplication.
Theseparameterscanbeusedtocontroltheminimumandmaximumwaittimetheslavewill(intentionally)idleandnotpollfor
masterlogchangesincasethemasterhadsentthefulllogsalready.The idleMaxWaitTimevaluewillonlybeusedwhen
adapativePollingissetto true.When adaptivePollingisdisabled,only idleMinWaitTimewillbeusedasaconstanttime
spaninwhichtheslavewillnotpollthemasterforfurtherchanges.Thedefaultvaluesare0.5secondsfor idleMinWaitTimeand2.5
secondsfor idleMaxWaitTime,whichcorrespondtothehard-codedvaluesusedinpreviousversionsofArangoDB.
added initialSyncMaxWaitTimeconfigurationparameterforinitialandcontinuousreplication
Thisoptioncontrolsthemaximumwaittime(inseconds)thattheinitialsynchronizationwillwaitforaresponsefromthemaster
whenfetchinginitialcollectiondata.Ifnoresponseisreceivedwithinthistimeperiod,theinitialsynchronizationwillgiveupand
fail.ThisoptionisalsorelevantforcontinuousreplicationincaseautoResyncissettotrue,asthenthecontinuousreplicationmay
triggerafulldatare-synchronizationincasethemastercannotthelogdatatheslavehadaskedfor.
HTTPrequestssentfromtheslavetothemasterduringinitialsynchronizationwillnowberetriediftheyfailwithconnection
problems.
theinitialsynchronizationnowlogsitsprogresssoitcanbequeriedusingtheregularreplicationstatuscheckAPIs.
added asyncattributefor syncand syncCollectionoperationscalledfromtheArangoShell.Setthingthisattributeto truewill
makethesynchronizationjobontheservergointothebackground,sothattheshelldoesnotblock.Thestatusofthestarted
asynchronoussynchronizationjobcanbequeriedfromtheArangoShelllikethis:
/*startsinitialsynchronization*/
varreplication=require("org/arangodb/replication");
varid=replication.sync({
username:"myuser",
async:true
});
/*nowquerytheidofthereturnedasyncjobandprintthestatus*/
Theresultof getSyncResult()willbe falsewhiletheserver-sidejobhasnotcompleted,anddifferentto falseifithas
completed.Whenithascompleted,alljobresultdetailswillbereturnedbythecallto getSyncResult().
thewebadmininterfacedashboardnowshowsaserver'sreplicationstatusatthebottomofthepage
WebAdminInterface
Thefollowingimprovementshavebeenmadeforthewebadmininterface:
theAQLeditornowhassupportforbindparameters.Thebindparametervaluescanbeeditedinthewebinterfaceandsavedwitha
queryforfutureuse.
WhatsNewin2.8
388
theAQLeditornowallowscancelingrunningqueries.Thiscanbeusedtocancellong-runningquerieswithoutswitchingtothequery
managementsection.
thedashboardnowprovidesinformationabouttheserver'sreplicationstatusatthebottomofthepage.Thiscanbeusedtotrack
eitherthestatusofaone-timesynchronizationorthecontinuousreplication.
thecompactionstatusandsomestatusinternalsaboutcollectionsarenowdisplayedinthedetailviewforacollectionintheweb
interface.Thesedatacanbeusedfordebuggingcompactionissues.
unloadingacollectionviathewebinterfacewillnowtriggergarbagecollectioninallv8contextsandforceaWALflush.This
increasesthechancesofperfomingtheunloadfaster.
thestatusterminologyforcollectionsforwhichanunloadrequesthasbeenissuedviathewebinterfacewaschangedfrom inthe
processofbeingunloadedto willbeunloaded.Thisismoreaccurateastheactualunloadmaybepostponeduntillaterifthere
arestillreferencespointingtodatainthecollection.
Foxximprovements
themoduleresolutionusedby requirenowbehavesmorelikeinnode.js
the org/arangodb/requestmodulenowreturnsresponsebodiesforerrorresponsesbydefault.Theoldbehaviorofnotreturning
bodiesforerrorresponsescanbere-enabledbyexplicitlysettingtheoption returnBodyOnErrorto false
Miscellaneouschanges
Thestartupoption --server.hide-product-headercanbeusedtomaketheservernotsendtheHTTPresponseheader "Server:
ArangoDB"initsHTTPresponses.ThiscanbeusedtoconcealtheservermakefromHTTPclients.Bydefault,theoptionisturnedoff
sotheheaderisstillsentasusual.
arangodumpandarangorestorenowhavebettererrorreporting.Additionally,arangodumpwillnowfailbydefaultwhentryingtodump
edgesthatrefertoalreadydroppedcollections.Thiscanbecircumventedbyspecifyingtheoption --forcetruewheninvoking
arangodump.
arangoimpnowprovidesanoption --create-collection-typetospecifythetypeofthecollectiontobecreatedwhen --createcollectionissetto true.Previously --create-collectionalwayscreateddocumentcollectionsandthecreationofedgecollections
wasnotpossible.
WhatsNewin2.8
389
Performanceimprovements
Indexbuckets
Theprimaryindexesandhashindexesofcollectionscannowbesplitintomultipleindexbuckets.Thisoptionwasavailableforedge
indexesonlyinArangoDB2.6.
Abucketcanbeconsideredacontainerforaspecificrangeofindexvalues.Forprimary,hashandedgeindexes,determiningthe
responsiblebucketforanindexvalueisdonebyhashingtheactualindexvalueandapplyingasimplearithmeticoperationonthehash.
Becauseanindexvaluewillbepresentinatmostonebucketandbucketsareindependent,usingmultiplebucketsprovidesthefollowing
benefits:
initiallybuildingthein-memoryindexdatacanbeparallelizedevenforasingleindex,withonethreadperbucket(orwiththreads
beingresponsibleformorethanonebucketatatime).Thiscanhelpreducingtheloadingtimeforcollections.
resizinganindexwhenitisabouttorunoutofreservespaceisperformedperbucket.Aseachbucketonlycontainsafractionofthe
entireindex,resizingandrehashingabucketismuchfasterandlessintrusivethanresizingandrehashingtheentireindex.
Whencreatingnewcollections,thedefaultnumberofindexbucketsis 8sinceArangoDB2.7.Inpreviousversions,thedefaultvalue
was 1.Thenumberofbucketscanalsobeadjustedforexistingcollectionssotheycanbenefitfromtheoptimizations.Thenumberof
indexbucketscanbesetforacollectionatanytimebyusingacollection's propertiesfunction:
db.collection.properties({indexBuckets:16});
Thenumberofindexbucketsmustbeapowerof2.
Pleasenotethatforbuildingtheindexdataformultiplebucketsinparallelitisrequiredthatacollectioncontainsasignificantamountof
documentsbecauseforalownumberofdocumentstheoverheadofparallelizationwilloutweighitsbenefits.Thecurrentthresholdvalue
is256kdocuments,butthisvaluemaychangeinfutureversionsofArangoDB.Additionally,theconfigurationoption --database.indexthreadswilldeterminehowmanyparallelthreadsmaybeusedforbuildingtheindexdata.
Fasterupdateandremoveoperationsinnon-uniquehashindexes
TheuniquehashindexesinArangoDBprovidedanamortizedO(1)lookup,insert,updateandremoveperformance.Non-uniquehash
indexesprovidedamortizedO(1)insertperformance,buthadworseperformanceforupdateandremoveoperationsfornon-unique
values.Fordocumentswiththesameindexvalue,theymaintainedalistofcollisions.Whenadocumentwasupdatedorremoved,that
exactdocumenthadtobefoundinthecollisionslistfortheindexvalue.WhilegettingtothestartofthecollisionslistwasO(1),scanning
thelisthadO(n)performanceintheworstcase(withnbeingthenumberofdocumentswiththesameindexvalue).Overall,thismade
updateandremoveoperationsinnon-uniquehashindexesslowiftheindexcontainedmanyduplicatevalues.
ThishasbeenchangedinArangoDB2.7sothatnon-uniquehashindexesnowalsoprovideupdateandremoveoperationswithan
amortizedcomplexityofO(1),eveniftherearemanyduplicates.
Resizingnon-uniquehashindexesnowalsodoesn'trequirelookingintothedocumentdata(whichmayinvolveadiskaccess)becausethe
indexmaintainssomeinternalcachevalueperdocument.Whenresizingandrehashingtheindex(oranindexbucket),theindexwillfirst
compareonlythecachevaluesbeforepeekingintotheactualdocuments.Thischangecanalsoleadtoreducedindexresizingtimes.
Throughputenhancements
WhatsNewin2.7
390
TheArangoDB-internalimplementationsfordispatchingrequests,keepingstatisticsandassigningV8contextstothreadshavebeen
improvedinordertouselesslocks.Thesechangesallowhigherconcurrencyandthroughputinthesecomponents,whichcanalsomake
theserverhandlemorerequestsinagivenperiodoftime.
Whatgainscanbeexpecteddependsonwhichoperationsareexecuted,buttherearereal-worldcasesinwhichthroughputincreasedby
between25%and70%whencomparedto2.6.
Madvisehints
TheLinuxvariantforArangoDBprovidestheOSwithmadvisehintsaboutindexmemoryanddatafilememory.Thesehintscanspeed
upthingswhenmemoryistight,inparticularatcollectionloadtimebutalsoforrandomaccesseslater.Thereisnoformalguaranteethat
theOSactuallyusesthemadvisehintsprovidedbyArangoDB,butactualmeasurementshaveshownimprovementsforloadingbigger
collections.
AQLimprovements
Additionaldatefunctions
ArangoDB2.7providesseveralextraAQLfunctionsfordateandtimecalculationandmanipulation.Thesefunctionswerecontributedby
GitHubusers@[email protected]!
Thefollowingextradatefunctionsareavailablefrom2.7on:
DATE_DAYOFYEAR(date):Returnsthedayofyearnumberofdate.Thereturnvaluesrangefrom1to365,or366inaleapyear
respectively.
DATE_ISOWEEK(date):ReturnstheISOweekdateofdate.Thereturnvaluesrangefrom1to53.M ondayisconsideredthefirstday
oftheweek.Therearenofractionalweeks,thusthelastdaysinDecembermaybelongtothefirstweekofthenextyear,andthe
firstdaysinJanuarymaybepartofthepreviousyear'slastweek.
DATE_LEAPYEAR(date):Returnswhethertheyearofdateisaleapyear.
DATE_QUARTER(date):Returnsthequarterofthegivendate(1-based):
1:January,February,M arch
2:April,M ay,June
3:July,August,September
4:October,November,December
DATE_DAYS_IN_MONTH(date):Returnsthenumberofdaysindate'smonth(28..31).
DATE_ADD(date,amount,unit):Addsamountgiveninunittodateandreturnsthecalculateddate.
unitcanbeeitherofthefollowingtospecifythetimeunittoaddorsubtract(case-insensitive):
y,year,years
m,month,months
w,week,weeks
d,day,days
h,hour,hours
i,minute,minutes
s,second,seconds
f,millisecond,milliseconds
amountisthenumberofunitstoadd(positivevalue)orsubtract(negativevalue).
DATE_SUBTRACT(date,amount,unit):Subtractsamountgiveninunitfromdateandreturnsthecalculateddate.
Itworksthesameas DATE_ADD(),exceptthatitsubtracts.Itisequivalenttocalling DATE_ADD()withanegativeamount,except
that DATE_SUBTRACT()canalsosubtractISOdurations.NotethatnegativeISOdurationsarenotsupported(i.e.startingwith -P,
like -P1Y).
DATE_DIFF(date1,date2,unit,asFloat):Calculatethedifferencebetweentwodatesingiventimeunit,optionallywithdecimal
places.Returnsanegativevalueifdate1isgreaterthandate2.
WhatsNewin2.7
391
DATE_COMPARE(date1,date2,unitRangeStart,unitRangeEnd):Comparetwopartialdatesandreturntrueiftheymatch,false
otherwise.Thepartstocomparearedefinedbyarangeoftimeunits.
Thefullrangeis:years,months,days,hours,minutes,seconds,milliseconds.PasstheunittostartfromasunitRangeStart,andthe
unittoendwithasunitRangeEnd.Allunitsinbetweenwillbecompared.LeaveoutunitRangeEndtoonlycompareunitRangeStart.
DATE_FORMAT(date,format):Formatadateaccordingtothegivenformatstring.Itsupportsthefollowingplaceholders(case-
insensitive):
%t:timestamp,inmillisecondssincemidnight1970-01-01
%z:ISOdate(0000-00-00T00:00:00.000Z)
%w:dayofweek(0..6)
%y:year(0..9999)
%yy:year(00..99),abbreviated(lasttwodigits)
%yyyy:year(0000..9999),paddedtolengthof4
%yyyyyy:year(-009999..+009999),withsignprefixandpaddedtolengthof6
%m:month(1..12)
%mm:month(01..12),paddedtolengthof2
%d:day(1..31)
%dd:day(01..31),paddedtolengthof2
%h:hour(0..23)
%hh:hour(00..23),paddedtolengthof2
%i:minute(0..59)
%ii:minute(00..59),paddedtolengthof2
%s:second(0..59)
%ss:second(00..59),paddedtolengthof2
%f:millisecond(0..999)
%fff:millisecond(000..999),paddedtolengthof3
%x:dayofyear(1..366)
%xxx:dayofyear(001..366),paddedtolengthof3
%k:ISOweekdate(1..53)
%kk:ISOweekdate(01..53),paddedtolengthof2
%l:leapyear(0or1)
%q:quarter(1..4)
%a:daysinmonth(28..31)
%mmm:abbreviatedEnglishnameofmonth(Jan..Dec)
%mmmm:Englishnameofmonth(January..December)
%www:abbreviatedEnglishnameofweekday(Sun..Sat)
%wwww:Englishnameofweekday(Sunday..Saturday)
%&:specialescapesequenceforrareoccasions
%%:literal%
%:ignored
RETURNDISTINCT
Toreturnuniquevaluesfromaquery,AQLnowprovidesthe DISTINCTkeyword.Itcanbeusedasamodifierfor RETURNstatements,
asashorteralternativetothealreadyexisting COLLECTstatement.
Forexample,thefollowingqueryonlyreturnsdistinct(unique) statusattributevaluesfromthecollection:
FORdocINcollection
RETURNDISTINCTdoc.status
RETURNDISTINCTisnotallowedonthetop-levelofaqueryifthereisno FORloopinfrontofit. RETURNDISTINCTisallowedin
subqueries.
RETURNDISTINCTensuresthatthevaluesreturnedaredistinct(unique),butdoesnotguaranteeanyorderofresults.Inordertohave
certainresultorder,anadditional SORTstatementmustbeaddedtoaquery.
WhatsNewin2.7
392
Shorthandobjectnotation
AQLnowprovidesashorthandnotationforobjectliteralsinthestyleofES6objectliterals:
LETname="Peter"
LETage=42
RETURN{name,age}
Thisisequivalenttothepreviouslyavailablecanonicalform,whichisstillavailableandsupported:
LETname="Peter"
LETage=42
RETURN{name:name,age:age}
Arrayexpansionimprovements
Thealreadyexisting[*]operatorhasbeenimprovedwithoptionalfilteringandprojectionandlimitcapabilities.
Forexample,considerthefollowingexamplequerythatfiltersvaluesfromanarrayattribute:
FORuINusers
RETURN{
name:u.name,
friends:(
FORfINu.friends
FILTERf.age>u.age
RETURNf.name
)
}
Withthe[*]operator,thisquerycanbesimplifiedto
FORuINusers
RETURN{name:u.name,friends:u.friends[*FILTERCURRENT.age>u.age].name}
Thepseudo-variableCURRENTcanbeusedtoaccessthecurrentarrayelement.The FILTERconditioncanreferto CURRENTorany
variablesvalidintheouterscope.
Toreturnaprojectionofthecurrentelement,therecannowbeaninline RETURN:
FORuINusers
RETURNu.friends[*RETURNCONCAT(CURRENT.name,"isafriendof",u.name)]
whichisthesimplifiedvariantfor:
FORuINusers
RETURN(
FORfriendINu.friends
RETURNCONCAT(friend.name,"isafriendof",u.name)
)
Arraycontraction
Inordertocollapse(orflatten)resultsinnestedarrays,AQLnowprovidesthe[**]operator.Itworkssimilartothe[*]operator,but
additionallycollapsesnestedarrays.Howmanylevelsarecollapsedisdeterminedbytheamountof*charactersused.
Forexample,considerthefollowingquerythatproducesanestedresult:
FORuINusers
RETURNu.friends[*].name
WhatsNewin2.7
393
The[**]operatorcannowbeappliedtogetridofthenestedarrayandturnitintoaflatarray.Wesimplyapplythe[**]onthe
previousqueryresult:
RETURN(
FORuINusersRETURNu.friends[*].name
)[**]
Templatequerystrings
AssemblingquerystringsinJavaScripthasbeenerror-pronewhenusingsimplestringconcatenation,especiallybecauseplainJavaScript
stringsdonothavemultiline-support,andbecauseofpotentialparameterinjectionissues.Whilemultilinequerystringscanbeassembled
withES6templatestringssinceArangoDB2.5,andquerybindparametersaretheresinceArangoDB1.0topreventparameterinjection,
therewasnoJavaScript-ysolutiontocombinethese.
ArangoDB2.7nowprovidesanES6templatestringgeneratorfunctionthatcanbeusedtoeasilyandsafelyassembleAQLqueriesfrom
JavaScript.JavaScriptvariablesandexpressionscanbeusedeasilyusingregularES6templatestringsubstitutions:
letname='test';
letattributeName='_key';
letquery=aqlQuery`FORuINusers
FILTERu.name==${name}
RETURNu.${attributeName}`;
db._query(query);
ThisismorelegiblethanwhenusingaplainJavaScriptstringandalsodoesnotrequiredefiningthebindparametervaluesseparately:
letname='test';
letattributeName='_key';
letquery="FORuINusers"+
"FILTERu.name==@name"+
"RETURNu.@attributeName";
db._query(query,{
name,
attributeName
});
The aqlQuerytemplatestringgeneratorwillalsohandlecollectionobjectsautomatically:
db._query(aqlQuery`FORuIN${db.users}RETURNu.name`);
NotethatwhiletemplatestringsareavailableintheJavaScriptfunctionsprovidedtobuildqueries,theyaren'tafeatureofAQLitself.
AQLcouldalwayshandlemultilinequerystringsandprovidedbindparameters( @...)forseparatingthequerystringandthe
parametervalues.The aqlQuerytemplatestringgeneratorfunctionwilltakecareofthisseparation,too,butwilldoitbehindthescenes.
AQLqueryresultcache
TheAQLqueryresultcachecanoptionallycachethecompleteresultsofallorjustselectedAQLqueries.Itcanbeoperatedinthe
followingmodes:
off:thecacheisdisabled.Noqueryresultswillbestored
on:thecachewillstoretheresultsofallAQLqueriesunlesstheir cacheattributeflagissetto false
demand:thecachewillstoretheresultsofAQLqueriesthathavetheir cacheattributesetto true,butwillignoreallothers
Themodecanbesetatserverstartupusingthe --database.query-cache-modeconfigurationoptionandlaterchangedatruntime.The
defaultvalueis off,meaningthatthequeryresultcacheisdisabled.Thisisbecausethecachemayconsumeadditionalmemorytokeep
queryresults,andalsobecauseitmustbeinvalidatedwhenchangeshappenincollectionsforwhichresultshavebeencached.
Thequeryresultcachemaythereforehavepositiveornegativeeffectsonqueryexecutiontimes,dependingontheworkload:itwillnot
makemuchsenseturningonthecacheinwrite-onlyorwrite-mostlyscenarios,butthecachemaybeverybeneficialincaseworkloadsare
read-onlyorread-mostly,andqueryarecomplex.
WhatsNewin2.7
394
Ifthequerycacheisoperatedin demandmode,itcanbecontrolledperqueryifthecacheshouldbecheckedforaresult.
Optimizer
TheAQLoptimizerrule patch-update-statementshasbeenadded.ThisrulecanoptimizecertainAQLUPDATEqueriesthatupdate
documentsintheacollectionthattheyalsoiterateover.
Forexample,thefollowingqueryreadsdocumentsfromacollectioninordertoupdatethem:
FORdocINcollection
UPDATEdocWITH{newValue:doc.oldValue+1}INcollection
Inthiscase,onlyasinglecollectionisaffectedbythequery,andthereisnoindexlookupinvolvedtofindtheto-be-updateddocuments.
Inthiscase,theUPDATEquerydoesnotrequiretakingafull,memory-intensivesnapshotofthecollection,butitcanbeperformedin
smallchunks.Thiscanleadtomemorysavingswhenexecutingsuchqueries.
Functioncallargumentsoptimization
ThisoptimizationwillleadtoargumentsinfunctioncallsinsideAQLqueriesnotbeingcopiedbutbeingpassedbyreference.Thismay
speedupcallstofunctionswithbiggerargumentvaluesorqueriesthatcallAQLfunctionsalotoftimes.
WebAdminInterface
Thewebinterfacenowhasanewdesign.
The"Applications"tabinthewebinterfaceshasbeenrenamedto"Services".
TheArangoDBAPIdocumentationhasbeenmovedfromthe"Tools"menutothe"Links"menu.Thenewdocumentationisbasedon
Swagger2.0andopensinaseparatewebpage.
Foxximprovements
ES2015Classes
AllFoxxconstructorshavebeenreplacedwithES2015classesandcanbeextendedusingtheclasssyntax.The extendmethodisstill
supportedatthemomentbutwillbecomedeprecatedinArangoDB2.8andremovedinArangoDB2.9.
Before:
varMyModel=Foxx.Model.extend({
//...
schema:{/*...*/}
});
After:
classMyModelextendsFoxx.Model{
//...
}
MyModel.prototype.schema={/*...*/};
Confidentialconfiguration
WhatsNewin2.7
395
Itisnowpossibletospecifyconfigurationoptionswiththetype password.Thepasswordtypeisequivalenttothetexttypebutwill
bemaskedinthewebfrontendtopreventaccidentalexposureofconfidentialoptionslikeAPIkeysandpasswordswhenconfiguring
yourFoxxapplication.
Dependencies
Thesyntaxforspecifyingdependenciesinmanifestshasbeenextendedtoallowspecifyingoptionaldependencies.Unmetoptional
dependencieswillnotpreventanappfrombeingmounted.Thetraditionalshorthandsyntaxforspecifyingnon-optionaldependencies
willstillbesupportedintheupcomingversionsofArangoDB.
Before:
{
...
"dependencies":{
"notReallyNeeded":"users:^1.0.0",
"totallyNecessary":"sessions:^1.0.0"
}
}
After:
{
"dependencies":{
"notReallyNeeded":{
"name":"users",
"version":"^1.0.0",
"required":false
},
"totallyNecessary":{
"name":"sessions",
"version":"^1.0.0"
}
}
}
Replication
TheexistingreplicationHTTPAPIhasbeenextendedwithmethodsthatreplicationclientscanusetodeterminewhetheragivendate,
identifiedbyatickvalue,isstillpresentonamasterforreplication.BycallingtheseAPIs,clientscanmakeaninformeddecisionabout
whetherthemastercanstillprovideallmissingdatastartingfromthepointuptowhichtheclienthadalreadysynchronized.Thiscanbe
helpfulincaseareplicationclientisre-startedafterapause.
M asterserversnowalsotrackupthepointuptowhichtheyhavesentchangestoclientsforreplication.Thisinformationcanbeusedto
determinethepointofdatathatreplicationclientshavereceivedfromthemaster,andifandhowfarapproximatelytheylagbehind.
Finally,restartingthereplicationapplieronaslaveserverhasbeenmademorerobustincasetheapplierwasstoppedwhiletherewere
pendingtransactionsonthemasterserver,andre-startingthereplicationapplierneedstorestorethestateofthesetransactions.
Clienttools
Thefilenamesindumpscreatedbyarangodumpnowcontainnotonlythenameofthedumpedcollection,butalsoanadditional32-digit
hashvalue.Thisisdonetopreventoverwritingdumpfilesincase-insensitivefilesystemswhenthereexistmultiplecollectionswiththe
samename(butwithdifferentcases).
Forexample,ifadatabasehadtwocollectionstestandTest,previousversionsofarangodumpcreatedthefollowingfiles:
test.structure.jsonand test.data.jsonforcollectiontest
Test.structure.jsonand Test.data.jsonforcollectionTest
WhatsNewin2.7
396
Thisdidnotworkincase-insensitivefilesystems,becausethefilesforthesecondcollectionwouldhaveoverwrittenthefilesofthefirst.
arangodumpin2.7willcreatetheuniquefilesinthiscase,byappendingthe32-digithashvaluetothecollectionnameinallcase.These
filenameswillbeunambiguousevenincase-insensitivefilesystems.
Bettercontrol-Csupportinarangosh
WhenCTRL-Cispressedinarangosh,itwillnowabortthelocallyrunningcommand(ifany).Ifnocommandwasrunning,pressing
CTRL-Cwillprinta ^Cfirst.PressingCTRL-Cagainwillthenquitarangosh.
CTRL-Ccanalsobeusedtoresetthecurrentpromptwhileenteringcomplexnestedobjectswhichspanmultipleinputlines.
CTRL-CsupporthasbeenaddedtotheArangoShellversionsbuiltwithReadline-support(LinuxandM acOSonly).TheWindows
versionofArangoDBusesadifferentlibraryforhandlinginput,andsupportforCTRL-Chasnotbeenaddedthereyet.
Start/stop
LinuxstartupscriptsandsystemdconfigurationforarangodnowtrytoadjusttheNOFILE(numberofopenfiles)limitsfortheprocess.
Thelimitvalueissetto131072(128k)whenArangoDBisstartedviastart/stopcommands.
ThiswillpreventarangodrunningoutofavailablefiledescriptorsincaseofmanyparallelHTTPconnectionsorlargecollectionswith
manydatafiles.
Additionally,whenArangoDBisstarted/stoppedmanuallyviathestart/stopcommands,themainprocesswillwaitforupto10seconds
afteritforksthesupervisorandarangodchildprocesses.Ifthestartupfailswithinthatperiod,thestart/stopscriptwillfailwithanonzeroexitcode,allowinganyinvokingscriptstohandlethiserror.Previousversionsalwaysreturnedanexitcodeof0,evenwhenarangod
couldn'tbestarted.
Ifthestartupofthesupervisororarangodisstillongoingafter10seconds,themainprogramwillstillreturnwithexitcode0inorderto
notblockanyscripts.Thelimitof10secondsisarbitrarybecausethetimerequiredforanarangodstartupisnotknowninadvance.
Non-sparselogfiles
WALlogfilesanddatafilescreatedbyarangodarenownon-sparse.ThispreventsSIGBUSsignalsbeingraisedwhenamemory-mapped
regionbackedbyasparsedatafilewasaccessedandthememoryregionwasnotactuallybackedbydisk,forexamplebecausethediskran
outofspace.
arangodnowalwaysfullyallocatesthediskspacerequiredforalogfileordatafilewhenitcreatesone,sothememoryregioncanalways
bebackedbydisk,andmemorycanbeaccessedwithoutSIGBUSbeingraised.
WhatsNewin2.7
397
APIsadded
Batchdocumentremovalandlookupcommands
Thefollowingcommandshavebeenaddedfor collectionobjects:
collection.lookupByKeys(keys)
Thesecommandscanbeusedtoperformmulti-documentlookupandremovaloperationsefficientlyfromtheArangoShell.Theargument
totheseoperationsisanarrayofdocumentkeys.
ThesecommandscanalsobeusedviatheHTTPRESTAPI.Theirendpointsare:
PUT/_api/simple/lookup-by-keys
PUT/_api/simple/remove-by-keys
CollectionexportHTTPRESTAPI
ArangoDBnowprovidesadedicatedcollectionexportAPI,whichcantakesnapshotsofentirecollectionsmoreefficientlythanthe
general-purposecursorAPI.TheexportAPIisusefultotransferthecontentsofanentirecollectiontoaclientapplication.Itprovides
optionalfilteringonspecificattributes.
TheexportAPIisavailableatendpoint POST/_api/export?collection=....TheAPIhasthesamereturnvaluestructureasthealready
establishedcursorAPI( POST/_api/cursor).
AnintroductiontotheexportAPIisgiveninthisblogpost:http://jsteemann.github.io/blog/2015/04/04/more-efficient-data-exports/
AQLimprovements
EDGESAQLFunction
TheAQLfunctionEDGESgotanewfifthoptionalparameter,whichmustbeanobjectifspecified.Rightnowonlyoneoptionis
availableforit:
includeVerticesthisisabooleanparameterthatallowstomodifytheresultof EDGES().Thedefaultvaluefor includeVertices
is false,whichdoesnothaveanyeffect.Settingitto truewillmodifytheresult,suchthatalsotheconnectedverticesare
returnedalongwiththeedges:
{vertex:<vertexDocument>,edge:<edgeDocument>}
SubqueryoptimizationsforAQLqueries
Thisoptimizationavoidscopyingintermediateresultsintosubqueriesthatarenotrequiredbythesubquery.
Abriefdescriptioncanbefoundhere:http://jsteemann.github.io/blog/2015/05/04/subquery-optimizations/
ReturnvalueoptimizationforAQLqueries
Thisoptimizationavoidscopyingthefinalqueryresultinsidethequery'smain ReturnNode.
Abriefdescriptioncanbefoundhere:http://jsteemann.github.io/blog/2015/05/04/return-value-optimization-for-aql/
WhatsNewin2.6
398
SpeedupAQLqueriescontainingbig INlistsforindexlookups
INlistsusedforindexlookupshadperformanceissuesinpreviousversionsofArangoDB.Theseissueshavebeenaddressedin2.6so
usingbigger INlistsforfilteringismuchfaster.
Abriefdescriptioncanbefoundhere:http://jsteemann.github.io/blog/2015/05/07/in-list-improvements/
AddedalternativeimplementationforAQLCOLLECT
Thealternativemethodusesahashtableforgroupinganddoesnotrequireitsinputelementstobesorted.Itwillbetakenintoaccount
bytheoptimizerfor COLLECTstatementsthatdonotusean INTOclause.
Incasea COLLECTstatementcanusethehashtablevariant,theoptimizerwillcreateanextraplanforitatthebeginningoftheplanning
phase.Inthisplan,noextra SORTnodewillbeaddedinfrontofthe COLLECTbecausethehashtablevariantof COLLECTdoesnot
requiresortedinput.Instead,a SORTnodewillbeaddedafterittosortitsoutput.This SORTnodemaybeoptimizedawayagainin
laterstages.Ifthesortorderoftheresultisirrelevanttotheuser,addinganextra SORTnullafterahash COLLECToperationwillallow
theoptimizertoremovethesortsaltogether.
Inadditiontothehashtablevariantof COLLECT,theoptimizerwillmodifytheoriginalplantousetheregular COLLECTimplementation.
Asthisimplementationrequiressortedinput,theoptimizerwillinserta SORTnodeinfrontofthe COLLECT.This SORTnodemaybe
optimizedawayinlaterstages.
Thecreatedplanswillthenbeshippedthroughtheregularoptimizationpipeline.Intheend,theoptimizerwillpicktheplanwiththe
lowestestimatedtotalcostasusual.Thehashtablevariantdoesnotrequireanup-frontsortoftheinput,andwillthusbepreferredover
theregular COLLECTiftheoptimizerestimatesmanyinputelementsforthe COLLECTnodeandcannotuseanindextosortthem.
Theoptimizercanbeexplicitlytoldtousetheregularsortedvariantof COLLECTbysuffixinga COLLECTstatementwith OPTIONS{
"method":"sorted"}.Thiswilloverridetheoptimizerguessworkandonlyproducethesortedvariantof COLLECT.
Ablogpostonthenew COLLECTimplementationcanbefoundhere:http://jsteemann.github.io/blog/2015/04/22/collecting-with-a-hashtable/
Simplifiedreturnvaluesyntaxfordata-modificationAQLqueries
ArangoDB2.4sinceversionallowstoreturnresultsfromdata-modificationAQLqueries.Thesyntaxforthiswasquitelimitedand
verbose:
FORiIN1..10
INSERT{value:i}INtest
LETinserted=NEW
RETURNinserted
The LETinserted=NEWRETURNinsertedwasrequiredliterallytoreturntheinserteddocuments.Nocalculationscouldbemadeusing
theinserteddocuments.
Thisisnowmoreflexible.Afteradata-modificationclause(e.g. INSERT, UPDATE, REPLACE, REMOVE, UPSERT)therecanfollowany
numberof LETcalculations.Thesecalculationscanrefertothepseudo-values OLDand NEWthatarecreatedbythedata-modification
statements.
Thisallowsreturningprojectionsofinsertedorupdateddocuments,e.g.:
FORiIN1..10
INSERT{value:i}INtest
RETURN{_key:NEW._key,value:i}
Stillnoteveryconstructisallowedafteradata-modificationclause.Forexample,nofunctionscanbecalledthatmayaccessdocuments.
M oreinformationcanbefoundhere:http://jsteemann.github.io/blog/2015/03/27/improvements-for-data-modification-queries/
AddedAQL UPSERTstatement
WhatsNewin2.6
399
Thisaddsan UPSERTstatementtoAQLthatisacombinationofboth INSERTand UPDATE/ REPLACE.The UPSERTwillsearchfora
matchingdocumentusingauser-providedexample.Ifnodocumentmatchestheexample,theinsertpartofthe UPSERTstatementwillbe
executed.Ifthereisamatch,theupdate/replacepartwillbecarriedout:
UPSERT{page:'index.html'}/*searchexample*/
INSERT{page:'index.html',pageViews:1}/*insertpart*/
UPDATE{pageViews:OLD.pageViews+1}/*updatepart*/
INpageViews
UPSERTcanbeusedwithan UPDATEor REPLACEclause.The UPDATEclausewillperformapartialupdateofthefounddocument,
whereasthe REPLACEclausewillreplacethefounddocumententirely.The UPDATEor REPLACEpartscanrefertothepseudo-value
OLD,whichcontainsallattributesofthefounddocument.
UPSERTstatementscanoptionallyreturnvalues.Inthefollowingquery,thereturnattribute foundwillreturnthefounddocument
beforethe UPDATEwasapplied.Ifnodocumentwasfound, foundwillcontainavalueof null.The updatedresultattributewill
containtheinserted/updateddocument:
UPSERT{page:'index.html'}/*searchexample*/
INSERT{page:'index.html',pageViews:1}/*insertpart*/
UPDATE{pageViews:OLD.pageViews+1}/*updatepart*/
INpageViews
RETURN{found:OLD,updated:NEW}
Amoredetaileddescriptionof UPSERTcanbefoundhere:http://jsteemann.github.io/blog/2015/03/27/preview-of-the-upsert-command/
WhenerrorsoccurinsideAQLuserfunctions,theerrormessagewillnowcontainastacktrace,indicatingthelineofcodeinwhichthe
erroroccurred.ThisshouldmakedebuggingAQLuserfunctionseasier.
WebAdminInterface
ArangoDB'sbuilt-inwebinterfacenowusessessions.Sessioninformationisstoredincookies,soclientsusingthewebinterfacemust
acceptcookiesinordertouseit.
Thenewstartupoption --server.session-timeoutcanbeusedforadjustingthesessionlifetime.
TheAQLeditorinthewebinterfacenowprovidesanexplainfunctionality,whichcanbeusedforinspectingandperformance-tuning
AQLqueries.ThequeryexecutiontimeisnowalsodisplayedintheAQLeditor.
Foxxappsthatrequireconfigurationoraremissingdependenciesarenowindicatedintheappoverviewanddetails.
Foxximprovements
ConfigurationandDependencies
Foxxappmanifestscannowdefineconfigurationoptions,aswellasdependenciesonotherFoxxapps.
AnintroductiontoFoxxconfigurationscanbefoundintheblog:https://www.arangodb.com/2015/05/reusable-foxx-apps-withconfigurations/
AndtheblogpostonFoxxdependenciescanbefoundhere:https://www.arangodb.com/2015/05/foxx-dependencies-for-morecomposable-foxx-apps/
MochaTests
YoucannowwritetestsforyourFoxxappsusingtheM ochatestingframework:https://www.arangodb.com/2015/04/testing-foxx-mocha/
ArecipeforwritingtestsforyourFoxxappscanbefoundinthecookbook:https://docs.arangodb.com/2.8/cookbook/FoxxTesting.html
WhatsNewin2.6
400
APIDocumentation
TheAPIdocumentationhasbeenupdatedtoSwagger2.YoucannowalsomountAPIdocumentationinyourownFoxxapps.
Alsoseetheblogpostintroducingthisfeature:https://www.arangodb.com/2015/05/document-your-foxx-apps-with-swagger-2/
CustomScriptsandFoxxQueue
InadditiontotheexistingsetupandteardownscriptsyoucannowdefinecustomscriptsinyourFoxxmanifestandinvoketheseusing
thewebadmininterfaceortheFoxxmanagerCLI.Thesescriptscannowalsotakepositionalargumentsandexportreturnvalues.
JobtypesfortheFoxxQueuecannowbedefinedasascriptnameandappmountpathallowingtheuseofFoxxscriptsasjobtypes.
Thepre-2.6jobtypesareknowntocauseissueswhenrestartingtheserverandareerror-prone;westronglyrecommendedconverting
anyexistingjobtypestothenewformat.
Clienttools
Thedefaultconfigurationvaluefortheoption --server.request-timeoutwasincreasedfrom300to1200secondsforallclienttools
(arangosh,arangoimp,arangodump,arangorestore).
Thedefaultconfigurationvaluefortheoption --server.connect-timeoutwasincreasedfrom3to5secondsforclienttools(arangosh,
arangoimp,arangodump,arangorestore).
Arangorestore
Theoption --create-databasewasaddedforarangorestore.
Settingthisoptionto truewillnowcreatethetargetdatabaseifitdoesnotexist.Whencreatingthetargetdatabase,theusernameand
passwordspassedtoarangorestorewillbeusedtocreateaninitialuserforthenewdatabase.
Thedefaultvalueforthisoptionis false.
Arangoimp
Arangoimpcannowoptionallyupdateorreplaceexistingdocuments,providedtheimportdatacontainsdocumentswith _key
attributes.
Previously,theimportcouldbeusedforinsertingnewdocumentsonly,andre-insertingadocumentwithanexistingkeywouldhave
failedwithauniquekeyconstraintviolatederror.
Thebehaviorofarangoimp(insert,update,replaceonduplicatekey)cannowbecontrolledwiththeoption --on-duplicate.Theoption
canhaveoneofthefollowingvalues:
error:whenauniquekeyconstrainterroroccurs,donotimportorupdatethedocumentbutreportanerror.Thisisthedefault.
update:whenauniquekeyconstrainterroroccurs,tryto(partially)updatetheexistingdocumentwiththedataspecifiedinthe
import.Thismaystillfailifthedocumentwouldviolatesecondaryuniqueindexes.Onlytheattributespresentintheimportdata
willbeupdatedandotherattributesalreadypresentwillbepreserved.Thenumberofupdateddocumentswillbereportedinthe
updatedattributeoftheHTTPAPIresult.
replace:whenauniquekeyconstrainterroroccurs,trytofullyreplacetheexistingdocumentwiththedataspecifiedinthe
import.Thismaystillfailifthedocumentwouldviolatesecondaryuniqueindexes.Thenumberofreplaceddocumentswillbe
reportedinthe updatedattributeoftheHTTPAPIresult.
ignore:whenauniquekeyconstrainterroroccurs,ignorethiserror.Therewillbenoinsert,updateorreplacefortheparticular
document.Ignoreddocumentswillbereportedseparatelyinthe ignoredattributeoftheHTTPAPIresult.
Thedefaultvalueis error.
Afewexamplesforusingarangoimpwiththe --on-duplicateoptioncanbefoundhere:
http://jsteemann.github.io/blog/2015/04/14/updating-documents-with-arangoimp/
WhatsNewin2.6
401
SomeLinux-basedArangoDBpackagesarenowusingtcmallocformemoryallocator.
UpgradedICUlibrarytoversion54.Thisincreasesperformanceinmanyplaces.
Allowtosplitanedgeindexintobucketswhichareresizedindividually.Thedefaultvalueis 1,resemblingthepre-2.6behavior.
Usingmultiplebucketswillleadtotheindexentriesbeingdistributedtotheindividualbuckets,witheachbucketbeingresponsible
onlyforafractionofthetotalindexentries.Usingmultiplebucketsmayleadtomorefrequentbutmuchfasterindexbucketresizes,
andisrecommendedforbiggeredgecollections.
Defaultconfigurationvalueforoption --server.backlog-sizewaschangedfrom10to64.
Defaultconfigurationvalueforoption --database.ignore-datafile-errorswaschangedfrom trueto false
Documentkeyscannowcontain @and .characters
Fulltextindexcannowindextextvaluescontainedindirectsub-objectsoftheindexedattribute.
PreviousversionsofArangoDBonlyindexedtheattributevalueifitwasastring.Sub-attributesoftheindexattributewereignored
whenfulltextindexing.
Now,iftheindexattributevalueisanobject,theobject'svalueswilleachbeincludedinthefulltextindexiftheyarestrings.Ifthe
indexattributevalueisanarray,thearray'svalueswilleachbeincludedinthefulltextindexiftheyarestrings.
Forexample,withafulltextindexpresentonthe translationsattribute,thefollowingtextvalueswillnowbeindexed:
varc=db._create("example");
c.ensureFulltextIndex("translations");
c.insert({translations:{en:"fox",de:"Fuchs",fr:"renard",ru:"лиса"}});
c.insert({translations:"FoxistheEnglishtranslationoftheGermanwordFuchs"});
c.insert({translations:["ArangoDB","document","database","Foxx"]});
c.fulltext("translations","лиса").toArray();//returnsonlyfirstdocument
c.fulltext("translations","Fox").toArray();//returnsfirstandseconddocuments
c.fulltext("translations","prefix:Fox").toArray();//returnsallthreedocuments
Addedconfigurationoption --server.foxx-queues-poll-interval
ThisstartupoptioncontrolsthefrequencywithwhichtheFoxxqueuesmanagerischeckingthequeue(orqueues)forjobstobe
executed.
Thedefaultvalueis 1second.Loweringthisvaluewillresultinthequeuemanagerwakingupandcheckingthequeuesmore
frequently,whichmayincreaseCPUusageoftheserver.WhennotusingFoxxqueues,thisvaluecanberaisedtosavesomeCPU
time.
Addedconfigurationoption --server.foxx-queues
ThisstartupoptioncontrolswhethertheFoxxqueuemanagerwillcheckqueueandjobentriesinthe _systemdatabaseonly.
RestrictingtheFoxxqueuemanagertothe _systemdatabasewillleadtothequeuemanagerhavingtocheckonlythequeues
collectionofasingledatabase,whereasmakingitcheckthequeuesofalldatabasesmightresultinmoreworktobedoneandmore
CPUtimetobeusedbythequeuemanager.
WhatsNewin2.6
402
V8versionupgrade
Thebuilt-inversionofV8hasbeenupgradedfrom3.29.54to3.31.74.1.ThisallowsactivatingadditionalES6(alsodubbedHarmonyor
ES.next)featuresinArangoDB,bothintheArangoShellandtheArangoDBserver.Theycanbeusedforscriptingandinserver-side
actionssuchasFoxxroutes,traversalsetc.
ThefollowingadditionalES6featuresbecomeavailableinArangoDB2.5bydefault:
iteratorsandgenerators
templatestrings
enhancedobjectliterals
enhancednumericliterals
blockscopingwith letandconstantvariablesusing const(note:constantvariablesrequireusingstrictmode,too)
additionalstringmethods(suchas startsWith, repeatetc.)
Indeximprovements
Sparsehashandskiplistindexes
Hashandskiplistindexescanoptionallybemadesparse.Sparseindexesexcludedocumentsinwhichatleastoneoftheindexattributesis
eithernotsetorhasavalueof null.
Assuchdocumentsareexcludedfromsparseindexes,theymaycontainfewerdocumentsthantheirnon-sparsecounterparts.This
enablesfasterindexingandcanleadtoreducedmemoryusageincasetheindexedattributedoesoccuronlyinsome,butnotalldocuments
ofthecollection.Sparseindexeswillalsoreducethenumberofcollisionsinnon-uniquehashindexesincasenon-existingoroptional
attributesareindexed.
Inordertocreateasparseindex,anobjectwiththeattribute sparsecanbeaddedtotheindexcreationcommands:
db.collection.ensureHashIndex(attributeName,{sparse:true});
db.collection.ensureHashIndex(attributeName1,attributeName2,{sparse:true});
db.collection.ensureUniqueConstraint(attributeName,{sparse:true});
db.collection.ensureUniqueConstraint(attributeName1,attributeName2,{sparse:true});
db.collection.ensureSkiplist(attributeName,{sparse:true});
db.collection.ensureSkiplist(attributeName1,attributeName2,{sparse:true});
db.collection.ensureUniqueSkiplist(attributeName,{sparse:true});
db.collection.ensureUniqueSkiplist(attributeName1,attributeName2,{sparse:true});
Notethatinplaceoftheabovespecializedindexcreationcommands,itisrecommendedtousethemoregeneralindexcreationcommand
ensureIndex:
db.collection.ensureIndex({type:"hash",sparse:true,unique:true,fields:[attributeName]});
db.collection.ensureIndex({type:"skiplist",sparse:false,unique:false,fields:["a","b"]});
Whennotexplicitlyset,the sparseattributedefaultsto falsefornewhashorskiplistindexes.
Thiscausesachangeinbehaviorwhencreatingauniquehashindexwithoutspecifyingthesparseflag:in2.4,uniquehashindexeswere
implicitlysparse,alwaysexcluding nullvalues.Therewasnooptiontocontrolthisbehavior,andsparsitywasneithersupportedfor
non-uniquehashindexesnorskiplistsin2.4.Thisimplicitsparsityofuniquehashindexeswasconsideredaninconsistency,andtherefore
thebehaviorwascleanedupin2.5.Asof2.5,indexeswillonlybecreatedsparseifsparsityisexplicitlyrequested.Existinguniquehash
indexesfrom2.4orbeforewillautomaticallybemigratedsotheyarestillsparseaftertheupgradeto2.5.
WhatsNewin2.5
403
Geoindexesareimplicitlysparse,meaningdocumentswithouttheindexedlocationattributeorcontaininginvalidlocationcoordinate
valueswillbeexcludedfromtheindexautomatically.Thisisalsoachangewhencomparedtopre-2.5behavior,whendocumentswith
missingorinvalidcoordinatevaluesmayhavecausederrorsoninsertionwhenthegeoindex' uniqueflagwassetandits ignoreNull
flagwasnot.Thiswasconfusingandhasbeenrectifiedin2.5.Themethod ensureGeoConstraint()nowdoesthesameas
ensureGeoIndex().Furthermore,theattributes constraint, unique, ignoreNulland sparseflagsarenowcompletelyignored
whencreatinggeoindexes.
Thesameistrueforfulltextindexes.Thereisnoneedtospecifynon-uniquenessorsparsityforgeoorfulltextindexes.
Assparseindexesmayexcludesomedocuments,theycannotbeusedforeverytypeofquery.Sparsehashindexescannotbeusedtofind
documentsforwhichatleastoneoftheindexedattributeshasavalueof null.Forexample,thefollowingAQLquerycannotusea
sparseindex,evenifonewascreatedonattribute attr:
FORdocIncollection
FILTERdoc.attr==null
RETURNdoc
Ifthelookupvalueisnon-constant,asparseindexmayormaynotbeused,dependingontheothertypesofconditionsinthequery.If
theoptimizercansafelydeterminethatthelookupvaluecannotbe null,asparseindexmaybeused.Whenuncertain,theoptimizer
willnotmakeuseofasparseindexinaqueryinordertoproducecorrectresults.
Forexample,thefollowingqueriescannotuseasparseindexon attrbecausetheoptimizerwillnotknowbeforehandwhetherthe
comparisonvaluesfor doc.attrwillinclude null:
FORdocIncollection
FILTERdoc.attr==SOME_FUNCTION(...)
RETURNdoc
FORotherINotherCollection
FORdocIncollection
FILTERdoc.attr==other.attr
RETURNdoc
Sparseskiplistindexescanbeusedforsortingiftheoptimizercansafelydetectthattheindexrangedoesnotinclude nullforanyof
theindexattributes.
Selectivityestimates
Indexesoftype primary, edgeand hashnowprovideselectivityestimates.ThesewillbeusedbytheAQLqueryoptimizerwhen
decidingaboutindexusage.Usingselectivityestimatescanleadtofasterqueryexecutionwhenmoreselectiveindexesareused.
Theselectivityestimatesarealsoreturnedbythe GET/_api/indexRESTAPImethodinasub-attribute selectivityEstimateforeach
indexthatsupportsit.Thisattributewillbeomittedforindexesthatdonotprovideselectivityestimates.Ifprovided,theselectivity
estimatewillbeanumericvaluebetween0and1.
Selectivityestimateswillalsobereportedintheresultof collection.getIndexes()forallindexesthatsupportthis.Ifnoselectivity
estimatecanbedeterminedforanindex,theattribute selectivityEstimatewillbeomittedhere,too.
Thewebinterfacealsoshowsselectivityestimatesforeachindexthatsupportsthis.
Currentlythefollowingindextypescanprovideselectivityestimates:
primaryindex
edgeindex
hashindex(uniqueandnon-unique)
Noselectivityestimateswillbeprovidedforindexeswhenrunninginclustermode.
AQLOptimizerimprovements
Sortremoval
WhatsNewin2.5
404
TheAQLoptimizerrule"use-index-for-sort"willnowremovesortsalsoincaseanon-sortedindex(e.g.ahashindex)isusedforonly
equalitylookupsandallsortattributesarecoveredbytheequalitylookupconditions.
Forexample,inthefollowingquerytheextrasorton doc.valuewillbeoptimizedawayprovidedthereisanindexon doc.value):
FORdocINcollection
FILTERdoc.value==1
SORTdoc.value
RETURNdoc
TheAQLoptimizerrule"use-index-for-sort"nowalsoremovessortincasethesortcriteriaexcludestheleft-mostindexattributes,but
theleft-mostindexattributesareusedbytheindexforequality-onlylookups.
Forexample,inthefollowingquerywithaskiplistindexon value1, value2,thesortcanbeoptimizedaway:
FORdocINcollection
FILTERdoc.value1==1
SORTdoc.value2
RETURNdoc
Constantattributepropagation
ThenewAQLoptimizerrule propagate-constant-attributeswilllookforattributesthatareequality-comparedtoaconstantvalue,and
willpropagatethecomparisonvalueintootherequalitylookups.Thisrulewillonlylookinside FILTERconditions,andinsertconstant
valuesfoundin FILTERs,too.
Forexample,therulewillinsert 42insteadof i.valueinthesecond FILTERofthefollowingquery:
FORiINc1
FORjINc2
FILTERi.value==42
FILTERj.value==i.value
RETURN1
Interleavedprocessing
TheoptimizerwillnowinspectAQLdata-modificationqueriesanddetectifthequery'sdata-modificationpartcanruninlockstepwith
thedataretrievalpartofthequery,orifthedataretrievalpartmustbeexecutedandcompletedfirstbeforethedata-modificationcan
start.
Executingbothdataretrievalanddata-modificationinlockstepallowsusingmuchsmallerbuffersforintermediateresults,reducingthe
memoryusageofqueries.Notallqueriesareeligibleforthisoptimization,andtheoptimizerwillonlyapplytheoptimizationwhenit
cansafelydetectthatthedata-modificationpartofthequerywillnotmodifydatatobefoundbytheretrievalpart.
Queryexecutionstatistics
The filteredattributewasaddedtoAQLqueryexecutionstatistics.Thevalueofthisattributeindicateshowmanydocumentswere
filteredby FilterNodesintheAQLquery.Notethat IndexRangeNodescanalsofilterdocumentsbyselectingonlytherequiredranges
fromtheindex.The filteredvaluewillnotincludetheworkdoneby IndexRangeNodes,butonlytheworkperformedby
FilterNodes.
Languageimprovements
DynamicattributenamesinAQLobjectliterals
ThischangeallowsusingarbitraryexpressionstoconstructattributenamesinobjectliteralsspecifiedinAQLqueries.Todisambiguate
expressionsandotherunquotedattributenames,dynamicattributenamesneedtobeenclosedinbrackets( [and ]).
Example:
WhatsNewin2.5
405
FORiIN1..100
RETURN{[CONCAT('value-of-',i)]:i}
AQLfunctions
ThefollowingAQLfunctionswereaddedin2.5:
MD5(value):generatesanM D5hashof value
SHA1(value):generatesanSHA1hashof value
RANDOM_TOKEN(length):generatesarandomstringvalueofthespecifiedlength
SimplifyFoxxusage
ThankstoouruserfeedbackwelearnedthatFoxxisapowerful,yetrathercomplicatedconcept.With2.5wemadeitlesscomplicated
whilekeepingallitsstrength.Thatincludesarewriteofthedocumentationaswellassomecodechangesasfollows:
MovedFoxxapplicationstoadifferentfolder.
Until2.4foxxappswerestoredinthefollowingfolderstructure: <app-path>/databases/<dbname>/<appname>:<appversion>.Thiscaused
sometroubleasappswherecachedbasedonnameandversionandupdatesdidnotapply.Alsothepathonfilesystemandtheapp's
accessURLhadnorelationtooneanother.NowthepathonfilesystemisidenticaltotheURL(excepttheappendedAPP): <apppath>/_db/<dbname>/<mointpoint>/APP
RewriteofFoxxrouting
TheroutingofFoxxhasbeenexposedtomajorinternalchangesweadjustedbecauseofuserfeedback.Thisallowsustosetthe
developmentmodepermountpointwithouthavingtochangepathsandholdappsatseparatelocations.
FoxxDevelopmentmode
Thedevelopmentmodeuseduntil2.4isgone.Ithasbeenreplacedbyamuchmorematureversion.Thisincludesthedeprecationofthe
javascript.dev-app-pathparameter,whichisuselesssince2.5.Insteadofhavingtwoseparateappdirectoriesforproductionand
development,appsnowresideinoneplace,whichisusedforproductionaswellasfordevelopment.Appscanstillbeputinto
developmentmode,changingtheirbehaviorcomparedtoproductionmode.Developmentmodeappsarestillrereadfromdiskatevery
request,andstilltheyshipmoredebugoutput.
Thischangehasalsomadethestartupoptions --javascript.frontend-development-modeand --javascript.dev-app-pathobsolete.The
formeroptionwillnothaveanyeffectwhenset,andthelatteroptionisonlyreadandusedduringtheupgradeto2.5anddoesnothave
anyeffectslater.
Foxxinstallprocess
InstallingFoxxappshasbeenatwostepprocess:importthemintoArangoDBandmountthemataspecificmountpoint.These
operationshavebeenjoinedtogether.Youcaninstallanappatonemountpoint,that'sit.Nofetch,mount,unmount,purgecycle
anymore.Thecommandshavebeensimplifiedtojust:
install:getyourFoxxappupandrunning
uninstall:shutitdownanderaseitfromdisk
Foxxerroroutput
Until2.4theerrorsproducedbyFoxxwerenotoptimal.Often,theerrormessagewasjust unabletoparsemanifestandcontained
onlyaninternalstacktrace.In2.5wemademajorimprovementsthere,includingamuchmorefinegrainederroroutputthathelpsyou
debugyourFoxxapps.Theerrormessageprintedisnowmuchclosertoitssourceandshouldhelpyoutrackitdown.
AlsoweaddedthedefaulthandlersforunhandlederrorsinFoxxapps:
YouwillgetaniceinternalerrorpagewheneveryourFoxxappiscalledbutwasnotinstalledduetoanyerror
WhatsNewin2.5
406
Youwillgetapropererrormessagewhenhavinganuncaughterrorappearsinanyapproute
InproductionmodethemessagesabovewillNOTcontainanyinformationaboutyourFoxxinternalsandaresafetobeexposedtothird
partyusers.Indevelopmentmodethemessagesabovewillcontainthestacktrace(ifavailable),makingiteasierforyourin-housedevsto
trackdownerrorsintheapplication.
Foxxconsole
Weaddeda consoleobjecttoFoxxapps.AllFoxxappsnowhaveaconsoleobjectimplementingthefamiliarConsoleAPIintheir
globalscope,whichcanbeusedtologdiagnosticmessagestothedatabase.Thisconsolealsoallowstoreadtheerroroutputofone
specificfoxx.
Foxxrequests
Weadded org/arangodb/requestmodule,whichprovidesasimpleAPIformakingHTTPrequeststoexternalservices.Thisisenables
Foxxtobedirectlypartofamicroservicearchitecture.
WhatsNewin2.5
407
V8versionupgrade
Thebuilt-inversionofV8hasbeenupgradedfrom3.16.14to3.29.59.ThisactivatesseveralES6(alsodubbedHarmonyorES.next)
featuresinArangoDB,bothintheArangoShellandtheArangoDBserver.Theycanbeusedforscriptingandinserver-sideactionssuch
asFoxxroutes,traversalsetc.
ThefollowingES6featuresareavailableinArangoDB2.4bydefault:
iterators
the ofoperator
symbols
predefinedcollectionstypes(M ap,Setetc.)
typedarrays
M anyotherES6featuresaredisabledbydefault,butcanbemadeavailablebystartingarangodorarangoshwiththeappropriateoptions:
arrowfunctions
proxies
generators
String,Array,andNumberenhancements
constants
enhancedobjectandnumericliterals
ToactivatealltheseES6featuresinarangodorarangosh,startitwiththefollowingoptions:
arangosh--javascript.v8-options="--harmony--harmony_generators"
M oredetailsontheavailableES6featurescanbefoundinthisblog.
FoxxGenerator
ArangoDB2.4isshippedwithFoxxGenerator,aframeworkforbuildingstandardizedHypermediaAPIseasily.ThegeneratedAPIscan
beconsumedwithclienttoolsthatunderstandSiren.
HypermediaisthesimpleideathatourHTTPAPIsshouldhavelinksbetweentheirendpointsinthesamewaythatourwebsiteshave
linksbetweenthem.FoxxGeneratorisbasedontheideathatyoucanrepresentanAPIasastatechart:Everyendpointisastateandthe
linksarethetransitionsbetweenthem.Usingyourdescriptionofstatesandtransitions,itcanthencreateanAPIforyou.
TheFoxxGeneratorcancreateAPIsbasedonasemanticdescriptionofentitiesandtransitions.Ablogseriesontheusecasesandhowto
usetheFoxxgeneratorishere:
part1
part2
part3
AcookbookrecipeforgettingstartedwithFoxxGeneratorishere.
AQLimprovements
WhatsNewin2.4
408
TheAQLoptimizerhasbeenenhancedtouseofindexesinqueriesinseveraladditionalcases.Filterscontainingthe INoperatorcan
nowmakeuseofindexes,andmultipleOR-orAND-combinedfilterconditionscannowalsouseindexesifthefilterconditionsreferto
thesameindexedattribute.
Hereareafewexamplesofqueriesthatcannowuseindexesbutcouldn'tbefore:
FORdocINcollection
FILTERdoc.indexedAttribute==1||doc.indexedAttribute>99
RETURNdoc
FORdocINcollection
FILTERdoc.indexedAttributeIN[3,42]||doc.indexedAttribute>99
RETURNdoc
FORdocINcollection
FILTER(doc.indexedAttribute>2&&doc.indexedAttribute<10)||
(doc.indexedAttribute>23&&doc.indexedAttribute<42)
RETURNdoc
Additionally,theoptimizerrule remove-filter-covered-by-indexhasbeenadded.ThisruleremovesFilterNodesandCalculationNodes
fromanexecutionplanifthefilterconditionisalreadycoveredbyapreviousIndexRangeNode.Removingthefilter'sCalculationNode
andtheFilterNodeitselfwillspeedupqueryexecutionbecausethequeryrequireslesscomputation.
Furthermore,thenewoptimizerrule remove-sort-randwillremovea SORTRAND()statementandmovetherandomiterationintothe
appropriate EnumerateCollectionNode.Thisisusuallymoreefficientthanindividuallyenumeratingandsorting.
Data-modificationqueriesreturningdocuments
INSERT, REMOVE, UPDATEor REPLACEqueriesnowcanoptionallyreturnthedocumentsinserted,removed,updated,orreplaced.This
ishelpfulfortrackingtheauto-generatedattributes(e.g. _key, _rev)createdbyan INSERTandinalotofothersituations.
Inordertoreturndocumentsfromadata-modificationquery,thestatementmustimmediatelybeimmediatelyfollowedbya LET
statementthatassignseitherthepseudo-value NEWor OLDtoavariable.This LETstatementmustbefollowedbya RETURN
statementthatreturnsthevariableintroducedby LET:
FORiIN1..100
INSERT{value:i}INtestLETinserted=NEWRETURNinserted
FORuINusers
FILTERu.status=='deleted'
REMOVEuINusersLETremoved=OLDRETURNremoved
FORuINusers
FILTERu.status=='notactive'
UPDATEuWITH{status:'inactive'}INusersLETupdated=NEWRETURNupdated
NEWreferstotheinsertedormodifieddocumentrevision,and OLDreferstothedocumentrevisionbeforeupdateorremoval. INSERT
statementscanonlyrefertothe NEWpseudo-value,and REMOVEoperationsonlyto OLD. UPDATEand REPLACEcanrefertoeither.
Inallcasesthefulldocumentswillbereturnedwithalltheirattributes,includingthepotentiallyauto-generatedattributessuchas _id,
_key,or _revandtheattributesnotspecifiedintheupdateexpressionofapartialupdate.
COUNTclause
Anoptional COUNTclausewasaddedtothe COLLECTstatement.The COUNTclauseallowsformoreefficientcountingofvalues.
InpreviousversionsofArangoDBonehadtowritethefollowingtocountdocuments:
WhatsNewin2.4
409
RETURNLENGTH(
FORdocINcollection
FILTER...somecondition...
RETURNdoc
)
Withthe COUNTclause,thequerycanbemodifiedto
FORdocINcollection
COLLECTWITHCOUNTINTOlength
RETURNlength
Thelatterquerywillbemuchmoreefficientbecauseitwillnotproduceanyintermediateresultswithneedtobeshippedfroma
subqueryintothe LENGTHfunction.
The COUNTclausecanalsobeusedtocountthenumberofitemsineachgroup:
FORdocINcollection
COLLECTgroup=doc.groupWITHCOUNTINTOlength
return{group:group,length:length}
COLLECTmodifications
InArangoDB2.4, COLLECToperationscanbemademoreefficientifonlyasmallfragmentofthegroupvaluesisneededlater.Forthese
cases, COLLECTprovidesanoptionalconversionexpressionforthe INTOclause.Thisexpressioncontrolsthevaluethatisinsertedinto
thearrayofgroupvalues.Itcanbeusedforprojections.
Thefollowingqueryonlycopiesthe dateRegisteredattributeofeachdocumentintothegroups,potentiallysavingalotofmemoryand
computationtimecomparedtocopying doccompletely:
FORdocINcollection
COLLECTgroup=doc.groupINTOdates=doc.dateRegistered
return{group:group,maxDate:MAX(dates)}
Comparethistothefollowingvariantofthequery,whichwastheonlywaytoachievethesameresultinpreviousversionsof
ArangoDB:
FORdocINcollection
COLLECTgroup=doc.groupINTOdates
return{group:group,maxDate:MAX(dates[*].doc.dateRegistered)}
Theabovequerywillneedtocopythefull docattributeintothe lengthsvariable,whereasthenewvariantwillonlycopythe
dateRegisteredattributeofeach doc.
Subquerysyntax
InpreviousversionsofArangoDB,subqueriesrequiredextraparenthesesaroundthem,andthiscausedconfusionwhensubquerieswere
usedasfunctionparameters.Forexample,thefollowingquerydidnotwork:
LETvalues=LENGTH(
FORdocINcollectionRETURNdoc
)
buthadtobewrittenasfollows:
WhatsNewin2.4
410
LETvalues=LENGTH((
FORdocINcollectionRETURNdoc
))
Thiswasunintuitiveandisfixedinversion2.4sothatbothvariantsofthequeryareacceptedandproducethesameresult.
Webinterface
The ApplicationstabforFoxxapplicationsinthewebinterfacehasgotacompleteredesign.
ItwillnowonlyshowapplicationsthatarecurrentlyrunninginArangoDB.Foraselectedapplication,anewdetailedviewhasbeen
created.Thisviewprovidesabetteroverviewoftheapp,e.g.:
author
license
version
contributors
downloadlinks
APIdocumentation
InstallinganewFoxxapplicationontheserverismadeeasyusingthenew Addapplicationbutton.The Addapplicationdialog
providesallthefeaturesalreadyavailableinthe foxx-managerconsoleapplicationplussomemore:
installaFoxxapplicationfromGithub
installaFoxxapplicationfromazipfile
installaFoxxapplicationfromArangoDB'sapplicationstore
createanewFoxxapplicationfromscratch:thisfeatureusesageneratortocreateaFoxxapplicationwithpre-definedCRUD
methodsforagivenlistofcollections.ThegeneratedFoxxappcaneitherbedownloadedasazipfileorbeinstalledontheserver.
StartingwithanewFoxxapphasneverbeeneasier.
Defaultendpointis127.0.0.1
ThedefaultendpointfortheArangoDBserverhasbeenchangedfrom 0.0.0.0to 127.0.0.1.ThiswillmakenewArangoDB
installationsunaccessiblefromclientsotherthanlocalhostunlesstheconfigurationischanged.Thisisasecurityprecautionmeasurethat
hasbeenrequestedasafeaturealotoftimes.
Ifyouarethedevelopmentoption --enable-relative,theendpointwillstillbe 0.0.0.0.
Systemcollectionsinreplication
Bydefault,systemcollectionsarenowincludedinreplicationandallreplicationAPIreturnvalues.Thiswillleadtouseraccountsand
credentialsdatabeingreplicatedfrommastertoslaveservers.Thismayoverwriteslave-specificdatabaseusers.
Ifthisisundesired,the _userscollectioncanbeexcludedfromreplicationeasilybysettingthe includeSystemattributeto falsein
thefollowingcommands:
replication.sync({includeSystem:false});
replication.applier.properties({includeSystem:false});
Thiswillexcludeallsystemcollections(including _aqlfunctions, _graphsetc.)fromtheinitialsynchronizationandthecontinuous
replication.
Ifthisisalsoundesired,itisalsopossibletospecifyalistofcollectionstoexcludefromtheinitialsynchronizationandthecontinuous
replicationusingthe restrictCollectionsattribute,e.g.:
WhatsNewin2.4
411
require("org/arangodb/replication").applier.properties({
includeSystem:true,
restrictType:"exclude",
restrictCollections:["_users","_graphs","foo"]
});
WhatsNewin2.4
412
AQLimprovements
Frameworkimprovements
AQLqueriesarenowsentthroughaqueryoptimizerframeworkbeforeexecution.Thequeryoptimizerframeworkwillfirstconvertthe
internalrepresentationofthequery,theabstractsyntaxtree,intoaninitialexecutionplan.
Theexecutionplanisthensendthroughoptimizerrulesthatmaydirectlymodifytheplaninplaceorcreateanewvariantoftheplan.
Newplansmightagainbeoptimized,allowingtheoptimizertocarryoutseveraloptimizations.
Aftercreatingplans,theoptimizerwillestimatethecostsforeachplanandpicktheplanwiththelowestcost(termedtheoptimalplan)
fortheactualqueryexecution.
Withthe explain()methodof ArangoStatementuserscancheckwhichexecutionplantheoptimizerpickorretrievealistofother
plansthatoptimizerdidnotchoose.Theplanwillrevealmanydetailsaboutwhichindexesareusedetc. explain()willalsoreturnthe
ofoptimizerrulesappliedsouserscanvalidatewhetherornotaqueryallowsusingaspecificoptimization.
ExecutionofAQLquerieshasbeenrewritteninC++,allowingmanyqueriestoavoidtheconversionofdocumentsbetweenArangoDB's
internallow-leveldatastructureandtheV8objectrepresentationformat.
Theframeworkforoptimizerrulesisnowalsogenerallycluster-aware,allowingspecificoptimizationsforqueriesthatruninacluster.
Additionally,theoptimizerwasdesignedtobeextensibleinordertoaddmoreoptimizationsinthefuture.
Alternativeoperatorsyntax
ArangoDB2.3allowstousethefollowingalternativeformsforthelogicaloperators:
AND:logicaland
OR:logicalor
NOT:negation
Thisnewsyntaxisjustanalternativetotheoldsyntax,allowingeasiermigrationfromSQL.Theoldsyntaxisstillfullysupportedand
willbe:
&&:logicaland
||:logicalor
!:negation
NOTINoperator
AQLnowhasadedicated NOTINoperator.
Previously,a NOTINwasonlyachievablebywritinganegated INcondition:
FORiIN...FILTER!(iIN[23,42])...
InArangoDB2.3,thesameresultcannowalternativelybeachievedbywritingthemoreintuitivevariant:
FORiIN...FILTERiNOTIN[23,42]...
WhatsNewin2.3
413
Improvementsofbuilt-infunctions
ThefollowingAQLstringfunctionshavebeenadded:
LTRIM(value,characters):left-trimsastringvalue
RTRIM(value,characters):right-trimsastringvalue
FIND_FIRST(value,search,start,end):findsthefirstoccurrenceofasearchstring
FIND_LAST(value,search,start,end):findsthelastoccurrenceofasearchstring
SPLIT(value,separator,limit):splitsastringintoanarray,usingaseparator
SUBSTITUTE(value,search,replace,limit):replacescharactersorstringsinsideanother
ThefollowingotherAQLfunctionshavebeenadded:
VALUES(document):returnsthevaluesofanobjectasaarray(thisisthecounterparttothealreadyexisting ATTRIBUTESfunction)
ZIP(attributes,values):returnsanobjectconstructedfromattributesandvaluespassedinseparateparameters
PERCENTILE(values,n,method):returnsthenthspercentileofthevaluesprovided,usingrankorinterpolationmethod
Thealreadyexistingfunctions CONCATand CONCAT_SEPARATORnowsupportarrayarguments,e.g.:
/*"foobarbaz"*/
CONCAT(['foo','bar','baz'])
/*"foo,bar,baz"*/
CONCAT_SEPARATOR(",",['foo','bar','baz'])
AQLqueriesthrowlessexceptions
InpreviousversionsofArangoDB,AQLqueriesabortedwithanexceptioninmanysituationsandthrewaruntimeexception.For
example,exceptionswerethrownwhentryingtofindavalueusingthe INoperatorinanon-arrayelement,whentryingtousenonbooleanvalueswiththelogicaloperands &&or ||or !,whenusingnon-numericvaluesinarithmeticoperations,whenpassing
wrongparametersintofunctionsetc.
ThefactthatmanyAQLoperatorscouldthrowexceptionsledtoalotofquestionsfromusers,andalotofmore-verbose-than-necessary
queries.Forexample,thefollowingqueryfailedwhenthereweredocumentsthatdidnothavea topicsattributeatall:
FORdocINmycollection
FILTERdoc.topicsIN["something","whatever"]
RETURNdoc
Thisforceduserstorewritethequeryasfollows:
FILTERIS_LIST(doc.topics)&&doc.topicsIN["something","whatever"]
RETURNdoc
InArangoDB2.3thishasbeenchangedtomakeAQLeasiertouse.Thechangeprovidesanextrabenefit,andthatisthatnon-throwing
operatorsallowthequeryoptimizertoperformmuchmoretransformationsinthequerywithoutchangingitsoverallresult.
Hereisasummaryofchanges:
whenanon-arrayvalueisusedontheright-handsideofthe INoperator,theresultwillbe falseinArangoDB2.3,andno
exceptionwillbethrown.
thebooleanoperators &&and ||donotthrowinArangoDB2.3ifanyoftheoperandsisnotabooleanvalue.Instead,theywill
performanimplicitcastofthevaluestobooleans.Theirresultwillbeasfollows:
lhs&&rhswillreturn lhsifitis falseorwouldbe falsewhenconvertedintoaboolean.If lhsis trueorwould
be truewhenconvertedtoaboolean, rhswillbereturned.
lhs||rhswillreturn lhsifitis trueorwouldbe truewhenconvertedintoaboolean.If lhsis falseorwouldbe
falsewhenconvertedtoaboolean, rhswillbereturned.
!valuewillreturnthenegatedvalueof valueconvertedintoaboolean
thearithmeticoperators( +, -, *, /, %)canbeappliedtoanyvalueandwillnotthrowexceptionswhenappliedtononnumericvalues.Instead,anyvalueusedintheseoperatorswillbecastedtoanumericvalueimplicitly.Ifnonumericresultcanbe
WhatsNewin2.3
414
producedbyanarithmeticoperator,itwillreturn nullinArangoDB2.3.Thisisalsotruefordivisionbyzero.
passingargumentsofinvalidtypesintoAQLfunctionsdoesnotthrowaruntimeexceptioninmostcases,butmayproduceruntime
warnings.Built-inAQLfunctionsthatreceiveinvalidargumentswillthenreturn null.
Non-uniquehashindexes
Theperformanceofinsertionintonon-uniquehashindexeshasbeenimprovedsignificantly.Thisfixesperformanceproblemsincase
attributeswereindexesthatcontainedonlyveryfewdistinctvalues,orwhenmostofthedocumentsdidnotevencontaintheindexed
attribute.Thisalsofixesproblemswhenloadingcollectionswithsuchindexes.
Theinsertiontimenowscaleslinearlywiththenumberofdocumentsregardlessofthecardinalityoftheindexedattribute.
Reverseiterationoverskiplistindexes
AQLqueriescannowuseasortedskiplistindexforreverseiteration.Thisallowsseveralqueriestorunfasterthaninpreviousversions
ofArangoDB.
Forexample,thefollowingAQLquerycannowusetheindexon doc.value:
FILTERdoc.value>23
SORTdoc.valuesDESC
RETURNdoc
PreviousversionsofArangoDBdidnotusetheindexbecauseofthedescending( DESC)sort.
Additionally,thenewAQLoptimizercanuseanindexforsortingnoweveniftheAQLquerydoesnotcontaina FILTERstatement.
ThisoptimizationwasnotavailableinpreviousversionsofArangoDB.
AddedbasicsupportforhandlingbinarydatainFoxx
BufferobjectscannowbeusedwhensettingtheresponsebodyofanyFoxxaction.ThisallowsFoxxactionstoreturnbinarydata.
RequestswithbinarypayloadcanbeprocessedinFoxxapplicationsbyusingthenewmethod res.rawBodyBuffer().Thiswillreturn
theunparsedrequestbodyasaBufferobject.
Thereisnowalsothemethod req.requestParts()availableinFoxxtoretrievetheindividualcomponentsofamultipartHTTPrequest.
Thatcanbeusedforexampletoprocessfileuploads.
Additionally,the res.send()methodhasbeenaddedasaconveniencemethodforreturningstrings,JSONobjectsorBuffersfroma
Foxxaction.Itprovidessomeauto-detectionbasedonitsparametervalue:
res.send("<p>someHTML</p>");//returnsanHTMLstring
res.send({success:true});//returnsaJSONobject
res.send(newBuffer("somebinarydata"));//returnsbinarydata
Theconveniencemethod res.sendFile()cannowbeusedtoreturnthecontentsofafilefromaFoxxaction.Theyfilemaycontain
binarydata:
res.sendFile(applicationContext.foxxFilename("image.png"));
Thefilesystemmethods fs.write()and fs.readBuffer()canbeusedtoworkwithbinarydata,too:
fs.write()willperformanauto-detectionofitssecondparameter'svaluesoitworkswithBufferobjects:
fs.write(filename,"somedata");//savesastringvalueinfile
fs.write(filename,newBuffer("somebinarydata"));//saves(binary)contentsofabuffer
WhatsNewin2.3
415
fs.readBuffer()hasbeenaddedasamethodtoreadthecontentsofanarbitraryfileintoaBufferobject.
Webinterface
Batchdocumentremovalandmovefunctionalityhasbeenaddedtothewebinterface,makingiteasiertoworkwithmultipledocuments
atonce.Additionally,basicJSONimportandexporttoolshavebeenadded.
Command-lineoptionsadded
Thecommand-lineoption --javascript.v8-contextswasaddedtoarangodtoprovidebettercontroloverthenumberofV8contexts
createdinarangod.
Previously,thenumberofV8contextsarangodcreatedatstartupwasequaltothenumberofserverthreads(asspecifiedbyoption -server.threads).
InsomesituationsitmaybemoresensibletocreatedifferentamountsofthreadsandV8contexts.ThisisbecauseeachV8contexts
createdwillconsumememoryandrequiresCPUresourcesforperiodicgarbagecollection.Contrary,serverthreadsdonothavesuchhigh
memoryorCPUfootprint.
Iftheoption --javascript.v8-contextsisnotspecified,thenumberofV8contextscreatedatstartupwillremainequaltothenumberof
serverthreads.ThusnochangeinconfigurationisrequiredtokeepthesamebehaviorasinpreviousArangoDBversions.
Thecommand-lineoption --log.use-local-timewasaddedtoprintdatesandtimesinArangoDB'slogintheserver-localtimezone
insteadofUTC.Ifitisnotset,thetimezonewilldefaulttoUTC.
Theoption --backslash-escapehasbeenaddedtoarangoimp.Specifyingthisoptionwillusethebackslashastheescapecharacterfor
literalquoteswhenparsingCSVfiles.Theescapecharacterforliteralquotesisstillthedoublequotecharacter.
ArangoDB'sbuilt-inHTTPservernowsupportsHTTPpipelining.
TheArangoShelltutorialfromthearangodb.comwebsiteisnowintegratedintotheArangoDBshell.
PowerfulFoxxEnhancements
Withthenewjobqueuefeatureyoucanrunasyncjobstocommunicatewithexternalservices,Foxxqueriesmakewritingcomplex
AQLqueriesmucheasierandFoxxsessionswillhandletheauthenticationandsessionhassleforyou.
FoxxQueries
WritinglongAQLqueriesinJavaScriptcanquicklybecomeunwieldy.Asof2.3ArangoDBbundlestheArangoDBQueryBuilder
modulethatprovidesaJavaScriptAPIforwritingcomplexAQLquerieswithoutstringconcatenation.Allbuilt-infunctionsthataccept
AQLstringsnowsupportquerybuilderinstancesdirectly.AdditionallyFoxxprovidesamethod Foxx.createQueryforcreating
parametrizedqueriesthatcanreturnFoxxmodelsorapplyarbitrarytransformationstothequeryresults.
FoxxSessions
ThesessionfunctionalityinFoxxhasbeencompletelyrewritten.Theold activateAuthenticationAPIisstillsupportedbutmaybe
deprecatedinthefuture.Thenew activateSessionsAPIsupportscookiesorconfigurableheaders,providesoptionalJSONWebToken
andcryptographicsigningsupportandusesthenewsessionsFoxxapp.
ArangoDB2.3providesFoxxappsforusermanagementandsaltedhash-basedauthenticationwhichcanbereplacedwithor
supplementedbyalternativeimplementations.Foranexampleappusingboththebuilt-inauthenticationandOAuth2seetheFoxx
SessionsExampleapp.
FoxxQueues
WhatsNewin2.3
416
FoxxnowprovidesasyncworkersviatheFoxxQueuesAPI.Jobsenqueuedinajobqueuewillbeexecutedasynchronouslyoutsideof
therequest/responsecycleofFoxxcontrollersandcanbeusedtocommunicatewithexternalservicesorperformtasksthattakealong
timetocompleteormayrequiremultipleattempts.
Jobscanbescheduledinadvanceorsettobeexecutedimmediately,thenumberofretryattempts,theretrydelayaswellassuccessand
failurehandlerscanbedefinedforeachjobindividually.Jobtypesthatintegratevariousexternalservicesfortransactionale-mails,logging
andusertrackingcanbefoundintheFoxxappregistry.
Misc
TherequestandresponseobjectsinFoxxcontrollersnowprovidemethodsforreadingandwritingrawcookiesandsignedcookies.
M ountedFoxxappswillnowbeloadedwhenarangodstartsratherthanatthefirstdatabaserequest.Thismayresultinslightlyslower
startuptimes(butafasterresponseforthefirstrequest).
WhatsNewin2.3
417
AQLimprovements
DatamodificationAQLqueries
Uptoincludingversion2.1,AQLsupporteddataretrievaloperationsonly.StartingwithArangoDBversion2.2,AQLalsosupportsthe
followingdatamodificationoperations:
INSERT:insertnewdocumentsintoacollection
UPDATE:partiallyupdateexistingdocumentsinacollection
REPLACE:completelyreplaceexistingdocumentsinacollection
REM OVE:removeexistingdocumentsfromacollection
Data-modificationoperationsarenormallycombinedwithotherAQLstatementssuchasFORloopsandFILTERconditionsto
determinethesetofdocumentstooperateon.Forexample,thefollowingquerywillfindalldocumentsincollectionusersthatmatcha
specificconditionandsettheirstatusvariabletoinactive:
FORuINusers
FILTERu.status=='notactive'
UPDATEuWITH{status:'inactive'}INusers
Thefollowingquerycopiesalldocumentsfromcollectionusersintocollectionbackup:
FORuINusers
INSERTuINbackup
Andthisqueryremovesdocumentsfromcollectionbackup:
FORdocINbackup
FILTERdoc.lastModified<DATE_NOW()-3600
REMOVEdocINbackup
Formoreinformationondata-modificationqueries,pleaserefertoDatamodificationqueries.
Updatablevariables
Previously,thevalueofavariableassignedinanAQLquerywiththe LETkeywordwasnotupdatableinanAQLquery.This
preventedstatementslikethefollowingfrombeingexecutable:
LETsum=0
FORvINvalues
SORTv.year
LETsum=sum+v.value
RETURN{year:v.year,value:v.value,sum:sum}
OtherAQLimprovements
addedAQLTRANSLATEfunction
Thisfunctioncanbeusedtoperformlookupsfromstaticobjects,e.g.
WhatsNewin2.2
418
LETcountryNames={US:"UnitedStates",UK:"UnitedKingdom",FR:"France"}
RETURNTRANSLATE("FR",countryNames)
LETlookup={foo:"foo-replacement",bar:"bar-replacement",baz:"baz-replacement"}
RETURNTRANSLATE("foobar",lookup,"notcontained!")
Write-aheadlog
AllwriteoperationsinanArangoDBserverwillnowbeautomaticallyloggedintheserver'swrite-aheadlog.Thewrite-aheadlogisaset
ofappend-onlylogfiles,anditisusedincaseofacrashrecoveryandforreplication.
Datafromthewrite-aheadlogwilleventuallybemovedintothejournalsordatafilesofcollections,allowingtheservertoremoveolder
write-aheadlogfiles.
Cross-collectiontransactionsinArangoDBshouldbenefitconsiderablybythischange,aslesswritesthaninpreviousversionsare
requiredtoensurethedataofmultiplecollectionsareatomicallyanddurablycommitted.Alldata-modifyingoperationsinside
transactions(insert,update,remove)willwritetheiroperationsintothewrite-aheadlogdirectlynow.Inpreviousversions,such
operationswerebuffereduntilthecommitorrollbackoccurred.Transactionswithmultipleoperationsshouldthereforerequireless
physicalmemorythaninpreviousversionsofArangoDB.
Thedatainthewrite-aheadlogcanalsobeusedinthereplicationcontext.InpreviousversionsofArangoDB,replicatingfromamaster
requiredturningonaspecialreplicationloggeronthemaster.Thereplicationloggercausedanextrawriteoperationintothe_replication
systemcollectionforeachactualwriteoperation.Thisextrawriteisnowsuperfluous.Instead,slavescanreaddirectlyfromthemaster's
write-aheadlogtogetinformedaboutmostrecentdatachanges.Thisremovestheneedtostoredata-modificationoperationsinthe
_replicationcollectionaltogether.
Fortheconfigurationofthewrite-aheadlog,pleaserefertoWrite-aheadlogoptions.
Theintroductionofthewrite-aheadlogalsoremovestheneedtoconfigureandstartthereplicationloggeronamaster.Thoughthe
replicationloggerobjectisstillavailableinArangoDB2.2toensureAPIcompatibility,starting,stopping,orconfiguringitwillhaveno
effect.
Removedsortingofattributenameswhenincollectionshaper
InpreviousversionsofArangoDB,addingadocumentwithpreviouslynot-usedattributenamescausedafullsortofallattribute
namesusedinthecollection.Thesortingwasdonetoensurefastcomparisonsofattributenamesinsomerareedgecases,butit
considerablysloweddowninsertsintocollectionswithmanydifferentorevenuniqueattributenames.
Specializedprimaryindeximplementationtoallowfasterhashtablerebuildingandreducelookupsindatafilesfortheactualvalueof
_key.Thisalsoreducestheamountofrandommemoryaccessesforprimaryindexinserts.
Reclamationofindexmemorywhendeletinglastdocumentincollection
Deletingdocumentsfromacollectiondidnotleadtoindexsizesbeingreduced.Instead,theindexmemorywaskeptallocatedandreusedlaterwhenacollectionwasrefilledwithnewdocuments.Now,indexmemoryofprimaryindexesandhashindexesisreclaimed
instantlywhenthelastdocumentinacollectionisremoved.
Preventbufferingoflongprintresultsinarangosh'sandarangod'sprintcommand
Thischangewillemitbufferedintermediateprintresultsanddiscardtheoutputbuffertoquicklydeliverprintresultstotheuser,
andtopreventconstructingverylargebuffersforlargeresults.
Added insertmethodasanaliasfor save.Documentscannowbeinsertedintoacollectionusingeithermethod:
WhatsNewin2.2
419
db.test.save({foo:"bar"});
db.test.insert({foo:"bar"});
Cleanupofoptionsfordata-modificationoperations
M anyofthedata-modificationoperationshadsignatureswithmanyoptionalboolparameters,e.g.:
db.test.update("foo",{bar:"baz"},true,true,true)
db.test.replace("foo",{bar:"baz"},true,true)
db.test.remove("foo",true,true)
db.test.save({bar:"baz"},true)
Suchlongparameterlistswereunintuitiveandhardtousewhenonlyoneoftheoptionalparametersshouldhavebeenset.
TomaketheAPIsmoreusable,theoperationsnowunderstandthefollowingalternativesignature:
collection.update(key,update-document,options)
collection.replace(key,replacement-document,options)
collection.remove(key,options)
collection.save(document,options)
Examples:
db.test.update("foo",{bar:"baz"},{overwrite:true,keepNull:true,waitForSync:true})
db.test.replace("foo",{bar:"baz"},{overwrite:true,waitForSync:true})
db.test.remove("foo",{overwrite:true,waitForSync:true})
db.test.save({bar:"baz"},{waitForSync:true})
Added --overwriteoptiontoarangoimp
Thisallowsremovingalldocumentsinacollectionbeforeimportingintoitusingarangoimp.
Honorstartupoption --server.disable-statisticswhendecidingwhetherornottostartperiodicstatisticscollectionjobs
Previously,thestatisticscollectionjobswerestartedeveniftheserverwasstartedwiththe --server.disable-statisticsflagbeing
setto true.Nowiftheoptionissetto true,nostatisticswillbecollectedontheserver.
DisallowstoringofJavaScriptobjectsthatcontainJavaScriptnativeobjectsoftype Date, Function, RegExpor External,e.g.
db.test.save({foo:/bar/});
db.test.save({foo:newDate()});
Thiswillnowprint
Error:<data>cannotbeconvertedintoJSONshape:couldnotshapedocument
Previously,objectsofthesetypesweresilentlyconvertedintoanemptyobject(i.e. {})andnowarningwasissued.
Tostoresuchobjectsinacollection,explicitlyconvertthemintostringslikethis:
db.test.save({foo:String(/bar/)});
db.test.save({foo:String(newDate())});
Removedfeatures
MRubyintegrationforarangod
ArangoDBhadanexperimentalM Rubyintegrationinsomeofthepublishbuilds.Thiswasn'tcontinuouslydeveloped,andsoithasbeen
removedinArangoDB2.2.
Thischangehasledtothefollowingstartupoptionsbeingsuperfluous:
WhatsNewin2.2
420
--ruby.gc-interval
--ruby.action-directory
--ruby.modules-path
--ruby.startup-directory
SpecifyingthesestartupoptionswilldonothinginArangoDB2.2,sousingtheseoptionsshouldbeavoidedfromnowonastheymight
beremovedinafutureversionofArangoDB.
Removedstartupoptions
ThefollowingstartupoptionshavebeenremovedinArangoDB2.2.Specifyingthemintheserver'sconfigurationfilewillnotproducean
errortomakemigrationeasier.Still,usageoftheseoptionsshouldbeavoidedastheywillnothaveanyeffectandmightfullyberemoved
inafutureversionofArangoDB:
--database.remove-on-drop
--database.force-sync-properties
--random.no-seed
--ruby.gc-interval
--ruby.action-directory
--ruby.modules-path
--ruby.startup-directory
--server.disable-replication-logger
MultiCollectionGraphs
ArangoDBisamultimodeldatabasewithnativegraphsupport.Inversion2.2thefeaturesforgraphshavebeenimprovedbyintegration
ofanewgraphmodule.Allgraphscreatedwiththeoldmoduleareautomaticallymigratedintothenewmodulebutcanstillbeusedby
theoldmodule.
Newgraphmodule
Uptoincludingversion2.1,ArangoDBofferedamoduleforgraphsandgraphoperations.Thismoduleallowedyoutouseexactlyone
edgecollectiontogetherwithonevertexcollectioninagraph.WithArangoDBversion2.2thisgraphmoduleisdeprecatedandanew
graphmoduleisoffered.Thisnewmoduleallowstocombineanarbitrarynumberofvertexcollectionsandedgecollectionsinthesame
graph.Foreachedgecollectionalistofcollectionscontainingsourceverticesandalistofcollectionscontainingtargetverticescanbe
defined.IfanedgeisstoredArangoDBchecksifthisedgeisvalidinthiscollection.Furthermoreifavertexisremovedfromoneofthe
collectionsallconnectededgeswillberemovedaswell,givingtheguaranteeofnolooseendsinthegraphs.Thelayoutofthegraphcanbe
modifiedatruntimebyaddingorremovingcollectionsandchangingthedefinitionsforedgecollections.Alloperationsonthegraphlevel
aretransactionalbydefault.
GraphsinAQL
M ulticollectiongraphshavebeenaddedtoAQLaswell.Basicfunctionality(gettingvertices,edges,neighbors)canbeexecutedusingthe
entiregraph.Alsomoreadvancedfeatureslikeshortestpathcalculations,characteristicfactorsofthegraphortraversalshavebeen
integratedintoAQL.Forthesefunctionsallgraphscreatedwiththegraphmodulecanbeused.
WhatsNewin2.2
421
NewEdgesIndex
Theedgesindex(usedtostoreconnectionsbetweennodesinagraph)internallyusesanewdatastructure.Thisdatastructureimproves
theperformancewhenpopulatingtheedgeindex(i.e.whenloadinganedgecollection).Forlargegraphsloadingcanbe20timesfaster
thanwithArangoDB2.0.
Additionally,thenewindexfixesperformanceproblemsthatoccurredwhenmanyduplicate _fromor _tovalueswerecontainedin
theindex.Furthermore,thenewindexsupportsfasterremovalofedges.
Finally,whenloadinganexistingcollectionandbuildingtheedgesindexforthecollection,lessmemoryre-allocationswillbeperformed.
Overall,thisshouldconsiderablyspeeduploadingedgecollections.
Thenewindextypereplacestheoldedgesindextypeautomatically,withoutanychangesbeingrequiredbytheenduser.
TheAPIofthenewindexiscompatiblewiththeAPIoftheoldindex.Stillitispossiblethatthenewindexreturnsedgesinadifferent
orderthantheoldindex.Thisisstillconsideredtobecompatiblebecausetheoldindexhadneverguaranteedanyresultordereither.
AQLImprovements
AQLoffersfunctionalitytoworkwithdates.DatesarenodatatypesoftheirowninAQL(neithertheyareinJSON,whichisoften
usedasaformattoshipdataintoandoutofArangoDB).Instead,datesinAQLareinternallyrepresentedbyeithernumbers
(timestamps)orstrings.ThedatefunctionsinAQLprovidemechanismstoconvertfromanumerictimestamptoastringrepresentation
andviceversa.
TherearetwodatefunctionsinAQLtocreatedatesforfurtheruse:
DATE_TIMESTAMP(date)CreatesaUTCtimestampvaluefrom date
DATE_TIMESTAMP(year,month,day,hour,minute,second,millisecond):Sameasbefore,butallowsspecifyingtheindividualdate
componentsseparately.Allparametersafter dayareoptional.
DATE_ISO8601(date):ReturnsanISO8601datetimestringfrom date.ThedatetimestringwillalwaysuseUTCtime,indicatedby
the Zatitsend.
DATE_ISO8601(year,month,day,hour,minute,second,millisecond):sameasbefore,butallowsspecifyingtheindividualdate
componentsseparately.Allparametersafter dayareoptional.
Thesetwoabovedatefunctionsacceptthefollowinginputvalues:
numerictimestamps,indicatingthenumberofmillisecondselapsedsincetheUNIXepoch(i.e.January1st197000:00:00UTC).An
exampletimestampvalueis 1399472349522,whichtranslatesto 2014-05-07T14:19:09.522Z.
datetimestringsinformats YYYY-MM-DDTHH:MM:SS.MMM, YYYY-MM-DDHH:MM:SS.MMM,or YYYY-MM-DD.M illisecondsarealways
optional.
Atimezonedifferencemayoptionallybeaddedattheendofthestring,withthehoursandminutesthatneedtobeaddedor
subtractedtothedatetimevalue.Forexample, 2014-05-07T14:19:09+01:00canbeusedtospecifyaonehouroffset,and 2014-0507T14:19:09+07:30canbespecifiedforsevenandhalfhoursoffset.Negativeoffsetsarealsopossible.Alternativelytoanoffset,a
ZcanbeusedtoindicateUTC/Zulutime.
Anexamplevalueis 2014-05-07T14:19:09.522ZmeaningM ay7th2014,14:19:09and522milliseconds,UTC/Zulutime.Another
examplevaluewithouttimecomponentis 2014-05-07Z.
WhatsNewin2.1
422
Pleasenotethatifnotimezoneoffsetisspecifiedinadatestring,ArangoDBwillassumeUTCtimeautomatically.Thisisdoneto
ensureportabilityofqueriesacrossserverswithdifferenttimezonesettings,andbecausetimestampswillalwaysbeUTC-based.
individualdatecomponentsasseparatefunctionarguments,inthefollowingorder:
year
month
day
hour
minute
second
millisecond
Allcomponentsfollowing dayareoptionalandcanbeomitted.Notethatnotimezoneoffsetscanbespecifiedwhenusing
separatedatecomponents,andUTC/Zulutimewillbeused.
Thefollowingcallsto DATE_TIMESTAMPareequivalentandwillallreturn 1399472349522:
DATE_TIMESTAMP("2014-05-07T14:19:09.522")
DATE_TIMESTAMP("2014-05-07T14:19:09.522Z")
DATE_TIMESTAMP("2014-05-0714:19:09.522")
DATE_TIMESTAMP("2014-05-0714:19:09.522Z")
DATE_TIMESTAMP(2014,5,7,14,19,9,522)
DATE_TIMESTAMP(1399472349522)
Thesameistrueforcallsto DATE_ISO8601thatalsoacceptsvariableinputformats:
DATE_ISO8601("2014-05-07T14:19:09.522Z")
DATE_ISO8601("2014-05-0714:19:09.522Z")
DATE_ISO8601(2014,5,7,14,19,9,522)
DATE_ISO8601(1399472349522)
Theabovefunctionsareallequivalentandwillreturn "2014-05-07T14:19:09.522Z".
Thefollowingdatefunctionscanbeusedwithdatescreatedby DATE_TIMESTAMPand DATE_ISO8601:
DATE_DAYOFWEEK(date):Returnstheweekdaynumberof date.Thereturnvalueshavethefollowingmeanings:
0:Sunday
1:M onday
2:Tuesday
3:Wednesday
4:Thursday
5:Friday
6:Saturday
DATE_YEAR(date):Returnstheyearpartof dateasanumber.
DATE_MONTH(date):Returnsthemonthpartof dateasanumber.
DATE_DAY(date):Returnsthedaypartof dateasanumber.
DATE_HOUR(date):Returnsthehourpartof dateasanumber.
DATE_MINUTE(date):Returnstheminutepartof dateasanumber.
DATE_SECOND(date):Returnsthesecondspartof dateasanumber.
DATE_MILLISECOND(date):Returnsthemillisecondspartof dateasanumber.
Thefollowingotherdatefunctionsarealsoavailable:
DATE_NOW():Returnsthecurrenttimeasatimestamp.
Notethatthisfunctionisevaluatedoneveryinvocationandmayreturndifferentvalueswheninvokedmultipletimesinthesame
query.
ThefollowingotherAQLfunctionshavebeenaddedinArangoDB2.1:
WhatsNewin2.1
423
FLATTEN:thisfunctioncanturnanarrayofsub-arraysintoasingleflatarray.Allarrayelementsintheoriginalarraywillbe
expandedrecursivelyuptoaconfigurabledepth.Theexpandedvalueswillbeaddedtothesingleresultarray.
Example:
FLATTEN([1,2,[3,4],5,[6,7],[8,[9,10]])
willexpandthesub-arraysonthefirstlevelandproduce:
[1,2,3,4,5,6,7,8,[9,10]]
Tofullyflattenthearray,themaximumdepthcanbespecified(e.g.withavalueof 2):
FLATTEN([1,2,[3,4],5,[6,7],[8,[9,10]],2)
Thiswillfullyexpandthesub-arraysandproduce:
[1,2,3,4,5,6,7,8,9,10]
CURRENT_DATABASE:thisfunctionwillreturnthenameofthedatabasethecurrentqueryisexecutedin.
CURRENT_USER:thisfunctionreturnsthenameofthecurrentuserthatisexecutingthequery.Ifauthorizationisturnedofforthe
queryisexecutedoutsideofarequestcontext,nouserispresentandthefunctionwillreturn null.
ClusterDumpandRestore
Thedumpandrestoretools,arangodumpandarangorestore,cannowbeusedtodumpandrestorecollectionsinacluster.Additionally,
acollectiondumpfromastandaloneArangoDBservercanbeimportedintoacluster,andviceversa.
WebInterfaceImprovements
Thewebinterfaceinversion2.1hasamorecompactdashboard.Itprovideschartswithtime-seriesforincomingrequests,HTTPtransfer
volumeandsomeserverresourceusagefigures.
Additionallyitprovidestrendindicators(e.g.15minaverages)anddistributioncharts(akahistogram)forsomefigures.
FoxxImprovements
ToeasilyaccessafileinsidethedirectoryofaFoxxapplicationfromwithinFoxx,Foxx's applicationContextnowprovidesthe
foxxFilename()function.Itcanbeusedtoassemblethefullfilenameofafileinsidetheapplication'sdirectory.The
applicationContextcanbeaccessedasglobalvariablefromanymodulewithinaFoxxapplication.
ThefilenamecanbeusedinsideFoxxactionsorsetup/teardownscripts,e.g.topopulateaFoxxapplication'scollectionwithdata.
The requirefunctionnowalsopreferslocalmoduleswhenusedfrominsideaFoxxapplication.Thisallowsputtingmodulesinsidethe
Foxxapplicationdirectoryandrequiringthemeasily.Italsoallowsusingapplication-specificversionsoflibrariesthatarebundledwith
ArangoDB(suchasunderscore.js).
WindowsInstaller
TheWindowsinstallershippedwithArangoDBnowsupportsinstallationofArangoDBforthecurrentuserorallusers,withthe
requiredprivileges.ItalsosupportstheinstallationofArangoDBasaservice.
Fixesfor32bitsystems
WhatsNewin2.1
424
SeveralissueshavebeenfixedthatoccurredonlywhenusingArangoDBona32bitsoperatingsystem,specifically:
acrashinathirdpartycomponentusedtomanageclusterdata
athirdpartylibrarythatfailedtoinitializeon32bitWindows,makingarangodandarangoshcrashimmediately.
overflowsofvaluesusedfornanosecond-precisiontimeouts:theseoverflowshaveledtoinvalidvaluesbeingpassedtosocket
operations,makingthemfailandre-trytoooften
Updateddrivers
SeveraldriversforArangoDBhavebeencheckedforcompatibilitywith2.1.Thecurrentlistofdriverswithcompatibilitynotescanbe
foundonlinehere.
C++11usage
WehavemovedseveralfilesfromCtoC++,allowingmorecodereuseandreducingtheneedforshippingdatabetweenthetwo.Wehave
alsodecidedtorequireC++11supportforArangoDB,whichallowsustousesomeofthesimplifications,featuresandguaranteesthat
thisstandardhasinstock.
ThatalsomeansacompilerwithC++11supportisrequiredtobuildArangoDBfromsource.ForinstanceGNUCCofatleastversion
4.8.
MiscellaneousImprovements
Cancelableasynchronousjobs:severalpotentiallylong-runningjobscannowbecanceledviaanexplicitcanceloperation.This
allowsstoppinglong-runningqueries,traversalsorscriptswithoutshuttingdownthecompleteArangoDBprocess.Jobcancelation
isprovidedforasynchronouslyexecutedjobsasisdescribedin@refHttpJobCancel.
Server-sideperiodictaskmanagement:anArangoDBservernowprovidesfunctionalitytoregisterandunregisterperiodictasks.
Tasksareuser-definedJavaScriptactionsthatcanberunperiodicallyandautomatically,independentofanyHTTPrequests.
Thefollowingtaskmanagementfunctionsareprovided:
require("org/arangodb/tasks").register():registersaperiodictask
require("org/arangodb/tasks").unregister():unregistersandremovesaperiodictask
require("org/arangodb/tasks").get():retrievesaspecifictasksorallexistingtasks
Anexampletask(tobeexecutedevery15seconds)canberegisteredlikethis:
vartasks=require("org/arangodb/tasks");
tasks.register({
name:"thisisanexampletaskwithparameters",
period:15,
command:function(params){
vargreeting=params.greeting;
vardata=JSON.stringify(params.data);
require('console').log('%sfromparametertask:%s',greeting,data);
},
params:{greeting:"hi",data:"howareyou?"}
});
Pleaserefertothesection@refTasksformoredetails.
The figuresmethodofacollectionnowreturnsdataaboutthecollection'sindexmemoryconsumption.Thereturnedvalue
indexes.sizewillcontainthetotalamountofmemoryacquiredbyallindexesofthecollection.Thisfigurecanbeusedtoassess
thememoryimpactofindexes.
CapitalizedHTTPresponseheaders:fromversion2.1,ArangoDBwillreturncapitalizedHTTPheadersbydefault,e.g. ContentLengthinsteadof content-length.ThoughtheHTTPspecificationstatesthatheadersfieldnamearecase-insensitive,severalolder
clienttoolsrelyonaspecificcaseinHTTPresponseheaders.ThischangesmakeArangoDBabitmorecompatiblewiththose.
WhatsNewin2.1
425
Simplifiedusageof db._createStatement():toeasilyrunanAQLquery,themethod db._createStatementnowallowspassingthe
AQLqueryasastring.Previouslyitrequiredtheusertopassanobjectwitha queryattribute(whichthencontainedthequery
string).
ArangoDBnowsupportsbothversions:
db._createStatement(queryString);
db._createStatement({query:queryString});
WhatsNewin2.1
426
IncompatiblechangesinArangoDB3.0
ItisrecommendedtocheckthefollowinglistofincompatiblechangesbeforeupgradingtoArangoDB3.0,andadjustanyclientprograms
ifnecessary.
Buildsystem
BuildingArangoDB3.0fromsourcenowrequiresCM ake.
Thepre-3.0buildsystemusedaconfigure-basedapproach.ThestepstobuildArangoDB2.8fromsourcecodewere:
makesetup
./configure<options>
make
Thesestepswillnotworkanymore,asArangoDB3.0doesnotcomewithaconfigurescript.
Tobuild3.0onLinux,createaseparatebuilddirectoryfirst:
mkdir-pbuild
andthencreatetheinitialbuildscriptsonceusingCM ake:
(cdbuild&&cmake<options>..)
Theabovecommandwillconfigurethebuildandcheckfortherequireddependencies.Ifeverythingworkswelltheactualbuildcanbe
startedwith
(cdbuild&&make)
ThebinariesfortheArangoDBserverandallclienttoolswillthenbecreatedinsidethe builddirectory.TostartArangoDBlocally
fromthe builddirectory,use
build/bin/arangod<options>
Datafilesanddatafilenames
ArangoDB3.0usesanewVelocyPack-basedformatforstoringdatainWALlogfilesandcollectiondatafiles.Thefileformatisnot
compatiblewiththefilesusedinpriorversionsofArangoDB.ThatmeansdatafileswrittenbyArangoDB3.0cannotbeusedinearlier
versionsandviceversa.
Thepatternforcollectiondirectorynameswaschangedin3.0toincludearandomidcomponentattheend.Thenewpatternis
collection-<id>-<random>,where <id>isthecollectionidand <random>isarandomnumber.PreviousversionsofArangoDBuseda
pattern collection-<id>withouttherandomnumber.
UserManagement
UnlikeArangoDB2.x,ArangoDB3.0usersarenowseparatedfromdatabases,andyoucangrantoneormoredatabasepermissionstoa
user.
IfyouwanttomimicthebehaviorofArangoDB,youshouldnameyouruserslike username@dbname.
Usersthatcanaccessthe_systemdatabaseareallowedtomanageusersandpermissionsforalldatabases.
427
Edgesandedgesattributes
InArangoDBpriorto3.0theattributes _fromand _toofedgesweretreatedspeciallywhenloadingorstoringedges.Thatspecial
handlingledtotheseattributesbeingnotasflexibleasregulardocumentattributes.Forexample,the _fromand _toattributevaluesof
anexistingedgecouldnotbeupdatedoncetheedgewascreated.Additionally,the _fromand _toattributescouldnotbeindexedin
user-definedindexes,e.g.tomakeeachcombinationof _fromand _tounique.Finally,as _fromand _toreferencedthelinked
collectionsbycollectionidandnotbycollectionname,theirmeaningbecameunclearonceareferencedcollectionwasdropped.The
collectionidstoredinedgesthenbecameunusable,andwhenaccessingsuchedgethecollectionnamepartofitwasalwaystranslatedto
_undefined.
InArangoDB3.0,the _fromand _tovaluesofedgesaresavedasregularstrings.Thisallowsusing _fromand _toinuser-defined
indexes.Additionallythisallowsupdatingthe _fromand _tovaluesofexistingedges.Furthermore,collectionsreferencedby _from
and _tovaluesmaybedroppedandre-createdlater.Any _fromand _tovaluesofedgespointingtosuchdroppedcollectionare
unaffectedbythedropoperationnow.Alsonotethatrenamingthecollectionreferencedin _fromand _toinArangoDB2.8also
relinkedtheedges.In3.0theedgesareNOTautomaticallyrelinkedtothenewcollectionanymore.
Documents
Documents(incontrasttoedges)cannotcontaintheattributes _fromor _toonthemainlevelinArangoDB3.0.Theseattributeswill
beautomaticallyremovedwhensavingdocuments(i.e.non-edges). _fromand _tocanbestillusedinsub-objectsinsidedocuments.
The _fromand _toattributeswillofcoursebepreservedandarestillrequiredwhensavingedges.
AQL
Edgeshandling
WhenupdatingorreplacingedgesviaAQL,anymodificationstothe _fromand _toattributesofedgeswereignoredbyprevious
versionsofArangoDB,withoutsignalinganyerrors.Thiswasduetothe _fromand _toattributesbeingimmutableinearlierversions
ofArangoDB.
From3.0on,the _fromand _toattributesofedgesaremutable,soanyAQLqueriesthatmodifythe _fromor _toattributevalues
ofedgeswillattempttoactuallychangetheseattributes.Clientsshouldbeawareofthischangeandshouldreviewtheirqueriesthat
modifyedgestoruleoutunintendedside-effects.
Additionally,whencompletelyreplacingthedataofexistingedgesviatheAQL REPLACEoperation,itisnowrequiredtospecifyvalues
forthe _fromand _toattributes,as REPLACErequirestheentirenewdocumenttobespecified.Ifeither _fromor _toaremissing
fromthereplacementdocument,an REPLACEoperationwillfail.
Graphfunctions
Inversion3.0allformergraphrelatedfunctionshavebeenremovedfromAQLtobereplacedbynativeAQLconstructs.These
constructsallowformorefine-grainedfilteringonseveralgraphlevels.AlsothisallowstheAQLoptimizertoautomaticallyimprove
thesequeriesbyenhancingthemwithappropriateindexes.Wehavecreatedrecipestoupgradefrom2.8to3.0whenusingthese
functions.
Thefunctions:
GRAPH_COM M ON_NEIGHBORS
GRAPH_COM M ON_PROPERTIES
GRAPH_DISTANCE_TO
GRAPH_EDGES
GRAPH_NEIGHBORS
GRAPH_TRAVERSAL
GRAPH_TRAVERSAL_TREE
GRAPH_SHORTEST_PATH
GRAPH_PATHS
428
GRAPH_VERTICES
arecoveredinM igratingGRAPH_*Functionsfrom2.8orearlierto3.0
GRAPH_ABSOLUTE_BETWEENNESS
GRAPH_ABSOLUTE_CLOSENESS
GRAPH_ABSOLUTE_ECCENTRICITY
GRAPH_BETWEENNESS
GRAPH_CLOSENESS
GRAPH_DIAM ETER
GRAPH_ECCENTRICITY
GRAPH_RADIUS
arecoveredinM igratingGRAPH_*M easurementsfrom2.8orearlierto3.0
EDGES
NEIGHBORS
PATHS
TRAVERSAL
TRAVERSAL_TREE
arecoveredinM igratinganonymousgraphfunctionsfrom2.8orearlierto3.0
Typecastingfunctions
Thetypecastingappliedbythe TO_NUMBER()AQLfunctionhaschangedasfollows:
stringvaluesthatdonotcontainavalidnumericvaluearenowconvertedtothenumber 0.InpreviousversionsofArangoDBsuch
stringvalueswereconvertedtothevalue null.
arrayvalueswithmorethan1memberarenowconvertedtothenumber 0.InpreviousversionsofArangoDBsucharrayswere
convertedtothevalue null.
objects/documentsarenowconvertedtothenumber 0.InpreviousversionsofArangoDBobjects/documentswereconvertedto
thevalue null.
Additionally,the TO_STRING()AQLfunctionnowconverts nullvaluesintoanemptystring( "")insteadofthestring "null",
whichismoreinlinewith LENGTH(null)returning 0andnot 4sincev2.6.
Theoutputof TO_STRING()hasalsochangedforarraysandobjectsasfollows:
arraysarenowconvertedintotheirJSON-stringifyequivalents,e.g.
[]isnowconvertedto []
[1,2,3]isnowconvertedto [1,2,3]
["test",1,2]isnowconvertedto["test",1,2]`
PreviousversionsofArangoDBconvertedarrayswithnomembersintotheemptystring,andnon-emptyarraysintoacommaseparatedlistofmembervalues,withoutthesurroundingangularbrackets.Additionally,stringarraymemberswerenotenclosedin
quotesintheresultstring:
[]wasconvertedto``
[1,2,3]wasconvertedto 1,2,3
["test",1,2]wasconvertedtotest,1,2`
objectsarenowconvertedtotheirJSON-stringifyequivalents,e.g.
{}isconvertedto {}
{a:1,b:2}isconvertedto {"a":1,"b":2}
{"test":"foobar"}isconvertedto {"test":"foobar"}
PreviousversionsofArangoDBalwaysconvertedobjectsintothestring [objectObject]
ThischangealsoaffectsotherpartsinAQLthatused TO_STRING()toimplicitlycastoperandstostrings.ItalsoaffectstheAQL
functions CONCAT()and CONCAT_SEPARATOR()whichtreatedarrayvaluesdifferently.PreviousversionsofArangoDBautomatically
flattenedarrayvaluesinthefirstlevelofthearray,e.g. CONCAT([1,2,3,[4,5,6]])produced 1,2,3,4,5,6.Nowthiswillproduce
429
[1,2,3,[4,5,6]].Toflattenarraymembersonthetoplevel,youcannowusethemoreexplicit CONCAT(FLATTEN([1,2,3,[4,5,6]],
1)).
Arithmeticoperators
AsthearithmeticoperationsinAQLimplicitlyconverttheiroperandstonumericvaluesusing TO_NUMBER(),theircastingbehaviorhas
alsochangedasdescribedabove.
Someexamplesofthechangedbehavior:
"foo"+1produces 1now.Inpreviousversionsthisproduced null.
[1,2]+1produces 1.Inpreviousversionsthisproduced null.
1+"foo"+1´produces2 now.Inpreviousversionthisproduced1`.
Attributenamesandparameters
PreviousversionsofArangoDBhadsometroublewithattributenamesthatcontainedthedotsymbol( .).SomecodepartsinAQL
usedthedotsymboltosplitanattributenameintosub-components,soanattributenamed a.bwasnotcompletelydistinguishable
fromanattribute awithasub-attribute b.Thisinconsistentbehaviorsometimesallowed"hacks"toworksuchaspassingsubattributesinabindparameterasfollows:
FORdocINcollection
FILTERdoc.@name==1
RETURNdoc
Ifthebindparameter @namecontainedthedotsymbol(e.g. @bind= a.b,itwasunclearwhetherthisshouldtriggersub-attribute
access(i.e. doc.a.b)oraaccesstoanattributewithexactlythespecifiedname(i.e. doc["a.b"]).
ArangoDB3.0nowhandlesattributenamescontainingthedotsymbolproperly,andsendingabindparameter @name= a.bwillnow
alwaystriggeranaccesstotheattribute doc["a.b"],notthesub-attribute bof ain doc.
Forusersthatusedthe"hack"ofpassingbindparameterscontainingdotsymboltoaccesssub-attributes,ArangoDB3.0allows
specifyingtheattributenamepartsasanarrayofstrings,e.g. @name= ["a","b"],whichwillberesolvedtothesub-attributeaccess
doc.a.bwhenthequeryisexecuted.
Keywords
LIKEisnowakeywordinAQL.Using LIKEineithercaseasanattributeorcollectionnameinAQLqueriesnowrequiresquoting.
SHORTEST_PATHisnowakeywordinAQL.Using SHORTEST_PATHineithercaseasanattributeorcollectionnameinAQLqueriesnow
requiresquoting.
Subqueries
Queriesthatcontainsubqueriesthatcontaindata-modificationoperationssuchas INSERT, UPDATE, REPLACE, UPSERTor REMOVE
willnowrefusetoexecuteifthecollectionaffectedbythesubquery'sdata-modificationoperationisread-accessedinanouterscopeof
thequery.
Forexample,thefollowingquerywillrefusetoexecuteasthecollection myCollectionismodifiedinthesubquerybutalsoreadaccessedintheouterscope:
FORdocINmyCollection
LETchanges=(
FORwhatINmyCollection
FILTERwhat.value==1
REMOVEwhatINmyCollection
)
RETURNdoc
Itisstillpossibletowritetocollectionsfromwhichdataisreadinthesamequery,e.g.
430
FORdocINmyCollection
FILTERdoc.value==1
REMOVEdocINmyCollection
andtomodifydataindifferentcollectionviasubqueries.
Otherchanges
TheAQLoptimizerrule"merge-traversal-filter"thatalreadyexistedin3.0wasrenamedto"optimize-traversals".Thisshouldbeofno
relevancetoclientapplicationsexceptiftheyprogramaticallylookforappliedoptimizerrulesintheexplainoutofAQLqueries.
TheorderofresultscreatedbytheAQLfunctions VALUES()and ATTRIBUTES()wasneverguaranteedanditonlyhadthe"correct"
orderingbyaccidentwheniteratingoverobjectsthatwerenotloadedfromthedatabase.Assomeofthefunctioninternalshavechanged,
the"correct"orderingwillnotappearanymore,andstillnoresultorderisguaranteedbythesefunctionsunlessthe sortparameteris
specified(forthe ATTRIBUTES()function).
UpgradedV8version
TheV8enginethatisusedinsideArangoDBtoexecuteJavaScriptcodehasbeenupgradedfromversion4.3.61to5.0.71.39.Thenew
versionshouldbemostlycompatibletotheoldversion,buttheremaybesubtledifferences,includingchangesoferrormessagetexts
thrownbytheengine.Furthermore,someV8startupparametershavechangedtheirmeaningorhavebeenremovedinthenewversion.
ThisisonlyrelevantwhenArangoDBorArangoShellarestartedwithacustomvalueforthe --javascript.v8-optionsstartupoption.
Amongothers,thefollowingV8optionschangeinthenewversionofArangoDB:
--es_staging:in2.8ithadthemeaning enableallcompletedharmonyfeatures,in3.0theoptionmeans enabletest-worthy
harmonyfeatures(forinternaluseonly)
--strong_this:thisoptionwasn'tpresentin2.8.In3.0itmeans don'tallow'this'toescapefromconstructorsanddefaultsto
true.
--harmony_regexps:thisoptionsmeans enable"harmonyregularexpressionextensions"andchangesitsdefaultvaluefromfalse
totrue
--harmony_proxies:thisoptionsmeans enable"harmonyproxies"andchangesitsdefaultvaluefromfalsetotrue
--harmony_reflect:thisoptionsmeans enable"harmonyReflectAPI"andchangesitsdefaultvaluefromfalsetotrue
--harmony_sloppy:thisoptionsmeans enable"harmonyfeaturesinsloppymode"andchangesitsdefaultvaluefromfalsetotrue
--harmony_tostring:thisoptionsmeans enable"harmonytoString"andchangesitsdefaultvaluefromfalsetotrue
--harmony_unicode_regexps:thisoptionsmeans enable"harmonyunicoderegexps"andchangesitsdefaultvaluefromfalsetotrue
--harmony_arrays, --harmony_array_includes, --harmony_computed_property_names, --harmony_arrow_functions, -harmony_rest_parameters, --harmony_classes, --harmony_object_literals, --harmony_numeric_literals, --harmony_unicode:
theseoptionhavebeenremovedinV85.
AsaconsequenceoftheupgradetoV8version5,theimplementationoftheJavaScript Bufferobjecthadtobechanged.JavaScript
BufferobjectsinArangoDBnowalwaysstoretheirdataontheheap.ThereisnosharedpoolforsmallBuffervalues,andnopointing
intoexistingBufferdatawhenextractingslices.ThischangemayincreasethecostofcreatingBufferswithshortcontentsorwhen
peekingintoexistingBuffers,butwasrequiredforsafermemorymanagementandtopreventleaks.
JavaScriptAPIchanges
ThefollowingincompatiblechangeshavebeenmadetotheJavaScriptAPIinArangoDB3.0:
Foxx
TheFoxxframeworkhasbeencompletelyrewrittenfor3.0withanew,simplerandmorefamiliarAPI.TomakeFoxxservicesdeveloped
for2.8orearlierArangoDBversionsrunin3.0,theservice'smanifestfileneedstobeedited.
431
ToenablethelegacymodeforaFoxxservice,add "engines":{"arangodb":"^2.8.0"}(orsimilarversionrangesthatexclude3.0andup)
totheservicemanifestfile(named"manifest.json",locatedintheservice'sbasedirectory).
Require
M odulesshippedwithArangoDBcannowberequiredusingthepattern @arangodb/<module>insteadof org/arangodb/<module>,e.g.
varcluster=require("@arangodb/cluster");
Theoldformatcanstillbeusedforcompatibility:
varcluster=require("org/arangodb/cluster");
ArangoDBpriortoversion3.0allowedatransparentuseofCoffeeScriptsourcefileswiththe require()function.Fileswithafile
nameextensionof coffeewereautomaticallysentthroughaCoffeeScriptparserandtranspiledintoJavaScripton-the-fly.Thissupport
isgonewithArangoDB3.0.TorunanyCoffeeScriptsourcefiles,theymustbeconvertedtoJavaScriptbytheclientapplication.
Responseobject
The @arangodb/requestresponseobjectnowstorestheparsedJSONresponsebodyinaproperty jsoninsteadof bodywhenthe
requestwasmadeusingthe jsonoption.The bodyinsteadcontainstheresponsebodyasastring.
EdgesAPI
Whencompletelyreplacinganedgeviaacollection's replace()functionthereplacingedgedatanowneedstocontainthe _fromand
_toattributesforthenewedge.PreviousversionsofArangoDBdidnotrequiretheedgedatatocontain _fromand _toattributes
whenreplacinganedge,since _fromand _tovalueswereimmutableforexistingedges.
Forexample,thefollowingcallworkedinArangoDB2.8butwillfailin3.0:
db.edgeCollection.replace("myKey",{value:"test"});
TomakethisworkinArangoDB3.0, _fromand _toneedtobeaddedtothereplacementdata:
db.edgeCollection.replace("myKey",{_from:"myVertexCollection/1",_to:"myVertexCollection/2",value:"test"});
Notethatthisonlyaffectsthe replace()functionbutnot update(),whichwillonlyupdatethespecifiedattributesoftheedgeand
leaveallothersintact.
Additionally,thefunctions edges(), outEdges()and inEdges()withanarrayofedgeidswillnowmaketheedgeidsuniquebefore
returningtheconnectededges.Thisisprobablydesiredanyway,asresultswillbereturnedonlyonceperdistinctinputedgeid.However,
itmaybreakclientapplicationsthatrelyontheoldbehavior.
DatabasesAPI
The _listDatabases()functionofthe dbobjecthasbeenrenamedto _databases(),makingitconsistentwiththe _collections()
function.Alsothe _listEndpoints()functionhasbeenrenamedto _endpoints().
CollectionAPI
Examplematching
Thecollectionfunction byExampleHash()and byExampleSkiplist()havebeenremovedin3.0.Theirfunctionalityisprovidedby
collection's byExample()function,whichwillautomaticallyuseasuitableindexifpresent.
Thecollectionfunction byConditionSkiplist()hasbeenremovedin3.0.ThesamefunctionalitycanbeachievedbyissuinganAQL
querywiththetargetcondition,whichwillautomaticallyuseasuitableindexifpresent.
432
Revisionidhandling
The exists()methodofacollectionnowthrowsanexceptionwhenthespecifieddocumentexistsbutitsrevisioniddoesnotmatch
therevisionidspecified.PreviousversionsofArangoDBsimplyreturned falseifeithernodocumentexistedwiththespecifiedkeyor
whentherevisioniddidnotmatch.Itwasthereforeimpossibletodistinguishthesetwocasesfromthereturnvaluealone.3.0corrects
this.Additionally, exists()inpreviousversionsalwaysreturnedabooleanifonlythedocumentkeywasgiven.3.0nowreturnsthe
document'smeta-data,whichincludesthedocument'scurrentrevisionid.
Giventhereisadocumentwithkey testincollection myCollection,thenthebehaviorof3.0isasfollows:
/*testifdocumentexists.thisreturnedtruein2.8*/
db.myCollection.exists("test");
{
"_key":"test",
"_id":"myCollection/test",
"_rev":"9758059"
}
/*testifdocumentexists.thisreturnedtruein2.8*/
db.myCollection.exists({_key:"test"});
{
"_key":"test",
"_rev":"9758059"
}
/*testifdocumentexists.thisalsoreturnedfalsein2.8*/
db.myCollection.exists("foo");
false
/*testifdocumentwithagivenrevisionidexists.thisreturnedtruein2.8*/
db.myCollection.exists({_key:"test",_rev:"9758059"});
{
"_key":"test",
"_rev":"9758059"
}
/*testifdocumentwithagivenrevisionidexists.thisreturnedfalsein2.8*/
db.myCollection.exists({_key:"test",_rev:"1234"});
JavaScriptexception:ArangoError1200:conflict
Capconstraints
Thecapconstraintsfeaturehasbeenremoved.Thischangehasledtotheremovalofthecollectionoperations first()and last(),
whichwereinternallybasedondatafromcapconstraints.
AscapconstraintshavebeenremovedinArangoDB3.0itisnotpossibletocreateanindexoftype"cap"withacollection's
ensureIndex()function.Thededicatedfunction ensureCapConstraint()hasalsobeenremovedfromthecollectionAPI.
GraphBlueprintsJSModule
Thedeprecatedmodule graph-blueprintshasbeendeleted.Allit'sfeaturesarecoveredbythe general-graphmodule.
GeneralGraphFluentAQLinterface
ThefluentinterfacehasbeenremovedfromArangoDB.It'sfeatureswerecompletelyoverlappingwith"aqb"whichcomespreinstalled
aswell.PleaseswitchtoAQBinstead.
UndocumentedAPIs
Theundocumentedfunctions BY_EXAMPLE_HASH()and BY_EXAMPLE_SKIPLIST(), BY_CONDITION_SKIPLIST, CPP_NEIGHBORSand
CPP_SHORTEST_PATHhavebeenremoved.ThesefunctionswerealwayshiddenandnotintendedtobepartofthepublicJavaScriptAPI
forcollections.
433
HTTPAPIchanges
CRUDoperations
ThefollowingincompatiblechangeshavebeenmadetotheHTTPAPIinArangoDB3.0:
General
TheHTTPinsertoperationsforsingledocumentsandedges(POST /_api/document)donotsupporttheURLparameter
"createCollection"anymore.InpreviousversionsofArangoDBthisparametercouldbeusedtoautomaticallycreateacollectionupon
insertionofthefirstdocument.ItisnowrequiredthatthetargetcollectionalreadyexistswhenusingthisAPI,otherwiseitwillreturnan
HTTP404error.ThesameistruefortheimportAPIatPOST /_api/import.
CollectionscanstillbecreatedeasilyviaaseparatecalltoPOST /_api/collectionasbefore.
The"location"HTTPheaderreturnedbyArangoDBwheninsertinganewdocumentoredgenowalwayscontainsthedatabasename.
ThiswasalsothedefaultbehaviorinpreviousversionsofArangoDB,butitcouldbeoverriddenbyclientssendingtheHTTPheader xarango-version:1.4intherequest.ClientscancontinuetosendthisheadertoArangoDB3.0,buttheheaderwillnotinfluencethe
locationresponseheadersproducedbyArangoDB3.0anymore.
AdditionallytheCRUDoperationsAPIsdonotreturnanattribute"error"intheresponsebodywithanattributevalueof"false"incase
anoperationsucceeded.
Revisionidhandling
Theoperationsforupdating,replacingandremovingdocumentscanoptionallychecktherevisionnumberofthedocumenttobeupdated,
replacedorremovedsothecallercanensuretheoperationworksonaspecificversionofthedocumentandtherearenolostupdates.
PreviousversionsofArangoDBallowedpassingtherevisionidofthepreviousdocumenteitherintheHTTPheader If-Matchorinthe
URLparameter rev.Forexample,removingadocumentwithaspecificrevisionidcouldbeachievedasfollows:
curl-XDELETE\
"http://127.0.0.1:8529/_api/document/myCollection/myKey?rev=123"
ArangoDB3.0doesnotsupportpassingtherevisionidviathe"rev"URLparameteranymore.Insteadthepreviousrevisionidmustbe
passedintheHTTPheader If-Match,e.g.
curl-XDELETE\
--header"If-Match:'123'"\
"http://127.0.0.1:8529/_api/document/myCollection/myKey"
TheURLparameter"policy"wasalsousableinpreviousversionsofArangoDBtocontrolrevisionhandling.Usingitwasredundantto
specifyingtheexpectedrevisionidviathe"rev"parameteror"If-M atch"HTTPheaderandthereforesupportforthe"policy"parameter
wasremovedin3.0.
Inordertocheckforapreviousrevisionidwhenupdating,replacingorremovingdocumentspleaseusethe If-MatchHTTPheaderas
describedabove.WhennorevisioncheckifrequiredtheHTTPheadercanbeomitted,andtheoperationswillworkonthecurrent
revisionofthedocument,regardlessofitsrevisionid.
AlldocumentsAPI
TheHTTPAPIforretrievingtheids,keysorURLsofalldocumentsfromacollectionwaspreviouslylocatedatGET /_api/document?
collection=....ThisAPIwasmovedtoPUT /_api/simple/all-keysandisnowexecutedasanAQLquery.Thenameofthe
collectionmustnowbepassedintheHTTPrequestbodyinsteadofintherequestURL.Thesameistrueforthe"type"parameter,
whichcontrolsthetypeoftheresulttobecreated.
CallstothepreviousAPIcanbetranslatedasfollows:
old:GET /_api/document?collection=<collection>&type=<type>withoutHTTPrequestbody
3.0:PUT /_api/simple/all-keyswithHTTPrequestbody {"collection":"<collection>","type":"id"}
434
TheresultformatofthisAPIhasalsochangedslightly.InpreviousversionscallstotheAPIreturnedaJSONobjectwitha documents
attribute.AsthefunctionalityisbasedonAQLinternallyin3.0,theAPInowreturnsaJSONobjectwitha resultattribute.
EdgesAPI
CRUDoperations
TheAPIfordocumentsandedgeshavebeenunifiedinArangoDB3.0.TheCRUDoperationsfordocumentsandedgesarenowhandled
bythesameendpointat /_api/document.ForCRUDoperationsthereisnodistinctionanymorebetweendocumentsandedgesAPIwise.
ThatmeansCRUDoperationsconcerningedgesneedtobesenttotheHTTPendpoint /_api/documentinsteadof /_api/edge.Sending
requeststo /_api/edgewillresultinanHTTP404errorin3.0.Thefollowingmethodsareavailableat /_api/documentfordocuments
andedge:
HTTPPOST:insertnewdocumentoredge
HTTPGET:fetchanexistingdocumentoredge
HTTPPUT:replaceanexistingdocumentoredge
HTTPPATCH:partiallyupdateanexistingdocumentoredge
HTTPDELETE:removeanexistingdocumentoredge
WhencompletelyreplacinganedgeviaHTTPPUTpleasenotethatthereplacingedgedatanowneedstocontainthe _fromand _to
attributesfortheedge.PreviousversionsofArangoDBdidnotrequiresending _fromand _towhenreplacingedges,as _fromand
_tovalueswereimmutableforexistingedges.
The _fromand _toattributesofedgesnowalsoneedtobepresentinsidetheedgesobjectssenttotheserver:
curl-XPOST\
--data'{"value":1,"_from":"myVertexCollection/1","_to":"myVertexCollection/2"}'\
"http://127.0.0.1:8529/_api/document?collection=myEdgeCollection"
PreviousversionsofArangoDBrequiredthe _fromand _toattributesofedgesbesentseparatelyinURLparameter fromand to:
curl-XPOST\
--data'{"value":1}'\
"http://127.0.0.1:8529/_api/edge?collection=e&from=myVertexCollection/1&to=myVertexCollection/2"
Queryingconnectededges
TheRESTAPIforqueryingconnectededgesatGET /_api/edges/<collection>willnowmaketheedgeidsuniquebeforereturningthe
connectededges.Thisisprobablydesiredanywayasresultswillnowbereturnedonlyonceperdistinctinputedgeid.However,itmay
breakclientapplicationsthatrelyontheoldbehavior.
GraphAPI
Somedata-modificationoperationsinthenamedgraphsAPIat /_api/gharialnowreturneitherHTTP202(Accepted)orHTTP201
(Created)iftheoperationsucceeds.Whichstatuscodeisreturneddependsonthe waitForSyncattributeoftheaffectedcollection.In
previousversionssomeoftheseoperationsreturnHTTP200regardlessofthe waitForSyncvalue.
ThedeprecatedgraphAPI /_api/graphhasbeenremoved.Allit'sfeaturescanbereplacedusing /_api/gharialandAQLinstead.
SimplequeriesAPI
TheRESTroutesPUT /_api/simple/firstand /_api/simple/lasthavebeenremovedentirely.TheseAPIswereresponsiblefor
returningthefirst-insertedandlast-inserteddocumentsinacollection.Thisfeaturewasbuiltoncapconstraintsinternally,whichhave
beenremovedin3.0.
Callingoneoftheseendpointsin3.0willresultinanHTTP404error.
IndexesAPI
435
Itisnotsupportedin3.0tocreateanindexwithtype cap(capconstraint)in3.0asthecapconstraintsfeaturehasbeeremoved.
CallingtheindexcreationendpointHTTPAPIPOST /_api/index?collection=...withanindextype capwillthereforeresultinan
HTTP400error.
LogentriesAPI
TheRESTrouteHTTPGET /_admin/logisnowaccessiblefromwithinalldatabases.InpreviousversionsofArangoDB,thisroute
wasaccessiblefromwithinthe _systemdatabaseonly,andanHTTP403(Forbidden)wasthrownbytheserverforanyaccessfrom
withinanotherdatabase.
FiguresAPI
TheRESTrouteHTTPGET /_api/collection/<collection>/figureswillnotreturnthefollowingresultattributesastheybecame
meaninglessin3.0:
shapefiles.count
shapes.fileSize
shapes.count
shapes.size
attributes.count
attributes.size
DatabasesandCollectionsAPIs
WhencreatingadatabaseviatheAPIPOST /_api/database,ArangoDBwillnowalwaysreturntheHTTPstatuscode202(created)if
theoperationsucceeds.PreviousversionsofArangoDBreturnedHTTP202aswell,butthisbehaviorwaschangablebysendingan
HTTPheader x-arango-version:1.4.Whensendingthisheader,previousversionsofArangoDBreturnedanHTTPstatuscode200
(ok).ClientscanstillsendthisheadertoArangoDB3.0butthiswillnotinfluencetheHTTPstatuscodeproducedbyArangoDB.
The"location"headerproducedbyArangoDB3.0willnowalwayscontainthedatabasename.Thiswasalsothedefaultinprevious
versionsofArangoDB,butthebehaviorcouldbeoverriddenbysendingtheHTTPheader x-arango-version:1.4.Clientscanstillsend
theheader,butthiswillnotmakethedatabasenameinthe"location"responseheaderdisappear.
TheresultformatforqueryingallcollectionsviatheAPIGET /_api/collectionhasbeenchanged.
PreviousversionsofArangoDBreturnedanobjectwithanattributenamed collectionsandanattributenamed names.Bothcontained
allavailablecollections,but collectionscontainedthecollectionsasanarray,and namescontainedthecollectionsagain,containedin
anobjectinwhichtheattributenameswerethecollectionnames,e.g.
{
"collections":[
{"id":"5874437","name":"test","isSystem":false,"status":3,"type":2},
{"id":"17343237","name":"something","isSystem":false,"status":3,"type":2},
...
],
"names":{
"test":{"id":"5874437","name":"test","isSystem":false,"status":3,"type":2},
"something":{"id":"17343237","name":"something","isSystem":false,"status":3,"type":2},
...
}
}
Thisresultstructurewasredundant,andthereforehasbeensimplifiedtojust
{
"result":[
{"id":"5874437","name":"test","isSystem":false,"status":3,"type":2},
{"id":"17343237","name":"something","isSystem":false,"status":3,"type":2},
...
]
}
inArangoDB3.0.
436
ReplicationAPIs
TheURLparameter"failOnUnknown"wasremovedfromtheRESTAPIGET /_api/replication/dump.Thisparametercontrolled
whetherdumpingorreplicatingedgesshouldfailifoneofthevertexcollectionslinkedintheedge's _fromor _toattributeswasnot
presentanymore.Inthiscasethe _fromand _tovaluescouldnotbetranslatedintomeaningfulidsanymore.
Thereweretwowaysforhandlingthis:
setting failOnUnknownto truecausedtheHTTPrequesttofail,leavingerrorhandlingtotheuser
setting failOnUnknownto falsecausedtheHTTPrequesttocontinue,translatingthecollectionnamepartinthe _fromor _to
valueto _unknown.
InArangoDB3.0thisparameterisobsolete,as _fromand _toarestoredasself-containedstringvaluesallthetime,sotheycannot
getinvalidwhenreferencedcollectionsaredropped.
TheresultformatoftheAPIGET /_api/replication/logger-followhaschangedslightlyinthefollowingaspects:
documentsandedgesarereportedinthesameway.Thetypefordocumentinsertions/updatesandedgeinsertions/updatesisnow
always 2300.PreviousversionsofArangoDBreturneda typevalueof 2300fordocumentsand 2301foredges.
recordsaboutinsertions,updatesorremovalsofdocumentsandedgesdonothavethe keyand revattributesonthetop-level
anymore.Instead, keyand revcanbeaccessedbypeekingintothe _keyand _revattributesofthe datasub-attributesof
thechangerecord.
Thesameistrueforthecollection-specificchangesAPIGET /_api/replication/dump.
UsermanagementAPIs
TheRESTAPIendpointPOST /_api/userforaddingnewusersnowrequirestherequesttocontainaJSONobjectwithanattribute
named user,containingthenameoftheusertobecreated.PreviousversionsofArangoDBalsocheckedthisattribute,butadditionally
lookedforanattribute usernameifthe userattributedidnotexist.
UndocumentedAPIs
ThefollowingundocumentedHTTPRESTendpointshavebeenremovedfromArangoDB'sRESTAPI:
/_open/cerberusand /_system/cerberus:theseendpointswereintendedforsomeArangoDB-internalapplicationsonly
PUT /_api/simple/by-example-hash,PUT /_api/simple/by-example-skiplistandPUT /_api/simple/by-condition-skiplist:
thesemethodsweredocumentedinearlyversionsofArangoDBbuthavebeenmarkedasnotintendedtobecalledbyendusers
sinceArangoDBversion2.3.ThesemethodsshouldnothavebeenpartofanyArangoDBmanualsinceversion2.4.
/_api/structure:anolderunfinishedandunpromotedAPIfordataformatandtypechecks,supersededbyFoxxapplications.
AdministrationAPIs
/_admin/shutdownnowneedstobecalledwiththeHTTPDELETEmethod
HandlingofCORSrequests
ItcannowbecontrolledindetailforwhichoriginhostsCORS(Cross-originresourcesharing)requestswithcredentialswillbeallowed.
ArangoDB3.0providesthestartupoption --http.trusted-originthatcanbeusedtospecifyoneormanyoriginsfromwhichCORS
requestsaretreatedas"trustworthy".
Theoptioncanbespecifiedmultipletimes,oncepertrustedorigin,e.g.
--http.trusted-originhttp://127.0.0.1:8529--http.trusted-originhttps://127.0.0.1:8599
ThiswillmaketheArangoDBserverrespondtoCORSrequestsfromtheseoriginswithan Access-Control-Allow-CredentialsHTTP
headerwithavalueof true.WebbrowserscaninspectthisheaderandcanallowpassingArangoDBwebinterfacecredentials(ifstored
inthebrowser)totherequestingsite.ArangoDBwillnotforwardorprovideanycredentials.
437
SettingthisoptionisonlyrequiredifapplicationsonotherhostsneedtoaccesstheArangoDBwebinterfaceorotherHTTPRESTAPIs
fromawebbrowserwiththesamecredentialsthattheuserhasenteredwhenloggingintothewebinterface.Whenawebbrowserfinds
the Access-Control-Allow-CredentialsHTTPresponseheader,itmayforwardthecredentialsenteredintothebrowserforthe
ArangoDBwebinterfacelogintotheothersite.
Thisisapotentialsecurityissue,sotherearenotrustedoriginsbydefault.Itmayberequiredtosetsometrustedoriginsifyou're
planningtoissueAJAXrequeststoArangoDBfromothersitesfromthebrowser,withthecredentialsenteredduringtheArangoDB
interfacelogin(i.e.singlesign-on).Ifsuchfunctionalityisnotused,theoptionshouldnotbeset.
Tospecifyatrustedorigin,specifytheoptiononcepertrustedoriginasshownabove.Notethatthetrustedoriginvaluesspecifiedin
thisoptionwillbecomparedbytewisewiththe OriginHTTPheadervaluesentbyclients,andonlyexactmatcheswillpass.
Thereisalsothewildcard allforenablingCORSaccessfromalloriginsinatestordevelopmentsetup:
--http.trusted-originall
SettingthisoptionwillleadtotheArangoDBserverrespondingwithan Access-Control-Allow-Credentials:trueHTTPheadertoall
incomingCORSrequests.
Command-lineoptions
QuiteafewstartupoptionsinArangoDB2weredoublenegations(like --server.disable-authenticationfalse).InArangoDB3these
arenowexpressedaspositives(e.g. --server.authentication).AlsotheoptionsbetweentheArangoDBserveranditsclienttoolshave
beingunified.Forexample,theloggeroptionsarenowthesamefortheserverandtheclienttools.Additionallymanyoptionshavebeen
movedintomoreappropriatetopicsections.
Renamedoptions
Thefollowingoptionshavebeenavailablebefore3.0andhavechangedtheirnamein3.0:
--server.disable-authenticationwasrenamedto --server.authentication.Notethatthemeaningoftheoption -server.authenticationistheoppositeoftheprevious --server.disable-authentication.
--server.disable-authentication-unix-socketswasrenamedto --server.authentication-unix-sockets.Notethatthemeaningof
theoption --server.authentication-unix-socketsistheoppositeoftheprevious --server.disable-authentication-unix-sockets.
--server.authenticate-system-onlywasrenamedto --server.authentication-system-only.Themeaningoftheoptionin
unchanged.
--server.disable-statisticswasrenamedto --server.statistics.Notethatthemeaningoftheoption --server.statisticsis
theoppositeoftheprevious --server.disable-statistics.
--server.cafilewasrenamedto --ssl.cafile.Themeaningoftheoptionisunchanged.
--server.keyfilewasrenamedto --ssl.keyfile.Themeaningoftheoptionisunchanged.
--server.ssl-cachewasrenamedto --ssl.session-cache.Themeaningoftheoptionisunchanged.
--server.ssl-cipher-listwasrenamedto --ssl.cipher-list.Themeaningoftheoptionisunchanged.
--server.ssl-optionswasrenamedto --ssl.options.Themeaningoftheoptionisunchanged.
--server.ssl-protocolwasrenamedto --ssl.protocol.Themeaningoftheoptionisunchanged.
--server.backlog-sizewasrenamedto --tcp.backlog-size.Themeaningoftheoptionisunchanged.
--server.reuse-addresswasrenamedto --tcp.reuse-address.Themeaningoftheoptionisunchanged.
--server.disable-replication-applierwasrenamedto --database.replication-applier.Themeaningoftheoption -database.replication-applieristheoppositeoftheprevious --server.disable-replication-applier.
--server.allow-method-overridewasrenamedto --http.allow-method-override.Themeaningoftheoptionisunchanged.
--server.hide-product-headerwasrenamedto --http.hide-product-header.Themeaningoftheoptionisunchanged.
--server.keep-alive-timeoutwasrenamedto --http.keep-alive-timeout.Themeaningoftheoptionisunchanged.
--server.foxx-queueswasrenamedto --foxx.queues.Themeaningoftheoptionisunchanged.
--server.foxx-queues-poll-intervalwasrenamedto --foxx.queues-poll-interval.Themeaningoftheoptionisunchanged.
--no-serverwasrenamedto --server.rest-server.Notethatthemeaningoftheoption --server.rest-serveristheopposite
oftheprevious --no-server.
--database.query-cache-modewasrenamedto --query.cache-mode.Themeaningoftheoptionisunchanged.
--database.query-cache-max-resultswasrenamedto --query.cache-entries.Themeaningoftheoptionisunchanged.
438
--database.disable-query-trackingwasrenamedto --query.tracking.Themeaningoftheoption --query.trackingisthe
oppositeoftheprevious --database.disable-query-tracking.
--log.ttywasrenamedto --log.foreground-tty.Themeaningoftheoptionisunchanged.
--upgradehasbeenrenamedto --database.auto-upgrade.Incontrastto2.8thisoptionnowrequiresabooleanparameter.To
actuallyperformanautomaticdatabaseupgradeatstartupuse --database.auto-upgradetrue.Tonotperformit,use -database.auto-upgradefalse.
--check-versionhasbeenrenamedto --database.check-version.
--temp-pathhasbeenrenamedto --temp.path.
Logverbosity,topicsandoutputfiles
Loggingnowsupportslogtopics.Youcancontrolthesebyspecifyingalogtopicinfrontofalogleveloranoutput.Forexample
--log.levelstartup=trace--log.levelinfo
willlogmessagesconcerningstartupattracelevel,everythingelseatinfolevel. --log.levelcanbespecifiedmultipletimesatstartup,
forasmanytopicsasneeded.
mmap:informationaboutmemory-mappingoperations
performance:someperformance-relatedinformation
queries:executedAQLqueries
Thenewlogoption --log.output<definition>allowsdirectingtheglobalorper-topiclogoutputtodifferentoutputs.Theoutput
definition""canbeoneof
"-"forstdin
"+"forstderr
"syslog://"
"syslog:///"
"file://"
Theoptioncanbespecifiedmultipletimesinordertoconfiguretheoutputfordifferentlogtopics.Tosetupaper-topicoutput
configuration,use --log.output<topic>=<definition>,e.g.
queries=file://queries.txt
logsallqueriestothefile"queries.txt".
Theoldoption --log.fileisstillavailablein3.0forconveniencereasons.In3.0itisashortcutforthemoregeneraloption -log.outputfile://filename.
Theoldoption --log.requests-fileisstillavailablein3.0.Itisnowashortcutforthemoregeneraloption --log.output
requests=file://....
Theoldoption --log.performanceisstillavailablein3.0.Itisnowashortcutforthemoregeneraloption --log.level
performance=trace.
Removedoptionsforlogging
Theoptions --log.content-filterand --log.source-filterhavebeenremoved.TheyhavemostbeenusedduringArangoDB's
internaldevelopment.
439
Thesyslog-relatedoptions --log.applicationand --log.facilityhavebeenremoved.Theyaresupersededbythemoregeneral -log.outputoptionwhichcanalsohandlesyslogtargets.
Removedotheroptions
Theoption --server.default-api-compatibilitywaspresentinearlierversionofArangoDBtocontrolvariousaspectsoftheserver
behavior,e.g.HTTPreturncodesortheformatofHTTP"location"headers.ClientapplicationscouldsendanHTTPheader"x-arangoversion"withaversionnumbertorequesttheserverbehaviorofacertainArangoDBversion.
Thisoptionwasonlyhonoredinahandfulofcases(describedabove)andwasremovedin3.0becausethechangesinserverbehavior
controlledbythisoptionwerechangedevenbeforeArangoDB2.0.Thisshouldhaveleftenoughtimeforclientapplicationstoadaptto
thenewbehavior,makingtheoptionsuperfluousin3.0.
Threadoptions
Theoptions --server.threadsand --scheduler.threadsnowhaveadefaultvalueof 0.When --server.threadsissetto 0on
startup,thesuitablenumberofthreadswillbedeterminedbyArangoDBbyaskingtheOSforthenumberofavailableCPUsandusing
thatasabaseline.IfthenumberofCPUsislowerthan4,ArangoDBwillstillstart4dispatcherthreads.When --scheduler.threadsis
setto 0,thenArangoDBwillautomaticallydeterminethenumberofschedulerthreadstostart.Thiswillnormallycreate2scheduler
threads.
Iftheexactnumberofthreadsneedstobesetbytheadmin,thenitisstillpossibletoset --server.threadsand --scheduler.threads
tonon-zerovalues.ArangoDBwillusethesevaluesandstartthatmanythreads(notethatsomethreadsmaybecreatedlazilysothey
maynotbepresentdirectlyafterstartup).
ThenumberofV8JavaScriptcontextstobecreated( --javascript.v8-contexts)nowhasadefaultvalueof 0too,meaningthat
ArangoDBwillcreateasmanyV8contextsastherewillbedispatcherthreads(controlledbythe --server.threadsoption).Settingthis
optiontoanon-zerovaluewillcreateexactlyasmanyV8contextsasspecified.
Settingtheseoptionsexplicitlytonon-zerovaluesmaybebeneficialinenvironmentsthathavefewresources(processingtime,maximum
threadcount,availablememory).
Authentication
Thedefaultvaluefor --server.authenticationisnow trueintheconfigurationfilesshippedwithArangoDB.Thismeanstheserver
willbestartedwithauthenticationenabledbydefault,requiringallclientconnectionstoprovideauthenticationdatawhenconnectingto
ArangoDBAPIs.PreviousArangoDBversionsusedthesetting --server.disable-authenticationtrue,effectivelydisabling
authenticationbydefault.
Thedefaultvaluefor --server.authentication-system-onlyisnow trueinArangoDB.ThatmeansthatFoxxapplicationsrunningin
ArangoDBwillbepublicaccessible(atleasttheywillnotuseArangoDB'sbuiltinauthenticationmechanism).Onlyrequeststo
ArangoDBAPIsatURLpathprefixes /_api/and /_adminwillrequireauthentication.Tochangethat,andusethebuiltin
authenticationmechanismforFoxxapplicationstoo,set --server.authentication-system-onlyto false,andmakesuretohavethe
option --server.authenticationsetto trueaswell.
Thoughenablingtheauthenticationisrecommendedforproductionsetups,itmaybeoverkillinadevelopmentenvironment.Toturnoff
authentication,theoption --server.authenticationcanbesetto falseinArangoDB'sconfigurationfileoronthecommand-line.
WebAdminInterface
TheJavaScriptshellhasbeenremovedfromArangoDB'swebinterface.Thefunctionalitytheshellprovidedisstillfullyavailableinthe
ArangoShell(arangosh)binaryshippedwithArangoDB.
ArangoShellandclienttools
440
TheArangoShell(arangosh)andtheotherclienttoolsbundledwithArangoDBcanonlyconnecttoanArangoDBserverofversion3.0or
higher.TheywillnotconnecttoanArangoDB2.8.ThisisbecausetheserverHTTPAPIshavechangedbetween2.8and3.0,andall
clienttoolsusestheseAPIs.
InordertoconnecttoearlierversionsofArangoDBwiththeclienttools,anolderversionoftheclienttoolsneedstobekeptinstalled.
Thepreferrednameforthetemplatestringgeneratorfunction aqlQueryisnow aqlandisautomaticallyavailableinarangosh.
Elsewhere,itcanbeloadedlike constaql=require('@arangodb').aql.
Command-lineoptionsadded
Allclienttoolsin3.0provideanoption --server.max-packet-sizeforcontrollingthemaximumsizeofHTTPpacketstobehandledby
theclienttools.Thedefaultvalueis128M B,asinpreviousversionsofArangoDB.Incontrasttopreviousversionsinwhichthevalue
washard-coded,theoptionisnowconfigurable.ItcanbeincreasedtomaketheclienttoolshandleverylargeHTTPresultmessagessent
bytheserver.
Command-lineoptionschanged
Forallclienttools,theoption --server.disable-authenticationwasrenamedto --server.authentication.Notethatthemeaningof
theoption --server.authenticationistheoppositeoftheprevious --server.disable-authentication.
Theoption --server.ssl-protocolwasrenamedto --ssl.protocol.Themeaningoftheoptionisunchanged.
Thecommand-lineoption --quietwasremovedfromallclienttoolsexceptarangoshbecauseithadnoeffectinthem.
Arangobench
Inordertomakeitspurposemoreapparenttheformer arangobclienttoolhasbeenrenamedto arangobenchin3.0.
Thechecksumcalculationalgorithmforthe collection.checksum()methodanditscorrespondingRESTAPIGET
/_api/collection/<collection</checksumhaschangedin3.0.Checksumscalculatedin3.0willdifferfromchecksumscalculatedwith2.8
orbefore.
TheArangoDBserverin3.0doesnotreadafile ENDPOINTScontainingalistofadditionalendpointsonstartup.In2.8thisfilewas
automaticallyreadifpresentinthedatabasedirectory.
Thenamesofthesub-threadsstartedbyArangoDBhavechangedin3.0.ThisisrelevantonLinuxonly,wherethreadscanbenamedand
threadnamesmaybevisibletosystemtoolssuchastopormonitoringsolutions.
441
ifnecessary.
AQL
Keywordsadded
ThefollowingAQLkeywordswereaddedinArangoDB2.8:
GRAPH
OUTBOUND
INBOUND
ANY
ALL
NONE
AGGREGATE
Usageofthesekeywordsforcollectionnames,variablenamesorattributenamesinAQLquerieswillnotbepossiblewithoutquoting.
Forexample,thefollowingAQLquerywillstillworkasitusesaquotedcollectionnameandaquotedattributename:
FORdocINÒUTBOUND`
RETURNdoc.àny`
Changedbehavior
TheAQLfunctions NEARand WITHINnowhavestrictervalidationsfortheirinputparameters limit, radiusand distance.They
maynowthrowexceptionswheninvalidparametersarepassedthatmayhavenotledtoexceptionsinpreviousversions.
Additionally,theexpansion( [*])operatorinAQLhaschangeditsbehaviorwhenhandlingnon-arrayvalues:
InArangoDB2.8,callingtheexpansionoperatoronanon-arrayvaluewillalwaysreturnanemptyarray.PreviousversionsofArangoDB
expandednon-arrayvaluesbycallingthe TO_ARRAY()functionforthevalue,whichforexamplereturnedanarraywithasinglevaluefor
boolean,numericandstringinputvalues,andanarraywiththeobject'svaluesforanobjectinputvalue.Thisbehaviorwasinconsistent
withhowtheexpansionoperatorworksforthearrayindexesin2.8,sothebehaviorisnowunified:
iftheleft-handsideoperandof [*]isanarray,thearraywillbereturnedasiswhencalling [*]onit
iftheleft-handsideoperandof [*]isnotanarray,anemptyarraywillbereturnedby [*]
AQLqueriesthatrelyontheoldbehaviorcanbechangedbyeithercalling TO_ARRAYexplicitlyorbyusingthe [*]atthecorrect
position.
Thefollowingexamplequerywillchangeitsresultin2.8comparedto2.7:
LETvalues="foo"RETURNvalues[*]
In2.7thequeryhasreturnedthearray ["foo"],butin2.8itwillreturnanemptyarray [].Tomakeitreturnthearray ["foo"
]again,anexplicit TO_ARRAYfunctioncallisneededin2.8(whichinthiscaseallowstheremovalofthe [*]operatoraltogether).This
alsoworksin2.7:
LETvalues="foo"RETURNTO_ARRAY(values)
Anotherexample:
LETvalues=[{name:"foo"},{name:"bar"}]
RETURNvalues[*].name[*]
442
Theabovereturned [["foo"],["bar"]]in2.7.In2.8itwillreturn[[],[]] ,becausethevalueofnameìsnotanarray.
Tochangetheresultstothe2.7style,thequerycanbechangedto
LETvalues=[{name:"foo"},{name:"bar"}]
RETURNvalues[*RETURNTO_ARRAY(CURRENT.name)]
Theabovealsoworksin2.7.Thefollowingtypesofquerieswon'tchange:
LETvalues=[1,2,3]RETURNvalues[*]
LETvalues=[{name:"foo"},{name:"bar"}]RETURNvalues[*].name
LETvalues=[{names:["foo","bar"]},{names:["baz"]}]RETURNvalues[*].names[*]
LETvalues=[{names:["foo","bar"]},{names:["baz"]}]RETURNvalues[*].names[**]
Deadlockhandling
Clientapplicationsshouldbepreparedtohandleerror29( deadlockdetected)thatArangoDBmaynowthrowwhenitdetectsa
deadlockacrossmultipletransactions.Whenaclientapplicationreceiveserror29,itshouldretrytheoperationthatfailed.
TheerrorcanonlyoccurforAQLqueriesorusertransactionsthatinvolvemorethanasinglecollection.
Optimizer
TheAQLexecutionnodetype IndexRangeNodewasreplacedwithanewmorecapableexecutionnodetype IndexNode.Thatmeansin
executionplanexplainoutputtherewillbenomore IndexRangeNodesbutonly IndexNode.Thisaffectsexplainoutputthatcanbe
retrievedvia require("org/arangodb/aql/explainer").explain(query), db._explain(query),andtheHTTPqueryexplainAPI.
TheoptimizerrulethatmakesAQLqueriesactuallyuseindexeswasalsorenamedfrom use-index-rangeto use-indexes.Againthis
affectsexplainoutputthatcanberetrievedvia require("org/arangodb/aql/explainer").explain(query), db._explain(query),andthe
HTTPqueryexplainAPI.
Thequeryoptimizerrule remove-collect-intowasrenamedto remove-collect-variables.Thisaffectsexplainoutputthatcanbe
retrievedvia require("org/arangodb/aql/explainer").explain(query), db._explain(query),andtheHTTPqueryexplainAPI.
HTTPAPI
Whenaserver-sideoperationgotcanceledduetoanexplicitclientcancelrequestviaHTTP DELETE/_api/job,previousversionsof
ArangoDBreturnedanHTTPstatuscodeof408(requesttimeout)fortheresponseofthecanceledoperation.
TheHTTPreturncode408hascausedproblemswithsomeclientapplications.Somebrowsers(e.g.Chrome)handleda408responseby
resendingtheoriginalrequest,whichistheoppositeofwhatisdesiredwhenajobshouldbecanceled.
ThereforeArangoDBwillreturnHTTPstatuscode410(gone)forcanceledoperationsfromversion2.8on.
Foxx
ModelandRepository
DuetocompatibilityissuestheM odelandRepositorytypesarenolongerimplementedasES2015classes.
Thepre-2.7"extend"stylesubclassingissupportedagainandwillnotemitanydeprecationwarnings.
varMyModel=Foxx.Model.extend({
//...
schema:{/*...*/}
});
Moduleresolution
443
ThebehavioroftheJavaScriptmoduleresolutionusedbythe requirefunctionhasbeenmodifiedtoimprovecompatibilitywith
moduleswrittenforNode.js.
Specifically
absolutepaths(e.g. /some/absolute/path)arenowalwaysinterpretedasabsolutefilesystempaths,relativetothefilesystemroot
globalnames(e.g. global/name)arenowfirstintepretedasreferencestomodulesresidinginarelevant node_modulesfolder,a
built-inmoduleoramatchingdocumentintheinternal _modulescollection,andonlyresolvedtolocalfilepathsifnoothermatchis
found
Previouslythetwoformatsweretreatedinterchangeablyandwouldberesolvedtolocalfilepathsfirst,leadingtoproblemswhenlocal
filesusedthesamenamesasothermodules(e.g.alocalfile chai.jswouldcauseproblemswhentryingtoloadthe chaimodule
installedin node_modules).
Formoreinformationseetheblogannouncementofthischangeandtheupgradeguide.
Module org/arangodb/request
Themodulenowalwaysreturnsresponsebodies,evenforerrorresponses.Inversionspriorto2.8themodulewouldsilentlydrop
responsebodiesiftheresponseheaderindicatedanerror.
Theoldbehaviorofnotreturningbodiesforerrorresponsescanberestoredbyexplicitlysettingtheoption returnBodyOnErrorto
false:
letresponse=request({
//...
returnBodyOnError:false
});
Garbagecollection
TheV8garbagecollectionstrategywasslightlyadjustedsothatiteventuallyhappensinallV8contextsthatholdV8externalobjects
(referencestoArangoDBdocumentsandcollections).Thisenablesabettercleanupoftheseresourcesandpreventsotherprocessessuch
ascompactionbeingstalledwhilewaitingfortheseresourcestobereleased.
InthiscontextthedefaultvaluefortheJavaScriptgarbagecollectionfrequency( --javascript.gc-frequency)wasalsoincreasedfrom10
secondsto15seconds,aslessinternaloperationsinArangoDBarecarriedoutinJavaScript.
Clienttools
arangodumpwillnowfailbydefaultwhentryingtodumpedgesthatrefertoalreadydroppedcollections.Thiscanbecircumventedby
specifyingtheoption --forcetruewheninvokingarangodump
444
ifnecessary.
AQLchanges
DISTINCTisnowakeywordinAQL.
AQLqueriesthatuse DISTINCT(inlower,upperormixedcase)asanidentifier(i.e.asavariable,acollectionnameorafunctionname)
willstopworking.Tomakesuchqueriesworkingagain,eachoccurrenceof DISTINCTinanAQLqueryshouldbeenclosedinbackticks.
Thiswillturn DISTINCTfromakeywordintoanidentifieragain.
TheAQLfunction SKIPLIST()hasbeenremovedinArangoDB2.7.ThisfunctionwasdeprecatedinArangoDB2.6.Itwasaleft-over
fromtimeswhenthequeryoptimizerwasn'tabletouseskiplistindexestogetherwithfilters,skipandlimitvalues.Sincethisissuebeen
fixedsinceversion2.3,thereisnoAQLreplacementfunctionfor SKIPLIST.Queriesthatusethe SKIPLISTfunctioncanbefixedby
usingtheusualcombinationof FOR, FILTERand LIMIT,e.g.
FILTERdoc.value>=@value
SORTdoc.valueDESC
LIMIT1
RETURNdoc
Foxxchanges
Bundlingandcompilation
The assetspropertyisnolongersupportedinFoxxmanifestsandisscheduledtoberemovedinafutureversionofArangoDB.The
filespropertycanstillbeusedtoservestaticassetsbutitisrecommendedtouseseparatetoolingtocompileandbundleyourassets.
Manifestscripts
Theproperties setupand teardownhavebeenmovedintothe scriptspropertymap:
Before:
{
...
"teardown":"scripts/teardown.js"
}
After:
{
...
"scripts":{
"teardown":"scripts/teardown.js"
}
}
FoxxQueues
Function-basedFoxxQueuejobtypesarenolongersupported.Tolearnabouthowyoucanusethenewscript-basedjobtypesfollow
theupdatedrecipeinthecookbook.
445
FoxxSessions
The jwtand typeoptionshavebeenremovedfromthe activateSessionsAPI.
Ifyouwanttoreplicatethebehaviorofthe jwtoptionyoucanusetheJWTfunctionsinthe cryptomodule.AJWT-basedsession
storagethatdoesn'twritesessionstothedatabaseisavailableasthesessions-jwtappintheFoxxappstore.
Thesessiontypeisnowinferredfromthepresenceofthe cookieor headeroptions(allowingyoutoenablesupportforboth).Ifyou
wanttousethedefaultsettingsfor cookieor headeryoucanpassthevalue trueinstead.
The sessionStorageAppoptionhasbeenremovedinfavourofthe sessionStorageoption.
Before:
varctrl=newFoxx.Controller(applicationContext);
sessionStorageApp:'some-sessions-app',
type:'cookie'
});
After:
sessionStorage:applicationContext.dependencies.sessions.sessionStorage,
cookie:true
});
Requestmodule
Themodule org/arangodb/requestusesaninternallibraryfunctionforsendingHTTPrequests.Thislibraryfunctionally
unconditionallysetanHTTPheader Accept-Encoding:gzipinalloutgoingHTTPrequests,withoutclientcodehavingtosetthisheader
explicitly.
Thishasbeenfixedin2.7,so Accept-Encoding:gzipisnotsetautomaticallyanymore.Additionallytheheader User-Agent:ArangoDB
isnotsetautomaticallyeither.Ifclientapplicationsrelyontheseheadersbeingsent,theyarefreetoadditwhenconstructingrequests
usingtherequestmodule.
The internal.download()functionisalsoaffectedbythischange.Again,theheadercanbeaddedhereifrequiredbypassingitviaa
headerssub-attributeinthethirdparameter( options)tothisfunction.
arangodump/backups
Thefilenamesindumpscreatedbyarangodumpnowcontainnotonlythenameofthedumpedcollection,butalsoanadditional32-digit
hashvalue.Thisisdonetopreventoverwritingdumpfilesincase-insensitivefilesystemswhenthereexistmultiplecollectionswiththe
samename(butwithdifferentcases).
Thischangeleadstochangedfilenamesindumpscreatedbyarangodump.Ifanyclientscriptsdependonthefilenamesinthedump
outputdirectorybeingequaltothecollectionnameplusoneofthesuffixes .structure.jsonand .data.json,theyneedtobeadjusted.
StartingwithArangoDB2.7,thefilenameswillcontainanunderscoreplusthe32-digitM D5value(representedinhexadecimalnotation)
ofthecollectionname.
Forexample,whenarangodumpdumpsdataoftwocollectionstestandTest,thefilenamesinpreviousversionsofArangoDBwere:
test.structure.json(definitionsforcollectiontest)
test.data.json(dataforcollectiontest)
Test.structure.json(definitionsforcollectionTest)
Test.data.json(dataforcollectionTest)
In2.7,thefilenameswillbe:
446
test_098f6bcd4621d373cade4e832627b4f6.structure.json(definitionsforcollectiontest)
test_098f6bcd4621d373cade4e832627b4f6.data.json(dataforcollectiontest)
Test_0cbc6611f5540bd0809a388dc95a615b.structure.json(definitionsforcollectionTest)
Test_0cbc6611f5540bd0809a388dc95a615b.data.json(dataforcollectionTest)
Starting/stopping
Whenstartingarangod,theserverwillnowdroptheprocessprivilegestothespecifiedvaluesinoptions --server.uidand -server.gidinstantlyafterparsingthestartupoptions.
Thatmeanswheneither --server.uidor --server.gidareset,theprivilegechangewillhappenearlier.Thismaypreventbindingthe
servertoanendpointwithaportnumberlowerthan1024ifthearangodbuserhasnoprivilegesforthat.PreviousversionsofArangoDB
changedtheprivilegeslater,sosomestartupactionswerestillcarriedoutundertheinvokinguser(i.e.likelyrootwhenstartedviainit.d
orsystemscripts)andespeciallybindingtolowportnumberswasstillpossiblethere.
Thedefaultprivilegesforuserarangodbwillnotbesufficientforbindingtoportnumberslowerthan1024.TohaveanArangoDB2.7
bindtoaportnumberlowerthan1024,itneedstobestartedwitheitheradifferentprivilegeduser,ortheprivilegesofthearangodbuser
havetoraisedmanuallybeforehand.
Additionally,LinuxstartupscriptsandsystemdconfigurationforarangodnowwilladjusttheNOFILE(numberofopenfiles)limitsfor
theprocess.Thelimitvalueissetto131072(128k)whenArangoDBisstartedviastart/stopcommands.Thegoalofthischangeisto
preventarangodfromrunningoutofavailablefiledescriptorsforsocketconnectionsanddatafiles.
Connectionhandling
arangodwillnowactuallycloselingeringclientconnectionswhenidleforatleastthedurationspecifiedinthe --server.keep-alivetimeoutstartupoption.
InpreviousversionsofArangoDB,idleconnectionswerenotclosedbytheserverwhenthetimeoutwasreachedandtheclientwasstill
connected.Nowtheconnectionisproperlyclosedbytheserverincaseoftimeout.Clientapplicationsrelyingontheoldbehaviormay
nowneedtoreconnecttotheserverwhentheiridleconnectionstimeoutandgetclosed(note:connectionsbeingidleforalongtimemay
beclosedbytheOSorfirewallsanyway-clientapplicationsshouldbeawareofthatandtrytoreconnect).
Optionchanges
Configureoptionsremoved
Thefollowingoptionsfor configurehavebeenremovedbecausetheywereunusedorexotic:
--enable-timings
--enable-figures
Startupoptionsadded
Thefollowingconfigurationoptionshavebeenaddedin2.7:
--database.query-cache-max-results:setsthemaximumnumberofresultsinAQLqueryresultcacheperdatabase
--database.query-cache-mode:setsthemodefortheAQLqueryresultscache.Possiblevaluesare on, offand demand.The
defaultvalueis off
Simplequeries
447
M anysimplequeriesprovidea skip()functionthatcanbeusedtoskipoveracertainnumberofdocumentsintheresult.Thisfunction
allowedspecifyingnegativeoffsetsinpreviousversionsofArangoDB.Specifyinganegativeoffsetledtothequeryresultbeingiterated
inreverseorder,soskippingwasperformedfromthebackoftheresult.Asmostsimplequeriesdonotprovideaguaranteedresultorder,
skippingfromthebackofaresultwithunspecificorderseemsaratherexoticusecaseandwasremovedtoincreaseconsistencywith
AQL,whichalsodoesnotprovidenegativeskipvalues.
NegativeskipvaluesweredeprecatedinArangoDB2.6.
TasksAPI
Theundocumentedfunction addJob()hasbeenremovedfromthe org/arangodb/tasksmoduleinArangoDB2.7.
RuntimeendpointsmanipulationAPI
ThefollowingHTTPRESTAPImethodsforruntimemanipulationofserverendpointshavebeenremovedinArangoDB2.7:
POST /_api/endpoint:todynamicallyaddanendpointwhiletheserverwasrunning
DELETE /_api/endpoint:todynamicallyremoveanendpointwhiletheserverwasrunning
ThischangealsoaffectstheequivalentJavaScriptendpointmanipulationmethodsavailableinFoxx.Thefollowingfunctionshavebeen
removedinArangoDB2.7:
db._configureEndpoint()
db._removeEndpoint()
448
ifnecessary.
Requirements
ArangoDB'sbuilt-inwebinterfacenowusescookiesforsessionmanagement.Sessioninformationidsarestoredincookies,soclients
usingthewebinterfacemustacceptcookiesinordertologinanduseit.
Foxxchanges
FoxxQueues
FoxxQueuejobtypedefinitionswerepreviouslybasedonfunctionsandhadtoberegisteredbeforeuse.Duetochangesin2.5this
resultedinproblemswhenrestartingtheserverordefiningjobtypesincorrectly.
Function-basedjobtypeshavebeendeprecatedin2.6andwillberemovedentirelyin2.7.
Inordertoconvertexistingfunction-basedjobtypestothenewscript-basedjobtypes,createcustomscriptsinyourFoxxappand
referencethembytheirnameandthemountpointoftheapptheyaredefinedin.OfficialjobtypesfromtheFoxxappstorecanbe
upgradedbyupgradingfromthe1.xversiontothe2.xversionofthesameapp.
Inordertoupgradequeuedjobstothenewjobtypes,youneedtoupdatethe typepropertyoftheaffectedjobsinthedatabase's
_jobssystemcollection.Inordertoseethecollectioninthewebinterfaceyouneedtoenablethecollectiontype"System"inthe
collectionlistoptions.
Example:
Before: "type":"mailer.postmark"
After: "type":{"name":"mailer","mount":"/my-postmark-mailer"}
FoxxSessions
Theoptions jwtand typeofthecontrollermethod controller.activateSessionshavebeendeprecatedin2.6andwillberemoved
entirelyin2.7.
IfyouwanttousepureJWTsessions,youcanusethe sessions-jwtFoxxappfromtheFoxxappstore.
IfyouwanttouseyourownJWT-basedsessions,youcanusetheJWTfunctionsinthe cryptomoduledirectly.
Insteadofusingthe typeoptionyoucanjustusethe cookieand headeroptionsontheirown,whichbothnowacceptthevalue
truetoenablethemwiththeirdefaultconfigurations.
Theoption sessionStorageApphasbeenrenamedto sessionStorageandnowalsoacceptssessionstoragesdirectly.Theoldoption
sessionStorageAppwillberemovedentirelyin2.7.
Libraries
Thebundledversionofthe joilibraryusedinFoxxwasupgradedtoversion6.0.8.ThismayaffectFoxxapplicationsthatdependon
thelibrary.
AQLchanges
AQLLENGTHfunction
449
ThereturnvalueoftheAQL LENGTHfunctionwaschangedif LENGTHisappliedon nullorabooleanvalue:
LENGTH(null)nowreturns 0.InpreviousversionsofArangoDB,thisreturned 4.
LENGTH(false)nowreturns 0.InpreviousversionsofArangoDB,thereturnvaluewas 5.
LENGTH(true)nowreturns 1.InpreviousversionsofArangoDB,thereturnvaluewas 4.
AQLgraphfunctions
In2.6thegraphfunctionsdidundergoaperformancelifting.Duringthisprocesswehadtoadopttheresultformatandtheoptionsfor
someofthem.M anygraphfunctionsnowhaveanoption includeDatawhichallowstotriggeriftheresultofthisfunctionshould
containfullyextracteddocuments includeData:trueoronlythe _idvalues includeData:false.Inmostusecasesthe _idis
sufficienttocontinueandtheextractionofdataisanunnecessaryoperation.TheAQLfunctionssupportingthisadditionaloptionare:
SHORTEST_PATH
NEIGHBORS
GRAPH_SHORTEST_PATH
GRAPH_NEIGHBORS
GRAPH_EDGES
Furthermoretheresult SHORTEST_PATHhaschanged.Theoldformatreturnedalistofallverticesonthepath.Optionallyitcouldinclude
eachsub-pathforthesevertices.Allofthedocumentswerefullyextracted.Example:
450
[
{
vertex:{
_id:"vertex/1",
_key:"1",
_rev:"1234"
name:"Alice"
},
path:{
vertices:[
{
_id:"vertex/1",
_key:"1",
_rev:"1234"
name:"Alice"
}
],
edges:[]
}
},
{
vertex:{
_id:"vertex/2",
_key:"2",
_rev:"5678"
name:"Bob"
},
path:{
vertices:[
{
_id:"vertex/1",
_key:"1",
_rev:"1234"
name:"Alice"
},{
_id:"vertex/2",
_key:"2",
_rev:"5678"
name:"Bob"
}
],
edges:[
{
_id:"edge/1",
_key:"1",
_rev:"9876",
type:"loves"
}
]
}
}
]
Thenewversionismorecompact.Each SHORTEST_PATHwillonlyreturnonedocumenthavingtheattributes vertices, edges,
distance.The distanceiscomputedtakingintoaccountthegivenweight.Optionallythedocumentscanbeextractedwith
includeData:trueExample:
{
vertices:[
"vertex/1",
"vertex/2"
],
edges:[
"edge/1"
],
distance:1
}
Thenextfunctionthatreturnsadifferentformatis NEIGHBORS.Since2.5itreturnedanobjectwith edgeand vertexforeach
connectededge.Example:
451
[
{
vertex:{
_id:"vertex/2",
_key:"2",
_rev:"5678"
name:"Bob"
},
edge:{
_id:"edge/1",
_key:"1",
_rev:"9876",
type:"loves"
}
}
]
With2.6itwillonlyreturnthevertexdirectly,againusing includeData:true.Bydefaultitwillreturnadistinctsetofneighbors,using
theoption distinct:falsewillincludethesamevertexforeachedgepointingtoit.
Example:
[
"vertex/2"
]
FunctionandAPIchanges
Graphmeasurementsfunctions
AllgraphmeasurementsfunctionsinJavaScriptmodule general-graphthatcalculatedasinglefigurepreviouslyreturnedanarray
containingjustthefigure.Nowthesefunctionswillreturnthefiguredirectlyandnotputitinsideanarray.
Theaffectedfunctionsare:
graph._absoluteEccentricity
graph._eccentricity
graph._absoluteCloseness
graph._closeness
graph._absoluteBetweenness
graph._betweenness
graph._radius
graph._diameter
Clientprogramscallingthesefunctionsshouldbeadjustedsotheyprocessthescalarvaluereturnedbythefunctioninsteadofthe
previousarrayvalue.
CursorAPI
AbatchSizevalue 0isnowdisallowedwhencallingthecursorAPIviaHTTP POST/_api/cursor.
TheHTTPRESTAPI POST/_api/cursordoesnotaccepta batchSizeparametervalueof 0anylonger.Abatchsizeof0never
mademuchsense,butpreviousversionsofArangoDBdidnotcheckforthisvalue.Nowcreatingacursorusinga batchSizevalue0will
resultinanHTTP400errorresponse.
DocumentURLsreturned
TheRESTAPImethodGET /_api/document?collection=...(thatmethodwillreturnpartialURLstoalldocumentsinthecollection)
willnowproperlyprefixdocumentaddressURLswiththecurrentdatabasename.
452
PreviousversionsofArangoDBreturnedtheURLsstartingwith /_api/butwithoutthecurrentdatabasename,e.g.
/_api/document/mycollection/mykey.Startingwith2.6,theresponseURLswillincludethedatabasenameaswell,e.g.
/_db/_system/_api/document/mycollection/mykey.
Fulltextindexing
Fulltextindexeswillnowalsoindextextvaluescontainedindirectsub-objectsoftheindexedattribute.
PreviousversionsofArangoDBonlyindexedtheattributevalueifitwasastring.Sub-attributesoftheindexattributewereignoredwhen
fulltextindexing.
Now,iftheindexattributevalueisanobject,theobject'svalueswilleachbeincludedinthefulltextindexiftheyarestrings.Iftheindex
attributevalueisanarray,thearray'svalueswilleachbeincludedinthefulltextindexiftheyarestrings.
Deprecatedserverfunctionality
Simplequeries
Thefollowingsimplequeryfunctionsarenowdeprecated:
collection.near
collection.within
collection.geo
collection.fulltext
collection.range
collection.closedRange
ThisalsoleadtothefollowingRESTAPImethodsbeingdeprecatedfromnowon:
PUT/_api/simple/near
PUT/_api/simple/within
PUT/_api/simple/fulltext
PUT/_api/simple/range
ItisrecommendedtoreplacecallstothesefunctionsorAPIswithequivalentAQLqueries,whicharemoreflexiblebecausetheycanbe
combinedwithotheroperations:
FORdocINNEAR(@@collection,@latitude,@longitude,@limit)
RETURNdoc
FORdocINWITHIN(@@collection,@latitude,@longitude,@radius,@distanceAttributeName)
RETURNdoc
FORdocINFULLTEXT(@@collection,@attributeName,@queryString,@limit)
RETURNdoc
FILTERdoc.value>=@left&&doc.value<@right
LIMIT@skip,@limit
RETURNdoc`
TheabovesimplequeryfunctionsandRESTAPImethodsmayberemovedinfutureversionsofArangoDB.
Usingnegativevaluesfor SimpleQuery.skip()isalsodeprecated.ThisfunctionalitywillberemovedinfutureversionsofArangoDB.
AQLfunctions
TheAQL SKIPLISTfunctionhasbeendeprecatedbecauseitisobsolete.
ThefunctionwasintroducedinolderversionsofArangoDBwithalesspowerfulqueryoptimizertoretrievedatafromaskiplistindex
usinga LIMITclause.
Since2.3thesamegoalcanbeachievedbyusingregularAQLconstructs,e.g.
453
FILTERdoc.value>=@value
SORTdoc.value
LIMIT1
RETURNdoc
Startupoptionchanges
Optionsadded
Thefollowingconfigurationoptionshavebeenaddedin2.6:
--server.session-timeout:allowscontrollingthetimeoutofusersessionsinthewebinterface.Thevalueisspecifiedinseconds.
--server.foxx-queues:controlswhethertheFoxxqueuemanagerwillcheckqueueandjobentries.Disablingthisoptioncanreduce
serverloadbutwillpreventjobsaddedtoFoxxqueuesfrombeingprocessedatall.
Thedefaultvalueis true,enablingtheFoxxqueuesfeature.
--server.foxx-queues-poll-interval:allowsadjustingthefrequencywithwhichtheFoxxqueuesmanagerischeckingthequeue(or
queues)forjobstobeexecuted.
Thedefaultvalueis 1second.Loweringthisvaluewillresultinthequeuemanagerwakingupandcheckingthequeuesmore
frequently,whichmayincreaseCPUusageoftheserver.
Note:thisoptiononlyhasaneffectwhen --server.foxx-queuesisnotsetto false.
Optionsremoved
Thefollowingconfigurationoptionshavebeenremovedin2.6.:
--log.severity:thedocsfor --log.severitymentionedlotsofseverities(e.g. exception, technical, functional,
development)butonlyafewseverities(e.g. all, human)wereactuallyused,with humanbeingthedefaultand allenabling
theadditionalloggingofincomingrequests.
Theoptionpretendedtocontrolalotofthingswhichitactuallydidn't.Additionally,theoption --log.requests-filewasaround
foralongtimealready,alsocontrollingrequestlogging.
Becausethe --log.severityoptioneffectivelydidnotcontrolthatmuch,itwasremoved.Asideeffectofremovingtheoptionis
that2.5installationsstartedwithoption --log.severityallwillnotlogrequestsaftertheupgradeto2.6.Thiscanbeadjustedby
settingthe --log.requests-fileoptioninstead.
Defaultvalueschanged
Thedefaultvaluesforthefollowingoptionshavechangedin2.6:
--database.ignore-datafile-errors:thedefaultvalueforthisoptionwaschangedfrom trueto false.
Ifthenewdefaultvalueof falseisused,thenarangodwillrefuseloadingcollectionsthatcontaindatafileswithCRCmismatches
orothererrors.Acollectionwithdatafileerrorswillthenbecomeunavailable.Thispreventsfollowuperrorsfromhappening.
Theonlywaytoaccesssuchcollectionistousethedatafiledebugger(arango-dfdb)andtrytorepairortruncatethedatafilewithit.
--server.request-timeout:thedefaultvaluewasincreasedfrom300to1200secondsforallclienttools(arangosh,arangoimp,
arangodump,arangorestore).
--server.connect-timeout:thedefaultvaluewasincreasedfrom3to5secondsforallclienttools(arangosh,arangoimp,
arangodump,arangorestore).
454
ifnecessary.
Changedbehavior
V8
TheV8versionshippedwithArangoDBwasupgradedfrom3.29.59to3.31.74.1.ThisleadstoadditionalECM AScript6(ES6or
"harmony")featuresbeingenabledbydefaultinArangoDB'sscriptingenvironment.
Apartfromthat,achangeintheinterpretationofcommand-lineoptionsbyV8mayaffectusers.ArangoDBpassesthevalueofthe
command-lineoption --javascript.v8-optionstoV8andleavesinterpretationofthecontentstoV8.Forexample,theArangoDB
option --javascript.v8-options="--harmony"couldbeusedtotellV8toenableitsharmonyfeatures.
InArangoDB2.4,thefollowingharmonyoptionsweremadeavailablebyV8:
--harmony_scoping(enableharmonyblockscoping)
--harmony_modules(enableharmonymodules(impliesblockscoping))
--harmony_proxies(enableharmonyproxies)
--harmony_generators(enableharmonygenerators)
--harmony_numeric_literals(enableharmonynumericliterals(0o77,0b11))
--harmony_strings(enableharmonystring)
--harmony_arrays(enableharmonyarrays)
--harmony_arrow_functions(enableharmonyarrowfunctions)
--harmony_classes(enableharmonyclasses)
--harmony_object_literals(enableharmonyobjectliteralextensions)
--harmony(enableallharmonyfeatures(exceptproxies))
Therewastheoption --harmony,whichturnedonalmostallharmonyfeatures.
InArangoDB2.5,V8providesthefollowingharmony-relatedoptions:
--harmony(enableallcompletedharmonyfeatures)
--harmony_shipping(enableallshippedharmonyfeatures)
--harmony_modules(enable"harmonymodules(impliesblockscoping)"(inprogress))
--harmony_arrays(enable"harmonyarraymethods"(inprogress))
--harmony_array_includes(enable"harmonyArray.prototype.includes"(inprogress))
--harmony_regexps(enable"harmonyregularexpressionextensions"(inprogress))
--harmony_arrow_functions(enable"harmonyarrowfunctions"(inprogress))
--harmony_proxies(enable"harmonyproxies"(inprogress))
--harmony_sloppy(enable"harmonyfeaturesinsloppymode"(inprogress))
--harmony_unicode(enable"harmonyunicodeescapes"(inprogress))
--harmony_tostring(enable"harmonytoString")
--harmony_numeric_literals(enable"harmonynumericliterals")
--harmony_strings(enable"harmonystringmethods")
--harmony_scoping(enable"harmonyblockscoping")
--harmony_classes(enable"harmonyclasses(impliesblockscoping&objectliteralextension)")
--harmony_object_literals(enable"harmonyobjectliteralextensions")
--harmony_templates(enable"harmonytemplateliterals")
Notethatthereareextraoptionsforbettercontrollingthededicatedfeatures,andespeciallythatthemeaningofthe --harmonyoption
haschangedfromenablingallharmonyfeaturestoallcompletedharmonyfeatures!
Usersshouldadjustthevalueof --javascript.v8-optionsaccordingly.
PleasenotethatincompleteharmonyfeaturesaresubjecttochangeinfutureV8releases.
455
Sparseindexes
Hashindexesandskiplistindexescannowbecreatedinasparsevariant.Whennotexplicitlyset,the sparseattributedefaultsto
falsefornewindexes.
Thiscausesachangeinbehaviorwhencreatingauniquehashindexwithoutspecifyingthesparseflag.Theuniquehashindexwillbe
createdinanon-sparsevariantinArangoDB2.5.
In2.4andbefore,uniquehashindexeswereimplicitlysparse,alwaysexcluding nullvaluesfromtheindex.Therewasnooptionto
controlthisbehavior,andsparsitywasneithersupportedfornon-uniquehashindexesnorskiplistsin2.4.Thisimplicitsparsityofjust
uniquehashindexeswasconsideredaninconsistency,andthereforethebehaviorwascleanedupin2.5.
Asof2.5,hashandskiplistindexeswillonlybecreatedsparseifsparsityisexplicitlyrequested.Thismayrequireachangeinindexcreatingclientcode,butonlyiftheclientcodecreatesuniquehashindexesandiftheyarestillintendedtobesparse.Inthiscase,the
clientcodeshouldexplicitlysetthe sparseflagto truewhencreatingauniquehashindex.
Existinguniquehashindexesfrom2.4orbeforewillautomaticallybemigratedsotheyarestillsparseaftertheupgradeto2.5.Forthese
indexes,the sparseattributewillbepopulatedautomaticallywithavalueof true.
Geoindexesareimplicitlysparse,meaningdocumentswithouttheindexedlocationattributeorcontaininginvalidlocationcoordinate
valueswillbeexcludedfromtheindexautomatically.Thisisalsoachangewhencomparedtopre-2.5behavior,whendocumentswith
missingorinvalidcoordinatevaluesmayhavecausederrorsoninsertionwhenthegeoindex' uniqueflagwassetandits ignoreNull
flagwasnot.
Thiswasconfusingandhasbeenrectifiedin2.5.Themethod ensureGeoConstraint()nowdoesthesameas ensureGeoIndex().
Furthermore,theattributes constraint, unique, ignoreNulland sparseflagsarenowcompletelyignoredwhencreatinggeo
indexes.Clientindexcreationcodethereforedoesnotneedtosetthe ignoreNullor constraintattributeswhencreatingageoindex.
Thesameistrueforfulltextindexes.Thereisnoneedtospecifynon-uniquenessorsparsityforgeoorfulltextindexes.Theywillalways
benon-uniqueandsparse.
MovedFoxxapplicationstoadifferentfolder.
Until2.4foxxappswerestoredinthefollowingfolderstructure: <app-path>/databases/<dbname>/<appname>:<appversion>.Thiscaused
sometroubleasappswherecachedbasedonnameandversionandupdatesdidnotapply.Alsothepathonfilesystemandtheapp's
accessURLhadnorelationtooneanother.NowthepathonfilesystemisidenticaltotheURL(excepttheappendedAPP): <apppath>/_db/<dbname>/<mointpoint>/APP
FoxxDevelopmentmode
Thedevelopmentmodeuseduntil2.4isgone.Ithasbeenreplacedbyamuchmorematureversion.Thisincludesthedeprecationofthe
javascript.dev-app-pathparameter,whichisuselesssince2.5.Insteadofhavingtwoseparateappdirectoriesforproductionand
development,appsnowresideinoneplace,whichisusedforproductionaswellasfordevelopment.Appscanstillbeputinto
developmentmode,changingtheirbehaviorcomparedtoproductionmode.Developmentmodeappsarestillrereadfromdiskatevery
request,andstilltheyshipmoredebugoutput.
Thischangehasalsomadethestartupoptions --javascript.frontend-development-modeand --javascript.dev-app-pathobsolete.The
formeroptionwillnothaveanyeffectwhenset,andthelatteroptionisonlyreadandusedduringtheupgradeto2.5anddoesnothave
anyeffectslater.
Foxxinstallprocess
InstallingFoxxappshasbeenatwostepprocess:importthemintoArangoDBandmountthemataspecificmountpoint.These
operationshavebeenjoinedtogether.Youcaninstallanappatonemountpoint,that'sit.Nofetch,mount,unmount,purgecycle
anymore.Thecommandshavebeensimplifiedtojust:
install:getyourFoxxappupandrunning
uninstall:shutitdownanderaseitfromdisk
Deprecatedfeatures
456
Foxx:method Model#toJSONSchema(id)isdeprecated,itwillraiseawarningifyouuseit.Pleaseuse Foxx.toJSONSchema(id,model)
instead.
Removedfeatures
Startupswitch --javascript.frontend-development-mode:Itsmajorpurposewasinternaldevelopmentanyway.Nowtheweb
frontendcanbesettodevelopmentmodesimilartoanyotherfoxxapp.
Startupswitch --javascript.dev-app-path:WasusedforthedevelopmentmodeofFoxx.Thisisintegratedwiththenormalapppathnowandcanbetriggeredonapplevel.Thesecondapp-pathissuperfluous.
Foxx: controller.collection:Pleaseuse appContext.collectioninstead.
Foxx: FoxxRepository.modelPrototype:Pleaseuse FoxxRepository.modelinstead.
Foxx: Model.extend({},{attributes:{}}):Pleaseuse Model.extend({schema:{}})instead.
Foxx: requestContext.bodyParam(paramName,description,Model):Pleaseuse requestContext.bodyParam(paramName,options)
instead.
Foxx: requestContext.queryParam({type:string}):Pleaseuse requestContext.queryParam({type:joi})instead.
Foxx: requestContext.pathParam({type:string}):Pleaseuse requestContext.pathParam({type:joi})instead.
Graph:Themodules org/arangodb/graphand org/arangodb/graph-blueprint:Pleaseusemodule org/arangodb/general-graph
instead.NOTE:Thisdoesnotmeanwedonotsupportblueprintsanymore.Generalgraphcoverseverythingthegraph--blueprint
did,plusmanymorefeatures.
General-Graph:Inthemodule org/arangodb/general-graphthefunctions _undirectedRelationand _directedRelationareno
longeravailable.Bothfunctionshavebeenunifiedto _relation.
457
ifnecessary.
Changedbehavior
V8upgrade
ThebundledV8versionhasbeenupgradedfrom3.16.14to3.29.59.
Thenewversionprovidesbettererrorchecking,whichcanleadtosubtlechangesintheexecutionofJavaScriptcode.
Thefollowingcode,thoughnonsense,runswithouterrorin2.3and2.4whenstrictmodeisnotenabled:
(function(){
a=true;
a.foo=1;
})();
Whenenablingstrictmode,thefunctionwillthrowanerrorin2.4butnotin2.3:
(function(){
"usestrict";
a=true;
a.foo=1;
})();
TypeError:Cannotassigntoreadonlyproperty'foo'oftrue
Thoughthisisachangeinbehavioritcanbeconsideredanimprovement.Thenewversionactuallyuncoversanerrorthatwent
undetectedintheoldversion.
Errormessageshavealsochangedslightlyinthenewversion.ApplicationsthatrelyontheexacterrormessagesoftheJavaScriptengine
mayneedtobeadjustedsotheylookfortheupdatederrormessages.
Defaultendpoint
Thedefaultendpointforarangodisnow 127.0.0.1.
ThischangewillmodifytheIPaddressArangoDBlistensonto127.0.0.1bydefault.ThiswillmakenewArangoDBinstallations
unaccessiblefromclientsotherthanlocalhostunlesstheconfigurationischanged.Thisisasecurityfeature.
TomakeArangoDBaccessiblefromanyclient,changetheserver'sconfiguration( --server.endpoint)toeither tcp://0.0.0.0:8529or
theserver'spubliclyvisibleIPaddress.
Replication
SystemcollectionsarenowincludedinthereplicationandallreplicationAPIreturnvaluesbydefault.
Thiswillleadtouseraccountsandcredentialsdatabeingreplicatedfrommastertoslaveservers.Thismayoverwriteslave-specific
databaseusers.
Thismaybeconsideredafeatureorananti-feature,soitisconfigurable.
Ifreplicationofsystemcollectionsisundesired,theycanbeexcludedfromreplicationbysettingthe includeSystemattributeto false
inthefollowingcommands:
initialsynchronization: replication.sync({includeSystem:false})
continuousreplication: replication.applier.properties({includeSystem:false})
458
Thiswillexcludeallsystemcollections(including _aqlfunctions, _graphsetc.)fromtheinitialsynchronizationandthecontinuous
replication.
Ifthisisalsoundesired,itisalsopossibletospecifyalistofcollectionstoexcludefromtheinitialsynchronizationandthecontinuous
replicationusingthe restrictCollectionsattribute,e.g.:
require("org/arangodb/replication").applier.properties({
includeSystem:true,
restrictType:"exclude",
restrictCollections:["_users","_graphs","foo"]
});
Theaboveexamplewillingeneralincludesystemcollections,butwillexcludethespecifiedthreecollectionsfromcontinuousreplication.
TheHTTPRESTAPImethodsforfetchingthereplicationinventoryandfordumpingcollectionsalsosupportthe includeSystem
controlflagviaaURLparameterofthesamename.
Buildprocesschanges
Severaloptionsforthe configurecommandhavebeenremovedin2.4.Theoptions
--enable-all-in-one-v8
--enable-all-in-one-icu
--enable-all-in-one-libev
--with-libev=DIR
--with-libev-lib=DIR
--with-v8=DIR
--with-v8-lib=DIR
--with-icu-config=FILE
arenotavailableanymorebecausethebuildprocesswillalwaysusethebundledversionsofthelibraries.
WhenbuildingArangoDBfromsourceinadirectorythatalreadycontainedapre-2.4version,itwillbenecessarytoruna make
supercleancommandonceandafullrebuildafterwards:
gitpull
makesuperclean
makesetup
./configure<optionsgohere>
make
Asaconsequenceofglobalrenaminginthecodebase,theoption mergeArrayshasbeenrenamedto mergeObjects.Thisoptioncontrols
whetherJSONobjectswillbemergedonanupdateoperationoroverwritten.Thedefaulthasbeen,andstillis,tomerge.Notspecifying
theparameterwillleadtoamerge,asithasbeenthebehaviorinArangoDBeversince.
ThisaffectstheHTTPRESTAPImethodPATCH /_api/document/collection/key.ItsoptionalURLparameter mergeArraysforthe
optionhasbeenrenamedto mergeObjects.
TheAQL UPDATEstatementisalsoaffected,asitsoption mergeArrayshasalsobeenrenamedto mergeObjects.The2.3query
UPDATEdocINcollectionWITH{...}INcollectionOPTIONS{mergeArrays:false}
shouldthusberewrittentothefollowingin2.4:
UPDATEdocINcollectionWITH{...}INcollectionOPTIONS{mergeObjects:false}
Deprecatedfeatures
459
For FoxxControllerobjects,themethod collection()isdeprecatedandwillberemovedinfutureversionofArangoDB.Usingthis
methodwillissueawarning.Pleaseuse applicationContext.collection()instead.
For FoxxRepositoryobjects,theproperty modelPrototypeisnowdeprecated.Usingitwillissueawarning.Pleaseuse
FoxxRepository.modelinstead.
In FoxxController/ RequestContext,callingmethod bodyParam()withthreeargumentsisdeprecated.Pleaseuse
.bodyParam(paramName,options)instead.
In FoxxController/ RequestContextcallingmethod queryParam({type:string})isdeprecated.Pleaseuse
requestContext.queryParam({type:joi})instead.
In FoxxController/ RequestContextcallingmethod pathParam({type:string})isdeprecated.Pleaseuse
requestContext.pathParam({type:joi})instead.
For FoxxModel,calling Model.extend({},{attributes:{}})isdeprecated.Pleaseuse Model.extend({schema:{}})instead.
Inmodule org/arangodb/general-graph,thefunctions _undirectedRelation()and _directedRelation()aredeprecatedandwillbe
removedinafutureversionofArangoDB.Bothfunctionshavebeenunifiedto _relation().
Themodules org/arangodb/graphand org/arangodb/graph-blueprintaredeprecated.Pleaseusemodule org/arangodb/general-graph
instead.
TheHTTPRESTAPI _api/graphandallitsmethodsaredeprecated.PleaseusethegeneralgraphAPI _api/gharialinstead.
Removedfeatures
Thefollowingreplication-relatedJavaScriptmethodsbecameobsoleteinArangoDB2.2andhavebeenremovedinArangoDB2.4:
require("org/arangodb/replication").logger.start()
require("org/arangodb/replication").logger.stop()
require("org/arangodb/replication").logger.properties()
TheRESTAPImethodsforthesefunctionshavealsobeenremovedinArangoDB2.4:
HTTPPUT /_api/replication/logger-start
HTTPPUT /_api/replication/logger-stop
HTTPGET /_api/replication/logger-config
HTTPPUT /_api/replication/logger-config
Clientapplicationsthatcalloneofthesemethodsshouldbeadjustedbyremovingthecallstothesemethods.Thisshouldn'tbe
problematicasthesemethodshavebeenno-opssinceArangoDB2.2anyway.
460
ifnecessary.
Configurationfilechanges
Threadsandcontexts
Thenumberofserverthreadsspecifiedisnowtheminimumofthreadsstarted.Therearesituationinwhichthreadsarewaitingforresults
ofdistributeddatabaseservers.Inthiscasethenumberofthreadsisdynamicallyincreased.
WithArangoDB2.3,thenumberofserverthreadscanbeconfiguredindependentlyofthenumberofV8contexts.Theconfiguration
option --javascript.v8-contextswasaddedtoarangodtoprovidebettercontroloverthenumberofV8contextscreatedinarangod.
Previously,thenumberofV8contextsarangodcreatedatstartupwasequaltothenumberofserverthreads(asspecifiedbyoption -server.threads).
InsomesituationsitmaybemoresensibletocreatedifferentamountsofthreadsandV8contexts.ThisisbecauseeachV8contexts
createdwillconsumememoryandrequiresCPUresourcesforperiodicgarbagecollection.Contrary,serverthreadsdonothavesuchhigh
memoryorCPUfootprint.
Iftheoption --javascript.v8-contextsisnotspecified,thenumberofV8contextscreatedatstartupwillremainequaltothenumberof
serverthreads.ThusnochangeinconfigurationisrequiredtokeepthesamebehaviorasinpreviousArangoDBversions.
Ifyouareusingthedefaultconfigfilesormergethemwithyourlocalconfigfiles,pleasereviewifthedefaultnumberofserverthreadsis
okayinyourenvironment.AdditionallyyoushouldverifythatthenumberofV8contextscreated(asspecifiedinoption -javascript.v8-contexts)isokay.
Syslog
Thecommand-lineoption --log.syslogwasusedinpreviousversionsofArangoDBtoturnloggingtosyslogonoroff:whensettingto
anon-emptystring,syslogloggingwasturnedon,otherwiseturnedoff.Whensyslogloggingwasturnedon,loggingwasdonewiththe
applicationnamespecifiedin --log.application,whichdefaultedto triagens.Therewasalsoacommand-lineoption -log.hostnamewhichcouldbesetbutdidnothaveanyeffect.
Thisbehaviorturnedouttobeunintuitiveandwaschangedin2.3asfollows:
thecommand-lineoption --log.syslogisdeprecatedanddoesnothaveanyeffectwhenstartingArangoDB.
toturnonsyslogloggingin2.3,theoption --log.facilityhastobesettoanon-emptystring.Thevaluefor facilityisOSdependent(possiblevaluescanbefoundin /usr/include/syslog.horthelike- usershouldbeavailableonmanysystems).
thedefaultvaluefor --log.applicationhasbeenchangedfrom triagensto arangod.
thecommand-lineoption --log.hostnameisdeprecatedanddoesnothaveanyeffectwhenstartingArangoDB.Instead,thehost
namewillbesetbysyslogautomatically.
whenloggingtosyslog,ArangoDBnowomitsthedatetimeprefixandtheprocessid,becausethey'llbeaddedbysyslog
automatically.
AQL
AQLqueriesthrowlessexceptions
ArangoDB2.3containsacompletelyrewrittenAQLqueryoptimizerandexecutionengine.ThismeansthatAQLquerieswillbe
executedwithadifferentenginethaninArangoDB2.2andearlier.PartsofAQLqueriesmightbeexecutedindifferentorderthanbefore
becausetheAQLoptimizerhasmorefreedomtomovethingsaroundinaquery.
461
InpreviousversionsofArangoDB,AQLqueriesabortedwithanexceptioninmanysituationsandthrewaruntimeexception.Exceptions
werethrownwhentryingtofindavalueusingthe INoperatorinanon-arrayelement,whentryingtousenon-booleanvalueswiththe
logicaloperands &&or ||or !,whenusingnon-numericvaluesinarithmeticoperations,whenpassingwrongparametersinto
functionsetc.
InArangoDB2.3thishasbeenchangedinmanycasestomakeAQLmoreuser-friendlyandtoallowtheoptimizationtoperformmuch
morequeryoptimizations.
Hereisasummaryofchanges:
whenanon-arrayvalueisusedontheright-handsideofthe INoperator,theresultwillbe falseinArangoDB2.3,andno
exceptionwillbethrown.
thebooleanoperators &&and ||donotthrowinArangoDB2.3ifanyoftheoperandsisnotabooleanvalue.Instead,theywill
performanimplicitcastofthevaluestobooleans.Theirresultwillbeasfollows:
lhs&&rhswillreturn lhsifitis falseorwouldbe falsewhenconvertedintoaboolean.If lhsis trueorwould
be truewhenconvertedtoaboolean, rhswillbereturned.
lhs||rhswillreturn lhsifitis trueorwouldbe truewhenconvertedintoaboolean.If lhsis falseorwouldbe
falsewhenconvertedtoaboolean, rhswillbereturned.
!valuewillreturnthenegatedvalueof valueconvertedintoaboolean
thearithmeticoperators( +, -, *, /, %)canbeappliedtoanyvalueandwillnotthrowexceptionswhenappliedtononnumericvalues.Instead,anyvalueusedintheseoperatorswillbecastedtoanumericvalueimplicitly.Ifnonumericresultcanbe
producedbyanarithmeticoperator,itwillreturn nullinArangoDB2.3.Thisisalsotruefordivisionbyzero.
passingargumentsofinvalidtypesintoAQLfunctionsdoesnotthrowaruntimeexceptioninmostcases,butmayproduceruntime
warnings.Built-inAQLfunctionsthatreceiveinvalidargumentswillthenreturn null.
NestedFORloopexecutionorder
Thequeryoptimizerin2.3maypermutetheorderofnested FORloopsinAQLqueries,providedthatexchangingtheloopswillnot
alteraqueryresult.However,achangeintheorderofreturnedvaluesisallowedbecausenosortorderisguaranteedbyAQL(andwas
never)unlessanexplicit SORTstatementisusedinaquery.
ChangedreturnvaluesofArangoQueryCursor.getExtra()
Thereturnvalueof ArangoQueryCursor.getExtra()hasbeenchangedinArangoDB2.3.Itnowcontainsa statsattributewith
statisticsaboutthequerypreviouslyexecuted.Italsocontainsa warningsattributewithwarningsthathappenedduringquery
execution.Thereturnvaluestructurehasbeenunifiedin2.3forbothread-onlyanddata-modificationqueries.
Thereturnvaluelookslikethisforaread-onlyquery:
arangosh>stmt=db._createStatement("FORiINmycollectionRETURNi");stmt.execute().getExtra()
{
"stats":{
"writesExecuted":0,
"writesIgnored":0,
"scannedFull":2600,
"scannedIndex":0
},
"warnings":[]
}
Fordata-modificationqueries,ArangoDB2.3returnsaresultwiththesamestructure:
arangosh>stmt=db._createStatement("FORiINxxREMOVEiINxx");stmt.execute().getExtra()
{
"stats":{
"writesExecuted":2600,
"writesIgnored":0,
"scannedFull":2600,
"scannedIndex":0
},
"warnings":[]
}
462
InArangoDB2.2,thereturnvalueof ArangoQueryCursor.getExtra()wasemptyforread-onlyqueriesandcontainedanattribute
operationswithtwosub-attributesfordata-modificationqueries:
arangosh>stmt=db._createStatement("FORiINmycollectionRETURNi");stmt.execute().getExtra()
{
}
arangosh>stmt=db._createStatement("FORiINmycollectionREMOVEiINmycollection");stmt.execute().getExtra()
{
"operations":{
"executed":2600,
"ignored":0
}
}
ChangedreturnvaluesinHTTPmethod POST/_api/cursor
ThepreviouslymentionedchangealsoleadstothestatisticsbeingreturnedintheHTTPRESTAPImethod POST/_api/cursor.
Previously,thereturnvaluecontainedanoptional extraattributethatwasfilledonlyfordata-modificationqueriesandinsomeother
casesasfollows:
{
"result":[],
"hasMore":false,
"extra":{
"operations":{
"executed":2600,
"ignored":0
}
}
}
WiththechangedresultstructureinArangoDB2.3,the extraattributeintheresultwilllooklikethis:
{
"result":[],
"hasMore":false,
"extra":{
"stats":{
"writesExecuted":2600,
"writesIgnored":0,
"scannedFull":0,
"scannedIndex":0
},
"warnings":[]
}
}
Ifthequeryoption fullCountisrequested,the fullCountresultvaluewillalsobereturnedinsidethe statsattributeofthe extra
attribute,andnotdirectlyasanattributeinsidethe extraattributeasin2.2.Notethata fullCountwillonlybepresentin
extra. statsifitwasrequestedasanoptionforthequery.
TheresultinArangoDB2.3willalsocontaina warningsattributewiththearrayofwarningsthathappenedduringqueryexecution.
ChangedreturnvaluesinArangoStatement.explain()
Thereturnvalueof ArangoStatement.explain()haschangedsignificantlyinArangoDB2.3.Thenewreturnvaluestructureisnot
compatiblewiththestructurereturnedby2.2.
InArangoDB2.3,thefullexecutionplanforanAQLqueryisreturnedalongsideallappliedoptimizerrules,optimizationwarningsetc.It
isalsopossibletohavetheoptimizerreturnallexecutionplans.Thisrequiredanewdatastructure.
Clientprogramsthatuse ArangoStatement.explain()ortheHTTPRESTAPImethod POST/_api/explainmayneedtobeadjustedto
usethenewreturnformat.
463
Thereturnvalueof ArangoStatement.parse()hasbeenextendedinArangoDB2.3.Inadditiontotheexistingattributes,ArangoDB2.3
willalsoreturnan astattributecontainingtheabstractsyntaxtreeofthestatement.Thisextraattributecansafelybeignoredbyclient
programs.
Variablesnotupdatableinqueries
PreviousversionsofArangoDBallowedthemodificationofvariablesinsideAQLqueries,e.g.
LETcounter=0
FORiIN1..10
LETcounter=counter+1
RETURNcounter
Whilethisisadmittedlyaconvenientfeature,thenewqueryoptimizerdesigndidnotallowtokeepit.Additionally,updatingvariables
insideaquerywouldpreventalotofoptimizationstoqueriesthatwewouldliketheoptimizertomake.Additionally,updatingvariables
inqueriesthatrunondifferentnodesinaclusterwouldlikecausenon-deterministicbehaviorbecausequeriesarenotexecutedlinearly.
Changedreturnvalueof TO_BOOL
TheAQLfunction TO_BOOLnowalwaysreturnstrueifitsargumentisaarrayoranobject.InpreviousversionsofArangoDB,the
functionreturnedfalseforemptyarraysorforobjectswithoutattributes.
Changedreturnvalueof TO_NUMBER
TheAQLfunction TO_NUMBERnowreturnsnullifitsargumentisanobjectoranarraywithmorethanonemember.Inpreviousversion
ofArangoDB,thereturnvalueinthesecaseswas0. TO_NUMBERwillreturn0foremptyarray,andthenumericequivalentofthearray
member'svalueforarrayswithasinglemember.
NewAQLkeywords
ThefollowingkeywordshavebeenaddedtoAQLinArangoDB2.3:
NOT
AND
OR
UnquotedusageofthesekeywordsforattributenamesinAQLquerieswilllikelyfailinArangoDB2.3.Ifanysuchattributenameneeds
tobeusedinaquery,itshouldbeenclosedinbacktickstoindicatetheusageofaliteralattributename.
Removedfeatures
Bitarrayindexes
Bitarrayindexeswereonlyhalf-waydocumentedandintegratedinpreviousversionsofArangoDBsotheirbenefitwaslimited.The
supportforbitarrayindexeshasthusbeenremovedinArangoDB2.3.Itisnotpossibletocreateindexesoftype"bitarray"with
ArangoDB2.3.
WhenacollectionisopenedthatcontainsabitarrayindexdefinitioncreatedwithapreviousversionofArangoDB,ArangoDBwillignore
itandlogthefollowingwarning:
indextype'bitarray'isnotsupportedinthisversionofArangoDBandisignored
FutureversionsofArangoDBmayautomaticallyremovesuchindexdefinitionssothewarningswilleventuallydisappear.
Otherremovedfeatures
TheHTTPRESTAPImethodat POST/_admin/modules/flushhasbeenremoved.
464
Knownissues
InArangoDB2.3.0,AQLqueriescontainingfilterconditionswithanINexpressionwillnotyetuseanindex:
FORdocINcollectionFILTERdoc.indexedAttributeIN[...]RETURNdoc
FORdocINcollection
FILTERdoc.indexedAttributeIN[...]
RETURNdoc
We’recurrentlyworkingongettingtheINoptimizationsdone,andwillshipthemina2.3maintenancereleasesoon(e.g.2.3.1or2.3.2).
465
Appendix
References:Briefoverviewsoverinterfacesandobjects
db:the dbobject
collection:the collectionobject
JavaScriptM odules:Listofbuilt-inandsupportedJSmodules
Deprecated:Featuresthatareconsideredobsoleteandmaygetremovedeventually
Errorcodesandmeanings:Listofallpossibleerrorsthatcanbeencountered
Glossary:Disambiguationpage
Appendix
466
References
References
467
The"db"Object
The dbobjectisavailableinarangoshbydefault,andcanalsobeimportedandusedinFoxxservices.
db.namereturnsacollectionobjectforthecollectionname.
Thefollowingmethodsexistsonthe_dbobject:
Database
db._createDatabase(name,options,users)
db._databases()
db._dropDatabase(name,options,users)
db._useDatabase(name)
Indexes
db._index(index)
db._dropIndex(index)
Properties
db._id()
db._isSystem()
db._name()
db._path()
db._version()
Collection
db._collection(name)
db._create(name)
db._drop(name)
db._truncate(name)
AQL
db._createStatement(query)
db._query(query)
db._explain(query)
Document
db._document(object)
db._exists(object)
db._remove(selector)
db._replace(selector,data)
db._update(selector,data)
db
468
The"collection"Object
Thefollowingmethodsexistonthecollectionobject(returnedbydb.name):
Collection
collection.checksum()
collection.count()
collection.drop()
collection.figures()
collection.load()
collection.properties()
collection.reserve()
collection.revision()
collection.rotate()
collection.toArray()
collection.truncate()
collection.type()
collection.unload()
Indexes
collection.dropIndex(index)
collection.ensureIndex(description)
collection.getIndexes(name)
collection.index(index)
Document
collection.all()
collection.any()
collection.closedRange(attribute,left,right)
collection.document(object)
collection.documents(keys)
collection.edges(vertex-id)
collection.exists(object)
collection.firstExample(example)
collection.inEdges(vertex-id)
collection.insert(data)
collection.edges(vertices)
collection.iterate(iterator,options)
collection.outEdges(vertex-id)
collection.queryByExample(example)
collection.range(attribute,left,right)
collection.remove(selector)
collection.rename()
collection.replace(selector,data)
collection.replaceByExample(example,data)
collection.save(data)
collection.update(selector,data)
collection.updateByExample(example,data)
collection
469
JavaScriptModules
ArangoDBusesaNode.jscompatiblemodulesystem.Youcanusethefunctionrequireinordertoloadamoduleorlibrary.Itreturnsthe
exportedvariablesandfunctionsofthemodule.
Theglobalvariables global, process, console, Buffer, __filenameand __dirnameareavailablethroughoutArangoDBand
Foxx.
Nodecompatibilitymodules
ArangoDBsupportsanumberofmodulesforcompatibilitywithNode.js,including:
assertimplementsbasicassertionandtestingfunctions.
bufferimplementsabinarydatatypeforJavaScript.
consoleisawellknownloggingfacilitytoalltheJavaScriptdevelopers.ArangoDBimplementsmostoftheConsoleAPI,withthe
exceptionsofprofileandcount.
eventsimplementsaneventemitter.
fsprovidesafilesystemAPIforthemanipulationofpaths,directories,files,links,andtheconstructionoffilestreams.ArangoDB
implementsmostFilesystem/Afunctions.
moduleprovidesdirectaccesstothemodulesystem.
pathimplementsfunctionsdealingwithfilenamesandpaths.
punycodeimplementsconversionfunctionsforpunycodeencoding.
querystringprovidesutilitiesfordealingwithquerystrings.
streamprovidesastreaminginterface.
string_decoderimplementslogicfordecodingbuffersintostrings.
urlprovidesutilitiesforURLresolutionandparsing.
utilprovidesgeneralutilityfunctionslike formatand inspect.
AdditionallyArangoDBprovidespartialimplementationsforthefollowingmodules:
net:only isIP, isIPv4and isIPv6.
process:only envand cwd;stubsfor argv, stdout.isTTY, stdout.write, nextTick.
timers:stubsfor setImmediate, setTimeout, setInterval, clearImmediate, clearTimeout, clearIntervaland ref.
tty:only isatty(alwaysreturns false).
vm:only runInThisContext.
ThefollowingNode.jsmodulesarenotavailableatall: child_process, cluster, constants, crypto(butsee @arangodb/crypto
below), dgram, dns, domain, http, https, os, sys, tls, v8, zlib.
ArangoDBSpecificModules
TherearealargenumberofArangoDB-specificmodulesusingthe @arangodbnamespace,mostlyforinternalusebyArangoDBitself.
Thefollowinghoweverarenoteworthy:
@arangodb/cryptoprovidesvariouscryptographyfunctionsincludinghashingalgorithms.
@arangodb/foxxisthenamespaceprovidingthevariousbuildingblocksoftheFoxxmicroserviceframework.
JavaScriptM odules
470
BundledNPMModules
ThefollowingNPM modulesarepreinstalled:
aqbistheArangoDBQueryBuilderandcanbeusedtoconstructAQLquerieswithachainingJavaScriptAPI.
chaiisafull-featuredassertionlibraryforwritingJavaScripttests.
dedentisasimpleutilityfunctionforformattingmulti-linestrings.
error-stack-parserparsesstacktracesintoamoreusefulformat.
graphql-syncisanArangoDB-compatibleGraphQLserver/schemaimplementation.
highlight.jsisanHTM Lsyntaxhighlighter.
i(inflect)isautilitylibraryforinflecting(e.g.pluralizing)words.
joiisavalidationlibrarythatissupportedthroughouttheFoxxframework.
js-yamlisaJavaScriptimplementationoftheYAM Ldataformat(apartialsupersetofJSON).
lodashisautilitybeltforJavaScriptprovidingvarioususefulhelperfunctions.
minimatchisaglobmatcherformatchingwildcardsinfilepaths.
node-semverisautilitylibraryforhandlingsemverversionnumbers.
qsprovidesutilitiesfordealingwithquerystringsusingadifferentformatthanthequerystringmodule.
sinonisamockinglibraryforwritingteststubs,mocksandspies.
JavaScriptM odules
471
ConsoleModule
require('console')
TheimplementationfollowstheCommonJSspecificationConsole.
console.assert
console.assert(expression,format,argument1,...)
Teststhatanexpressionistrue.Ifnot,logsamessageandthrowsanexception.
Examples
console.assert(value==="abc","expected:value===abc,actual:",value);
console.debug
console.debug(format,argument1,...)
Formatstheargumentsaccordingtoformatandlogstheresultasdebugmessage.Notethatdebugmessageswillonlybeloggedifthe
serverisstartedwithloglevelsdebugortrace.
Stringsubstitutionpatterns,whichcanbeusedinformat.
%%sstring
%%d,%%iinteger
%%ffloatingpointnumber
%%oobjecthyperlink
Examples
console.debug("%s","thisisatest");
console.dir
console.dir(object)
Logsalistingofallpropertiesoftheobject.
Exampleusage:
console.dir(myObject);
console.error
console.error(format,argument1,...)
Formatstheargumentsaccordingto@FA{format}andlogstheresultaserrormessage.
%%sstring
%%d,%%iinteger
%%oobjecthyperlink
Exampleusage:
console.error("error'%s':%s",type,message);
console
472
console.getline
console.getline()
Readsinalinefromtheconsoleandreturnsitasstring.
console.group
console.group(format,argument1,...)
Formatstheargumentsaccordingtoformatandlogstheresultaslogmessage.Opensanestedblocktoindentallfuturemessagessent.
CallgroupEndtoclosetheblock.Representationofblockisuptotheplatform,itcanbeaninteractiveblockorjustasetofindentedsub
messages.
Exampleusage:
console.group("userattributes");
console.log("name",user.name);
console.log("id",user.id);
console.groupEnd();
console.groupCollapsed
console.groupCollapsed(format,argument1,...)
Sameasconsole.group.
console.groupEnd
console.groupEnd()
Closesthemostrecentlyopenedblockcreatedbyacalltogroup.
console.info
console.info(format,argument1,...)
Formatstheargumentsaccordingtoformatandlogstheresultasinfomessage.
%%sstring
%%d,%%iinteger
%%oobjecthyperlink
Exampleusage:
console.info("The%sjumpedover%dfences",animal,count);
console.log
console.log(format,argument1,...)
Formatstheargumentsaccordingtoformatandlogstheresultaslogmessage.Thisisanaliasforconsole.info.
console.time
console.time(name)
Createsanewtimerunderthegivenname.CalltimeEndwiththesamenametostopthetimerandlogthetimeelapsed.
Exampleusage:
console
473
console.time("mytimer");
...
console.timeEnd("mytimer");//thiswillprinttheelapsedtime
console.timeEnd
console.timeEnd(name)
Stopsatimercreatedbyacalltotimeandlogsthetimeelapsed.
console.timeEnd
console.trace()
LogsastacktraceofJavaScriptexecutionatthepointwhereitiscalled.
console.warn
console.warn(format,argument1,...)
Formatstheargumentsaccordingtoformatandlogstheresultaswarnmessage.
%%sstring
%%d,%%iinteger
%%oobjecthyperlink
console
474
CryptoModule
constcrypto=require('@arangodb/crypto')
Thecryptomoduleprovidesimplementationsofvarioushashingalgorithmsaswellascryptographyrelatedfunctions.
Nonces
Thesefunctionsdealwithcryptographicnonces.
createNonce
crypto.createNonce():string
Createsacryptographicnonce.
Returnsthecreatednonce.
checkAndMarkNonce
crypto.checkAndMarkNonce(nonce):void
Checksandmarksanonce.
Arguments
nonce: string
Thenoncetocheckandmark.
Returnsnothing.
Randomvalues
Thefollowingfunctionsdealwithgeneratingrandomvalues.
rand
crypto.rand():number
Generatesarandomintegerthatmaybepositive,negativeorevenzero.
Returnsthegeneratednumber.
genRandomAlphaNumbers
crypto.genRandomAlphaNumbers(length):string
Generatesastringofrandomalpabeticalcharactersanddigits.
Arguments
length: number
Thelengthofthestringtogenerate.
Returnsthegeneratedstring.
genRandomNumbers
crypto.genRandomNumbers(length):string
Generatesastringofrandomdigits.
crypto
475
Arguments
length: number
genRandomSalt
crypto.genRandomSalt(length):string
Generatesastringofrandom(printable)ASCIIcharacters.
Arguments
length: number
JSONWebTokens(JWT)
ThesemethodsimplementtheJSONWebTokenstandard.
jwtEncode
crypto.jwtEncode(key,message,algorithm):string
GeneratesaJSONWebTokenforthegivenmessage.
Arguments
key: string|null
Thesecretcryptographickeytobeusedtosignthemessageusingthegivenalgorithm.Notethatthisfunctionwillraiseanerrorif
thekeyisomittedbutthealgorithmexpectsakey,andalsoifthealgorithmdoesnotexpectakeybutakeyisprovided(e.g.when
using "none").
message: string
M essagetobeencodedasJWT.Notethatthemessagewillonlybebase64-encodedandsigned,notencrypted.Donotstore
sensitiveinformationintokensunlesstheywillonlybehandledbytrustedparties.
algorithm: string
Nameofthealgorithmtouseforsigningthemessage,e.g. "HS512".
ReturnstheJSONWebToken.
jwtDecode
crypto.jwtDecode(key,token,noVerify):string|null
Arguments
key: string|null
Thesecretcryptographickeythatwasusedtosignthemessageusingthealgorithmindicatedbythetoken.Notethatthisfunction
willraiseanerrorifthekeyisomittedbutthealgorithmexpectsakey.
Ifthealgorithmdoesnotexpectakeybutakeyisprovided,thetokenwillfailtoverify.
token: string
Thetokentodecode.
crypto
476
Notethatthefunctionwillraiseanerrorifthetokenismalformed(e.g.doesnothaveexactlythreesegments).
noVerify: boolean(Default: false)
Whetherverificationshouldbeskipped.Ifthisissetto truethesignatureofthetokenwillnotbeverified.Otherwisethefunction
willraiseanerrorifthesignaturecannotbeverifiedusingthegivenkey.
ReturnsthedecodedJSONmessageor nullifnotokenisprovided.
jwtAlgorithms
AhelperobjectcontainingthesupportedJWTalgorithms.EachattributenamecorrespondstoaJWT algandthevalueisanobject
with signand verifymethods.
jwtCanonicalAlgorithmName
crypto.jwtCanonicalAlgorithmName(name):string
AhelperfunctionthattranslatesaJWT algvaluefoundinaJWTheaderintothecanonicalnameofthealgorithmin jwtAlgorithms.
Raisesanerrorifnoalgorithmwithamatchingnameisfound.
Arguments
name: string
Algorithmnametolookup.
Returnsthecanonicalnameforthealgorithm.
Hashingalgorithms
md5
crypto.md5(message):string
HashesthegivenmessageusingtheM D5algorithm.
Arguments
message: string
Themessagetohash.
Returnsthecryptographichash.
sha1
crypto.sha1(message):string
HashesthegivenmessageusingtheSHA-1algorithm.
Arguments
message: string
Themessagetohash.
sha224
Arguments
crypto
477
message: string
Themessagetohash.
sha256
Arguments
message: string
Themessagetohash.
sha384
Arguments
message: string
Themessagetohash.
sha512
Arguments
message: string
Themessagetohash.
Miscellaneous
constantEquals
crypto.constantEquals(str1,str2):boolean
Comparestwostrings.Thisfunctioniteratesovertheentirelengthofbothstringsandcanhelpmakingcertaintimingattacksharder.
Arguments
str1: string
Thefirststringtocompare.
str2: string
Thesecondstringtocompare.
Returns trueifthestringsareequal, falseotherwise.
pbkdf2
crypto
478
crypto.pbkdf2(salt,password,iterations,keyLength):string
GeneratesaPBKDF2-HM AC-SHA1hashofthegivenpassword.
Arguments
salt: string
Thecryptographicsalttohashthepasswordwith.
password: string
Themessageorpasswordtohash.
iterations: number
Thenumberofiterations.Thisshouldbeaveryhighnumber.OWASPrecommended64000iterationsin2012andrecommends
doublingthatnumbereverytwoyears.
WhenusingPBKDF2forpasswordhashesitisalsorecommendedtoaddarandomvalue(typicallybetween0and32000)tothat
numberthatisdifferentforeachuser.
keyLength: number
Thekeylength.
hmac
crypto.hmac(key,message,algorithm):string
GeneratesanHM AChashofthegivenmessage.
Arguments
key: string
Thecryptographickeytousetohashthemessage.
message: string
Themessagetohash.
algorithm: string
Thenameofthealgorithmtouse.
crypto
479
FilesystemModule
require('fs')
TheimplementationtriestofollowtheCommonJSspecificationwherepossible.Filesystem/A/0.
SingleFileDirectoryManipulation
exists
checksifafileofanytypeordirectoryexists fs.exists(path)
Returnstrueifafile(ofanytype)oradirectoryexistsatagivenpath.Ifthefileisabrokensymboliclink,returnsfalse.
isFile
testsifpathisafile fs.isFile(path)
Returnstrueifthepathpointstoafile.
isDirectory
testsifpathisadirectory fs.isDirectory(path)
Returnstrueifthepathpointstoadirectory.
size
getsthesizeofafile fs.size(path)
Returnsthesizeofthefilespecifiedbypath.
mtime
getsthelastmodificationtimeofafile fs.mtime(filename)
Returnsthelastmodificationdateofthespecifiedfile.ThedateisreturnedasaUnixtimestamp(numberofsecondselapsedsince
January11970).
pathSeparator
fs.pathSeparator
Ifyouwanttocombinetwopathsyoucanusefs.pathSeparatorinsteadof/or\.
join
fs.join(path,filename)
Thefunctionreturnsthecombinationofthepathandfilename,e.g.fs.join(Hello/World,foo.bar)wouldreturnHello/World/foo.bar.
getTempFile
returnsthenamefora(new)temporaryfile fs.getTempFile(directory,createFile)
Returnsthenameforanewtemporaryfileindirectorydirectory.IfcreateFileistrue,anemptyfilewillbecreatedsonootherprocess
cancreateafileofthesamename.
Note:Thedirectorydirectorymustexist.
fs
480
getTempPath
returnsthetemporarydirectory fs.getTempPath()
Returnstheabsolutepathofthetemporarydirectory
makeAbsolute
makesagivenpathabsolute fs.makeAbsolute(path)
Returnsthegivenstringifitisanabsolutepath,otherwiseanabsolutepathtothesamelocationisreturned.
chmod
setsfilepermissionsofspecifiedfiles(nonwindowsonly) fs.exists(path)
Returnstrueonsuccess.
list
returnsthedirectorylisting fs.list(path)
Thefunctionsreturnsthenamesofallthefilesinadirectory,inlexicallysortedorder.Throwsanexceptionifthedirectorycannotbe
traversed(orpathisnotadirectory).
Note:thismeansthatlist("x")ofadirectorycontaining"a"and"b"wouldreturn["a","b"],not["x/a","x/b"].
listTree
returnsthedirectorytree fs.listTree(path)
Thefunctionreturnsanarraythatstartswiththegivenpath,andallofthepathsrelativetothegivenpath,discoveredbyadepthfirst
traversalofeverydirectoryinanyvisiteddirectory,reportingbutnottraversingsymboliclinkstodirectories.Thefirstpathisalways
"",thepathrelativetoitself.
makeDirectory
createsadirectory fs.makeDirectory(path)
Createsthedirectoryspecifiedbypath.
makeDirectoryRecursive
createsadirectory fs.makeDirectoryRecursive(path)
Createsthedirectoryhierarchyspecifiedbypath.
remove
removesafile fs.remove(filename)
Removesthefilefilenameatthegivenpath.Throwsanexceptionifthepathcorrespondstoanythingthatisnotafileorasymboliclink.
If"path"referstoasymboliclink,removesthesymboliclink.
removeDirectory
removesanemptydirectory fs.removeDirectory(path)
Removesadirectoryifitisempty.Throwsanexceptionifthepathisnotanemptydirectory.
removeDirectoryRecursive
fs
481
removesadirectory fs.removeDirectoryRecursive(path)
Removesadirectorywithallsubelements.Throwsanexceptionifthepathisnotadirectory.
FileIO
read
readsinafile fs.read(filename)
Readsinafileandreturnsthecontentasstring.PleasenotethatthefilecontentmustbeencodedinUTF-8.
read64
readsinafileasbase64 fs.read64(filename)
Readsinafileandreturnsthecontentasstring.ThefilecontentisBase64encoded.
readBuffer
readsinafile fs.readBuffer(filename)
ReadsinafileandreturnsitscontentinaBufferobject.
readFileSync
fs.readFileSync(filename,encoding)
Readsthecontentsofthefilespecifiedin filename.If encodingisspecified,thefilecontentswillbereturnedasastring.Supported
encodingsare:
utf8or utf-8
ascii
base64
ucs2or ucs-2
utf16leor utf16be
hex
Ifno encodingisspecified,thefilecontentswillbereturnedinaBufferobject.
write
fs.write(filename,content)
Writesthecontentintoafile.ContentcanbeastringoraBufferobject.Ifthefilealreadyexists,itistruncated.
writeFileSync
fs.writeFileSync(filename,content)
Thisisanaliasfor fs.write(filename,content).
append
fs.append(filename,content)
Writesthecontentintoafile.ContentcanbeastringoraBufferobject.Ifthefilealreadyexists,thecontentisappendedattheend.
RecursiveManipulation
copyRecursive
fs
482
copiesadirectorystructure fs.copyRecursive(source,destination)
Copiessourcetodestination.Exceptionswillbethrownon:
Failuretocopythefile
specifyingadirectoryfordestinationwhensourceisafile
specifyingadirectoryassourceanddestination
CopyFile
copiesafileintoatargetfile fs.copyFile(source,destination)
Copiessourcetodestination.IfDestinationisadirectory,afileofthesamenamewillbecreatedinthatdirectory,elsethecopywillget
thespecifiedfilename.
move
renamesafile fs.move(source,destination)
M ovessourcetodestination.Failuretomovethefile,orspecifyingadirectoryfordestinationwhensourceisafilewillthrowan
exception.Likewise,specifyingadirectoryassourceanddestinationwillfail.
ZIP
unzipFile
unzipsafile fs.unzipFile(filename,outpath,skipPaths,overwrite,password)
Unzipsthezipfilespecifiedbyfilenameintothepathspecifiedbyoutpath.Overwritesanyexistingtargetfilesifoverwriteissettotrue.
Returnstrueifthefilewasunzippedsuccessfully.
zipFile
zipsafile fs.zipFile(filename,chdir,files,password)
Storesthefilesspecifiedbyfilesinthezipfilefilename.Ifthefilefilenamealreadyexists,anerroristhrown.Thelistofinputfilesfiles
mustbegivenasalistofabsolutefilenames.Ifchdirisnotempty,thechdirprefixwillbestrippedfromthefilenameinthezipfile,so
whenitisunzippedfilenameswillberelative.Specifyingapasswordisoptional.
Returnstrueifthefilewaszippedsuccessfully.
fs
483
Module"request"
TherequestmoduleprovidesthefunctionalityformakingHTTPrequests.
require('@arangodb/request')
MakingHTTPrequests
HTTPmethodhelpers
InadditiontotherequestfunctionconvenienceshorthandsareavailableforeachHTTPmethodintheformof,i.e.:
request.head(url,options)
request.get(url,options)
request.post(url,options)
request.put(url,options)
request.delete(url,options)
request.patch(url,options)
Theseareequivalenttousingtherequestfunctiondirectly,i.e.:
request[method](url,options)
//isequivalentto
request({method,url,...options});
Forexample:
constrequest=require('@arangodb/request');
request.get('http://localhost',{headers:{'x-session-id':'keyboardcat'}});
//isequivalentto
request({
method:'get',
url:'http://localhost',
headers:{'x-session-id':'keyboardcat'}
});
Therequestfunction
TherequestfunctioncanbeusedtomakeHTTPrequests.
request(options)
PerformsanHTTPrequestandreturnsaResponseobject.
Parameter
Therequestfunctiontakesthefollowingoptions:
urloruri:thefully-qualifiedURLoraparsedURLfrom url.parse.
qs(optional):objectcontainingquerystringvaluestobeappendedtotheURL.
useQuerystring:if true,use querystringmoduletohandlequerystrings,otherwiseuse qsmodule.Default: false.
method(optional):HTTPmethod(case-insensitive).Default: "GET".
headers(optional):HTTPheaders(case-insensitive).Default: {}.
body(optional):requestbody.M ustbeastringor Buffer,oraJSONserializablevalueifjsonis true.
json:if true,bodywillbeserializedtoaJSONstringandtheContent-Typeheaderwillbesetto "application/json".
AdditionallytheresponsebodywillalsobeparsedasJSON(unlessencodingissetto null).Default: false.
form(optional):whensettoastringorobjectandnobodyhasbeenset,bodywillbesettoaquerystringrepresentationofthatvalue
andtheContent-Typeheaderwillbesetto "application/x-www-form-urlencoded".AlsoseeuseQuerystring.
auth(optional):anobjectwiththepropertiesusernameandpasswordforHTTPBasicauthenticationorthepropertybearerfor
request
484
HTTPBearertokenauthentication.
followRedirect:whetherHTTP3xxredirectsshouldbefollowed.Default: true.
maxRedirects:themaximumnumberofredirectstofollow.Default: 10.
encoding:encodingtobeusedfortheresponsebody.Ifsetto null,theresponsebodywillbereturnedasa Buffer.Default:
"utf-8".
timeout:numberofmillisecondstowaitforaresponsebeforeabortingtherequest.
returnBodyOnError:whethertheresponsebodyshouldbereturnedevenwhentheserverresponseindicatesanerror.Default:
true.
ThefunctionreturnsaResponseobjectwiththefollowingproperties:
rawBody:therawresponsebodyasa Buffer.
body:theparsedresponsebody.Ifencodingwasnotsetto null,thisisastring.Ifadditionallyjsonwassetto trueandthe
responsebodyiswell-formedJSON,thisistheparsedJSONdata.
headers:anobjectcontainingtheresponseheaders.OtherwisethisisidenticaltorawBody.
statusCodeandstatus:theHTTPstatuscodeoftheresponse,e.g. 404.
message:theHTTPstatusmessageoftheresponse,e.g. NotFound.
Forms
Therequestmodulesupports application/x-www-form-urlencoded(URLencoded)formuploads:
varres=request.post('http://service.example/upload',{form:{key:'value'}});
//or
varres=request.post({url:'http://service.example/upload',form:{key:'value'}});
//or
varres=request({
method:'post',
url:'http://service.example/upload',
form:{key:'value'}
});
Formdatawillbeencodedusingtheqsmodulebydefault.
Ifyouwanttousethequerystringmoduleinstead,simplyusetheuseQuerystringoption.
JSON
IfyouwanttosubmitJSON-serializablevaluesasrequestbodies,justsetthejsonoption:
varres=request.post('http://service.example/notify',{body:{key:'value'},json:true});
//or
varres=request.post({url:'http://service.example/notify',body:{key:'value'},json:true});
//or
varres=request({
method:'post',
url:'http://service.example/notify',
body:{key:'value'},
json:true
});
HTTPauthentication
TherequestmodulesupportsbothHTTPBasicauthentication.Justpassthecredentialsviatheauthoption:
request
485
varres=request.get(
'http://service.example/secret',
{auth:{username:'jcd',password:'bionicman'}}
);
//or
varres=request.get({
url:'http://service.example/secret',
auth:{username:'jcd',password:'bionicman'}
});
//or
varres=request({
method:'get',
auth:{username:'jcd',password:'bionicman'}
});
AlternativelyyoucansupplythecredentialsviatheURL:
varusername='jcd';
varpassword='bionicman';
varres=request.get(
'http://'+
encodeURIComponent(username)+
':'+
encodeURIComponent(password)+
'@service.example/secret'
);
YoucanalsouseBearertokenauthentication:
varres=request.get(
'http://service.example/secret',
{auth:{bearer:'keyboardcat'}}
);
//or
varres=request.get({
auth:{bearer:'keyboardcat'}
});
//or
varres=request({
method:'get',
auth:{bearer:'keyboardcat'}
});
request
486
Module"actions"
Theactionmoduleprovidestheinfrastructurefordefininglow-levelHTTPactions.
IfyouwanttodefineHTTPendpointsinArangoDByoushouldprobablyusetheFoxxmicroserviceframeworkinstead.
Basics
Errormessage
actions.getErrorMessage(code)
Returnstheerrormessageforanerrorcode.
StandardHTTPResultGenerators
actions.defineHttp(options)
Definesanewaction.Theoptionsareasfollows:
options.url
TheURL,whichcanbeusedtoaccesstheaction.Thispathmightcontainslashes.Notethatthisactionwillalsobecalled,ifaurlis
givensuchthatoptions.urlisaprefixofthegivenurlandnolongerdefinitionmatches.
options.prefix
Iffalse,thenonlyusetheactionforexactmatches.Thedefaultistrue.
options.callback(request,response)
Therequestargumentcontainsadescriptionoftherequest.Arequestparameterfooisaccessibleasrequest.parametrs.foo.Arequest
headerbarisaccessibleasrequest.headers.bar.Assumethattheactionisdefinedfortheurl/foo/barandtherequesturlis
/foo/bar/hugo/egon.Thenthesuffixparts["hugo","egon"]areavailibleinrequest.suffix.
Thecallbackmustdefinefilltheresponse.
response.responseCode:theresponsecode
response.contentType:thecontenttypeoftheresponse
response.body:thebodyoftheresponse
YoucanusethefunctionsResultOkandResultErrortoeasilygeneratearesponse.
Resultok
actions.resultOk(req,res,code,result,headers)
Thefunctiondefinesaresponse.codeisthestatuscodetoreturn.resultistheresultobject,whichwillbereturnedasJSONobjectinthe
body.headersisanarrayofheaderstoreturned.Thefunctionaddstheattributeerrorwithvaluefalseandcodewithvaluecodetothe
result.
Resultbad
actions.resultBad(req,res,error-code,msg,headers)
Thefunctiongeneratesanerrorresponse.
Resultnotfound
actions.resultNotFound(req,res,code,msg,headers)
actions
487
Resultunsupported
actions.resultUnsupported(req,res,headers)
Resulterror
actions.resultError(req,res,code,errorNum,errorM essage,headers,keyvals)*
Thefunctiongeneratesanerrorresponse.TheresponsebodyisanarraywithanattributeerrorMessagecontainingtheerrormessage
errorMessage,errorcontainingtrue,codecontainingcode,errorNumcontainingerrorNum,anderrorMessagecontainingtheerror
messageerrorMessage.keyvalsaremixedintotheresult.
ResultnotImplemented
actions.resultNotImplemented(req,res,msg,headers)
Resultpermanentredirect
actions.resultPermanentRedirect(req,res,options,headers)
Thefunctiongeneratesaredirectresponse.
Resulttemporaryredirect
actions.resultTemporaryRedirect(req,res,options,headers)
Thefunctiongeneratesaredirectresponse.
ArangoDBResultGenerators
Collectionnotfound
actions.collectionNotFound(req,res,collection,headers)
Indexnotfound
actions.indexNotFound(req,res,collection,index,headers)
Resultexception
actions.resultException(req,res,err,headers,verbose)
Thefunctiongeneratesanerrorresponse.If@FA{verbose}issettotrueornotspecified(thedefault),thentheerrorstacktracewillbe
includedintheerrormessageifavailable.If@FA{verbose}isastringitwillbeprependedbeforetheerrormessageandthestacktrace
willalsobeincluded.
actions
488
Module"queries"
ThequerymoduleprovidestheinfrastructureforworkingwithcurrentlyrunningAQLqueriesviaarangosh.
Properties
queries.properties()Returnstheserverscurrentquerytrackingconfiguration;wechangetheslowquerythresholdtogetbetterresults:
arangosh>varqueries=require("@arangodb/aql/queries");
arangosh>queries.properties();
arangosh>queries.properties({slowQueryThreshold:1});
Currentlyrunningqueries
Wecreateataskthatspawnsqueries,sowehaveniceoutput.Sincethistaskusesresources,youmaywanttoincrease period(andnot
forgettoremoveit...afterwards):
arangosh>vartheQuery='FORsleepLoooongIN1..5LETsleepLoooonger=SLEEP(1000)RETURNsleepLoooong'
arangosh>vartasks=require("@arangodb/tasks");
arangosh>tasks.register({
........>id:"mytask-1",
........>name:"thisisasampletasktospawnaslowaqlquery",
........>command:"require('@arangodb').db._query('"+theQuery+"');"
........>});
arangosh>queries.current();
ThefunctionreturnsthecurrentlyrunningAQLqueriesasanarray.
Slowqueries
ThefunctionreturnsthelastAQLqueriesthatexceededtheslowquerythresholdasanarray:
arangosh>queries.slow();
[]
Clearslowqueries
ClearthelistofslowAQLqueries:
arangosh>queries.clearSlow();
{
"code":200
}
arangosh>queries.slow();
[]
Kill
KillarunningAQLquery:
queries
489
arangosh>varrunningQueries=queries.current().filter(function(query){
........>returnquery.query===theQuery;
........>});
arangosh>queries.kill(runningQueries[0].id);
{
"code":200
}
queries
490
Write-aheadlog
Thismoduleprovidesfunctionalityforadministeringthewrite-aheadlogs.
Configuration
retrievestheconfigurationofthewrite-aheadlog internal.wal.properties()
Retrievestheconfigurationofthewrite-aheadlog.TheresultisaJSONarraywiththefollowingattributes:
allowOversizeEntries:whetherornotoperationsthatarebiggerthanasinglelogfilecanbeexecutedandstored
logfileSize:thesizeofeachwrite-aheadlogfile
historicLogfiles:themaximumnumberofhistoriclogfilestokeep
reserveLogfiles:themaximumnumberofreservelogfilesthatArangoDBallocatesinthebackground
syncInterval:theintervalforautomaticsynchronizationofnot-yetsynchronizedwrite-aheadlogdata(inmilliseconds)
throttleWait:themaximumwaittimethatoperationswillwaitbeforetheygetabortedifcaseofwrite-throttling(inmilliseconds)
throttleWhenPending:thenumberofunprocessedgarbage-collectionoperationsthat,whenreached,willactivatewrite-throttling.A
valueof0meansthatwrite-throttlingwillnotbetriggered.
Examples
arangosh>require("internal").wal.properties();
configuresthewrite-aheadlog internal.wal.properties(properties)
Configuresthebehaviorofthewrite-aheadlog.propertiesmustbeaJSONJSONobjectwiththefollowingattributes:
allowOversizeEntries:whetherornotoperationsthatarebiggerthanasinglelogfilecanbeexecutedandstored
logfileSize:thesizeofeachwrite-aheadlogfile
historicLogfiles:themaximumnumberofhistoriclogfilestokeep
reserveLogfiles:themaximumnumberofreservelogfilesthatArangoDBallocatesinthebackground
throttleWait:themaximumwaittimethatoperationswillwaitbeforetheygetabortedifcaseofwrite-throttling(inmilliseconds)
throttleWhenPending:thenumberofunprocessedgarbage-collectionoperationsthat,whenreached,willactivatewrite-throttling.A
valueof0meansthatwrite-throttlingwillnotbetriggered.
Specifyinganyoftheaboveattributesisoptional.Notspecifiedattributeswillbeignoredandtheconfigurationforthemwillnotbe
modified.
Examples
arangosh>require("internal").wal.properties({
........>allowOverSizeEntries:true,
........>logfileSize:32*1024*1024});
Flushing
flushesthecurrentlyopenWALlogfile internal.wal.flush(waitForSync,waitForCollector)
Flushesthewrite-aheadlog.Byflushingthecurrentlyactivewrite-aheadlogfile,thedatainitcanbetransferredtocollectionjournalsand
datafiles.Thisisusefultoensurethatalldataforacollectionispresentinthecollectionjournalsanddatafiles,forexample,when
dumpingthedataofacollection.
ThewaitForSyncoptiondetermineswhetherornottheoperationshouldblockuntilthenot-yetsynchronizeddatainthewrite-aheadlog
wassynchronizedtodisk.
Write-aheadlog
491
ThewaitForCollectoroperationcanbeusedtospecifythattheoperationshouldblockuntilthedataintheflushedloghasbeencollected
bythewrite-aheadloggarbagecollector.Notethatsettingthisoptiontotruemightblockforalongtimeiftherearelong-running
transactionsandthewrite-aheadloggarbagecollectorcannotfinishgarbagecollection.
Examples
arangosh>require("internal").wal.flush();
Write-aheadlog
492
TaskManagement
IntroductiontoTaskManagementinArangoDB
ArangoDBcanexecuteuser-definedJavaScriptfunctionsasone-shotorperiodictasks.Thisfunctionalitycanbeusedtoimplement
timedorrecurringjobsinthedatabase.
TasksinArangoDBconsistofaJavaScriptsnippetorfunctionthatisexecutedwhenthetaskisscheduled.Ataskcanbeaone-shottask
(meaningitisrunonceandnotrepeated)oraperiodictask(meaningthatitisre-scheduledaftereachexecution).Taskscanhaveoptional
parameters,whicharedefinedattasksetuptime.Theparametersspecifiedattasksetuptimewillbepassedasargumentstothetask
wheneveritgetsexecuted.PeriodicTaskshaveanexecutionfrequencythatneedstobespecifiedwhenthetaskissetup.One-shottasks
haveaconfigurabledelayafterwhichthey'llgetexecuted.
Taskswillbeexecutedontheservertheyhavebeensetupon.Taskswillnotbeshippedaroundinacluster.Ataskwillbeexecutedin
thecontextofthedatabaseitwascreatedin.However,whendroppingadatabase,anytasksthatwerecreatedinthecontextofthis
databasewillremainactive.Itisthereforesensibletofirstunregisterallactivetasksforadatabasebeforedroppingthedatabase.
TasksregisteredinArangoDBwillbeexecuteduntiltheservergetsshutdownorrestarted.Afterarestartoftheserver,anyuser-defined
one-shotorperiodictaskswillbelost.
CommandsforWorkingwithTasks
ArangoDBprovidesthefollowingcommandsforworkingwithtasks.Allcommandscanbeaccessedviathetasksmodule,whichcanbe
loadedlikethis:
require("@arangodb/tasks")
PleasenotethatthetasksmoduleisavailableinsidetheArangoDBserveronly.ItcannotbeusedfromtheArangoShellorArangoDB's
webinterface.
Registeratask
Toregisteratask,theJavaScriptsnippetorfunctionneedstobespecifiedinadditiontotheexecutionfrequency.Optionally,ataskcan
haveanidandaname.Ifnoidisspecified,itwillbeauto-assignedforanewtask.Thetaskidisalsothemeanstoaccessorunregistera
tasklater.Tasknamesareinformationalonly.Theycanbeusedtomakeataskdistinguishablefromothertasksalsorunningontheserver.
Thefollowingserver-sidecommandsregisteratask.ThecommandtobeexecutedisaJavaScriptstringsnippetwhichprintsamessageto
theserver'slogfile:
consttasks=require("@arangodb/tasks");
tasks.register({
id:"mytask-1",
name:"thisisasnippettask",
period:15,
command:"require('console').log('hellofromsnippettask');"
});
Theabovehasregisterataskwithidmytask-1,whichwillbeexecutedevery15secondsontheserver.Thetaskwillwritealogmessage
wheneveritisinvoked.
TaskscanalsobesetupusingaJavaScriptcallbackfunctionlikethis:
TaskM anagement
493
tasks.register({
id:"mytask-2",
name:"thisisafunctiontask",
period:15,
command:function(){
require('console').log('hellofromfunctiontask');
}
});
Itisimportanttonotethatthecallbackfunctionislateboundandwillbeexecutedinadifferentcontextthaninthecreationcontext.The
callbackfunctionmustthereforenotaccessanyvariablesdefinedoutsideofitsownscope.Thecallbackfunctioncanstilldefineanduse
itsownvariables.
Topassparameterstoatask,theparamsattributecanbesetwhenregisteringatask.Notethattheparametersarelimitedtodatatypes
usableinJSON(meaningnocallbackfunctionscanbepassedasparametersintoatask):
tasks.register({
id:"mytask-3",
name:"thisisaparametertask",
period:15,
vargreeting=params.greeting;
vardata=JSON.stringify(params.data);
require('console').log('%sfromparametertask:%s',greeting,data);
},
params:{greeting:"hi",data:"howareyou?"}
});
Registeringaone-shottaskworksthesameway,exceptthattheperiodattributemustbeomitted.Ifperiodisomitted,thenthetaskwill
beexecutedjustonce.Thetaskinvocationdelaycanoptionallybespecifiedwiththeoffsetattribute:
tasks.register({
id:"mytask-once",
name:"thisisaone-shottask",
offset:10,
require('console').log('youwillseemejustonce!');
}
});
Note:Whenspecifyinganoffsetvalueof0,ArangoDBwillinternallyaddaverysmallvaluetotheoffsetsowillbeslightlygreaterthan
zero.
Note:Normally offsetcannotbegreaterthan interval.Theonlywaytoset offsetgreaterthan intervalistoregisteranonperiodictimerwiththespecifiedoffsetandwhenthisfiresthiscaninstallaperiodictimerwithanoffsetof0.
Unregisteratask
Afterataskhasbeenregistered,itcanbeunregisteredusingitsid:
tasks.unregister("mytask-1");
Notethatunregisteringanon-existingtaskwillthrowanexception.
Listalltasks
Togetanoverviewofwhichtasksareregistered,thereisthegetmethod.Ifthegetmethodiscalledwithoutanyarguments,itwillreturn
anarrayofalltasks:
TaskM anagement
494
tasks.get();
Ifgetiscalledwithataskidargument,itwillreturninformationaboutthisparticulartask:
tasks.get("mytask-3");
Thecreatedattributeofataskrevealswhenataskwascreated.ItisreturnedasaUnixtimestamp.
TaskM anagement
495
Deprecated
Deprecated
496
SimpleQueries
Simplequeriescanbeusedifthequeryconditionisstraightforward,i.e.,adocumentreference,alldocuments,aquery-by-example,ora
simplegeoquery.Inasimplequeryyoucanspecifyexactlyonecollectionandonequerycriteria.Inthefollowingsectionswedescribe
theJavaScriptshellinterfaceforsimplequeries,whichyoucanusewithintheArangoDBshellandwithinactionsandtransactions.For
otherlanguagesseethecorrespondinglanguageAPIdocumentation.
YoucanfindalistofqueriesatCollectionM ethods.
SimpleQueries
497
SequentialAccessandCursors
Ifaqueryreturnsacursor,thenyoucanusehasNextandnexttoiterateovertheresultsetortoArraytoconvertittoanarray.
Ifthenumberofqueryresultsisexpectedtobebig,itispossibletolimittheamountofdocumentstransferredbetweentheserverandthe
clienttoaspecificvalue.ThisvalueiscalledbatchSize.ThebatchSizecanoptionallybesetbeforeorwhenasimplequeryisexecuted.If
theserverhasmoredocumentsthanshouldbereturnedinasinglebatch,theserverwillsetthehasMoreattributeintheresult.Itwillalso
returntheidoftheserver-sidecursorintheidattributeintheresult.ThisidcanbeusedwiththecursorAPItofetchanyoutstanding
resultsfromtheserveranddisposetheserver-sidecursorafterwards.
TheinitialbatchSizevaluecanbesetusingthesetBatchSizemethodthatisavailableforeachtypeofsimplequery,orwhenthesimple
queryisexecutedusingitsexecutemethod.IfnobatchSizevalueisspecified,theserverwillpickareasonabledefaultvalue.
HasNext
checksifthecursorisexhausted cursor.hasNext()
ThehasNextoperatorreturnstrue,thenthecursorstillhasdocuments.Inthiscasethenextdocumentcanbeaccessedusingthenext
operator,whichwilladvancethecursor.
Examples
arangosh>vara=db.five.all();
arangosh>while(a.hasNext())print(a.next());
Next
returnsthenextresultdocument cursor.next()
IfthehasNextoperatorreturnstrue,thentheunderlyingcursorofthesimplequerystillhasdocuments.Inthiscasethenextdocument
canbeaccessedusingthenextoperator,whichwilladvancetheunderlyingcursor.Ifyouusenextonanexhaustedcursor,thenundefined
isreturned.
Examples
arangosh>db.five.all().next();
SetBatchsize
setsthebatchsizeforanyfollowingrequests cursor.setBatchSize(number)
Setsthebatchsizeforqueries.Thebatchsizedetermineshowmanyresultsareatmosttransferredfromtheservertotheclientinone
chunk.
GetBatchsize
returnsthebatchsize cursor.getBatchSize()
Returnsthebatchsizeforqueries.Ifthereturnedvalueisundefined,theserverwilldetermineasensiblebatchsizeforanyfollowing
requests.
ExecuteQuery
executesaquery query.execute(batchSize)
SimpleQueries
498
Executesasimplequery.IftheoptionalbatchSizevalueisspecified,theserverwillreturnatmostbatchSizevaluesinoneroundtrip.The
batchSizecannotbeadjustedafterthequeryisfirstexecuted.
Note:Thereisnoneedtoexplicitlycalltheexecutemethodifanothermeansoffetchingthequeryresultsischosen.Thefollowingtwo
approachesleadtothesameresult:
arangosh>result=db.users.all().toArray();
arangosh>q=db.users.all();q.execute();result=[];while(q.hasNext()){result.push(q.next());}
ThefollowingtwoalternativesbothuseabatchSizeandreturnthesameresult:
arangosh>q=db.users.all();q.setBatchSize(20);q.execute();while(q.hasNext()){print(q.next());}
arangosh>q=db.users.all();q.execute(20);while(q.hasNext()){print(q.next());}
Dispose
disposestheresult cursor.dispose()
Ifyouarenolongerinterestedinanyfurtherresults,youshouldcalldisposeinordertofreeanyresourcesassociatedwiththecursor.
Aftercallingdisposeyoucannolongeraccessthecursor.
Count
countsthenumberofdocuments cursor.count()
Thecountoperatorcountsthenumberofdocumentintheresultsetandreturnsthatnumber.Thecountoperatorignoresanylimitsand
returnsthetotalnumberofdocumentsfound.
Note:Notallsimplequeriessupportcounting.Inthiscasenullisreturned.
cursor.count(true)
Iftheresultsetwaslimitedbythelimitoperatorordocumentswereskipedusingtheskipoperator,thecountoperatorwithargument
truewillusethenumberofelementsinthefinalresultset-afterapplyinglimitandskip.
Note:Notallsimplequeriessupportcounting.Inthiscasenullisreturned.
Examples
Ignoreanylimit:
arangosh>db.five.all().limit(2).count();
null
Countinganylimitorskip:
arangosh>db.five.all().limit(2).count(true);
2
SimpleQueries
499
Pagination
If,forexample,youdisplaytheresultofausersearch,thenyouareingeneralnotinterestedinthecompletedresultset,butonlythefirst
10orsodocuments.Ormaybethenext10documentsforthesecondpage.Inthiscase,youcantheskipandlimitoperators.These
operatorsworklikeLIM ITinM ySQL.
skipusedtogetherwithlimitcanbeusedtoimplementpagination.Theskipoperatorskipsoverthefirstndocuments.So,inorderto
createresultpageswith10resultdocumentsperpage,youcanuseskip(n*10).limit(10)toaccessthe10documentsonthenthpage.
Thisresultshouldbesorted,sothatthepaginationworksinapredicableway.
Limit
limit query.limit(number)
Limitsaresulttothefirstnumberdocuments.Specifyingalimitof0willreturnnodocumentsatall.Ifyoudonotneedalimit,justdo
notaddthelimitoperator.Thelimitmustbenon-negative.
Ingeneraltheinputtolimitshouldbesorted.Otherwiseitwillbeunclearwhichdocumentswillbeincludedintheresultset.
Examples
arangosh>db.five.all().limit(2).toArray();
Skip
skip query.skip(number)
Skipsthefirstnumberdocuments.Ifnumberispositive,thenthisnumberofdocumentsareskippedbeforereturningthequeryresults.
Ingeneraltheinputtoskipshouldbesorted.Otherwiseitwillbeunclearwhichdocumentswillbeincludedintheresultset.
Note:usingnegativeskipvaluesisdeprecatedasofArangoDB2.6andwillnotbesupportedinfutureversionsofArangoDB.
Examples
arangosh>db.five.all().skip(3).toArray();
SimpleQueries
500
ModificationQueries
ArangoDBalsoallowsremoving,replacing,andupdatingdocumentsbasedonanexampledocument.Everydocumentinthecollection
willbecomparedagainstthespecifiedexampledocumentandbedeleted/replaced/updatedifallattributesmatch.
Thesemethodshouldbeusedwithcautionastheyareintendedtoremoveormodifylotsofdocumentsinacollection.
Allmethodscanoptionallyberestrictedtoaspecificnumberofoperations.However,ifalimitisspecificbutislessthanthenumberof
matches,itwillbeundefinedwhichofthematchingdocumentswillgetremoved/mo

ArangoDB v3.0.8 Documentation

Transcription

Similar documents

Flemish String Board Attachment

The University of Malaya Library

celebrity icons one-sheet

Quick Intro to SSRS for New WINGAP SQL users

Kuhl-Linscomb - Haley Shapley

PDF Document maunal_for_airmatic_capper

Photo search by face positions and facial attributes on

Fred Bahnson Amity Gaige Brian Turner Jim Shepard First

Creating an Ubuntu scope