Nginx Variables

Transcription

Nginx Variables

agentzh'sNginxTutorials(version2015.03.19)
TableofContents
Foreword
WritingPlanfortheTutorials
NginxVariables(01)
NginxVariables(02)
NginxVariables(03)
NginxVariables(04)
NginxVariables(05)
NginxVariables(06)
NginxVariables(07)
NginxVariables(08)
NginxDirectiveExecutionOrder(01)
Foreword
I'vebeendoingalotofworkintheNginxworldoverthelastfewyearsandI'vealsobeenthinkingaboutwritingaseriesoftutorial-likearticlestoexplaintomore
peoplewhatI'vedoneandwhatI'velearnedinthisarea.NowIhavefinallydecidedtopostserialarticlestotheSinaBloghttp://blog.sina.com.cn/openrestyin
Chinese.Everyarticlewillroughlycoverasingletopicandwillbeinarathercasualstyle.ButatsomepointinthefutureImayrestructurethearticlesandtheirstyle
inordertoturnthemintoa"real"book.
Thearticlesaredividedintoseries.Forexample,thefirstseriesis"NginxVariables".EachseriescanbethoughtofasmappingtoachapterintheNginxbookthatI
maypublishinthefuture.
ThearticlesareintendedforNginxusersofallexperiencelevels,includinguserswithextensiveApacheandLighttpdexperiencewhomayhaveneverusedNginx
before.
TheexamplesinthearticlesareatleastcompatiblewithNginx0.8.54.DonottrytheexampleswitholderversionsofNginx.ThelateststableversionofNginxasof
thiswritingis1.7.9.
AlloftheNginxmodulesreferencedinthearticlesareproduction-ready.IwillnotbecoveringanyNginxcoremodulesthatareeitherexperimentalorbuggy.
Additionally,Iwillbemakingextensiveuseof3rd-partyNginxmodulesintheexamples.Ifit'sinconvenientforyoutodownloadandinstalltheindividualmodules
oneatatimethenIhighlyrecommendthatyoudownloadandinstallthengx_openrestysoftwarebundlethatImaintain.
http://openresty.org/
Allofthemodulesreferencedinthearticles,includingthecoreNginxmodulesthatarenew(butstable),areincludedintheOpenRestybundle.
AprinciplethatIwillbetryingtoadheretoistousesmallconciseexamplestoexplainandvalidatetheconceptsandbehaviorsbeingdescribed.M yhopeisthatit
willhelpthereadertodevelopthegoodhabitofnotacceptingothers'viewpointsorstatementsatfacevaluewithouttestingthemfirst.Thisapproachmayhave
somethingtodowithmyQAbackground.Infact,Ikeeptweakingandcorrectingthearticlesbasedontheresultsofrunningtheexampleswhilewriting.
Theexamplesinthearticlesfallintooneoftwocategories,goodandproblematic.Thepurposeoftheproblematicexamplesistohighlightpotentialpitfallsandother
areaswhereNginxoritsmodulesbehaveinwaysthatreadersmaynotexpect.Problematicexamplesareeasytoidentifybecauseeachlineoftextintheexamplewill
beprefixedwithaquestionmark,i.e.,"?".Hereisanexample:
?server{
?listen8080;
?
?location/bad{
?echo$foo;
?}
?}
Donotreproducethesearticleswithoutexplicitpermissionsfromus.Copyrightreserved.
Iencouragereaderstosendfeedback([email protected]),especiallyconstructivecriticism.
ThesourceforallthearticlesisonGitHub:
http://github.com/agentzh/nginx-tutorials/
Thesourcefilesareundertheen/directory.IamusingalittlemarkuplanguagethatisamixtureofWikiandPODtowritethesearticles.Theyarethe.tutfiles.
Youarewelcometocreateforksand/orprovidepatches.
Thee-booksfilesthataresuitableforcellphones,Kindle,iPad/iPhone,SonyReaders,andotherdevicescanbedownloadedfromhere:
http://openresty.org/#eBooks
SpecialthanksgotoKaiWu(kai10k)whokindlytranslatesthesearticlestoEnglish.
agentzhathomeintheFuzhoucity
October30,2011
WritingPlanfortheTutorials
Hereliststhetutorialseriesthathavealreadybeenpublishedortobepublished.
GettingStartedwithNginx
HowNginxM atchesURIs
NginxVariables
NginxDirectiveExecutionOrder
Nginx'sifisEvil
NginxSubrequests
NginxStaticFileServices
NginxLogServices
ApplicationGatewaysbasedonNginx
Reverse-ProxiesbasedonNginx
NginxandM emcached
NginxandRedis
NginxandM ySQL
NginxandPostgreSQL
ApplicationcachingBasedonNginx
SecurityandAccessControlinNginx
WebServicesBasedonNginx
AJAXApplicationsDrivenbyNginx
PerformanceTestingforNginxanditsApplications
StrengthoftheNginxCommunity
TheseriesnamescanroughlycorrespondtothechapternamesinmyfinalNginxbook,buttheyareunlikelytostayexactlythesame.Theactualseriesnamesmay
changeandtherelativeorderoftheseriesmaychangeaswell.
Thelistabovewillbeconstantlyupdatedtoalwaysreflectthelatestplan.
NginxVariables(01)
VariablesasValueContainers
Nginx'sconfigurationfilesuseamicroprogramminglanguage.M anyreal-worldNginxconfigurationfilesareessentiallysmallprograms.Thislanguage'sdesignis
heavilyinfluencedbyPerlandBourneShellasfarasIcansee,despitethefactthatitmightnotbeTuring-Completeanditisdeclarativeinmanyplaces.Thisisa
distinguishingfeatureofNginx,ascomparedtootherwebserverslikeApacheorLighttpd.Beingaprogramminglanguage,"variables"arethusanaturalpartofit
(exceptionsdoexist,ofcourse,asinpurefunctionallanguageslikeHaskell).
VariablesarejustcontainersholdingvariousvaluesinimperativelanguageslikePerl,BourneShell,andC/C++.
And"values"canbenumberslike3.14,stringslikehelloworld,orevencomplicatedthingslikereferences
toarraysorhashtablesinthoselanguages.FortheNginxconfigurationlanguage,however,variablescanhold
onlyonetypeofvalues,thatis,strings(thereisaninterestingexception:the3rd-partymodulengx_array_var
extendsNginxvariablestoholdarrays,butitisimplementedbyencodingaCpointerasabinarystringvalue
behindthescene).
Variablesarevaluecontainers
VariableSyntaxandInterpolation
Let'ssayournginx.confconfigurationfilehasthefollowingline:
set$a"helloworld";
Weassignavaluetothevariable$aviathesetconfigurationdirectivecomingfromthestandardngx_rewritemodule.Inparticular,weassignthestringvaluehello
worldto$a.
WecanseethattheNginxvariablenametakesadollarsign($)infrontofit.Thisisrequiredbythelanguagesyntax:wheneverwewanttoreferenceanNginxvariable
intheconfigurationfile,wemustadda$prefix.ThislooksveryfamiliartothosePerlandPHPprogrammers.
SuchvariableprefixmodifiersmaydiscomfortsomeJavaandC#programmers,thisnotationdoeshaveanobviousadvantagethough,thatis,variablescanbe
embeddeddirectlyintoastringliteral:
set$ahello;
set$b"$a,$a";
HereweusethevalueoftheexistingNginxvariable$atoconstructthevalueforthevariable$b.Soafterthesetwodirectivescompleteexecution,thevalueof$ais
hello,and$bishello,hello.Thistechniqueiscalled"variableinterpolation"inthePerlworld,whichmakesad-hocstringconcatenationoperatorsnolonger
thatnecessary.Let'susethesametermfortheNginxworldfromnowon.
Let'sseeanothercompleteexample:
server{
listen8080;
location/test{
set$foohello;
echo"foo:$foo";
}
}
Thisexampleomitsthehttpdirectiveandeventsconfigurationblocksintheouter-mostscopeforbrevity.Torequestthis/testinterfaceviacurl,anHTTP
clientutility,onthecommandline,weget
$curl'http://localhost:8080/test'
foo:hello
Hereweusetheechodirectiveofthe3rdpartymodulengx_echotoprintoutthevalueofthe$foovariableastheHTTPresponse.
Apparentlytheargumentsoftheechodirectivedoessupport"variableinterpolation",butwecannottakeitforgrantedforotherdirectives.Becausenotallthe
configurationdirectivessupport"variableinterpolation"anditisinfactuptotheimplementationofthedirectiveinthatmodule.Alwayslookupthedocumentation
tobesure.
Escaping"$"
We'vealreadylearnedthatthe$characterisspecialanditservesasthevariablenameprefix,butnowconsiderthatwewanttooutputaliteral$characterviatheecho
directive.Thefollowingnaiveexampledoesnotworkatall:
?:nginx
?location/t{
?echo"$";
?}
Wewillgetthefollowingerrormessagewhileloadingthisconfiguration:
[emerg]invalidvariablenamein...
ObviouslyNginxtriestoparse$"asavariablename.Isthereawaytoescape$inthestringliteral?Theansweris"no"(itisstillthecaseinthelatestNginxstable
release1.2.7)andIhavebeenhopingthatwecouldwritesomethinglike$$toobtainaliteral$.
Luckily,workaroundsdoexistandhereisoneproposedbyM aximDounin:firstweassigntoavariablealiteralstringcontainingadollarsigncharacterviaa
configurationdirectivethatdoesnotsupport"variableinterpolation"(rememberthatnotallthedirectivessupport"variableinterpolation"?),andthenreferencethis
variablelaterwheneverweneedadollarsign.Hereissuchanexampletodemonstratetheidea:
geo$dollar{
default"$";
}
server{
listen8080;
location/test{
echo"Thisisadollarsign:$dollar";
}
}
Let'stestitout:
Thisisadollarsign:$
Herewemakeuseofthegeodirectiveofthestandardmodulengx_geotoinitializethe$dollarvariablewiththestring"$",thereaftervariable$dollarcanbe
usedinplacesthatrequireadollarsign.Thisworksbecausethegeodirectivedoesnotsupport"variableinterpolation"atall.However,thengx_geomoduleis
originallydesignedtosetaNginxvariabletodifferentvaluesaccordingtotheremoteclientaddress,andinthisexample,wejustabuseittoinitializethe$dollar
variablewiththestring"$"unconditionally.
DisambiguatingVariableNames
Thereisaspecialcasefor"variableinterpolation",thatis,whenthevariablenameisfolloweddirectlybycharactersallowedinvariablenames(likeletters,digits,and
underscores).Insuchcases,wecanuseaspecialnotationtodisambiguatethevariablenamefromthesubsequentliteralcharacters,forinstance,
server{
listen8080;
location/test{
set$first"hello";
echo"${first}world";
}
}
Herethevariable$firstisconcatenatedwiththeliteralstringworld.Ifitwerewrittendirectlyas"$firstworld",Nginx's"variableinterpolation"engine(also
knownasthe"scriptengine")wouldtrytoaccessthevariable$firstworldinsteadof$first.Toresolvetheambiguityhere,curlybracesmustbeusedaround
thevariablename(excludingthe$prefix),asin${first}.Let'stestthissample:
$curl'http://localhost:8080/test
helloworld
VariableDeclarationandCreation
InlanguageslikeC/C++,variablesmustbedeclared(orcreated)beforetheycanbeusedsothatthecompilercanallocatestorageandperformtypecheckingat
compile-time.Similarly,NginxcreatesalltheNginxvariableswhileloadingtheconfigurationfile(orinotherwords,at"configurationtime"),thereforeNginxvariables
arealsorequiredtobedeclaredsomehow.
FortunatelythesetdirectiveandthegeodirectivementionedabovedohavethesideeffectofdeclaringorcreatingNginxvariablesthattheywillassignvaluestolater
at"requesttime".Ifwedonotdeclareavariablethiswayanduseitdirectlyin,say,theechodirective,wewillgetanerror.Forexample,
?server{
?listen8080;
?
?location/bad{
?echo$foo;
?}
?}
Herewedonotdeclarethe$foovariableandaccessitsvaluedirectlyinecho.Nginxwilljustrefuseloadingthisconfiguration:
[emerg]unknown"foo"variable
Yes,wecannotevenstarttheserver!
Nginxvariablecreationandassignmenthappenatcompletelydifferentphasesalongthetime-line.VariablecreationonlyoccurswhenNginxloadsitsconfiguration.On
theotherhand,variableassignmentoccurswhenrequestsareactuallybeingserved.ThisalsomeansthatwecannevercreatenewNginxvariablesat"requesttime".
VariableScope
OnceanNginxvariableiscreated,itisvisibletotheentireconfiguration,evenacrossdifferentvirtualserverconfigurationblocks,regardlessoftheplacesitisdeclared
at.Hereisanexample:
server{
listen8080;
location/foo{
echo"foo=[$foo]";
}
location/bar{
set$foo32;
echo"foo=[$foo]";
}
}
Herethevariable$fooiscreatedbythesetdirectivewithinlocation/bar,andthisvariableisvisibletotheentireconfiguration,thereforewecanreferenceitin
location/foowithoutworries.Belowistheresultoftestingthesetwointerfacesviathecurltool.
$curl'http://localhost:8080/foo'
foo=[]
$curl'http://localhost:8080/bar'
foo=[32]
foo=[]
Wecanseethattheassignmentoperationisonlyperformedinrequeststhataccesslocation/bar,sincethecorrespondingsetdirectiveisonlyusedinthat
location.Whenrequestingthe/foointerface,wealwaysgetanemptyvalueforthe$foovariablebecausethatiswhatwegetwhenaccessinganuninitialized
variable.
AnotherimportantcharacteristicthatwecanobservefromthisexampleisthateventhoughthescopeofNginxvariablesistheentireconfiguration,eachrequestdoes
haveitsownversionofallthosevariables'containers.Requestsdonotinterferewitheachothereveniftheyarereferencingavariablewiththesamename.Thisis
verymuchlikelocalvariablesinC/C++functionbodies.EachinvocationoftheC/C++functiondoesuseitsownversionofthoselocalvariables(onthestack).
Forinstance,inthissample,werequest/barandthevariable$foogetsthevalue32,whichdoesnotaffectthevalueof$fooinsubsequentrequeststo/foo(itis
stilluninitialized!),becausetheycorrespondtodifferentvaluecontainers.
OnecommonmistakeforNginxnewcomersistoregardNginxvariablesassomethingsharedamongalltherequests.EventhoughthescopeofNginxvariablenamesgo
acrossconfigurationblocksat"configurationtime",itsvaluecontainer'sscopenevergoesbeyondrequestboundariesat"requesttime".Essentiallyherewedohave
twodifferentkindsofscopehere.
NginxVariables(02)
VariableLifetime&InternalRedirection
WealreadyknowthatNginxvariablesareboundtoeachrequesthandledbyNginx,forthisreasontheyhaveexactlythesamelifetimeasthecorrespondingrequest.
Thereisanothercommonmisunderstandingherethough:somenewcomerstendtoassumethatthelifetimeofNginxvariablesisboundtothelocationconfiguration
block.Let'sconsiderthefollowingcounterexample:
server{
listen8080;
location/foo{
set$ahello;
echo_exec/bar;
}
location/bar{
echo"a=[$a]";
}
}
Hereinlocation/fooweusetheecho_execdirective(providedbythe3rd-partymodulengx_echo)toinitiatean"internalredirection"tolocation/bar.The
"internalredirection"isanoperationthatmakesNginxjumpfromonelocationtoanotherwhileprocessingarequest.This"jumping"happenscompletelywithin
theserveritself.Thisisdifferentfromthose"externalredirections"basedontheHTTP301and302responsesbecausethelatteriscollaboratedexternally,bythe
HTTPclients.Also,incaseof"externalredirections",theendusercouldusuallyobservethechangeoftheURLinherwebbrowser'saddressbarwhilethisisnotthe
caseforinternalones."Internalredirections"areverysimilartotheexeccommandinBourneShell;itisa"onewaytrip"andneverreturns.Anothersimilarexample
isthegotostatementintheClanguage.
Beingan"internalredirection",therequestaftertheredirectionremainstheoriginalone.Itisjustthecurrentlocationthatischanged,sowearestillusingthe
originalcopyoftheNginxvariablecontainers.Backtoourexample,thewholeprocesslookslikethis:Nginxfirstassignstothe$avariablethestringvaluehellovia
thesetdirectiveinlocation/foo,andthenitissuesaninternalredirectionviatheecho_execdirective,thusleavinglocation/fooandenteringlocation
/bar,andfinallyitoutputsthevalueof$a.Becausethevaluecontainerof$aremainsuntouched,wecanexpecttheresponseoutputtobehello.Thetestresult
confirmsthis:
$curllocalhost:8080/foo
a=[hello]
Butwhenaccessing/bardirectlyfromtheclientside,wewillgetanemptyvalueforthe$avariable,sincethisvariablereliesonlocation/footogetinitialized.
Itcanbeobservedthatduringarequest'slifetime,thecopyofNginxvariablecontainersdoesnotchangeatallevenwhenNginxgoesacrossdifferentlocation
configurationblocks.Herewealsoencountertheconceptof"internalredirections"forthefirsttimeandit'sworthmentioningthattherewritedirectiveofthe
ngx_rewritemodulecanalsobeusedtoinitiate"internalredirections".Forinstance,wecanrewritetheexampleabovewiththerewritedirectiveasfollows:
server{
listen8080;
location/foo{
set$ahello;
rewrite^/bar;
}
location/bar{
echo"a=[$a]";
}
}
It'sfunctionallyequivalenttoecho_exec.Wewilldiscusstherewritedirectiveinmoredepthinlaterchapters,likeinitiating"externalredirections"like301and302.
Toconclude,thelifetimeofNginxvariablecontainersisindeedboundtotherequestbeingprocessed,andisirrelevanttolocation.
NginxBuilt-inVariables
TheNginxvariableswehaveseensofarareall(implicitly)createdbydirectiveslikeset.Weusuallycallsuchvariables"user-definedvaraibles",orsimply"user
variables".ThereisalsoanotherkindofNginxvariablesthatarepre-definedbyeithertheNginxcoreorNginxmodules.Let'scallthiskindofvariables"built-in
variables".
$uri&$request_uri
OnecommonuseofNginxbuilt-invariablesistoretrievevarioustypesofinformationaboutthecurrentrequestorresponse.Forinstance,thebuilt-invariable$uri
providedbyngx_http_coreisusedtofetchthe(decoded)URIofthecurrentrequest,excludinganyquerystringarguments.Anotherexampleisthe$request_uri
variableprovidedbythesamemodule,whichisusedtofetchtheraw,non-decodedformoftheURI,includinganyquerystring.Let'slookatthefollowingexample.
location/test{
echo"uri=$uri";
echo"request_uri=$request_uri";
}
Weomittheserverconfigurationblockhereforbrevity.Justasallthosesamplesabove,westilllistentothe8080localport.Inthisexample,weoutputboththe
$uriand$request_uriintotheresponsebody.Belowistheresultoftestingthis/testinterfacewithdifferentrequests:
uri=/test
request_uri=/test
$curl'http://localhost:8080/test?a=3&b=4'
uri=/test
request_uri=/test?a=3&b=4
$curl'http://localhost:8080/test/hello%20world?a=3&b=4'
uri=/test/helloworld
request_uri=/test/hello%20world?a=3&b=4
VariableswithInfiniteNames
Thereisanotherverycommonbuilt-invariablethatdoesnothaveafixedvariablename.Instead,Ithasinfinitevariations.Thatis,allthosevariableswhosenames
havetheprefixarg_,like$arg_fooand$arg_bar.Let'sjustcallitthe$arg_XXX"variablegroup".Forexample,the$arg_namevariableisevaluatedtothe
valueofthenameURIargumentforthecurrentrequest.Also,theURIargument'svalueobtainedhereisnotdecodedyet,potentiallycontainingthe%XXsequences.
Let'scheckoutacompleteexample:
location/test{
echo"name:$arg_name";
echo"class:$arg_class";
}
ThenwetestthisinterfacewithvariousdifferentURIargumentcombinations:
name:
class:
$curl'http://localhost:8080/test?name=Tom&class=3'
name:Tom
class:3
$curl'http://localhost:8080/test?name=hello%20world&class=9'
name:hello%20world
class:9
Infact,$arg_namedoesnotonlymatchthenameargumentname,butalsoNAMEorevenName.Thatis,thelettercasedoesnotmatterhere:
$curl'http://localhost:8080/test?NAME=Marry'
name:Marry
class:
$curl'http://localhost:8080/test?Name=Jimmy'
name:Jimmy
class:
Behindthescene,NginxjustconvertstheURIargumentnamesintothepurelower-caseformbeforematchingagainstthenamespecifiedby$arg_XXX.
Ifyouwanttodecodethespecialsequenceslike%20intheURIargumentvalues,thenyoucouldusetheset_unescape_uridirectiveprovidedbythe3rd-party
modulengx_set_misc.
location/test{
set_unescape_uri$name$arg_name;
set_unescape_uri$class$arg_class;
echo"name:$name";
echo"class:$class";
}
Let'scheckouttheactualeffect:
$curl'http://localhost:8080/test?name=hello%20world&class=9'
name:helloworld
class:9
Thespacehasindeedbeendecoded!
Anotherthingthatwecanobservefromthisexampleisthattheset_unescape_uridirectivecanalsoimplicitlycreateNginxuser-definedvariables,justliketheset
directive.Wewilldiscussthengx_set_miscmoduleinmoredetailinfuturechapters.
Thistypeofvariableslike$arg_XXXpossessesinfinitenumberofpossiblenames,sotheydonotcorrespondtoanyvaluecontainers.Furthermore,suchvariables
arehandledinaveryspecificwaywithintheNginxcore.Itisthusnotpossiblefor3rd-partymodulestointroducesuchmagicalbuilt-invariablesoftheirown.
TheNginxcoreoffersalotofsuchbuilt-invariablesinadditionto$arg_XXX,likethe$cookie_XXXvariablegroupforfetchingHTTPcookievalues,the
$http_XXXvariablegroupforfetchingrequestheaders,aswellasthe$sent_http_XXXvariablegroupforretrievingresponseheaders.Wewillnotgointothedetails
foreachofthemhere.Interestedreaderscanrefertotheofficialdocumentationforthengx_http_coremodule.
Read-onlyBuilt-inVariables
Alltheuser-definedvariablesarewritable.Actuallythewaythatwedeclareorcreatesuchvariablessofaristouseaconfiguredirective,likeset,thatperformsvalue
assignmentatrequesttime.Butitisnotnecessarilythecaseforbuilt-invariables.
M ostofthebuilt-invariablesareeffectivelyread-only,likethe$uriand$request_urivariablesthatwejustintroducedearlier.Assignmentstosuchread-onlyvariables
mustalwaysbeavoided.Otherwiseitwillleadtounexpectedconsequences,forexample,
?location/bad{
?set$uri/blah;
?echo$uri;
?}
ThisproblematicconfigurationjusttriggersaconfusingerrormessagewhenNginxisstarted:
[emerg]theduplicate"uri"variablein...
Attemptsofwritingtosomeotherread-onlybuilt-invariableslike$arg_XXXwilljustleadtoservercrashesinsomeparticularNginxversions.
NginxVariables(03)
WritableBuilt-inVariable$args
Somebuilt-invariablesarewritableaswell.Forinstance,whenreadingthebuilt-invariable$args,wegettheURLquerystringofthecurrentrequest,butwhenwriting
toit,weareeffectivelymodifyingthequerystring.Hereissuchanexample:
location/test{
set$orig_args$args;
set$args"a=3&b=4";
echo"originalargs:$orig_args";
echo"args:$args";
}
HerewefirstsavetheoriginalURLquerystringintoourownvariable$orig_args,thenmodifythecurrentquerystringbyoverridingthe$argsvariable,and
finallyoutputthevariables$orig_argsand$args,respectively,withtheechodirective.Let'stestitlikethis:
originalargs:
args:a=3&b=4
$curl'http://localhost:8080/test?a=0&b=1&c=2'
originalargs:a=0&b=1&c=2
args:a=3&b=4
Inthefirsttest,wedidnotprovideanyURLquerystring,hencetheemptyoutputforthe$orig_argsvariable.Andinbothtests,thecurrentquerystringwas
forciblyoverriddentothenewvaluea=3&b=4,regardlessofthepresenceofaquerystringintheoriginalrequest.
Itshouldbenotedthatthe$argsvariableherenolongerownsavaluecontainerasuservariables,justlike$arg_XXX.Whenreading$args,Nginxwillexecuteaspecial
pieceofcode,fetchingdatafromaparticularplacewheretheNginxcorestorestheURLquerystringforthecurrentrequest.Ontheotherhand,whenweoverwrite
$args,Nginxwillexecuteanotherspecialpieceofcode,storingnewvalueintothesameplaceinthecore.OtherpartsofNginxalsoreadthesameplacewheneverthe
querystringisneeded,soourmodificationto$argswillimmediatelyaffectalltheotherparts'functionalitylateron.Let'sseeanexampleforthis:
location/test{
set$orig_a$arg_a;
set$args"a=5";
echo"originala:$orig_a";
echo"a:$arg_a";
}
Herewefirstsavethevalueofthebuilt-invaraible$arg_a,thevalueoftheoriginalrequest'sURLargumenta,intoouruservariable$orig_a,thenchangethe
URLquerystringtoa=5byassigningthenewvaluetothebuilt-invariable$args,andfinallyoutputthevariables$orig_aand$arg_a,respectively.Because
modificationsto$argseffectivelychangetheURLquerystringofthecurrentrequestforthewholeserver,thevalueofthebuilt-invariable$arg_XXXshouldalso
changeaccordingly.Thetestresultverifiesthis:
$curl'http://localhost:8080/test?a=3'
originala:3
a:5
Wecanseethattheinitialvalueof$arg_ais3sincetheURLquerystringoftheoriginalrequestisa=3.Butthefinalvalueof$arg_aautomaticallybecomes5
afterwemodify$argswiththevaluea=5.
Belowisanotherexampletodemonstratethatassignmentsto$argsalsoaffecttheHTTPproxymodulengx_proxy.
server{
listen8080;
location/test{
set$args"foo=1&bar=2";
proxy_passhttp://127.0.0.1:8081/args;
}
}
server{
listen8081;
location/args{
echo"args:$args";
}
}
Twovirtualserversaredefinedhereinthehttpconfigurationblock(omittedforbrevity).
Thefirstvirtualserverislisteningatthelocalport8080.Its/testlocationfirstupdatesthecurrentURLquerystringtothevaluefoo=1&bar=2bywritingto
$args,thensetsupanHTTPreverseproxyviatheproxy_passdirectiveofthengx_proxymodule,targetingtheHTTPservice/argsonthelocalport8081.By
defaultthengx_proxymoduleautomaticallyforwardsthecurrentURLquerystringtotheremoteHTTPservice.
The"remoteHTTPservice"onthelocalport8081isprovidedbythesecondvirtualserverdefinedbyourselves,whereweoutputthecurrentURLquerystringvia
theechodirectiveinlocation/args.Bydoingthis,wecaninvestigatetheactualURLquerystringforwardedbythengx_proxymodulefromthefirstvirtual
server.
Let'saccessthe/testinterfaceexposedbythefirstvirtualserver.
$curl'http://localhost:8080/test?blah=7'
args:foo=1&bar=2
WecanseethattheURLquerystringisfirstrewrittentofoo=1&bar=2eventhoughtheoriginalrequesttakesthevalueblah=7,thenitisforwardedtothe/args
interfaceofthesecondvirtualserverviatheproxy_passdirective,andfinallyitsvalueisoutputtotheclient.
Tosummarize,theassignmentto$argsalsosuccessfullyinfluencesthebehaviorofthengx_proxymodule.
Variable"GetHandlers"and"SetHandlers"
Wehavealreadylearnedinprevioussectionsthatwhenreadingthebuilt-invariable$args,Nginxexecutesaspecialpieceofcodetoobtainavalueon-the-flyandwhen
writingtothisvariable,Nginxexecutesanotherspecialpieceofcodetopropagatethechange.InNginx'sterminology,thespecialcodeexecutedforreadingthevariable
iscalled"gethandler"andthecodeforwritingtothevariableiscalled"sethandler".DifferentNginxmodulesusuallypreparedifferent"gethandlers"and"set
handlers"fortheirownvariables,whicheffectivelyputmagicintothesevariables'behavior.
Suchtechniquesarenotuncommoninthecomputingworld.Forexample,inobject-orientedprogramming(OOP),theclassdesignerusuallydoesnotexposethe
membervariableoftheclassdirectlytotheuserprogrammer,butinsteadprovidestwomethodsforreadingfromandwritingtothemembervariable,respectively.
Suchclassmethodsareoftencalled"accessors".BelowisanexampleintheC++programminglanguage:
#include<string>
usingnamespacestd;
classPerson{
public:
conststringget_name(){
returnm_name;
}
voidset_name(conststringname){
m_name=name;
}
private:
stringm_name;
};
InthisC++classPerson,weprovidetwopublicmethods,get_nameandset_name,toserveasthe"accessors"fortheprivatemembervariablem_name.
Thebenefitsofsuchdesignareobvious.Theclassdesignercanexecutearbitrarycodeinthe"accessors",toimplementanyextrabusinesslogicorusefulsideeffects,
likeautomaticallyupdatingothermembervariablesdependingonthecurrentmember,orupdatingthecorrespondingfieldinadatabaseassociatedwiththecurrent
object.Forthelattercase,itispossiblethatthemembervariabledoesnotexistatall,orthatthemembervariablejustservesasadatacachetomitigatethepressureon
theback-enddatabase.
Correspondingtotheconceptof"accessors"inOOP,Nginxvariablesalsosupportbindingcustom"gethandlers"and"sethandlers".Additionally,notallNginx
variablesownacontainertoholdvalues.Somevariableswithoutacontainerjustbehavelikeamagicalcontainerbymeansofitsfancy"gethandler"and"sethandler".
Infact,whenavariableisbeingcreatedat"configuretime",thecreatingNginxmodulemustmakeadecisiononwhethertoallocateavaluecontainerforitandwhether
toattachacustom"gethandler"and/ora"sethandler"toit.
Thosevariablesowningavaluecontainerarecalled"indexedvariables"inNginx'sterminology.Otherwise,theyaresaidtobenotindexed.
Wealreadyknowthatthe"variablegroups"like$arg_XXXdiscussedinearliersectionsdonothaveavaluecontainerandthusarenotindexed.Whenreading
$arg_XXX,itisits"gethandler"atwork,thatis,its"gethandler"scansthecurrentURLquerystringon-the-fly,extractingthevalueofthespecifiedURLargument.
M anybeginnersmisunderstandtheway$arg_XXXisimplemented;theyassumethatNginxwillparsealltheURLargumentsinadvanceandpreparethevaluesfor
allthosenon-empty$arg_XXXvariablesbeforetheyareactuallyread.Thisisnottrue,however.NginxnevertriestoparsealltheURLargumentsbeforehand,but
ratherscansthewholeURLquerystringforaparticularargumentina"gethandler"everytimethatargumentisrequestedbyreadingthecorresponding$arg_XXX
variable.Similarly,whenreadingthebuilt-invariable$cookie_XXX,its"gethandler"justscanstheCookierequestheadersforthecookienamespecified.
NginxVariables(04)
ValueContainersforCaching&ngx_map
SomeNginxvariableschoosetousetheirvaluecontainersasadatacachewhenthe"gethandler"isconfigured.Inthissetting,the"gethandler"isrunonlyonce,i.e.,at
thefirsttimethevariableisread,whichreducesoverheadwhenthevariableisreadmultipletimesduringitslifetime.Let'sseeanexampleforthis.
map$args$foo{
default0;
debug1;
}
server{
listen8080;
location/test{
set$orig_foo$foo;
set$argsdebug;
echo"originalfoo:$orig_foo";
echo"foo:$foo";
}
}
Hereweusethemapdirectivefromthestandardmodulengx_mapforthefirsttime,whichdeservessomeintroduction.Thewordmapheremeansmappingor
correspondence.Forexample,functionsinM athsareakindof"mapping".AndNginx'smapdirectiveisusedtodefinea"mapping"relationshipbetweentwoNginx
variables,orinotherwords,"functionrelationship".Backtothisexample,weusethemapdirectivetodefinethe"mapping"relationshipbetweenuservariable$foo
andbuilt-invariable$args.WhenusingtheM athfunctionnotation,y=f(x),our$argsvariableiseffectivelythe"independentvariable",x,while$fooisthe
"dependentvariable",y.Thatis,thevalueof$foodependsonthevalueof$args,orrather,wemapthevalueof$argsontothe$foovariable(insomeway).
Nowlet'slookattheexactmappingruledefinedbythemapdirectiveinthisexample.
map$args$foo{
default0;
debug1;
}
Thefirstlinewithinthecurlybracesisaspecialrulecondition,thatis,thisconditionholdsifandonlyifotherconditionsallfail.Whenthis"default"conditionholds,
the"dependentvariable"$fooisassignedbythevalue0.Thesecondlinewithinthecurlybracesmeansthatthe"dependentvariable"$fooisassignedbythevalue
1ifthe"independentvariable"$argsmatchesthestringvaluedebug.Combiningthesetwolines,weobtainthefollowingcompletemappingrule:ifthevalueof
$argsisdebug,variable$foogetsthevalue1;otherwise$foogetsthevalue0.Soessentially,thisisaconditionalassignmenttothevariable$foo.
Nowthatweunderstandwhatthemapdirectivedoes,let'slookatthedefinitionoflocation/test.Wefirstsavethevalueof$foointoanotheruservariable
$orig_foo,thenoverwritethevalueof$argstodebug,andfinallyoutputthevaluesof$orig_fooand$foo,respectively.
Intuitively,afterweoverwritethevalueof$argstodebug,thevalueof$fooshouldautomaticallygetadjustedto1accordingtothemappingruledefinedearlier,
regardlessoftheoriginalvalueof$foo.Butthetestresultsuggeststheotherwayaround.
originalfoo:0
foo:0
Thefirstoutputlineindicatesthatthevalueof$orig_foois0,whichisexactlywhatweexpected:theoriginalrequestdoesnottakeaURLquerystring,sothe
initialvalueof$argsisempty,leadingtothe0initialvalueof$foo,accordingtothe"default"conditioninourmappingrule.
Butsurprisingly,thesecondoutputlineindicatesthatthefinalvalueof$fooisstill0,evenafterweoverwrite$argstothevaluedebug.Thisapparentlyviolates
ourmappingrulebecausewhen$argstakesthevaluedebug,thevalueof$fooshouldreallybe1.Sowhatishappeninghere?
Actuallythereasonisprettysimple:whenthefirsttimevariable$fooisread,itsvaluecomputedbyngx_map's"gethandler"iscachedinitsvaluecontainer.We
alreadylearnedearlierthatNginxmodulesmaychoosetousethevaluecontainerofthevariablecreatedbythemselvesasadatacacheforits"gethandler".Obviously,
thengx_mapmoduleconsidersthemappingcomputationbetweenvariablesexpensiveenoughandcachestheresultautomatically,sothatthenexttimethesame
variableisreadwithinthelifetimeofthecurrentrequest,Nginxcanjustreturnthecachedresultwithoutinvokingthe"gethandler"again.
Toverifythisfurther,wecantryspecifyingtheURLquerystringasdebugintheoriginalrequest.
$curl'http://localhost:8080/test?debug'
originalfoo:1
foo:1
Itcanbeseenthatthevalueof$orig_foobecomes1,complyingwithourmappingrule.Andsubsequentreadingsof$fooalwaysyieldthesamecachedresult,1,
regardlessofthenewvalueof$argslateron.
Themapdirectiveisactuallyauniqueexample,becauseitnotonlyregistersa"gethandler"fortheuservariable,butalsoallowstheusertodefinethecomputingrule
inthe"gethandler"directlyintheNginxconfigurationfile.Ofcourse,therulethatcanbedefinedhereislimitedtosimplemappingrelationswithanothervariable.
M eanwhile,itmustbemadeclearthatnotallthevariablesusinga"gethandler"willcachetheresult.Forinstance,wehavealreadyseenearlierthatthe$arg_XXX
variabledoesnotuseitsvaluecontaineratall.
Similartothengx_mapmodule,thestandardmodulengx_geothatweencounteredearlieralsoenablesvaluecachingforthevariablescreatedbyitsgeodirective.
ASideNoteforUseContextsofDirectives
Inthepreviousexample,weshouldalsonotethatthemapdirectiveisputoutsidetheserverconfigurationblock,thatis,itisdefineddirectlywithintheoutermost
httpconfigurationblock.Somereadersmaybecuriousaboutthissetting,sinceweonlyuseitinlocation/testafterall.Ifwetryputtingthemapstatement
withinthelocationblock,however,wewillgetthefollowingerrorwhilestartingNginx:
[emerg]"map"directiveisnotallowedherein...
Soitisexplicitlyprohibited.Infact,itisonlyallowedtousethemapdirectiveinthehttpblock.Everyconfiguredirectivedoeshaveapre-definedsetofuse
contextsintheconfigurationfile.Whenindoubt,alwaysrefertothecorrespondingdocumentationfortheexactusecontextsofaparticulardirective.
LazyEvaluationofVariableValues
M anyNginxfreshmenwouldworrythattheuseofthemapdirectivewithintheglobalscope(i.e.,thehttpblock)willleadtounnecessaryvariablevalue
computationandassignmentforallthelocationsinallthevirtualserversevenifonlyonelocationblockactuallyusesit.Fortunately,thisisnotwhatis
happeninghere.Wehavealreadylearnedhowthemapdirectiveworks.Itisthe"gethandler"(registeredbythengx_mapmodule)thatperformsthevaluecomputation
andrelatedassignment.Andthe"gethandler"willnotrunatallunlessthecorrespondinguservariableisactuallybeingread.Therefore,forthoserequeststhatnever
accessthatvariable,therecannotbeany(useless)computationinvolved.
Thetechniquethatpostponesthevaluecomputationofftothepointwherethevalueisactuallyneedediscalled"lazyevaluation"inthecomputingworld.
Programminglanguagesnativelyoffering"lazyevaluation"isnotverycommonthough.ThemostfamousexampleistheHaskellprogramminglanguage,wherelazy
evaluationisthedefaultsemantics.Incontrastwith"lazyevaluation",itismuchmorecommontosee"eagerevaluation".Weareluckytoseeexamplesoflazy
evaluationhereinthengx_mapmodule,butthe"eagerevaluation"semanticsisalsomuchmorecommonintheNginxworld.Considerthefollowingsetstatementthat
cannotbesimpler:
set$b"$a,$a";
Whenrunningthesetdirective,Nginxeagerlycomputesandassignsthenewvalueforthevariable$bwithoutpostponingtothepointwhen$bisactuallyreadlater
on.Similarly,theset_unescape_uridirectivealsoevaluateseagerly.
NginxVariables(05)
VariablesinSubrequests
ADetourtoSubrequests
Wehaveseenearlierthatthelifetimeofvariablecontainersisboundtotherequest,butIownyouaformaldefinitionof"requests"there.Youmighthaveassumedthat
the"requests"inthatcontextarejustthoseHTTPrequestsinitiatedfromtheclientside.Infact,therearetwokindsof"requests"intheNginxworld.Oneiscalled
"mainrequests",andtheotheriscalled"subrequests".
M ainrequestsarethoseinitiatedexternallybyHTTPclients.Alltheexamplesthatwehaveseensofarinvolvemainrequestsonly,includingthosedoing"internal
redirections"viatheecho_execorrewritedirective.
WhereassubrequestsareaspecialkindofrequestsinitiatedfromwithintheNginxcore.ButpleasedonotconfusesubrequestswiththoseHTTPrequestscreatedby
thengx_proxymodules!SubrequestsmaylookverymuchlikeanHTTPrequestinappearance,theirimplementation,however,hasnothingtodowithneitherthe
HTTPprotocolnoranykindofsocketcommunication.Asubrequestisanabstractinvocationfordecomposingthetaskofthemainrequestintosmaller"internal
requests"thatcanbeservedindependentlybymultipledifferentlocationblocks,eitherinseriesorinparallel."Subrequests"canalsoberecursive:anysubrequest
caninitiatemoresub-subrequests,targetingotherlocationblocksoreventhecurrentlocationitself.AccordingtoNginx'sterminology,ifrequestAinitiatesa
subrequestB,thenAiscalledthe"parentrequest"ofB.ItisworthmentioningthattheApachewebserveralsohastheconceptofsubrequestsforlong,soreaders
comingfromthatworldshouldbenostrangertothis.
Let'scheckoutanexampleusingsubrequests:
location/main{
echo_location/foo;
echo_location/bar;
}
location/foo{
echofoo;
}
location/bar{
echobar;
}
Hereinlocation/main,weusetheecho_locationdirectivefromthengx_echomoduletoinitiatetwoGET-typedsubrequeststargeting/fooand/bar,
respectively.Thesubrequestsinitiatedbyecho_locationarealwaysrunningsequentiallyaccordingtotheirliteralorderintheconfigurationfile.Therefore,thesecond
/barrequestwillnotbefireduntilthefirst/foorequestcompletesprocessing.Theresponsebodyofthesetwosubrequestsgetconcatenatedtogetheraccordingto
theirrunningorder,toformthefinalresponsebodyoftheirparentrequest(for/main):
$curl'http://localhost:8080/main'
foo
bar
Itshouldbenotedthatthecommunicationoflocationblocksviasubrequestsislimitedwithinthesameserverblock(i.e.,thesamevirtualserverconfiguration),
sowhentheNginxcoreprocessesasubrequest,itjustcallsafewCfunctionsbehindthescene,withoutdoinganykindofnetworkorUNIXdomainsocket
communication.Forthisreason,subrequestsareextremelyefficient.
IndependentVariableContainersinSubrequests
BacktoourearlierdiscussionforthelifetimeofNginxvariablecontainers,nowwecanstillstatethatthelifetimeisboundtothecurrentrequest,andeveryrequest
doeshaveitsowncopyofallthevariablecontainers.Itisjustthatthe"request"herecanbeeitheramainrequest,orasubrequest.Variableswiththesamename
betweenaparentrequestandasubrequestwillgenerallynotinterferewitheachother.Let'sdoasmallexperimenttoconfirmthis:
location/main{
set$varmain;
echo_location/foo;
echo_location/bar;
echo"main:$var";
}
location/foo{
set$varfoo;
echo"foo:$var";
}
location/bar{
set$varbar;
echo"bar:$var";
}
Inthissample,weassigndifferentvaluestothevariable$varinthreelocationblocks,/main,/foo,and/bar,andoutputthevalueof$varinallthese
locations.Inparticular,weintentionallyoutputthevalueof$varinlocation/mainaftercallingthetwosubrequests,soifvaluechangesof$varinthe
subrequestscanaffecttheirparentrequest,weshouldseeanewvalueoutputinlocation/main.Theresultofrequesting/mainisasfollows:
foo:foo
bar:bar
main:main
Apparently,theassignmentstovariable$varinthosetwosubrequestsdonotaffectthemainrequest/mainatall.Thissuccessfullyverifiesthatboththemain
requestanditssubrequestsdoowndifferentcopiesofvariablecontainers.
SharedVariableContainersamongRequests
Unfortunately,subrequestsinitiatedbycertainNginxmodulesdosharevariablecontainerswiththeirparentrequests,likethoseinitiatedbythe3rd-partymodule
ngx_auth_request.Belowissuchanexample:
location/main{
set$varmain;
auth_request/sub;
echo"main:$var";
}
location/sub{
set$varsub;
echo"sub:$var";
}
Hereinlocation/main,wefirstassigntheinitialvaluemaintovariable$var,thenfireasubrequestto/subviatheauth_requestdirectivefromthe
ngx_auth_requestmodule,andfinallyoutputthevalueof$var.Notethatinlocation/subweintentionallyoverwritethevalueof$vartosub.When
accessing/main,weget
main:sub
Obviously,thevaluechangeof$varinthesubrequestto/subdoesaffectthemainrequestto/main.Thusthevariablecontainerof$varisindeedsharedbetween
themainrequestandthesubrequestcreatedbythengx_auth_requestmodule.
Forthepreviousexample,somereadersmightask:"whydoesn'ttheresponsebodyofthesubrequestappearinthefinaloutput?"Theanswerissimple:itisjust
becausetheauth_requestdirectivediscardstheresponsebodyofthesubrequestitmanages,andonlycheckstheresponsestatuscodeofthesubrequest.When
thestatuscodelooksgood,like200,auth_requestwilljustallowNginxcontinueprocessingthemainrequest;otherwiseitwillimmediatelyabortthemain
requestbyreturninga403errorpage,forexample.Inourexample,thesubrequestto/subjustreturna200responseimplicitlycreatedbytheechodirectivein
location/sub.
Eventhoughsharingvariablecontainersamongthemainrequestandallitssubrequestscouldmakebidirectionaldataexchangeeasier,itcouldalsoleadtounexpected
subtleissuesthatarehardtodebuginreal-worldconfigurations.Becauseusersoftenforgetthatavariablewiththesamenameisactuallyusedinsomedeeply
embeddedsubrequestandjustuseitforsomethingelseinthemainrequest,thisvariablecouldgetunexpectedlymodifiedduringprocessing.Suchbadsideeffects
makemany3rd-partymoduleslikengx_echo,ngx_luaandngx_srcachechoosetodisablethevariablesharingbehaviorforsubrequestsbydefault.
NginxVariables(06)
Built-inVariablesinSubrequests
TherearesomesubtletiesinvolvedinusingNginxbuilt-invariablesinthecontextofasubrequest.Wewilldiscussthedetailsinthissection.
Built-inVariablesSensitivetotheSubrequestContext
Wealreadyknowthatmostbuilt-invariablesarenotsimplevaluecontainers.Theybehavedifferentlythanuservariablesbyregistering"gethandlers"and/or"set
handlers".Evenwhentheydoownavaluecontainer,theyusuallyjustusethecontainerasaresultcachefortheir"gethandlers".The$argsvariablewediscussed
earlier,forexample,justusesits"gethandler"toreturntheURLquerystringforthecurrentrequest.Thecurrentrequestherecanalsobeasubrequest,sowhen
reading$argsinasubrequest,its"gethandler"shouldnaturallyreturnthequerystringforthesubrequest.Let'sseesuchanexample:
location/main{
echo"mainargs:$args";
echo_location/sub"a=1&b=2";
}
location/sub{
echo"subargs:$args";
}
Hereinthe/maininterface,wefirstechooutthevalueof$argsforthecurrentrequest,andthenuseecho_locationtoinitiateasubrequestto/sub.Itshouldbe
notedthatherewegiveasecondargumenttotheecho_locationdirective,tospecifytheURLquerystringforthesubrequestbeingfired(thefirstargumentistheURI
forthesubrequest,aswealreadyknow).Finally,wedefinethe/subinterfaceandprintoutthevalueof$argsinthere.Queryingthe/maininterfacegives
$curl'http://localhost:8080/main?c=3'
mainargs:c=3
subargs:a=1&b=2
Itisclearthatwhen$argsisreadinthemainrequest(to/main),itsvalueistheURLquerystringofthemainrequest;whereaswheninthesubrequest(to/foo),it
isthequerystringofthesubrequest,a=1&b=2.Thisbehaviorindeedmatchesourintuition.
Justlike$args,whenthebuilt-invariable$uriisusedinasubrequest,its"gethandler"alsoreturnsthe(decoded)URIofthecurrentsubrequest:
location/main{
echo"mainuri:$uri";
echo_location/sub;
}
location/sub{
echo"suburi:$uri";
}
Belowistheresultofquerying/main:
mainuri:/main
suburi:/sub
Theoutputiswhatwewouldexpect.
Built-inVariablesforMainRequestsOnly
Unfortunately,notallbuilt-invariablesaresensitivetothecontextofsubrequests.Severalbuilt-invariablesalwaysactonthemainrequestevenwhentheyareusedin
asubrequest.Thebuilt-invariable$request_methodissuchanexception.
Whenever$request_methodisread,wealwaysgettherequestmethodname(suchasGETandPOST)forthemainrequest,nomatterwhetherthecurrentrequestisa
subrequestornot.Let'stestitout:
location/main{
echo"mainmethod:$request_method";
echo_location/sub;
}
location/sub{
echo"submethod:$request_method";
}
Inthisexample,the/mainand/subinterfacesbothoutputthevalueof$request_method.M eanwhile,weinitiateaGETsubrequestto/subviatheecho_location
directivein/main.Nowlet'sdoaPOSTrequestto/main:
$curl--datahello'http://localhost:8080/main'
mainmethod:POST
submethod:POST
Hereweusethe--dataoptionofthecurlutilitytospecifyourPOSTrequestbody,alsothisoptionmakescurlusethePOSTmethodfortherequest.Thetest
resultturnsoutaswepredicted:thevariable$request_methodisevaluatedtothemainrequest'smethodname,POST,despiteitsuseinaGETsubrequest.
Somereadersmightchallengeourconclusionherebypointingoutthatwedidnotruleoutthepossibilitythatthevalueof$request_methodgotcachedatitsfirst
readinginthemainrequestandwhatwewereseeinginthesubrequestwasactuallythecachedvaluethatwasevaluatedearlierinthemainrequest.Thisconcernis
unnecessary,however,becausewehavealsolearnedthatthevariablecontainerrequiredbydatacaching(ifany)isalwaysboundtothecurrentrequest,alsothe
subrequestsinitiatedbythengx_echomodulealwaysdisablevariablecontainersharingwiththeirparentrequests.Backtothepreviousexample,evenifthebuilt-in
variable$request_methodinthemainrequestusedthevaluecontainerasthedatacache(actuallyitdoesnot),itcannotaffectthesubrequestbyanymeans.
Tofurtheraddresstheconcernofthesereaders,let'sslightlymodifythepreviousexamplebyputtingtheechostatementfor$request_methodin/mainafterthe
echo_locationdirectivethatrunsthesubrequest:
location/main{
echo_location/sub;
echo"mainmethod:$request_method";
}
location/sub{
echo"submethod:$request_method";
}
Let'stestitagain:
submethod:POST
mainmethod:POST
Nochangeintheoutputcanbeobserved,exceptthatthetwooutputlinesreversedtheorder(sinceweexchangetheorderofthosetwongx_echomodule'sdirectives).
Consequently,wecannotobtainthemethodnameofasubrequestbyreadingthe$request_methodvariable.Thisisacommonpitfallforfreshmenwhendealingwith
methodnamesofsubrequests.Toovercomethislimitation,weneedtoturntothebuilt-invariable$echo_request_methodprovidedbythengx_echomodule:
location/main{
echo"mainmethod:$echo_request_method";
echo_location/sub;
}
location/sub{
echo"submethod:$echo_request_method";
}
Wearefinallygettingwhatwewant:
mainmethod:POST
submethod:GET
Nowwithinthesubrequest,wegetitsownmethodname,GET,asexpected,andthemainrequestmethodremainsPOST.
Similarto$request_method,thebuilt-invariable$request_urialsoalwaysreturnsthe(non-decoded)URLforthemainrequest.Thisismoreunderstandable,however,
becausesubrequestsareessentiallyfakedrequestsinsideNginx,whichdonotreallytakeanon-decodedrawURL.
VariableContainerSharingandValueCachingTogether
Intheprevioussection,someofthereaderswereworriedaboutthecasethatvariablecontainersharinginsubrequestsandvaluecachingforvariable's"gethandlers"
wereworkingtogether.Ifitwereindeedthecase,thenitwouldbeanightmarebecauseitwouldbereallyreallyhardtopredictwhatisgoingonbyjustlookingatthe
configurationfile.Inprevioussections,wealreadylearnedthatthesubrequestsinitiatedbythengx_auth_requestmodulearesharingthesamevariablecontainerswith
theirparents,sowecanmaliciouslyconstructsuchahorribleexample:
map$uri$tag{
default0;
/main1;
/sub2;
}
server{
listen8080;
location/main{
auth_request/sub;
echo"maintag:$tag";
}
location/sub{
echo"subtag:$tag";
}
}
Hereweuseouroldfriend,themapdirective,tomapthevalueofthebuilt-invariable$uritoouruservariable$tag.When$uritakesthevalue/main,thevalue1is
assignedto$tag;when$uritakesthevalue/sub,thevalue2isassignedinsteadto$tag;underalltheotherconditions,0isassigned.Next,in/main,wefirst
initiateasubrequestto/subbyusingtheauth_requestdirective,andthenoutputthevalueof$tag.Andwithin/sub,wedirectlyoutputthevalueof$tag.
Guesswhatwewillgetwhenweaccess/main?
maintag:2
Ouch!Didn'twemapthevalue/mainto1?Whytheactualoutputfor/mainisthevalue,2,for/sub?Whatisgoingonhere?
Actuallyitworkedlikethis:our$tagvariablewasfirstreadinthesubrequestto/sub,andthe"gethandler"registeredbymapcomputedthevalue2for$tagin
thatcontext(because$uriwas/subinthesubrequest)andthevalue2gotcachedinthevaluecontainerof$tagfromthenon.Becausetheparentrequestsharedthe
samecontainerasthesubrequestcreatedbyauth_request,whentheparentrequestread$taglater(afterthesubrequestwasfinished),thecachedvalue2was
directlyreturned!Suchresultscanindeedbeverysurprisingatfirstglance.
Fromthisexample,wecanconcludeagainthatitcanhardlybeagoodideatoenablevariablecontainersharinginsubrequests.
NginxVariables(07)
SpecialValue"Invalid"and"NotFound"
WehavementionedthatthevaluesofNginxvariablescanonlybeofonesingletype,thatis,thestringtype,butvariablescouldalsohavenomeaningfulvaluesatall.
Variableswithoutanymeaningfulvaluesstilltakeaspecialvaluethough.Therearetwopossiblespecialvalues:"invalid"and"notfound".
Forexample,whenauservariable$fooiscreatedbutnotassignedyet,$footakesthespecialvalueof"invalid".AndwhenthecurrentURLquerystringdoesnot
havetheXXXargumentatall,thebuilt-invariable$arg_XXXtakesthespecialvalueof"notfound".
Both"invalid"and"notfound"arespecialvalues,completelydifferentfromanemptystringvalue("").Thisisverysimilartothosedistinctspecialvaluesinsome
dynamicprograminglanguages,likeundefinPerl,nilinLua,andnullinJavaScript.
Wehaveseenearlierthatanuninitializedvariableisevaluatedtoanemptystringwhenusedinaninterpolatedstring,itsrealvalue,however,isnotanemptystringat
all.Itisthe"gethandler"registeredbythesetdirectivethatautomaticallyconvertsthe"invalid"specialvalueintoanemptystring.Toverifythis,let'sreturntothe
examplewehavediscussedbefore:
location/foo{
echo"foo=[$foo]";
}
location/bar{
set$foo32;
echo"foo=[$foo]";
}
Whenaccessing/foo,theuservariable$fooisuninitializedwhenusedintheinterpolatedstringfortheechodirective.Theoutputshowsthatthevariableis
evaluatedtoanemptystring:
foo=[]
Fromtheoutput,theuninitialized$foovariablebehavesjustliketakinganemptystringvalue.Butcarefulreadersshouldhavealreadynoticedthat,fortherequest
above,thereisawarningintheNginxerrorlogfile(whichislogs/error.logbydefault):
[warn]5765#0:*1usinguninitialized"foo"variable,...
Whoonearthgeneratesthiswarning?Theansweristhe"gethandler"of$foo,registeredbythesetdirective.When$fooisread,Nginxfirstchecksthevalueinits
containerbutseesthe"invalid"specialvalue,thenNginxdecidestocontinuerunning$foo's"gethandler",whichfirstprintsthewarning(asshownabove)andthen
returnsanemptystringvalue,whichthereaftergetscachedin$foo'svaluecontainer.
Carefulreadersshouldhaveidentifiedthatthisprocessforuservariablesisexactlythesameasthemechanismwediscussedearlierforbuilt-invariablesinvolving"get
handlers"andresultcachinginvaluecontainers.Yes,itisthesamemechanisminaction.Itisalsoworthnotingthatonlythe"invalid"specialvaluewilltriggerthe"get
handler"invocationintheNginxcorewhile"notfound"willnot.
Thewarningmessageaboveusuallyindicatesatypointhevariablenameormisuseofuninitializedvariables,notnecessarilyinthecontextofaninterpolatedstring.
Becauseoftheexistenceofvaluecachinginthevariablecontainer,thiswarningwillnotgetprintedmultipletimesinthelifetimeofthecurrentrequest.Also,the
ngx_rewritemoduleprovidestheuninitialized_variable_warndirectivefordisablingthiswarningaltogether.
TestingSpecialValuesofNginxVariablesinLua
Aswehavejustmentioned,thebuilt-invariable$arg_XXXtakesthespecialvalue"notfound"whentheURLargumentXXXdoesnotexist,butunfortunately,itis
noteasytodistinguishitfromtheemptystringvaluedirectlyintheNginxconfigurationfile,forexample:
location/test{
echo"name:[$arg_name]";
}
HereweintentionallyomittheURLargumentnameinourrequest:
name:[]
Wecanseethatwearestillgettinganemptystringvalue,becausethistimeitistheNginx"scriptengine"thatautomaticallyconvertsthe"notfound"specialvalueto
anemptystringwhenperformingvariableinterpolation.
Thenhowcanwetestthespecialvalue"notfound"?Orinotherwords,howcanwedistinguishitfromnormalemptystringvalues?Obviously,inthefollowing
example,theURLargumentnamedoestakeanordinaryvalue,whichisatrueemptystring:
$curl'http://localhost:8080/test?name='
name:[]
Butwecannotreallydifferentiatethisfromtheearliercasethatdoesnotmentionthenameargumentatall.
Luckily,wecaneasilyachievethisinLuabymeansofthe3rd-partymodulengx_lua.Pleaselookatthefollowingexample:
location/test{
content_by_lua'
ifngx.var.arg_name==nilthen
ngx.say("name:missing")
else
ngx.say("name:[",ngx.var.arg_name,"]")
end
';
}
Thisexampleisveryclosetothepreviousoneintermsoffunctionality.Weusethecontent_by_luadirectivefromthengx_luamoduletoembedasmallpieceofour
ownLuacodetotestagainstthespecialvalueoftheNginxvariable$arg_name.When$arg_nametakesaspecialvalue(either"notfound"or"invalid"),wewill
getthefollowingoutputwhenrequesting/foo:
name:missing
Thisisourfirsttimemeetingthengx_luamodule,whichdeservesabriefintroduction.ThismoduleembedstheLualanguageinterpreter(orLuaJIT'sJust-in-Time
compiler)intotheNginxcore,toallowNginxusersdirectlyruntheirownLuaprogramsinsidetheserver.TheusercanchoosetoinsertherLuacodeintodifferent
runningphasesoftheserver,tofulfilldifferentrequirements.SuchLuacodeareeitherspecifieddirectlyasliteralstringsintheNginxconfigurationfile,orresidein
external.luasourcefiles(orLuabinarybytecodefiles)whosepathsarespecifiedintheNginxconfiguration.
Backtoourexample,wecannotdirectlywritesomethinglike$arg_nameinourLuacode.Instead,wereferenceNginxvariablesinLuabymeansofthengx.var
APIprovidedbythengx_luamodule.Forexample,toreferencetheNginxvariable$VARIABLEinLua,wejustwritengx.var.VARIABLE.WhentheNginxvariable
$arg_nametakesthespecialvalue"notfound"(or"invalid"),ngx.var.arg_nameisevaluatedtothenilvalueintheLuaworld.Itshouldalsobenotingthatwe
usetheLuafunctionngx.saytoprintouttheresponsebodycontents,whichisfunctionallyequivalenttotheechodirectivewearealreadyveryfamiliarwith.
IfweprovideanameURIargumentthattakesanemptyvalueintherequest,theoutputisnowverydifferent:
$curl'http://localhost:8080/test?name='
name:[]
Inthistest,thevalueoftheNginxvariable$arg_nameisatrueemptystring,neither"notfound"nor"invalid".SoinLua,theexpressionngx.var.arg_name
evaluatestotheLuaemptystring(""),clearlydistinguishedfromtheLuanilvalueintheprevioustest.
Thisdifferentiationisimportantincertainapplicationscenarios.Forinstance,somewebserviceshavetodecidewhethertouseacolumnvaluetofilterthedatasetby
checkingtheexistenceofthecorrespondingURIargument.Fortheseserives,whenthenameURIargumentisabsent,thewholedatasetarejustreturned;whenthe
nameargumenttakesanemptyvalue,however,onlythoserecordsthattakeanemptyvaluearereturned.
Itisworthmentioningafewlimitationsinthestandard$arg_XXXvariable.Considerusingthefollowingrequesttotest/testinourpreviousexampleusingLua:
$curl'http://localhost:8080/test?name'
name:missing
Nowthe$arg_namevariablestillreadsthe"notfound"specialvalue,whichisapparentlycounter-intuitive.Additionally,whenmultipleURIargumentswiththe
samenamearespecifiedintherequest,$arg_XXXjustreturnsthefirstvalueoftheargument,discardingothervaluessilently:
$curl'http://localhost:8080/test?name=Tom&name=Jim&name=Bob'
name:[Tom]
Tosolvetheseproblems,wecanusetheLuafunctionngx.req.get_uri_argsprovidedbythengx_luamoduleinstead.
NginxVariables(08)
In(02)wementionedthatanothercategoryofbuiltinvariables$cookie_XXXarelike$arg_XXX.SimilarlywhenthereexistnocookienamedXXX,itscorresponding
Nginxvariable$cookie_XXXhasnon-value"notfound".
location/test{
content_by_lua'
ifngx.var.cookie_user==nilthen
ngx.say("cookieuser:missing")
else
ngx.say("cookieuser:[",ngx.var.cookie_user,"]")
end
';
}
Thecurlutilityoffersthe--cookiename=valueoption,whichdesignatesname=valueasacookieofitsrequest(byaddingtheCookieheader).Let'stesta
fewcasescontainingcookies.
$curl--cookieuser=agentzh'http://localhost:8080/test'
cookieuser:[agentzh]
$curl--cookieuser='http://localhost:8080/test'
cookieuser:[]
cookieuser:missing
Asexpected,whencookieuserdoesnotexist,Luavariablengx.var.cookie_userisnil.Sowehavesuccessfullydistinguishedthecasewithemptystring
andthecasewithnon-value.
Aniceadd-onwithmodulengx_luaiswhenluareferencesanundeclaredvariableofNginx,thevariableisnilandNginxwillnotabortsitloadingasbefore.
location/test{
content_by_lua'
ngx.say("$blah=",ngx.var.blah)
';
}
Uservariable$blahisneverdeclaredintheNginxconfigurationnginx.conf,butitisreferencedasngx.var.blahinLuacode.Nginxcanbestartedstill,
becausewhenNginxloadsitsconfiguration,Luacodeisonlycompiledbutnotexecuted,SoNginxhasnoideaavariable$blahisreferenced.Whenluacommandis
executedinruntimebycommandcontent_by_lua,theluavariableisevaluatedasnil.M odulengx_luaanditscommandngx.saywillconvertLuanilintostring
"nil"beforeitisprinted,sotheoutputwillbe:
curl'http://localhost:8080/test'
$blah=nil
Thisisindeedwhatwewant.
Weshouldhavenoticedalso,whencommandcontent_by_luaincludes$blahinitsparameter,itisneverevaluatedas"variableinterpolation"does(otherwiseNginx
willbecomplainingvariable$blahisnotdeclared).Thisisbecausecommandcontent_by_luadoesnotreallysupport"variableinterpolation".Aswehavesaid
earlierin(01),Nginxcommanddoesnotnecessarilysupport"variableinterpolation"anditisentirelyuptothemoduleimplementation.
It'sactuallydifficulttoreturnan"invalid"non-value.Aswelearntin(07),variableswhicharedeclaredbutnotinitializedbysethasnon-value"invalid".However,as
soonasthevariableisdevalued,the"gethandler"isexecutedandanemptystringiscomputedandcached,soeventuallyemptystringisreturned,notthe"invalid"
non-value.Followingluacodecanprovethis:
location/foo{
content_by_lua'
ifngx.var.foo==nilthen
ngx.say("$fooisnil")
else
ngx.say("$foo=[",ngx.var.foo,"]")
end
';
}
location/bar{
set$foo32;
echo"foo=[$foo]";
}
Byrequestingtolocation/foowehave:
$foo=[]
Aswecantell,whenLuareferencesuninitializedNginxvariable$foo,itobtainsemptystring.
Lastnottheleast,weshouldhavepointedout,althoughNginxvariablecanhaveonlystringsasvalidvalue.The3rdpartymodulengx_array_varcansupportarray
likeoperationsforNginxvariable.Hereisanexample:
location/test{
array_split","$arg_namesto=$array;
array_map"[$array_it]"$array;
array_join""$arrayto=$res;
echo$res;
}
M odulengx_array_varprovidescommandsarray_split,array_mapandarray_join.Thesemanticsisprettyclosetothebuiltinfunctionssplit,mapand
joininPerl(otherlanguagessupportsimilarfunctionalitiestoo).Nowlet'scheckwhathappenswhenlocation/testisrequested:
$curl'http://localhost:8080/test?names=Tom,Jim,Bob
[Tom][Jim][Bob]
Clearlymodulengx_array_varmakeiteasiertohandleinputswithvariablelength,suchastheURLparametername,whichcomposesofmultiplecommadelimited
names.Stillwemustemphasize,modulengx_luaisamuchbetterchoicetoexecutethiskindofcomplicatedtasks,usuallyitismoreflexibleandmaintainable.
TillnowthetutorialcoverstheNginxvariable.Intheprocesswehavebeendiscussingmanybuiltinand3rdpartyNginxmodules,thesemoduleshelpusbetter
understandfeaturesandinternalsofNginxvariablebycomposingvariousminiconstructs.Lateronthetutorialwillbecoveringmoredetailsofthosemodules.
Withtheseexamples,weshouldunderstandthatNginxvariableplaysakeyroleintheNginxminilanguage:variablesarethewaysandmeansNginxcommunicate
internally,theycontainalltheneededinformation(includingtherequestinformation)andtheyarethecornerstoneelementswhichbridgeeveryotherNginxmodules.
Nginxvariablesareeverywhereinthecomingtutorials,understandthemisabsolutelynecessary.
Inthecomingtutorial"NginxDirectiveExecutionOrder",wewillbediscussingindetailtheNginxexecutionorderingandthephaseseveryrequesttraverses.It's
indispensabletounderstandthemsincefortheNginxminilanguage,theorderingofwritingcanbedramaticallydifferentfromtheorderingofexecutinginthetimeline.
ItusuallyconfusesmanyNginxusers.
Nginxdirectiveexecutionorder(01)
WhentherearemultipleNginxmodulecommandsinalocationdirective,theexecutionordercanbedifferentfromwhatyouexpect.BusyNginxuserswho
attempttoconfigureNginxby"trialanderror"maybeveryconfusedbythisbehavior.Thisseriesistouncoverthemysteriesandhelpyoubetterunderstandthe
executionorderingbehindthescenes.
Westartwithaconfusedexample:
?location/test{
?set$a32;
?echo$a;
?
?set$a56;
?echo$a;
?}
Clearly,we'dexpecttooutput32,followedby56.Becausevariable$ahasbeenresetaftercommandecho"isexecuted".Really?therealityis:
$curl'http://localhost:8080/test
56
56
Wow,statementset$a56musthavehadbeenexecutedbeforethefirstecho$acommand,butwhy?IsitaNginxbug?
No,thisisnotanNginxbug.WhenNginxhandleseveryrequest,theexecutionfollowsafewpredefinedphases.
Therecanbealtogether11phaseswhenNginxhandlesarequest,let'sstartwiththreemostcommonones:rewrite,accessandcontent(Theotherphaseswill
beaddressedlater.)
UsuallyanNginxmoduleanditscommandsregistertheirexecutioninonlyoneofthosephases.Forexamplecommandsetrunsinphaserewrite,andcommand
echorunsinphasecontent.Sincephaserewriteoccursbeforephasecontentforeveryrequestprocessing,itscommandsareexecutedearlieraswell.
Therefore,commandsetalwaysgetsexecutedbeforecommandechowithinonelocationdirective,regardlessoftheirstatementorderingintheconfiguration.
Backtoourexample:
set$a32;
echo$a;
set$a56;
echo$a;
Theactualexecutionorderingis:
set$a32;
set$a56;
echo$a;
echo$a;
It'sclearnow,twocommandssetareexecutedinphaserewrite,twocommandsechoareexecutedafterwardsinphasecontent.Commandsindifferentphases
cannotbeexecutedbackandforth.
Toprovethis,wecanenableNginx's"debuglog".
IfyouhavenotworkedwithNginx"debuglog"before,hereisabriefintroduction.The"debuglog"isdisabledbydefaultbecauseperformanceisdegradedwhenitis
enabled.Toenable"debuglog"youmustreconfigureandrecompileNginx,andsetthe--with-debugoptionforthepackage's./configurescript.When
buildingunderLinuxorM acOSXfromsource:
tarxvfnginx-1.0.10.tar.gz
cdnginx-1.0.10/
./configure--with-debug
make
sudomakeinstall
Incasethepackagengx_openrestyisused.Theoption--with-debugcanbeusedwithits./configurescriptaswell.
AfterwerebuildtheNginxdebugbinarywith--with-debugoption,westillneedtoexplicitlyusethedebugloglevel(it'sthelowestlevel)forcommand
error_log,inNginxconfiguration:
error_loglogs/error.logdebug;
debug,thesecondparameterofcommanderror_logiscrucial.Itsfirstparameteriserrorlog'sfilepath,logs/error.log.Certainlywecanuseanotherfilepath
butdorememberthelocationbecauseweneedtocheckitscontentrightaway.
Nowlet'srestartNginx(Attention,it'snotenoughtoreloadNginx.Itneedstobekilledandrestartedbecausewe'veupdatedtheNginxbinary).Thenwecansendthe
requestagain:
56
56
It'stimetocheckNginx'serrorlog,whichisbecomingalotmoreverbose(morethan700linesfortherequestinmysetup).Solet'sapplythegrepcommandtofilter
whatwewouldbeinterested:
grep-E'http(outputfilter|script(set|value))'logs/error.log
It'sapproximatelylikebelow(forclearness,I'veeditedthegrepoutputandremoveitstimestampetc):
[debug]5363#0:*1httpscriptvalue:"32"
[debug]5363#0:*1httpscriptset$a
[debug]5363#0:*1httpoutputfilter"/test?"
Itbarelymakesanysenses,doesit?Soletmeinterpret.Commandsetdumpstwolinesofdebuginfowhichstartwithhttpscript,thefirstlinetellsthevalue
whichcommandsethaspossessed,andthesecondlinebeingthevariablenameitwillbegivento,sofortheleadingfilteredlog:
Thesetwolinesaregeneratedbythisstatement:
set$a32;
Andforthefollowingfilteredlog:
Theyaregeneratedbythisstatement:
set$a56;
Besides,wheneverNginxoutputsitsresponse,its"outputfilter"willbeexecuted,ourfavoritecommandechoisnoexception.AssoonasNginx's"outputfilter"is
executed,itgeneratesdebugloglikebelow:
Ofcoursethedebuglogmightnothave"/test?",sincethispartcorrespondstotheactualrequestURI.Byputtingeverythingtogether,wecanfinallyconclude
thosetwocommandssetareindeedexecutedbeforetheothertwocommandsecho.
Consideratereadersmusthavenoticedthattherearethreelinesofhttpoutputfilterdebuglogbutwewerehavingonlytwooutputcommandsecho.Infact,
onlythefirsttwodebuglogsaregeneratedbythetwoechostatements.Thelastdebuglogisaddedbymodulengx_echobecauseitneedstoflagtheendofoutput.
TheflagoperationitselfcausesNginx's"outputfilter"tobeexecutedagain.M anymodulesincludingngx_proxyhassimilarbehavior,whentheyneedtogiveoutput
data.
Allright,therearenosurpriseswiththoseduplicated56outputs.Wearenotgivenachancetoexecuteechoinfrontofthesecondsetcommand.Luckily,wecanstill
achievethiswithafewtechniques:
location/test{
set$a32;
set$saved_a$a;
set$a56;
echo$saved_a;
echo$a;
}
Nowwehavewhatwehavewanted:
32
56
Withthehelpofanotheruservariable$saved_a,thevalueof$aissavedbeforeitisoverwritten.Becareful,theexecutionorderofmultiplesetcommandsare
ensuredtobeliketheirorderofwritingbymodule.Similarly,modulengx_echoensuresmultipleechocommandsgetexecutedinthesameorderoftheirwriting.
IfwerecallexamplesinNginxVariables,thistechniquehasbeenappliedextensively.ItbypassestheexecutionorderingdifficultiesintroducedbyNginxphased
processing.
Youmightneedtoask:"howwouldIknowthephaseaNginxcommandbelongsto?"Indeed,theanswerisRTFD.(SurelyadvanceddeveloperscanexaminetheC
sourcecodedirectly).M anymodulemarksexplicitlyitsapplicablephaseinthemodule'sdocumentation,suchascommandechowritesbelowinitsdocumentation:
phase:content
Itsaysthecommandisexecutedinphasecontent.Ifyouencountersamodulewhichmissestheapplicablephaseinthedocument,youcanwritetoitsauthorsright
awayandaskforit.However,weshallbereminded,noteverycommandhasanapplicablephase.ExamplesarecommandgeointroducedinNginxVariables(01)and
commandmapintroducedinNginxVariables(04).Thesecommands,whohavenoexplicitapplicablephase,aredeclarativeandunrelatedtotheconceptionof
executionordering.IgorSysoev,theauthorofNginx,hasmadethestatementsafewtimespublicly,thatNginxminilanguageinitsconfigurationis"declarative"not
"procedural".
We'vejustlearnt,allsetcommandswithinlocationareexecutedinrewritephase.Infact,almostallcommandsimplementedbymodulerewriteareexecuted
inrewritephaseunderthespecificcontext.CommadrewriteintroducedinNginxVariables(02)isoneofthem.However,weshallpointoutthatwhenthese
commandsarefoundinserverdirective,theywillbeexecutedinanearlierphasewe'venotaddressed:theserverrewritephase.
Commandset_unescape_uri,introducedinNginxVariables(02)isalsoexecutedinrewritephase.Actually,commandsimplementedbymodulengx_set_misccan
mixwithcommandsimplementedbymodulengx_rewriteandtheexecutionorderingisensured.Let'scheckanexample:
location/test{
set$a"hello%20world";
set_unescape_uri$b$a;
set$c"$b!";
echo$c;
}
Bysendingarequestaccordinglywehave:
helloworld!
Apparently,theset_unescape_uricommandanditsneighboringsetcommandsareallexecutedintheorderoftheirwriting.
Tofurtherdemonstrateourassertion,wecheckagainNginx"debuglog"(incaseit'sunclearforyouhowtocheck"debuglog",pleasereferencestepsfoundin(01)).
grep-E'httpscript(value|copy|set)'logs/error.log
Thedebuglogsarefilteredas:
[debug]11167#0:*1httpscriptvalue:"hello%20world"
[debug]11167#0:*1httpscriptvalue(postfilter):"helloworld"
[debug]11167#0:*1httpscriptset$b
[debug]11167#0:*1httpscriptcopy:"!"
[debug]11167#0:*1httpscriptset$c
Theleadingtwolines:
[debug]11167#0:*1httpscriptvalue:"hello%20world"
Theycorrespondtothecommand
Thefollowingtwolines:
[debug]11167#0:*1httpscriptvalue(postfilter):"helloworld"
[debug]11167#0:*1httpscriptset$b
Theyaregeneratedbycommand
Thereareminordifferencesinthefirstline,ifwecomparetothelogsgeneratedbycommandset:the"(postfilter)"addition.Intheendoftheline,URL
decodinghassuccessfullyexecutedaswewish."hello%20world"isdecodedas"helloworld".
Thelasttwolinesofdebuglog:
[debug]11167#0:*1httpscriptcopy:"!"
[debug]11167#0:*1httpscriptset$c
Theyaregeneratedbythelastsetcommand
set$c"$b!";
Asyoumighthavenoticed,since"variableinterpolation"isevaluatedwhenvariable$cisdeclaredandinitialized,thedebuglogstartswithhttpscriptcopy.In
theendofthelogitisthestringconstant"!"tobeconcatenated.
Withtheloginformation,it'sfairlyeasytotellthecommandexecutionordering:
set$c"$b!";
Itisaperfectmatchtothestatementsordering.
Justlikethecommandsimplementedinmodulengx_set_misc,commandset_by_luaimplementedin3rdpartymodulengx_lua,canmixwithcommandsofmodule
ngx_rewriteaswell.AsintroducedinNginxVariables(07),commandset_by_luasupportscomputationwithgivenLuacode,andassignsthecomputedresulttoa
Nginxvariable.Ascommandsetdoes,commandset_by_luadeclaresNginxvariablebeforeinitializationifthevariabledoesnotexist.
Let'scheckamixedexamplewhichcomprisescommandset_by_luaandset:
location/test{
set$a32;
set$b56;
set_by_lua$c"returnngx.var.a+ngx.var.b";
set$equation"$a+$b=$c";
echo$equation;
}
Variable$aand$bareinitializedwithnumericalvalue32and56respectively,thencommandset_by_luaisusedtogetherwithgivenLuacodetocomputethesumof
$aand$b.Variable$cisinitializedwiththecomputedvalue.Finally,variables$a,$band$careconcatenatedby"variableinterpolation"andassignstheresultto
variable$equation,whichisprintedbycommandecho.
Weshallpayattentiontoafewpointsintheexample:FirstlyNginxvariable$VARIABLEisreferencedasngx.var.VARIABLEinLuacode.Secondly,sinceNginx
variablesarestrings,thevalueofvariablengx.var.aandngx.var.bareactuallystrings"32"and"56",howevertheyareautomaticallyconvertedtonumerical
valuesbyLuaintheadditionoperation.ThirdlyLuacodereturnstoNginxvariable$cthecomputedsumvaluebystatementreturn.FinallywhenLuacode
returns,itactuallyconvertsthenumericalvaluebacktostring.(becausestringistheonlyvalidvalueforNginxvariable)
Theactualoutputmeetsourexpectation:
32+56=88
Thisinfactassertsthatcommandset_by_luacanmixwithcommandsimplementedbymodulengx_rewrite,suchasset.
M anyother3rdpartymodulessupportthemixwithmodulengx_rewriteaswell.Theexamplesincludemodulengx_array_var,discussedinNginxVariables(08)and
modulengx_encrypted_session,whichencryptssessions.Thelatterwillbestudiedindetailshortly.
Sincebuiltinmodulengx_rewriteisvirtuallyindispensable,it'sagreatadvantageforthe3rdpartymodulehasthecaliberofbeingmixedwith.Truthis,allofthose3rd
partymoduleshaveadoptedaspecialtechnique,whichallowsthe"injection"oftheirexecutionintocommandsofmodulerewrite(withthehelpofa3rdparty
modulengx_devel_kitdevelopedbyM arcusClyne).Fortherestregular3rdpartymodules,whichalsoregistertheirexecutioninphaserewrite,theircommandsare
executedseparatelyfrommodulengx_rewriteinruntime.Infact,it'shardlyaccuratetotellthecommandsexecutionorderinginbetweendifferentmodules(strictly
speakingtheyareusuallyexecutedintheorderofloading,butexceptiondoesexist).Forexamplebothmodules,AandBregistertheircommandstobeexecutedin
phaserewrite,thenitiseitherthecaseinwhichcommandsofAareexecutedfollowedbyBortheothercompletewayaround.Unlessitisexplicitlydocumented,
wecannotrelyontheuncertainorderinginourconfigurations.
Asdiscussedearlier,unlessspecialtechniquesareutilizedasmodulengx_set_miscdoes,amodulecannotmixitscommandswithngx_rewrite,andexpectsthecorrect
executionorder.Evenifthecommandsareregisteredintherewritephaseaswell.Wecandemonstratewithsomeexamples.
3rdpartymodulengx_headers_moreprovidesafewcommands,whichdealwiththecurrentrequestheaderandresponseheader.Oneofthemis
more_set_input_header.Thecommandcanmodifyagivenrequestheaderinrewritephase(oraddthespecificheaderifit'snotavailableincurrentrequest).As
describedinitsdocumentation,thecommandalwaysexecutesintheendofrewritephase:
phase:rewritetail
Beingtersethough,rewritetailmeanstheendofphaserewrite.
Sinceitexecutesintheendofphaserewrite,theimplicationisitsexecutionisalwaysafterthecommandsimplementedinmodulengx_rewrite.Evenifitis
writtenattheverybeginning:
?location/test{
?set$valuedog;
?more_set_input_headers"X-Species:$value";
?set$valuecat;
?
?echo"X-Species:$http_x_species";
?}
AsbrieflyintroducedinNginxVariables(02),Builtinvariable$http_XXXhastheheaderXXXforthecurrentrequest.Wemustbecarefulthough,variable
<$http_XXX>matchestothenormalizedrequestheader,i.e.itlowercasescapitallettersandturnsminus-intounderscore_fortherequestheadernames.
Thereforevariable$http_x_speciescansuccessfullycatchestherequestheaderX-Species,whichisdeclaredbycommandmore_set_input_header.
Becauseofthestatementordering,wemighthavemistakenlyconcludedheaderX-Specieshasthevaluedogwhen/testisrequested.Buttheactualresultis
different:
X-Species:cat
Clearly,statementset$valuecatisexecutedearlierthanmore_set_input_headers,althoughitiswrittenafterwards.
Thisexampletellsusthatcommandsofdifferentmodulesareexecutedindependentlyfromeachother,eveniftheyareallregisteredinthesameprocessingphase.
(unlessitisimplementedasmodulengx_set_misc,whosecommandsarespecificallytunedwithmodulengx_rewrite).Inotherwords,everyprocessingphaseis
furtherdividedintosub-phasesbyNginxmodules.
Similartomore_set_input_headers,commandrewrite_by_luaprovidedby3rdpartymodulengx_luaexecuteintheendofrewritephaseaswell.Wecanverifythis:
?location/test{
?set$a1;
?rewrite_by_lua"ngx.var.a=ngx.var.a+1";
?set$a56;
?
?echo$a;
?}
ByusingLuacodespecifiedbycommandrewrite_by_luaNginxvariable$aisincrementedby1.Wemighthaveexpectedtheresultbe56ifwearelookingatthe
writingsequence.Theactualresultis57becausecommandisalwaysexecutedafterallthesetstatements.
57
Admittedlycommandrewrite_by_luahasdifferentbehaviorthancommandset_by_lua,whichisdiscussedin(02).
Outofsheercuriosity,weshallaskimmediatelythatwhatwouldbeexecutionorderinginbetweenmore_set_input_headersandrewrite_by_lua,sincetheybothride
onrewritetail?Theansweris:undefined.Wemustavoidaconfigurationwhichreliesontheirexecutionorders.
Nginxphaserewriteisaratherearlyprocessingphase.Usuallycommandsregisteredinthisphaseexecutevariousrewritetasksontherequest(forexamplerewrite
theURLortheURLparameters),thecommandsmightalsodeclareandinitializeNginxvariableswhichareneededinthesubsequenthandling.Certainly,onecannot
forbidotherstocomplicatethemselvesbycheckingtherequestbody,orvisitadatabaseetc.Afterall,commandlikerewrite_by_luaoffersthecalibertostuffinany
potentiallymindtwistedLuacode.
Afterphaserewrite,Nginxhasanotherphasecalledaccess.Thecommandsprovidedby3rdpartymodulengx_auth_request,whichisdiscussedinNginx
Variables(05),executeinphaseaccess.CommandsregisteredinaccessphasemostlycarryoutACLfunctionalities,suchasguardinguserclearance,checkinguser
origins,examiningsourceIPvalidityetc.
Forexamplecommandallowanddenyprovidedbybuiltinmodulengx_accesscancontrolwhichIPaddresseshavetheprivilegestovisit,orwhichIPaddressesare
rejected:
location/hello{
allow127.0.0.1;
denyall;
echo"helloworld";
}
Location/helloallowsvisitfromlocalhost(IPaddress127.0.0.1)andrejectrequestsfromallotherIPaddresses(returnshttperror403)Therulesdefinedby
ngx_accesscommandsareassertedinthewritingsequence.Onceoneruleismatched,theassertionstopsandalltherestallowordenycommandsareignored.Ifno
ruleismatched,handlingcontinuesinthefollowingstatements.Ifthematchedruleisdeny,handingisabortedanderror403isreturnedimmediately.Inourexample,
requestissuedfromlocalhostmatchestotheruleallow127.0.0.1andhandingcontinuestotheotherstatements,howeverrequestissuedfromeveryotherIP
addresseswillmatchruledenyallhandlingisthereforeabortedanderror403isreturned.
Wecangiveitatest,bysendingrequestfromlocalhost:
$curl'http://localhost:8080/hello'
helloworld
Ifrequestissentfromanothermachine(supposeNginxrunsonIP192.168.1.101)wehave:
$curl'http://192.168.1.101:8080/hello'
<html>
<head><title>403Forbidden</title></head>
<bodybgcolor="white">
<center><h1>403Forbidden</h1></center>
<hr><center>nginx</center>
</body>
</html>
Bytheway,modulengx_accesssupportsthe"CIDRnotation"todesignateasub-network.Forexample169.200.179.4/24representsthesub-networkwhich
hastheroutingprefix169.200.179.0(orsubnetmask255.255.255.0)
Becausecommandsofmodulengx_accessexecuteinaccessphase,andphaseaccessisbehindrewritephase.Soforthosecommandswehavebeendiscussing,
regardlessofthewritingordertheyalwaysexecuteinrewritephase,whichisearlierthanallowordeny.Keepthisinmind,weshalltryourbesttokeepthe
writingandexecutionorderconsistent.
M odulengx_luaimplementsanothercommandaccess_by_lua.Thecommandallowsluacodetobeexecutedintheendofaccessphase,whichmeansitalways
executesafterallowanddenyeventheybelongtothesamephase.Inmanycases,weexaminetherequest'ssourceIPaddresswithngx_access,andusecommand
access_by_luatoexecutemorecomplicatedverificationswithLua.Forexamplebyqueryingadatabaseorotherbackendservices,thecurrentuser'sidentityand
privilegesareexamined.
Wecancheckasimpleexample,whichusescommandaccess_by_luatoimplementtheIPfilteringfunctionalityofmodulengx_access
location/hello{
access_by_lua'
ifngx.var.remote_addr=="127.0.0.1"then
return
end
ngx.exit(403)
';
echo"helloworld";
}
Nginx'sbuiltinvariable$remote_addrisreferencedinLuatogettheclient'sIPaddress.ThenLuastatementifisusedtodetermineiftheaddressequals127.0.0.1.
Luareturnsifitequals,Nginxthuscontinuesthesubsequenthandling(includingthecontentphasewherecommandechoappliesto).Ifitisnotthelocalhost
address,currenthandlingisabortedbyusingngx_luamodule'sLuafunctionngx.exitClientgetsahttperror403.
Theexampleisequivalenttotheotherexampleusingngx_accessmoduleintermsoffunctionality,whichwasdiscussedin(03):
location/hello{
allow127.0.0.1;
denyall;
echo"helloworld";
}
Howeverweshallpointout,performancewisethetwostillhavedifferences.M odulengx_accessperformsbetterbecauseitisspecificallyimplementedasaNginx
moduleinC.
Wecanmeasuretheperformancedifferencesofthetwo.Afterall,performanceiswhatweareafterbyusingNginx.Ontheotherhand,it'sabsolutelynecessarytobe
equippedwithmeasuringtechniques,becauseonlyactualdatadistinguishesamateursandprofessionals.Infact,bothngx_luaandngx_accessperformprettygoodfor
IPfiltering.Tominimizemeasuringerrorswecouldmeasuredirectlytheelapsedtimeofaccessphase.Traditionally,thismeanshackingNginxsourcecodewith
timingcodeandstatisticalcode,orrecompileNginxbinarysothatitcanbemonitoredbyspecificprofilingtoolslikeGNUgprof.
Wearelucky,becausecurrentreleasesofSolaris,M acOSXorFreeBSDofferasystemutilitydtrace,whichallowsmicromonitoringofuserprocessintermsof
performance(andfunctionalityaswell).Thetoolsparesusfromhackingsourcecodeorrecompilationwithprofiling.Let'sdemonstratethemeasuringscenarioonthe
M acBookAirbecausedtraceisavailablesinceM acOSX10.5
First,opentheTerminalapplicationofM acOSX,changetoyourpreferablepathandcreateafilenamedasnginx-access-time.d,editthefilewithfollowing
content:
#!/usr/bin/envdtrace-s
pid$1::ngx_http_handler:entry
{
elapsed=0;
}
pid$1::ngx_http_core_access_phase:entry
{
begin=timestamp;
}
pid$1::ngx_http_core_access_phase:return
/begin>0/
{
elapsed+=timestamp-begin;
begin=0;
}
pid$1::ngx_http_finalize_request:return
/elapsed>0/
{
@elapsed=avg(elapsed);
elapsed=0;
}
Savethefileandmakeitexecutable.
$chmod+x./nginx-access-time.d
The.dfileactuallycontainscodewritteninDlanguageofferedbyutilitydtrace(attention,theDlanguageisnottheotherDlanguage,whichisadvocatedbyWalter
BrightforabetterC++).SofarwecannotreallyexplainindetailthecodebecauseitrequiresathoroughunderstandingofNginxinternals.Anywayweshallbeclearof
thecode'spurpose:measurerequestsbeinghandledbyspecificNginxworkerprocessandcalculatetheaveragetimeelapsedinaccessphase.
NowwecangettheDscriptrunning.Thescripttakesacommandlineparameter,whichistheprocessid(pid)ofNginxworker.SinceNginxsupportsmultipleworker
processesandtherequestscanberandomlyhandledbyanyoneofthem,we'dliketoconfigureNginxinitsconfigurationnginx.confsothatonlyoneworkeris
requested.
worker_processes1;
AfterNginxbinaryisrestarted,theworkerprocessidcanbeobtainedbycommandps.
$psax|grepnginx|grepworker|grep-vgrep
Typicallywehave:
10975??S0:34.28nginx:workerprocess
10975ismyNginxworkerpid.Incaseyouhavemultiplelines,youmusthavestartedmultipleNginxserverinstancesorthecurrentNginxserverhasstarted
multipleworkerprocesses.
Thenasroot,scriptnginx-access-time.disexecutedwiththeworkerpid
$sudo./nginx-access-time.d10975
WeshallhaveoneoutputmessageifeverythinggoesOK.
dtrace:script'./nginx-access-time.d'matched4probes
ThemessagesaysourDscripthassuccessfullydeployed4probesonthetargetprocess.Thenthescriptisreadytotraceprocess10975constantly.
Let'sopenanotherTerminal,andsendmultiplerequestswithcurltoourmonitoredprocess
helloworld
helloworld
BacktoourTerminalwhereDscriptisrunning,presskeysCtrl-Ctointerruptit.Whenthescriptbailsoutitprintsonconsolethestatisticalresult.Forexample
myconsolehasfollowingresult:
$sudo./nginx-access-time.d10975
dtrace:script'./nginx-access-time.d'matched4probes
^C
19219
Thefinal19219istheaveragetimeelapsedinaccessphaseinnanoseconds(1second=1000x1000x1000nanoseconds)
Donewiththesteps.Wecanrunthenginx-access-time.dscripttocalculateaverageelapsedtimeinphaseaccessforthreedifferentNginxsetups
respectively.TheyareIPfilteringwithmodulengx_access,IPfilteringwithcommandaccess_by_lua,andfinallynofilteringforaccessphase.Thelastresulthelps
eliminatethesideeffectcausedbyprobesorother"systematicerrors".Besides,wecanusetrafficloadertoolssuchasabtosendshalfamillionrequeststominimize
"randomerrors",asbelow:
$ab-k-c1-n100000'http://127.0.0.1:8080/hello'
ThereforethestatisticalresultofDscriptisascloseaspossibletothe"actual"time.
IntheM acOSX,atypicalrunhasfollowingresults:
ngx_access18146
access_by_lua35011
nofiltering15887
Weminusthelastvaluefromtheformertwo:
ngx_access2259
access_by_lua19124
Well,modulengx_accessoutperformscommandaccess_by_luabyamagnitude,aswemighthaveexpected.Stilltheabsolutedifferenceistiny.FortheIntel
Core2Due1.86GHzCPUofmine,thereisonlyafewmicroseconds.
Infacttheaccess_by_luaexamplecanbefurtheroptimizedusingbuiltinvariable$binary_remote_addr.ThisvariablehastheIPaddressinbinaryformwhereas
variable$remote_addrhastheaddressinalongerstringformat.ShorteraddresscanbecomparedquickerwhenLuaexecutesitsstringoperations.
Becareful,if"debuglog"isenabledasintroducedin(01)thecomputedelapsedtimewillincreasedramatically,because"debuglog"hasahugeoverhead.
contentisbyallmeansthemostsignificantphaseinNginx'srequesthandling,becausecommandsrunninginthephasehavetheresponsibilitytogenerate"content"
andoutputHTTPresponse.Becauseofitsimportance,Nginxhasarichsetofcommandsrunninginit.Thecommandsincludeecho,echo_exec,proxy_pass,
echo_location,content_by_lua,whichwerediscussedinNginxVariables(02),NginxVariables(03),NginxVariables(05)andNginxVariables(07)respectively.
contentisaphasewhichrunslaterthanrewriteandaccess.Thereforeitscommandsalwaysexecuteintheendwhentheyareusedtogetherwithcommandsof
rewriteandaccess.
location/test{
#rewritephase
set$age1;
rewrite_by_lua"ngx.var.age=ngx.var.age+1";
#accessphase
deny10.32.168.49;
access_by_lua"ngx.var.age=ngx.var.age*3";
#contentphase
echo"age=$age";
}
Thisisaperfectexample,inwhichcommandsareexecutedinanexactsequenceastheyarewritten.Thetestingresultmatchestoourexpectationstoo.
age=6
Infact,thecommands'writingordercanbecompletelyshuffledanditwon'thaveanyimpacttotheirexecutionsequence.Commandset,whichisimplementedby
modulengx_rewrite,executesinrewritephase.Commandrewrite_by_luafrommodulengx_luaexecutesintheendofrewritephase.Commanddenyfrom
modulengx_accessexecutesinaccessphase.Commandaccess_by_luafrommodulengx_luaexecutesintheendofaccessphase.Finally,ourfavoritecommand
echo,implementedbymodulengx_echo,executesincontentphase.
TheexamplealsodemonstratesthecollaboratinginbetweencommandsrunningoneachdifferentNginxphase.Intheprocess,Nginxvariableisthedatacarrier
interconnectingcommandsandmodules.Theexecutionorderofthesecommandsislargelydecidedbythephaseeachappliesto.
Asmatteroffact,multiplecommandsfromdifferentmodulescouldcoexistinphaserewriteandaccess.Astheexampleshows,commandsetandcommand
rewrite_by_luabothbelongtophaserewrite.Commanddenyandcommandaccess_by_luabothbelongtophaseaccess.Howeveritisnotthesamestoryfor
phasecontent.
M ostmodules,whentheyimplementcommandsforphasecontent,theyareactuallyinserting"contenthandler"forthecurrentlocationdirective,however
therecanbeoneandonlyone"contenthandler"foralocation.Soonlyonemodulecouldbeattherestwhenmultiplemodulesarecontendingtherole.Consider
followingproblematicexample:
?location/test{
?echohello;
?content_by_lua'ngx.say("world")';
?}
Commandechofrommodulengx_echoandcommandcontent_by_luafrommodulengx_luabothexecuteinphasecontent.Butonlyoneofthemcouldsuccessfully
become"contenthandler":
world
Ourtestindicates,thatthewinneriscontent_by_luaalthoughitiswrittenafterwards,andcommandechoneverreallyhasachancetorun.Wecannotbeassured
whichmodulewinsinthecircumstance.Forexample,modulengx_echowinsandtheoutputbecomeshelloifweswapthecontent_by_luaandechostatements.So
weshallavoidtousemultiplecommandsforphasecontent,ifthecommandsareimplementedbydifferentmodules.
Theexamplecanbemodifiedbyreplacingcommandcontent_by_luawithcommandechoandwewillgetwhatweneed:
location/test{
echohello;
echoworld;
}
Againtestproves:
hello
world
Wecanusemultipleechocommands,thereisnoproblemwiththisbecausetheyallbelongtomodulengx_echo.M odulengx_echoregulatestheexecutionorderingof
them.Becarefulthough,noteverymodulesupportsthecommandsbeingexecutedmultipletimeswithinonelocation.Commandcontent_by_luaforaninstance,
canbeusedonlyonce,sofollowingexampleisincorrect:
?location/test{
?content_by_lua'ngx.say("hello")';
?content_by_lua'ngx.say("world")';
?}
Nginxdumpserrorfortheconfiguration:
[emerg]"content_by_lua"directiveisduplicate...
Thecorrectwayofdoingitis:
location/test{
content_by_lua'ngx.say("hello")ngx.say("world")';
}
Insteadofusingtwicethecontent_by_luacommandinlocation,theapproachistocallfunctionngx.saytwiceintheLuacode,whichisexecutedbycommand
content_by_lua
Similarly,commandproxy_passfrommodulengx_proxycannotcoexistwithcommandechowithinonelocationbecausetheybothexecuteincontentphase.
M anyNginxnewbiesmakefollowingmistake:
?location/test{
?echo"before...";
?proxy_passhttp://127.0.0.1:8080/foo;
?echo"after...";
?}
?
?location/foo{
?echo"contentstobeproxied";
?}
Theexampletriestooutputstrings"before..."and"after..."withcommandechobeforeandaftermodulengx_proxyreturnsitscontent.Howeveronlyone
modulecouldexecuteincontent.Thetestindicatesmodulengx_proxywinsandcommandechofrommodulengx_echoneverruns
contentstobeproxied
Toimplementwhattheexamplehadwantedto,weshallusetwoothercommandsprovidedbymodulengx_echo,echo_before_bodyandecho_after_body:
location/test{
echo_before_body"before...";
proxy_passhttp://127.0.0.1:8080/foo;
echo_after_body"after...";
}
location/foo{
echo"contentstobeproxied";
}
Testtellswemakeit:
before...
contentstobeproxied
after...
Thereasoncommandsecho_before_bodyandecho_after_bodycouldcoexistwithothermodulesincontentphase,istheyarenot"contenthandler"but"output
filter"ofNginx.Backin(01)whenweexaminethe"debuglog"generatedbycommandecho,we'velearntNginxcallsits"outputfilter"wheneverNginxoutputsdata.
Sothatmodulengx_echotakestheadvantageofittomodifycontentgeneratedbymodulengx_proxy(byaddingsurroundingcontent).Weshallpointoutthough,
"outputfilter"isnotoneofthose11phasesmentionedin(01)(manyphasescouldtrigger"outputfilter"whentheyoutputdata).Stillit'sperfectlyallrightto
documentcommandsecho_before_bodyandecho_after_bodyasfollowing:
phase:outputfilter
Itmeansthecommandexecutesin"outputfilter".
We'velearntin(05)thatwhenacommandexecutesincontentphaseforaspecificlocation,itusuallymeansitsNginxmoduleregistersa"contenthandler"for
thelocation.However,whathappensifnomoduleregistersitscommandas"contenthandler"forphasecontent?Whowillbetakingthegloryofgenerate
contentandoutputresponses?Theansweristhestaticresourcemodule,whichmapstherequestURItothefilesystem.Staticresourcemoduleonlycomesintoplay
whenthereisnone"contenthandler",otherwiseithandsoffthedutyto"contenthandler".
TypicallyNginxhasthreestaticresourcemodulesforthecontentphase(unlessoneormoreofthosemodulesaredisabledexplicitly,orsomeotherconflicting
modulesareenabledwhenNginxisbuilt)Thethreemodules,intheorderoftheirexecutionorder,arengx_indexmodule,ngx_autoindexmoduleandngx_static
module.Let'sdiscussthemonebyone.
M odulengx_indexandngx_autoindexonlyapplytothoserequestURI,whichendswith/.FortheotherrequestURIwhichdoesnotendwith/,bothmodules
ignorethemandletthefollowingcontentphasemodulehandle.M odulengx_statichowever,hasanexactoppositestrategy.ItignorestherequestURIwhich
endswith/andhandlestherest.
M odulengx_indexmainlylooksforaspecifichomepagefile,suchasindex.htmlorindex.htminthefilesystem.Forexample:
location/{
root/var/www/;
indexindex.htmindex.html;
}
Whenaddress/isrequested,Nginxlooksforfileindex.htmandindex.html(inthisorder)inapathinthefilesystem.Thepathisspecifiedbycommandroot.
Iffileindex.htmexists,Nginxjumpsinternallytolocationindex.htm;ifitdoesnotexistandfileindex.htmlexists,Nginxjumpsinternallytolocation
index.html.Iffileindex.htmldoesnotexisteither,andhandlingistransferredtotheothermodulewhichexecutesitcommandsinphasecontent.
WehavelearntinNginxVariables(02),commandsecho_execandrewritecantrigger"internalredirects"aswell.ThejumpmodifiestherequestURI,andlooksforthe
correspondinglocationdirectiveforsubsequenthandling.Intheprocess,phasesrewrite,accessandcontentarereiteratedforthelocation.The
"internalredirect"isdifferentfromthe"externalredirect"definedbyHTTPresponsecode302and301,clientbrowserwon'tupdateitsURIaddresses.Thereforeas
soonasinternaljumpoccurswhenmodulengx_indexfindsthefilesspecifiedbycommandindex,theneteffectislikeclientwouldhavebeenrequestingthefile'sURI
attheverybeginning.
Wecancheckfollowingexampletowitnessthe"internalredirect"triggeredbymodulengx_index,whenitfindstheneededfile.
location/{
root/var/www/;
indexindex.html;
}
location/index.html{
set$a32;
echo"a=$a";
}
Weneedtocreateanemptyfileindex.htmlunderthepath/var/www/,andmakesurethefileisreadablefortheNginxworkerprocess.Thenwecouldsend
requestto/:
$curl'http://localhost:8080/'
a=32
Whathappened?Whytheoutputisnotthecontentoffileindex.html(whichshallbeempty)?FirstlyNginxusesdirectivelocation/tohandleoriginalGET
/request,thenmodulengx_indexexecutesincontentphase,anditfindsfileindex.htmlunderpath/var/www/.Atthismoment,ittriggersan"internal
redirect"tolocation/index.html.
Sofarsogood.Butherecomesthesurprises!WhenNginxlooksforlocationdirectivewhichmatchesto/index.html,location/index.htmlhasa
higherprioritythanlocation/.ThisisbecauseNginxuses"longestmatchedsubstring"semanticstomatchlocationdirectivestorequestURI'sprefix.When
directiveischosen,phasesrewrite,accessandcontentarereiterated,andeventuallyitoutputsa=32.
Whatifweremovefile/var/www/index.htmlintheexample,andrequestto/again?Theansweriserror403Forbidden.Why?Whenmodulengx_index
cannotfindthefilespecifiedbycommandindex(index.htmlinhere),ittransfersthehandlingtothefollowingmodulewhichexecutesincontent.Butnoneof
thosefollowingmodulescanfulfilltherequest,Nginxbailsoutanddumpsuserror.M eanwhileitlogstheerrorinNginxerrorlog:
[error]28789#0:*1directoryindexof"/var/www/"isforbidden
Themeaningofdirectoryindexistogenerate"indexes".Usuallythisimpliestogenerateawebpage,whichlistseveryfileandsubdirectoriesunderpath
/var/www/.Ifweusemodulengx_autoindexrightafterngx_index,itcangeneratesuchapagejustlikewhatweneed.Nowlet'smodifytheexamplealittlebit:
location/{
root/var/www/;
indexindex.html;
autoindexon;
}
When/isrequestedagainmeanwhilefile/var/www/index.htmliskeptmissing.Anicehtmlpageisgenerated:
$curl'http://localhost:8080/'
<html>
<head><title>Indexof/</title></head>
<h1>Indexof/</h1><hr><pre><ahref="../">../</a>
<ahref="cgi-bin/">cgi-bin/</a>08-Mar-201019:36-
<ahref="error/">error/</a>08-Mar-201019:36<ahref="htdocs/">htdocs/</a>05-Apr-201003:55<ahref="icons/">icons/</a>08-Mar-201019:36</pre><hr></body>
</html>
Thepageshowsthereareafewsubdirectoriesundermy/var/www/.Theyarecgi-bin/,error/,htdocs/andicons/.Theoutputmightbedifferentifyou
havetriedbyyourself.
Again,iffile/var/www/index.hmtldoesexist,modulengx_indexwilltrigger"internalredirect",andmodulengx_autoindexwillnothaveachancetoexecute,you
maytestityourselftoo.
The"goalkeeper"moduleexecutedinphasecontentisngx_static.whichisalsousedintensively.Themoduleservesthestaticfiles,includingthestatic
resourcesofawebsite,suchasstatic.htmlfiles,static.cssfiles,static.jsfilesandstaticimagefilesetc.Althoughngx_indexcouldtriggeran"internal
redirect"tothespecifiedhomepage,buttheactualoutputtask(takesthefilecontentasresponse,andmarksthecorrespondingresponseheaders)iscarriedoutby
modulengx_static.
Let'scheckanexampleinwhichmodulengx_staticservesdiskfiles,withfollowingconfigurationsnippet:
location/{
root/var/www/;
}
M eanwhiletwofilesarecreatedunder/var/www/.Onefileisnamedindex.htmlanditscontentcontainsonelineoftextthisismyhome.Anotherfileis
namedhello.htmlanditscontentcontainsonelineoftexthelloworld.Againbeawareofthefiles'privilegesandmakesuretheyarereadablebyNginxworker
process.
Nowwesendrequeststothefiles'correspondingURI:
$curl'http://localhost:8080/index.html'
thisismyhome
$curl'http://localhost:8080/hello.html'
helloworld
Aswecansee,thecreatedfilecontentsaresentasoutputs.
Wecanexaminewhatishappeninghere:location/doesnothaveanycommandtoexecuteinphasecontent,thereforenomodulehasregistereda"content
handler"inthelocation.Thehandlingthusfallstothethreestaticresourcemoduleswhicharethelastresortsofphasecontent.Theformertwomodules
ngx_indexandngx_autoindexnoticesthattherequestURIdoesnotendwith/sotheyhandoffimmediatelytomodulengx_static,whichrunsintheend.
Accordingtothe"documentroot"specifiedbycommandroot,modulengx_staticmapstherequestURIs/index.htmland/hello.htmltodiskfiles
/var/www/index.htmland/var/www/hello.htmlrespectively.Asbothfilescanbefound,theircontentareoutputtedasresponse,meanwhileresponse
headerContent-Type,Content-LengthandLast-Modifiedareaccordinglyindicated.
Toverifymodulengx_statichasexecuted,wecouldenablethe"debuglog"introducedin(01).Againwesendrequestto/index.htmlandNginxerrorlogwill
containfollowingdebuginformation:
[debug]3033#0:*1httpstaticfd:8
Thislineisgeneratedbymodulengx_static.Itsmeaningis"outputtingstaticresourcewhosefilehandleis8".Ofcoursethenumericalfilehandlechangesevery
time,andthelineisonlyatypicaloutputinmysetup.Tobereminded,builtinmodulengx_gzip_staticcouldgeneratethesamedebuginfoaswell,bydefaultitisnot
enabledthough,whichwillbediscussedlater.
Commandrootonlydeclaresa"documentroot",itdoesnotenablesthengx_staticmodule.Themoduleisasmatteroffact,alwaysenabledalready,butitmight
nothavethechancetoexecute.Thisisentirelyuptotheothermodules,whichexecuteearlierincontentphase.M odulengx_staticexecuteonlywhenallof
themhave"gaveup".Toprovethis,checkfollowingblanklocationdefinition:
location/{
}
Becausethereisnorootcommand,Nginxcomputesadefault"documentroot"whenthelocationisrequested.Thedefaultshallbethehtml/subdirectoryunder
"configureprefix".Forexamplesupposeour"configureprefix"is/foo/bar/,thedefault"documentroot"is/foo/bar/html/.
Sowhodecides"configureprefix"?ActuallyittheNginxrootdirectorywhenitisinstalled(orthevalueof--prefixoptionofscript./configurewhenNginx
isbuilt).IfNginxisinstalledinto/usr/local/nginx/,"configureprefix"is/usr/local/nginx/anddefault"documentroot"istherefore
/usr/local/nginx/html/.Certainlyacommandlineoption--prefixcanbegivenwhenNginxisstarted,tochangethe"configureprefix"(sothatwecan
easilytestmultiplesetups).SupposeNginxisstartedasfollowing:
nginx-p/home/agentzh/test/
Forthisserver,its"configureprefix"becomes/home/agentzh/test/andits"documentroot"becomes/home/agentzh/test/html/.The"configure
prefix"notonlydetermines"documentroot",itactuallydeterminesthewaymanyrelationalpathresolutestoabsolutepathinNginxconfiguration.Wewillencounter
manyexampleswhichreference"configureprefix".
Infactthereisasimplewayoftellingcurrent"documentroot",whichistorequestanon-existedfile,Suchas:
$curl'http://localhost:8080/blah-blah.txt'
<html>
<head><title>404NotFound</title></head>
<center><h1>404NotFound</h1></center>
</body>
</html>
Naturally,the404errorpageisreturned.AgainwhenwecheckNginxerrorlog,weshallhavefollowingerrormessage:
[error]9364#0:*1open()"/home/agentzh/test/html/blah-blah.txt"failed(2:Nosuchfileordirectory)
Theerrormessageisprintedbymodulengx_static,sinceitcannotfindafileblah-blah.txtinitscorrespondingpath.Andbecausetheerrormessagecontains
theabsolutepath,whichngx_staticattemptstoopenwith,it'squiteobviousthatcurrent"documentroot"is/home/agentzh/test/html/.
M anynewbiesmighttakeitforgrantedthaterror404iscausedwhentheneededlocationdoesnotexist.Theformerexampletellsus,404errorcouldbereturned
eveniftheneededlocationisconfiguredandmatched.Thisisbecauseerror404meansthenon-existenceofanabstract"resource",notthespecificlocation.
Anotherfrequentmistakeismissingthecommandforphasecontent,whentheyactuallydon'texpectthedefaultstaticmodulestocomeintoplay,forexample:
location/auth{
access_by_lua'
--alotofLuacodeomittedhere...
';
}
Apparently,onlycommandsforphaseaccessaregivenfor/auth,whichisaccess_by_lua.Andithasnocommandsforphasecontent.Sowhen/authis
requested,theLuacodespecifiedinaccessphasewillexecute,thenthestaticresourcewillbeservedinphasecontentbymodulengx_static.Sinceitactually
looksforthefile/authonthedisknormallyitdumpsa404errorunlessweareluckilyandfile/authiscreatedonthecorrespondingpath.Sothethumbofrule,
whenerror404isencounteredundernostaticresourcecircumstances,weshallfirstcheckifthelocationhasproperlyconfigureditscommandsforphase
content,thecommandscanbecontent_by_lua,echoandproxy_passetc.Infact,Nginxerrorlogerror.logcouldonlygiveveryconfusingmessageforthecase.
Astheonesbelow,whichisfoundfortheaboveexample:
[error]9364#0:*1open()"/home/agentzh/test/html/auth"failed(2:Nosuchfileordirectory)
Sofarwehaveaddressedindetailrewrite,accessandcontent,whicharealsothemostfrequentlyencounteredphasesinNginxrequestprocessing.Wehave
learntmanyNginxmodulesandtheircommandsthatexecuteinthosephases,andit'scleartousthatthecommands'executionorderisdirectlydecidedbythephase
theyarerunningin.UnderstandingthephaseisourkeynoteforcorrectconfigurationwhichorchestratesvariousNginxmodules.Thereforelet'scovertherestphases
we'venotmet.
Asmentionedin(01),altogethertherecanbe11phaseswhenNginxhandlesarequest.Intheirexecutionorderthephasesarepost-read,server-rewrite,
find-config,rewrite,post-rewrite,preaccess,access,post-access,try-files,content,andfinallylog.
Phasepost-readistheveryfirst,commandsregisteredinthisphaseexecuterightafterNginxhasprocessedtherequestheaders.Similartophaserewritewe've
learntearlier,post-readsupportshooksbyNginxmodules.Built-inmodulengx_realipisanexample,ithooksitshandlerinpost-readphase,andforcefully
rewritetherequest'soriginaladdressasthevalueofaspecificrequestheader.Thefollowingcaseillustratesngx_realipmoduleanditscommandsset_real_ip_from,
real_ip_header.
server{
listen8080;
set_real_ip_from127.0.0.1;
real_ip_headerX-My-IP;
location/test{
set$addr$remote_addr;
echo"from:$addr";
}
}
TheconfigurationtellsNginxtoforcefullyrewritetheoriginaladdressofeveryrequestcomingfrom127.0.0.1tobethevalueoftherequestheaderX-My-IP.
M eanwhileitusesthebuilt-invariable$remote_addrtooutputtherequest'soriginaladdress,sothatweknowiftherewriteissuccessful.
Firstwesendarequestto/testfromlocalhost:
$curl-H'X-My-IP:1.2.3.4'localhost:8080/test
from:1.2.3.4
Thetestutilizes-Hoptionprovidedbycurl,theoptionincorporatesanextraHTTPheaderX-My-IP:1.2.3.4intherequest.Aswecantell,variable
$remote_addrhasbecome1.2.3.4inrewritephase,thevaluecomesfromtherequestheaderX-My-IP.SowhendoesNginxrewritetherequest'soriginal
address?yesit'sinthepost-readphase.Sincephaserewriteisfarbehindphasepost-read,whencommandsetreadsvariable$remote_addr,itsvaluehas
alreadybeenrewritteninpost-readphase.
Ifhowever,therequestsentfromlocalhostto/testdoesnothaveaX-My-IPheaderortheheadervalueisaninvalidIPaddress,Nginxwillnotmodifytheoriginal
address.Forexample:
$curllocalhost:8080/test
from:127.0.0.1
$curl-H'X-My-IP:abc'localhost:8080/test
from:127.0.0.1
Ifarequestissentfromanothermachineto/test,itoriginaladdresswon'tbeoverwrittenbyNginxeither,evenifithasaperfectX-My-IPheader.Itisbecauseour
previouscasemarksexplicitlywithcommandset_real_ip_from,thattherewritingonlyoccursfortherequestscomingfrom127.0.0.1.Thisfilteringmechanism
protectNginxfrommaliciousrequestssentbyuntrustedsources.Asyoumighthaveexpected,commandset_real_ip_fromcandesignateaIPsubnet(byusingCIDR
notationintroducedearlierin(03)).Besides,commandset_real_ip_fromcanbeusedmultipletimessothatwecansetupmultipletrustedsources,belowisan
example:
set_real_ip_from127.0.0.0/24;
Youmightbeasking,what'sthebenefitmodulengx_realipbringstous?Whywouldwerewritearequest'soriginaladdress?Theansweris:whentherequesthascome
throughoneormoreHTTPproxies,themodulebecomesveryhandy.Whenarequestisforwardedbyaproxy,itsoriginaladdresswillbecometheproxyserver'sIP
address,consequentlyNginxandtheservicesrunningonitwillnolongerhavetheactualsource.However,wecouldletproxyserverrecordtheoriginaladdressina
specificheader(suchasX-My-IP)andrecoveritinNginx,sothatitssubsequentprocessing(andtheservicesrunningonNginx)willtaketherequestasifitcomes
rightfromitsoriginaladdressandtheproxiesinbetweenaretransparent.Forthisexactpurpose,modulengx_realipneedshookhandlersinthefirstphase,thepostreadphase,sotherewritingoccursasearlyaspossible.
Behindpost-readistheserver-rewritephase.Webrieflymentionedin(02),whenmodulengx_rewriteanditscommandsareconfiguredinserverdirective,
theybasicallyexecuteinserver-rewritephase.Wehaveanexamplebelow:
server{
listen8080;
location/test{
set$b"$a,world";
echo$b;
}
set$ahello;
}
Attentiontheset$ahellostatementisputinserverdirective,soitrunsinserver-rewritephase,whichrunsearlierthanrewritephase.Therefore
statementset$b"$a,world'"inlocationdirectiveisexecutedafterwardsanditobtainsthecorrect$avalue:
hello,world
Sincephaseserver-rewriteexecuteslaterthanpost-readphase,commandsetinserverdirectivealwaysrunslaterthanmodulengx_realip,whichrewrites
therequest'soriginaladdress,example:
server{
listen8080;
real_ip_headerX-Real-IP;
location/test{
echo"from:$addr";
}
}
Sendrequestto/testwehave:
$curl-H'X-Real-IP:1.2.3.4'localhost:8080/test
from:1.2.3.4
Again,commandsetiswritteninfrontofcommandsofngx_realip,itsactualexecutionisonlyafterwards.Sowhencommandsetassignsvariable$addrinserverrewritephase,thevariable$remote_addrhasbeenoverwritten.
Rightafterserver-rewriteisthephasefind-config.ThisphasedoesnotallowNginxmodulestoregistertheirhandlers,insteaditisaphasewhenNginx
corematchesthecurrentrequesttothelocationdirectives.Itmeansarequestisnotcateredbyanylocationdirectiveuntilitreachesfind-config.
Apparently,forphaseslikepost-readandserver-rewrite,theeffectivecommandsarethosewhichgetspecifiedonlyinserverdirectivesandtheirouter
directives,becausethetwophasesareexecutedearlierthanfind-config.Thisexplainsthatcommandsofmodulengx_rewriteareexecutedinphaseserverrewriteonlyiftheyarewrittenwithinseverdirective.Similarly,theformerexamplesconfigurethecommandsofmodulengx_realipinserverdirectivetomake
surethehandlersregisteredinpost-readphasecouldfunctioncorrectly.
AssoonasNginxmatchesalocationdirectiveinthefind-configphase,itprintsadebuglogintheerrorlogfile.Let'scheckfollowingexample:
location/hello{
echo"helloworld";
}
IfNginxenablesthe"debuglog",adebuglogcanbecapturedinfileerror.logwheneverinterface/helloisrequested.
$grep'usingconfig'logs/error.log
[debug]84579#0:*1usingconfiguration"/hello"
Forthepurposeofconvenience,thelog'stimestamphasbeenomitted.
Afterphasefind-config,itisouroldbuddyrewrite.SinceNginxalreadymatchestherequesttoaspecificlocationdirective,startingfromthisphase,
commandswrittenwithinlocationdirectivesarebecomingeffective.Asillustratedearlier,commandsofmodulengx_rewriteareexecutedinrewritephasewhen
theyarewritteninlocationdirectives.Likewise,commandsofmodulengx_set_miscandmodulengx_lua(set_by_luaandrewrite_by_lua)arealsoexecutedin
phaserewrite.
Afterrewrite,itisthepost-rewritephase.Justlikefind-config,thisphasedoesnotallowNginxmodulestoregistertheirhandlerseither,insteaditcarries
outtheneeded"internalredirects"byNginxcore(ifthishasbeenrequestedinrewritephase).Wehaveaddressedthe"internaljump"conceptin(02),and
demonstratedhowtoissuethe"internalredirect"withcommandecho_execorcommandrewrite.However,let'sfocusoncommandrewriteforthemomentsince
commandecho_execisexecutedincontentphaseandbecomesirrelevanttopost-rewrite,theformerdrawsgreaterinterestbecauseitexecutesinrewrite
phase.Backtoourexamplein(02):
server{
listen8080;
location/foo{
set$ahello;
rewrite^/bar;
}
location/bar{
echo"a=[$a]";
}
}
Thecommandrewritefoundindirectivelocation/foo,rewritestheURIofcurrentrequestas/barunconditionally,meanwhile,itissuesan"internalredirect"
andexecutioncontinuesfromlocation/bar.Whatultimatelyintriguesus,isthemagicalbitsandpiecesof"internalredirect"mechanism,"internalredirect"
effectivelyrewindsourprocessingofcurrentrequestbacktothefind-configphase,sothatthelocationdirectivescanbematchedagaintotherequestURI,
whichusuallyhasbeenrewritten.Justlikeourexample,whoseURIisrewrittenas/barbycommandrewrite,thelocation/bardirectiveismatchedand
executionrepeatstherewritephasethereafter.
Itmightnotbeobvious,thattheactualactofrewindingtofind-configdoesnotoccurinrewritephase,insteaditoccursinthefollowingpost-rewrite
phase.Commandrewriteintheformerexample,simplyrequestsNginxtoissuean"internalredirect"initspost-rewritephase.Thisdesignisusuallyquestioned
byNginxbeginnersandtheytendtocomeupwithanideatoexecutethe"internaljump"directlybycommandrewrite.Theanswerhowever,isfairlysimple.The
designallowsURIberewrittenmultipletimesinthelocationdirective,whichismatchedattheverybeginning.Suchas:
location/foo{
rewrite^/bar;
rewrite^/baz;
echofoo;
}
location/bar{
echobar;
}
location/baz{
echobaz;
}
TherequestURIhasbeenrewrittentwiceinlocation/foodirective:firstlyitbecomes/bar,secondlyitbecomes/baz.Astheneteffectofbothrewrite
statements,"internalredirect"occursonlyonceinpost-rewritephase.Ifitwouldhaveexecutedthe"internalredirect"atthefirstURIrewrite,thesecondwould
havenochancetobeexecutedsinceprocessingwouldhaveleftcurrentlocationdirective.Toprovethiswesendarequestto/foo:
baz
Itcanbeassertedfromtheoutput,theactualjumpisfrom/footo/baz.WecouldfurtherprovethisbyenablingNginx"debuglog"andinterrogatethedebuglog
generatedinfind-configphaseforthematched:
[debug]89449#0:*1usingconfiguration"/foo"
[debug]89449#0:*1usingconfiguration"/baz"
Clearly,forthespecificrequest,Nginxonlymatchestwolocationdirectives:/fooand/baz,and"internaljump"occursonlyonce.
Quiteobviously,ifcommandngx_rewrite/rewriteisusedtorewritetherequestURIinserverdirective,therewon'tbeany"internalredirects",thisis
becausetheURIrewriteishappeninginserver-rewritephase,whichgetsexecutedearlierthanfind-configphasethatmatchesinbetweenthelocation
directives.Wecanchecktheexamplebelow:
server{
listen8080;
rewrite^/foo/bar;
location/foo{
echofoo;
}
location/bar{
echobar;
}
}
Intheexample,everyrequestwhoseURIstartswith/foogetsitsURIrewrittenas/bar.Therewritingoccursinserver-rewritephase,andtherequesthas
neverbeenmatchedtoanylocationdirective.OnlyafterwardsNginxexecutesthematchesinfind-configphase.Soifwesendarequestto/foo,location
/foonevergetsmatchedbecausewhenthematchoccursinfind-configphase,therequestURIhasbeenrewrittenas/bar.Solocation/baristheoneand
theonlyonematcheddirective.Actualoutputillustratesthis:
bar
Againlet'scheckNginx"debuglog":
[debug]92693#0:*1usingconfiguration"/bar"
Aswecantell,Nginxaltogetherfinishesoncethelocationmatch,andthereisno"internalredirect".
Afterpost-rewrite,itisthepreaccessphase.Justasitsnameimplies,thephaseiscalledpreaccesssimplybecauseitisexecutedrightbeforeaccess
phase.
Built-inmodulengx_limit_reqandngx_limit_zoneareexecutedinthisphase.Theformerlimitsthenumberofrequestsperhour/minute,andthelatterlimitsthe
numberofsimultaneousrequests.Wewillbediscussingthemmorethoroughlyafterwards.
Actually,built-inmodulengx_realipregistersitshandlerinpreaccessaswell.Youmightneedtoaskthen:"whydoitagain?Diditregisteritshandlersinpostreadphasealready".Beforetheanswerisuncoveredlet'sstudyfollowingexample:
server{
listen8080;
location/test{
echo"from:$remote_addr";
}
}
Comparingtotheearlierexample,themajordifferenceisthatcommandsofmodulengx_realiparewritteninaspecificlocationdirective.Aswehavelearntbefore,
Nginxmatchesitslocationdirectivesinfind-configphase,whichisfarbehindpost-read,hencetherequesthasnothingtodowithcommandswrittenin
anylocationdirectiveinpost-readphase.Backtoourexample,itisexactlythecasewherecommandsarewritteninalocationdirectiveandmodule
ngx_realipwon'tcarryoutanyrewriteoftheremoteaddress,becauseitisnotinstructedassuchinpost-readphase.
Whatifwedoneedtherewrite?Tohelpresolvetheissue,modulengx_realipregistersitshandlersinpreaccessagain,sothatitisgiventhechancetoexecuteina
locationdirective.Nowtheexamplerunsaswewould'veexpected:
from:1.2.3.4
Bereallycarefulthough,modulengx_realipcouldeasilybemisused,asourfollowingexampleillustrates:
server{
listen8080;
location/test{
echo"from:$addr";
}
}
Intheexample,weintroducesavariable$addr,towhichthevalueof$remote_addrissavedinrewritephase.Thevariableisthenusedintheoutput.Slowdown
righthereandyoumighthavenoticedtheissue,phaserewriteoccursearlierthanpreaccess,sovariableassignmentactuallyhappensbeforemodulengx_realip
hasthechancetorewritetheremoteaddressinpreaccessphase.Theoutputprovesourobservation:
from:127.0.0.1
Theoutputgivestheactualremoteaddress(nottherewrittenone)AgainNginx"debuglog"helpsassertittoo:
$grep-E'httpscript(var|set)|realip'logs/error.log
[debug]32488#0:*1httpscriptvar:"127.0.0.1"
[debug]32488#0:*1httpscriptset$addr
[debug]32488#0:*1realip:"1.2.3.4"
[debug]32488#0:*1realip:0100007FFFFFFFFF0100007F
Amongthelogs,thefirstlinewrites:
Thelogisgeneratedwhenvariable$remote_addrisfetchedbycommandset,string"127.0.0.1"isthefetchedvalue.
Thesecondlinewrites:
[debug]32488#0:*1httpscriptset$addr
ItindicatesNginxassignsvaluetovariable$addr.
Forthefollowingtwolines:
[debug]32488#0:*1realip:"1.2.3.4"
[debug]32488#0:*1realip:0100007FFFFFFFFF0100007F
Theyaregeneratedwhenmodulengx_realiprewritestheremoteaddressinpreaccessphase.Aswecantell,thenewaddressbecomes1.2.3.4asexpectedbutit
happensonlyafterthevariableassignmentandthat'salreadytoolate.
Nowthelastline:
Itisgeneratedwhencommandechooutputsvariable$addr,clearlythevalueistheoriginalremoteaddress,nottherewrittenone.
Somepeoplemightcomeupwithasolutionimmediately:"whatifmodulengx_realipregistersitshandlersinrewritephaseinstead,notinpreacccessphase?"
Thesolutionhoweveris,notnecessarilycorrect.Thisisbecausemodulengx_rewriteregistersitshandlersinrewritephasetoo,andwehavelearntin(02)thatthe
executionorder,underthecircumstances,cannotbeguaranteed,sothereisagoodchancethatmodulengx_realipstillexecutesitscommandsaftercommandset.
Alwayswehavethebackupoption:insteadofpreaccess,tryusengx_realipmoduleinserverdirective,itbypassesthebothersomesituationsencountered
above.
Afterphasepreaccess,itisanotheroldfriend,theaccessphase.Aswe'velearnt,built-inmodulengx_access,3rdpartymodulengx_auth_requestand3rdparty
modulengx_lua(access_by_lua)havetheircommandsexecutedinthisphase.
Afterphaseaccess,itisthepost-accessphase.Againasthenameimplies,wecaneasilyspotthatthephaseisexecutedrightafteraccessphase.Similarto
post-rewrite,thephasedoesnotallowNginxmoduletoregistertheirhandlers,insteaditrunsafewtasksbyNginxcore,amongthem,primarilyisthesatisfy
functionality,providedbymodulengx_http_core.
WhenmultipleNginxmoduleexecutetheircommandsinaccessphase,commandsatisfycontrolstheirrelationshipsinbetween.Forexample,bothmoduleAand
moduleBregistertheiraccesscontrolhandlersinaccessphase,wemayhavetwoworkingmodes,oneistoletaccesswhenbothAandBpasstheircontrol,the
otheristoletaccesswheneitherAorBpasstheircontrol.Thefirstoneiscalledallmode("AND"relation),thesecondoneiscalledanymode("OR"relation)By
default,Nginxusesallmode,belowisanexample:
location/test{
satisfyall;
denyall;
access_by_lua'ngx.exit(ngx.OK)';
echosomethingimportant;
}
Under/testdirective,bothngx_accessandngx_luaareused,sowehavetwomodulesmonitoringaccessinaccessphase.Specifically,statementdenyalltells
modulengx_accesstorejectsallaccess,whereasstatementaccess_by_lua'ngx.exit(ngx.OK)'allowsallaccess.Whenallmodeisusedwithcommand
satisfy,itmeanstoletaccessonlyifeverymoduleallowsaccess.Sincemodulengx_accessalwaysrejectsinourcase,therequestisrejected:
<html>
<head><title>403Forbidden</title></head>
<center><h1>403Forbidden</h1></center>
</body>
</html>
CarefulreadersmightfindfollowingerrorlogintheNginxerrorlogfile:
[error]6549#0:*1accessforbiddenbyrule
Ifhowever,wechangethesatisfyallstatementtosatisfyany.
location/test{
satisfyany;
denyall;
access_by_lua'ngx.exit(ngx.OK)';
}
Theoutcomeiscompletelydifferent:
somethingimportant
Therequestisallowedtoaccess.Becauseoverallaccessisallowedwheneveronemodulepassesthecontrolinanymode.Inourexample,modulengx_luaandits
commandaccess_by_luaalwaysallowtheaccess.
Certainly,ifeverymodulerejectstheaccessinthesatisfyanycircumstances,therequestwillberejected:
location/test{
satisfyany;
denyall;
access_by_lua'ngx.exit(ngx.HTTP_FORBIDDEN)';
}
Nowrequestto/testwillencounter403Forbiddenerrorpage.Intheprocess,the"OR"relationofaccesscontrolofeachaccessmodule,isimplementedin
post-access.
Pleasenotethatthisexamplerequiresatleastngx_lua0.5.0rc19orlater;earlierversionscannotworkwiththesatisfyanystatement.

Nginx Variables

Transcription

Similar documents

“I wanted People to use it, so I made it Open-Source”

Chronic Pain and Headache Management

Dr. Rychik`s slides

Student Essay Prize

Sensus Fidelium: Listening for the Echo

NGXINSIGHT

Vermivorous Vincent - Furman University Scholar Exchange

Johnny Knoxville - Furman University Scholar Exchange

to hack an asp .net website? hard, but possible!