Large (UDP) Packets in IPv6 - Labs

Large (UDP)
Packets in IPv6
Geoff Huston
APNIC
What’s the problem?
What’s the problem?
So what?
Packet Networks like
variable packet sizes
• Therangeofpacketsizessupportedinanetwork
representsasetofengineeringtrade-offs:
• Biterrorrateoftheunderlyingmedia
• Desiredcarriageefficiency
• Transmissionspeedvspacketswitchingspeed
IPv4 Packet Design
FORWARD fragmentation
• Ifaroutercannotforwardapacketonitsnexthopdue
toapacketsizemismatchthenitispermittedto
fragmentthepacket,preservingtheoriginalIPheaderin
eachofthefragments
IPv4 and the “Don’t Fragment”
bit
IfFragmentationisnotpermittedbythesource,thenthe
routerdiscardsthepacket.TheroutermaysendanICMP
tothepacketsourcewithanUnreacahble code(Type3,
Code4)
LaterIPv4implementationsaddedaMTUsizetothis
ICMPmessage
BUT:ICMPmessagesareextensivelyfilteredinthe
Internetsoapplicationsshouldnotcountonreceiving
thesemessages!
Trouble at the Packet Mill
• Lostfragsrequirearesendoftheentirepacket–
thisisfarlessefficientthanrepairingalostpacket
• Fragmentsrepresentasecurityvulnerabilityasthey
areeasilyspoofed
• Fragmentsrepresentaproblemtofirewalls–
withoutthetransportheadersitisunclearwhether
fragsshouldbeadmittedordenied
• Packetreassemblyconsumesresourcesatthe
destination
The thinking at the time…
FragmentationwasaBadIdea!
Kent,C.andJ.Mogul,"FragmentationConsideredHarmful",Proc.
SIGCOMM'87WorkshoponFrontiersinComputerCommunications
Technology,August1987
IPv6 Packet Design
• Attempttorepairtheproblembyeffectively
jammingtheDON’TFRAGMENTbittoalwaysON
• IPv6usesBACKWARDsignalling
• Whenapacketistoobigforthenexthoparouter
shouldsendanICMP6TYPE2(PacketTooBig)message
tothesourceaddressandincludetheMTUofthenext
hop.
IPv6 Source Fragmentation
What changed? What’s the
same?
• Bothprotocolsmayfragmentapacketatthesource
• BothprotocolssupportaPacketTooBigsignalfromthe
interiorofthenetworktothesource
• OnlyIPv4routersmaygeneratefragmentson-the-fly
• IPv6reliesonsupportforExtensionHeaderstosupportits
implementationofIPpacketfragmentation
• Butthathasitsownsetofimplications(Seeslide3)!
What does “Packet Too Big”
mean anyway?
errrrr
It’s a Layering Problem
• FragmentationwasseenasanIPlevelproblem
• Itwasmeanttobeagnosticwithrespecttotheupper
level(transport)protocol
• Butwedon’ttreatitlikethat
• Andweexpectdifferenttransportprotocolstoreactto
fragmentationnotificationindifferentways
What does “Packet Too Big”
mean anyway?
ForTCP itmeansthattheactivesessionreferredto
intheICMPpayload*shoulddropitssessionMSSto
matchtheMTU**
InanidealnetworkyoushouldneverseeIPv6fragments
inTCP!
*assumingthatthepayloadcontainstheoriginalend-to-endIPheader
**assumingthattheICMP isgenuine
What does “Packet Too Big”
mean anyway?
ForUDP itsnotclear:
• Theoffendingpackethasgoneaway!
• SomeIPimplementationsappeartoignoreit
• OthersaddahostentrytothelocalIPForwardingtable
thatrecordstheMTU
• OthersperformarudimentaryformofMTUreduction
inalocalMTUcache
Problems
ICMPisreadilyspoofed
• ICMPmessagescanconsumehostresources
• AnattackermayspoofahighvolumestreamofICMP
PTBmessages withrandomIPv6sourceaddresses
• AnattackermayspoofICMPPTBmessageswithverylow
MTUvalues
Problems
ICMPiswidelyfiltered
• leadingtoblackholesinTCPsessions
• GETisasmallHTTPpacket
• Theresponsecanbearbitrarilylarge,andifthereisapath
MTUmismatchtheresponsecanwedge
Get
Response
Problems
LeadingtoambiguityinUDP
• Isthislackofaresponseduetonetworkcongestion,
routing&addressingissues,orMTUmismatch?
• Shouldthereceiverjustgiveup,resendthetrigger
query,orreverttoTCP?(assumingthatitcan)
What did IPv6 do differently?
IPv6definedaminimumunfragmented packetsizeof
1,280bytes:
IPv6 Specification: RFC2460
What did IPv6 do differently?
IPv6definedaminimumunfragmented packetsizeof
1,280bytes:
IPv6 Specification: RFC2460
Bewteen 1280 and 1500
WhatshouldanIPv6hostuseasalocalMTUvalue?
1280
1500
• Ifyousetitat1280thenyouinvitefragmentationifyou
needtosendlargerpackets,whichwillriskEHlosson
fragmentedpackets
• Ifyousetitat1500thenyoumayencounterriskswithMTU
mismatchandPTBnotificationlosswhentalkingwithahost
withasmallerMTUandencounterMTUBlackHoles
Lets look
• SoiftheissueisthecombinationofIPv6,UDPand
largerpacketsthenperhapswecanexperiment
withthis
• It’scalled“theDNS”!
• Sowesetupanexperiment…
• Response1:131octets
• Response2:1400octets
• Response3:1700octets
AndsetupanameserverreachableonlyonIPv6andonly
onUDP
What we expect to see
SizeFetchedFailedReason
Small(150octets)
1160octets
1400octets
1700octets
99%
99%
?
<52%
1%
1%
?
>48%
Noise
Noise
PTB
EHLoss,
Fragloss,
PTB
What we saw
1280
1500
Tested
AlwaysFetched
Both
AlwaysMissed
150 11,719
8,792(75.02%)
377(3.22%)
2,550(21.76%)
1,160
2,004
1,353(67.51%)
5(0.25%)
646(32.24%)
1,400
9,789
7,374(75.33%)
385(3.93%)
2,030(20.74%)
1,425
1,977
1,313(66.41%)
7(0.35%)
657(33.23%)
1,453
1,987
1,298(65.32%)
9(0.45%)
680(34.22%)
172(1.54%)
5,139(46.01%)
1,70011,170
5,859(52.45%)
What we saw
1280
1500
Tested
AlwaysFetched
Both
AlwaysMissed
150 11,719
8,792(75.02%)
377(3.22%)
2,550(21.76%)
1,160
2,004
1,353(67.51%)
5(0.25%)
646(32.24%)
1,400
9,789
7,374(75.33%)
385(3.93%)
2,030(20.74%)
1,425
1,977
1,313(66.41%)
7(0.35%)
657(33.23%)
1,453
1,987
1,298(65.32%)
9(0.45%)
680(34.22%)
172(1.54%)
5,139(46.01%)
1,70011,170
5,859(52.45%)
??
Thereisquitesomenoiseinthisdata– thesmall-sizeresponse showsa21%
lossrate,whichislikelytobeduetoacombination of:
DNSmulti-slavequery engine farms
IPv6LinkLayeraddressmanipulation
ICMPv6Address unreachable
DNStimeouts
Unreachables
• DualStackconfigurationshideamultitudeofsins
• AndoneoftheseistheuseofunreachableIPv6
addressesforDNSresolvers
• 11,077distinctunreachableIPv6addresses!
• Outof22,000distinctIPv6/128addresses
• Whichisnotquiteasbadasitlooks– anumberofresolvers
are“aggressive”intheiruseof/64interfaceidentifiers
Filtering the results
• Joinindividualresolver/128addressesinto
common/64’s
• Onlylookatresolver/64’sthatfetcheitherofthe
twolow-sizecontrols
• WhichmeansthattheIPv6addressisreachable
• Andtheresolverwillsuccessfullyresolveaglueless
delegation
What we saw:
Size
Tested
AlwaysFetched
Both
AlwaysMissed
1505,4335,290(97%)143(3%)0
1,160654651(99%)3(1%)0
1280
14004,6584,495(96%)133(3%)30(1%)
1425636619(97%) 5(1%) 12(2%)
1453638609(95%) 6(1%) 23(4%)
1500
17004,6863,464(74%)79(1%)1,143(25%)
What we saw:
Size
Tested
AlwaysFetched
Both
AlwaysMissed
1505,4335,290(97%)143(3%)0
1,160654651(99%)3(1%)0
1280
14004,6584,495(96%)133(3%)30(1%)
1425636619(97%) 5(1%) 12(2%)
1453638609(95%) 6(1%) 23(4%)
1500
17004,6863,464(74%)79(1%)1,143(25%)
Between 1280 and 1500 the failure rate rises as the
packet size rises.
PTB MTU size distribution
1500
1480
1280
1460
1492
What we saw:
Size
Tested
AlwaysFetched
Both
AlwaysMissed
1505,4335,290(97%)143(3%)0
1,160654651(99%)3(1%)0
1280
14004,6584,495(96%)133(3%)30(1%)
1425636619(97%) 5(1%) 12(2%)
1453638609(95%) 6(1%) 23(4%)
1500
17004,6863,464(74%)79(1%)1,143(25%)
There is a visible signal here for packets > 1500 octets.
It is not a 48% drop rate, but it is certainly more than 20% over
and above the other packet sizes. There is a definite problem
here with large IPv6 packets.
EH drop? Or something more
mundane?
1,143 IPv6/64sconsistentlycannotfetcha1,700
octetUDPresponse
• 331/64’s generatedICMPFragmentationreassembly
ICMPmessages
• Firewallfrontenddiscardingtrailingfragments
• 61 /64’sgeneratedPacketTooBigmessages
• 751 failing/64’sgeneratednoICMPmessages
i.e.EHpacketdropwasamaximumof16%inthisexperiment
What we saw with a 1280 MTU:
Size
Tested
AlwaysFetched
Both
AlwaysMissed
1504,7774,600(96%)177(4%)0
1280
14004,6623,695(79%)80(2%)887(19%)
1500
17004,6353,429(74%)95(2%)1,111(24%)
Dropping the local MTU pushes a further 18% fragmentation drop
into the 1,400 Byte packet
What are we seeing?
WhetheritsEHdropoffragfiltering,thereis
somethingdeeplyconcerninginthesenumbers:
• Aprotocolthatsuffersa~20%packetdroprateon
fragmentedpacketspresentsaproblem!
• HostsshouldusethelargestlocallysupportedMTUfor
UDP(andusea1,220MSSforTCP)
• ApplicationsshouldassumethatlargeIPv6fragmented
packetsmaysilentlydieintransit.Theyshouldbe
preparedtoperformarapidcutovertoTCPintheevent
ofsuspectedpacketlossinUDP
• Shouldwerevivedraft-bonica-6man-frag-deprecate?
Thanks!