Interconnects - Past, Present, and Future

Transcription

Interconnects
Past, Present, and Future
Joel Goergen – Cisco Systems, Inc.
Version r2
Ethernet Technology Summit
April 2015 Santa Clara, CA
1
Ethernet Technology Summit April 2015 – Santa Clara, CA
•
What role does a SERDES play in the platform?
•
What are the 5 basic copper reaches today?
•
What is the electrical interconnect channel performance?
•
What modulation should be used for the higher speeds?
•
How much further can electrical go?
SERDES in the Platform
Past, Present, Future
3
1.25Gbps
2.5Gbps
3.125Gbps
6.25Gbps
** Channel count is double for
TX and RX
Mid 1993 - 1999
** C2C and C2M channels are in
2 to 4 Channels per line card to the fabric
addition to the fabric channels
4000-13 or Rogers material
Back Plane / Mid Plane
1998 – 2006
•
Platforms range 5 to 8 years
N6000-21 or Rogers material
•
Major undertaking / Major Cost
•
Takes advantage of new
•
3.125Gbps
6.25Gbps
10.325Gbps
SERDES, Materials, and
Connector technology
Involves smaller steps in ASIC
geometry
2003 – 2010
Broad range of materials
6.25Gbps
10.325Gbps
15.7Gbps
28Gbps
10+Gbps
15+Gbps
28Gbps
56Gbps
TX and RX
2009 – 2015
Back Plane / Mid Plane / Orthogonal
•
•
•
•
geometry
2011 – 2018
56Gbps
112Gbps
224Gbps
??Gbps
112Gbps
224Gbps
448Gbps
??Gbps
TX and RX
2016 – 2024
•
•
•
•
geometry
2018 – 2026
800++ Channels per line card to the fabric
•
The most popular SERDES cores are ones that significantly impact
platform performance per unit cost. 3.125Gbps, 10+Gbps, 25+Gbps
are all such cores. These are also cores that align well to 10GE and
100GE.
•
50+Gbps might not be as successful as 100+Gbps, given alignment to
100GE and 400GE. 50+Gbps will make an impact, but might be similar
to 6.25Gbps after 3.125Gbps.
•
100/200/400 SERDES path needs to start soon to capture new
platform starts.
Reach
Looking at the 5 basic reach definitions talked about most
8
IL
< 10mm/0.4in
1.5dB@14GHz
3dB@28GHz
Bump-to-bump
Inside MCM or
3D Stack
XSR
4dB@14GHz
8dB@28GHz
Ball-to-ball
Across PCB
< 200mm/7.9in
VSR
C2M
10dB@14GHz
20dB@28GHz
Ball-to-ball
< 500mm/19.7in
MR
C2C
20dB@14GHz
40dB@28GHz
Ball-to-ball
< 1000mm/39.4in
LR
C2F
35dB@14GHz
Ball-to-ball
< 50mm/2.0in
9
USR
IL
< 10mm/0.4in
Is it needed
for optical
Engines?
< 50mm/2.0in
USR
1.5dB@14GHz
3dB@28GHz
XSR
4dB@14GHz
8dB@28GHz
Ball-to-ball
Across PCB
From chip to
module
< 200mm/7.9in
VSR
C2M
10dB@14GHz
20dB@28GHz
Ball-to-ball
From chip to
chip
< 500mm/19.7in
MR
C2C
20dB@14GHz
40dB@28GHz
Ball-to-ball
< 1000mm/39.4in
LR
C2F
35dB@14GHz
Ball-to-ball
IL shown allows for: 1) 25 Gb/s NRZ or PAM4, 2) 50 Gb/s NRZ or PAM4, and 3) 100 Gb/s PAM4
10
Bump-to-bump
Inside MCM or
3D Stack
IL
< 10mm/0.4in
Is it needed
for optical
Engines?
< 50mm/2.0in
USR
1.5dB@14GHz
3dB@28GHz
XSR
4dB@14GHz
8dB@28GHz
Ball-to-ball
Across PCB
From chip to
module
< 200mm/7.9in
VSR
C2M
10dB@14GHz
20dB@28GHz
Ball-to-ball
From chip to
chip
< 500mm/19.7in
MR
C2C
20dB@14GHz
40dB@28GHz
Ball-to-ball
< 1000mm/39.4in
LR
C2F
35dB@14GHz
40dB@28GHz
Ball-to-ball
IL shown allows for: 1) 25 Gb/s NRZ or PAM4, 2) 50 Gb/s NRZ or PAM4, and 3) 100 Gb/s PAM4
But 1000mm is going to take lots of power - 35dB / 45dB @ 28 GHz will be expensive to achieve
11
Bump-to-bump
Inside MCM or
3D Stack
Notes:
 TE Channels developed using
HFSS/ADS modeling tools.
The 4” host trace is and should
be supported at mid-loss
materials by 100G standards… 7”
is certainly seen in designs, but
industries seem to recognize
and adjust with material and
design tradeoffs.
Propose: Use extended
CAUI4 C2M–like channels
for Modulation
discussion/comparison
12
It seems that -20dB is the
practical limit for electrical
transmission on shorter
reach copper, looking at
power per bit and die
area.
Electrical works most
effectively as long as we
stay above this level.
Less loss means a lower
power SERDES can be
implemented.
13
Notes:
 Channels are public on the .3bs
webpage… all channels include
connector and an 8% impedance
variation from motherboard to
daughtercard.
Are these channels
right to use for
modulation discussion?
Is ILD pessimistic for
educated 50G channel
design?
14
It seems that -35dB / 45dB is the Practical limit
for electrical transmission
on longer reach copper,
looking at power per bit
and die area.
Electrical works most
effectively as long as we
stay above this level.
Less loss means a lower
power SERDES can be
implemented.
15
Projecting Longer Reach and Fabric Channel
Loss Longer C2C and C2F Rough Channel Loss Limits
• Copper
options exist
for 28Gbps,
56Gbps, and
112Gbps.
0.0
-5.0
L
o -10.0
s
s -15.0
loss_28Gbps
-20.0
loss_56Gbps
i
n -25.0
loss_112Gbps
-30.0
loss_448Gbps
loss_224Gbps
d
B -35.0
Target Based on Modulation
Some Projections Target -40dB
-40.0
0
14
28
56
112
224
448
Frequency in Ghz
16
• Not much
industry effort
in 224Gbps
and 448Gbps
to determine
any level of
copper
suitability.
Projecting Shorter Reach Channel Loss
• Copper
options exist
for 28Gbps,
56Gbps, and
112Gbps.
Shorter C2C and C2M Rough Channel Loss Limits
0.0
L
-5.0
o
s
s -10.0
loss_28Gbps
loss_56Gbps
i
n -15.0
loss_112Gbps
loss_224Gbps
d -20.0
B
loss_448Gbps
Target Based on Modulation
-25.0
0
14
28
56
112
224
448
Frequency in Ghz
17
• Not much
industry effort
in 224Gbps
and 448Gbps
to determine
any level of
copper
suitability.
Electrical Channel
Performance
Defining the limits
18
Possible Implementations
-
System in package (SiP)
2.5D/3D Silicon interposer
Stacked die
Multi-chip module
Package-on-package
Offers system advantage/flexibility for routing and architecture.
Discussions still needed to have:
Reasonable to see C2EO interfaces commonly in industry by 2018
Die-to-Die definition (instead of ball to ball)?
Thermal Stability and proximity to ASIC die are key issues
19
No leap changes in this market from 100G Ethernet; albeit incremental changes
have are seen (materials, connectors, quality, system constraints, etc.)
Module Route*:
0.5”-2”
Host Route*: 1-4”
(up to 7” with low loss techniques)
Ball-to-ball definition (ball meaning BGA on the outside of the package)?
* Looking across the industry, across multiple platforms… typical channel length ranges shown.
20
• No leap changes in this market from
100G Ethernet; albeit incremental
changes have are seen (materials,
Stretches to 22”
connectors, quality, system
constraints, etc.)
Stretches to 15”
• However, there is a push in industry
to longer links… while making design
tradeoffs
Ball-to-ball definition (ball meaning
BGA on the outside of the package)?
http://www.ieee802.org/3/bm/public/may13/rabin
ovich_01_0513_optx.pdf
21
* Looking across the industry, across multiple platforms…
typical channel length ranges shown.
Fabric
PPU /NPU
C2C
C2F
0
X
25G NRZ
25G NRZ
25G NRZ
22
25G NRZ
7
25G NRZ
Mux
25G NRZ
SERDES Core
Fabric Interconnects
Optic Module
C2M
C2C
25G NRZ
40G/50G PAM4
25G NRZ
40G/50G PAM4
Possible
Re-Timer
between
here
0
Mux
Handles conversion
from what to what
8
15
25G NRZ
25G NRZ
SERDES Core
25G NRZ building blocks
What??
16 by 25G
8 by 50G
4 by 100G
Fabric
PPU /NPU
C2C
C2F
0
X
25G NRZ
50G PAM4
Optic Module
50G PAM4
C2M
C2C
25G NRZ
40G/50G PAM4
25G NRZ
40G/50G PAM4
Possible
Re-Timer
between
here
0
25G NRZ
50G PAM4
7
25G NRZ
50G PAM4
0
What??
SERDES Core
Fabric Interconnects
23
7
16 by 25G
8 by 50G
4 by 100G
Mux
50GPAM4
50G PAM4
What??
Mux
25G NRZ
50G PAM4
Mux
Handles conversion
from what to what
50GPAM4
Early Adopter
50G ???
25G NRZ / 50G ???
building block
building block
16 by 25G
8 by 50G
4 by 100G
Fabric
100 Gb/s PAM4
CDPPI
0
X
?? Tb/s
Optic Engine
PPU /NPU DIE
Optic Engine
0
X
400 Gb/s
4 by 100G
CDPPI
100 Gb/s PAM4
100 Gb/s PAM4
0
100 Gb/s PAM4
100 Gb/s PAM4
3
0
400 Gb/s
4 by 100G
CDPPI
100 Gb/s PAM4
100G SERDES
building block
24
100G SERDES
building block
3
Modulation
How to cover reach with the best coverage of Silicon SERDES cores
25
IL
S
E
R
D
E
S
C
o
R
e
T
y
p
e
N
R
Z
N
R
Z
P
A
M
4
N
R
Z
P
A
M
4
Not Really Practical
26
N
R
Z
P
A
M
4
N
R
Z
P
A
M
4
P
A
M
4
P
A
< 10mm/0.4in
P
M
A
4
P
M
A
4
P < 50mm/2.0in
M
A
4
M
4 < 200mm/7.9in
N
R
Z
N
R
< 500mm/19.7in
N
Z
R
N
< 1000mm/39.4in
Z
R
Z
USR
1.5dB@14GHz
3dB@28GHz
XSR
4dB@14GHz
8dB@28GHz
Ball-to-ball
Across PCB
VSR
C2M
10dB@14GHz
20dB@28GHz
Ball-to-ball
MR
C2C
20dB@14GHz
40dB@28GHz
Ball-to-ball
LR
C2F
35dB@14GHz
Ball-to-ball
Feasible Technology – Min
Compatibility Concern
Compatibility Concerns
Bump-to-bump
Inside MCM or
3D Stack
•
S
E
R
D
E
S
C
o
R
e
T
y
p
e
27
N
R
Z
P
A
M
4
N
R
Z
P
A
M
4
•
P
A
M
4
•
•
1 – NRZ for C2M / C2C
•
Long term covering C2F with PAM4 and covering USR with NRZ
•
Might be a challenge for C2M
•
Preserves SI test equipment and strategies in the lab
•
Optics can stay NRZ
•
Long term covering C2F with PAM4 and covering USR with NRZ
•
Compatibility between C2C and C2F
•
Complicates test equipment and SI strategy in some cases
•
Optics can stay NRZ
•
Compatibility across all reaches
•
Changes test equipment and SI strategies, but not sure the extent
•
Optics has to move to PAM4 unless a conversion step is specified
•
Perhaps the best use of power and die size for a long reach
•
Changes test equipment and SI strategies, but not sure the extent
•
Optics has to move to PAM4 unless a conversion step is specified
2 – NRZ for C2M and PAM4 for C2C
3 – PAM4 for C2M / C2C
4 – PAM4 for C2F
IL
S
E
R
D
E
S
N
R
Z
C
o
R
e
T
y
p
e
P
A
M
4
Not Really Practical
28
N
R
Z
P
A
M
4
N
R
Z
< 10mm/0.4in
P
A
M
4
1.5dB@14GHz
3dB@28GHz
6dB@56Ghz
Bump-to-bump
Inside MCM or
3D Stack
XSR
4dB@14GHz
8dB@28GHz
16dB@56GHz
< 200mm/7.9in
VSR
C2M
10dB@14GHz
20dB@28GHz
40dB@56GHz
< 500mm/19.7in
MR
C2C
20dB@14GHz
40dB@28GHz
Ball-to-ball
< 1000mm/39.4in
LR
C2F
35dB@14GHz
40dB@28GHz
Ball-to-ball
< 50mm/2.0in
P
A
M
4
USR
Feasible Technology – Min
Compatibility Concern
Compatibility Concerns
Ball-to-ball
Across PCB
Ball-to-ball
• PAM4 for ALL
S
E
R
D
E
S
C
o
R
e
T
y
p
e
29
• Compatibility across all reaches
P
A
M
4
• Changes test equipment and SI
strategies, but not sure the
extent
• Optics has to move to PAM4
unless a conversion step is
specified
How Fast Can Electrical
Go?
How to cover reach with the best coverage of Silicon SERDES cores
30
IL
S
E
R
D
E
S
C
o
R
e
T
y
p
e
31
P
A
M
4
40dB@100GHz
Bump-to-bump
Inside MCM or
3D Stack
XSR
40dB@100GHz
Ball-to-ball
Across PCB
< 200mm/7.9in
VSR
C2M
10dB@14GHz
20dB@28GHz
40dB@56GHz
< 500mm/19.7in
MR
C2C
20dB@14GHz
40dB@28GHz
Ball-to-ball
< 1000mm/39.4in
LR
C2F
35dB@14GHz
40dB@28GHz
Ball-to-ball
< 10mm/0.4in
< 50mm/2.0in
USR
Feasible Technology –
Maybe
Ball-to-ball
S
E
R
D
E
S
C
o
R
e
T
y
p
e
32
P
A
M
4
P
A
M
4
?
?
?
• PAM4 for USR / XSR
• Unless there is a significant
leap in Silicon, Materials,
Modulation or FEC, copper
may be done here
•
Copper may hold its own up to 100 Gb/s using PAM4.
•
After 100 Gb/s, there may exist opportunities for reaches up to 400
Gb/s. Everything else has to be optical.
•
After 400 Gb/s, there really isn’t an electrical reach.
•
To be fair – in July 2005, I said the copper limit was 25 Gb/s. Today, I
think we can achieve 100 Gb/s, with a very few 400 Gb/s
opportunities.
Thank you!
34

Interconnects - Past, Present, and Future

Transcription

Similar documents

DKS Brochure2 - BoardUp - Front

New Wafer Level Stacking Technologies and Their Applications

With Madhvi and Asim Mehta`s Group

10x10 Oct 11 Newsline Ad Santa`s Arrival

The Final Day

1947

Student`s Worksheet Christmas One

AWESOME VILLA IN SANTA PONSA