Deep Learning Image Classification for Histopathology.pptx

Deep Learning Image Classification
for Histopathology
Boris Murmann, Sean Fischer, Elaina Chai
November 19, 2015
2
ImageNet Large Scale Visual Recognition Challenge
http://image-net.org
http://devblogs.nvidia.com/parallelforall/mocha-jl-deep-learning-julia/
3
Best-Performing Topology: Convolutional Neural Network
Simple Example: Handwritten Digits
http://parse.ele.tue.nl/cluster/2/CNNArchitecture.jpg
§  Training: Use massive amounts of data to find the parameter set that
achieves the best possible fit
§  Testing: Classify input data using the trained model parameters
›  Our main interest
4
Software versus Hardware Camp
www.qualcomm.com/zeroth
§  Cloud computing versus mobile computing
§  CPU/GPU clusters versus custom chips
§  Virtually unconstrained power versus severely constrained power/energy
5
“Brain-Inspired” Computing Hardware
Degree of
brain
inspiration
6
Memory
Signaling
Computation
Analog
Analog Spikes
Analog
Carver Mead, ca. 1990
Digital
Digital Spikes
Digital
IBM True North, 2014
“Sloppy”
Digital
Mixed-signal
Mixed-signal
à Our interest
Digital
Digital
Digital
DianNao, 2014
Example Application: Lane Tracking
Unlabelled
Input Image
7
Output Image
with Label Points
Example Network Structure
Huge number of
coefficients
Power ~ 100W
on a GPU board
A. Coates
2014
8
Far to high for
practical
applications
Custom Machine Learning Chip with Embedded Memory
Area: 67.7 mm2
§  Read from 256-bit, 36MB eDRAM: ~19 pJ
§  Remaining issue: Can’t fit enough memory on chip
§  Ongoing work on “network compression” will solve this issue
Reference: Y. Chen, MICRO 2014
9
Pushing Further: Mixed-Signal Techniques for Machine Learning
Low-swing interconnect
ConvNet
Training
Trained
parameters
Bit flips and
quantization
error
Software
10
Hardware
MixedSignal
Compute
Blocks
Low
Voltage
Memory
w/ error
Deterministic
coupling
between bits
Charge Domain Dot Product Kernel
§  Externally digital, internally
analog compute block
§  Amenable for digital CMOS
VLSI integration
§  ADC energy amortized over
several multipliers
QP
§  Multiply via multi-phase charge
redistribution
§  Add via passive charge sharing
among multipliers
SIGN
+
8Cu
Cu
vOD
4Cu
SIGN
2Cu
Cu
−
QN
11
2Cu
VDD/2
8Cu
§  Small unit caps < 1fF
4Cu
16x 8-bit MAC Design in 28nm CMOS
§  Energy stored in capacitors essentially negligible
§  Total energy dominated by ADC and control wires/switches
§  Next step is to work on low-swing control and improved ADC
12
The Machine Learning Team
The Bio “Team”
Architecture
Memory
Signaling
ALU
ALU
VX∙Y
DX,1 DY,1
DX,2 DY,2
ADC
DX,N-1 DY,N-1 DX,N DY,N
Mixed-signal compute
13
DOUT,X∙Y
§ 
§ 
§ 
Grade III
Grade II
Grade I
Histological Features for Cancer Prognosis
Clear cellular differentiation,
Consistent cell size and shape
Normal mitotic activity
Subjective
curve
§ 
§ 
§ 
No clear cellular differentiation
Irregular cell size and shape
Accelerated mitotic activity
D.H. Patey, R.W. Scarff, The position of histology in the prognosis of carcinoma of the breast. Lancet 211, 801-804 (1928)
14
Manual Inspection Remains the Gold Standard
Relationship between histological grade and overall survival in 1830 breast cancer patients
Grade I
Log-Rank Test
% Survival
Grade II
Grade III
χ2 = 198.06
P < 0.0001
Time [years]
C.W. Elston, I.O. Ellis, Pathological prognostic factors in breast cancer I. The value of histological grade in breast cancer:
Experience from a large study with long-term follow-up. Histopathology 19, 403-410 (1991)
15
The Rise of Digital Histopathology
Whole Slide Scanner
Stanford Tissue Microarray Database
(http://tma.stanford.edu)
MikroScan D2
Array Block TA-274 Breast #07b VGH-011A
v3 (slice 1.00) - 10395
Growing repositories of digitized tissue samples enables machine learning in histopathology.
16
Machine Learning Uncovering New Prognostic Features
Training Images (NKI)
Testing Images (VGH)
Model Prediction
C-Path Model on VGH (n=286)
Alive
Training
Deceased
C-Path
Survival Model
New Prognostic
Features
Alive
Log-Rank P = 0.001
Low-Risk
High-Risk
Years
A. H. Beck, A. R. Sangoi, S. Leung, R. J. Marinelli, T. O. Nielsen, M. J. van de Vijver, R. B. West, M. van de Rijn, D. Koller.
"Systematic analysis of breast cancer morphology uncovers stromal features associated with survival." Science
translational medicine 3.108 (2011)
17
Eight New Epithelial Features for Cancer Prognosis
Epithelial tissue lines any surface in organs and blood vessels throughout the body.
Eight Epithelial Features on VGH
σ[σ[pixel intensity]/Mean[intensity in epi. nuclei]]
Min[elliptic fit of epi. cont. regions]
Sum of unclassified objects
σ(|epi. cytoplasmic obj. and nuclear obj.|)
σ(Max[blue pixel atypical epi nuclei])
Mean[border between epi. cytoplasmic obj.]
Max[|atypical epi. nuclei|]
Max[Min[green pixel intensity in epi. regions]]
Alive
Log-Rank P = 0.02
Low-Risk
High-Risk
Years
A. H. Beck, A. R. Sangoi, S. Leung, R. J. Marinelli, T. O. Nielsen, M. J. van de Vijver, R. B. West, M. van de Rijn, D. Koller.
"Systematic analysis of breast cancer morphology uncovers stromal features associated with survival." Science
translational medicine 3.108 (2011)
18
Three New Stromal Features for Cancer Prognosis
Stromal tissue provides structural support and serves as a matrix in which other cells are embedded.
Intensity difference between
stromal region and neighbors
Presence of stromal
regions without nuclei
Three Stromal Features on VGH
Low
Risk
Log-Rank P = 0.004
Alive
High
Risk
Low
Risk
High
Risk
Avg. relative border of stromal spindle
nuclei to stromal round nuclei
Low-Risk
High-Risk
Years
A. H. Beck, A. R. Sangoi, S. Leung, R. J. Marinelli, T. O. Nielsen, M. J. van de Vijver, R. B. West, M. van de Rijn, D. Koller.
"Systematic analysis of breast cancer morphology uncovers stromal features associated with survival." Science
translational medicine 3.108 (2011)
19
The Future of Histopathology?
Real time digitally assisted analysis
Fully digital analysis
New prognostic/diagnostic feature sets advance basic research
20
Summary
§  Machine learning has seen renewed interest in recent years
§  Impressive progress in software development and classification accuracy
§  Less impressive progress in hardware implementation of the most
promising networks (convolutional neural networks)
›  But we are hoping to fix this…
§  Given these trajectories, we hope to take steps toward fully digital
histology
›  Initial focus on algorithms, software
›  Potential end-game solution is a low-cost, power efficient custom
machine/computer
21