Multimedia – 3D Video Research - EECS International Graduate

Multimedia –
3D Video Research
Hsueh-Ming Hang 杭學鳴
Electronics Eng. Dept 電子工程系
National Chiao Tung Univ 國立交通大學
Hsinchu, TAIWAN
Hsueh-Ming Hang 杭學鳴











1984: Ph.D. in EE, RPI (Rensselaer Polytechnic Inst), NY, USA
1976,1980: BS and MS, Chiao Tung Univ, Hsinchu
1991.11~: Nat’l Chiao-Tung Univ., Professor
2005.8~2006.8/2010.2~: NCTU, ECE College, Associate Dean
2006.8~2009.7: Nat’l Taipei Univ Tech, EECS College, Dean
1998.11~2004.7: NCTU, Telecom Research Center, Director
1984.6~1991.11: AT & T Bell Labs., USA; MTS
2008: IET Fellow
2002: IEEE Fellow
2000: IEEE Third Millennium Medal
Associate Editor, IEEE Trans Image Processing (1992-1994, 2008-12),
IEEE Circuits and Systems for Video Technology (1997-1999)
Sept 2013
hmhang/EE, NCTU
2
Digital Media in Daily Life
MP3
MPEG2
Sept 2013
JPEG
MPEG4
JPEG
MP3
MPEG4
hmhang/EE, NCTU
3
Matrix “Bullet – Time”
-- 120 cameras
Sept 2013
hmhang/EE, NCTU
4
Multi-View System

Synthesize a “new” view at “any” 3D location from a
given set of views – Virtual-view Synthesis
Dense Camera
Array:
Disparity:5~40 pixels
Virtual View Synthesis
Sept 2013
hmhang/EE, NCTU
5
NCTU Virtual Studio (Hakka College)
Sept 2013
hmhang/EE, NCTU
6
3D Scene Composition

Implement Chroma key on 2 natural scenes!
&
X: 860 Y: 768
RGB: 124, 112, 70
X: 1381 Y: 944
RGB: 102, 106, 91
Sept 2013
hmhang/EE, NCTU
7
X: 860 Y: 768
RGB: 124, 112, 70
August 2013
hmhang/EE, NCTU
8
Some Lab Equipment



28-view Autostereoscopic 3D
display
9-camera Array: synchronized
PC cluster
Sept 2013
9
hmhang/EE, NCTU
Research Topics

(1) Multimedia Algorithms (Standards!)
-- 3-D video analysis, synthesis, and composition
-- 3-D image/video quality assessment: human
perceptual modeling

(2) Multimedia Compression and
Implementation
-- HEVC (High Efficiency Video Coding, H.265)
-- 3D-HEVC
Sept 2013
10
hmhang/EE, NCTU
Wanted!
Background: S&S, DSP, Communication
 Skills: Programming, Algorithms, (Math)
 Self-motivated
 Self-study
 Write thesis, papers, ..
 Give oral reports
 Techniques in problem solving
 Knowledge in multimedia

Sept 2013
11
hmhang/EE, NCTU
Research Roadmap Driven by Network Benchmarking Lab (NBL):
Deep Packet Inspection, Traffic Forensics, Embedded Benchmarking,
Software Defined Networking (SDN)
林盈達 Ying-Dar Lin, IEEE Fellow
Dept of Computer Science, NCTU
[email protected]
www.cs.nctu.edu.tw/~ydlin
www.nbl.org.tw
9-14-2013
1











B.S., NTU-CSIE, 1988; Ph.D., UCLA-CS, 1993

Professor (1999~)/Associate Professor (1993~1999), NCTUCS; IEEE Fellow (2013)
Founder and Director, III-NCTU Embedded Benchmarking
Lab (EBL; www.ebl.org.tw), 2011~
Founder and Director, ITRI-NCTU Network
Benchmarking Lab (NBL; www.nbl.org.tw), 2002~
Editorial Boards: IEEE Wireless Comm. (2013~), IEEE
Transactions on Computers (2011~), IEEE Computer
(2012~), IEEE Network (2011~), IEEE Communications

Magazine – Network Testing Series (2010~), IEEE
Communications Letters (2010~), Computer
Communications (2010~), Computer Networks (2010~) ,
IEEE Communications Surveys and Tutorials (2008~),
IEICE Transactions on Information and Systems
(11/2011~), IEICE Transactions on Communications
(5/2012~)
CEO, Telecom Technology Center (www.ttc.org.tw),
7/2010~5/2011
Director, Computer and Network Center, NCTU, 2007~2010
Consultant, ICL/ITRI, 2002~2010

Visiting Scholar, Cisco, San Jose, 7/2007-7/2008
Director, Institute of Network Engineering, NCTU,
2005~2007
Co-Founder, L7 Networks Inc. (www.L7.com.tw), 2002
Areas of research interests

Deep Packet Inspection
 Attack, virus, spam, porno, P2P
 Software, algorithm, hardware, SoC
 Real traffic, beta site, botnet

Internet security and QoS

Wireless communications

Test technologies of switch, router, WLAN,
security, VoIP, 4G/LTE and embedded systems
Publications

International journal: 92

International conference: 50

IETF Internet Draft: 1

Industrial articles: 153

Textbooks: 4 (Ying-Dar Lin, Ren-Hung Hwang,
Fred Baker, Computer Networks: An Open
Source Approach, McGraw-Hill, Feb 2011)

Patents: 26

Tech transfers: 8
Well-cited paper: Multihop Cellular: A New
Architecture for Wireless Communications,
INFOCOM 2000, YD Lin and YC Hsu; #citations: 600;
standardized into WLAN mesh, Bluetooth, WiMAX,
and LTE-advanced
2
From Development to Research
• Sources of research topics
1.
Literature repository: minor improvement on existing or pseudo
problems
2. Development projects: feasible solutions on real problems
3. Industrial discussions: real problems but not necessarily feasible
solutions
• D(development)  R(research)
 Enabling resource: Linux
 Research is the non-trivial part within the development process.
 If I don’t know how to develop it, I would not research on it.
• Roadmap and footprints: cable TV networks (1996-1999) 
multi-hop cellular (1998-2000)  QoS (1998~2003)  deep
packet inspection (2004~2009)  traffic forensics (2008~)
 embedded benchmarking (2011~)  software defined
networking (SDN, 2013~)
3
System Research with Three Side Products
7-in-1
Linux QoS Router Security L7 Inc.
Development
Gateway Startup
Public Testing
with a Magazine
Network Benchmarking Lab (NBL)
RealFlow
Embedded Benchmarking Lab (EBL)
4G LTE
Development Plane
1996
1998
2000
2002
QoS
Cable TV
Networks
Multi-hop
Cellular
Research Plane
Computer Networks:
An Open Source Approach
2004
2008
2011 2012 2013
Deep Packet Inspection
Traffic Forensics
Embedded
Benchmarking
& 4G LTE
4
Development Plane:
L7 Networks Inc.
Computer Networks: An Open Source Approach
Research Plane:
QoS
Deep Packet Inspection (DPI)
THE BLUE TRACK
5
7-in-1 System Prototyping and Benchmarking
•
•
•
7-in-1: VPN, Firewall, NAT, Routing, Content Filtering, Intrusion
Detection, Bandwidth Management
Launched a startup in 2002: L7 Networks Inc.
Appeared in IEEE Communications Surveys & Tutorials, 3rd quarter
2002, http://speed.cis.nctu.edu.tw/~ydlin/wei.pdf
LAN/DMZ
WAN
LAN/DMZ to WAN Outbound Traffic
MAC
Filter
Redirect
Y
In-LAN
Filter
Policy
Route
Route
Out-WAN
Filter
Y
Out-LAN
Filter
IPsec
VPN
Bandwidth
Mgt.
Y
FTP/POP3/SMTP/
Web/URL Filter with
Many-to-One NAT
Bandwidth
Mgt.
NAT
Alerting
System
Intrusion
Detection
Y
Route
In-WAN
Filter
WAN to DMZ/LAN Inbound Traffic
Redirect
sniff
Y
deNAT
IPsec
deVPN
6
Ying-Dar Lin, Ren-Hung Hwang, Fred Baker, Computer
Networks: An Open Source Approach, McGraw-Hill,
Feb 2011.
www.mhhe.com/lin; available now at amazon.com
Facebook Q&A Communit: www.facebook.com/CNFBs
ISBN: 0-07-337624-8 / 978-007-337624-0
Computer Networks: An Open Source Approach
considers why a protocol, designed a specific way, is
more important than how a protocol works. Key
concepts and underlying principles are conveyed
while explaining protocol behaviors. To further bridge
the long-existing gap between design and
implementation, it illustrates where and how protocol
designs are implemented in Linux-based systems. A
comprehensive set of fifty-six live open source
implementations spanning across hardware (8B/10B,
OFDM, CRC32, CSMA/CD, and crypto), driver
(Ethernet and PPP), kernel (longest prefix matching,
checksum, NAT, TCP traffic control, socket, shaper,
scheduler, firewall, and VPN), and daemon
(RIP/OSPF/BGP, DNS, FTP, SMTP/POP3/IMAP4, HTTP,
SNMP, SIP, streaming, and P2P) are interleaved with
the text.
7
7
Quotes from Reviewers:
•
•
•
•
•
•
“The exposure to real life implementation details in this book is phenomenal...
Definitely one of the better books written in the area of Computer Networks.” –
Mahasweta Sarkar, San Diego State University
“I have never seen a book giving such details on explaining the design and
implementation of such practical systems...Those open source implementations
are excellent demonstrations for practical networking systems.” – Fang Liu,
University of Texas-Pan American
“This is a solid textbook with strong emphasis on technical (implementation)
details of computer network protocols.” – Oge Marques, Florida Atlantic
University
“Written by RFC and open source contributors, this book definitely is an
authentic guide for network engineers.” – Wen Chen, Cisco Fellow
“Interleaving designs and implementations into the same book bridges the longexisting gap and makes this an ideal text to teach from.” – Mario Gerla,
University of California, Los Angeles
“The sidebars of Historical Evolution and Principle in Action make the reading
more enjoyable, while Performance Matters treat computer networking
quantitatively.” – H. T. Kung, Harvard University
8
Development Plane:
Network Benchmarking Lab (NBL)
Embedded Benchmarking Lab (EBL)
Broadband Mobile Lab (BML): 4G LTE
Research Plane:
Traffic Forensics
Embedded Benchmarking & 4G LTE
THE GREEN TRACK
9
NBL Solutions
編號
1
2
3
4
5
技術名稱
自動控制測試系統 - ACTS (Automatically Controlled Test System)
真實流量錄製與重播工具 – ILLT (In-Lab Live Testing)
真實流量資料庫 – PCAP Lib
測試涵蓋率分析與最佳化之技術 – Test Coverage Analysis and Optimization
惡意程式收集分析之工具與資料庫 – Malware Tool-chain and Malware Lib
6
無線區域網路流量與訊號之錄製與重播工具 – WLAN Capture and Replay of Traffic
and Environment
7
第四代無線行動通訊之協定測試環境 – LTE Conformance and Interoperability
Testing
8
第四代無線行動通訊之多重輸入出之測試環境與工具 – LTE MIMO OTA (Over-theAir)
9
手持裝置耗時耗電與穩定度之自動測試工具 – Android AKL (Automatic Key
Logger)
10
NBL Industrial Customers

服務業界廠商超過100家、測試產品超過600件
11
A Spin-Off: EBL (Embedded Benchmarking Lab)
AKL (stand-alone)
Dynamic Multi-Level Profiler
Cross Layer Bottleneck
Detector
Bottleneck Analyzer
Android System
H-Profile
Power Measurer (System Level)
Battery Use Extension (App Level)
Power Memo (Function Level)
12
LTE 4-stage Testbeds
•
•
•
•
Stage 1
 eNB emulator
 Test purposes
 Conformance Test
 Design verification
Stage 2
 eNB/EPC of diff. vendors
 Test purposes
 Interoperability Test
 Capacity verification
Stage 3
 OTA chamber/channel emulator
 Test purposes
 Operator-IOT
 Performance test for mobile devices (CTIA)
Stage 4
 Experimental band in NCTU campus
 Test purposes
 Field Trials
 BetaSite
13
Software Defined Networking for Future Internet
14
Summary
 From development to research




Embedded systems: performance, power consumption, GUI
Network security: malware, APT (Advanced Persistent Threats)
Wireless communications: 4G LTE, micro/macro-cell
Software defined networking (SDN): architectures, test lab
 Fundamental problems in emerging areas




Micro/macro-cell modeling: femto, relay, base stations
SDN architecture modeling: mapping switches to controllers
Serial/parallel multi-filter modeling: false positive/negative
Optimal scheduling in cloud computing: 2-tier projects/ jobs
 Call for students
 Highly motivated
 Skillful in (1) hands-on, (2) writing, and/or (3) math
 Graduated and current international students: India, Ukraine, Vietnam, Thailand,
Burkina Faso, El Salvador, etc.
 Lab locations: Lab 701 (701 EECS Building), NBL (601 EECS Building), LTE
(103 EECS Building)
15
Research Overview
Carrson C. Fung, Ph.D.
Assistant Professor
IMOD Group
Communication Electronics and Signal Processing Lab (CommLab)
Dept. of Electronics Engineering
[email protected]
http://cwww.ee.nctu.edu.tw/~cfung
IMOD Group



Learn about modeling and optimal
design for signal processing related
problems
Weekly group and joint group (with
other CommLab members) meetings
Recent funded projects

Integrated project (2011 – 2013) (with 5
other professors)


Individual project (2013 – 2014)




Opportunistic transmission techniques for
cooperative cognitive communications
Coordinated and cooperative techniques for
enhanced communications
Possible summer internship abroad
(dependent on performance and
funding)
Small group setting (1 M.S. + 1 Ph.D.)
Possible to get jobs with skills you
learned in my group

Skills that employers desire and cannot be
easily replaced
http://cwww.ee.nctu.edu.tw/~cfung
When can I
transmit??
When can I
transmit??
2
Current Scope of Research

Signal processing research

Robust detection and classification






Applications include radar, intruder detection,
cognitive sensing
“Channel” model mismatch, noisy signal
Parametric vs. non-parametric (clustering)
One vs. many sensors
Centralized vs. distributed
Flexible radio
network
F1
W1
Signal processing for communications

Training sequence design for channel
estimation



Fp3




Wp3
Quantized design
Channel parameters mismatch
Interference mitigation (greedy/selfish) and
management (cooperative)

primary
network
Centralized vs. distributed
Channel aging
CSI mismatch
Opportunistic transmission
For security as well
(Eavesdropper)
Eve
noise
noise
Alice
Bob
http://cwww.ee.nctu.edu.tw/~cfung
3
The Pitch




If you like to do interesting work in signal
processing…
If you like mathematics…
If you like opportunities to think out of the box
(explore your innovative side)…
If you like close interaction with your adviser…
THEN MY GROUP IS FOR YOU!!!
Stop by and talk to me (ED 639)!
[email protected]
http://cwww.ee.nctu.edu.tw/~cfung
4
The Internet of Things & M2M
Communications
Presented by Juan Jose Celada
on Behalf of
Prof. Fuchun Joseph Lin 林甫俊教授
[email protected]
Office EC 125C
Department of Computer Science
1
Vision of Connected World
• Everything that can be connected will be connected
to the global Internet.
• These “things” form the “Internet of Things (IoT)” to
offer human smart services.
• The IoT involves “machine-to-machine (M2M)
communications” where machines (a.k.a. things)
interact with other machines or men to provide
services.
• Such a connected world greatly improves human life.
Major applications include smart home, smart grid,
connected car and eHealth etc.
2
Gaps in Achieving the Vision
• Lack of addresses and bandwidth has limited the
development of the IoT and M2M in the past.
• Most M2M is thus restricted to a limited, local area.
• With the widespread of mobile broadband and IPv6,
these gaps can now be overcome.
• It is predicted that on the planet earth there will be
50 billions of connected devices by 2020, each with
an IPv6 address.
• It is also predicted that in average, everyone of us
will be associated with at least 10 connected devices.
3
Achieving Connected World
with M2M
M2M
Common
Service
Layer
Mobile
Core
Networks
M2M Cloud
Wireless
Sensor
Network
Source: ETSI M2M TC
4
Research Focus and Interest
•
•
•
•
•
•
M2M Architecture and Protocols
M2M Common Service Layer
Core Network Optimization for M2M
M2M Gateways
M2M Cloud
Big Data for M2M
5
For Further Information
• Connected World Technologies Laboratory
http://cwt.nctu.edu.tw/
• National Science Council M2M Deep Plowing
Project
http://m2mdpc.nctu.edu.tw/
• Prof. Fuchun Joseph Lin’s Home Page
http://people.cs.nctu.edu.tw/~fuchun_lin/
6
Nelson G. Chen
Assistant Professor
Department of Electrical and Computer Engineering
National Chiao Tung University
September 14, 2013
陳稷康
電機系助理教授
國立交通大學
102年9月14日 [email protected]
My work
• Nanoparticle assisted ultrasound therapy
(NAUT)
– Tumors collect more nanoparticles than regular
cells
– When exposed to ultrasound, cancerous cells
preferentially killed
– Work in collaboration with Academia Sinica
Results
BEAS-2B – normal cells; A-549-cancer cells
My work (continued)
– Dynamics and mechanism of effect is what?
• Spot heating? (Overall temperature didn’t change)
• Additional cavitation from nucleation?
– Safety of imaging with nanoparticles in body?
• Ultrasound levels were in diagnostic range
• Such range is safe without other items; but known to
create effects with microbubbles
• Nanoparticle assisted ultrasound imaging
– See nanoparticles or clusters of nanoparticles?
– With cancer-specific drugs, see tiny tumors?
Other planned projects
• Development of ways to • Behavior based
perform ultrasound CT
impairment detector
– Ultrasound CT produces
images with decreased
speckle, and about 1
order of magnitude
increase in resolution
– Currently impractical
due to data size,
processing time, and
transducer positioning
– Alcohol/drug
consumption with
driving/operating
dangerous machinery
illegal and dangerous
– Development of series of
tasks to estimate
sobriety
– Ethically and legally test
with volunteers
Charles T. M. Choi, PhD (蔡德明教授)
Institute of Biomedical Engineering & Electrical
Engineering Dept. (生醫工程研究所/電機系)
[email protected]




Cochlear implants for treatment of severe to profound
hearing impairment patients
Brain surgery navigation & deep brain stimulation for
treatment of Parkinson’s disease, epilepsy, chronic
depression
Electrical Impedance Imaging Technology for real time lung
ventilation imaging, breast cancer detection, brain stroke
detection.
Related patents granted: 10 (US/TW/CHN); patents
pending 26
1
Biomedical/Electrical Engineering,, NCTU