A Variety of Terminals

Embedded and Realtime Linux Conference
Valencia, 10 November 2003
The fundamental role of open systems
in future applications
Andrea Cuomo, Vice President
Advanced System Technology
A Variety of Terminals
Advanced System Technology
STMicroelectronics
1
A Variety of Networks
AdAd-HocHoc-Net’s
WLAN
2.5 G, 3G, 4G
GPS
DVBDVB-S
BASE STATION
Advanced System Technology
STMicroelectronics
A Variety of Applications
•
•
•
•
•
•
•
•
•
•
•
•
Interactive games
Positioning
MMMessaging
Navigation
Web surfing
Safety
E-Commerce
E-Business
MMBroadcasting
P2P
……
…….
Advanced System Technology
STMicroelectronics
2
But One Single Convergence Layer
Presentation
Any Application
Source
Transport
IP
MAC
L1
RF
Any Network
Advanced System Technology
STMicroelectronics
Defining Convergence
MULTIMEDIA
CONNECTIVITY
CONVERGENCE
STORAGE
Advanced System Technology
MOBILITY
SECURITY
STMicroelectronics
3
Convergence: a Practical Example:
Multimedia Over Wireless
VOICE TECHNOLOGY
IMAGE SENSOR
Speech recognition
Text to speech
Biometric verification
Repartition for lower system cost &
power
Alternative sensor I/F for power
VGA & CIF
ACOUSTIC TECHNOLOGY
GRAPHICS
New Function
AHB1
2D: PIM/PDA, basic action games
3D: fully texture games
CUBA IF
ARM926EJ-S Subsystem
AHB1-M
Quiet zone microphone (selective lobe
microphone)
Noise/Echo cancellation
Enhancement to speech recog.
Voice intelligibility (Noise dependent
equalization)
D-Cache
D-SRAM
VIDEO COMPRESSION
I-Cache
ARM926EJ-S
Core
I-SRAM
MPEG4: QCIF to PAL resolution
H263
MPEG4 Hardware Engine
DIGITAL AUDIO
Decoder (MP3, MPEG1,2 L1&2 DTS,
DOLBY DIGITAL,AAC, PAC)
Encoder (MP3, MPEG L1&2,AAC)
3D surround (SRS, …)
Audio pre/post processing
Secure digital music
APB Bridge 1
DISPLAY
DMA Controler
AHB1-S AHB1-M APB-S
SECURITY/DRM SUPPORT
Video response STN with advanced
low power drive
Multimode power/quality
Optimum refresh partitioning for sub
mW power
LOCALIZATION
GPS
Cellular based terminals
Keypad
IMAGING
Integration allows multiple video
<<types>>
Optimized for local feedback from
MPEG4
JPEG Codec
CONNECTIVITY
Bluetooth
802.11
USB
Advanced System Technology
STMicroelectronics
Moore’s Law
Transistors/chip
10G
4Gb
1G
1Gb
100M
(DRAM)
256Kb
64Kb
100K
16Kb
4Kb
4004 8080
1970
1975
Advanced System Technology
P8
16Mb
4Mb
1Mb
1M
1K
256Mb
64Mb
Memory
10M
10K
Memory
CD (µm)
0.10
0.15
0.25
0.4
0.6
0.8
1.2
2.0
3.0
80386
68020
80286
68000
8086
8085
P6
Pentium
80486,68040
Microprocessor
1980
1985
1990
1995
2000
5.0
2005 Year
STMicroelectronics
4
Convergence & SoCs
Go Hand In Hand
Connectivity
Storage
Connectivity
Content
elaboration
(base band)
MMI
Security
Storage
MMI
Connectivity
Security
Power
management
Advanced System Technology
Content
elaboration
(base band)
Connectivity
Power
management
STMicroelectronics
A Huge IP's Portfolio
Video
MPEG2/4 CoDecs
Motion Estimation
Transcoding
Memory Reduction
Imaging
Audio
MP2, AAC, MP3
AC3
VoiceRec
Best
Implementation
2D/3D GFX Engines
...
- Performance
- Cost
- Flexibility
- Power
- Quality
Graphics
HDD
UDrive
DataPlay
Storage
SmartCard
Secure Content
Distribution
Security
SYMBIAN
WCE
MiddleWare, Java
SW
Advanced System Technology
CMOS Sensors
Picture Generation & Enhan.
Enhanced JPEG/JPEG2000
Display Dependent Processing
BlueTooth
GPS
OFDM (Wi-Fi)
Connectivity
STMicroelectronics
5
The Future of Embedded Systems
Music
CONTENT
News
Media Player
APPLICATIONS
PLATFORMS
Real
…
Linux
E-mail
Games
DRM
Messaging
Intertrust Playmedia …
Windows
Microcore
Banking
Symbian
SMS
Palm
MMS …
VxWorks
…
Browsing
Browser
…
…
Media Accelerator
HARDWARE
Secure Hardware
Advanced System Technology
Memory
STMicroelectronics
Two Paths forward
BRUTE FORCE
• GIPS
10’s GIPS
100’s GIPS
• …
ALL IN SOFTWARE
Advanced System Technology
SMART
• IDENTIFY SYSTEM INVARIANT
• HARDWIRE INVARIANTS
SOFTWARE TO PERSONALIZE
THE VARIABLE PART
STMicroelectronics
6
IC Design Challenges
Growing Cost per IC
R&D
cost
per
device
The Verification Challenge
10,000x more vectors
required to validate
100x / 6 years
Exponential
growth
2002
10B
1996
100M
1M
100 x 10,000 =
1 million times
more simulation load
1990
100k
1M
10M
10x / 6 years
100x Gate Count
1990
1994
1997
2000
Verification effort growth outpaces design
Process Impact on Design
1.0
0.8
0.7
0.5 0.35 0.25 0.18 0.15 0.13
Area
Speed
Power
MetalMigration
Signal
Integrity
EMI
Advanced System Technology
STMicroelectronics
The Algorithmic Driving Force
Shannon asks for more than Moore can deliver...
Algorithmic Complexity
(Shannon’s Law)
10000000
1000000
Processor Performance
(Moore’s Law)
3G
100000
10000
2G
1000
100
10
Advanced System Technology
20
20
20
16
20
12
20
08
20
04
20
00
19
96
19
92
19
88
19
84
19
80
1
Battery Capacity
1G
STMicroelectronics
7
The Power Challenge
1000
Rocket
Nozzle
Nuclear Reactor
P4 @ 1.4GHz, 75W
Watts/cm2
100
P5
P4
Sun’s
surface
Pentium III ®
Hot plate
10
Pentium II ®
PentiumPro ®
Pentium ®
i386
i486
1 1.5µ
1µ
0.7µ
0.5µ
0.35µ
0.25µ
0.18µ 0.13µ 0.1µ
0.07µ 0.05µ
Courtesy of Fred Pollack, Intel
Keynote speech, MICRO-32
Advanced System Technology
STMicroelectronics
The smart alternative: HW/SW Plug & Play
Functions library
Function implementation
SW Motion Est.
RLC - VLC lib
SW API
DCT / iDCT lib
Motion. Est. lib
CBR lib
µP
HW Motion Est.
HW 1
HW 2
STBUS
Advanced System Technology
STMicroelectronics
8
A Practical Case:
The Multimedia Over Wireless Platform
Microprocessor:
• OS
• User Interface
• Services (MMS, Streaming, ..)
MEMO
AHB1-M
SRAM
Multilayer
AHB
ARM926EJ-S Subsystem
D-Cache
ARM
ARM926EJ-S
I-Cache
SUBSYSTEM
Core
• VIDEO accelerator:
Mem Ctrl.
• MPEG4
• H263
AHB Master
GRAPHICS
GRAPHICS
DMA
HAMAC
Video Audio
SENSOR
Codec Codec
IGP
Picture
Post
Processing
LCD Ctrl
CO-PROCESSOR
AUDIO – DSP
• MP3
• AAC
• MPEG 1 / 2
• AMR
• MIDI
APB Bridge 1
APB
Advanced System Technology
STMicroelectronics
Application
Specific Signal
Processors
Field
Programmable
Devices
Application
Specific
ICs
Physically
Optimized
ICs
Log
105 - 106
Digital
Signal
Processors
Log F L E X I B I L I T Y
General
Purpose
Processors
Log P O W E R D I S S I P A T I O N
Implementation Space
PERFORMANCE
103 - 104
Source: T.Noll, RWTH Aachen
Advanced System Technology
STMicroelectronics
9
S/W Arch. & Application
For Mobile MM
STM HW/SW platform
Makes easy application dev.
MMS
Streaming
PTD Application (DMD, mComm.)
Mobile IP, Location
Games, Connectivity…
Standard EPOC Framework API
Applications / Java
High level API
Hw
Acceleration
Low level API
Hw abstraction level
Device drivers
Media &
Graphics
Networking
framework
Linux
Symbian
Microsoft
Other STD
framework
OS
} OS
Co-processors
HW
Hardware platform
Advanced System Technology
STMicroelectronics
Open Platforms
• A technical need
• A business must
• An innovation catalyst
Advanced System Technology
STMicroelectronics
10
OMA Driven Strategic Change
PRESENT
FUTURE
APPLICATIONS
APPLICATIONS
API
API
LIBRARIES
O/S
O/S & LIBRARIES
API
API
HARDWARE
HARDWARE
Advanced System Technology
STD (OMA)
OPEN
COMPETITION
STD (MIPI)
STMicroelectronics
The Linux Opportunity
•
•
•
•
•
Simple
Open
Standard
Competitive
Fair
For embedded systems,
an old idea of ST, back to ’97 …………
Advanced System Technology
STMicroelectronics
11
Linux Platform@ST
OpenGL - ES
streaming
applications
The System Call Interface
Process
management
Memory
management
File
systems
Device
control
Ad-hoc
Networks
Real-Time
Concurrency,
Multitasking
Arch-dependent code
Power Mgmt
Virtual
Memory
Memory
manager
Embedded
Multiprocessing
CPU
Memory
Files and Dirs:
the VFS
File
system
types
Disks - CDs
= from Linux CE Forum
= from other Open Std Initiatives
Advanced System Technology
Networking
STMicroelectronics
device
access
Character
devices
Connectivity
Network
Subsystem
IF Drivers
Graphics
A/V Streaming
Consoles etc.
NW interfaces
Kernel
subsystems
Features
Implemented
Software
Support
Source: Linux Device Drivers,
2nd Edition
Where do we see Linux in embedded systems?
Consumer Boxes
Mobile Terminals
Car Navigation
Advanced System Technology
STMicroelectronics
12
What does it take to become a leader
in this world?
(and why is ST a good partner for Linux?)
Advanced System Technology
STMicroelectronics
A Global World
Security
Networking
Consumer
Wireless
Storage
Automotive
Computer Science
Advanced System Technology
STMicroelectronics
13
Learning From The World
Lead Customers R&D and
Engineering: Seagate,
Western Digital
(California,Colorado, ...)
Joint Design center with
Seagate: Scotts Valley,
CA
Competence on R/W
technology: Joint Design
with EXAR - CA
Engineering and Design Capability /
Close understanding of customer
application / Design Center: S. Jose
CA
Engineering and
Design skills in digital
servo controllers:
Design team with
SSD/Parthus - Dublin,
Ir.
Engineering and
Design skills in fast
microprocessors:
Bristol, U.K.
Coordination and
strategic capability:
Agrate (Italy) and
Geneva
(Switzerland)
Process Technology
R&D in Bipolar and
BCD; Design
competence on
analog and mixed
signal ICs :
Castelletto, Agrate
and Pavia (Italy)
Process Technology R&D
in BICMOS (mixed) and
CMOS (digital) :
Grenoble, France
Design of ‘packaging’, testing
and final assembly (Back End)
capability: Malaysia, Malta,
Singapore
Customers’
Manufacturing :
Singapore, and other Far
East locations
(Source: Doz, Santos, & Williamson, “From Global to Metanational: How Companies Win in the Knowledge Economy”,
Harvard Business School Press, 2001)
Advanced System Technology
STMicroelectronics
The Metanational Process
Sensing
Sensing Units
Mobilizing
Magnet
Operating
Operations
Courtesy of Y. Doz, J. Santos & P. Williamson.
Reprinted from : “From Global to Metanational : How Companies Win in the Knowledge Economy” Harvard Business School Press, 2001
Advanced System Technology
STMicroelectronics
14
Organization Culture
Combining the machine model
and complex adaptive systems
Predictable
Hierarchical
Defined
Uncertain
Ambiguous
Volatile
control
adaptability
procedures
culture
Advanced System Technology
STMicroelectronics
WORKING WITH LEADERS
ACCESS
ACCESS
AUTOMOTIVE
AUTOMOTIVE
SECURITY
SECURITY
VIDEO
VIDEO
STORAGE
STORAGE
NETWORKING
NETWORKING
IMAGING
IMAGING
A WORLD-CLASS NETWORK
OF PARTNERSHIPS
A WORLD-CLASS
ACADEMIC NETWORK
USI
IRISA
CNR
ƒ EDA VENDORS
ƒ IP PROVIDERS
ƒ FOUNDRIES &
SUBCONTRACTORS
ƒ COMPETITORS
ƒ START-UPS
ƒ SOFTWARE PROVIDERS
ƒ OPERATORS
ENST
UBO
Stanford
Harvard
Catalunya
Milano
Berkeley
UCLA
Bologna
ENSSAT
Politecnico
di Torino
UNC
UCSD
CNET
Catania
EPFL
Pavia
UCI
Pisa
ENS
LETI
INRIA
Hong
Kong
USC
CNRS
Palermo
Lecce
Carnegie
Mellon
INPG
ESA
Advanced System Technology
Ancona
SISSA
Politecnico
di Milano
UC
Davis
IMEC
Eurecom
MIT
Washington
STMicroelectronics
15
A World of Opportunities
Standard
committee
Service
prov.
IP
provider
University
IP
provider
IP
provider
ST
OEM
Customer
University
Service
provider
Univ.
Customer
Standard
committee
“…they are easy to work with”
Cesar Cesaratto, President, Nortel Enterprise Networks Europe,
interviewed by Business Week, January 1994, answering about
the reasons why Nortel chose ST as a strategic partner.
Advanced System Technology
STMicroelectronics
You have to be able to live with a lot of different
cultures, and above all you have to get used to the
fact that you can’t give orders.
P. Drucker
Advanced System Technology
STMicroelectronics
16
And, besides, we are already a Linux Company!
VOICE TECHNOLOGY
IMAGE SENSOR
Speech recognition
Text to speech
Biometric verification
Repartition for lower system cost &
power
Alternative sensor I/F for power
VGA & CIF
ACOUSTIC TECHNOLOGY
GRAPHICS
New Function
AHB1
CUBA IF
ARM926EJ-S Subsystem
AHB1-M
Quiet zone microphone (selective lobe
microphone)
Noise/Echo cancellation
Enhancement to speech recog.
Voice intelligibility (Noise dependent
equalization)
D-Cache
D-SRAM
I-Cache
ARM926EJ-S
Core
MPEG4 Hardware Engine
DIGITAL AUDIO
Decoder (MP3, MPEG1,2 L1&2 DTS,
DOLBY DIGITAL,AAC, PAC)
Encoder (MP3, MPEG L1&2,AAC)
3D surround (SRS, …)
Audio pre/post processing
Secure digital music
APB Bridge 1
AHB1-S AHB1-M APB-S
SECURITY/DRM SUPPORT
DMA Controler
VIDEO COMPRESSION
MPEG4: QCIF to PAL resolution
H263
DISPLAY
Video response STN with advanced
low power drive
Multimode power/quality
Optimum refresh partitioning for sub
mW power
LOCALIZATION
GPS
Cellular based terminals
Keypad
IMAGING
I-SRAM
2D: PIM/PDA, basic action games
3D: fully texture games
Integration allows multiple video
<<types>>
Optimized for local feedback from
MPEG4
JPEG Codec
Advanced System Technology
CONNECTIVITY
Bluetooth
802.11
USB
STMicroelectronics
PGI CDK ™ = PGI Compilers +
Open Source Clustering Software
Workstation Clusters
The PGI CDK is a turn-key package for configuration of an
HPC cluster from a group of networked Linux workstations or
dedicated cluster using a simple question-and-answer installer
Advanced System Technology
STMicroelectronics
17
ST Leverages Linux Internally
• Many ST compiler and tools engineers use Linux workstations as
their primary development platform on a day-to-day basis
• ST C/C++ compiler toolchains, simulators and application
development tools for embedded processors are hosted on Linux
• Linux ports are under way for selected ST embedded processors
• ST develops and markets compilers and tools for Linux/x86/AMD64
workstations, servers and clusters worldwide under The Portland
Group brand name as a way to measure ST compiler technology
against the best compiler infrastructures in the world
• Formal and ad hoc Linux clusters are used to increase throughput
of quality assurance tests of various software products produced by
ST
Advanced System Technology
STMicroelectronics
18