Wide Dynamic Range for Automotive Machine Vision

WHITE PAPER
Autobrite® Imaging Technology:
Wide Dynamic Range for Automotive Machine Vision
Abstract
This paper addresses the requirements for wide dynamic range cameras in automotive machine vision applications. The paper describes
the architecture of a CMOS image sensor pixel and explores mechanisms for overcoming the effects of noise and saturation to achieve
wide dynamic range.
The linear response curve of a standard pixel is used to illustrate the effect of integration time on dynamic range. Alternative methods
are presented for expanding dynamic range of image sensors, including multiple integration and logarithmic response. Each method is
shown to have significant disadvantages.
Autobrite® proprietary technology from Sensata Technologies is described. An explanation of variable height/multiple reset and
barrier voltage demonstrates how Autobrite offers superior interscene and intrascene dynamic range. A description of the control
mechanism shows how Autobrite provides adaptive and programmable wide dynamic range.
The paper concludes that only cameras equipped with Autobrite match the dynamic range of the image sensor to both the scene and
the application. Autobrite-enabled cameras offer superior wide dynamic range performance for capturing complete and accurate visual
information in driver assistance systems.
Contents
1. Introduction
1. Introduction
2. The CMOS Image Sensor
3. Dynamic Range Overview
4. Dynamic Range of Cameras Using Linear
Image Sensors
5. Methods for Expanding Dynamic Range
6. The Autobrite Solution for Wide Dynamic
Range
7. Autobrite in Automotive Applications
8. Conclusion
Machine vision applications in the automotive industry face critical imaging
challenges. Driver assistance systems must
process and respond quickly and accurately to visual details from uncontrolled
environments. This ability relies on obtaining complete and accurate video information from a camera that adapts to widely
varying light levels.
Driver assistance systems rely on
complete and accurate video information
from a camera that adapts to widely
varying light levels.
The ratio of the highest to the lowest intensity of light in a scene is known as the dynamic range. Conditions such as approaching headlights, glare from other vehicles,
tunnel entrances and exits, and rising or
setting sun produce extreme variations of
illumination within a scene. If the dynamic
range of a camera is not sufficiently wide to
accommodate the dynamic range of a scene,
1
the resulting image will fail to capture details
that the automotive machine vision system
requires. Most of these details cannot be
recovered through post-processing.
Figure 1 lists typical automotive imaging
applications for machine vision. In each of
these applications, wide dynamic range is essential for capturing all available details for
further electronic processing by the driver
assistance system. For example, a blind spot
monitoring system compares video frames
and alerts the driver when a vehicle is moving into the monitored area. In order to
detect moving vehicles but ignore stationary
items such as road barriers, the processor
must receive accurate and complete digital
information from the camera in both daylight and darkness.
WHITE PAPER
Autobrite® Imaging Technology:
Wide Dynamic Range for Automotive Machine Vision
Figure 1. Applications of Machine Vision in Driver Assistance
Figure 2 compares two images captured in real-world automotive lighting conditions. The image on the left, captured using a camera
without wide dynamic range, has lost much of the detail in the brightest and darkest areas. The image on the right, captured with a wide
dynamic range camera, accommodates the extremes of lighting and includes all of the important details in the scene.
In applications such as blind spot monitoring or lane departure warning, the information provided in the image on the right is critical. Only
the image on the right provides adequate detail for the machine vision system to analyze and inform the driver about safe lane changing
conditions. An automotive machine vision system using the image on the left has a greater chance of failing.
Figure 2. Images captured without (left) and with (right) Wide Dynamic Range
2
WHITE PAPER
Autobrite® Imaging Technology:
Wide Dynamic Range for Automotive Machine Vision
The image sensor determines the camera’s
ability to accommodate the dynamic range
of a scene.
The camera is the front end of any machine
vision system. The image sensor is the critical component that determines the camera’s
ability to accommodate the dynamic range
of a scene. This section introduces a basic
CMOS pixel architecture and explains
charge accumulation, saturation, and the
reset function.
The image sensor contains an array of
pixels, each with a photodiode that converts light (photons) to charge (electrons).
Figure 3 shows a typical structure of an acti
ve CMOS pixel. The charge collected in
the pixel is proportional to the intensity of
light that strikes the photodiode. Charge is
accumulated over time and then sent to the
output circuit at the end of the integration
period.
V uc
R es et
M1
M2
P hotodiode
M3
R ow
S elect
Output
is applied, the potential reaches its minimum value and the pixel is saturated. Also
referred to as the fill or full-well capacity of
the pixel, the saturation point is measured in
electrons and is a function of the photodiode size.
b. During the integration period:
Integration starts by lowering the gate
voltage at M1, bringing the gate to
ground. M1 stops conducting and the
starting voltage is stored on the photodiode. As photons strike the photodiode, photo current is integrated onto
the photodiode capacitance, reducing
the stored voltage.
Figures 5 and 6 illustrate charge accumulation and pixel saturation. In Figure 5, charge
accumulates linearly over time, as indicated
by the sloped curve, until the pixel saturates.
At saturation, the slope becomes horizontal
and no further charge accumulates. Figure
6 shows the photodiode at saturation. Any
additional photons striking the photodiode
are not detected until the pixel is reset.
Therefore, it is impossible to differentiate
between illumination levels that exceed the
saturation level.
c. At end of the integration period:
The final voltage, representing the
amount of light impinging on the
photodiode, is buffered by M2 and
selectively read to an output device by
turning on M3.
Before the next integration period starts, the
reset voltage is applied, causing electrons
to be “spilled off ” or removed from the
photodiode. Figure 4 illustrates the function
of the pixel reset.
Saturation
Charge
2.The CMOS Image Sensor
In Figure 4a (left), the reset transistor is
off during the integration period, while
light strikes the photodiode and electrons
accumulate. In Figure 4b (center), the reset
transistor is turned on and electrons are
extracted from the photodiode out to the
power supply. In Figure 4c (right), the photodiode is empty after the reset, except for
residual electrons.
Illumination
Figure 5.
Charge Accumulation and Pixel Saturation
During the integration period, charge
accumulates linearly on the photodiode
capacitance as a function of time. If
enough charge accumulates before the reset
Light
Figure 3. CMOS pixel architecture
Reset
off
The CMOS pixel functions as follows:
e-
e-
a. Before the start of the integration period:
Reset voltage VReset (typically the
positive power supply voltage Vuc) is
applied to the gate of reset transistor
M1. Current flows through M1 until
the photodiode reaches a voltage near
the power supply voltage Vuc.
Reset
on
Vuc
(a)
Photodiode during
integration
Vuc
(b)
Photodiode
during reset
Figure 4. Pixel Reset
3
Vuc
(c)
Photodiode
after reset
WHITE PAPER
Autobrite® Imaging Technology:
Wide Dynamic Range for Automotive Machine Vision
e-
Vuc
Figure 6. Saturated Pixel
Any illumination level that results in
saturation will produce the same result in
the image: a white pixel. For automotive
applications that require precise details even
in bright areas of an image, this is suboptimal—thus the need for a wide dynamic
range imaging solution.
3. Dynamic Range Overview
The dynamic range of a camera is the
ratio of the maximum to the minimum detectable signal of the image
sensor, limited by saturation and the
noise floor.
The dynamic range of a camera determines
its ability to capture the dynamic range of
a scene. The uncontrolled lighting that is
typical of automotive environments,outside,
inside, or at night presents a significant
challenge.
Scenes with some regions that are very
bright and other regions that are very dim
pose the greatest difficulty. If the actual
scene has a greater dynamic range than a
camera can capture, the resultant images
will lack detail in the dark or bright areas. In
the darkest areas, where the signal is lower
than the noise, information such as trees in
shadows or people in dark clothing is not
resolved. In the brightest areas, where the
signal is greater than saturation, information
such as reflections off oncoming vehicles
or glare off a roadway is washed out. Either
situation results in incomplete digital detail
for the machine vision system to process.
Figure 7 lists the luminance, measured in
candelas per square meter, for typical scenes
in an automotive environment. In a scene
that includes, for example, a bright object in
sunlight (105 cd/m2) as well as dark colors
in shadows (10-1 cd/m2), the dynamic
range is calculated as:
20 Log
This section introduces dynamic range of
a camera, the ratio of the maximum to the
minimum detectable signal of the image
sensor. The maximum signal is limited by
saturation, while the minimum signal is
limited by the noise floor. Noise floor represents the level of noise inherent in all pixels,
resulting from process technology limitations such as dark current and circuit design
factors such as reset noise.
10 5
= 120 dB
10 −1
Signal ( saturation)
Signal (noise)
4. Dynamic Range of Cameras
Using Linear Image Sensors
This section describes a typical image
sensor with linear response, in which the
stored charge is directly proportional to the
amount of illumination. Figure 8 shows a
response curve for a linear image sensor.
The slope of the curve represents the image
sensor’s responsivity, the change in signal
due to change in illumination
Signal lower
than noise
Dyanmic range
Signal higher
than saturation
Saturation
Illumination
Figure 8. Linear response curve
Figure 7
Luminance Values in Outdoor Scenes
4
An automotive system designer using
a camera with a linear image sensor
must choose between detecting the
bright or dark details, but cannot
detect both.
Noise Floor
Dynamic range is expressed in decibel units
(dB) according to the equation:
Dynamic Range = 20 Log
To accurately capture the details in a scene
with such extremes of lighting, a camera
must provide a dynamic range of at least
120 dB. This requirement is typical of
scenes in automotive machine vision applications. For comparison, cameras built with
traditional CMOS image sensors provide
dynamic range of around 60 dB.
Charge
Light
At the lowest illumination level, all cameras
lose some information because the signal
falls below the noise floor. Figure 8 shows
this response as the gray part of the linear
curve. An image sensor’s response to the
illumination must rise above the noise floor
before the camera can distinguish between
noise and the signal that results from
WHITE PAPER
Autobrite® Imaging Technology:
Wide Dynamic Range for Automotive Machine Vision
The dynamic range is the part of the curve
between the noise floor and saturation. To
capture the maximum amount of information, the response curve must cross
the noise floor quickly and not reach the
saturation point until the highest possible
illumination.
Linear image sensors typically use integration time to increase the dynamic range.
Integration time is the duration over which
the pixel accumulates charge due to exposure to illumination. Figure 9 compares the
linear response curve for two integration
periods.
Figure 9b (bottom diagram) shows that the
shallow slope of the short integration time
response curve does not reach saturation
until much higher illumination. In other
words, good detail can be captured from
bright objects without “washing out.”
However, the short integration time requires
more illumination to rise above the noise
floor. When an image sensor fails to capture
enough photons to provide detail in the
darker regions, darker pixels will blend
together and information about the original
image is lost.
(a) Long Integration
Saturation
Charge
At a high illumination level, a standard
camera loses information due to saturation.
Once a pixel saturates, no further information about its brightness is available. Even
if the illumination continues to increase,
there is no difference in the output signal,
as shown in Figure 8 where the slope of the
curve becomes horizontal.
Short Integration Time
Noise Floor
Crosses noise floor
at low illumination
Reaches saturation
at low illumination
Illumination
(b) Short Integration
Saturation
Charge
photons striking the photodiode. In other
words, if the signal-to-noise ratio is too low,
details in the dark areas are not distinguishable from noise. If details are important at
low levels of brightness, the noise floor is
of greatest concern.
Noise Floor
Long Integration Time
Figure 9a (top diagram) shows that the steep
slope of a long integration time enables the
response curve to rise quickly above the
noise floor. In other words, relatively little
light is required to produce a detectable
response and good detail can be captured
in the dark regions of a scene.
However, the steep slope of the long
integration time response curve reaches
saturation quickly. The image sensor reaches
its maximum output in relatively low light,
resulting in large portions of the image
saturating and appearing to be uniformly
white.
Reaches saturation
at high illumination
Crosses noise floor
at high illumination
Illumination
Figure 9
Comparison of linear response curves for long
and short integration time
Table 1 summarizes the observations about
the response curves in Figure 9.
As Table 1 shows, long integration requires
little light to cross the noise floor and
capture information in the dark areas, but
saturates quickly. Short integration time
does not reach saturation until high illumination, but also requires high illumination to
cross the noise floor. An automotive system
designer using a camera with a linear image
sensor must choose between detecting the
bright or dark details, but cannot detect
both.
In general, integration time is useful for
expanding the interscene dynamic range
in cameras using linear image sensors. If
one scene is predominantly dark and the
next scene is predominantly bright, a long
integration time will work for the first image and a short integration for the second.
However, integration time has limited usefulness for expanding intrascene dynamic
range. If extremes of light and dark exist in
the same scene, a short integration time will
lose details in the dark while a long integration will wash out the bright areas.
Automotive scenes often have extremes
of lighting at the same time, requiring high
intrascene dynamic range. A camera with a
linear image sensor, therefore, is inadequate
for automotive applications because it cannot detect both bright and dark details in
a single scene. An alternative solution is
needed that captures both.
Table 1. Effects of integration time on linear response curve
Integration Time
Illumination Level
Required to Reach Noise Floor
Illumination Level
at which Saturation Occurs
Short
High
High
Long
Low
Low
5
WHITE PAPER
Autobrite® Imaging Technology:
Wide Dynamic Range for Automotive Machine Vision
Figure 10 proposes an ideal response curve.
5. Methods for Expanding
Dynamic Range
However, having an image sensor that
produces the curve shown in Figure 10 is not
enough to define the ideal automotive camera. For example, a camera is impractical if
it is prohibitively expensive to manufacture.
Furthermore, a camera should be flexible to
change the response curve according to the
conditions.
As shown in the previous section, traditional cameras with linear image sensors have
a number of failings. Current linear image
sensor technology does not capture as much
dynamic range as might be available in a
scene. While integration time is an effective
tool for expanding the interscene dynamic
range (from scene to scene), it is not useful
for expanding the intrascene dynamic range
(within a scene).
This section describes several methods for
expanding dynamic range and evaluates
each method against criteria for an ideal
automotive camera. The ideal automotive
camera uses an image sensor that produces
a wide dynamic range response curve with
the following characteristics:
• In low light, the response curve is linear
with a steep slope to maximize responsiv- ity and ensure that the signal exceeds the noise floor at the lowest possible illumination.
• In bright light, the response curve is nonlinear to expand the dynamic range as far
as possible before reaching saturation.
Saturation
Charge
Methods for expanding dynamic
range should meet the criteria for an
ideal automotive camera, including
adaptablility and programmability.
Linear curve,
steep slope at
low illumination
Non-linear
curve, shallow
slope at high
illumination
Noise Floor
Illumination
Figure 10. Ideal Response Curve
The ideal automotive camera and image sensor technology must also be:
• Built on reliable technology to ensure longlife operation
• Cost-effective to facilitate product introduction into the marketplace.
• Adaptive so that only the right amount of
dynamic range expansion is applied to
accommodate each scene.
• Programmable so that system designers can
tailor the camera’s response to specific
conditions.
An optimal method for achieving wide
dynamic range should meet all of the criteria
summarized in Table 2.
The following are just a few examples of
the methods that have been developed for
expanding dynamic range. Each example
uses a different approach to achieving the
criteria shown in Table 2.
Multiple Integration
A common technique for providing wide
dynamic range involves collecting two or
more frames of the same scene to produce
a single image. Each collected signal is
stored in a separate frame buffer and processed at the end of the integration period.
One variation of multiple integration is
based on capturing two frames—one with
a long integration time followed by one
short—and combining them. Information is
captured from the darkest areas during the
longer integration time and from the brightest areas during the shorter time. After a
scaling factor is applied, the information
from the separate integrations is merged to
produce a wide dynamic range image
Another variation of multiple integration
is based on storing a preliminary image at a
designated time and then continuing to collect charge until a final image is stored the
end of the integration period. The two images are compared, and if the charge at the
Table 2. Criteria for Ideal Automotive Wide Dynamic Range Cameras
Criteria
Linear response at low illumination
Expanded high illumination response
Reliable
Cost-effective
Adaptive
Programmable
Description
Accumulated charge rises quickly above the noise floor. Curve is linear and steep-sloped at low illumination
(high responsivity).
Response is non-linear and shallow sloped at high illumination, expanding the capability to capture the bright
light without reaching saturation.
Uses proven technology and simple designs with minimal number of transistors per pixel. Requires minimal
external processing, complex circuitry, or additional devices that could contribute to failure rate.
Camera uses standard pixels and minimal external circuitry or devices that increase production expenses.
Manufacturers can use one camera to perform multiple applications.
Camera provides “intelligent” technology that adjusts its response curve to the conditions. Adaptive cameras
improve contrast in low dynamic range scenes that don’t require a highly non-linear response curve.
Technology is flexible and controllable. Manufacturers can program the response for specific conditions so
that one camera model can be tailored to meet the needs of multiple applications.
6
WHITE PAPER
Autobrite® Imaging Technology:
Wide Dynamic Range for Automotive Machine Vision
end of the integration period is saturated, the
previously collected charge is used instead.
Multiple integration can expand dynamic
range. However, a major drawback is the external electronics required to implement this
method. In addition to the image sensor,
multiple integration requires an analog-todigital (A to D) conversion for each capture,
a separate frame buffer to store each image,
and external signal processing to perform
operations on the multiple images.
Table 3 summarizes the performance of
multiple integration against the criteria for
wide dynamic range cameras.
Table 4 summarizes the performance of
logarithmic response against the criteria for
wide dynamic range cameras.
6. The Autobrite Solution for Wide Dynamic Range
Logarithmic Response
Some image sensors use a MOS transistor,
operating below its threshold voltage to make
use of a logarithmic dependence between the
drain current and gate voltage. For this mode
of operation, the response curve to achieve
wide dynamic range is purely logarithmic.
The output is no longer integrated charge,
but a continuously changing voltage that is
related to the photocurrent. This mode of
operation has significantly higher fixed pattern noise, making the low light illumination
response considerably inferior to charge integration techniques. In addition, the frequency
response of the pixel is dependent on the
level of illumination. At low light level, the
slow response causes image lag or ghosting.
Autobrite uses the variable height/
multiple reset method and a feedback
loop to meet the criteria for an ideal
automotive camera.
Autobrite is a complete solution from
Sensata Camera Technologies that meets
all of the criteria proposed in Table 2 for
wide dynamic range cameras. Autobrite uses
combined technologies to expand dynamic
range and a straightforward standard, threetransistor CMOS pixel to provide reliability
and cost-effectiveness. No additional frame
buffers are required.
Based on research at the Massachusetts
Institute of Technology, Autobrite controls
the pixel through Sensata’s proprietary
variable height/multiple reset method. A
complete feedback loop simultaneously
controls the integration time and dynamic
range expansion for total adaptability and
programmability. This section explains the
variable height/multiple reset method and
the feedback loop. This section also demonstrates how Autobrite meets all of the
criteria for an ideal automotive camera.
Variable Height/Multiple Reset
Several methods of expanding dynamic
range function by resetting the pixel at
strategic times during the integration period
to delay saturation by removing charge from
the photodiode. Autobrite takes this concept further, using a variable height/mul-
Table 3. Performance Summary for Multiple Integration
Criteria
Low illumination response
High illumination response
Reliable
Cost-effective
Adaptive
Programmable
Evaluation
Good. Expands dynamic range by applying a steep-sloped linear response curve, which raises the signal above
the noise floor quickly.
Good. Expands dynamic range by applying a shallow-sloped response curve, which delays the approach to
saturation level.
Poor. Requires additional frame buffers and external signal processing to store and process the images,
increasing the number of electronic components that may fail.
Poor. The cost of the external memory and complex circuitry required to store and process the images is a major
drawback. In addition, multiple A/D conversions require faster converters and incrrease power consumption.
Moderate
Moderate
Table 4. Performance Summary for Logarithmic Response
Criteria
Low illumination response
High illumination response
Reliable
Cost-effective
Adaptive
Programmable
Evaluation
Poor. logarithmic sensors have inherently more fixed pattern noise than linear sensors. Image lag and ghosting
are also apparent at low illumination.
Good. Expands dynamic range by providing shallower-sloped response at high illumination.
Untested. Logarithmic sensors have not been produced or field utilized in high volumes.
Untested. Logarithmic sensors have not been produced or field utilized in high volumes.
Poor. Not adaptive and thus unable to provide maximum contrast and data integrity in all scenes.
Poor. Not programmable and thus limited in flexibility.
7
WHITE PAPER
Autobrite® Imaging Technology:
Wide Dynamic Range for Automotive Machine Vision
B arrier
M1
M2
P hotodiode
M3
R ow
S elect
Output
d
n
a
e
d
o
i
d
o
t
o
h
P
V3
e
g
a
tl
o
V
r
e
ir
r
a
B
possible, ranging from linear to various
degrees of non-linearity, some of which
are shown in Figure 14. Multiple response
curves provide flexibility to select the best
dynamic range for the given lighting conditions or application.
Charge
V uc
If the voltage of the pixel is lower than V1 at
the time of the pulse, as shown by the solid
black line (representing high illumination),
the voltage is reset to V1, essentially holding
the pixel voltage at the barrier voltage and
producing
VReset a nonlinear response. If the voltage of the pixel is higher than V1, as shown
V1
by Voltage
the dotted
black line (representing low
illumination), the reset pulse does not affect
2
the pixelVvoltage
and the response is linear.
Photodiode and
Barrier Voltage
tiple reset to control not only the timing of
the reset, but the height as well. To control
the height of the reset, Autobrite uses a
barrier voltage instead of a reset voltage, as
shown in Figure 11.
Charge
Photodiode
VReset
Illumination
Time
V1
Figure 14 Multiple response curves
Feedback Loop
Barrier
Voltage
Figure 11. Autobrite pixel architecture
During the integration process, the barrier voltage is increased to a programmable
level. If the photocurrent reduces the pixel
voltage to below the barrier voltage, M1 begins to conduct and holds the pixel voltage
at the barrier voltage. This will only happen
if the pixel voltage falls below the barrier
voltage, indicating a high light intensity. If
the pixel voltage remains above the barrier,
it continues to integrate as a normal linear
pixel.
Figure 12 illustrates a simple variable height/
multiple reset example with two barriers
or reset pulses. At the start of the integration period, reset pulse VReset is applied to
the gate. During the integration period, a
second reset pulse V1 is applied. The second
pulse, which is lower in height than VReset,
performs a conditional reset.
Time
Figure 12. Variable height/multiple reset
The variable height/multiple reset method
accommodates more complex arrangements of reset pulses. The spacing between
the pulses and the height of the individual
pulses are programmable and determine
the response curve of the signal. Figure 13
shows a scenario with four reset pulses at
diminishing barrier voltages VReset, V1, V2,
and V3.
Voltage
Before the integration period begins, the
pixel voltage is driven near VUC, as with
a standard active pixel. At the start of
integration, the barrier voltage drops near
zero and the photodiode voltage decreases
with a slope that is dependent on the light
intensity.
Barrier
Voltage
VReset
V1
Voltage
V2
V3
Time
Figure 13. Multiple barrier operation
Variable height/multiple reset is capable of
producing a linear response in low light as
well as a non-linear response in bright light
instantaneously—all in the same image. In
fact, a wide range of response curves are
8
Achieving wide dynamic range through an
image sensor with linear response at low
illumination and non-linear response at high
illumination solves only part of the problem. To complete the solution, the automotive camera must be able to decide which
response curve to use. Furthermore, system
designers must be able to override the automated decision to customize the response
for specific applications. Autobrite uniquely
meets these criteria with its key features:
adaptability and programmability.
Autobrite includes a mechanism to dynamically adjust both the response curve and the
total integration time based on the scene
being observed. Essentially, the dynamic
range is expanded in real time by changing
Charge
the timing and height of the reset signal.
The control mechanism can be configured
to automatically adapt to each environment
or programmed for a specific application,
Illumination
thereby providing performance that is unmatched by other approaches for achieving
wide dynamic range.
Figure 15 illustrates the Autobrite control
mechanism.
The control loop starts with the image sensor capturing an image. Registers acquire
statistics of the scene such as average
WHITE PAPER
Autobrite® Imaging Technology:
Wide Dynamic Range for Automotive Machine Vision
intensity and number of pixels that exceed
a threshold. A proprietary control mechanism, which can be tailored to a specific
application, uses the statistics to select the
optimum response curve and integration
time. A multiplexing device inputs the signals and allows either the calculated values
or user-supplied values to be fed to the voltage and timing control, which generates the
barrier voltage(s) for the image sensor.
Simultaneously calculating both the integration time and the required dynamic range
expansion allows the image sensor to settle
on optimal settings very quickly. This fast
response time is critical in applications
where dramatic changes in the lighting conditions occur quickly, such as the appearance of headlights from other vehicles.
Figure 15. Autobrite feedback and control mechanism
Another advantage of this approach is that
the collection of the statistics and the control algorithms can be tailored to a specific
application. Original equipment manufacturers can program Autobrite to meet
specific requirements of their application.
For example, system engineers can program
Autobrite to:
• Select a specific region of interest within
the image frame that Autobrite will use to
optimize the integration time and
response curve.
• Select a specific integration time and
response curve, overriding the automated
adjustment.
• Select a maximum integration time or
response curve that the automated adjustment is not to exceed.
• Adjust the speed of adaptibility to respond
more quickly or slowly to lighting changes.
Figure 16. Images captured without (left) and with (right) Autobrite
Table 5 summarizes Autobrite’s ability to meet the criteria for wide dynamic range automotive cameras.
Table 5. Performance Summary for Autobrite
Low illumination response Good. At low illumination, Autobrite responds linearly, ensuring that the
signal rises above the noise floor quickly.
High illumination response
Good. Expands dynamic range by modifying the response curve as needed
by changing the height and timing of the barrier voltage pulses. Provides
high illumination response varying from linear to highly non-linear.
Reliable
Good. Uses a standard, three-transistor CMOS pixel with no extra transistors or external storage. Using a standard pixel ensures access to proven
and easily available technology as well as excellent position to incorporate
advances in CMOS technology. SMaL image sensors incorporating Autobrite technology have been produced in volumes of millions
Cost-effective
Good. Uses the same pixel as linear image sensors with no extra cost associated with frame buffers. In addition to competitive production costs, no
extra storage or A/D conversion means lower power consumption.
Adaptive
Good. Autobrite includes a control mechanism to select the best response
curve and integration time in real time to optimize dynamic range based on
the intensity characteristics of the scene.
Programmable
Good. System engineers can apply control algorithms to customize Autobrite for specific conditions or applications.
Autobrite Advantages
Figure 16 provides a side-by-side comparison of images captured with and without
Autobrite. In the image on the left, the
intrascene dynamic range clearly exceeds the
dynamic range of the camera, resulting in
“clipping” or lost detail in the light and dark
regions. In the image on the right, Autobrite
enables the same scene to be captured with
complete visual details even in the extremes
of brightness and darkness.
9
WHITE PAPER
Autobrite® Imaging Technology:
Wide Dynamic Range for Automotive Machine Vision
Table 6 lists the methods described in this paper and summarizes their ability to meet the criteria for wide dynamic range.
Table 6 Summary of methods for expanding dynamic range
Multiple Integration
Logarithmic Response
Autobrite
Linear response at low illumination
Good
Poor
Good
Non-linear response at high illumination
Good
Good
Good
Reliable
Poor
Untested
Good
Cost-effective
Poor
Untested
Good
Adaptive
Moderate
Poor
Good
Programmable
Moderate
Poor
Good
7. Autobrite in Automotive
Applications
Autobrite provides wide dynamic
range as well as the ability to
adapt to each environment or be
programmed in real time.
Autobrite provides original equipment
manufacturers with more options to optimize image capture for driver assistance
applications. In addition to providing wide
dynamic range, Autobrite has the ability to
adapt to each environment or be programmed in real time—providing machine
vision manufacturers with unlimited flexibility. In many cases, only one camera is
needed for multiple applications.
This section describes typical automotive
applications for Autobrite-enabled cameras.
These are only a few of the many examples
of implementations in which Autobrite
excels.
In the same tunnel scenario, the camera
must adjust quickly to the lighting conditions as the driver exits the tunnel into
the bright sunlight. Autobrite adapts the
response curve and the integration time to
the new scene, enabling the camera to continue capturing critical information about
the road lane markings. Similarly, Autobrite
automatically adapts between night and day
lighting conditions, adjusting every frame to
ensure total data integrity.
characteristics of the person. Autobrite can
ignore the area next to or behind the person, eliminating extraneous information.
In Figure 17, the image on the left is captured with a conventional camera that has
adjusted the integration time for the bright
area of the window, leaving the interior
scene too dark. The image on the right,
optimized with Autobrite for the most
important physical region in the scene,
resolves the interior occupant.
Vehicle Occupant Sensing
Driver Drowsiness
In applications where only a portion of the
scene is of interest, system engineers can
apply algorithms that ignore either physical
or luminance regions when selecting the response curve. Autobrite optimizes only the
designated area of the image, rather than
the entire image.
As an example, a vehicle occupant sensing application must determine whether or
not a person is present and detect specific
In some situations, too much dynamic range
can be detrimental. If a scene requires
precise digital detail within a narrow range
of illumination, an excessively wide dynamic
range sensor would capture unnecessary
information at the extremes of illumination and not enough information within the
critical range.
Lane Departure Warning
A lane departure warning system must
distinguish the road lane markings in all
lighting and weather conditions in order to
alert the driver when the vehicle moves out
of its lane. Autobrite’s wide dynamic range
enables the camera to capture critical visual
information, detecting the lane markings
even in a challenging scenario such as a dark
tunnel with glare from the tunnel exit ahead.
Figure 17.
Vehicle occupant sensing images captured with conventional (left) and Autobrite (right) technology
10
WHITE PAPER
Autobrite® Imaging Technology:
Wide Dynamic Range for Automotive Machine Vision
For example, a driver drowsiness system
that tracks eye movements might use a
strobe flash to illuminate the subject. In this
case, dynamic range is less important but
high contrast is critical. Autobrite’s settings
can be optimized to limit the dynamic range
and provide maximum contrast.
The automotive industry is rapidly developing new applications for electronically processing captured images. Autobrite-enabled
cameras provide the wide dynamic range,
adaptability, and programmability needed to
optimize the front end of machine vision
products.
8. Conclusion
Autobrite ensures that complete
and accurate visual details are
captured even in the most challenging lighting conditions.
The effectiveness of any automotive
machine vision application depends on a
camera that can provide complete and accurate video information. As machine vision is
increasingly applied to driver assistance and
safety applications, manufacturers must develop systems that can process and respond
to visual details in uncontrolled environments. The camera is the critical component
that adapts to widely varying light levels.
Autobrite proprietary technology from
Sensata Technologies offers superior wide
dynamic range performance to capture
complete and accurate visual information
for automotive machine vision systems.
Autobrite is the only technology that offers
adaptability and programmability—the
ability to match the dynamic range of the
image sensor to both the scene and the application.
Wide dynamic range is the measure of a
camera’s ability to capture the difference
between the brightest and the darkest details
in a scene. Conventional cameras using
image sensors with linear response do not
provide the dynamic range needed for automotive applications.
Figure 18 illustrates Autobrite’s superior
wide dynamic range performance for a
back-up assistance camera. In the image
on the left, captured with a conventional
camera, shadows obscure an object behind
the vehicle because the camera has automatically adjusted the exposure to capture
detail in the bright region. In the image on
the right, Autobrite is able to capture both
bright and dark details and clearly shows the
obstacle in the car’s path.
Alternative methods have been developed
for expanding dynamic range, including multiple integration and logarithmic
response. While these methods are successful in expanding dynamic range, each has
significant disadvantages.
Figure 18.
Back-up assistance images captured with conventional (left) and Autobrite (right) technology
Furthermore, Autobrite uses the same
pixel structure as conventional linear sensors, providing the benefits of competitive
production costs, proven and easily available
technology, and excellent position to incorporate advances in CMOS technology. In
addition, no extra storage or A/D conversion means lower power consumption.
Autobrite overcomes dynamic range limitations of machine vision in automotive
applications. Autobrite’s adaptability and
programmability ensure that complete and
accurate visual details are captured even in
the most challenging lighting conditions.
11
WHITE PAPER
Autobrite® Imaging Technology:
Wide Dynamic Range for Automotive Machine Vision
References
1. S. Decker, R.D. McGrath, K. Brehmer, and C.G. Sodini,
“A 256 x 256 CMOS Imaging Array with Wide Dynamic Range Pixels and Column-Parallel Digital Output,” IEEE Journal of Solid-State Circuits, vol. SC-33, No. 12, pp. 2081-2091, Dec. 1998.
2. O. Yadid-Pecht and E.R. Fossum, “Wide Intrascene Dynamic Range CMOS APS Using Dual Sampling,” IEEE Transaction on Electron Devices, vol. ED-44, No. 10, pp 1721-1723, Oct. 1997.
3. D.X.D. Yang, A. El Gamal, B. Fowler, and H. Tian, “A 640 x 512 CMOS Image Sensor with Ultrawide Dynamic Range Floating-Point Pixel-Level ADC,” IEEE Journal of Solid-State Circuits, vol. SC-34, No. 12, pp. 1821-1834, Dec. 1999.
4. S.G. Chamberlain and J.P.Y. Lee, “A Novel Wide Dynamic Range Silicon Photodetector and Linear Imaging
Array,” IEEE Journal of Solid-State Circuits, vol. SC-19, No. 2, pp. 41-48, Feb. 1984.
5. F.J. Kub and G.W. Anderson, “Compressing Photodetectors for Long Optical Pulses Using a Lateral Blooming Drain Structure,” IEEE Trans. Electron Devices, vol. ED-40, pp. 1740-1744, Oct. 1993.
6. S. Decker, A Wide Dynamic Range CMOS Imager with Parallel On-Chip Analog-to-Digital Conversion.
PhD thesis, Massachusetts Institute of Technology, Sept. 1997.
7. K. Fife, A Stereo Vision System with Automatic Brightness Adaptation. Masters thesis,
Massachusetts Institute of Technology, May 1995.
8. P. Acosta-Serafini, A 256x256 Compact, High Resolution Imager for Machine Vision Applications. Masters thesis,
Massachusetts Institute of Technology, Aug. 1997.
Autobrite is a registered trademark of Sensata Technologies.
All company names, trademarks and service marks are the property of their respective owners.
Sensata Technologies
529 Pleasant Street
Attleboro, MA 02703-2964
U.S.A.
Phone 1-508-236-3800
www.sensata.com
© Copyright 2007, Sensata Technologies
Important Notice: Sensata Technologies (Sensata) reserves the right to make changes to or discontinue any product
or service identified in this publication without notice. Sensata advises its customers to obtain the latest version of the
relevant information to verify, before placing any orders, that the information being relied upon is current. Sensata
assumes no responsibility for infringement of patents or rights of others based on Sensata applications assistance or
product specifications since Sensata does not possess full access concerning the use or application of customers’ products. Sensata also assumes no responsibility for customers’ product designs.
12
Printed in U.S.A., June, 2007