Jean Caron

Jean Caron
Course objectives
This course will give you:
y An overview of the industrial vision system components y Hints to select vision components for a specific project
y How to use image processing tools to create machine vision applications
2
Course contents
y Image acquisition hardware survey
y Lighting overview
y Image processing techniques (No slide available as mentioned during the course) 3
Image acquisition ‐ Contents
y Camera:
y Sensors
y Color
y Key features
y Video output standards
y Frame Grabbers:
y Typical architecture
y Available PCI buses
y Bandwidth issues
5
Camera ‐ Sensors
y An image sensor is a device that converts an optical image to an electric signal.
y An image sensor is typically a charge‐coupled device (CCD) or a complementary metal–oxide–semiconductor (CMOS) active‐pixel sensor
y Sensors produce image information by converting light (photons) into electrical current (electrons)
y Area scan and line‐scan sensors are used
6
Camera ‐ Sensors
y CCD:
y Charge‐Coupled Device is an analog shift register, that enables the transportation of analog signals (electric charges) through successive stages (capacitors) controlled by a clock signal.
y Used in arrays of photoelectric light (UV, Visible, IR) sensors (pixels), to serialize parallel analog signals
y "CCD" refers to the way that the image signal is read out from the chip. Under the control of an external circuit, each capacitor can transfer its electric charge to one or other of its neighbours.
7
Camera ‐ Sensors
y Interline CCD:
y Uses two CCD arrays of the same size
y One array is sensible to the light. It’s the image area y One array is masked to be used as a storage area
y Contain a microlens over the pixels to increase the
quantum efficiency or fill factor. The microlens is a
spherical plastic material similar to a photo‐resist used
in semiconductor wafer fab.
8
Camera ‐ Sensors
y Interline CCD:
9
Camera ‐ Sensors
y Key parameters of a CCD:
y fill factor: percentage of the pixel area that is actually sensitive to light
y well capacity: measure of the amount of charge the pixel can hold, thus giving the dynamic range of the sensor. y shutter speed
y pixel size: main factor for the CCD sensitivity y resolution
10
Camera ‐ Sensors
y CMOS:
y Complementary Metal–Oxide–Semiconductor (CMOS)
is a major class of integrated circuits
y Same principle than for the CCD (array of photoelectric
light)
y Transistor managed the transfer of the electric charges
11
Camera ‐ Sensors
y CMOS:
12
Camera ‐ Sensors
y CCD advantages:
y Larger true fill factors
y low noise, y high sensitivity
y CMOS advantages:
y low cost because easy to produce (like a chip)
y Small size
y Low power consumption
y Windowing
13
Camera – Monochrome
14
Camera – Color
y Bayer pattern sensor:
y Each square of four pixels has one filtered red, one blue,
and two green
y Human eye is more sensitive to green than either red or
blue
y Result is that luminance information is collected at
every pixel, but the color resolution is lower than the
luminance resolution
y Require a bayer decoding of the acquired image to
retrieve the color image
15
Camera – Color
y Bayer pattern sensor:
16
Camera ‐ Color
y Three CCD sensor:
y Bonded to the respective colour output ports of the prism enabling independent detection of Red, Green and Blue signals
17
Camera – Color
y Three CCD sensor:
y Excellent spectral distribution of the three colour channels
18
Camera – Scanning method
y Interlaced cameras: y Interlaced cameras are characterized by the interlaced
readout mechanism inherited from both European and
US television standards
y RS‐170 standard for monochrome (or EIA) and NTSC for color
y
y
y
US video signal standard (60 Hz)
Full frame acquisition (2 fields) in 33 ms
CCD resolution = 640 x 480 pixels
y CCIR standard for monochrome and PAL for color
y
y
y
European & Asian video signal standard (50 Hz)
Full frame acquisition (2 fields) in 40 ms
CCD resolution = 760 x 574 pixels
19
Camera – Scanning method
y Interlaced cameras: y The readout of a full resolution image, also called frame,
is performed in two phases called “fields”: the odd field
delivers the odd lines of the image; the even field
delivers the even lines of the image
y The number of lines of cells of the storage array is one‐
half of the number of lines of cells in the active array
20
Camera – Scanning method
y Interlaced cameras: y The Odd CCD transfer initiates the readout of the odd‐
field; the Even CCD transfer initiates the readout of the
even field
y In order to obtain a full‐resolution image; it is required
to perform both CCD transfers with the nominal
interval of one‐field period. Consequently, not all pixels
terminate the exposure simultaneously!
y See demo movie
21
Camera – Scanning method
y Progressive scan cameras: y All the active cells are transferred every CCD transfer
y Enable ultra short exposure times (highest scan frequencies)
y Sensor resolution not limited by a TV norm (highest resolution) 22
Camera – Scanning method
23
Camera – Main functionalities
y Rapid Reset or Asynchronous reset: Control the start of the sensor exposure
y Shutter: Control the exposure time of the sensor
y Otherwise use a Strobe to control the lighting (for cameras without shutter like interlaced cameras)
y See demo movie + camera configuration methods
24
Camera – Key features
y Number of pixels: determine the image resolution y Pixel size ( or size of the sensor): determine the sensor y
y
y
y
y
y
y
sensitivity
Signal to Noise Ratio (SNR)
Spectral sensibility
FPS: frame per second
Pixel depth 8/10/12/14/16 bits (digital cameras only)
Monochrome / Bayer / Tri CCD
CCD / CMOS
Rapid reset, shutter, progressive or interlaced, expose overlap
25
Camera – Video standards
y Analog standards
y
y
y
y
A composite analogue video signal is output which can be accompanied by many other timing/synchronization signals
The video signal is limited by bandwidth and analogue noise constraints of the cabling. Also, any small timing errors between the camera and the ADC on the frame grabber can cause loss in image fidelity
A composite analogue video signal contains the pixel information together with all timing signals on a few wires, which makes simple and inexpensive cabling
analogue video has a limited dynamic range 26
Camera – Video standards
y Digital standards
y
y
y
Digital cameras use the same CCD technology as analogue models but the ADC is also inside the camera, which digitizes the video directly and outputs a digital signal
This technique of digitizing immediately overcomes many of the shortcomings of analogue cameras giving very high fidelity
the cabling is generally more complex and expensive than for analog cameras 27
Camera – Video standards
y Analog standards
y
y
Analog standard video (broadcast): only 2 image format and fps
Industrial analog video : mega pixel image format and up to 90 fps
y Digital standards
y
y
y
y
y
Camera Link
USB
IEEE‐1394
GigE Vision
HD‐SDI (serial data interface)
28
Camera – Video standards
Digital Signal Options Data Transfer Rate:
Max Cable Length:
FireWire 1394.a FireWire 1394.b Camera Link® USB 2.0 GigE 400 Mb/s 800 Mb/s up to 3.6 Gb/s 480 Mb/s 1000 Mb/s 4.5m 100m (with GOF cable) 10m 5m 100m # Devices:
up to 63 up to 63 1 up to 127 unlimited Connector:
6pin‐6pin 9pin‐9pin 26pin USB RJ45/CAT5 Capture Board:
Optional
Optional Required Optional
Optional
Power:
Optional
Optional Optional
(PoCL)
Optional
Required
29
Frame grabbers
y Definition:
y A frame grabber is an electronic device that captures individual, digital still frames from an analog video signal or a digital video stream
y It is usually employed as a component of a computer vision system, in which video frames are captured in digital form and then displayed, stored or transmitted in raw or compressed digital form.
30
31
Frame grabbers
y Frame grabber out put (bandwidth issue) :
y PCI 32bits/33 MHz 32/66MHz 32/133 (PCI‐X) y PCIe serial bus so no bandwidth issue due to bus capacity sharing y POCL power over camera link
y Color format / LUT
y Video input mux or not (nb of digitizer in analog) y Trigger / strobe / I/Os
y Free run / master / Asynchronous y Camera interfacing dedicated files
32
Frame grabbers
y Digitalized image
y
y
Digitalized images are encoded on 8 bits (256 levels) or more
The grey level 0= black and the grey level 255 = white
y Image size
y
y
y
Monochrome image 512 × 512 coded on 8 bits => 256 KBYTES
Monochrome image 512 × 512 coded on 12 bits => 384 kB
Color image 512 × 512 coded on 24 (3x8) bits => 768 kB
33
Diffuse lighting
35
Diffuse lighting
y Avantage:
y Donne un éclairage doux et homogène dans toutes les directions.
y Minimise les ombres.
y Inconvenients:
y Diminue légèrement les contrastes.
y Applications :
y Pièces brillantes, métalliques.
y Pièces 3D où les ombres peuvent être un problème.
36
Direct lighting
Shadow
37
Eclairage direct
y Avantages :
y Les lampes ponctuelles halogènes ou non sont faciles à mettre en place et à utiliser.
y L’utilisation d’un éclairage direct donne un contraste maximum.
y Inconvénients :
y Ombrage important sur les pièces 3D (utiliser au moins 2 éclairage pour minimiser les ombres ou anneau à fibres optique autour de la caméra).
y Réflexion spéculaire sur les pièces brillantes.
y Applications :
y Améliorer le contraste dans l’image.
y Mise en évidence de défauts ou de formes
38
Direct lighting
y Avantages :
y Les lampes ponctuelles halogènes ou non sont faciles à mettre en place et à utiliser.
y L’utilisation d’un éclairage direct donne un contraste maximum.
y Inconvénients :
y Ombrage important sur les pièces 3D (utiliser au moins 2 éclairage pour minimiser les ombres ou anneau à fibres optique autour de la caméra).
y Réflexion spéculaire sur les pièces brillantes.
y Applications :
y Améliorer le contraste dans l’image.
y Mise en évidence de défauts ou de formes
39
Backlight
Table lumineuse
40
Backlight
y Avantages :
y Donne un contraste maximum entre la pièce et le fond.
y Simplifie l’image en donnant seulement la silhouette de la pièce.
y Inconvénients :
y Les détails de surface sont perdus:
pièce noire sur fond blanc.
y Applications:
y Très utile pour les application de contrôle dimensionnel extérieur des pièces ou contrôle intérieur de trous
(Collimater l’éclairage pour des mesures précises).
41
On Axis Lighting
Miroir semi-réfléchissant
42
On Axis lighting
y Avantages :
y La caméra est normale à l’objet, pas d’effet de perspective.
y Crée un effet de fond brillant (les surfaces brillantes sont claires et les surfaces diffusantes sont sombres). y Inconvénients :
y L’épaisseur du miroir peut produire une image double.
y Applications :
y Détection de défauts rayures, fissures sur des surfaces lisses.
y Eclairage de petites cavités.
43
Structured lighting
Source Laser
ou à Fibres
44
Structured lighting
y Avantages :
y Méthode permettant une mesure sur l’axe z.
y Met en évidence le profil de surface sur de très faibles contrastes.
y Différentes formes d’éclairages sont disponibles :
grilles, points, cercles.
y Inconvénients :
y Peu fiable pour des mesures précises sur des surfaces non réfléchissantes.
y La pièce doit être balayée pour une mesure complète (long, analyse difficile).
45
Polarized filters
Analyseur
Polarisant
46
Polarized filters
y Avantages :
y Permet de séparer réflexions spéculaire et diffuse (la réflexion directe est minimum lorsque les filtres sont croisés).
y Inconvénient :
y L’effet de filtrage demande une intensité d’éclairage accrue.
y Applications :
y Permet de contrôler la quantité de réflexion spéculaire reçue par la caméra. orientation du filtre caméra
orientation du filtre de l’éclairage
47
Color filters
Lumière blanche
Filtre Rouge
Objet Bleu
Fond Rouge
Un filtre Rouge assombrit des objets bleus ou verts.
Un filtre Bleu assombrit des objets rouges ou jaunes.
Un filtre Vert assombrit des objets bleus ou rouges.
48
Color filters
y Avantages :
y Augmente les contrastes sur les pièces de couleur.
y Relativement simple et bon marché.
y Inconvénient :
y Peut demander une augmentation de l’intensité de l’éclairage.
y En fonction de la saturation, ce filtrage peut être de peu d’utilité.
y Applications :
y Augmentation du contraste entre un objet et un fond de couleurs différentes, mais qui donnent le même niveau de gris sur une image noir et blanc (bleu et rouge).
49
Thank you
50