Contents - imtuoradea.ro

Contents
Overview and Introduction
1
Systems analysis
1
Hardware architecture
3
Stability theory
5
Control engineering
8
Signal processing
11
Feedback
14
Negative feedback
22
Positive feedback
24
Systems design
33
Systems design
33
Requirement
35
Systems architecture
39
Dynamic models
41
Function model
41
Function block diagram
48
Transfer function
49
Convolution
52
Dynamical system
62
Graph dynamical system
72
Dynamic response
75
First-order logic
75
List of first-order theories
94
Pole splitting
103
Damping ratio
108
Second-order logic
111
Impulse response
116
LTI system theory
118
Routh–Hurwitz stability criterion
129
Feedback properties
133
Control theory
133
Control system
142
Controller (control theory)
146
Process control
149
Three-term control
151
Proportionality (mathematics)
151
Integral
154
Derivative
176
Control methods
191
OLE for process control
191
Nonlinear control
193
Proportional control
196
Optimal control
197
Robust control
203
Adaptive control
205
Stochastic control
207
Neural network
208
Fuzzy logic
217
Expert system
223
The Root-Locus Design Method
231
Root locus
231
Compensation (engineering)
234
Frequency compensation
235
Error detection and correction
237
Linear encoder
244
Gray code
249
Rotary encoder
264
The Frequency-Response Design Method
271
Bode plot
271
Nyquist stability criterion
281
Instability
284
Frequency response
286
Nyquist plot
288
Nichols plot
289
Bandwidth (signal processing)
290
Lead–lag compensator
293
Static VAR compensator
295
PID controller
298
References
Article Sources and Contributors
314
Image Sources, Licenses and Contributors
320
Article Licenses
License
323
1
Overview and Introduction
Systems analysis
Systems analysis is the study of sets of interacting entities, including computer systems analysis. This field is
closely related to requirements analysis or operations research. It is also "an explicit formal inquiry carried out to
help someone (referred to as the decision maker) identify a better course of action and make a better decision than he
might otherwise have made."[1]
Overview
The terms analysis and synthesis come from Greek where they mean respectively "to take apart" and "to put
together". These terms are in scientific disciplines from mathematics and logic to economy and psychology to denote
similar investigative procedures. Analysis is defined as the procedure by which we break down an intellectual or
substantial whole into parts. Synthesis is defined as the procedure by which we combine separate elements or
components in order to form a coherent whole.[2] Systems analysis researchers apply methodology to the analysis of
systems involved to form an overall picture. System analysis is used in every field where there is a work of
developing something. Analysis can also be defined as a series of components that perform organic function
together.
Information technology
The development of a computer-based information system includes a systems analysis phase which produces or
enhances the data model which itself is a precursor to creating or enhancing a database (see Christopher J. Date "An
Introduction to Database Systems"). There are a number of different approaches to system analysis. When a
computer-based information system is developed, systems analysis (according to the Waterfall model) would
constitute the following steps:
• The development of a feasibility study, involving determining whether a project is economically, socially,
technologically and organizationally feasible.
• Conducting fact-finding measures, designed to ascertain the requirements of the system's end-users. These
typically span interviews, questionnaires, or visual observations of work on the existing system.
• Gauging how the end-users would operate the system (in terms of general experience in using computer hardware
or software), what the system would be used for etc.
Another view outlines a phased approach to the process. This approach breaks systems analysis into 5 phases:
•
•
•
•
•
Scope definition
Problem analysis
Requirements analysis
Logical design
Decision analysis
Use cases are a widely-used systems analysis modeling tool for identifying and expressing the functional
requirements of a system. Each use case is a business scenario or event for which the system must provide a defined
response. Use cases evolved out of object-oriented analysis; however, their use as a modeling tool has become
common in many other methodologies for system analysis and design.
Systems analysis
Practitioners
Practitioners of systems analysis are often called up to dissect systems that have grown haphazardly to determine the
current components of the system. This was shown during the year 2000 re-engineering effort as business and
manufacturing processes were examined as part of the Y2K automation upgrades. Employment utilizing systems
analysis include systems analyst, business analyst, manufacturing engineer, enterprise architect, etc.
While practitioners of systems analysis can be called upon to create new systems, they often modify, expand or
document existing systems (processes, procedures and methods). A set of components interact with each other to
accomplish some specific purpose. Systems are all around us. Our body is itself a system. A business is also a
system. People, money, machine, market and material are the components of business system that work together that
achieve the common goal of the organization.
References
[1] SYSTEMS ANALYSIS (http:/ / web. archive. org/ web/ 20070822025602/ http:/ / pespmc1. vub. ac. be/ ASC/ SYSTEM_ANALY. html)
[2] Tom Ritchey, [http://www.swemorph.com/pdf/anaeng-r.pdf Analysis and .
External links
• Systems Analysis, Modelling and Prediction (SAMP), University of Oxford (http://www.eng.ox.ac.uk/samp)
• Software Requirement Analysis using UML (http://www.slideshare.net/dhirajmusings/
software-requirement-analysis-using-uml) article by Dhiraj Shetty.
• Introduction to Social Macrodynamics (http://urss.ru/cgi-bin/db.pl?cp=&page=Book&id=34250&lang=en&
blang=en&list=Found)
• A useful set of guides and a case study about the practical application of business and systems analysis methods
(http://www.cilco.co.uk/briefing-studies/index.html)
• Complete online tutorial for system analysis and design (http://www.systemsanalysis.co.nr)
• A comprehensive description of the discipline of systems analysis from Simmons College, Boston, MA, USA
(www.simmons.edu) (http://web.simmons.edu/~benoit/LIS486/SystemsAnalysis.html)
2
Hardware architecture
3
Hardware architecture
In engineering, hardware architecture refers to the identification of a
system's physical components and their interrelationships. This
description, often called a hardware design model, allows hardware
designers to understand how their components fit into a system
architecture and provides software component designers important
information needed for software development and integration. Clear
definition of a hardware architecture allows the various traditional
engineering disciplines (e.g., electrical and mechanical engineering) to
work more effectively together to develop and manufacture new
machines, devices and components.[1]
An orthographically projected diagram of the
F-117A Nighthawk.
Hardware is also an expression used within the computer engineering
industry to explicitly distinguish the (electronic computer) hardware
from the software which runs in it. But hardware, within the
automation and software engineering disciplines, need not simply be a
computer of some sort. A modern automobile runs vastly more
software than the Apollo spacecraft. Also, modern aircraft cannot
function without running tens of millions of computer instructions
embedded and distributed throughout the aircraft and resident in both
standard computer hardware and in specialized hardward components
An F-117 conducts a live exercise bombing run
such as IC wired logic gates, analog and hybrid devices, and other
using GBU-27 laser-guided bombs.
digital components. The need to effectively model how separate
physical components combine to form complex systems is important over a wide range of applications, including
computers, personal digital assistants (PDAs), cell phones, surgical instrumentation, satellites, and submarines.
Hardware architecture is therefore the representation of an engineered (or to be engineered) electronic or
electromechanical hardware system, and the process and discipline for effectively implementing the design(s) for
such a system. It is generally part of a larger integrated system encompassing information, software, and device
prototyping.[2]
It is a representation because it is used to convey information about the related elements comprising a hardware
system, the relationships among those elements, and the rules governing those relationships.
It is a process because a sequence of steps is prescribed to produce or
change the architecture, and/or a design from that architecture, of a
hardware system within a set of constraints.
It is a discipline because a body of knowledge is used to inform
practitioners as to the most effective way to design the system within a
set of constraints.
A hardware architecture is primarily concerned with the internal
electrical (and, more rarely, the mechanical) interfaces among the
system's components or subsystems, and the interface between the
Electric multi-turn actuator with controls.
system and its external environment, especially the devices operated by
or the electronic displays viewed by a user. (This latter, special
interface, is known as the computer human interface, AKA human computer interface, or HCI; formerly called the
Hardware architecture
4
man-machine interface.)[3] Integrated circuit (IC) designers are driving current technologies into innovative
approaches for new products. Hence, multiple layers of active devices are being proposed as single chip, opening up
opportunities for disruptive microelectronic, optoelectronic, and new microelectromechanical hardware
implementation.[4] [5]
Background
Prior to the advent of digital computers, the electronics and other
engineering disciplines used the terms system and hardware as
they are still commonly used today. However, with the arrival of
digital computers on the scene and the development of software
engineering as a separate discipline, it was often necessary to
distinguish among engineered hardware artifacts, software
artifacts, and the combined artifacts.
A programmable hardware artifact, or machine, that lacks its
software program is impotent; even as a software artifact, or
program, is equally impotent unless it can be used to alter the
sequential states of a suitable (hardware) machine. However, a
hardware machine and its software program can be designed to
perform an almost illimitable number of abstract and physical
tasks. Within the computer and software engineering disciplines
(and, often, other engineering disciplines, such as
communications), then, the terms hardware, software, and system
came to be used to distinguish among the hardware which could
run a software program, the software program itself, and the
hardware device complete with its program, respectively.
A hardware architecture example, which is integrated
as a handheld medical device for diabetes monitoring.
The hardware engineer or architect deals (more or less)
exclusively with the hardware device; the software engineer or
architect deals (more or less) exclusively with the software
U-Boat layout, with detailed equipment hardware
specification and functionality.
program; and the systems engineer or systems architect is
responsible for seeing that the software program is capable of
properly running within the hardware device, and that the system composed of the two entities is capable of properly
interacting with its external environment, especially the user, and performing its intended function.
A hardware architecture, then, is an abstract representation of an electronic and/or an electromechanical device
which is capable of running a fixed or changeable program.[6] [7]
A hardware architecture generally includes some form of analog, digital, or hybrid electronic computer, along with
electronic and mechanical sensors and actuators. Hardware architecting may be viewed as a 'partitioning scheme,' or
algorithm, which considers all of the system's present and foreseeable requirements and arranges the necessary
hardware components into a workable set of cleanly bounded subsystems with no more parts than are required. That
is, it is a partitioning scheme which is exclusive, inclusive, and exhaustive. A major purpose of the partitioning is to
arrange the elements in the hardware subsystems so that there is a minimum of electrical connections and electronic
communications needed among them. In both software and hardware, a good subsystem tends to be seen as a
meaningful "object." Moreover, a clear allocation of user requirements to the architecture (hardware and software)
provides an effective basis for validation tests of the user's requirements in the as-built system.
Hardware architecture
References
[1] Rai, L. & Kang, S.J. (2008). "Rule-based modular software and hardware architecture for multi-shaped robots using real-time dynamic
behavior identification and selection". Knowledge-Based Systems 21 (4): 273–283. doi:10.1016/j.knosys.2007.05.008.
[2] Frampton, K.D., Martin, S.E. & Minor, K. (2003). "The scaling of acoustic streaming for application in micro-fluidic devices". Applied
Acoustics 64 (7): 681–692. doi:10.1016/S0003-682X(03)00005-7.
[3] Brunelli, C., Garzia, F. & Nurmi, J. (2008). "A coarse-grain reconfigurable architecture for multimedia applications featuring subword
computation capabilities". Journal of Real-Time Image Processing 3 (1–2): 21–32. doi:10.1007/s11554-008-0071-3.
[4] Cale, T.S., Lu, J.-Q. & Gutmann, R.J. (2008). "Three-dimensional integration in microelectronics: Motivation, processing, and
thermomechanical modeling". Chemical Engineering Communications 195 (8): 847–888. doi:10.1080/00986440801930302.
[5] Cavalcanti, A., Shirinzadeh, B., Zhang, M. & Kretly, L.C. (2008). "Nanorobot Hardware Architecture for Medical Defense". Sensors 8 (5):
2932–2958. doi:10.3390/s8052932.
[6] Assif, D., Himel, R. & Grajower, Y. (1988). "A new electromechanical device to measure the accuracy of interocclusal records". Journal of
Prosthetic Dentistry 59 (6): 672–676. doi:10.1016/0022-3913(88)90380-0. PMID 3165452.
[7] Zimmermann, M., Volden, T., Kirstein, K.-U., Hafizovic, S., Lichtenberg, J., Brand, O. & Hierlemann, A. (2008). "A CMOS-based
integrated-system architecture for a static cantilever array". Sensors and Actuators B: Chemical 131 (1): 254–264.
doi:10.1016/j.snb.2007.11.016.
Stability theory
In mathematics, stability theory addresses the stability of solutions of differential equations and of trajectories of
dynamical systems under small perturbations of initial conditions. The heat equation, for example, is a stable partial
differential equation because small perturbations of initial data lead to small variations in temperature at a later time
as a result of the maximum principle. More generally, a theorem is stable if small changes in the hypothesis lead to
small variations in the conclusion. One must specify the metric used to measure the perturbations when claiming a
theorem is stable. In partial differential equations one may measure the distances between functions using Lp norms
or the sup norm, while in differential geometry one may measure the distance between spaces using the
Gromov-Hausdorff distance.
In Dynamical Systems, an orbit is called Lyapunov stable if the forward orbit of any point in a small enough
neighborhood of it stays in a small (but perhaps, larger) neighborhood. Various criteria have been developed to prove
stability or instability of an orbit. Under favorable circumstances, the question may be reduced to a well-studied
problem involving eigenvalues of matrices. A more general method involves Lyapunov functions.
Overview in Dynamical Systems
Many parts of the qualitative theory of differential equations and dynamical systems deal with asymptotic properties
of solutions and the trajectories: what happens with the system after a large time? The simplest kind of behavior is
exhibited by equilibrium points, or fixed points, and by periodic orbits. If a particular orbit is well understood, it is
natural to ask next whether a small change in the initial condition will lead to similar behavior. Stability theory
addresses the following questions: will a nearby orbit indefinitely stay close to a given orbit? will it converge to the
given orbit (this is a stronger property)? In the former case, the orbit is called stable and in the latter case,
asymptotically stable, or attracting. Stability means that the trajectories do not change too much under small
perturbations. The opposite situation, where a nearby orbit is getting repelled from the given orbit, is also of interest.
In general, perturbing the initial state in some directions results in the trajectory asymptotically approaching the
given one and in other directions to the trajectory getting away from it. There may also be directions for which the
behavior of the perturbed orbit is more complicated (neither converging nor escaping completely), and then stability
theory does not give sufficient information about the dynamics.
One of the key ideas in stability theory is that the qualitative behavior of an orbit under perturbations can be
analyzed using the linearization of the system near the orbit. In particular, at each equilibrium of a smooth dynamical
system with an n-dimensional phase space, there is a certain n×n matrix A whose eigenvalues characterize the
5
Stability theory
behavior of the nearby points (Hartman-Grobman theorem). More precisely, if all eigenvalues are negative real
numbers or complex numbers with negative real parts then the point is a stable attracting fixed point, and the nearby
points converge to it at an exponential rate, cf Lyapunov stability and exponential stability. If none of the
eigenvalues is purely imaginary (or zero) then the attracting and repelling directions are related to the eigenspaces of
the matrix A with eigenvalues whose real part is negative and, respectively, positive. Analogous statements are
known for perturbations of more complicated orbits.
Stability of fixed points
The simplest kind of an orbit is a fixed point, or an equilibrium. If a mechanical system is in a stable equilibrium
state then a small push will result in a localized motion, for example, small oscillations as in the case of a pendulum.
In a system with damping, a stable equilibrium state is moreover asymptotically stable. On the other hand, for an
unstable equilibrium, such as a ball resting on a top of a hill, certain small pushes will result in a motion with a large
amplitude that may or may not converge to the original state.
There are useful tests of stability for the case of a linear system. Stability of a nonlinear system can often be inferred
from the stability of its linearization.
Maps
Let f: R → R be a continuously differentiable function with a fixed point a, f(a) = a. Consider the dynamical system
obtained by iterating the function f:
The fixed point a is stable if the absolute value of the derivative of f at a is strictly less than 1, and unstable if it is
strictly greater than 1. This is because near the point a, the function f has a linear approximation with slope f′(a):
Thus
which means that the derivative measures the rate at which the successive iterates approach the fixed point a or
diverge from it. If the derivative at a is exactly 1 or −1, then more information is needed in order to decide stability.
There is an analogous criterion for a continuously differentiable map f: Rn → Rn with a fixed point a, expressed in
terms of its Jacobian matrix at a, J = Ja(f). If all eigenvalues of J are real or complex numbers with absolute value
strictly less than 1 then a is a stable fixed point; if at least one of them has absolute value strictly greater than 1 then
a is unstable. Just as for n=1, the case of all eigenvalues having absolute value 1 needs to be investigated further —
the Jacobian matrix test is inconclusive. The same criterion holds more generally for diffeomorphisms of a smooth
manifold.
6
Stability theory
Linear autonomous systems
The stability of fixed points of a system of constant coefficient linear differential equations of first order can be
analyzed using the eigenvalues of the corresponding matrix.
An autonomous system
where x(t)∈Rn and A is an n×n matrix with real entries, has a constant solution
(In a different language, the origin 0∈Rn is an equilibrium point of the corresponding dynamical system.) This
solution is asymptotically stable as t → ∞ ("in the future") if and only if for all eigenvalues λ of A, Re(λ) < 0.
Similarly, it is asymptotically stable as t → −∞ ("in the past") if and only if for all eigenvalues λ of A, Re(λ) > 0. If
there exists an eigenvalue λ of A with Re(λ) > 0 then the solution is unstable for t → ∞.
Application of this result in practice, in order to decide the stability of the origin for a linear system, is facilitated by
the Routh–Hurwitz stability criterion. The eigenvalues of a matrix are the roots of its characteristic polynomial. A
polynomial in one variable with real coefficients is called a Hurwitz polynomial if the real parts of all roots are
strictly negative. The Routh–Hurwitz theorem implies a characterization of Hurwitz polynomials by means of an
algorithm that avoids computing the roots.
Non-linear autonomous systems
Asymptotic stability of fixed points of a non-linear system can often be established using the Hartman–Grobman
theorem.
Suppose that v is a C1-vector field in Rn which vanishes at a point p, v(p)=0. Then the corresponding autonomous
system
has a constant solution
Let J = Jp(v) be the n×n Jacobian matrix of the vector field v at the point p. If all eigenvalues of J have strictly
negative real part then the solution is asymptotically stable. This condition can be tested using the Routh–Hurwitz
criterion.
Lyapunov function for general dynamical systems
A general way to establish Lyapunov stability or asymptotic stability of a dynamical system is by means of
Lyapunov functions.
References
• Stability [1] at Scholarpedia, curated by Philip Holmes and Eric T. Shea-Brown.
External links
• Stable Equilibria [2] by Michael Schreiber, The Wolfram Demonstrations Project.
References
[1] http:/ / www. scholarpedia. org/ article/ Stability
[2] http:/ / demonstrations. wolfram. com/ StableEquilibria/
7
Control engineering
8
Control engineering
Control engineering or Control
systems engineering is the engineering
discipline that applies control theory to
design systems with predictable
behaviors. The practice uses sensors to
measure the output performance of the
device being controlled (often a
vehicle) and those measurements can be
used to give feedback to the input
actuators that can make corrections
toward desired performance. When a
device is designed to perform without
the need of human inputs for correction
it is called automatic control (such as
cruise control for regulating a car's
speed). Multi-disciplinary in nature,
control systems engineering activities
focus on implementation of control
systems
mainly
derived
by
mathematical modeling of systems of a diverse range.
Control systems play a critical role in space flight
Overview
Modern day control engineering (also called control systems engineering) is a relatively new field of study that
gained a significant attention during 20th century with the advancement in technology. It can be broadly defined as
practical application of control theory. Control engineering has an essential role in a wide range of control systems,
from simple household washing machines to high-performance F-16 fighter aircraft. It seeks to understand physical
systems, using mathematical modeling, in terms of inputs, outputs and various components with different behaviors;
use control systems design tools to develop controllers for those systems; and implement controllers in physical
systems employing available technology. A system can be mechanical, electrical, fluid, chemical, financial and even
biological, and the mathematical modeling, analysis and controller design uses control theory in one or many of the
time, frequency and complex-s domains, depending on the nature of the design problem.
History
Automatic control Systems were first developed over two thousand years ago. The first feedback control device on
record is thought to be the ancient Ktesibios's water clock in Alexandria, Egypt around the third century B.C. It kept
time by regulating the water level in a vessel and, therefore, the water flow from that vessel. This certainly was a
successful device as water clocks of similar design were still being made in Baghdad when the Mongols captured the
city in 1258 A.D. A variety of automatic devices have been used over the centuries to accomplish useful tasks or
simply to just entertain. The latter includes the automata, popular in Europe in the 17th and 18th centuries, featuring
dancing figures that would repeat the same task over and over again; these automata are examples of open-loop
control. Milestones among feedback, or "closed-loop" automatic control devices, include the temperature regulator
of a furnace attributed to Drebbel, circa 1620, and the centrifugal flyball governor used for regulating the speed of
steam engines by James Watt in 1788.
Control engineering
In his 1868 paper "On Governors", J. C. Maxwell (who discovered the Maxwell electromagnetic field equations) was
able to explain instabilities exhibited by the flyball governor using differential equations to describe the control
system. This demonstrated the importance and usefulness of mathematical models and methods in understanding
complex phenomena, and signaled the beginning of mathematical control and systems theory. Elements of control
theory had appeared earlier but not as dramatically and convincingly as in Maxwell's analysis.
Control theory made significant strides in the next 100 years. New mathematical techniques made it possible to
control, more accurately, significantly more complex dynamical systems than the original flyball governor. These
techniques include developments in optimal control in the 1950s and 1960s, followed by progress in stochastic,
robust, adaptive and optimal control methods in the 1970s and 1980s. Applications of control methodology have
helped make possible space travel and communication satellites, safer and more efficient aircraft, cleaner auto
engines, cleaner and more efficient chemical processes, to mention but a few.
Before it emerged as a unique discipline, control engineering was practiced as a part of mechanical engineering and
control theory was studied as a part of electrical engineering, since electrical circuits can often be easily described
using control theory techniques. In the very first control relationships, a current output was represented with a
voltage control input. However, not having proper technology to implement electrical control systems, designers left
with the option of less efficient and slow responding mechanical systems. A very effective mechanical controller that
is still widely used in some hydro plants is the governor. Later on, previous to modern power electronics, process
control systems for industrial applications were devised by mechanical engineers using pneumatic and hydraulic
control devices, many of which are still in use today.
Control theory
There are two major divisions in control theory, namely, classical and modern, which have direct implications over
the control engineering applications. The scope of classical control theory is limited to single-input and single-output
(SISO) system design. The system analysis is carried out in time domain using differential equations, in complex-s
domain with Laplace transform or in frequency domain by transforming from the complex-s domain. All systems are
assumed to be second order and single variable, and higher-order system responses and multivariable effects are
ignored. A controller designed using classical theory usually requires on-site tuning due to design approximations.
Yet, due to easier physical implementation of classical controller designs as compared to systems designed using
modern control theory, these controllers are preferred in most industrial applications. The most common controllers
designed using classical control theory are PID controllers.
In contrast, modern control theory is carried out in the state space, and can deal with multi-input and multi-output
(MIMO) systems. This overcomes the limitations of classical control theory in more sophisticated design problems,
such as fighter aircraft control. In modern design, a system is represented as a set of first order differential equations
defined using state variables. Nonlinear, multivariable, adaptive and robust control theories come under this division.
Being fairly new, modern control theory has many areas yet to be explored. Scholars like Rudolf E. Kalman and
Aleksandr Lyapunov are well-known among the people who have shaped modern control theory.
Control systems
Control engineering is the engineering discipline that focuses on the modeling of a diverse range of dynamic systems
(e.g. mechanical systems) and the design of controllers that will cause these systems to behave in the desired manner.
Although such controllers need not be electrical many are and hence control engineering is often viewed as a
subfield of electrical engineering. However, the falling price of microprocessors is making the actual implementation
of a control system essentially trivial. As a result, focus is shifting back to the mechanical engineering discipline, as
intimate knowledge of the physical system being controlled is often desired.
9
Control engineering
Electrical circuits, digital signal processors and microcontrollers can all be used to implement Control systems.
Control engineering has a wide range of applications from the flight and propulsion systems of commercial airliners
to the cruise control present in many modern automobiles.
In most of the cases, control engineers utilize feedback when designing control systems. This is often accomplished
using a PID controller system. For example, in an automobile with cruise control the vehicle's speed is continuously
monitored and fed back to the system, which adjusts the motor's torque accordingly. Where there is regular feedback,
control theory can be used to determine how the system responds to such feedback. In practically all such systems
stability is important and control theory can help ensure stability is achieved.
Although feedback is an important aspect of control engineering, control engineers may also work on the control of
systems without feedback. This is known as open loop control. A classic example of open loop control is a washing
machine that runs through a pre-determined cycle without the use of sensors.
Control engineering education
At many universities, control engineering courses are taught in Electrical and Electronic Engineering, Mechatronics
Engineering, Mechanical engineering, and Aerospace engineering; in others it is connected to computer science, as
most control techniques today are implemented through computers, often as Embedded systems (as in the
automotive field). The field of control within chemical engineering is often known as process control. It deals
primarily with the control of variables in a chemical process in a plant. It is taught as part of the undergraduate
curriculum of any chemical engineering program, and employs many of the same principles in control engineering.
Other engineering disciplines also overlap with control engineering, as it can be applied to any system for which a
suitable model can be derived.
Control engineering has diversified applications that include science, finance management, and even human
behavior. Students of control engineering may start with a linear control system course dealing with the time and
complex-s domain, which requires a thorough background in elementary mathematics and Laplace transform (called
classical control theory). In linear control, the student does frequency and time domain analysis. Digital control and
nonlinear control courses require z transformation and algebra respectively, and could be said to complete a basic
control education. From here onwards there are several sub branches.
Recent advancement
Originally, control engineering was all about continuous systems. Development of computer control tools posed a
requirement of discrete control system engineering because the communications between the computer-based digital
controller and the physical system are governed by a computer clock. The equivalent to Laplace transform in the
discrete domain is the z-transform. Today many of the control systems are computer controlled and they consist of
both digital and analog components.
Therefore, at the design stage either digital components are mapped into the continuous domain and the design is
carried out in the continuous domain, or analog components are mapped in to discrete domain and design is carried
out there. The first of these two methods is more commonly encountered in practice because many industrial systems
have many continuous systems components, including mechanical, fluid, biological and analog electrical
components, with a few digital controllers.
Similarly, the design technique has progressed from paper-and-ruler based manual design to computer-aided design,
and now to computer-automated design (CAutoD), which has been made possible by evolutionary computation.
CAutoD can be applied not just to tuning a predefined control scheme, but also to controller structure optimisation,
system identification and invention of novel control systems, based purely upon a performance requirement,
independent of any specific control scheme.[1] [2]
10
Control engineering
Literature
• Christopher Kilian (2005). Modern Control Technology. Thompson Delmar Learning. ISBN 1-4018-5806-6.
• Arnold Zankl (2006). Milestones in Automation: From the Transistor to the Digital Factory. Wiley-VCH.
ISBN 3-89578-259-9.
References
[1] Tan, K.C. and Li, Y. (2001) Performance-based control system design automation via evolutionary computing. Engineering Applications of
Artificial Intelligence, 14 (4). pp. 473-486. ISSN 0952-1976, http:/ / eprints. gla. ac. uk/ 3807/
[2] Li, Y., et al. (2004). CAutoCSD - Evolutionary search and optimisation enabled computer automated control system design. International
Journal of Automation and Computing, 1(1). pp.76-88. ISSN 1751-8520, http:/ / eprints. gla. ac. uk/ 3818/
• Bennett, Stuart (1986-06). A history of control engineering, 1800-1930. IET. ISBN 978-0-86341047-5.
• Bennett, Stuart (1993). A history of control engineering, 1930-1955. IET. ISBN 978-0-86341299-8.
External links
• Control Labs Worldwide (http://www-control.eng.cam.ac.uk/extras/Virtual_Library/Control_VL.html)
• The Michigan Chemical Engineering Process Dynamics and Controls Open Textbook (http://controls.engin.
umich.edu/wiki)
Signal processing
Signal processing is an area of systems engineering, electrical engineering and applied mathematics that deals with
operations on or analysis of signals, in either discrete or continuous time. Signals of interest can include sound,
images, time-varying measurement values and sensor data, for example biological data such as electrocardiograms,
control system signals, telecommunication transmission signals, and many others. Signals are analog or digital
electrical representations of time-varying or spatial-varying physical quantities. In the context of signal processing,
arbitrary binary data streams and on-off signalling are not considered as signals, but only analog and digital signals
that are representations of analog physical quantities.
Typical operations and applications
Processing of signals includes the following operations and algorithms with application examples:[1]
• Filtering (for example in tone controls and equalizers)
• Smoothing, deblurring (for example in image enhancement)
• Adaptive filtering (for example for echo-cancellation in a conference telephone, or denoising for aircraft
identification by radar)
• Spectrum analysis (for example in magnetic resonance imaging, tomographic reconstruction and OFDM
modulation)
• Digitization, reconstruction and compression (for example, image compression, sound coding and other source
coding)
• Storage (in digital delay lines and reverb)
• Modulation (in modems and radio receivers and transmitters)
• Wavetable synthesis (in modems and music synthesizers)
• Feature extraction (for example speech-to-text conversion and optical character recognition)
• Pattern recognition and correlation analysis (in spread spectrum receivers and computer vision)
• Prediction
11
Signal processing
• A variety of other operations
In communication systems, signal processing may occur at OSI layer 1, the Physical Layer (modulation,
equalization, multiplexing, etc.) in the seven layer OSI model, as well as at OSI layer 6, the Presentation Layer
(source coding, including analog-to-digital conversion and data compression).
History
According to Alan V. Oppenheim and Ronald W. Schafer, the principles of signal processing can be found in the
classical numerical analysis techniques of the 17th century. They further state that the "digitalization" or digital
refinement of these techniques can be found in the digital control systems of the 1940s and 1950s.[2]
Mathematical topics embraced by signal processing
•
•
•
•
•
Linear signals and systems, and transform theory
System identification and classification
Calculus
Differential equations
Vector spaces and Linear algebra
•
•
•
•
•
•
•
•
Functional analysis
Probability and stochastic processes
Detection theory
Estimation theory
Optimization
Programming
Numerical methods
Iterative methods
Categories of signal processing
Analog signal processing
Analog signal processing is for signals that have not been digitized, as in legacy radio, telephone, radar, and
television systems. This involves linear electronic circuits such as passive filters, active filters, additive mixers,
integrators and delay lines. It also involves non-linear circuits such as compandors, multiplicators (frequency mixers
and voltage-controlled amplifiers), voltage-controlled filters, voltage-controlled oscillators and phase-locked loops.
Discrete time signal processing
Discrete time signal processing is for sampled signals that are considered as defined only at discrete points in time,
and as such are quantized in time, but not in magnitude.
Analog discrete-time signal processing is a technology based on electronic devices such as sample and hold circuits,
analog time-division multiplexers, analog delay lines and analog feedback shift registers. This technology was a
predecessor of digital signal processing (see below), and is still used in advanced processing of gigahertz signals.
The concept of discrete-time signal processing also refers to a theoretical discipline that establishes a mathematical
basis for digital signal processing, without taking quantization error into consideration.
12
Signal processing
Digital signal processing
Digital signal processing is the processing of digitised discrete time sampled signals. Processing is done by
general-purpose computers or by digital circuits such as ASICs, field-programmable gate arrays or specialized digital
signal processors (DSP chips). Typical arithmetical operations include fixed-point and floating-point, real-valued
and complex-valued, multiplication and addition. Other typical operations supported by the hardware are circular
buffers and look-up tables. Examples of algorithms are the Fast Fourier transform (FFT), finite impulse response
(FIR) filter, Infinite impulse response (IIR) filter, and adaptive filters such as the Wiener and Kalman filters.
Fields of signal processing
• Statistical signal processing – analyzing and extracting information from signals and noise based on their
stochastic properties
• Audio signal processing – for electrical signals representing sound, such as speech or music
• Speech signal processing – for processing and interpreting spoken words
• Image processing – in digital cameras, computers, and various imaging systems
• Video processing – for interpreting moving pictures
• Array processing – for processing signals from arrays of sensors
•
•
•
•
Time-frequency analysis – for processing non-stationary signals[3]
Filtering – used in many fields to process signals
Seismic signal processing
Data mining
Notes and references
[1] Mathematical Methods and Algorithms for Signal Processing, Todd K. Moon, Wynn C. Stirling, Prentice Hall, 2000, ISBN 0-201-36186-8,
page 4.
[2] Oppenheim, Alan V.; Schafer, Ronald W. (1975). Digital Signal Processing. Prentice Hall. p. 5. ISBN 0-13-2146355.
[3] Boashash, B. (ed.), (2003) Time-Frequency Signal Analysis and Processing: A Comprehensive Reference, Elsevier Science, Oxford, 2003;
ISBN 0080443354
External links
• Signal Processing for Communications (http://www.sp4comm.org/) — free online textbook by Paolo Prandoni
and Martin Vetterli (2008)
• Scientists and Engineers Guide to Digital Signal Processing (http://www.dspguide.com) — free online
textbook by Stephen Smith
13
Feedback
Feedback
Feedback describes the situation when output from (or information about the result of) an event or phenomenon in
the past will influence an occurrence or occurrences of the same (i.e. same defined) event / phenomenon (or the
continuation / development of the original phenomenon) in the present or future. When an event is part of a chain of
cause-and-effect that forms a circuit or loop, then the event is said to "feed back" into itself.
Feedback is also a synonym for:
• Feedback signal - the information about the initial event that is the basis for subsequent modification of the event
• Feedback loop - the causal path that leads from the initial generation of the feedback signal to the subsequent
modification of the event
• Audio feedback - the special kind of positive feedback that occurs when a loop exists between an audio input and
output.
Overview
Feedback is a mechanism, process or signal that is looped back to control a system within itself. Such a loop is called
a feedback loop. In systems containing an input and output, feeding back part of the output so as to increase the input
is positive feedback (regeneration); feeding back part of the output in such a way as to partially oppose the input is
negative feedback (degeneration).
Generally, a control system has input from an external signal source and output to an external load; this defines a
natural sense (or direction) or path of propagation of signal; the feedforward sense or path describes the signal
propagation from input to output; feedback describes signal propagation in the reverse sense. When a sample of the
output of the system is fed back, in the reverse sense, by a distinct feedback path into the interior of the system, to
contribute to the input of one of its internal feedforward components, especially an active device or a substance that
is consumed in an irreversible reaction, it is called the "feedback". The propagation of the signal around the feedback
loop takes a finite time because it is causal.
The natural sense of feedforward is defined chemically by some irreversible reaction, or electronically by an active
circuit element that has access to an auxiliary power supply, so as to be able to provide power gain to amplify the
signal as it propagates from input to output. For example, an amplifier can use power from its controlled power
reservoir, such as its battery, to provide power gain to amplify the signal; but the reverse is not possible: the signal
cannot provide power to re-charge the battery of the amplifier.
Feedforward, feedback and regulation are self related. The feedforward carries the signal from source to load.
Negative feedback helps to maintain stability in a system in spite of external changes. It is related to homeostasis.
For example, in a population of foxes (predators) and rabbits (prey), an increase in the number of foxes will cause a
reduction in the number of rabbits; the smaller rabbit population will sustain fewer foxes, and the fox population will
fall back. In an electronic amplifier feeding back a negative copy of the output to the input will tend to cancel
distortion, making the output a more accurate replica of the input signal.[1]
Positive feedback amplifies possibilities of divergences (evolution, change of goals); it is the condition to change,
evolution, growth; it gives the system the ability to access new points of equilibrium.
For example, in an organism, most positive feedback provides for fast autoexcitation of elements of endocrine and
nervous systems (in particular, in stress responses conditions) and are believed to play a key role in morphogenesis,
growth, and development of organs, all processes that are, in essence, a rapid escape from the initial state.
Homeostasis is especially visible in the nervous and endocrine systems when considered at organism level. Chemical
potential energy for irreversible reactions or electrical potential energy for irreversible cell-membrane current powers
the feedforward sense of the process. However, in the case of morphogenesis, feedback may only be enough to
explain the increase in momentum of the system, and may not be sufficient in itself to account for the movement or
14
Feedback
15
direction of its parts.
When a public-address system is used with a microphone to amplify speech, the output from a random sound at the
microphone may produce sound at a loudspeaker that reaches the microphone such as to reinforce and amplify the
original signal (positive feedback), building up to a howl (of frequency dependent upon the acoustics of the hall). A
similar process is used deliberately to produce oscillating electrical signals.
Feedback is distinctly different from reinforcement that occurs in learning, or in conditioned reflexes. Feedback
combines immediately with the immediate input signal to drive the responsive power gain element, without changing
the basic responsiveness of the system to future signals. Reinforcement changes the basic responsiveness of the
system to future signals, without combining with the immediate input signal. Reinforcement is a permanent change
in the responsiveness of the system to all future signals. Feedback is only transient, being limited by the duration of
the immediate signal.
Types of feedback
When feedback acts in response to an
event/phenomenon, it can influence the
input signal in one of two ways:
1. An in-phase feedback signal, where a
positive-going wave on the input leads to
a positive-going change on the output,
will amplify the input signal, leading to
more modification. This is known as
positive feedback.
Figure 1: Ideal feedback model. The feedback is negative if B < 0
2. A feedback signal which is inverted,
where a positive-going change on the input leads to a negative-going change on the output, will dampen the effect
of the input signal, leading to less modification. This is known as negative feedback.
• Note that an increase or decrease of the feedback signal here refers to the magnitude relative to the input signal's
absolute value, without regard to the polarity or sign of the feedback signal. For example if the input signal
changes by 100 then a change in feedback signal value from +5 to +10 is a positive feedback. If the input signal
changes by -100 then a change in the feedback signal from -5 to -10 is also a positive feedback.
Positive feedback tends to increase the event that caused it, such as in a nuclear chain-reaction. It is also known as a
self-reinforcing loop.[2] An event influenced by positive feedback can increase or decrease its output/activation until
it hits a limiting constraint. Such a constraint may be destructive, as in thermal runaway or a nuclear chain reaction.
Self-reinforcing loops can be a smaller part of a larger balancing loop, especially in biological systems such as
regulatory circuits.
Negative feedback, which tends to reduce the input signal that caused it, is also known as a self-correcting or
balancing loop.[2] Such loops tend to be goal-seeking, as in a thermostat, which compares actual temperature with
desired temperature and seeks to reduce the difference. Balancing loops are sometimes prone to hunting: an
oscillation caused by an excessive or delayed negative feedback signal, resulting in over-correction, wherein the
signal becomes a positive feedback.
The terms negative and positive feedback can be used loosely or colloquially to describe or imply criticism and
praise, respectively. This may lead to confusion with the more technically accurate terms positive and negative
reinforcement, which refer to something that changes the likelihood of a future behaviour. Moreover, when used
technically, negative feedback leads to stability that is, in general, considered good, whereas positive feedback can
lead to unstable and explosive situations that are considered bad. Thus, when used colloquially, these terms imply
the opposite desirability to that when used technically.
Feedback
Negative feedback was applied by Harold Stephen Black to electrical amplifiers in 1927, but he could not get his
idea patented until 1937.[3] Arturo Rosenblueth, a Mexican researcher and physician, co-authored a seminal 1943
paper Behavior, Purpose and Teleology[4] that, according to Norbert Wiener (another co-author of the paper), set the
basis for the new science cybernetics. Rosenblueth proposed that behaviour controlled by negative feedback,
whether in animal, human or machine, was a determinative, directive principle in nature and human creations. This
kind of feedback is studied in cybernetics and control theory.
Applications
Biology
In biological systems such as organisms, ecosystems, or the biosphere, most parameters must stay under control
within a narrow range around a certain optimal level under certain environmental conditions. The deviation of the
optimal value of the controlled parameter can result from the changes in internal and external environments. A
change of some of the environmental conditions may also require change of that range to change for the system to
function. The value of the parameter to maintain is recorded by a reception system and conveyed to a regulation
module via an information channel. An example of this is Insulin oscillations.
Biological systems contain many types of regulatory circuits, both positive and negative. As in other contexts,
positive and negative do not imply consequences of the feedback have good or bad final effect. A negative feedback
loop is one that tends to slow down a process, whereas the positive feedback loop tends to accelerate it. The mirror
neurons are part of a social feedback system, when an observed action is "mirrored" by the brain - like a
self-performed action.
Feedback is also central to the operations of genes and gene regulatory networks. Repressor (see Lac repressor) and
activator proteins are used to create genetic operons, which were identified by Francois Jacob and Jacques Monod in
1961 as feedback loops. These feedback loops may be positive (as in the case of the coupling between a sugar
molecule and the proteins that import sugar into a bacterial cell), or negative (as is often the case in metabolic
consumption).
Any self-regulating natural process involves feedback and/or is prone to hunting. A well-known example in ecology
is the oscillation of the population of snowshoe hares due to predation from lynxes.
In zymology, feedback serves as regulation of activity of an enzyme by its direct product(s) or downstream
metabolite(s) in the metabolic pathway (see Allosteric regulation).
Hypothalamo-pituitary-adrenal and gonadal axis is largely controlled by positive and negative feedback, much of
which is still unknown.
In psychology, the body receives a stimulus from the environment or internally that causes the release of hormones.
Release of hormones then may cause more of those hormones to be released, causing a positive feedback loop. This
cycle is also found in certain behaviour. For example, "shame loops" occur in persons who blush easily. When they
realize that they are blushing, they become even more embarrassed, which leads to further blushing, and so on.[5]
16
Feedback
Climate science
The climate system is characterized by strong positive and negative feedback loops between processes that affect the
state of the atmosphere, ocean, and land. A simple example is the ice-albedo positive feedback loop whereby melting
snow exposes more dark ground (of lower albedo), which in turn absorbs heat and causes more snow to melt.
Control theory
Feedback is extensively used in control theory, using a variety of methods including state space (controls), full state
feedback (also known as pole placement), and so forth.
Further information: PID controller
The most common general-purpose controller using a control-loop feedback mechanism is a
proportional-integral-derivative (PID) controller. Heuristically, the terms of a PID controller can be interpreted as
corresponding to time: the proportional term depends on the present error, the integral term on the accumulation of
past errors, and the derivative term is a prediction of future error, based on current rate of change.[6]
Mechanical engineering
In ancient times, the float valve was used to regulate the flow of water in Greek and Roman water clocks; similar
float valves are used to regulate fuel in a carburettor and also used to regulate tank water level in the flush toilet.
In 1745, the windmill was improved with by blacksmith Edmund Lee who added a fantail to keep the face of the
windmill pointing into the wind. In 1787, Thomas Mead regulated the rotation speed of a windmill by using a
centrifugal pendulum to adjust the distance between the bedstone and the runner stone (i.e., to adjust the load).
The use of the centrifugal governor by James Watt in 1788 to regulate the speed of his steam engine was one factor
leading to the Industrial Revolution. Steam engines also use float valves and pressure release valves as mechanical
regulation devices. A mathematical analysis of Watt's governor was done by James Clerk Maxwell in 1868.
The Great Eastern was one of the largest steamships of its time and employed a steam powered rudder with
feedback mechanism designed in 1866 by J.McFarlane Gray. Joseph Farcot coined the word servo in 1873 to
describe steam-powered steering systems. Hydraulic servos were later used to position guns. Elmer Ambrose Sperry
of the Sperry Corporation designed the first autopilot in 1912. Nicolas Minorsky published a theoretical analysis of
automatic ship steering in 1922 and described the PID controller.
Internal combustion engines of the late 20th century employed mechanical feedback mechanisms such as the vacuum
timing advance but mechanical feedback was replaced by electronic engine management systems once small, robust
and powerful single-chip microcontrollers became affordable.
Electronic engineering
The main applications of feedback in electronics are in the designs of amplifiers, oscillators, and logic circuit
elements.
The processing and control of feedback is engineered into many electronic devices and may also be embedded in
other technologies.
If the signal is inverted on its way round the control loop, the system is said to have negative feedback; otherwise,
the feedback is said to be positive. Negative feedback is often deliberately introduced to increase the stability and
accuracy of a system by correcting unwanted changes. This scheme can fail if the input changes faster than the
system can respond to it. When this happens, the lag in arrival of the correcting signal results in unintended positive
feedback, causing the output to oscillate or "hunt".[7] Oscillation is usually an unwanted consequence of system
behaviour.
17
Feedback
18
Harry Nyquist contributed the Nyquist plot for assessing the stability of feedback systems. An easier assessment, but
less general, is based upon gain margin and phase margin using Bode plots (contributed by Hendrik Bode). Design to
ensure stability often involves frequency compensation, one method of compensation being pole splitting.
The high-pitched squeal that sometimes occurs in audio systems, PA systems, and rock music is known as audio
feedback. If a microphone is in front of a speaker that it is connected to, the noise put into the microphone will come
out of the speaker. Since the microphone is in front of the speaker, the original sound (now coming from the speaker)
goes back into the microphone. This happens over and over, getting louder each time. This process produces the
squeal.
Electronic feedback loops
They are used to control the output of electronic devices, such as amplifiers. A feedback loop is created when
all or some portion of the output from an electronic device is fed-back to the input. A device is said to be
operating open loop if no output feedback is being employed and closed loop if feedback is being used.
Electronic feedback loops take two forms: negative feedback loops and positive feedback loops.[8]
Negative feedback loops
They exist when the fed-back output signal is out of phase with the input signal. This occurs when the
fed-back signal is anywhere from 90° to 270° with respect to the input signal. Negative feedback is generally
used to correct output errors or to lower device output gain to a pre-determined level. In feedback amplifiers,
this correction is generally for waveform distortion reduction or to establish a specified gain level. A general
expression for the gain of a negative feedback amplifier is the asymptotic gain model.
Positive feedback loops
They occur when the fed-back signal is in phase with the input signal. Under certain gain conditions, positive
feedback reinforces the input signal to the point where the output of the device oscillates between its
maximum and minimum possible states. Positive feedback may also introduce hysteresis into a circuit. This
can cause the circuit to ignore small signals and respond only to large ones. It is sometimes used to eliminate
noise from a digital signal. Under some circumstances, positive feedback may cause a device to latch, i.e., to
reach a condition in which the output is locked to its maximum or minimum state.
Software engineering and computing systems
Feedback loops provide generic mechanisms for controlling the running, maintenance, and evolution of software and
computing systems.[9] Feedback-loops are important models in the engineering of adaptive software, as they define
the behaviour of the interactions among the control elements over the adaptation process, to guarantee system
properties at run-time. Feedback loops and foundations of control theory has been successfully applied to computing
systems.[10] In particular, they have been applied to the development of products such as IBM's Universal Database
server and IBM Tivoli. From a software perspective, the autonomic (MAPE, Monitor Analyze Plan Execute) loop
proposed by researchers of IBM is another valuable contribution to the application of feedback loops to the control
of dynamic properties and the design and evolution of autonomic software systems.[11] [12]
Social sciences
A feedback loop to control human behaviour involves four distinct stages.[13] Firstly - Evidence. A behaviour must
be measured, captured, and data stored. Secondly - Relevance. The information must be relayed to the individual, not
in the raw-data form in which it was captured but in a context that makes it emotionally resonant. Thirdly Consequence. The information must illuminate one or more paths ahead. Fourthly - Action. There must be a clear
moment when the individual can recalibrate a behavior, make a choice, and act. Then that action is measured, and
the feedback loop can run once more, every action stimulating new behaviors that inch the individual closer to their
goals.
Feedback
Reflexive feedback
A sociological concept that states a feedback association is created within a certain relationship whereby the
subject/object that delivers a stimulus to a second subject/object, will in response receive the stimulus back. This first
impulse is reflected back and forth over and over again.
Economics and finance
A system prone to hunting (oscillating) is the stock market, which has both positive and negative feedback
mechanisms. This is due to cognitive and emotional factors belonging to the field of behavioural finance. For
example,
• When stocks are rising (a bull market), the belief that further rises are probable gives investors an incentive to buy
(positive feedback - reinforcing the rise, see also stock market bubble); but the increased price of the shares, and
the knowledge that there must be a peak after which the market will fall, ends up deterring buyers (negative
feedback - stabilizing the rise).
• Once the market begins to fall regularly (a bear market), some investors may expect further losing days and
refrain from buying (positive feedback - reinforcing the fall), but others may buy because stocks become more
and more of a bargain (negative feedback - stabilizing the fall).
George Soros used the word reflexivity, to describe feedback in the financial markets and developed an investment
theory based on this principle.
The conventional economic equilibrium model of supply and demand supports only ideal linear negative feedback
and was heavily criticized by Paul Ormerod in his book "The Death of Economics", which, in turn, was criticized by
traditional economists. This book was part of a change of perspective as economists started to recognise that Chaos
Theory applied to nonlinear feedback systems including financial markets.
World-system development
The hyperbolic growth of the world population observed till the 1970s has recently been correlated to a non-linear
second-order positive feedback between the demographic growth and technological development that can be spelled
out as follows: technological growth - increase in the carrying capacity of land for people - demographic growth more people - more potential inventors - acceleration of technological growth - accelerating growth of the carrying
capacity - the faster population growth - accelerating growth of the number of potential inventors - faster
technological growth - hence, the faster growth of the Earth's carrying capacity for people, and so on.[14]
Education
Learners have different conceptions of learning activities, and preconceptions about hierarchy in education. Some
may look up to instructors as experts in the field and take to heart most of the things instructors say. Thus, it is
believed that spending a fair amount of time and effort thinking about how to respond to students may be a
worthwhile time investment. The Educational research literature on formative feedback also referred to as formative
assessment was reviewed recently by V. Shute.
Some general types of reinforcement that can be used in many types of student assessment are:
19
Feedback
20
Confirmation Your answer was correct.
Corrective
Your answer was incorrect. The correct answer was Jefferson.
Explanatory
Your answer was incorrect because Carter was from Georgia; only Jefferson called Virginia home.
Diagnostic
Your answer was incorrect. Your choice of Carter suggests some extra instruction on the home states of past presidents might be
helpful.
Elaborative
Your answer, Jefferson, was correct. The University of Virginia, a campus rich with Jeffersonian architecture and writings, is
sometimes referred to as "Mr. Jefferson's University".
(Adapted from Flemming and Levie.[15] )
A different application of feedback in education is the system for "continuous improvement" of engineering curricula
monitored by the Accreditation Board for Engineering and Technology (ABET).[16]
Government
Examples of feedback in government are:
•
•
•
•
•
Elections
Mass media
Revolution
Curfews
Surveys
Email administration
A mechanism to alert the purported sender of an email with information about the email.
In organizations
As an organization seeks to improve its performance, feedback helps it to make required adjustments. Feedback
serves as motivation for many people in the work place. When one receives either negative or positive feedback,
they decide how they will apply it to his or her job. Joseph Folkman says that to find the greatest level of success in
an organization, working with other people, a person should learn how to accept any kind of feedback, analyze it in
the most positive manner possible, and use it to further impact future decision making.[17]
Examples of feedback in organizations:
•
•
•
•
•
•
•
Financial audit
Performance appraisal
Shareholders' meeting
Marketing research
360-degree feedback
Walkout
Lockout
Feedback
References
[1] H.S. Black, "Stabilized feed-back amplifiers", Electrical Engineering, vol. 53, pp. 114-120, Jan. 1934.
[2] Peter M. Senge (1990). The Fifth Discipline: The Art and Practice of the Learning Organization. New York: Doubleday. pp. 424.
ISBN 0-385-260-946.
[3] Richard R Spencer & Ghausi MS (2003). Introduction to electronic circuit design (http:/ / worldcat. org/ isbn/ 0-201-36183-3). Upper Saddle
River NJ: Prentice Hall/Pearson Education. pp. 661. ISBN 0-201-36183-3. .
[4] Rosenblueth A, Wiener N & Bigelow J: Behavior, Purpose and Teleology (http:/ / links. jstor. org/
sici?sici=0031-8248(194301)10:1<18:BPAT>2. 0. CO;2-8)
[5] http:/ / www. psychologytoday. com/ blog/ lets-connect/ 200909/ the-emotionalrelational-world
[6] Araki, M., PID Control (http:/ / www. eolss. net/ ebooks/ Sample Chapters/ C18/ E6-43-03-03. pdf),
[7] With mechanical devices, hunting can be severe enough to destroy the device.
[8] P. Horowitz & W. Hill, The Art of Electronics, Cambridge University Press (1980), Chapter 3, relating to operational amplifiers.
[9] H. Giese, Y. Brun, J. D. M. Serugendo, C. Gacek, H. Kienle, H. Müller, M. Pezzè, and M. Shaw (2009). "Engineering self-adaptive and
self-managing systems". Springer-Verlag.
[10] J. L. Hellerstein, Y. Diao, S. Parekh, and D. M. Tilbury (2004). Feedback Control of Computing Systems. John Wiley & Sons.
[11] J. O. Kephart and D. M. Chess (2003). "The vision of autonomic computing".
[12] H. A. Müller, H. M. Kienle, and U. Stege, (2009). "Autonomic computing: Now you see it, now you don’t—design and evolution of
autonomic software systems".
[13] Thomas, Goetz. "Feedback Loops Are Changing What People Do" (http:/ / www. wired. com/ magazine/ 2011/ 06/ ff_feedbackloop/ ).
Wired Magazine. Wired Magazine. . Retrieved 20 June 2011.
[14] Introduction to Social Macrodynamics (http:/ / cliodynamics. ru/ index. php?option=com_content& task=view& id=172& Itemid=70) by
Andrey Korotayev et al.
[15] Fleming, M., & Levie, W.H. (1993). Instructional message design: principles from the behavioral and cognitive sciences (http:/ / books.
google. com/ books?id=vLxPAAAAMAAJ& dq=intitle:Instructional+ intitle:message+ intitle:design& pgis=1) (Second Edition ed.).
Englewood Cliffs NJ: Educational Technology Publications. ISBN 0877782539. .
[16] Accreditation provides you with a structured mechanism to assess and improve the quality of your program: The two-loop feedback diagram
(http:/ / www. vanth. org/ curriculum/ twoloop. html)
[17] Folkman, Joseph R. The Power of Feedback. 1st ed. New Jersey: John Wiley & Sons Inc., 2006. Print.
Further reading
• Katie Salen and Eric Zimmerman. Rules of Play. MIT Press. 2004. ISBN 0-262-24045-9. Chapter 18: Games as
Cybernetic Systems.
• Korotayev A., Malkov A., Khaltourina D. Introduction to Social Macrodynamics: Secular Cycles and Millennial
Trends. (http://cliodynamics.ru/index.php?option=com_content&task=view&id=172&Itemid=70) Moscow:
URSS, 2006. ISBN 5-484-00559-0
• Dijk, E., Cremer, D.D., Mulder, L.B., and Stouten, J. "How Do We React to Feedback in Social Dilemmas?" In
Biel, Eek, Garling & Gustafsson, (eds.), New Issues and Paradigms in Research on Social Dilemmas, New York:
Springer, 2008.
21
Negative feedback
22
Negative feedback
Negative feedback occurs when the output
of a system acts to oppose changes to the
input of the system, with the result that the
changes are attenuated. If the overall
feedback of the system is negative, then the
system will tend to be stable.
Overview
Figure 1: Ideal feedback model. The feedback is negative if B < 0
In many physical and biological systems,
qualitatively different influences can oppose each other. For example, in biochemistry, one set of chemicals drives
the system in a given direction, whereas another set of chemicals drives it in an opposing direction. If one, or both of
these opposing influences are non-linear, equilibrium point(s) result.
In biology, this process (generally biochemical) is often referred to as homeostasis; whereas in mechanics, the more
common term is equilibrium.
In engineering, mathematics and the physical and biological sciences, common terms for the points around which the
system gravitates include: attractors, stable states, eigenstates/eigenfunctions, equilibrium points, and setpoints.
Negative refers to the sign of the multiplier in mathematical models for feedback. In delta notation,
output is
added to or mixed into the input. In multivariate systems, vectors help to illustrate how several influences can both
partially complement and partially oppose each other.
In contrast, positive feedback is feedback in which the system responds so as to increase the magnitude of any
particular perturbation, resulting in amplification of the original signal instead of stabilization. Any system where
there is a net positive feedback will result in a runaway situation. Both positive and negative feedback require a
feedback loop to operate.
Negative feedback is used to describe the act of reversing any discrepancy between desired and actual output.
Examples
Mechanical engineering
Negative feedback was first implemented in the 16th Century with the invention of the centrifugal governor. Its
operation is most easily seen in its use by James Watt to control the speed of his steam engine. Two heavy balls on
an upright frame rotate at the same speed as the engine. As their speed increases they swing up and outwards due to
the centrifugal force. This causes them to lift a mechanism which closes the steam inlet valve and the engine slows.
When the speed of the engine falls too far, the balls will fall by gravity and open the steam valve.
Control systems
Examples of the use of negative feedback to control its system are: thermostat control, phase-locked loop, hormonal
regulation (see diagram above), and temperature regulation in animals.
A simple and practical example is a thermostat. When the temperature in a heated room reaches a certain upper limit
the room heating is switched off so that the temperature begins to fall. When the temperature drops to a lower limit,
the heating is switched on again. Provided the limits are close to each other, a steady room temperature is
maintained. Similar control mechanisms are used in cooling systems, such as an air conditioner, a refrigerator, or a
freezer.
Negative feedback
23
Biology and chemistry
Some biological systems exhibit negative feedback such as the
baroreflex in blood pressure regulation and erythropoiesis. Many
biological process (e.g., in the human anatomy) use negative feedback.
Examples of this are numerous, from the regulating of body
temperature, to the regulating of blood glucose levels. The disruption
of feedback loops can lead to undesirable results: in the case of blood
glucose levels, if negative feedback fails, the glucose levels in the
blood may begin to rise dramatically, thus resulting in diabetes.
For hormone secretion regulated by the negative feedback loop: when
gland X releases hormone X, this stimulates target cells to release
hormone Y. When there is an excess of hormone Y, gland X "senses"
this and inhibits its release of hormone X.
Economics
In economics, automatic stabilisers are government programs which
work as negative feedback to dampen fluctuations in real GDP.
Electronic amplifiers
Most endocrine hormones are controlled by a
physiologic negative feedback inhibition loop,
such as the glucocorticoids secreted by the
adrenal cortex. The hypothalamus secretes
corticotropin-releasing hormone (CRH), which
directs the anterior pituitary gland to secrete
adrenocorticotropic hormone (ACTH). In turn,
ACTH directs the adrenal cortex to secrete
glucocorticoids, such as cortisol. Glucocorticoids
not only perform their respective functions
throughout the body but also negatively affect the
release of further stimulating secretions of both
the hypothalamus and the pituitary gland,
effectively reducing the output of glucocorticoids
[1]
once a sufficient amount has been released.
The negative feedback amplifier was invented by Harold Stephen
Black at Bell Laboratories in 1927, and patented by him in 1934.
Fundamentally, all electronic devices (e.g. vacuum tubes, bipolar
transistors, MOS transistors) exhibit some nonlinear behavior.
Negative feedback corrects this by trading unused gain for higher
linearity (lower distortion). An amplifier with too large an open-loop
gain, possibly in a specific frequency range, will additionally produce too large a feedback signal in that same range.
This feedback signal, when subtracted from the original input, will act to reduce the original input, also by "too
large" an amount. This "too small" input will be amplified again by the "too large" open-loop gain, creating a signal
that is "just right". The net result is a flattening of the amplifier's gain over all frequencies (desensitising). Though
much more accurate, amplifiers with negative feedback can become unstable if not designed correctly, causing them
to oscillate. Harry Nyquist of Bell Laboratories managed to work out a theory about how to make this behaviour
stable.
Negative feedback is used in this way in many types of amplification systems to stabilize and improve their
operating characteristics (see e.g., operational amplifiers).
References
[1] Raven, PH; Johnson, GB. Biology, Fifth Edition, Boston: Hill Companies, Inc. 1999. page 1058.
Negative feedback
24
External links
• http://www.biology-online.org/4/1_physiological_homeostasis.htm
Positive feedback
Positive feedback is a process in which the effects of a small
disturbance on (a perturbation of) a system include an increase in the
magnitude of the perturbation. That is, A produces more of B which in
turn produces more of A.[1] In contrast, a system that responds to a
perturbation in a way that reduces its effect is said to exhibit negative
feedback. These concepts were first recognized as broadly applicable
by Norbert Wiener in his 1948 work on cybernetics.[2]
Positive feedback tends to cause system instability, divergence from
equilibrium, oscillation and often exponential growth. When there is
more positive feedback than there are stabilizing tendencies, systems
will typically accelerate towards a non-linear region, which may
stabilise the system or (depending on the system) even damage or
destroy it. Positive feedback may end with the system 'latched' into a
new stable state. Positive feedback is controlled by signals in the
system being filtered, damped or limited, or it can cancelled or reduced
by adding negative feedback.
Figure 1: Ideal feedback model. The feedback is
positive if B > 0
Alarm or panic can spread by positive feedback
among a herd of animals to cause a stampede.
Positive feedback is used in digital electronics to force voltages away
from intermediate voltages into '0' and '1' states. On the other hand,
thermal runaway is a positive feedback that can destroy semiconductor
junctions. Positive feedback in chemical reactions can increase the rate
of reactions, and in some cases can lead to explosions. Positive
feedback in mechanical design causes tipping-point, or 'over-centre',
mechanisms to snap into position, for example in switches and locking
pliers. Out of control, it can cause bridges to collapse. Positive
feedback in economic systems can cause boom-then-bust cycles.
In sociology a network effect can quickly create
In the term "positive feedback", "positive" refers to the mathematical
the positive feedback of a bank run. The above
photo
is of the UK Northern Rock 2007 bank run.
sign of the direction of change rather than the desirability of the
See also viral video.
outcome. To avoid this confusion, it is sometimes better to use other
terms such as self-reinforcing feedback.[3] In social and financial
systems, positive feedback effects may also be referred to as 'virtuous' or 'vicious' circles.
Overview
The key feature of positive feedback is that small disturbances are amplified. When positive feedback is present,
there is some causal loop where a small change creates an effect that causes an even bigger change, like a ball rolling
down an increasingly steep hill.
When a change in a variable occurs in a system which exhibits positive feedback, the system responds by changing
that variable even more in the same direction.
Positive feedback
25
The end result of positive feedback is to amplify so that small perturbations may result in big changes. For example,
if a chemical reaction causes the release of heat, and the reaction itself happens faster at higher temperatures, then
there is a high likelihood of positive feedback. If the heat produced is not removed from the reactants fast enough,
thermal runaway can occur and very quickly lead to a chemical explosion. Equally, if a PA system microphone picks
up sounds from its own loudspeakers and these sounds are re-amplified enough, the effect can be loud squealing or
howling noises from the loudspeakers.
Formally, a system in equilibrium in which there is positive feedback to any change from its current state is said to
be in an unstable equilibrium. The magnitude of the forces which act to move such a system away from its set point
are an increasing function of the "distance" from the set point.
In the real world, positive feedback loops do not cause ever-increasing growth, but are modified by limiting effects
of some sort. Acoustic feedback causes a PA system to reach its maximum volume and so it cannot amplify any
further; a flask may crack and the chemical reactants spray into the air or spill onto the floor, where they spread out
and cool. Negative feedback effects within the same system can also modulate the effect of positive feedback so that
it may add to the responsiveness of the system but does not necessarily lead to a runaway process. Chaotic systems
exhibit positive feedback at the small scale, but non linearities prevent the systems from expanding infinitely, but do
so without reaching equilibrium.
Examples and Applications
In biology
]
In physiology
A number of examples of positive feedback systems may be found in
physiology, some of which are well described in works such as Arthur
Guyton's 'Textbook of Medical Physiology'.[4]
• One example is the onset of contractions in childbirth, known as the
Ferguson reflex. When a contraction occurs, the hormone oxytocin
is released into the body, which stimulates further contractions. This
results in contractions increasing in amplitude and frequency.[]
Positive feedback is a mechanism by which an
output is enhanced, such as protein levels.
However, in order to avoid any fluctuation in the
protein level, the mechanism is inhibited
stochastically (I), therefore when the
concentration of the activated protein (A) is past
the threshold ([I]), the loop mechanism is
activated and the concentration of A increases
exponentially if d[A]=k [A
• Another example is the process of blood clotting. The loop is
initiated when injured tissue releases signal chemicals that activate
platelets in the blood. An activated platelet releases chemicals to
activate more platelets, causing a rapid cascade and the formation of a blood clot.[]
• Lactation also involves positive feedback in that the more the baby sucks, the more milk is produced, via a surge
in prolactin secretion.[5]
• Estrogen that functions during the follicular phase of menstruation is also an example of positive feedback.[6]
• The generation of nerve signals is another example, in which the membrane of a nerve fibre causes slight leakage
of sodium ions through sodium channels, resulting in a change in the membrane potential, which in turn causes
more opening of channels, and so on. So a slight initial leakage results in an explosion of sodium leakage which
creates the nerve action potential.[7]
In most cases, such feedback loops culminate in counter-signals being released that suppress or breaks the loop.
Childbirth contractions stop when the baby is out of the mother's body. Chemicals break down the blood clot.
Lactation stops when the baby no longer nurses.[4]
Positive feedback
In evolutionary biology
Positive feedback loops have been used to describe aspects of the dynamics of change in biological evolution. For
example, beginning at the macro level, Alfred J. Lotka (1945) argued that the evolution of the species was most
essentially a matter of selection that fed back energy flows to capture more and more energy for use by living
systems.[8] At the human level, Richard Alexander (1989) proposed that social competition between and within
human groups fed back to the selection of intelligence thus constantly producing more and more refined human
intelligence.[9] Crespi (2004) discussed several other examples of positive feedback loops in evolution.[10] The
analogy of Evolutionary arms races provide further examples of positive feedback in biological systems.[11]
It have been shown that changes in
biodiversity through the Phanerozoic
correlate much better with hyperbolic model
(widely used in demography and
macrosociology) than with exponential and
logistic models (traditionally used in
population biology and extensively applied
to fossil biodiversity as well). The latter
models imply that changes in diversity are
guided by a first-order positive feedback
(more ancestors, more descendants) and/or a
negative feedback arising from resource
During the Phanerozoic the biodiversity shows a steady but not monotonic increase
from near zero to several thousands of genera.
limitation. Hyperbolic model implies a
second-order positive feedback. The
hyperbolic pattern of the world population growth has been demonstrated (see below) to arise from a second-order
positive feedback between the population size and the rate of technological growth. The hyperbolic character of
biodiversity growth can be similarly accounted for by a positive feedback between the diversity and community
structure complexity. It has been suggested that the similarity between the curves of biodiversity and human
population probably comes from the fact that both are derived from the interference of the hyperbolic trend
(produced by the positive feedback) with cyclical and stochastic dynamics.[12]
In psychology
Winner (1996) described gifted children as driven by positive feedback loops involving setting their own learning
course, this feeding back satisfaction, thus further setting their learning goals to higher levels and so on.[13] Winner
termed this positive feedback loop as a “rage to master.” Vandervert (2009a, 2009b) proposed that the child prodigy
can be explained in terms of a positive feedback loop between the output of thinking/performing in working
memory, which then is fed to the cerebellum where it is streamlined, and then fed back to working memory thus
steadily increasing the quantitative and qualitative output of working memory.[14] [15] Vandervert also argued that
this working memory/cerebellar positive feedback loop was responsible for language evolution in working memory.
26
Positive feedback
27
In electronics
Electronic amplification devices may have positive feedback signal paths intentionally added, or such paths may
come into being inadvertently. In addition, thermal positive feedback may take place in electronic circuits. This is
called thermal runaway and can be destructive.
In networking, a form of positive feedback known as a broadcast storm can result when multiple switches are
connected in such a way that they form a loop. Say for example, you have two switches, each with 4 ports. By
accident, ports 1 and 2 are connected to the other switches ports 1 and 2. A single multi-cast packet is sent, switch
one receives it, and sends it out through every port besides the one it came in on. Switch 2 receives 2 multi-casts, and
sends each of them out on every port besides the ones they came in on. Switch 1 then receives 2 again, and the
process repeats. This begins flooding the network with packets being rapidly bounced back and forth until the entire
network is crippled.
Regenerative circuits were invented and patented in 1914[16] for the
amplification and reception of very weak radio signals. Carefully
controlled positive feedback around a single transistor amplifier can
multiply its gain by 1,000 or more.[17] Therefore a signal can be
amplified 20,000 or even 100,000 times in one stage, that would
normally have a gain of only 20 to 50. The problem with regenerative
amplifiers working at these very high gains is that they easily become
unstable and start to oscillate. The radio operator has to be prepared to
tweak the amount of feedback fairly continuously for good reception.
Modern radio receivers use the superheterodyne design, with many
more amplification stages, but much more stable operation and no
positive feedback.
A vintage style regenerative radio receiver. Due
to the controlled use of positive feedback,
sufficient amplification can be derived from a
single vacuum tube or valve (centre).
The oscillation that can break out in a regenerative radio circuit can be
put to good use in the design of electronic oscillators. By the use of tuned circuits or a piezoelectric crystal
(commonly quartz), the signal that is amplified by the positive feedback remains linear and sinusoidal. There are
several designs for such harmonic oscillators, including the Armstrong oscillator, hartley oscillator, colpitts
oscillator, and the wien bridge oscillator. They all use positive feedback to maintain the oscillations.[18]
Many electronic circuits, especially amplifiers, incorporate negative feedback. This reduces their gain, but improves
their input impedance, output impedance, and bandwidth, and stabilises all of these parameters, including the
closed-loop gain. These parameters also become less dependent on the details of the amplifying device itself, and
more dependent on the feedback components, which are less likely to vary with manufacturing tolerance, age and
temperature. The difference between positive and negative feedback for AC signals is one of phase: if the signal is
fed back out of phase, the feedback is negative and if it is in-phase the feedback is positive. One problem for
amplifier designers who use negative feedback is that some of the components of the circuit will introduce phase
shift in the feedback path. If there is a frequency (usually a high frequency) where the phase shift reaches 180°, then
the designer must ensure that the amplifier gain at that frequency is very low (usually by low-pass filtering). If the
loop gain (the product of the amplifier gain and the extent of the positive feedback) at any frequency is greater than
one, then the amplifier will oscillate at that frequency (Barkhausen stability criterion). Such oscillations are
sometimes called parasitic oscillations. An amplifier that is stable in one set of conditions can break into parasitic
oscillation in another. This may be due to changes in temperature, supply voltage, adjustment of front-panel controls,
or even the proximity of a person or other conductive item. Amplifiers may oscillate gently in ways that are hard to
detect without an oscilloscope, or the oscillations may be so extensive that only a very distorted or no required signal
at all gets through, or that damage occurs. Low frequency parasitic oscillations have been called 'motorboating' due
to the similarity to the sound of a low-revving exhaust note.[19]
Positive feedback
Digital electronic circuits are sometimes designed to benefit from
positive feedback. Normal logic gates usually rely simply on gain to
push digital signal voltages away from intermediate values to the
values that are meant to represent boolean '0' and '1'. When an input
voltage is expected to vary in an analogue way, but sharp thresholds
are required for later digital processing, the Schmitt trigger circuit uses
positive feedback to ensure that if the input voltage creeps gently
above the threshold, the output is forced smartly and rapidly from one
logic state to the other. One of the corollaries of the Schmitt trigger's
use of positive feedback is that, should the input voltage move gently
down again past the same threshold, the positive feedback will hold the
The effect of using a Schmitt trigger (B) instead
output in the same state with no change. This effect is called hysteresis:
of a comparator (A)
the input voltage has to drop past a different, lower threshold to
'un-latch' the output and reset it to its original digital value. By reducing the extent of the positive feedback, the
hysteresis-width can be reduced, but it can not entirely be eradicated. The Schmitt trigger is, to some extent, a
latching circuit.[20]
An electronic latch is a circuit that has the effect of memory. For
example, a Schmitt trigger circuit in a stable state, with an input
voltage between its two threshold levels, in the middle of its hysteresis
band, can tell the observer whether the last input swing it saw was
higher or lower than the outer edges of its hysteresis band. This is the
basis of one bit of electronic memory. The actual memory circuits used
in most digital electronics are based on the 'flip-flop' or 'bistable
multivibrator'. A bistable multivibrator uses logic gates connected to
each other so that positive feedback maintains the state of the circuit
Illustration of an R-S ('reset-set') flip-flop made
after the input signal has been removed, until a suitable alternative
from two digital nor gates with positive feedback.
signal is applied to change the state.[21] This is usually a form of
Red and black mean logical '1' and '0',
respectively.
volatile memory, in the sense that removing power from the flip-flop
circuit will usually cause it to lose state. Computer random access
memory (RAM) can be made in this way, with one latching circuit for each bit of memory, eight to the byte.[22]
Thermal runaway occurs in electronic systems because some aspect of a circuit is allowed to pass more current when
it gets hotter, then the hotter it gets, the more current it passes, which, directly or indirectly, makes it pass yet more
current. The effects are usually catastrophic for the device in question. If devices have to be used near to their
maximum power-handling capacity, and thermal runaway is possible or likely under certain conditions,
improvements can usually be achieved by careful design.[23]
28
Positive feedback
29
Audio and video systems can easily be made to demonstrate positive
feedback. If a microphone picks up the amplified sound loudspeakers
in the same circuit, then howling and screeching sounds of audio
feedback (at up to the maximum power capacity of the amplifier) will
be heard, as random noise is re-amplified by positive feedback and
filtered by the characteristics of the audio system and the room.
Microphones are not the only transducers subject to this effect.
Phonograph turntables are also prone to picking up acoustic feedback,
usually in the low frequency range < 100 Hz, manifesting as a low
A phonograph turntable is prone to acoustic
rumble. Usually moving the placement or orientation (due to the
feedback
directional nature of all transducers) of the mic, turntable, or speakers
is the cure for this, although using EQ can help if the system tends to
be resonant mostly in certain bands. This resonance is due to acoustical characteristics of the room and nature of
electrical circuits.
Similarly, if a video camera is pointed at a monitor screen that
is displaying the camera's own signal, then weird repeating
patterns can be formed on the screen by positive feedback.
This video feedback effect was used in the opening sequences
to early series of the television programme Dr Who. Jimi
Hendrix helped to develop the controlled and musical use of
audio feedback in electric guitar playing,[24] and later Brian
May was a famous proponent of the technique.[25]
Video feedback.
In economics
In the World System development
The exponential growth of the world population observed until the 1970s has recently been correlated to a non-linear
second order positive feedback between the demographic growth and technological development that can be spelled
out as follows: technological growth - increase in the carrying capacity of land for people - demographic growth more people - more potential inventors - acceleration of technological growth - accelerating growth of the carrying
capacity - the faster population growth - accelerating growth of the number of potential inventors - faster
technological growth - hence, the faster growth of the Earth's carrying capacity for people, and so on (see, e.g.,
Introduction to Social Macrodynamics [26] by Andrey Korotayev et al.).
Systemic risk
Systemic risk is the risk that an amplification or leverage or positive feedback process is built into a system, this is
usually unknown, and under certain conditions this process can amplify exponentially and rapidly lead to destructive
or chaotic behavior. A Ponzi scheme is a good example of a positive-feedback system, because its output (profit) is
fed back to the input (new investors), causing rapid growth toward collapse. W. Brian Arthur has also studied and
written on positive feedback in the economy (e.g. W. Brian Arthur, 1990)[27]
Simple systems that clearly separate the inputs from the outputs are not prone to systemic risk. This risk is more
likely as the complexity of the system increases, because it becomes more difficult to see or analyze all the possible
combinations of variables in the system even under careful stress testing conditions. The more efficient a complex
Positive feedback
system is, the more likely it is to be prone to systemic risks, because it takes only a small amount of deviation to
disrupt the system. Therefore well-designed complex systems generally have built-in features to avoid this condition,
such as a small amount of friction, or resistance, or inertia, or time delay to decouple the outputs from the inputs
within the system. These factors amount to an inefficiency, but they are necessary to avoid instabilities.
Population and agriculture
Agriculture and human population can be considered to be in a positive feedback mode,[28] which means that one
drives the other with increasing intensity. It is suggested that this positive feedback system will end sometime with a
catastrophe, as modern agriculture is using up all of the easily available phosphate and is resorting to highly-efficient
monocultures which are more susceptible to systemic risk.
Prejudice, social institutions and poverty
Gunnar Myrdal described a vicious circle of increasing inequalities, and poverty, which is known as "Circular
Cumulative Causation (CCC)".[29]
In climatology
Within climate a positive feedback subsystem never acts in isolation, but is always embedded within the overall
climate system, which itself is always subject to a very powerful negative feedback, the Stefan–Boltzmann law: that
emitted radiation rises with the fourth power of temperature. Hence, on earth the gain of the overall system is always
less than one, stopping the system from suffering runaway effects. While there may have been periods of time such
as the exit from an ice age where the gain was greater than one, this has not lasted long enough for extreme effects
such as the evaporation of the oceans as is believed to have happened on Venus.
Examples of positive feedback subsystems in climatology include:
• A warmer atmosphere will melt ice and this changes the albedo which further warms the atmosphere.
• Methane hydrates can be unstable so that a warming ocean could release more methane, which is also a
greenhouse gas.
In sociology
A self-fulfilling prophecy is a social positive feedback loop between beliefs and behavior: if enough people believe
that something is true, their behavior can make it true, and observations of their behavior may turn increase belief. A
classic example is a bank run.
Another sociological example of positive feedback is the network effect. When more people are encouraged to join a
network this increases the reach of the network therefore the network expands ever more quickly. A viral video is an
example of the network effect in which links to a popular video are shared and redistributed, ensuring that more
people see the video and then re-publish the links. This is the basis for many social phenomena, including Ponzi
schemes and chain letters. In many cases population size is the limiting factor to the feedback effect.
30
Positive feedback
References
[1] Keesing, R.M. (1981). Cultural anthropology: A contemporary perspective (2nd ed.) p.149. Sydney: Holt, Rinehard & Winston, Inc.
[2] Norbert Wiener (1948), Cybernetics or Control and Communication in the Animal and the Machine, Paris, Hermann et Cie - MIT Press,
Cambridge, MA.
[3] Peter M. Senge (1990). The Fifth Discipline: The Art and Practice of the Learning Organization. New York: Doubleday. pp. 424.
ISBN 0-385-260-946.
[4] Guyton, Arthur C. (1991) Textbook of Medical Physiology. (8th ed). Philadelphia: W.B. Saunders. ISBN 0-7216-3994-1
[5] Guyton, Arthur C. (1991), p.926.
[6] Guyton, Arthur C. (1991), p.907.
[7] Guyton, Arthur C. (1991), p.59.
[8] Lotka, A. (1945). The law of evolution as a maximal principle. Human Biology, 17, 168-194.
[9] Alexander, R. (1989). Evolution of the human psyche. In P. Millar & C. Stringer (Eds.), The human revolution: Behavioral and biological
perspectives on the origins of modern humans (pp. 455-513). Princeton: Princeton University Press.
[10] Crespi B. J. (2004) Vicious circles: positive feedback in major evolutionary and ecological transitions. Trends in Ecology and Evolution, 19,
627-633.
[11] Dawkins, R. 1991. The Blind Watchmaker London: Penguin. Note: W.W. Norton also published this book, and some citations may refer to
that publication. However, the text is identical, so it depends on which book is at hand
[12] Markov A., Korotayev A. Phanerozoic marine biodiversity follows a hyperbolic trend // Palaeoworld. Volume 16, Issue 4, December 2007,
Pages 311-318 (http:/ / www. sciencedirect. com/ science?_ob=ArticleURL& _udi=B83WC-4N0HJMK-2& _user=10& _rdoc=1& _fmt=&
_orig=search& _sort=d& _docanchor=& view=c& _acct=C000050221& _version=1& _urlVersion=0& _userid=10&
md5=74a80d7c55ff987c9fc8d9c7963feab9); Markov A., Korotayev A. Hyperbolic growth of marine and continental biodiversity through the
Phanerozoic and community evolution // Journal of General Biology. Volume 69, 2008. N 3, pp. 175–194 (http:/ / elementy. ru/ genbio/
abstracts?artid=177).
[13] Winner, E. (1996). Gifted children: Myths and Realities. New York: Basic Books.
[14] Vandervert, L. (2009a). Working memory, the cognitive functions of the cerebellum and the child prodigy. In L.V. Shavinina (Ed.),
International handbook on giftedness (pp. 295-316). The Netherlands: Springer Science.
[15] Vandervert, L. (2009b). The emergence of the child prodigy 10,000 years ago: An evolutionary and developmental explanation. The Journal
of Mind and Behavior, 30, 15-32.
[16] Armstrong, E. H., U.S. Patent 1113149 (http:/ / www. google. com/ patents?vid=1113149), Wireless receiving system, 1914.
[17] Kitchin, Charles. "A SHORT WAVE REGENERATIVE RECEIVER PROJECT" (http:/ / www. electronics-tutorials. com/ receivers/
regen-radio-receiver. htm). . Retrieved 23 September 2010.
[18] "Sinewave oscillators" (http:/ / www. educypedia. be/ electronics/ analogosciltypes. htm). EDUCYPEDIA - electronics. . Retrieved 23
September 2010.
[19] Self, Douglas (2009). Audio Power Amplifier Design Handbook (http:/ / books. google. com/ books?id=Qpmi4ia2nhcC& pg=PA254&
lpg=PA254#v=onepage& q& f=false). Focal Press. pp. 254–255. ISBN 978-0240521626. .
[20] "CMOS Schmitt Trigger—A Uniquely Versatile Design Component" (http:/ / www. fairchildsemi. com/ an/ AN/ AN-140. pdf). Fairchild
Semiconductor Application Note 140. Fairchild Semiconductors. 1975. . Retrieved 29 September 2010.
[21] Strandh, Robert. "Latches and flip-flops" (http:/ / www. labri. fr/ perso/ strandh/ Teaching/ AMP/ Common/ Strandh-Tutorial/ flip-flops.
html). Laboratoire Bordelais de Recherche en Informatique. . Retrieved 4 November 2010.
[22] Wayne, Storr. "Sequential Logic Basics: SR Flip-Flop" (http:/ / www. electronics-tutorials. ws/ sequential/ seq_1. html).
Electronics-Tutorials.ws. . Retrieved 29 September 2010.
[23] Sharma, Bijay Kumar (2009). "Analog Electronics Lecture 4 Part C RC coupled Amplifier Design Procedure" (http:/ / cnx. org/ content/
m31058/ latest/ ). . Retrieved 29 September 2010.
[24] Shadwick, Keith (2003). Jimi Hendrix, Musician. Backbeat Books. p. 92. ISBN 0879307641.
[25] May, Brian. "Burns Brian May Tri-Sonic Pickups" (http:/ / www. brianmayguitars. co. uk/ accessories/ 19). House Music & Duck
Productions. . Retrieved 2 February 2011.
[26] http:/ / urss. ru/ cgi-bin/ db. pl?cp=& page=Book& id=37484& lang=en& blang=en& list=14
[27] W. Brian Arthur (February 1990). "Positive Feedbacks in the Economy". Scientific American, Vol 262. No.2, p.80
[28] Brown, A. Duncan. (2003) (http:/ / www. amazon. com/ dp/ 905727048X) Feed or Feedback. Publisher: International Books.
[29] Berger, Sebastian. "Circular Cumulative Causation (CCC) à la Myrdal and Kapp — Political Institutionalism for Minimizing Social Costs"
(http:/ / www. kwilliam-kapp. de/ pdf/ Circular Cumulative Causation a la Myrdal & Kapp. pdf). . Retrieved 26 November 2011.
31
Positive feedback
Further reading
• Norbert Wiener (1948), Cybernetics or Control and Communication in the Animal and the Machine, Paris,
Hermann et Cie - MIT Press, Cambridge, MA.
• Katie Salen and Eric Zimmerman. Rules of Play. MIT Press. 2004. ISBN 0-262-24045-9. Chapter 18: Games as
Cybernetic Systems.
32
33
Systems design
Systems design
Systems design is the process of defining the architecture, components, modules, interfaces, and data for a system to
satisfy specified requirements. One could see it as the application of systems theory to product development. There is
some overlap with the disciplines of systems analysis, systems architecture and systems engineering.[1] [2] If the
broader topic of product development "blends the perspective of marketing, design, and manufacturing into a single
approach to product development,"[3] then design is the act of taking the marketing information and creating the
design of the product to be manufactured. Systems design is therefore the process of defining and developing
systems to satisfy specified requirements of the user. Until the 1990s systems design had a crucial and respected role
in the data processing industry. In the 1990s standardization of hardware and software resulted in the ability to build
modular systems. The increasing importance of software running on generic platforms has enhanced the discipline of
software engineering.
Object-oriented analysis and design methods are becoming the most widely used methods for computer systems
design. The UML has become the standard language in object-oriented analysis and design. It is widely used for
modeling software systems and is increasingly used for high designing non-software systems and organizations.
Logical design
The logical design of a system pertains to an abstract representation of the data flows, inputs and outputs of the
system. This is often conducted via modelling, using an over-abstract (and sometimes graphical) model of the actual
system. In the context of systems design are included.
Physical design
The physical design relates to the actual input and output processes of the system. This is laid down in terms of how
data is input into a system, how it is verified/authenticated, how it is processed, and how it is displayed as output.
Physical design, in this context, does not refer to the tangible physical design of an information system. To use an
analogy, a personal computer's physical design involves input via a keyboard, processing within the CPU, and output
via a monitor, printer, etc. It would not concern the actual layout of the tangible hardware, which for a PC would be
a monitor, CPU, motherboard, hard drive, modems, video/graphics cards, USB slots, etc. it involve detail design of
user and a product database structure processing and control processor .The H/S personal specification a develop for
the proposed system .
Systems design
Alternative design methodologies
Rapid application development (RAD)
Rapid application development (RAD) is a methodology in which a systems designer produces prototypes for an
end-user. The end-user reviews the prototype, and offers feedback on its suitability. This process is repeated until the
end-user is satisfied with the final system.
Joint application design (JAD)
Joint application design (JAD) is a methodology which evolved from RAD, in which a systems designer consults
with a group consisting of the following parties:
• Executive sponsor
• Systems Designer
• Managers of the system
JAD involves a number of stages, in which the group collectively develops an agreed pattern for the design and
implementation of the system.
Systems design: Topics
• Requirements analysis - analyzes the needs of the end users or customers
• Benchmarking — is an effort to evaluate how current systems perform
• Systems architecture - creates a blueprint for the design with the necessary specifications for the hardware,
software, people and data resources. In many cases, multiple architectures are evaluated before one is selected.
• Design — designers will produce one or more 'models' of what they see a system eventually looking like, with
ideas from the analysis section either used or discarded. A document will be produced with a description of the
system, but nothing is specific — they might say 'touchscreen' or 'GUI operating system', but not mention any
specific brands;
• Computer programming and debugging in the software world, or detailed design in the consumer, enterprise or
commercial world - specifies the final system components.
• System testing - evaluates the system's actual functionality in relation to expected or intended functionality,
including all integration aspects.
References
[1] This article incorporates public domain material from the General Services Administration document "Federal Standard 1037C" (http:/ /
www. its. bldrdoc. gov/ fs-1037/ fs-1037c. htm).
[2] This article incorporates public domain material from the United States Department of Defense document "Dictionary of Military and
Associated Terms".
[3] Ulrich & Eppinger (2000). Product Design & Development. Irwin McGraw-Hill. ISBN 0-07-229647-X.
Further reading
• C. West Churchman The Design of Inquiring Systems: Basic Concepts of Systems and Organization. Basic Books,
New York, 1971, SBN 465-01608-1
• Levin, Mark Sh., Composite Systems Decisions, Springer, New York, 2006
• Maier, Mark W., and Rechtin, Eberhardt, The Art of Systems Architecting, Second Edition, CRC Press, Boca
Raton, 2000
• Saltzer, J.H, et al., End-to-End arguments in Systems Design in: ACM Transactions in Computer Systems Vol. 2,
nr 4 (Nov 1984), pp 277–288.
34
Systems design
• Ulrich, Karl T. and Eppinger, Steven D., Product Design and Development, Second Edition, Irwin McGraw-Hill,
Boston, 2000
External links
• (http://www.google.com/search?hl=en&defl=en&q=define:Systems+design&sa=X&
oi=glossary_definition&ct=title) Internet definitions
• Interactive Systems Design (http://www.dcs.gla.ac.uk/~johnson/teaching/isd/course.html) by Chris
Johnson, 1993
• Department of Systems Design Engineering (http://www.systems.uwaterloo.ca/), Faculty of Engineering at
the University of Waterloo, Ontario, Canada
• System Design and Management (http://sdm.mit.edu/), Systems Design and Management at the Massachusetts
Institute of Technology, Cambridge, MA, USA
• Course on System Design (http://www.mslevin.iitp.ru/) by Mark Sh. Levin
Requirement
In engineering, a requirement is a singular documented physical and functional need that a particular product or
service must be or perform. It is most commonly used in a formal sense in systems engineering, software
engineering, or enterprise engineering. It is a statement that identifies a necessary attribute, capability, characteristic,
or quality of a system for it to have value and utility to a user.
In the classical engineering approach, sets of requirements are used as inputs into the design stages of product
development. Requirements are also an important input into the verification process, since tests should trace back to
specific requirements. Requirements show what elements and functions are necessary for the particular project. This
is reflected in the waterfall model of the software life-cycle. However, when iterative methods of software
development or agile methods are used, the system requirements are incrementally developed in parallel with design
and implementation.
Requirements engineering is the set of activities that lead to the derivation of the system or software requirements.
Requirements engineering may involve a feasibility study, or a conceptual analysis phase of the project and
requirements elicitation (gathering, understanding, reviewing, and articulating the needs of the stakeholders) and
requirements analysis,[1] analysis (checking for consistency and completeness), specification (documenting the
requirements) and validation (making sure the specified requirements are correct).[2] [3]
Product versus process requirements
Projects are subject to three sorts of requirements:
• Business requirements describe in business terms what must be delivered or accomplished to provide value.
• Product requirements describe properties of a system or product (which could be one of several ways to
accomplish a set of business requirements.)
• Process requirements describe activities performed by the developing organization. For instance, process
requirements could specify specific methodologies that must be followed, and constraints that the organization
must obey.
Product and process requirements are closely linked. Process requirements often specify the activities that will be
performed to satisfy a product requirement. For example, a maximum development cost requirement (a process
requirement) may be imposed to help achieve a maximum sales price requirement (a product requirement); a
requirement that the product be maintainable (a Product requirement) often is addressed by imposing requirements to
35
Requirement
follow particular development styles (e.g., object-oriented programming), style-guides, or a review/inspection
process (process requirements).
Requirements in systems and software engineering
In systems engineering, a requirement can be a description of what a system must do, referred to as a Functional
Requirement. This type of requirement specifies something that the delivered system must be able to do. Another
type of requirement specifies something about the system itself, and how well it performs its functions. Such
requirements are often called Non-functional requirements, or 'performance requirements' or 'quality of service
requirements.' Examples of such requirements include usability, availability, reliability, supportability, testability and
maintainability.
A collection of requirements define the characteristics or features of the desired system. A 'good' list of requirements
as far as possible avoids saying how the system should implement the requirements, leaving such decisions to the
system designer. Specifying how the system should be implemented is called "implementation bias" or "solution
engineering". However, implementation constraints on the solution may validly be expressed by the future owner,
for example for required interfaces to external systems; for interoperability with other systems; and for commonality
(e.g. of user interfaces) with other owned products.
In software engineering, the same meanings of requirements apply, except that the focus of interest is the software
itself.
Product requirements
Types of product requirements
Requirements are typically placed into these categories:
• Architectural requirements describe what must be done by identifying the necessary system architecture of a
system,
• Functional requirements describe the functionality that the system is to execute; for example, formatting some
text or modulating a signal. They are sometimes known as capabilities.
• Non-functional requirements describe characteristics of the system that the user cannot affect or (immediately)
perceive. Nonfunctional requirements are sometimes known as quality requirements or ilities.
• Constraint requirements impose limits upon the design alternatives or project/process operations. No matter how
the problem is solved the constraint requirements must be adhered to.
Non-functional requirements can be further classified according to whether they are usability requirements, look and
feel requirements, humanity requirements, performance requirements, maintainability requirements, operational
requirements, safety requirements, reliability requirements, or one of many other types of requirements.
In software engineering this categorization is useful because only functional requirements can be directly
implemented in software. The non-functional requirements are controlled by other aspects of the system. For
example, in a computer system reliability is related to hardware failure rates, and performance is controlled by CPU
and memory. Non-functional requirements can in some cases be decomposed into functional requirements for
software. For example, a system level non-functional safety requirement can be decomposed into one or more
functional requirements. See FURPS. In addition, a non-functional requirement may be converted into a process
requirement when the requirement is not easily measurable. For example, a system level maintainability requirement
may be decomposed into restrictions on software constructs or limits on lines or code.
36
Requirement
37
Characteristics of good requirements
The characteristics of good requirements are variously stated by different writers, with each writer generally
emphasizing the characteristics most appropriate to their general discussion or the specific technology domain being
addressed. However, the following characteristics are generally acknowledged.[4]
Characteristic
Explanation
Unitary (Cohesive) The requirement addresses one and only one thing.
Complete
The requirement is fully stated in one place with no missing information.
Consistent
The requirement does not contradict any other requirement and is fully consistent with all authoritative external
documentation.
Non-Conjugated
(Atomic)
The requirement is atomic, i.e., it does not contain conjunctions. E.g., "The postal code field must validate American and
Canadian postal codes" should be written as two separate requirements: (1) "The postal code field must validate American
postal codes" and (2) "The postal code field must validate Canadian postal codes".
Traceable
The requirement meets all or part of a business need as stated by stakeholders and authoritatively documented.
Current
The requirement has not been made obsolete by the passage of time.
Feasible
The requirement can be implemented within the constraints of the project.
Unambiguous
The requirement is concisely stated without recourse to technical jargon, acronyms (unless defined elsewhere in the
Requirements document), or other esoteric verbiage. It expresses objective facts, not subjective opinions. It is subject to one
and only one interpretation. Vague subjects, adjectives, prepositions, verbs and subjective phrases are avoided. Negative
statements and compound statements are prohibited.
Mandatory
The requirement represents a stakeholder-defined characteristic the absence of which will result in a deficiency that cannot be
ameliorated. An optional requirement is a contradiction in terms.
Verifiable
The implementation of the requirement can be determined through one of four possible methods: inspection, demonstration,
test or analysis.
To the above some add Externally Observable, that is, the requirement specifies a characteristic of the product that is
externally observable or experienced by the user. Such advocates argue that requirements that specify internal
architecture, design, implementation, or testing decisions are probably constraints, and should be clearly articulated
in the Constraints section of the Requirements document. The contrasting view is that this perspective fails on two
points. First, the perspective does not recognize that the user experience may be supported by requirements not
perceivable by the user. For example, a requirement to present geocoded information to the user may be supported
by a requirement for an interface with an external third party business partner. The interface will be imperceptible to
the user, though the presentation of information obtained through the interface certainly would not. Second, a
constraint limits design alternatives, whereas a requirement specifies design characteristics. To continue the
example, a requirement selecting a web service interface is different from a constraint limiting design alternatives to
methods compatible with a Single Sign-On architecture.
Verification
All requirements should be verifiable. The most common method is by test. If this is not the case, another
verification method should be used instead (e.g. analysis, demonstration or inspection or review of design).
Certain requirements, by their very structure, are not verifiable. These include requirements that say the system must
never or always exhibit a particular property. Proper testing of these requirements would require an infinite testing
cycle. Such requirements must be rewritten to be verifiable. As stated above all requirements must be verifiable.
Non-functional requirements, which are unverifiable at the software level, must still be kept as a documentation of
customer intent. However, they may be traced to process requirements that are determined to be a practical way of
meeting them. For example, a non-functional requirement to be free from backdoors may be satisfied by replacing it
with a process requirement to use pair programming. Other non-functional requirements will trace to other system
Requirement
components and be verified at that level. For example system reliability is often verified by analysis at the system
level. Avionics software with its complicated safety requirements must follow the DO-178B development process.
Requirements analysis or requirements engineering
Requirements are prone to issues of ambiguity, incompleteness, and inconsistency. Techniques such as rigorous
inspection have been shown to help deal with these issues. Ambiguities, incompleteness, and inconsistencies that can
be resolved in the requirements phase typically cost orders of magnitude less to correct than when these same issues
are found in later stages of product development. Requirements analysis strives to address these issues.
There is an engineering trade off to consider between requirements which are too vague, and those which are so
detailed that they
1. take a long time to produce - sometimes to the point of being obsolete once completed
2. limit the implementation options available
3. are costly to produce
Documenting requirements
Requirements are usually written as a means for communication between the different stakeholders. This means that
the requirements should be easy to understand both for normal users and for developers. One common way to
document a requirement is stating what the system must do. Example: 'The contractor must deliver the product no
later than xyz date.' Other ways include use cases and user stories.
Changes in requirements
Requirements generally change with time. Once defined and approved, requirements should fall under change
control. For many projects, requirements are altered before the system is complete. This is partly due to the
complexity of computer software and the fact that users don't know what they want before they see it. This
characteristic of requirements has led to requirements management studies and practices.
Disputes regarding the necessity of rigour in software requirements
Most agile software development methodologies question the need for rigorously describing software requirements
upfront, which they consider a moving target. Instead, extreme programming for example describes requirements
informally using user stories (short summaries fitting on an index card explaining one aspect of what the system
should do), and considers it the developer's duty to directly ask the customer for clarification. Agile methodologies
also typically capture requirements in a series of automated acceptance tests.
References
[1] Stellman, Andrew; Greene, Jennifer (2005). Applied Software Project Management (http:/ / www. stellman-greene. com/ aspm/ ). O'Reilly
Media. p. 98. ISBN 978-0-596-00948-9. .
[2] Wiegers, Karl E. (2003). Software Requirements, Second Edition. Microsoft Press. ISBN 0-7356-1879-8.
[3] Young, Ralph R. (2001). Effective Requirements Practices. Addison-Wesley. ISBN 978-0201709124.
[4] Davis, Alan M. (1993). Software Requirements: Objects, Functions, and States, Second Edition. Prentice Hall. ISBN 0-13-805763-X.
External links
• Discovering System Requirements (http://www.sie.arizona.edu/sysengr/publishedPapers/ReqsSandiaReport.
pdf)
• The International Institute for Business Analysis and The IIBA's Business Analysis Body of Knowledge (http://
theiiba.org)
38
Requirement
• Writing Good Requirements (http://www.reqid.com/requirements-management/
writing-good-requirements-part1/)
• Gottesdiener, Ellen (2009). The Software Requirements Memory Jogger: A Desktop Guide to Help Business and
Technical Teams Develop and Manage Requirements (http://www.ebgconsulting.com/Pubs/srmj.php).
Addison-Wesley. ISBN 157681114X.
Systems architecture
A system architecture or systems architecture is the conceptual model that defines the structure, behavior, and
more views of a system.
An architecture description is a formal description and representation of a system, organized in a way that supports
reasoning about the structure of the system which comprises system components, the externally visible properties of
those components, the relationships (e.g. the behavior) between them, and provides a plan from which products can
be procured, and systems developed, that will work together to implement the overall system. The language for
architecture description is called the architecture description language (ADL).
History
It is important to keep in mind that the modern systems architecture did not appear out of nowhere. Systems
architecture depends heavily on practices and techniques which were developed over thousands of years in many
other fields most importantly being, perhaps, civil architecture.
Prior to the advent of digital computers, the electronics and other engineering disciplines used the term system as it
is still commonly used today. However, with the arrival of digital computers and the development of software
engineering as a separate discipline, it was often necessary to distinguish among engineered hardware artifacts,
software artifacts, and the combined artifacts. A programmable hardware artifact, or computing machine, that lacks
its software program is impotent; even as a software artifact, or program, is equally impotent unless it can be used to
alter the sequential states of a suitable (hardware) machine. However, a hardware machine and its software program
can be designed to perform an almost illimitable number of abstract and physical tasks. Within the computer and
software engineering disciplines (and, often, other engineering disciplines, such as communications), then, the term
system came to be defined as containing all of the elements necessary (which generally includes both hardware and
software) to perform a useful function.
Consequently, within these engineering disciplines, a system generally refers to a programmable hardware machine
and its included program. And a systems engineer is defined as one concerned with the complete device, both
hardware and software and, more particularly, all of the interfaces of the device, including that between hardware
and software, and especially between the complete device and its user (the CHI). The hardware engineer deals (more
or less) exclusively with the hardware device; the software engineer deals (more or less) exclusively with the
software program; and the systems engineer is responsible for seeing that the software program is capable of
properly running within the hardware device, and that the system composed of the two entities is capable of properly
interacting with its external environment, especially the user, and performing its intended function.
By analogy, then, a systems architecture makes use of elements of both software and hardware and is used to enable
design of such a composite system. A good architecture may be viewed as a 'partitioning scheme,' or algorithm,
which partitions all of the system's present and foreseeable requirements into a workable set of cleanly bounded
subsystems with nothing left over. That is, it is a partitioning scheme which is exclusive, inclusive, and exhaustive.
A major purpose of the partitioning is to arrange the elements in the sub systems so that there is a minimum of
communications needed among them. In both software and hardware, a good sub system tends to be seen to be a
meaningful "object". Moreover, a good architecture provides for an easy mapping to the user's requirements and the
39
Systems architecture
validation tests of the user's requirements. Ideally, a mapping also exists from every least element to every
requirement and test.
Types of systems architectures
Several types of systems architectures (underlain by the same fundamental principles[1] ) have been identified as
follows[2] :
•
•
•
•
•
•
•
•
Collaborative Systems (such as the Internet, intelligent transportation systems, and joint air defense systems)
Manufacturing Systems
Social Systems
Software and Information Technology Systems
Strategic Systems Architecture[3]
Hardware architecture
Software architecture
Enterprise architecture
References
[1] fundamental principles of Systems Architecture (http:/ / www. lix. polytechnique. fr/ ~golden/ systems_architecture. html#principles''The), by
Boris Golden
[2] The Art of Systems Architecture, Mark Maier and Eberhardt Rechtin, 2nd ed 2002
[3] Choosing A Strategic Systems Architecture (http:/ / www. forrester. com/ Research/ Document/ Excerpt/ 0,7211,39702,00. html), by Brad
Day
External links
•
•
•
•
Principles of system architecture (http://sysarch.pbwiki.com)
What is Systems Architecture ? (http://www.lix.polytechnique.fr/~golden/systems_architecture.html)
INCOSE Systems Architecture Working Group (http://www.incose.org/practice/techactivities/wg/sysarch/)
Journal of Systems Architecture (http://www.elsevier.com/wps/find/journaldescription.cws_home/505616/
description#description)
40
41
Dynamic models
Function model
A function model or functional model in systems engineering and software engineering is a structured
representation of the functions (activities, actions, processes, operations) within the modeled system or subject
area.[1]
A function model, also called an
activity model or process model, is a
graphical
representation
of
an
enterprise's function within a defined
scope. The purposes of the function
model are to describe the functions and
processes, assist with discovery of
information needs, help identify
opportunities, and establish a basis for
determining product and service
costs.[2]
History
The model originates in the 1950s,
after in the first half of the 20th
Example of a function model of the process of "Maintain Reparable Spares" in IDEF0
century other types of management
notation.
diagrams had already been developed.
The first known Gantt Chart was
developed in 1896 by Karol Adamiecki, who called it a harmonogram. Because Adamiecki did not publish his chart
until 1931 - and in any case his works were published in either Polish or Russian, languages not popular in the West
- the chart now bears the name of Henry Gantt (1861–1919), who designed his chart around the years 1910-1915 and
popularized it in the West.[3] [4] [5] The first structured method for documenting process flow, the flow process chart,
was introduced by Frank Gilbreth to members of American Society of Mechanical Engineers (ASME) in 1921 as the
presentation “Process Charts—First Steps in Finding the One Best Way”.[6] Gilbreth's tools quickly found their way
into industrial engineering curricula.
One of the first well defined function models, was the Functional Flow Block Diagram (FFBD) developed by the
defense-related TRW Incorporated in the 1950s.[7] In the 1960s it was exploited by the NASA to visualize the time
sequence of events in a space systems and flight missions.[8] It is further widely used in classical systems
engineering to show the order of execution of system functions.[9]
Function model
42
Functional modeling topics
Functional perspective
In systems engineering and software engineering a function model is created with a functional modeling perspective.
The functional perspective is one of the perspectives possible in business process modelling, other perspecifives are
for example behavioural, organisational or informational.[10]
A functional modeling perspective concentrates on describing the dynamic process. The main concept in this
modeling perspective is the process, this could be a function, transformation, activity, action, task etc. A well-known
example of a modeling language employing this perspective is data flow diagrams.
The perspective uses four symbols to describe a process, these being:
•
•
•
•
Process: Illustrates transformation from input to output.
Store: Data-collection or some sort of material.
Flow: Movement of data or material in the process.
External Entity: External to the modeled system, but interacts with it.
Now, with these symbols, a process can be represented as a network of these symbols. This decomposed process is a
DFD, data flow diagram.
In Dynamic Enterprise Modeling a division is made in the Control model, Function Model, Process model and
Organizational model.
Functional decomposition
Functional decomposition refers broadly to the process of resolving a
functional relationship into its constituent parts in such a way that the
original function can be reconstructed from those parts by function
composition. In general, this process of decomposition is undertaken
either for the purpose of gaining insight into the identity of the
constituent components, or for the purpose of obtaining a compressed
representation of the global function, a task which is feasible only
when the constituent processes possess a certain level of modularity.
Functional decomposition has a prominent role in computer
programming, where a major goal is to modularize processes to the
greatest extent possible. For example, a library management system
may be broken up into an inventory module, a patron information
module, and a fee assessment module. In the early decades of computer
programming, this was manifested as the "art of subroutining," as it
was called by some prominent practitioners.
Example of functional decomposition in a
Functional decomposition of engineering systems is a method for
systems analysis.
analyzing engineered systems. The basic idea is to try to divide a
system in such a way that each block of the block diagram can be described without an "and" or "or" in the
description.
This exercise forces each part of the system to have a pure function. When a system is composed of pure functions,
they can be reused, or replaced. A usual side effect is that the interfaces between blocks become simple and generic.
Since the interfaces usually become simple, it is easier to replace a pure function with a related, similar function.
Function model
43
Functional modeling methods
The functional approach is extended in multiple diagrammic techniques and modeling notations. This section gives
an overview of the important techniques in chronological order.
Functional Flow Block Diagram
The Functional flow block diagram
(FFBD) is a multi-tier, time-sequenced,
step-by-step flow diagram of the
system’s functional flow.[12] The
diagram is developed in the 1950s and
widely used in classical systems
engineering. The Functional Flow
Block Diagram is also referred to as
Functional Flow Diagram, functional
block diagram, and functional flow.[13]
Functional Flow Block Diagrams
(FFBD) usually define the detailed,
step-by-step operational and support
[11]
Functional Flow Block Diagram Format.
sequences for systems, but they are also
used effectively to define processes in
developing and producing systems. The software development processes also use FFBDs extensively. In the system
context, the functional flow steps may include combinations of hardware, software, personnel, facilities, and/or
procedures. In the FFBD method, the functions are organized and depicted by their logical order of execution. Each
function is shown with respect to its logical relationship to the execution and completion of other functions. A node
labeled with the function name depicts each function. Arrows from left to right show the order of execution of the
functions. Logic symbols represent sequential or parallel execution of functions.[14]
HIPO and IPO
HIPO for hierarchical input process output is a popular 1970s systems
analysis design aid and documentation technique[15] for representing
the modules of a system as a hierarchy and for documenting each
module.[16]
It was used to develop requirements, construct the design, and support
implementation of an expert system to demonstrate automated
rendezvous. Verification was then conducted systematically because of
the method of design and implementation.[17]
The overall design of the system is documented using HIPO charts or
structure charts. The structure chart is similar in appearance to an
organizational chart, but has been modified to show additional detail. Structure charts can be usedto display several
types of information, but are used most commonly to diagram either data structures or code structures.[16]
An expanded IPO Model.
Function model
44
N2 Chart
The N2 Chart is a diagram in the shape of a matrix,
representing functional or physical interfaces between
system elements. It is used to systematically identify,
define, tabulate, design, and analyze functional and
physical interfaces. It applies to system interfaces and
hardware and/or software interfaces.[12]
The N2 diagram has been used extensively to develop
data interfaces, primarily in the software areas. However,
it can also be used to develop hardware interfaces. The
basic N2 chart is shown in Figure 2. The system functions
are placed on the diagonal; the remainder of the squares
in the N x N matrix represent the interface inputs and
outputs. [18]
[18]
Figure 2. N2 chart definition.
Structured Analysis and Design Technique
SADT basis element.
Structured Analysis and Design Technique (SADT) is a software
engineering methodology for describing systems as a hierarchy of
functions, a diagrammatic notation for constructing a sketch for a
software application. It offers building blocks to represent entities and
activities, and a variety of arrows to relate boxes. These boxes and
arrows have an associated informal semantics.[19] SADT can be used
as a functional analysis tool of a given process, using successive levels
of details. The SADT method allows to define user needs for IT
developments, which is used in industrial Information Systems, but
also to explain and to present an activity’s manufacturing processes,
procedures.[20]
The SADT supplies a specific functional view of any enterprise by describing the functions and their relationships in
a company. These functions fulfill the objectives of a company, such as sales, order planning, product design, part
manufacturing, and human resource management. The SADT can depict simple functional relationships and can
reflect data and control flow relationships between different functions. The IDEF0 formalism is based on SADT,
developed by Douglas T. Ross in 1985.[21]
Function model
IDEF0
IDEF0 is a function modeling
methodology
for
describing
manufacturing functions, which offers a
functional modeling language for the
analysis, development, re-engineering,
and integration of information systems;
business processes; or software
engineering analysis.[22] It is part of the
IDEF family of modeling languages in
the field of software engineering, and is
built on the functional modeling
language building SADT.
The IDEF0 Functional Modeling
method is designed to model the
decisions, actions, and activities of an
organization or system.[23] It was
IDEF0 Diagram Example
derived from the established graphic
modeling language Structured Analysis
and Design Technique (SADT) developed by Douglas T. Ross and SofTech, Inc.. In its original form, IDEF0
includes both a definition of a graphical modeling language (syntax and semantics) and a description of a
comprehensive methodology for developing models.[1] The US Air Force commissioned the SADT developers to
develop a function model method for analyzing and communicating the functional perspective of a system. IDEF0
should assist in organizing system analysis and promote effective communication between the analyst and the
customer through simplified graphical devices.[23]
Business Process Model and Notation
Business Process Model and Notation
(BPMN) is a graphical representation
for specifying business processes in a
workflow. BPMN was developed by
Business
Process
Management
Initiative (BPMI), and is currently
maintained by the Object Management
Group since the two organizations
merged in 2005. The current version of
BPMN is 2.0.[24]
The Business Process Model and
Business Process Modeling Notation Example.
Notation
(BPMN)
specification
provides a graphical notation for
specifying business processes in a Business Process Diagram (BPD).[25] The objective of BPMN is to support
business process management for both technical users and business users by providing a notation that is intuitive to
business users yet able to represent complex process semantics. The BPMN specification also provides a mapping
between the graphics of the notation to the underlying constructs of execution languages, particularly BPEL4WS.[26]
45
Function model
Axiomatic Design
Axiomatic design is a top down hierarchical functional decomposition process used as a solution synthesis
framework for the analysis, development, re-engineering, and integration of products, information systems, business
processes or software engineering solutions.[27] Its structure is suited mathematically to analyze coupling between
functions in order to optimize the architectural robustness of potential functional solution models.
Other types of function models
In the field of systems and software engineering numerous specific function and functional models have been
defined. Here only a few general types will be explained.
Business function model
A Business Function Model (BFM) is a general description or category of operations performed routinely to carry
out an organization's mission. It can show the critical business processes in the context of the business area
functions. The processes in the business function model must be consistent with the processes in the value chain
models. Processes are a group of related business activities performed to produce an end product or to provide a
service. Unlike business functions that are performed on a continual basis, processes are characterized by the fact
that they have a specific beginning and an end point marked by the delivery of a desired output. The figure on the
right depicts the relationship between the business processes, business functions, and the business area’s business
reference model.[28]
Business reference model
A Business reference model is a
reference model, concentrating on the
functional and organizational aspects of
the core business of an enterprise,
service organization or government
agency. In enterprise engineering a
business reference model is part of an
Enterprise Architecture Framework or
Architecture Framework, which defines
how to organize the structure and views
associated
with
an
Enterprise
Architecture.
This FEA Business reference model depicts the relationship between the business
A reference model in general is a model
processes, business functions, and the business area’s business reference model.
of something that embodies the basic
goal or idea of something and can then
be looked at as a reference for various purposes. A business reference model is a means to describe the business
operations of an organization, independent of the organizational structure that perform them. Other types of business
reference model can also depict the relationship between the business processes, business functions, and the business
area’s business reference model. These reference model can be constructed in layers, and offer a foundation for the
analysis of service components, technology, data, and performance.
46
Function model
Operator function model
The Operator Function Model (OFM) is proposed as an alternative to traditional task analysis techniques used by
human factors engineers. An operator function model attempts to represent in mathematical form how an operator
might decompose a complex system into simpler parts and coordinate control actions and system configurations so
that acceptable overall system performance is achieved. The model represents basic issues of knowledge
representation, information flow, and decision making in complex systems. Miller (1985) suggests that the network
structure can be thought of as a possible representation of an operator's internal model of the system plus a control
structure which specifies how the model is used to solve the decision problems that comprise operator control
functions.[29]
References
This article incorporates public domain material from websites or documents of the National Institute of
Standards and Technology [30].
[1] FIPS Publication 183 (http:/ / www. itl. nist. gov/ fipspubs/ idef02. doc) released of IDEFØ December 1993 by the Computer Systems
Laboratory of the National Institute of Standards and Technology (NIST).
[2] Reader's Guide to IDEF0 Function Models (http:/ / www. archives. gov/ era/ pdf/ rmsc-19951006-dod-rm-function-and-information-models.
pdf). Accessed 27 Nov 2008.
[3] H.L. Gantt, Work, Wages and Profit, published by The Engineering Magazine, New York, 1910; republished as Work, Wages and Profits,
Easton, Pennsylvania, Hive Publishing Company, 1974, ISBN 0879600489.
[4] Gerard Blokdijk, Project Management 100 Success Secrets, Lulu.com, 2007, ISBN 0980459907, Google Print, p.76 (http:/ / books. google.
com/ books?id=dgB-QWHlnrUC& pg=PA76& dq=Adamiecki+ Gantt& as_brr=3& sig=Jp-mgVODNRJpxqBRM1PYJbs7mOU)
[5] Peter W. G. Morris, The Management of Projects, Thomas Telford, 1994, ISBN 0727725939, Google Print, p.18 (http:/ / books. google. com/
books?id=5ekyoWaeZ1UC& pg=PA18-IA7& dq=Adamiecki+ Gantt& as_brr=3& sig=xe_RAipoqlvhnu0xLkIsxx-8OAQ)
[6] Ben B. Graham (2002). Detail Process Charting. p.2.
[7] Tim Weilkiens (2008). Systems Engineering with SysML/UML: Modeling, Analysis, Design. Page 287.
[8] Harold Chestnut (1967). Systems Engineering Methods. Page 254.
[9] Thomas Dufresne & James Martin (2003). "Process Modeling for E-Business" (http:/ / mason. gmu. edu/ ~tdufresn/ paper. doc). INFS 770
Methods for Information Systems Engineering: Knowledge Management and E-Business. Spring 2003
[10] Process perspectives (http:/ / users. jyu. fi/ ~jpt/ ME2000/ Me14/ sld004. htm). In: Metamodeling and method engineering, Minna Koskinen,
2000.
[11] Systems Engineering Fundamentals. (http:/ / www. dau. mil/ pubs/ pdf/ SEFGuide 01-01. pdf) Defense Acquisition University Press, 2001
[12] The first version of this article is completely based on the NAS SYSTEM ENGINEERING MANUAL SECTION (http:/ / www. faa. gov/
about/ office_org/ headquarters_offices/ ato/ service_units/ operations/ sysengsaf/ seman/ SEM3. 1/ Section 4. 4. pdf) 4.4 VERSION 3.1
06/06/06.
[13] Task Analysis Tools Used Throughout Development (http:/ / www. hf. faa. gov/ webtraining/ Usability/ u17_tools2. htm). FAA 2008.
Retrieved 25 Sept 2008.
[14] FAA (2006). NAS SYSTEM ENGINEERING MANUAL SECTION (http:/ / www. faa. gov/ about/ office_org/ headquarters_offices/ ato/
service_units/ operations/ sysengsaf/ seman/ SEM3. 1/ Section 4. 4. pdf) 4.4 VERSION 3.1 06/06/06.
[15] IBM Corporation (1974).HIPO—A Design Aid and Documentation Technique, Publication Number GC20-1851, IBM Corporation, White
Plains, NY, 1974.
[16] Sandia National Laboratories (1992). Sandia Software Guidelines Volume 5 Tools, Techniques,and Methodologies (http:/ / www. prod.
sandia. gov/ cgi-bin/ techlib/ access-control. pl/ 1985/ 852348. pdf) SANDIA REPORTS 85–2348qUC–32
[17] Mary Ann Goodwin and Charles C. Robertson (1986). EXPERT SYSTEM VERIFICATION CONCERNS IN AN OPERATIONS
ENVIRONMENT (http:/ / ntrs. nasa. gov/ archive/ nasa/ casi. ntrs. nasa. gov/ 19880007850_1988007850. pdf). NASA paper N88-17234.
[18] NASA (1995). "Techniques of Functional Analysis". In: NASA Systems Engineering Handbook (http:/ / human. space. edu/ old/ docs/
Systems_Eng_Handbook. pdf) June 1995. p.142.
[19] John Mylopoulos (2004). Conceptual Modelling III. Structured Analysis and Design Technique (SADT) (http:/ / www. cs. toronto. edu/
~jm/ 2507S/ Notes04/ SADT. pdf). Retrieved 21 Sep 2008.
[20] SADT (http:/ / www. free-logistics. com/ index. php/ Download-document/ 22-SADT_eng. html) at Free-logisitcs.com. Retrieved 21 Sep
2008.
[21] Gavriel Salvendy (2001). Handbook of Industrial Engineering: Technology and Operations Management.. p.508.
[22] Systems Engineering Fundamentals. (http:/ / www. dau. mil/ pubs/ pdf/ SEFGuide 01-01. pdf) Defense Acquisition University Press, 2001.
[23] Varun Grover, William J. Kettinger (2000). Process Think: Winning Perspectives for Business Change in the Information Age. p.168.
[24] "BPMN Information" (http:/ / www. bpmn. org/ Documents/ FAQ. htm). . Retrieved 2008-11-02.
47
Function model
48
[25] Richard C. Simpson (2004). An XML Representation for Crew Procedures (http:/ / ntrs. nasa. gov/ archive/ nasa/ casi. ntrs. nasa. gov/
20050202022_2005202152. pdf). Final Report NASA Faculty Fellowship Program - 2004. Johnson Space Center.
[26] S.A. White, "Business Process Modeling Notation (BPMN)," In: Business Process Management Initiative (BPMI) 3 May 2004.
[27] Suh (2001). Axiomatic Design: Advances and Applications, Oxford University Press, 2001, ISBN 0-19-513466-4
[28] Analyze the Business and Define the Target Business Environment (http:/ / www. doi. gov/ ocio/ architecture/ mbt/ step3. htm). Accessed 27
Nov 2008.
[29] Operator Function Model (OFM) (http:/ / www2. hf. faa. gov/ workbenchtools/ default. aspx?rPage=Tooldetails& toolID=192). Accessed 27
Nov 2008.
[30] http:/ / www. nist. gov
Function block diagram
A function block diagram (FBD) is a block diagram that describes a function between input variables and output
variables. A function is described as a set of elementary blocks. Input and output variables are connected to blocks
by connection lines. An output of a block may also be connected to an input of another block:
Inputs and outputs of the blocks are
wired together with connection lines,
or links. Single lines may be used to
connect two logical points of the
diagram:
• An input variable and an input of a
block
• An output of a block and an input of
another block
• An output of a block and an output
variable
Functional block diagram of the attitude control and maneuvering electronics system.
June 1962.
The connection is oriented, meaning
that the line carries associated data
from the left end to the right end. The left and right ends of the connection line must be of the same type.
Multiple right connection, also called divergence can be used to broadcast information from its left end to each of its
right ends. All ends of the connection must be of the same type.
Function block diagram is one of five languages for logic or control configuration supported by standard IEC
61131-3 for a control system such as a Programmable Logic Controller (PLC) or a Distributed Control System
(DCS). The other supported languages are ladder logic, sequential function chart, structured text, and instruction list.
Transfer function
49
Transfer function
A transfer function (also known as the system function[1] or network function) is a mathematical representation,
in terms of spatial or temporal frequency, of the relation between the input and output of a linear time-invariant
system. With optical imaging devices, for example, it is the Fourier transform of the point spread function (hence a
function of spatial frequency) i.e. the intensity distribution caused by a point object in the field of view.
Explanation
Transfer functions are commonly used in the analysis of systems such as single-input single-output filters, typically
within the fields of signal processing, communication theory, and control theory. The term is often used exclusively
to refer to linear, time-invariant systems (LTI), as covered in this article. Most real systems have non-linear
input/output characteristics, but many systems, when operated within nominal parameters (not "over-driven") have
behavior that is close enough to linear that LTI system theory is an acceptable representation of the input/output
behavior.
In its simplest form for continuous-time input signal
mapping of the Laplace transform of the input,
and output
, to the output
, the transfer function is the linear
:
or
where
is the transfer function of the LTI system.
In discrete-time systems, the function is similarly written as
(see Z transform) and is often referred
to as the pulse-transfer function.
Direct derivation from differential equations
Consider a linear differential equation with constant coefficients
where u and r are suitably smooth functions of t, and L is the operator defined on the relevant function space, that
transforms u into r. That kind of equation can be used to constrain the output function u in terms of the forcing
function r. The transfer function, written as an operator
, is the right inverse of L, since
.
Solutions of the homogeneous equation
can be found by trying
. That substitution yields the
characteristic polynomial
The inhomogeneous case can be easily solved if the input function r is also of the form
substituting
one finds that
. In that case, by
if and only if
Taking that as the definition of the transfer function[2] requires careful disambiguation between complex vs. real
values, which is traditionally influenced by the interpretation of abs(H(s)) as the gain and -atan(H(s)) as the phase
lag.
Transfer function
50
Signal processing
Let
be the input to a general linear time-invariant system, and
transform of
and
be the output, and the bilateral Laplace
be
.
Then the output is related to the input by the transfer function
as
and the transfer function itself is therefore
.
In particular, if a complex harmonic signal with a sinusoidal component with amplitude
, angular frequency
and phase
where
is input to a linear time-invariant system, then the corresponding component in the output is:
and
Note that, in a linear time-invariant system, the input frequency has not changed, only the amplitude and the
phase angle of the sinusoid has been changed by the system. The frequency response
describes this change
for every frequency
in terms of gain:
and phase shift:
.
The phase delay (i.e., the frequency-dependent amount of delay introduced to the sinusoid by the transfer function)
is:
.
The group delay (i.e., the frequency-dependent amount of delay introduced to the envelope of the sinusoid by the
transfer function) is found by computing the derivative of the phase shift with respect to angular frequency ,
.
The transfer function can also be shown using the Fourier transform which is only a special case of the bilateral
Laplace transform for the case where
.
Transfer function
Common transfer function families
While any LTI system can be described by some transfer function or another, there are certain "families" of special
transfer functions that are commonly used. Typical infinite impulse response filters are designed to implement one of
these special transfer functions.
Some common transfer function families and their particular characteristics are:
•
•
•
•
•
•
•
•
•
Butterworth filter – maximally flat in passband and stopband for the given order
Chebyshev filter (Type I) – maximally flat in stopband, sharper cutoff than Butterworth of same order
Chebyshev filter (Type II) – maximally flat in passband, sharper cutoff than Butterworth of same order
Bessel filter – best pulse response for a given order because it has no group delay ripple
Elliptic filter – sharpest cutoff (narrowest transition between pass band and stop band) for the given order
Optimum "L" filter
Gaussian filter – minimum group delay; gives no overshoot to a step function.
Hourglass filter
Raised-cosine filter
Control engineering
In control engineering and control theory the transfer function is derived using the Laplace transform.
The transfer function was the primary tool used in classical control engineering. However, it has proven to be
unwieldy for the analysis of multiple-input multiple-output (MIMO) systems, and has been largely supplanted by
state space representations for such systems. In spite of this, a transfer matrix can be always obtained for any linear
system, in order to analyze its dynamics and other properties: each element of a transfer matrix is a transfer function
relating a particular input variable to an output variable.
Optics
In optics, modulation transfer function indicates the capability of optical contrast transmission.
For example, when observing a series of black-white-light fringes drawn with a specific spatial frequency, the image
quality may decay. White fringes fade while black ones turn brighter.
The modulation transfer function in a specific spatial frequency is defined by:
Where modulation (M) is computed from the following image or light brightness:
51
Transfer function
52
References
[1] Bernd Girod, Rudolf Rabenstein, Alexander Stenger, Signals and systems, 2nd ed., Wiley, 2001, ISBN 0471988006 p. 50
[2] The transfer function is defined by
in, e.g., Birkhoff, Garrett; Rota, Gian-Carlo (1978). Ordinary differential equations. New
York: John Wiley & Sons. ISBN 0-471-05224-8.
External links
• Transfer function (http://planetmath.org/?op=getobj&amp;from=objects&amp;id=5394) on PlanetMath
• ECE 209: Review of Circuits as LTI Systems (http://www.tedpavlic.com/teaching/osu/ece209/support/
circuits_sys_review.pdf) — Short primer on the mathematical analysis of (electrical) LTI systems.
• ECE 209: Sources of Phase Shift (http://www.tedpavlic.com/teaching/osu/ece209/lab3_opamp_FO/
lab3_opamp_FO_phase_shift.pdf) — Gives an intuitive explanation of the source of phase shift in two simple
LTI systems. Also verifies simple transfer functions by using trigonometric identities.
Convolution
In mathematics and, in particular,
functional analysis, convolution is a
mathematical operation on two
functions f and g, producing a third
function that is typically viewed as a
modified version of one of the original
functions. Convolution is similar to
cross-correlation. It has applications
that include probability, statistics,
computer vision, image and signal
processing, electrical engineering, and
differential equations.
Convolution of two square pulses: the resulting waveform is a triangular pulse. One of the
functions (in this case g) is first reflected about
and then offset by t, making it
. The area under the resulting product gives the convolution at t. The
horizontal axis is
for f and g, and t for
.
The convolution can be defined for
functions on groups other than
Euclidean space. In particular, the
circular convolution can be defined for
periodic functions (that is, functions on
the
circle),
and
the
discrete
Convolution of a square pulse (as input signal) with the impulse response of an RC circuit
convolution can be defined for
to obtain the output signal waveform. The integral of their product is the area of the
yellow region. In both animations the function g is symmetric, and so is unchanged under
functions on the set of integers. These
reflection.
generalizations of the convolution have
applications in the field of numerical
analysis and numerical linear algebra, and in the design and implementation of finite impulse response filters in
signal processing.
Computing the inverse of the convolution operation is known as deconvolution.
History
The operation
Convolution
53
is a particular case of composition products considered by the Italian mathematician Vito Volterra in 1913.[1]
Convolution is also sometimes called "Faltung" (which means folding in German); both Faltung and convolution
were used as early as 1903, though the definition is rather unfamiliar in older uses.[2] [3] The term Faltung was
sometimes used in English through the 1940s, before the notion of convolution became widely used, along with
other terms such as composition product, superposition integral, and Carson's integral.[4]
Definition
The convolution of ƒ and g is written ƒ∗g, using an asterisk or star. It is defined as the integral of the product of the
two functions after one is reversed and shifted. As such, it is a particular kind of integral transform:
(commutativity)
While the symbol t is used above, it need not represent the time domain. But in that context, the convolution formula
can be described as a weighted average of the function ƒ(τ) at the moment t where the weighting is given by g(−τ)
simply shifted by amount t. As t changes, the weighting function emphasizes different parts of the input function.
More generally, if f and g are complex-valued functions on Rd, then their convolution may be defined as the integral:
Visual explanation of convolution.
1. Express each function in terms of a dummy variable
2. Reflect one of the functions:
→
3. Add a time-offset, t, which allows
to slide along the -axis.
4. Start t at -∞ and slide it all the way to +∞. Wherever the two functions intersect,
find the integral of their product. In other words, compute a sliding,
weighted-average of function
, where the weighting function is
The resulting waveform (not shown here) is the convolution of functions f and
g. If f(t) is a unit impulse, the result of this process is simply g(t), which is
therefore called the impulse response.
Convolution
54
Circular convolution
When a function gT is periodic, with period T, then for functions, ƒ, such that ƒ∗gT exists, the convolution is also
periodic and identical to:
where to is an arbitrary choice. The summation is called a periodic summation of the function ƒ.
If gT is a periodic summation of another function, g, then ƒ∗gT is known as a circular, cyclic, or periodic convolution
of ƒ and g.
Discrete convolution
For complex-valued functions f, g defined on the set Z of integers, the discrete convolution of f and g is given by:
(commutativity)
When multiplying two polynomials, the coefficients of the product are given by the convolution of the original
coefficient sequences, extended with zeros where necessary to avoid undefined terms; this is known as the Cauchy
product of the coefficients of the two polynomials.
Circular discrete convolution
When a function gN is periodic, with period N, then for functions, f, such that f∗gN exists, the convolution is also
periodic and identical to:
The summation on k is called a periodic summation of the function f.
If gN is a periodic summation of another function, g, then f∗gN is known as a circular convolution of f and g.
When the non-zero durations of both f and g are limited to the interval [0, N − 1], f∗gN reduces to these common
forms:
(Eq.1)
The notation
for cyclic convolution denotes convolution over the cyclic group of integers modulo N.
Circular convolution is frequently used to characterized systems analyzed through the lens of the Discrete Fourier
Transform.
Convolution
Fast convolution algorithms
In many situations, discrete convolutions can be converted to circular convolutions so that fast transforms with a
convolution property can be used to implement the computation. For example, convolution of digit sequences is the
kernel operation in multiplication of multi-digit numbers, which can therefore be efficiently implemented with
transform techniques (Knuth 1997, §4.3.3.C; von zur Gathen & Gerhard 2003, §8.2).
Eq.1 requires N arithmetic operations per output value and N2 operations for N outputs. That can be significantly
reduced with any of several fast algorithms. Digital signal processing and other applications typically use fast
convolution algorithms to reduce the cost of the convolution to O(N log N) complexity.
The most common fast convolution algorithms use fast Fourier transform (FFT) algorithms via the circular
convolution theorem. Specifically, the circular convolution of two finite-length sequences is found by taking an FFT
of each sequence, multiplying pointwise, and then performing an inverse FFT. Convolutions of the type defined
above are then efficiently implemented using that technique in conjunction with zero-extension and/or discarding
portions of the output. Other fast convolution algorithms, such as the Schönhage–Strassen algorithm, use fast Fourier
transforms in other rings.
Domain of definition
The convolution of two complex-valued functions on Rd
is well-defined only if ƒ and g decay sufficiently rapidly at infinity in order for the integral to exist. Conditions for
the existence of the convolution may be tricky, since a blow-up in g at infinity can be easily offset by sufficiently
rapid decay in ƒ. The question of existence thus may involve different conditions on ƒ and g.
Compactly supported functions
If ƒ and g are compactly supported continuous functions, then their convolution exists, and is also compactly
supported and continuous (Hörmander). More generally, if either function (say ƒ) is compactly supported and the
other is locally integrable, then the convolution ƒ∗g is well-defined and continuous.
Integrable functions
The convolution of ƒ and g exists if ƒ and g are both Lebesgue integrable functions (in L1(Rd)), and in this case ƒ∗g is
also integrable (Stein & Weiss 1971, Theorem 1.3). This is a consequence of Tonelli's theorem. Likewise, if
ƒ ∈ L1(Rd) and g ∈ Lp(Rd) where 1 ≤ p ≤ ∞, then ƒ∗g ∈ Lp(Rd) and
In the particular case p= 1, this shows that L1 is a Banach algebra under the convolution (and equality of the two
sides holds if f and g are non-negative almost everywhere).
More generally, Young's inequality implies that the convolution is a continuous bilinear map between suitable Lp
spaces. Specifically, if 1 ≤ p,q,r ≤ ∞ satisfy
then
so that the convolution is a continuous bilinear mapping from Lp×Lq to Lr.
55
Convolution
Functions of rapid decay
In addition to compactly supported functions and integrable functions, functions that have sufficiently rapid decay at
infinity can also be convolved. An important feature of the convolution is that if ƒ and g both decay rapidly, then ƒ∗g
also decays rapidly. In particular, if ƒ and g are rapidly decreasing functions, then so is the convolution ƒ∗g.
Combined with the fact that convolution commutes with differentiation (see Properties), it follows that the class of
Schwartz functions is closed under convolution.
Distributions
Under some circumstances, it is possible to define the convolution of a function with a distribution, or of two
distributions. If ƒ is a compactly supported function and g is a distribution, then ƒ∗g is a smooth function defined by a
distributional formula analogous to
More generally, it is possible to extend the definition of the convolution in a unique way so that the associative law
remains valid in the case where ƒ is a distribution, and g a compactly supported distribution (Hörmander 1983, §4.2).
Measures
The convolution of any two Borel measures μ and ν of bounded variation is the measure λ defined by
This agrees with the convolution defined above when μ and ν are regarded as distributions, as well as the
convolution of L1 functions when μ and ν are absolutely continuous with respect to the Lebesgue measure.
The convolution of measures also satisfies the following version of Young's inequality
where the norm is the total variation of a measure. Because the space of measures of bounded variation is a Banach
space, convolution of measures can be treated with standard methods of functional analysis that may not apply for
the convolution of distributions.
Properties
Algebraic properties
The convolution defines a product on the linear space of integrable functions. This product satisfies the following
algebraic properties, which formally mean that the space of integrable functions with the product given by
convolution is a commutative algebra without identity (Strichartz 1994, §3.3). Other linear spaces of functions, such
as the space of continuous functions of compact support, are closed under the convolution, and so also form
commutative algebras.
Commutativity
Associativity
Distributivity
56
Convolution
57
Associativity with scalar multiplication
for any real (or complex) number
.
Multiplicative identity
No algebra of functions possesses an identity for the convolution. The lack of identity is typically not a major
inconvenience, since most collections of functions on which the convolution is performed can be convolved with a
delta distribution or, at the very least (as is the case of L1) admit approximations to the identity. The linear space of
compactly supported distributions does, however, admit an identity under the convolution. Specifically,
where δ is the delta distribution.
Inverse element
Some distributions have an inverse element for the convolution, S(−1), which is defined by
The set of invertible distributions forms an abelian group under the convolution.
Complex conjugation
Integration
If ƒ and g are integrable functions, then the integral of their convolution on the whole space is simply obtained as the
product of their integrals:
This follows from Fubini's theorem. The same result holds if ƒ and g are only assumed to be nonnegative measurable
functions, by Tonelli's theorem.
Differentiation
In the one-variable case,
where d/dx is the derivative. More generally, in the case of functions of several variables, an analogous formula
holds with the partial derivative:
A particular consequence of this is that the convolution can be viewed as a "smoothing" operation: the convolution
of ƒ and g is differentiable as many times as ƒ and g are together.
These identities hold under the precise condition that ƒ and g are absolutely integrable and at least one of them has an
absolutely integrable (L1) weak derivative, as a consequence of Young's inequality. For instance, when ƒ is
continuously differentiable with compact support, and g is an arbitrary locally integrable function,
These identities also hold much more broadly in the sense of tempered distributions if one of ƒ or g is a compactly
supported distribution or a Schwartz function and the other is a tempered distribution. On the other hand, two
positive integrable and infinitely differentiable functions may have a nowhere continuous convolution.
Convolution
58
In the discrete case, the difference operator D ƒ(n) = ƒ(n + 1) − ƒ(n) satisfies an analogous relationship:
Convolution theorem
The convolution theorem states that
where
denotes the Fourier transform of
, and
is a constant that depends on the specific normalization
of the Fourier transform (see “Properties of the Fourier transform”). Versions of this theorem also hold for the
Laplace transform, two-sided Laplace transform, Z-transform and Mellin transform.
See also the less trivial Titchmarsh convolution theorem.
Translation invariance
The convolution commutes with translations, meaning that
where τxƒ is the translation of the function ƒ by x defined by
If ƒ is a Schwartz function, then τxƒ is the convolution with a translated Dirac delta function τxƒ = ƒ∗τx δ. So
translation invariance of the convolution of Schwartz functions is a consequence of the associativity of convolution.
Furthermore, under certain conditions, convolution is the most general translation invariant operation. Informally
speaking, the following holds
• Suppose that S is a linear operator acting on functions which commutes with translations: S(τxƒ) = τx(Sƒ) for all x.
Then S is given as convolution with a function (or distribution) gS; that is Sƒ = gS∗ƒ.
Thus any translation invariant operation can be represented as a convolution. Convolutions play an important role in
the study of time-invariant systems, and especially LTI system theory. The representing function gS is the impulse
response of the transformation S.
A more precise version of the theorem quoted above requires specifying the class of functions on which the
convolution is defined, and also requires assuming in addition that S must be a continuous linear operator with
respect to the appropriate topology. It is known, for instance, that every continuous translation invariant continuous
linear operator on L1 is the convolution with a finite Borel measure. More generally, every continuous translation
invariant continuous linear operator on Lp for 1 ≤ p < ∞ is the convolution with a tempered distribution whose
Fourier transform is bounded. To wit, they are all given by bounded Fourier multipliers.
Convolutions on groups
If G is a suitable group endowed with a measure λ, and if f and g are real or complex valued integrable functions
on G, then we can define their convolution by
In typical cases of interest G is a locally compact Hausdorff topological group and λ is a (left-) Haar measure. In that
case, unless G is unimodular, the convolution defined in this way is not the same as
. The
preference of one over the other is made so that convolution with a fixed function g commutes with left translation in
the group:
Convolution
Furthermore, the convention is also required for consistency with the definition of the convolution of measures given
below. However, with a right instead of a left Haar measure, the latter integral is preferred over the former.
On locally compact abelian groups, a version of the convolution theorem holds: the Fourier transform of a
convolution is the pointwise product of the Fourier transforms. The circle group T with the Lebesgue measure is an
immediate example. For a fixed g in L1(T), we have the following familiar operator acting on the Hilbert space
L2(T):
The operator T is compact. A direct calculation shows that its adjoint T* is convolution with
By the commutativity property cited above, T is normal: T*T = TT*. Also, T commutes with the translation
operators. Consider the family S of operators consisting of all such convolutions and the translation operators. Then
S is a commuting family of normal operators. According to spectral theory, there exists an orthonormal basis {hk}
that simultaneously diagonalizes S. This characterizes convolutions on the circle. Specifically, we have
which are precisely the characters of T. Each convolution is a compact multiplication operator in this basis. This can
be viewed as a version of the convolution theorem discussed above.
A discrete example is a finite cyclic group of order n. Convolution operators are here represented by circulant
matrices, and can be diagonalized by the discrete Fourier transform.
A similar result holds for compact groups (not necessarily abelian): the matrix coefficients of finite-dimensional
unitary representations form an orthonormal basis in L2 by the Peter–Weyl theorem, and an analog of the
convolution theorem continues to hold, along with many other aspects of harmonic analysis that depend on the
Fourier transform.
Convolution of measures
Let G be a topological group. If μ and ν are finite Borel measures on a group G, then their convolution μ∗ν is
defined by
for each measurable subset E of G. The convolution is also a finite measure, whose total variation satisfies
In the case when G is locally compact with (left-)Haar measure λ, and μ and ν are absolutely continuous with respect
to a λ, so that each has a density function, then the convolution μ∗ν is also absolutely continuous, and its density
function is just the convolution of the two separate density functions.
If μ and ν are probability measures, then the convolution μ∗ν is the probability distribution of the sum X + Y of two
independent random variables X and Y whose respective distributions are μ and ν.
59
Convolution
Bialgebras
Let (X, Δ, ∇, ε, η) be a bialgebra with comultiplication Δ, multiplication ∇, unit η, and counit ε. The convolution is a
product defined on the endomorphism algebra End(X) as follows. Let φ, ψ ∈ End(X), that is, φ,ψ : X → X are
functions that respect all algebraic structure of X, then the convolution φ∗ψ is defined as the composition
The convolution appears notably in the definition of Hopf algebras (Kassel 1995, §III.3). A bialgebra is a Hopf
algebra if and only if it has an antipode: an endomorphism S such that
Applications
Convolution and related operations are found in many applications of engineering and mathematics.
• In electrical engineering, the convolution of one function (the input signal) with a second function (the impulse
response) gives the output of a linear time-invariant system (LTI). At any given moment, the output is an
accumulated effect of all the prior values of the input function, with the most recent values typically having the
most influence (expressed as a multiplicative factor). The impulse response function provides that factor as a
function of the elapsed time since each input value occurred.
•
•
•
•
•
•
•
•
•
•
• In digital signal processing and image processing applications, the entire input function is often available for
computing every sample of the output function. In that case, the constraint that each output is the effect of only
prior inputs can be relaxed.
• Convolution amplifies or attenuates each frequency component of the input independently of the other
components.
In statistics, as noted above, a weighted moving average is a convolution.
In probability theory, the probability distribution of the sum of two independent random variables is the
convolution of their individual distributions.
In optics, many kinds of "blur" are described by convolutions. A shadow (e.g., the shadow on the table when you
hold your hand between the table and a light source) is the convolution of the shape of the light source that is
casting the shadow and the object whose shadow is being cast. An out-of-focus photograph is the convolution of
the sharp image with the shape of the iris diaphragm. The photographic term for this is bokeh.
Similarly, in digital image processing, convolutional filtering plays an important role in many important
algorithms in edge detection and related processes.
In linear acoustics, an echo is the convolution of the original sound with a function representing the various
objects that are reflecting it.
In artificial reverberation (digital signal processing, pro audio), convolution is used to map the impulse response
of a real room on a digital audio signal (see previous and next point for additional information).
In time-resolved fluorescence spectroscopy, the excitation signal can be treated as a chain of delta pulses, and the
measured fluorescence is a sum of exponential decays from each delta pulse.
In radiotherapy treatment planning systems, most part of all modern codes of calculation applies a
convolution-superposition algorithm.
In physics, wherever there is a linear system with a "superposition principle", a convolution operation makes an
appearance.
In kernel density estimation, a distribution is estimated from sample points by convolution with a kernel, such as
an isotropic Gaussian. (Diggle 1995).
• In computational fluid dynamics, the large eddy simulation (LES) turbulence model uses the convolution
operation to lower the range of length scales necessary in computation thereby reducing computational cost.
60
Convolution
Notes
[1] According to [Lothar von Wolfersdorf (2000), "Einige Klassen quadratischer Integralgleichungen", Sitzungsberichte der Sächsischen
Akademie der Wissenschaften zu Leipzig, Mathematisch-naturwissenschaftliche Klasse, volume 128, number 2, 6–7], the source is Volterra,
Vito (1913), "Leçons sur les fonctions de linges". Gauthier-Villars, Paris 1913.
[2] John Hilton Grace and Alfred Young (1903). The algebra of invariants (http:/ / books. google. com/ books?id=NIe4AAAAIAAJ&
pg=PA40). Cambridge University Press. p. 40. .
[3] Leonard Eugene Dickson (1914). Algebraic invariants (http:/ / books. google. com/ books?id=LRGoAAAAIAAJ& pg=PA85). J. Wiley.
p. 85. .
[4] R. N. Bracewell (2005). "Early work on imaging theory in radio astronomy" (http:/ / books. google. com/ books?id=v2SqL0zCrwcC&
pg=PA172). In W. T. Sullivan. The Early Years of Radio Astronomy: Reflections Fifty Years After Jansky's Discovery. Cambridge University
Press. p. 172. ISBN 9780521616027. .
References
• Bracewell, R. (1986), The Fourier Transform and Its Applications (2nd ed.), McGraw–Hill, ISBN 0071160434.
• Hewitt, Edwin; Ross, Kenneth A. (1979), Abstract harmonic analysis. Vol. I, Grundlehren der Mathematischen
Wissenschaften [Fundamental Principles of Mathematical Sciences], 115 (2nd ed.), Berlin, New York:
Springer-Verlag, ISBN 978-3-540-09434-0, MR551496.
• Hewitt, Edwin; Ross, Kenneth A. (1970), Abstract harmonic analysis. Vol. II: Structure and analysis for compact
groups. Analysis on locally compact Abelian groups, Die Grundlehren der mathematischen Wissenschaften, Band
152, Berlin, New York: Springer-Verlag, MR0262773.
• Hörmander, L. (1983), The analysis of linear partial differential operators I, Grundl. Math. Wissenschaft., 256,
Springer, ISBN 3-540-12104-8, MR0717035.
• Kassel, Christian (1995), Quantum groups, Graduate Texts in Mathematics, 155, Berlin, New York:
Springer-Verlag, ISBN 978-0-387-94370-1, MR1321145.
• Knuth, Donald (1997), Seminumerical Algorithms (3rd. ed.), Reading, Massachusetts: Addison–Wesley,
ISBN 0-201-89684-2.
• Rudin, Walter (1962), Fourier analysis on groups, Interscience Tracts in Pure and Applied Mathematics, No. 12,
Interscience Publishers (a division of John Wiley and Sons), New York–London, ISBN 047152364X,
MR0152834.
• Sobolev, V.I. (2001), "Convolution of functions" (http://www.encyclopediaofmath.org/index.php?title=C/
c026430), in Hazewinkel, Michiel, Encyclopedia of Mathematics, Springer, ISBN 978-1556080104.
• Stein, Elias; Weiss, Guido (1971), Introduction to Fourier Analysis on Euclidean Spaces, Princeton University
Press, ISBN 0-691-08078-X.
• Strichartz, R. (1994), A Guide to Distribution Theory and Fourier Transforms, CRC Press, ISBN 0849382734.
• Titchmarsh, E (1948), Introduction to the theory of Fourier integrals (2nd ed.), New York, N.Y.: Chelsea Pub.
Co. (published 1986), ISBN 978-0828403245.
• Uludag, A. M. (1998), "On possible deterioration of smoothness under the operation of convolution", J. Math.
Anal. Appl. 227 no. 2, 335–358
• Treves, François (1967), Topological Vector Spaces, Distributions and Kernels, Academic Press,
ISBN 0486453529.
• von zur Gathen, J.; Gerhard, J. (2003), Modern Computer Algebra, Cambridge University Press,
ISBN 0-521-82646-2.
• Diggle, P. J. (1995), "A kernel method for smoothing point process data", Journal of the Royal Statistical Society,
Series C) 34 (1985) 138–147
61
Convolution
62
External links
• Earliest Uses: The entry on Convolution has some historical information. (http://jeff560.tripod.com/c.html)
• Convolution (http://rkb.home.cern.ch/rkb/AN16pp/node38.html#SECTION000380000000000000000), on
The Data Analysis BriefBook (http://rkb.home.cern.ch/rkb/titleA.html)
• http://www.jhu.edu/~signals/convolve/index.html Visual convolution Java Applet.
• http://www.jhu.edu/~signals/discreteconv2/index.html Visual convolution Java Applet for discrete-time
functions.
• Lectures on Image Processing: A collection of 18 lectures in pdf format from Vanderbilt University. Lecture 7 is
on 2-D convolution. (http://www.archive.org/details/Lectures_on_Image_Processing), by Alan Peters.
•
•
•
•
• http://www.vuse.vanderbilt.edu/~rap2/EECE253/EECE253_01_Intro.pdf
Convolution Kernel Mask Operation Interactive tutorial (http://micro.magnet.fsu.edu/primer/java/
digitalimaging/processing/kernelmaskoperation/)
Convolution (http://mathworld.wolfram.com/Convolution.html) at MathWorld
Freeverb3 Impulse Response Processor (http://freeverb3.sourceforge.net/): Opensource zero latency impulse
response processor with VST plugins
Stanford University CS 178 interactive Flash demo (http://graphics.stanford.edu/courses/cs178/applets/
convolution.html) showing how spatial convolution works.
Dynamical system
A dynamical system is a concept in mathematics where a fixed
rule describes the time dependence of a point in a geometrical
space. Examples include the mathematical models that describe
the swinging of a clock pendulum, the flow of water in a pipe, and
the number of fish each springtime in a lake.
At any given time a dynamical system has a state given by a set of
real numbers (a vector) that can be represented by a point in an
appropriate state space (a geometrical manifold). Small changes in
the state of the system create small changes in the numbers. The
evolution rule of the dynamical system is a fixed rule that
describes what future states follow from the current state. The rule
is deterministic; in other words, for a given time interval only one
future state follows from the current state.
Overview
The Lorenz attractor is an example of a non-linear
dynamical system. Studying this system helped give
rise to Chaos theory.
The concept of a dynamical system has its origins in Newtonian mechanics. There, as in other natural sciences and
engineering disciplines, the evolution rule of dynamical systems is given implicitly by a relation that gives the state
of the system only a short time into the future. (The relation is either a differential equation, difference equation or
other time scale.) To determine the state for all future times requires iterating the relation many times—each
advancing time a small step. The iteration procedure is referred to as solving the system or integrating the system.
Once the system can be solved, given an initial point it is possible to determine all its future positions, a collection of
points known as a trajectory or orbit.
Before the advent of fast computing machines, solving a dynamical system required sophisticated mathematical
techniques and could be accomplished only for a small class of dynamical systems. Numerical methods implemented
Dynamical system
on electronic computing machines have simplified the task of determining the orbits of a dynamical system.
For simple dynamical systems, knowing the trajectory is often sufficient, but most dynamical systems are too
complicated to be understood in terms of individual trajectories. The difficulties arise because:
• The systems studied may only be known approximately—the parameters of the system may not be known
precisely or terms may be missing from the equations. The approximations used bring into question the validity or
relevance of numerical solutions. To address these questions several notions of stability have been introduced in
the study of dynamical systems, such as Lyapunov stability or structural stability. The stability of the dynamical
system implies that there is a class of models or initial conditions for which the trajectories would be equivalent.
The operation for comparing orbits to establish their equivalence changes with the different notions of stability.
• The type of trajectory may be more important than one particular trajectory. Some trajectories may be periodic,
whereas others may wander through many different states of the system. Applications often require enumerating
these classes or maintaining the system within one class. Classifying all possible trajectories has led to the
qualitative study of dynamical systems, that is, properties that do not change under coordinate changes. Linear
dynamical systems and systems that have two numbers describing a state are examples of dynamical systems
where the possible classes of orbits are understood.
• The behavior of trajectories as a function of a parameter may be what is needed for an application. As a parameter
is varied, the dynamical systems may have bifurcation points where the qualitative behavior of the dynamical
system changes. For example, it may go from having only periodic motions to apparently erratic behavior, as in
the transition to turbulence of a fluid.
• The trajectories of the system may appear erratic, as if random. In these cases it may be necessary to compute
averages using one very long trajectory or many different trajectories. The averages are well defined for ergodic
systems and a more detailed understanding has been worked out for hyperbolic systems. Understanding the
probabilistic aspects of dynamical systems has helped establish the foundations of statistical mechanics and of
chaos.
It was in the work of Poincaré that these dynamical systems themes developed.
Basic definitions
A dynamical system is a manifold M called the phase (or state) space endowed with a family of smooth evolution
functions Φ t that for any element of
, the time, map a point of the phase space back into the phase space.
The notion of smoothness changes with applications and the type of manifold. There are several choices for the set T.
When T is taken to be the reals, the dynamical system is called a flow; and if T is restricted to the non-negative reals,
then the dynamical system is a semi-flow. When T is taken to be the integers, it is a cascade or a map; and the
restriction to the non-negative integers is a semi-cascade.
Examples
The evolution function Φ t is often the solution of a differential equation of motion
The equation gives the time derivative, represented by the dot, of a trajectory x(t) on the phase space starting at some
point x0. The vector field v(x) is a smooth function that at every point of the phase space M provides the velocity
vector of the dynamical system at that point. (These vectors are not vectors in the phase space M, but in the tangent
space TxM of the point x.) Given a smooth Φ t, an autonomous vector field can be derived from it.
There is no need for higher order derivatives in the equation, nor for time dependence in v(x) because these can be
eliminated by considering systems of higher dimensions. Other types of differential equations can be used to define
the evolution rule:
63
Dynamical system
is an example of an equation that arises from the modeling of mechanical systems with complicated constraints.
The differential equations determining the evolution function Φ t are often ordinary differential equations: in this
case the phase space M is a finite dimensional manifold. Many of the concepts in dynamical systems can be extended
to infinite-dimensional manifolds—those that are locally Banach spaces—in which case the differential equations are
partial differential equations. In the late 20th century the dynamical system perspective to partial differential
equations started gaining popularity.
Further examples
•
•
•
•
•
•
•
•
•
Logistic map
Dyadic transformation
Tent map
Double pendulum
Arnold's cat map
Horseshoe map
Baker's map is an example of a chaotic piecewise linear map
Billiards and outer billiards
Hénon map
•
•
•
•
•
•
•
•
Lorenz system
Circle map
Rössler map
Kaplan-Yorke map
List of chaotic maps
Swinging Atwood's machine
Quadratic map simulation system
Bouncing ball dynamics
Linear dynamical systems
Linear dynamical systems can be solved in terms of simple functions and the behavior of all orbits classified. In a
linear system the phase space is the N-dimensional Euclidean space, so any point in phase space can be represented
by a vector with N numbers. The analysis of linear systems is possible because they satisfy a superposition principle:
if u(t) and w(t) satisfy the differential equation for the vector field (but not necessarily the initial condition), then so
will u(t) + w(t).
Flows
For a flow, the vector field Φ(x) is a linear function of the position in the phase space, that is,
with A a matrix, b a vector of numbers and x the position vector. The solution to this system can be found by using
the superposition principle (linearity). The case b ≠ 0 with A = 0 is just a straight line in the direction of b:
When b is zero and A ≠ 0 the origin is an equilibrium (or singular) point of the flow, that is, if x0 = 0, then the orbit
remains there. For other initial conditions, the equation of motion is given by the exponential of a matrix: for an
initial point x0,
When b = 0, the eigenvalues of A determine the structure of the phase space. From the eigenvalues and the
eigenvectors of A it is possible to determine if an initial point will converge or diverge to the equilibrium point at the
64
Dynamical system
65
origin.
The distance between two different initial conditions in the case A ≠ 0 will change exponentially in most cases, either
converging exponentially fast towards a point, or diverging exponentially fast. Linear systems display sensitive
dependence on initial conditions in the case of divergence. For nonlinear systems this is one of the (necessary but not
sufficient) conditions for chaotic behavior.
Linear vector fields and a few trajectories.
Maps
A discrete-time, affine dynamical system has the form
with A a matrix and b a vector. As in the continuous case, the change of coordinates x → x + (1 − A) –1b removes the
term b from the equation. In the new coordinate system, the origin is a fixed point of the map and the solutions are of
the linear system A nx0. The solutions for the map are no longer curves, but points that hop in the phase space. The
orbits are organized in curves, or fibers, which are collections of points that map into themselves under the action of
the map.
As in the continuous case, the eigenvalues and eigenvectors of A determine the structure of phase space. For
example, if u1 is an eigenvector of A, with a real eigenvalue smaller than one, then the straight lines given by the
points along α u1, with α ∈ R, is an invariant curve of the map. Points in this straight line run into the fixed point.
There are also many other discrete dynamical systems.
Local dynamics
The qualitative properties of dynamical systems do not change under a smooth change of coordinates (this is
sometimes taken as a definition of qualitative): a singular point of the vector field (a point where v(x) = 0) will
remain a singular point under smooth transformations; a periodic orbit is a loop in phase space and smooth
deformations of the phase space cannot alter it being a loop. It is in the neighborhood of singular points and periodic
orbits that the structure of a phase space of a dynamical system can be well understood. In the qualitative study of
dynamical systems, the approach is to show that there is a change of coordinates (usually unspecified, but
computable) that makes the dynamical system as simple as possible.
Dynamical system
Rectification
A flow in most small patches of the phase space can be made very simple. If y is a point where the vector field
v(y) ≠ 0, then there is a change of coordinates for a region around y where the vector field becomes a series of
parallel vectors of the same magnitude. This is known as the rectification theorem.
The rectification theorem says that away from singular points the dynamics of a point in a small patch is a straight
line. The patch can sometimes be enlarged by stitching several patches together, and when this works out in the
whole phase space M the dynamical system is integrable. In most cases the patch cannot be extended to the entire
phase space. There may be singular points in the vector field (where v(x) = 0); or the patches may become smaller
and smaller as some point is approached. The more subtle reason is a global constraint, where the trajectory starts out
in a patch, and after visiting a series of other patches comes back to the original one. If the next time the orbit loops
around phase space in a different way, then it is impossible to rectify the vector field in the whole series of patches.
Near periodic orbits
In general, in the neighborhood of a periodic orbit the rectification theorem cannot be used. Poincaré developed an
approach that transforms the analysis near a periodic orbit to the analysis of a map. Pick a point x0 in the orbit γ and
consider the points in phase space in that neighborhood that are perpendicular to v(x0). These points are a Poincaré
section S(γ, x0), of the orbit. The flow now defines a map, the Poincaré map F : S → S, for points starting in S and
returning to S. Not all these points will take the same amount of time to come back, but the times will be close to the
time it takes x0.
The intersection of the periodic orbit with the Poincaré section is a fixed point of the Poincaré map F. By a
translation, the point can be assumed to be at x = 0. The Taylor series of the map is F(x) = J · x + O(x2), so a change
of coordinates h can only be expected to simplify F to its linear part
This is known as the conjugation equation. Finding conditions for this equation to hold has been one of the major
tasks of research in dynamical systems. Poincaré first approached it assuming all functions to be analytic and in the
process discovered the non-resonant condition. If λ1, ..., λν are the eigenvalues of J they will be resonant if one
eigenvalue is an integer linear combination of two or more of the others. As terms of the form λi – ∑ (multiples of
other eigenvalues) occurs in the denominator of the terms for the function h, the non-resonant condition is also
known as the small divisor problem.
Conjugation results
The results on the existence of a solution to the conjugation equation depend on the eigenvalues of J and the degree
of smoothness required from h. As J does not need to have any special symmetries, its eigenvalues will typically be
complex numbers. When the eigenvalues of J are not in the unit circle, the dynamics near the fixed point x0 of F is
called hyperbolic and when the eigenvalues are on the unit circle and complex, the dynamics is called elliptic.
In the hyperbolic case the Hartman–Grobman theorem gives the conditions for the existence of a continuous
function that maps the neighborhood of the fixed point of the map to the linear map J · x. The hyperbolic case is also
structurally stable. Small changes in the vector field will only produce small changes in the Poincaré map and these
small changes will reflect in small changes in the position of the eigenvalues of J in the complex plane, implying that
the map is still hyperbolic.
The Kolmogorov–Arnold–Moser (KAM) theorem gives the behavior near an elliptic point.
66
Dynamical system
Bifurcation theory
When the evolution map Φt (or the vector field it is derived from) depends on a parameter μ, the structure of the
phase space will also depend on this parameter. Small changes may produce no qualitative changes in the phase
space until a special value μ0 is reached. At this point the phase space changes qualitatively and the dynamical
system is said to have gone through a bifurcation.
Bifurcation theory considers a structure in phase space (typically a fixed point, a periodic orbit, or an invariant torus)
and studies its behavior as a function of the parameter μ. At the bifurcation point the structure may change its
stability, split into new structures, or merge with other structures. By using Taylor series approximations of the maps
and an understanding of the differences that may be eliminated by a change of coordinates, it is possible to catalog
the bifurcations of dynamical systems.
The bifurcations of a hyperbolic fixed point x0 of a system family Fμ can be characterized by the eigenvalues of the
first derivative of the system DFμ(x0) computed at the bifurcation point. For a map, the bifurcation will occur when
there are eigenvalues of DFμ on the unit circle. For a flow, it will occur when there are eigenvalues on the imaginary
axis. For more information, see the main article on Bifurcation theory.
Some bifurcations can lead to very complicated structures in phase space. For example, the Ruelle–Takens scenario
describes how a periodic orbit bifurcates into a torus and the torus into a strange attractor. In another example,
Feigenbaum period-doubling describes how a stable periodic orbit goes through a series of period-doubling
bifurcations.
Ergodic systems
In many dynamical systems it is possible to choose the coordinates of the system so that the volume (really a
ν-dimensional volume) in phase space is invariant. This happens for mechanical systems derived from Newton's laws
as long as the coordinates are the position and the momentum and the volume is measured in units of
(position) × (momentum). The flow takes points of a subset A into the points Φ t(A) and invariance of the phase
space means that
In the Hamiltonian formalism, given a coordinate it is possible to derive the appropriate (generalized) momentum
such that the associated volume is preserved by the flow. The volume is said to be computed by the Liouville
measure.
In a Hamiltonian system not all possible configurations of position and momentum can be reached from an initial
condition. Because of energy conservation, only the states with the same energy as the initial condition are
accessible. The states with the same energy form an energy shell Ω, a sub-manifold of the phase space. The volume
of the energy shell, computed using the Liouville measure, is preserved under evolution.
For systems where the volume is preserved by the flow, Poincaré discovered the recurrence theorem: Assume the
phase space has a finite Liouville volume and let F be a phase space volume-preserving map and A a subset of the
phase space. Then almost every point of A returns to A infinitely often. The Poincaré recurrence theorem was used
by Zermelo to object to Boltzmann's derivation of the increase in entropy in a dynamical system of colliding atoms.
One of the questions raised by Boltzmann's work was the possible equality between time averages and space
averages, what he called the ergodic hypothesis. The hypothesis states that the length of time a typical trajectory
spends in a region A is vol(A)/vol(Ω).
The ergodic hypothesis turned out not to be the essential property needed for the development of statistical
mechanics and a series of other ergodic-like properties were introduced to capture the relevant aspects of physical
systems. Koopman approached the study of ergodic systems by the use of functional analysis. An observable a is a
function that to each point of the phase space associates a number (say instantaneous pressure, or average height).
67
Dynamical system
68
The value of an observable can be computed at another time by using the evolution function φ t. This introduces an
operator U t, the transfer operator,
By studying the spectral properties of the linear operator U it becomes possible to classify the ergodic properties
of Φ t. In using the Koopman approach of considering the action of the flow on an observable function, the
finite-dimensional nonlinear problem involving Φ t gets mapped into an infinite-dimensional linear problem
involving U.
The Liouville measure restricted to the energy surface Ω is the basis for the averages computed in equilibrium
statistical mechanics. An average in time along a trajectory is equivalent to an average in space computed with the
Boltzmann factor exp(−βH). This idea has been generalized by Sinai, Bowen, and Ruelle (SRB) to a larger class of
dynamical systems that includes dissipative systems. SRB measures replace the Boltzmann factor and they are
defined on attractors of chaotic systems.
Nonlinear dynamical systems and chaos
Simple nonlinear dynamical systems and even piecewise linear systems can exhibit a completely unpredictable
behavior, which might seem to be random, despite the fact that they are fundamentally deterministic. This seemingly
unpredictable behavior has been called chaos. Hyperbolic systems are precisely defined dynamical systems that
exhibit the properties ascribed to chaotic systems. In hyperbolic systems the tangent space perpendicular to a
trajectory can be well separated into two parts: one with the points that converge towards the orbit (the stable
manifold) and another of the points that diverge from the orbit (the unstable manifold).
This branch of mathematics deals with the long-term qualitative behavior of dynamical systems. Here, the focus is
not on finding precise solutions to the equations defining the dynamical system (which is often hopeless), but rather
to answer questions like "Will the system settle down to a steady state in the long term, and if so, what are the
possible attractors?" or "Does the long-term behavior of the system depend on its initial condition?"
Note that the chaotic behavior of complex systems is not the issue. Meteorology has been known for years to involve
complex—even chaotic—behavior. Chaos theory has been so surprising because chaos can be found within almost
trivial systems. The logistic map is only a second-degree polynomial; the horseshoe map is piecewise linear.
Geometrical definition
A dynamical system is the tuple
, with
a manifold (locally a Banach space or Euclidean space),
the domain for time (non-negative reals, the integers, ...) and
an evolution rule t → f t (with
t
f is a diffeomorphism of the manifold to itself. So, f is a mapping of the time-domain
) such that
into the space of
diffeomorphisms of the manifold to itself. In other terms, f(t) is a diffeomorphism, for every time t in the domain
.
Measure theoretical definition
See main article Measure-preserving dynamical system.
A dynamical system may be defined formally, as a measure-preserving transformation of a sigma-algebra, the
quadruplet
. Here, X is a set, and Σ is a sigma-algebra on X, so that the pair
is a measurable
space. μ is a finite measure on the sigma-algebra, so that the triplet
is said to be Σ-measurable if and only if, for every
to preserve the measure if and only if, for every
map
, one has
is a probability space. A map
, one has
. A map
is said
. Combining the above, a
is said to be a measure-preserving transformation of X , if it is a map from X to itself, it is Σ-measurable,
and is measure-preserving. The quadruple
, for such a , is then defined to be a dynamical system.
The map embodies the time evolution of the dynamical system. Thus, for discrete dynamical systems the iterates
for integer n are studied. For continuous dynamical systems, the map is understood to be
Dynamical system
a finite time evolution map and the construction is more complicated.
Examples of dynamical systems
Internal links
•
•
•
•
•
•
•
•
•
•
•
•
Arnold's cat map
Baker's map is an example of a chaotic piecewise linear map
Circle map
Double pendulum
Billiards and Outer Billiards
Hénon map
Horseshoe map
Irrational rotation
List of chaotic maps
Logistic map
Lorenz system
Rossler map
External links
• Interactive applet for the Standard and Henon Maps [1] by A. Luhn
Multidimensional generalization
Dynamical systems are defined over a single independent variable, usually thought of as time. A more general class
of systems are defined over multiple independent variables and are therefore called multidimensional systems. Such
systems are useful for modeling, for example, image processing.
References
[1] http:/ / complexity. xozzox. de/ nonlinmappings. html
Further reading
Works providing a broad coverage:
• Ralph Abraham and Jerrold E. Marsden (1978). Foundations of mechanics. Benjamin–Cummings.
ISBN 0-8053-0102-X. (available as a reprint: ISBN 0-201-40840-6)
• Encyclopaedia of Mathematical Sciences (ISSN 0938-0396) has a sub-series on dynamical systems with reviews
of current research.
• Anatole Katok and Boris Hasselblatt (1996). Introduction to the modern theory of dynamical systems. Cambridge.
ISBN 0-521-57557-5.
• Christian Bonatti, Lorenzo J. Díaz, Marcelo Viana (2005). Dynamics Beyond Uniform Hyperbolicity: A Global
Geometric and Probabilistic Perspective. Springer. ISBN 3-540-22066-6.
• Stephen Smale (1967). "Differential dynamical systems". Bulletin of the American Mathematical Society 73 (6):
747–817. doi:10.1090/S0002-9904-1967-11798-1.
Introductory texts with a unique perspective:
• V. I. Arnold (1982). Mathematical methods of classical mechanics. Springer-Verlag. ISBN 0-387-96890-3.
• Jacob Palis and Wellington de Melo (1982). Geometric theory of dynamical systems: an introduction.
Springer-Verlag. ISBN 0-387-90668-1.
69
Dynamical system
• David Ruelle (1989). Elements of Differentiable Dynamics and Bifurcation Theory. Academic Press.
ISBN 0-12-601710-7.
• Tim Bedford, Michael Keane and Caroline Series, eds. (1991). Ergodic theory, symbolic dynamics and hyperbolic
spaces. Oxford University Press. ISBN 0-19-853390-X.
• Ralph H. Abraham and Christopher D. Shaw (1992). Dynamics—the geometry of behavior, 2nd edition.
Addison-Wesley. ISBN 0-201-56716-4.
Textbooks
• James Meiss (2007). Differential Dynamical Systems. SIAM. ISBN 0898716357.
• Steven H. Strogatz (1994). Nonlinear dynamics and chaos: with applications to physics, biology chemistry and
engineering. Addison Wesley. ISBN 0-201-54344-3.
• Kathleen T. Alligood, Tim D. Sauer and James A. Yorke (2000). Chaos. An introduction to dynamical systems.
Springer Verlag. ISBN 0-387-94677-2.
• Morris W. Hirsch, Stephen Smale and Robert Devaney (2003). Differential Equations, dynamical systems, and an
introduction to chaos. Academic Press. ISBN 0-12-349703-5.
• Julien Clinton Sprott (2003). Chaos and time-series analysis. Oxford University Press. ISBN 0-19-850839-5.
• Oded Galor (2011). Discrete Dynamical Systems. Springer. ISBN 978-3-642-07185-0.
Popularizations:
• Florin Diacu and Philip Holmes (1996). Celestial Encounters. Princeton. ISBN 0-691-02743-9.
• James Gleick (1988). Chaos: Making a New Science. Penguin. ISBN 0-14-009250-1.
• Ivar Ekeland (1990). Mathematics and the Unexpected (Paperback). University Of Chicago Press.
ISBN 0-226-19990-8.
• Ian Stewart (1997). Does God Play Dice? The New Mathematics of Chaos. Penguin. ISBN 0140256024.
External links
• A collection of dynamic and non-linear system models and demo applets (http://vlab.infotech.monash.edu.au/
simulations/non-linear/) (in Monash University's Virtual Lab)
• Arxiv preprint server (http://www.arxiv.org/list/math.DS/recent) has daily submissions of (non-refereed)
manuscripts in dynamical systems.
• DSWeb (http://www.dynamicalsystems.org/) provides up-to-date information on dynamical systems and its
applications.
• Encyclopedia of dynamical systems (http://www.scholarpedia.org/article/
Encyclopedia_of_Dynamical_Systems) A part of Scholarpedia — peer reviewed and written by invited experts.
• Nonlinear Dynamics (http://www.egwald.ca/nonlineardynamics/index.php). Models of bifurcation and chaos
by Elmer G. Wiens
• Oliver Knill (http://www.dynamical-systems.org) has a series of examples of dynamical systems with
explanations and interactive controls.
• Sci.Nonlinear FAQ 2.0 (Sept 2003) (http://amath.colorado.edu/faculty/jdm/faq-Contents.html) provides
definitions, explanations and resources related to nonlinear science
Online books or lecture notes:
• Geometrical theory of dynamical systems (http://arxiv.org/pdf/math.HO/0111177). Nils Berglund's lecture
notes for a course at ETH at the advanced undergraduate level.
• Dynamical systems (http://www.ams.org/online_bks/coll9/). George D. Birkhoff's 1927 book already takes a
modern approach to dynamical systems.
• Chaos: classical and quantum (http://chaosbook.org). An introduction to dynamical systems from the periodic
orbit point of view.
70
Dynamical system
• Modeling Dynamic Systems (http://www.embedded.com/2000/0008/0008feat2.htm). An introduction to the
development of mathematical models of dynamic systems.
• Learning Dynamical Systems (http://www.cs.brown.edu/research/ai/dynamics/tutorial/home.html).
Tutorial on learning dynamical systems.
• Ordinary Differential Equations and Dynamical Systems (http://www.mat.univie.ac.at/~gerald/ftp/book-ode/
). Lecture notes by Gerald Teschl
Research groups:
• Dynamical Systems Group Groningen (http://www.math.rug.nl/~broer/), IWI, University of Groningen.
• Chaos @ UMD (http://www-chaos.umd.edu/). Concentrates on the applications of dynamical systems.
• Dynamical Systems (http://www.math.sunysb.edu/dynamics/), SUNY Stony Brook. Lists of conferences,
researchers, and some open problems.
• Center for Dynamics and Geometry (http://www.math.psu.edu/dynsys/), Penn State.
• Control and Dynamical Systems (http://www.cds.caltech.edu/), Caltech.
• Laboratory of Nonlinear Systems (http://lanoswww.epfl.ch/), Ecole Polytechnique Fédérale de Lausanne
(EPFL).
• Center for Dynamical Systems (http://www.math.uni-bremen.de/ids.html/), University of Bremen
• Systems Analysis, Modelling and Prediction Group (http://www.eng.ox.ac.uk/samp/), University of Oxford
• Non-Linear Dynamics Group (http://sd.ist.utl.pt/), Instituto Superior Técnico, Technical University of Lisbon
• Dynamical Systems (http://www.impa.br/), IMPA, Instituto Nacional de Matemática Pura e Applicada.
• Nonlinear Dynamics Workgroup (http://ndw.cs.cas.cz/), Institute of Computer Science, Czech Academy of
Sciences.
Simulation software based on Dynamical Systems approach:
• FyDiK (http://fydik.kitnarf.cz/)
• iDMC (http://idmc.googlecode.com), simulation and dynamical analysis of nonlinear models
71
Graph dynamical system
72
Graph dynamical system
In mathematics, the concept of graph dynamical systems can be used to capture a wide range of processes taking
place on graphs or networks. A major theme in the mathematical and computational analysis of GDSs is to relate
their structural properties (e.g. the network connectivity) and the global dynamics that result.
The work on GDSs considers finite graphs and finite state spaces. As such, the research typically involves techniques
from, e.g., graph theory, combinatorics, algebra, and dynamical systems rather than differential geometry. In
principle, one could define and study GDSs over an infinite graph (e.g. cellular automata over
or interacting
particle systems), as well as GDSs with infinite state space (e.g.
as in coupled map lattices); see, e.g., Wu.[1] In
the following everything is implicitly assumed to be finite unless stated otherwise.
Formal definition
A graph dynamical system is constructed from the following components:
• A finite graph Y with vertex set v[Y] = {1,2, ... , n}. Depending on the context the graph can be directed or
undirected.
• A state xv for each vertex v of Y taken from a finite set K. The system state is the n-tuple x = (x1, x2, ... , xn), and
x[v] is the tuple consisting of the states associated to the vertices in the 1-neighborhood of v in Y (in some fixed
order).
• A vertex function fv for each vertex v. The vertex function maps the state of vertex v at time t to the vertex state
at time t + 1 based on the states associated to the 1-neighborhood of v in Y.
• An update scheme specifying the mechanism by which the mapping of individual vertex states is carried out so
as to induce a discrete dynamical system with map F: Kn → Kn.
The phase space associated to a dynamical system with map F: Kn → Kn is the finite directed graph with vertex set
Kn and directed edges (x, F(x)). The structure of the phase space is governed by the properties of the graph Y, the
vertex functions (fi)i, and the update scheme. The research in this area seeks to infer phase space properties based on
the structure of the system constituents. The analysis has a local-to-global character.
Generalized cellular automata (GCA)
If, for example, the update scheme consists of applying the vertex functions synchronously one obtains the class of
generalized cellular automata (CA). In this case, the global map F: Kn → Kn is given by
This class is referred to as generalized cellular automata since the classical or standard cellular automata are typically
defined and studied over regular graphs or grids, and the vertex functions are typically assumed to be identical.
Example: Let Y be the circle graph on vertices {1,2,3,4} with edges {1,2}, {2,3}, {3,4} and {1,4}, denoted Circ4.
Let K = {0,1} be the state space for each vertex and use the function nor3 : K3 → K defined by
nor3(x,y,z) = (1 + x)(1 + y)(1 + z) with arithmetic modulo 2 for all vertex functions. Then for example the system
state (0,1,0,0) is mapped to (0, 0, 0, 1) using a synchronous update. All the transitions are shown in the phase space
below.
Graph dynamical system
Sequential dynamical systems (SDS)
If the vertex functions are applied asynchronously in the sequence specified by a word w = (w1, w2, ... , wm) or
permutation = (
,
) of v[Y] one obtains the class of Sequential dynamical systems (SDS).[2] In this
case it is convenient to introduce the Y-local maps Fi constructed from the vertex functions by
The SDS map F = [FY , w] : Kn → Kn is the function composition
If the update sequence is a permutation one frequently speaks of a permutation SDS to emphasize this point.
Example: Let Y be the circle graph on vertices {1,2,3,4} with edges {1,2}, {2,3}, {3,4} and {1,4}, denoted Circ4.
Let K={0,1} be the state space for each vertex and use the function nor3 : K3 → K defined by nor3(x, y, z) =
(1 + x)(1 + y)(1 + z) with arithmetic modulo 2 for all vertex functions. Using the update sequence (1,2,3,4) then the
system state (0, 1, 0, 0) is mapped to (0, 0, 1, 0). All the system state transitions for this sequential dynamical system
are shown in the phase space below.
Stochastic graph dynamical systems
From, e.g., the point of view of applications it is interesting to consider the case where one or more of the
components of a GDS contains stochastic elements. Motivating applications could include processes that are not
fully understood (e.g. dynamics within a cell) and where certain aspects for all practical purposes seem to behave
according to some probability distribution. There are also applications governed by deterministic principles whose
description is so complex or unwieldy that it makes sense to consider probabilistic approximations.
Every element of a graph dynamical system can be made stochastic in several ways. For example, in a sequential
dynamical system the update sequence can be made stochastic. At each iteration step one may choose the update
sequence w at random from a given distribution of update sequences with corresponding probabilities. The matching
probability space of update sequences induces a probability space of SDS maps. A natural object to study in this
regard is the Markov chain on state space induced by this collection of SDS maps. This case is referred to as update
73
Graph dynamical system
sequence stochastic GDS and is motivated by, e.g., processes where "events" occur at random according to certain
rates (e.g. chemical reactions), synchronization in parallel computation/discrete event simulations, and in
computational paradigms described later.
This specific example with stochastic update sequence illustrates two general facts for such systems: when passing to
a stochastic graph dynamical system one is generally led to (1) a study of Markov chains (with specific structure
governed by the constituents of the GDS), and (2) the resulting Markov chains tend to be large having an exponential
number of states. A central goal in the study of stochastic GDS is to be able to derive reduced models.
One may also consider the case where the vertex functions are stochastic, i.e., function stochastic GDS. For example,
Random Boolean networks are examples of function stochastic GDS using a synchronous update scheme and where
the state space is K = {0, 1}. Finite probabilistic cellular automata (PCA) is another example of function stochastic
GDS. In principle the class of Interacting particle systems (IPS) covers finite and infinite PCA, but in practice the
work on IPS is largely concerned with the infinite case since this allows one to introduce more interesting topologies
on state space.
Applications
Graph dynamical systems constitute a natural framework for capturing distributed systems such as biological
networks and epidemics over social networks, many of which are frequently referred to as complex systems.
References
[1] Wu, Chai Wah (2005). "Synchronization in networks of nonlinear dynamical systems coupled via a directed graph". Nonlinearity 18 (3):
1057–1064. doi:10.1088/0951-7715/18/3/007.
[2] Mortveit, Henning S.; Reidys, Christian M. (2007). An introduction to sequential dynamical systems. Universitext. New York: Springer
Verlag. ISBN 978-0-387-30654-4.
External links
• Graph Dynamical Systems – A Mathematical Framework for Interaction-Based Systems, Their Analysis and
Simulations by Henning Mortveit (http://legacy.samsi.info/200809/algebraic/presentations/discrete/friday/
samsi-05-dec-08.pdf)
Further reading
• Macauley, Matthew; Mortveit, Henning S. (2009). "Cycle equivalence of graph dynamical systems". Nonlinearity
22 (2): 421–436. doi:10.1088/0951-7715/22/2/010.
• Golubitsky, Martin; Stewart, Ian (2003). The Symmetry Perspective. Basel: Birkhauser. ISBN 0817621717.
74
75
Dynamic response
First-order logic
First-order logic is a formal system used in mathematics, philosophy, linguistics, and computer science.
Over the past 100-odd years, what is now called first-order logic has gone by many names, including: first-order
predicate calculus, the lower predicate calculus, quantification theory, and predicate logic (a less precise term).
First-order logic is distinguished from propositional logic by its use of quantified variables. First-order logic with a
specified domain of discourse over which the quantified variables range, one or more interpreted predicate letters,
and proper axioms involving the interpreted predicate letters, is a first-order theory.
The adjective "first-order" distinguishes first-order logic from higher-order logic in which there are predicates having
predicates or functions as arguments, or in which one or both of predicate quantifiers or function quantifiers are
permitted.[1] In first-order theories, predicates are often associated with sets. In interpreted higher-order theories,
predicates may be interpreted as sets of sets.
There are many deductive systems for first-order logic that are sound (all provable statements are true) and complete
(all true statements are provable). Although the logical consequence relation is only semidecidable, much progress
has been made in automated theorem proving in first-order logic. First-order logic also satisfies several metalogical
theorems that make it amenable to analysis in proof theory, such as the Löwenheim–Skolem theorem and the
compactness theorem.
First-order logic is of great importance to the foundations of mathematics, because it is the standard formal logic for
axiomatic systems. Many common axiomatic systems, such as first-order Peano arithmetic and axiomatic set theory,
including the canonical Zermelo–Fraenkel set theory (ZF), can be formalized as first-order theories. No first-order
theory, however, has the strength to describe fully and categorically structures with an infinite domain, such as the
natural numbers or the real line. Categorical axiom systems for these structures can be obtained in stronger logics
such as second-order logic.
For a history of first-order logic and how it came to be the dominant formal logic, see Ferreirós (2001).
Introduction
While propositional logic deals with simple declarative propositions, first-order logic additionally covers predicates
and quantification.
A predicate resembles a function that returns either True or False. Consider the following sentences: "Socrates is a
philosopher", "Plato is a philosopher". In propositional logic these are treated as two unrelated propositions, denoted
for example by p and q. In first-order logic, however, the sentences can be expressed in a more parallel manner using
the predicate Phil(a), which asserts that the object represented by a is a philosopher. Thus if a represents Socrates
then Phil(a) asserts the first proposition, p; if a instead represents Plato then Phil(a) asserts the second proposition, q.
A key aspect of first-order logic is visible here: the string "Phil" is a syntactic entity which is given semantic
meaning by declaring that Phil(a) holds exactly when a is a philosopher. An assignment of semantic meaning is
called an interpretation.
First-order logic allows reasoning about properties that are shared by many objects, through the use of variables. For
example, let Phil(a) assert that a is a philosopher and let Schol(a) assert that a is a scholar. Then the formula
First-order logic
76
asserts that if a is a philosopher then a is a scholar. The symbol
is used to denote a conditional (if/then)
statement. The hypothesis lies to the left of the arrow and the conclusion to the right. The truth of this formula
depends on which object is denoted by a, and on the interpretations of "Phil" and "Schol".
Assertions of the form "for every a, if a is a philosopher then a is a scholar" require both the use of variables and the
use of a quantifier. Again, let Phil(a) assert a is a philosopher and let Schol(a) assert that a is a scholar. Then the
first-order sentence
asserts that no matter what a represents, if a is a philosopher then a is scholar. Here
, the universal quantifier,
expresses the idea that the claim in parentheses holds for all choices of a.
To show that the claim "If a is a philosopher then a is a scholar" is false, one would show there is some philosopher
who is not a scholar. This counterclaim can be expressed with the existential quantifier :
Here:
•
is the negation operator:
•
not a scholar.
is the conjunction operator:
is true if and only if
is false, in other words if and only if a is
asserts that a is a philosopher and also not a scholar.
The predicates Phil(a) and Schol(a) take only one parameter each. First-order logic can also express predicates with
more than one parameter. For example, "there is someone who can be fooled every time" can be expressed as:
Here Person(x) is interpreted to mean x is a person, Time(y) to mean that y is a moment of time, and Canfool(x,y) to
mean that (person) x can be fooled at (time) y. For clarity, this statement asserts that there is at least one person who
can be fooled at all times, which is stronger than asserting that at all times at least one person exists who can be
fooled. Asserting the latter (that there is always at least one foolable person) does not signify whether this foolable
person is always the same for all moments of time.
The range of the quantifiers is the set of objects that can be used to satisfy them. (In the informal examples in this
section, the range of the quantifiers was left unspecified.) In addition to specifying the meaning of predicate symbols
such as Person and Time, an interpretation must specify a nonempty set, known as the domain of discourse or
universe, as a range for the quantifiers. Thus a statement of the form
is said to be true, under a
particular interpretation, if there is some object in the domain of discourse of that interpretation that satisfies the
predicate that the interpretation uses to assign meaning to the symbol Phil.
Syntax
There are two key parts of first order logic. The syntax determines which collections of symbols are legal
expressions in first-order logic, while the semantics determine the meanings behind these expressions.
Alphabet
Unlike natural languages, such as English, the language of first-order logic is completely formal, so that it can be
mechanically determined whether a given expression is legal. There are two key types of legal expressions: terms,
which intuitively represent objects, and formulas, which intuitively express predicates that can be true or false. The
terms and formulas of first-order logic are strings of symbols which together form the alphabet of the language. As
with all formal languages, the nature of the symbols themselves is outside the scope of formal logic; they are often
regarded simply as letters and punctuation symbols.
It is common to divide the symbols of the alphabet into logical symbols, which always have the same meaning, and
non-logical symbols, whose meaning varies by interpretation. For example, the logical symbol always represents
First-order logic
77
"and"; it is never interpreted as "or". On the other hand, a non-logical predicate symbol such as Phil(x) could be
interpreted to mean "x is a philosopher", "x is a man named Philip", or any other unary predicate, depending on the
interpretation at hand.
Logical symbols
There are several logical symbols in the alphabet, which vary by author but usually include:
• The quantifier symbols and
• The logical connectives: for conjunction, for disjunction,
for implication,
for biconditional, for
negation. Occasionally other logical connective symbols are included. Some authors use
, or Cpq, instead of
, and
, or Epq, instead of
, especially in contexts where
is used for other purposes. Moreover, the
horseshoe
may replace
; the triple-bar may replace
, and a tilde (~), Np, or Fpq, may replace ;
||, or Apq may replace ; and &, or Kpq, may replace , especially if these symbols are not available for
technical reasons.
• Parentheses, brackets, and other punctuation symbols. The choice of such symbols varies depending on context.
• An infinite set of variables, often denoted by lowercase letters at the end of the alphabet x, y, z, … . Subscripts
are often used to distinguish variables: x0, x1, x2, … .
• An equality symbol (sometimes, identity symbol) =; see the section on equality below.
It should be noted that not all of these symbols are required - only one of the quantifiers, negation and conjunction,
variables, brackets and equality suffice. There are numerous minor variations that may define additional logical
symbols:
• Sometimes the truth constants T, Vpq, or
, for "true" and F, Opq, or
, for "false" are included. Without
any such logical operators of valence 0, these two constants can only be expressed using quantifiers.
• Sometimes additional logical connectives are included, such as the Sheffer stroke, Dpq (NAND), and exclusive
or, Jpq.
Non-logical symbols
The non-logical symbols represent predicates (relations), functions and constants on the domain of discourse. It used
to be standard practice to use a fixed, infinite set of non-logical symbols for all purposes. A more recent practice is to
use different non-logical symbols according to the application one has in mind. Therefore it has become necessary to
name the set of all non-logical symbols used in a particular application. This choice is made via a signature.[2]
The traditional approach is to have only one, infinite, set of non-logical symbols (one signature) for all applications.
Consequently, under the traditional approach there is only one language of first-order logic.[3] This approach is still
common, especially in philosophically oriented books.
1. For every integer n ≥ 0 there is a collection of n-ary, or n-place, predicate symbols. Because they represent
relations between n elements, they are also called relation symbols. For each arity n we have an infinite supply of
them:
Pn0, Pn1, Pn2, Pn3, …
2. For every integer n ≥ 0 there are infinitely many n-ary function symbols:
f n0, f n1, f n2, f n3, …
In contemporary mathematical logic, the signature varies by application. Typical signatures in mathematics are {1,
×} or just {×} for groups, or {0, 1, +, ×, <} for ordered fields. There are no restrictions on the number of non-logical
symbols. The signature can be empty, finite, or infinite, even uncountable. Uncountable signatures occur for example
in modern proofs of the Löwenheim-Skolem theorem.
In this approach, every non-logical symbol is of one of the following types.
First-order logic
78
1. A predicate symbol (or relation symbol) with some valence (or arity, number of arguments) greater than or
equal to 0. These which are often denoted by uppercase letters P, Q, R,... .
• Relations of valence 0 can be identified with propositional variables. For example, P, which can stand for any
statement.
• For example, P(x) is a predicate variable of valence 1. One possible interpretation is "x is a man".
• Q(x,y) is a predicate variable of valence 2. Possible interpretations include "x is greater than y" and "x is the
father of y".
2. A function symbol, with some valence greater than or equal to 0. These are often denoted by lowercase letters f,
g, h,... .
• Examples: f(x) may be interpreted as for "the father of x". In arithmetic, it may stand for "-x". In set theory, it
may stand for "the power set of x". In arithmetic, g(x,y) may stand for "x+y". In set theory, it may stand for
"the union of x and y".
• Function symbols of valence 0 are called constant symbols, and are often denoted by lowercase letters at the
beginning of the alphabet a, b, c,... . The symbol a may stand for Socrates. In arithmetic, it may stand for 0. In
set theory, such a constant may stand for the empty set.
The traditional approach can be recovered in the modern approach by simply specifying the "custom" signature to
consist of the traditional sequences of non-logical symbols.
Formation rules
The formation rules define the terms and formulas of first order logic. When terms and formulas are represented as
strings of symbols, these rules can be used to write a formal grammar for terms and formulas. These rules are
generally context-free (each production has a single symbol on the left side), except that the set of symbols may be
allowed to be infinite and there may be many start symbols, for example the variables in the case of terms.
Terms
The set of terms is inductively defined by the following rules:
1. Variables. Any variable is a term.
2. Functions. Any expression f(t1,...,tn) of n arguments (where each argument ti is a term and f is a function symbol
of valence n) is a term. In particular, symbols denoting individual constants are 0-ary function symbols, and are
thus terms.
Only expressions which can be obtained by finitely many applications of rules 1 and 2 are terms. For example, no
expression involving a predicate symbol is a term.
Formulas
The set of formulas (also called well-formed formulas[4] or wffs) is inductively defined by the following rules:
1.
2.
3.
4.
Predicate symbols. If P is an n-ary predicate symbol and t1, ..., tn are terms then P(t1,...,tn) is a formula.
Equality. If the equality symbol is considered part of logic, and t1 and t2 are terms, then t1 = t2 is a formula.
Negation. If φ is a formula, then φ is a formula.
Binary connectives. If φ and ψ are formulas, then (φ
ψ) is a formula. Similar rules apply to other binary
logical connectives.
5. Quantifiers. If φ is a formula and x is a variable, then
and
are formulas.
Only expressions which can be obtained by finitely many applications of rules 1–5 are formulas. The formulas
obtained from the first two rules are said to be atomic formulas.
For example,
First-order logic
79
is a formula, if f is a unary function symbol, P a unary predicate symbol, and Q a ternary predicate symbol. On the
other hand,
is not a formula, although it is a string of symbols from the alphabet.
The role of the parentheses in the definition is to ensure that any formula can only be obtained in one way by
following the inductive definition (in other words, there is a unique parse tree for each formula). This property is
known as unique readability of formulas. There are many conventions for where parentheses are used in formulas.
For example, some authors use colons or full stops instead of parentheses, or change the places in which parentheses
are inserted. Each author's particular definition must be accompanied by a proof of unique readability.
This definition of a formula does not support defining an if-then-else function ite(c,a,b) where "c" is a condition
expressed as a formula, that would return "a" if c is true, and "b" if it is false. This is because both predicates and
functions can only accept terms as parameters, but the first parameter is a formula. Some languages built on
first-order logic, such as SMT-LIB 2.0, add this. [5]
Notational conventions
For convenience, conventions have been developed about the precedence of the logical operators, to avoid the need
to write parentheses in some cases. These rules are similar to the order of operations in arithmetic. A common
convention is:
•
is evaluated first
•
and are evaluated next
• Quantifiers are evaluated next
•
is evaluated last.
Moreover, extra punctuation not required by the definition may be inserted to make formulas easier to read. Thus the
formula
might be written as
In some fields, it is common to use infix notation for binary relations and functions, instead of the prefix notation
defined above. For example, in arithmetic, one typically writes "2 + 2 = 4" instead of "=(+(2,2),4)". It is common to
regard formulas in infix notation as abbreviations for the corresponding formulas in prefix notation.
The definitions above use infix notation for binary connectives such as
. A less common convention is Polish
notation, in which one writes
,
, and so on in front of their arguments rather than between them. This
convention allows all punctuation symbols to be discarded. Polish notation is compact and elegant, but rarely used in
practice because it is hard for humans to read it. In Polish notation, the formula
becomes "∀x∀y→Pfx¬→ PxQfyxz".
Example
In mathematics the language of ordered abelian groups has one constant symbol 0, one unary function symbol −, one
binary function symbol +, and one binary relation symbol ≤. Then:
• The expressions +(x, y) and +(x, +(y, −(z))) are terms. These are usually written as x + y and x + y − z.
• The expressions +(x, y) = 0 and ≤(+(x, +(y, −(z))), +(x, y)) are atomic formulas.
These are usually written as x + y = 0 and x + y − z ≤ x + y.
• The expression
is a formula, which is usually written as
First-order logic
80
Free and bound variables
In a formula, a variable may occur free or bound. Intuitively, a variable is free in a formula if it is not quantified: in
, variable x is free while y is bound. The free and bound variables of a formula are defined inductively
as follows.
1. Atomic formulas. If φ is an atomic formula then x is free in φ if and only if x occurs in φ. Moreover, there are
no bound variables in any atomic formula.
2. Negation. x is free in φ if and only if x is free in φ. x is bound in φ if and only if x is bound in φ.
3. Binary connectives. x is free in (φ
ψ) if and only if x is free in either φ or ψ. x is bound in (φ
ψ) if and
only if x is bound in either φ or ψ. The same rule applies to any other binary connective in place of
.
4. Quantifiers. x is free in y φ if and only if x is free in φ and x is a different symbol from y. Also, x is bound in
y φ if and only if x is y or x is bound in φ. The same rule holds with
For example, in
x
y (P(x)
in place of
.
Q(x,f(x),z)), x and y are bound variables, z is a free variable, and w is neither
because it does not occur in the formula.
Freeness and boundness can be also specialized to specific occurrences of variables in a formula. For example, in
, the first occurrence of x is free while the second is bound. In other words, the x in
is
free while the
in
is bound.
A formula in first-order logic with no free variables is called a first-order sentence. These are the formulas that will
have well-defined truth values under an interpretation. For example, whether a formula such as Phil(x) is true must
depend on what x represents. But the sentence
will be either true or false in a given interpretation.
Semantics
An interpretation of a first-order language assigns a denotation to all non-logical constants in that language. It also
determines a domain of discourse that specifies the range of the quantifiers. The result is that each term is assigned
an object that it represents, and each sentence is assigned a truth value. In this way, an interpretation provides
semantic meaning to the terms and formulas of the language. The study of the interpretations of formal languages is
called formal semantics.
The domain of discourse D is a nonempty set of "objects" of some kind. Intuitively, a first-order formula is a
statement about these objects; for example,
states the existence of an object x such that the predicate P is
true where referred to it. The domain of discourse is the set of considered objects. For example, one can take
to
be the set of integer numbers.
The interpretation of a function symbol is a function. For example, if the domain of discourse consists of integers, a
function symbol f of arity 2 can be interpreted as the function that gives the sum of its arguments. In other words, the
symbol f is associated with the function I(f) which, in this interpretation, is addition.
The interpretation of a constant symbol is a function from the one-element set D0 to D, which can be simply
identified with an object in D. For example, an interpretation may assign the value
to the constant
symbol
.
The interpretation of an n-ary predicate symbol is a set of n-tuples of elements of the domain of discourse. This
means that, given an interpretation, a predicate symbol, and n elements of the domain of discourse, one can tell
whether the predicate is true of those elements according to the given interpretation. For example, an interpretation
I(P) of a binary predicate symbol P may be the set of pairs of integers such that the first one is less than the second.
According to this interpretation, the predicate P would be true if its first argument is less than the second.
First-order logic
81
First-order structures
The most common way of specifying an interpretation (especially in mathematics) is to specify a structure (also
called a model; see below). The structure consists of a nonempty set D that forms the domain of discourse and an
interpretation I of the non-logical terms of the signature. This interpretation is itself a function:
• Each function symbol f of arity n is assigned a function I(f) from
to
. In particular, each constant symbol
of the signature is assigned an individual in the domain of discourse.
• Each predicate symbol P of arity n is assigned a relation I(P) over
or, equivalently, a function from
to
. Thus each predicate symbol is interpreted by a Boolean-valued function on D.
Evaluation of truth values
A formula evaluates to true or false given an interpretation, and a variable assignment μ that associates an element
of the domain of discourse with each variable. The reason that a variable assignment is required is to give meanings
to formulas with free variables, such as
. The truth value of this formula changes depending on whether x
and y denote the same individual.
First, the variable assignment μ can be extended to all terms of the language, with the result that each term maps to a
single element of the domain of discourse. The following rules are used to make this assignment:
1. Variables. Each variable x evaluates to μ(x)
2. Functions. Given terms
that have been evaluated to elements
discourse, and a n-ary function symbol f, the term
of the domain of
evaluates to
.
Next, each formula is assigned a truth value. The inductive definition used to make this assignment is called the
T-schema.
1. Atomic formulas (1). A formula
is associated the value true or false depending on whether
, where
are the evaluation of the terms
interpretation of , which by assumption is a subset of
.
2. Atomic formulas (2). A formula
is assigned true if and
of discourse (see the section on equality below).
3. Logical connectives. A formula in the form
,
is the
evaluate to the same object of the domain
, etc. is evaluated according to the truth table for the
connective in question, as in propositional logic.
4. Existential quantifiers. A formula
is true according to M and
variables that only differs from
and
if there exists an evaluation
of the
regarding the evaluation of x and such that φ is true according to the
interpretation M and the variable assignment
. This formal definition captures the idea that
is true if
and only if there is a way to choose a value for x such that φ(x) is satisfied.
5. Universal quantifiers. A formula
is true according to M and if φ(x) is true for every pair composed
by the interpretation M and some variable assignment
that differs from
only on the value of x. This
captures the idea that
is true if every possible choice of a value for x causes φ(x) to be true.
If a formula does not contain free variables, and so is a sentence, then the initial variable assignment does not affect
its truth value. In other words, a sentence is true according to M and if and only if is true according to M and
every other variable assignment
.
There is a second common approach to defining truth values that does not rely on variable assignment functions.
Instead, given an interpretation M, one first adds to the signature a collection of constant symbols, one for each
element of the domain of discourse in M; say that for each d in the domain the constant symbol cd is fixed. The
interpretation is extended so that each new constant symbol is assigned to its corresponding element of the domain.
One now defines truth for quantified formulas syntactically, as follows:
1. Existential quantifiers (alternate). A formula
of discourse such that
holds. Here
is true according to M if there is some d in the domain
is the result of substituting cd for every free occurrence of x in
First-order logic
82
φ.
2. Universal quantifiers (alternate). A formula
is true according to M if, for every d in the domain of
discourse,
is true according to M.
This alternate approach gives exactly the same truth values to all sentences as the approach via variable assignments.
Validity, satisfiability, and logical consequence
If a sentence φ evaluates to True under a given interpretation M, one says that M satisfies φ; this is denoted
. A sentence is satisfiable if there is some interpretation under which it is true.
Satisfiability of formulas with free variables is more complicated, because an interpretation on its own does not
determine the truth value of such a formula. The most common convention is that a formula with free variables is
said to be satisfied by an interpretation if the formula remains true regardless which individuals from the domain of
discourse are assigned to its free variables. This has the same effect as saying that a formula is satisfied if and only if
its universal closure is satisfied.
A formula is logically valid (or simply valid) if it is true in every interpretation. These formulas play a role similar
to tautologies in propositional logic.
A formula φ is a logical consequence of a formula ψ if every interpretation that makes ψ true also makes φ true. In
this case one says that φ is logically implied by ψ.
Algebraizations
An alternate approach to the semantics of first-order logic proceeds via abstract algebra. This approach generalizes
the Lindenbaum–Tarski algebras of propositional logic. There are three ways of eliminating quantified variables
from first-order logic, that do not involve replacing quantifiers with other variable binding term operators:
• Cylindric algebra, by Alfred Tarski and his coworkers;
• Polyadic algebra, by Paul Halmos;
• Predicate functor logic, mainly due to Willard Quine.
These algebras are all lattices that properly extend the two-element Boolean algebra.
Tarski and Givant (1987) showed that the fragment of first-order logic that has no atomic sentence lying in the scope
of more than three quantifiers, has the same expressive power as relation algebra. This fragment is of great interest
because it suffices for Peano arithmetic and most axiomatic set theory, including the canonical ZFC. They also prove
that first-order logic with a primitive ordered pair is equivalent to a relation algebra with two ordered pair projection
functions.
First-order theories, models, and elementary classes
Further information: List of first-order theories
A first-order theory consists of a set of axioms in a particular first-order signature. The set of axioms is often finite
or recursively enumerable, in which case the theory is called effective. Some authors require theories to also include
all logical consequences of the axioms.
A first-order structure that satisfies all sentences in a given theory is said to be a model of the theory. An
elementary class is the set of all structures satisfying a particular theory. These classes are a main subject of study in
model theory.
Many theories have an intended interpretation, a certain model that is kept in mind when studying the theory. For
example, the intended interpretation of Peano arithmetic consists of the usual natural numbers with their usual
operations. However, the Löwenheim–Skolem theorem shows that most first-order theories will also have other,
nonstandard models.
First-order logic
A theory is consistent if it is not possible to prove a contradiction from the axioms of the theory. A theory is
complete if, for every formula in its signature, either that formula or its negation is a logical consequence of the
axioms of the theory. Gödel's incompleteness theorem shows that effective first-order theories that include a
sufficient portion of the theory of the natural numbers can never be both consistent and complete.
Empty domains
The definition above requires that the domain of discourse of any interpretation must be a nonempty set. There are
settings, such as inclusive logic, where empty domains are permitted. Moreover, if a class of algebraic structures
includes an empty structure (for example, there is an empty poset), that class can only be an elementary class in
first-order logic if empty domains are permitted or the empty structure is removed from the class.
There are several difficulties with empty domains, however:
• Many common rules of inference are only valid when the domain of discourse is required to be nonempty. One
example is the rule stating that
implies
when x is not a free variable in φ. This rule,
which is used to put formulas into prenex normal form, is sound in nonempty domains, but unsound if the empty
domain is permitted.
• The definition of truth in an interpretation that uses a variable assignment function cannot work with empty
domains, because there are no variable assignment functions whose range is empty. (Similarly, one cannot assign
interpretations to constant symbols.) This truth definition requires that one must select a variable assignment
function (μ above) before truth values for even atomic formulas can be defined. Then the truth value of a sentence
is defined to be its truth value under any variable assignment, and it is proved that this truth value does not depend
on which assignment is chosen. This technique does not work if there are no assignment functions at all; it must
be changed to accommodate empty domains.
Thus, when the empty domain is permitted, it must often be treated as a special case. Most authors, however, simply
exclude the empty domain by definition.
Deductive systems
A deductive system is used to demonstrate, on a purely syntactic basis, that one formula is a logical consequence of
another formula. There are many such systems for first-order logic, including Hilbert-style deductive systems,
natural deduction, the sequent calculus, the tableaux method, and resolution. These share the common property that a
deduction is a finite syntactic object; the format of this object, and the way it is constructed, vary widely. These finite
deductions themselves are often called derivations in proof theory. They are also often called proofs, but are
completely formalized unlike natural-language mathematical proofs.
A deductive system is sound if any formula that can be derived in the system is logically valid. Conversely, a
deductive system is complete if every logically valid formula is derivable. All of the systems discussed in this article
are both sound and complete. They also share the property that it is possible to effectively verify that a purportedly
valid deduction is actually a deduction; such deduction systems are called effective.
A key property of deductive systems is that they are purely syntactic, so that derivations can be verified without
considering any interpretation. Thus a sound argument is correct in every possible interpretation of the language,
regardless whether that interpretation is about mathematics, economics, or some other area.
In general, logical consequence in first-order logic is only semidecidable: if a sentence A logically implies a sentence
B then this can be discovered (for example, by searching for a proof until one is found, using some effective, sound,
complete proof system). However, if A does not logically imply B, this does not mean that A logically implies the
negation of B. There is no effective procedure that, given formulas A and B, always correctly decides whether A
logically implies B.
83
First-order logic
84
Rules of inference
Further information: List of rules of inference
A rule of inference states that, given a particular formula (or set of formulas) with a certain property as a
hypothesis, another specific formula (or set of formulas) can be derived as a conclusion. The rule is sound (or
truth-preserving) if it preserves validity in the sense that whenever any interpretation satisfies the hypothesis, that
interpretation also satisfies the conclusion.
For example, one common rule of inference is the rule of substitution. If t is a term and φ is a formula possibly
containing the variable x, then φ[t/x] (often denoted φ[x/t]) is the result of replacing all free instances of x by t in φ.
The substitution rule states that for any φ and any term t, one can conclude φ[t/x] from φ provided that no free
variable of t becomes bound during the substitution process. (If some free variable of t becomes bound, then to
substitute t for x it is first necessary to change the bound variables of φ to differ from the free variables of t.)
To see why the restriction on bound variables is necessary, consider the logically valid formula φ given by
, in the signature of (0,1,+,×,=) of arithmetic. If t is the term "x + 1", the formula φ[t/y] is
, which will be false in many interpretations. The problem is that the free variable x of t became
bound during the substitution. The intended replacement can be obtained by renaming the bound variable x of φ to
something else, say z, so that the formula after substitution is
, which is again logically valid.
The substitution rule demonstrates several common aspects of rules of inference. It is entirely syntactical; one can
tell whether it was correctly applied without appeal to any interpretation. It has (syntactically-defined) limitations on
when it can be applied, which must be respected to preserve the correctness of derivations. Moreover, as is often the
case, these limitations are necessary because of interactions between free and bound variables that occur during
syntactic manipulations of the formulas involved in the inference rule.
Hilbert-style systems and natural deduction
A deduction in a Hilbert-style deductive system is a list of formulas, each of which is a logical axiom, a hypothesis
that has been assumed for the derivation at hand, or follows from previous formulas via a rule of inference. The
logical axioms consist of several axiom schemes of logically valid formulas; these encompass a significant amount
of propositional logic. The rules of inference enable the manipulation of quantifiers. Typical Hilbert-style systems
have a small number of rules of inference, along with several infinite schemes of logical axioms. It is common to
have only modus ponens and universal generalization as rules of inference.
Natural deduction systems resemble Hilbert-style systems in that a deduction is a finite list of formulas. However,
natural deduction systems have no logical axioms; they compensate by adding additional rules of inference that can
be used to manipulate the logical connectives in formulas in the proof.
Sequent calculus
Further information: Sequent calculus
The sequent calculus was developed to study the properties of natural deduction systems. Instead of working with
one formula at a time, it uses sequents, which are expressions of the form
where A1, ..., An, B1, ..., Bk are formulas and the turnstile symbol
halves. Intuitively, a sequent expresses the idea that
is used as punctuation to separate the two
implies
.
First-order logic
85
Tableaux method
Further information: Method of analytic tableaux
Unlike the methods just described, the derivations in the tableaux method are
not lists of formulas. Instead, a derivation is a tree of formulas. To show that a
formula A is provable, the tableaux method attempts to demonstrate that the
negation of A is unsatisfiable. The tree of the derivation has
at its root;
the tree branches in a way that reflects the structure of the formula. For
example, to show that
is unsatisfiable requires showing that C and D
are each unsatisfiable; the corresponds to a branching point in the tree with
parent
and children C and D.
Resolution
The resolution rule is a single rule of inference that, together with unification,
is sound and complete for first-order logic. As with the tableaux method, a
formula is proved by showing that the negation of the formula is
unsatisfiable. Resolution is commonly used in automated theorem proving.
A tableau proof for the propositional
formula ((a ∨ ~b) & b) → a.
The resolution method works only with formulas that are disjunctions of atomic formulas; arbitrary formulas must
first be converted to this form through Skolemization. The resolution rule states that from the hypotheses
and
, the conclusion
can be
obtained.
Provable identities
The following sentences can be called "identities" because the main connective in each is the biconditional.
(where
must not occur free in
)
(where
must not occur free in
)
Equality and its axioms
There are several different conventions for using equality (or identity) in first-order logic. The most common
convention, known as first-order logic with equality, includes the equality symbol as a primitive logical symbol
which is always interpreted as the real equality relation between members of the domain of discourse, such that the
"two" given members are the same member. This approach also adds certain axioms about equality to the deductive
system employed. These equality axioms are:
1. Reflexivity. For each variable x, x = x.
First-order logic
86
2. Substitution for functions. For all variables x and y, and any function symbol f,
x = y → f(...,x,...) = f(...,y,...).
3. Substitution for formulas. For any variables x and y and any formula φ(x), if φ' is obtained by replacing any
number of free occurrences of x in φ with y, such that these remain free occurrences of y, then
x = y → (φ → φ').
These are axiom schemes, each of which specifies an infinite set of axioms. The third scheme is known as Leibniz's
law, "the principle of substitutivity", "the indiscernibility of identicals", or "the replacement property". The second
scheme, involving the function symbol f, is (equivalent to) a special case of the third scheme, using the formula
x = y → (f(...,x,...) = z → f(...,y,...) = z).
Many other properties of equality are consequences of the axioms above, for example:
1. Symmetry. If x = y then y = x.
2. Transitivity. If x = y and y = z then x = z.
First-order logic without equality
An alternate approach considers the equality relation to be a non-logical symbol. This convention is known as
first-order logic without equality. If an equality relation is included in the signature, the axioms of equality must
now be added to the theories under consideration, if desired, instead of being considered rules of logic. The main
difference between this method and first-order logic with equality is that an interpretation may now interpret two
distinct individuals as "equal" (although, by Leibniz's law, these will satisfy exactly the same formulas under any
interpretation). That is, the equality relation may now be interpreted by an arbitrary equivalence relation on the
domain of discourse that is congruent with respect to the functions and relations of the interpretation.
When this second convention is followed, the term normal model is used to refer to an interpretation where no
distinct individuals a and b satisfy a = b. In first-order logic with equality, only normal models are considered, and
so there is no term for a model other than a normal model. When first-order logic without equality is studied, it is
necessary to amend the statements of results such as the Löwenheim–Skolem theorem so that only normal models
are considered.
First-order logic without equality is often employed in the context of second-order arithmetic and other higher-order
theories of arithmetic, where the equality relation between sets of natural numbers is usually omitted.
Defining equality within a theory
If a theory has a binary formula A(x,y) which satisfies reflexivity and Leibniz's law, the theory is said to have
equality, or to be a theory with equality. The theory may not have all instances of the above schemes as axioms, but
rather as derivable theorems. For example, in theories with no function symbols and a finite number of relations, it is
possible to define equality in terms of the relations, by defining the two terms s and t to be equal if any relation is
unchanged by changing s to t in any argument.
Some theories allow other ad hoc definitions of equality:
• In the theory of partial orders with one relation symbol ≤, one could define s = t to be an abbreviation for s ≤ t
t ≤ s.
• In set theory with one relation
, one may define s = t to be an abbreviation for
x (s
x
t
x)
x (x
s
x
t). This definition of equality then automatically satisfies the axioms for equality. In this case,
one should replace the usual axiom of extensionality,
, by
, i.e. if x and y have the same elements, then they
belong to the same sets.
First-order logic
Metalogical properties
One motivation for the use of first-order logic, rather than higher-order logic, is that first-order logic has many
metalogical properties that stronger logics do not have. These results concern general properties of first-order logic
itself, rather than properties of individual theories. They provide fundamental tools for the construction of models of
first-order theories.
Completeness and undecidability
Gödel's completeness theorem, proved by Kurt Gödel in 1929, establishes that there are sound, complete, effective
deductive systems for first-order logic, and thus the first-order logical consequence relation is captured by finite
provability. Naively, the statement that a formula φ logically implies a formula ψ depends on every model of φ;
these models will in general be of arbitrarily large cardinality, and so logical consequence cannot be effectively
verified by checking every model. However, it is possible to enumerate all finite derivations and search for a
derivation of ψ from φ. If ψ is logically implied by φ, such a derivation will eventually be found. Thus first-order
logical consequence is semidecidable: it is possible to make an effective enumeration of all pairs of sentences (φ,ψ)
such that ψ is a logical consequence of φ.
Unlike propositional logic, first-order logic is undecidable (although semidecidable), provided that the language has
at least one predicate of arity at least 2 (other than equality). This means that there is no decision procedure that
determines whether arbitrary formulas are logically valid. This result was established independently by Alonzo
Church and Alan Turing in 1936 and 1937, respectively, giving a negative answer to the Entscheidungsproblem
posed by David Hilbert in 1928. Their proofs demonstrate a connection between the unsolvability of the decision
problem for first-order logic and the unsolvability of the halting problem.
There are systems weaker than full first-order logic for which the logical consequence relation is decidable. These
include propositional logic and monadic predicate logic, which is first-order logic restricted to unary predicate
symbols and no function symbols. The Bernays–Schönfinkel class of first-order formulas is also decidable.
The Löwenheim–Skolem theorem
The Löwenheim–Skolem theorem shows that if a first-order theory of cardinality λ has any infinite model then it has
models of every infinite cardinality greater than or equal to λ. One of the earliest results in model theory, it implies
that it is not possible to characterize countability or uncountability in a first-order language. That is, there is no
first-order formula φ(x) such that an arbitrary structure M satisfies φ if and only if the domain of discourse of M is
countable (or, in the second case, uncountable).
The Löwenheim–Skolem theorem implies that infinite structures cannot be categorically axiomatized in first-order
logic. For example, there is no first-order theory whose only model is the real line: any first-order theory with an
infinite model also has a model of cardinality larger than the continuum. Since the real line is infinite, any theory
satisfied by the real line is also satisfied by some nonstandard models. When the Löwenheim–Skolem theorem is
applied to first-order set theories, the nonintuitive consequences are known as Skolem's paradox.
The compactness theorem
The compactness theorem states that a set of first-order sentences has a model if and only if every finite subset of it
has a model. This implies that if a formula is a logical consequence of an infinite set of first-order axioms, then it is a
logical consequence of some finite number of those axioms. This theorem was proved first by Kurt Gödel as a
consequence of the completeness theorem, but many additional proofs have been obtained over time. It is a central
tool in model theory, providing a fundamental method for constructing models.
The compactness theorem has a limiting effect on which collections of first-order structures are elementary classes.
For example, the compactness theorem implies that any theory that has arbitrarily large finite models has an infinite
87
First-order logic
model. Thus the class of all finite graphs is not an elementary class (the same holds for many other algebraic
structures).
There are also more subtle limitations of first-order logic that are implied by the compactness theorem. For example,
in computer science, many situations can be modeled as a directed graph of states (nodes) and connections (directed
edges). Validating such a system may require showing that no "bad" state can be reached from any "good" state.
Thus one seeks to determine if the good and bad states are in different connected components of the graph. However,
the compactness theorem can be used to show that connected graphs are not an elementary class in first-order logic,
and there is no formula φ(x,y) of first-order logic, in the signature of graphs, that expresses the idea that there is a
path from x to y. Connectedness can be expressed in second-order logic, however, but not with only existential set
quantifiers, as
also enjoys compactness.
Lindström's theorem
Per Lindström showed that the metalogical properties just discussed actually characterize first-order logic in the
sense that no stronger logic can also have those properties (Ebbinghaus and Flum 1994, Chapter XIII). Lindström
defined a class of abstract logical systems, and a rigorous definition of the relative strength of a member of this class.
He established two theorems for systems of this type:
• A logical system satisfying Lindström's definition that contains first-order logic and satisfies both the
Löwenheim–Skolem theorem and the compactness theorem must be equivalent to first-order logic.
• A logical system satisfying Lindström's definition that has a semidecidable logical consequence relation and
satisfies the Löwenheim–Skolem theorem must be equivalent to first-order logic.
Limitations
Although first-order logic is sufficient for formalizing much of mathematics, and is commonly used in computer
science and other fields, it has certain limitations. These include limitations on its expressiveness and limitations of
the fragments of natural languages that it can describe.
Expressiveness
The Löwenheim–Skolem theorem shows that if a first-order theory has any infinite model, then it has infinite models
of every cardinality. In particular, no first-order theory with an infinite model can be categorical. Thus there is no
first-order theory whose only model has the set of natural numbers as its domain, or whose only model has the set of
real numbers as its domain. Many extensions of first-order logic, including infinitary logics and higher-order logics,
are more expressive in the sense that they do permit categorical axiomatizations of the natural numbers or real
numbers. This expressiveness comes at a metalogical cost, however: by Lindström's theorem, the compactness
theorem and the downward Löwenheim–Skolem theorem cannot hold in any logic stronger than first-order.
Formalizing natural languages
First-order logic is able to formalize many simple quantifier constructions in natural language, such as "every person
who lives in Perth lives in Australia". But there are many more complicated features of natural language that cannot
be expressed in (single-sorted) first-order logic. "Any logical system which is appropriate as an instrument for the
analysis of natural language needs a much richer structure than first-order predicate logic" (Gamut 1991, p. 75).
88
First-order logic
89
Type
Example
Comment
Quantification over
properties
If John is self-satisfied, then there is at least
one thing he has in common with Peter
Requires a quantifier over predicates, which cannot be implemented in
single-sorted first-order logic: Zj→ ∃X(Xj∧Xp)
Quantification over
properties
Santa Claus has all the attributes of a sadist
Requires quantifiers over predicates, which cannot be implemented in
single-sorted first-order logic: ∀X(∀x(Sx → Xx)→Xs)
Predicate adverbial
John is walking quickly
Cannot be analysed as Wj ∧ Qj; predicate adverbials are not the same kind
of thing as second-order predicates such as colour
Relative adjective
Jumbo is a small elephant
Cannot be analysed as Sj ∧ Ej; predicate adjectives are not the same kind of
thing as second-order predicates such as colour
Predicate adverbial
modifier
John is walking very quickly
-
Relative adjective
modifier
Jumbo is terribly small
An expression such as "terribly", when applied to a relative adjective such
as "small", results in a new composite relative adjective "terribly small"
Prepositions
Mary is sitting next to John
The preposition "next to" when applied to "John" results in the predicate
adverbial "next to John"
Restrictions, extensions and variations
There are many variations of first-order logic. Some of these are inessential in the sense that they merely change
notation without affecting the semantics. Others change the expressive power more significantly, by extending the
semantics through additional quantifiers or other new logical symbols. For example, infinitary logics permit
formulas of infinite size, and modal logics add symbols for possibility and necessity.
Restricted languages
First-order logic can be studied in languages with fewer logical symbols than were described above.
• Because
can be expressed as
the two quantifiers and can be dropped.
• Since
can be expressed as
or
, and
can be expressed as
and
, either of
can be expressed as
can be dropped. In other words, it is sufficient to have
and
, or
and
, either
, as the only logical
connectives.
• Similarly, it is sufficient to have only and
as logical connectives, or to have only the Sheffer stroke
(NAND) or the Peirce arrow (NOR) operator.
• It is possible to entirely avoid function symbols and constant symbols, rewriting them via predicate symbols in an
appropriate way. For example, instead of using a constant symbol one may use a predicate
(interpreted
as
), and replace every predicate such as
with
will similarly be replaced by a predicate
. A function such as
interpreted as
. This change requires adding additional axioms to the theory at hand, so that
interpretations of the predicate symbols used have the correct semantics.
Restrictions such as these are useful as a technique to reduce the number of inference rules or axiom schemes in
deductive systems, which leads to shorter proofs of metalogical results. The cost of the restrictions is that it becomes
more difficult to express natural-language statements in the formal system at hand, because the logical connectives
used in the natural language statements must be replaced by their (longer) definitions in terms of the restricted
collection of logical connectives. Similarly, derivations in the limited systems may be longer than derivations in
systems that include additional connectives. There is thus a trade-off between the ease of working within the formal
system and the ease of proving results about the formal system.
First-order logic
90
It is also possible to restrict the arities of function symbols and predicate symbols, in sufficiently expressive theories.
One can in principle dispense entirely with functions of arity greater than 2 and predicates of arity greater than 1 in
theories that include a pairing function. This is a function of arity 2 that takes pairs of elements of the domain and
returns an ordered pair containing them. It is also sufficient to have two predicate symbols of arity 2 that define
projection functions from an ordered pair to its components. In either case it is necessary that the natural axioms for
a pairing function and its projections are satisfied.
Many-sorted logic
Ordinary first-order interpretations have a single domain of discourse over which all quantifiers range. Many-sorted
first-order logic allows variables to have different sorts, which have different domains. This is also called typed
first-order logic, and the sorts called types (as in data type), but it is not the same as first-order type theory.
Many-sorted first-order logic is often used in the study of second-order arithmetic.
When there are only finitely many sorts in a theory, many-sorted first-order logic can be reduced to single-sorted
first-order logic. One introduces into the single-sorted theory a unary predicate symbol for each sort in the
many-sorted theory, and adds an axiom saying that these unary predicates partition the domain of discourse. For
example, if there are two sorts, one adds predicate symbols
and
and the axiom
.
Then the elements satisfying
are thought of as elements of the first sort, and elements satisfying
as elements
of the second sort. One can quantify over each sort by using the corresponding predicate symbol to limit the range of
quantification. For example, to say there is an element of the first sort satisfying formula φ(x), one writes
.
Additional quantifiers
Additional quantifiers can be added to first-order logic.
• Sometimes it is useful to say that "P(x) holds for exactly one x", which can be expressed as
notation, called uniqueness quantification, may be taken to abbreviate a formula such as
x P(x). This
x (P(x)
y (P(y)
(x = y))).
• First-order logic with extra quantifiers has new quantifiers Qx,..., with meanings such as "there are many x
such that ...". Also see branching quantifiers and the plural quantifiers of George Boolos and others.
• Bounded quantifiers are often used in the study of set theory or arithmetic.
Infinitary logics
Infinitary logic allows infinitely long sentences. For example, one may allow a conjunction or disjunction of
infinitely many formulas, or quantification over infinitely many variables. Infinitely long sentences arise in areas of
mathematics including topology and model theory.
Infinitary logic generalizes first-order logic to allow formulas of infinite length. The most common way in which
formulas can become infinite is through infinite conjunctions and disjunctions. However, it is also possible to admit
generalized signatures in which function and relation symbols are allowed to have infinite arities, or in which
quantifiers can bind infinitely many variables. Because an infinite formula cannot be represented by a finite string, it
is necessary to choose some other representation of formulas; the usual representation in this context is a tree. Thus
formulas are, essentially, identified with their parse trees, rather than with the strings being parsed.
The most commonly studied infinitary logics are denoted Lαβ, where α and β are each either cardinal numbers or the
symbol ∞. In this notation, ordinary first-order logic is Lωω. In the logic L∞ω, arbitrary conjunctions or disjunctions
are allowed when building formulas, and there is an unlimited supply of variables. More generally, the logic that
permits conjunctions or disjunctions with less than κ constituents is known as Lκω. For example, Lω1ω permits
First-order logic
countable conjunctions and disjunctions.
The set of free variables in a formula of Lκω can have any cardinality strictly less than κ, yet only finitely many of
them can be in the scope of any quantifier when a formula appears as a subformula of another.[6] In other infinitary
logics, a subformula may be in the scope of infinitely many quantifiers. For example, in Lκ∞, a single universal or
existential quantifier may bind arbitrarily many variables simultaneously. Similarly, the logic Lκλ permits
simultaneous quantification over fewer than λ variables, as well as conjunctions and disjunctions of size less than κ.
Non-classical and modal logics
• Intuitionistic first-order logic uses intuitionistic rather than classical propositional calculus; for example, ¬¬φ
need not be equivalent to φ.
• First-order modal logic allows one to describe other possible worlds as well as this contingently true world which
we inhabit. In some versions, the set of possible worlds varies depending on which possible world one inhabits.
Modal logic has extra modal operators with meanings which can be characterized informally as, for example "it
is necessary that φ" (true in all possible worlds) and "it is possible that φ" (true in some possible world). With
standard first-order logic we have a single domain and each predicate is assigned one extension. With first-order
modal logic we have a domain function that assigns each possible world its own domain, so that each predicate
gets an extension only relative to these possible worlds. This allows us to model cases where, for example, Alex is
a Philosopher, but might have been a Mathematician, and might not have existed at all. In the first possible world
P(a) is true, in the second P(a) is false, and in the third possible world there is no a in the domain at all.
• first-order fuzzy logics are first-order extensions of propositional fuzzy logics rather than classical propositional
calculus.
Higher-order logics
The characteristic feature of first-order logic is that individuals can be quantified, but not predicates. Thus
is a legal first-order formula, but
is not, in most formalizations of first-order logic. Second-order logic extends first-order logic by adding the latter
type of quantification. Other higher-order logics allow quantification over even higher types than second-order logic
permits. These higher types include relations between relations, functions from relations to relations between
relations, and other higher-type objects. Thus the "first" in first-order logic describes the type of objects that can be
quantified.
Unlike first-order logic, for which only one semantics is studied, there are several possible semantics for
second-order logic. The most commonly employed semantics for second-order and higher-order logic is known as
full semantics. The combination of additional quantifiers and the full semantics for these quantifiers makes
higher-order logic stronger than first-order logic. In particular, the (semantic) logical consequence relation for
second-order and higher-order logic is not semidecidable; there is no effective deduction system for second-order
logic that is sound and complete under full semantics.
Second-order logic with full semantics is more expressive than first-order logic. For example, it is possible to create
axiom systems in second-order logic that uniquely characterize the natural numbers and the real line. The cost of this
expressiveness is that second-order and higher-order logics have fewer attractive metalogical properties than
first-order logic. For example, the Löwenheim–Skolem theorem and compactness theorem of first-order logic
become false when generalized to higher-order logics with full semantics.
91
First-order logic
Automated theorem proving and formal methods
Further information: First-order theorem proving
Automated theorem proving refers to the development of computer programs that search and find derivations (formal
proofs) of mathematical theorems. Finding derivations is a difficult task because the search space can be very large;
an exhaustive search of every possible derivation is theoretically possible but computationally infeasible for many
systems of interest in mathematics. Thus complicated heuristic functions are developed to attempt to find a
derivation in less time than a blind search.
The related area of automated proof verification uses computer programs to check that human-created proofs are
correct. Unlike complicated automated theorem provers, verification systems may be small enough that their
correctness can be checked both by hand and through automated software verification. This validation of the proof
verifier is needed to give confidence that any derivation labeled as "correct" is actually correct.
Some proof verifiers, such as Metamath, insist on having a complete derivation as input. Others, such as Mizar and
Isabelle, take a well-formatted proof sketch (which may still be very long and detailed) and fill in the missing pieces
by doing simple proof searches or applying known decision procedures: the resulting derivation is then verified by a
small, core "kernel". Many such systems are primarily intended for interactive use by human mathematicians: these
are known as proof assistants. They may also use formal logics that are stronger than first-order logic, such as type
theory. Because a full derivation of any nontrivial result in a first-order deductive system will be extremely long for
a human to write,[7] results are often formalized as a series of lemmas, for which derivations can be constructed
separately.
Automated theorem provers are also used to implement formal verification in computer science. In this setting,
theorem provers are used to verify the correctness of programs and of hardware such as processors with respect to a
formal specification. Because such analysis is time-consuming and thus expensive, it is usually reserved for projects
in which a malfunction would have grave human or financial consequences.
Notes
[1] Mendelson, Elliott (1964). Introduction to Mathematical Logic. Van Nostrand Reinhold. pp. 56.
[2] The word language is sometimes used as a synonym for signature, but this can be confusing because "language" can also refer to the set of
formulas.
[3] More precisely, there is only one language of each variant of one-sorted first-order logic: with or without equality, with or without functions,
with or without propositional variables, ….
[4] Some authors who use the term "well-formed formula" use "formula" to mean any string of symbols from the alphabet. However, most
authors in mathematical logic use "formula" to mean "well-formed formula" and have no term for non-well-formed formulas. In every context,
it is only the well-formed formulas that are of interest.
[5] The SMT-LIB Standard: Version 2.0, by Clark Barrett, Aaron Stump, and Cesare Tinelli. http:/ / goedel. cs. uiowa. edu/ smtlib/
[6] Some authors only admit formulas with finitely many free variables in Lκω, and more generally only formulas with < λ free variables in Lκλ.
[7] Avigad et al. (2007) discuss the process of formally verifying a proof of the prime number theorem. The formalized proof required
approximately 30,000 lines of input to the Isabelle proof verifier.
92
First-order logic
References
• Peter Andrews. 2002. An Introduction to Mathematical Logic and Type Theory: To Truth Through Proof. 2nd ed.
Berlin: Kluwer Academic Publishers, available from Springer.
• Jeremy Avigad, Kevin Donnelly, David Gray, Paul Raff, 2007. "A formally verified proof of the prime number
theorem", ACM Transactions on Computational Logic, v. 9 n. 1. doi:10.1145/1297658.1297660
• Jon Barwise, 1977. "An introduction to first-order logic", in Barwise, Jon, ed (1982). Handbook of Mathematical
Logic. Studies in Logic and the Foundations of Mathematics. Amsterdam: North-Holland.
ISBN 978-0-444-86388-1
• Jon Barwise and John Etchemendy, 2000. Language Proof and Logic. Stanford, CA: CSLI Publications
(Distributed by the University of Chicago Press).
• Józef Maria Bocheński, 2007. A Précis of Mathematical Logic. Translated from the French and German editions
by Otto Bird. Dordrecht, South Holland: D. Reidel.
• José Ferreirós. The Road to Modern Logic—An Interpretation. (http://jstor.org/stable/2687794) Bulletin of
Symbolic Logic, Volume 7, Issue 4, 2001, pp. 441–484. DOI 10.2307/2687794. JStor (http://links.jstor.org/
sici?sici=1079-8986(200112)7:4<441:TRTMLI>2.0.CO;2-O)
• L. T. F. Gamut, 1991. Logic, Language, and Meaning, Volume 2: Introduction to Logic. Chicago: University Of
Chicago Press. ISBN 0-226-28088-8.
• David Hilbert and Wilhelm Ackermann 1950. Principles of Mathematical Logic (English translation). Chelsea.
The 1928 first German edition was titled Grundzüge der theoretischen Logik.
• Wilfrid Hodges, 2001, "Classical Logic I: First Order Logic," in Lou Goble, ed., The Blackwell Guide to
Philosophical Logic. Blackwell.
• Heinz-Dieter Ebbinghaus, Jörg Flum, and Wolfgang Thomas. 1994. Mathematical Logic. Berlin, New York:
Springer-Verlag. Second Edition. Undergraduate Texts in Mathematics. ISBN 978-0-387-94258-2.
External links
• Stanford Encyclopedia of Philosophy: " Classical Logic (http://plato.stanford.edu/entries/logic-classical/) -by Stewart Shapiro. Covers syntax, model theory, and metatheory for first-order logic in the natural deduction
style.
• forall x: an introduction to formal logic (http://www.fecundity.com/logic/), by P.D. Magnus, covers formal
semantics and proof theory for first-order logic.
• Metamath (http://us.metamath.org/index.html): an ongoing online project to reconstruct mathematics as a
huge first-order theory, using first-order logic and the axiomatic set theory ZFC. Principia Mathematica
modernized.
• Podnieks, Karl. Introduction to mathematical logic. (http://www.ltn.lv/~podnieks/)
• Cambridge Mathematics Tripos Notes (http://john.fremlin.de/schoolwork/logic/index.html) (typeset by John
Fremlin). These notes cover part of a past Cambridge Mathematics Tripos course taught to undergraduates
students (usually) within their third year. The course is entitled "Logic, Computation and Set Theory" and covers
Ordinals and cardinals, Posets and zorn’s Lemma, Propositional logic, Predicate logic, Set theory and Consistency
issues related to ZFC and other set theories.
93
List of first-order theories
List of first-order theories
In mathematical logic, a first-order theory is given by a set of axioms in some language. This entry lists some of the
more common examples used in model theory and some of their properties.
Preliminaries
For every natural mathematical structure there is a signature σ listing the constants, functions, and relations of the
theory together with their valences, so that the object is naturally a σ-structure. Given a signature σ there is a unique
first-order language Lσ that can be used to capture the first-order expressible facts about the σ-structure.
There are two common ways to specify theories:
1. List or describe a set of sentences in the language Lσ, called the axioms of the theory.
2. Give a set of σ-structures, and define a theory to be the set of sentences in Lσ holding in all these models. For
example, the "theory of finite fields" consists of all sentences in the language of fields that are true in all finite
fields.
An Lσ theory may:
• be consistent: no proof of contradiction exists;
• be satisfiable: there exists a σ-structure for which the sentences of the theory are all true (by the completeness
theorem, satisfiability is equivalent to consistency);
• be complete: for any statement, either it or its negation is provable;
• have quantifier elimination;
• eliminate imaginaries;
• be finitely axiomatizable;
• be decidable: There is an algorithm to decide which statements are provable;
• be recursively axiomatizable;
• be Model complete or sub-model complete;
• be κ-categorical: All models of cardinality κ are isomorphic;
• be Stable or unstable.
• be ω-stable (same as totally transcendental for countable theories).
• be superstable
• have an atomic model
• have a prime model
• have a saturated model
Pure identity theories
The signature of the pure identity theory is empty, with no functions, constants, or relations.
Pure identity theory has no (non-logical) axioms. It is decidable.
One of the few interesting properties that can be stated in the language of pure identity theory is that of being
infinite. This is given by an infinite set of axioms stating there are at least 2 elements, there are at least 3 elements,
and so on:
• ∃x1 ∃x2 ¬x1 = x2, ∃x1 ∃x2 ∃x3 ¬x1 = x2 ∧ ¬x1 = x3 ∧ ¬x2 = x3,...
These axioms define the theory of an infinite set.
The opposite property of being finite cannot be stated in first-order logic for any theory that has arbitrarily large
finite models: in fact any such theory has infinite models by the compactness theorem. In general if a property can be
stated by a finite number of sentences of first-order logic then the opposite property can also be stated in first-order
94
List of first-order theories
logic, but if a property needs an infinite number of sentences then its opposite property cannot be stated in first-order
logic.
Any statement of pure identity theory is equivalent to either σ(N) or to ¬σ(N) for some finite subset N of the
non-negative integers, where σ(N) is the statement that the number of elements is in N. It is even possible to describe
all possible theories in this language as follows. Any theory is either the theory of all sets of cardinality in N for
some finite subset N of the non-negative integers, or the theory of all sets whose cardinality is not in N, for some
finite or infinite subset N of the non-negative integers. (There are no theories whose models are exactly sets of
cardinality N if N is an infinite subset of the integers.) The complete theories are the theories of sets of cardinality n
for some finite n, and the theory of infinite sets.
One special case of this is the inconsistent theory defined by the axiom ∃x ¬x = x. It is a perfectly good theory with
many good properties: it is complete, decidable, finitely axiomatizable, and so on. The only problem is that it has no
models at all. By Gödel's completeness theorem, it is the only theory (for any given language) with no models.
Unary relations
A set of unary relations Pi for i in some set I is called independent if for every two disjoint finite subsets A and B of
I there is some element x such that Pi(x) is true for i in A and false for i in B. Independence can be expressed by a set
of first-order statements.
The theory of a countable number of independent unary relations is complete, but has no atomic models. It is
also an example of a theory that is superstable but not totally transcendental.
Equivalence relations
The signature of equivalence relations has one binary infix relation symbol ~, no constants, and no functions.
Equivalence relations satisfy the axioms:
• Reflexivity ∀x x~x;
• Symmetry ∀x ∀y x~y → y~x;
• Transitivity: ∀x ∀y ∀z (x~y ∧ y~z) → x~z.
Some first order properties of equivalence relations are:
•
•
•
•
~ has an infinite number of equivalence classes;
~ has exactly n equivalence classes (for any fixed positive integer n);
All equivalence classes are infinite;
All equivalence classes have size exactly n (for any fixed positive integer n).
The theory of an equivalence relation with exactly 2 infinite equivalence classes is an easy example of a theory
which is ω-categorical but not categorical for any larger cardinal.
The equivalence relation ~ should not be confused with the identity symbol '=': if x=y then x~y, but the converse is
not necessarily true. Theories of equivalence relations are not all that difficult or interesting, but often give easy
examples or counterexamples for various statements.
The following constructions are sometimes used to produce examples of theories with certain spectra; in fact by
applying them to a small number of explicit theories T one gets examples of complete countable theories with all
possible uncountable spectra. If T is a theory in some language, we define a new theory 2T by adding a new binary
relation to the language, and adding axioms stating that it is an equivalence relation, such that there are an infinite
number of equivalence classes all of which are models of T. It is possible to iterate this construction transfinitely:
given an ordinal α, define a new theory by adding an equivalence relation Eβ for each β<α, together with axioms
stating that whenever β<γ then each Eγ equivalence class is the union of infinitely many Eβ equivalence classes, and
each E0 equivalence class is a model of T. Informally, one can visualize models of this theory as infinitely branching
95
List of first-order theories
96
trees of height α with models of T attached to all leaves.
Orders
The signature of orders has no constants or functions, and one binary relation symbols ≤. (It is of course possible to
use ≥, < or > instead as the basic relation, with the obvious minor changes to the axioms.) We define x≥y, x<y, x>y as
abbreviations for y≤x, x≤y ∧¬y≤x, y<x,
Some first-order properties of orders:
•
•
•
•
•
•
•
•
•
Transitive: ∀x ∀y ∀z x ≤ y∧y ≤ z → x ≤ z
Reflexive: ∀x x ≤ x
Antisymmetric: ∀x ∀y x ≤ y ∧ y ≤ x → x = y
Partial: Transitive∧Reflexive∧Antisymmetric;
Linear (or total): Partial ∧ ∀x ∀y x≤y ∨ y≤x
Dense ∀x ∀z x<z → ∃y x<y ∧ y<z ("Between any 2 distinct elements there is another element")
There is a smallest element: ∃x ∀y x≤y
There is a largest element: ∃x ∀y y≤x
Every element has an immediate successor: ∀x ∃y ∀z x<z ↔ y≤z
The theory DLO of dense linear orders with no endpoints (i.e. no smallest or largest element) is complete,
ω-categorical, but not categorical for any uncountable cardinal. There are 3 other very similar theories: the theory of
dense linear orders with a:
• Smallest but no largest element;
• Largest but no smallest element;
• Largest and smallest element.
Being well ordered ("any non-empty subset has a minimal element") is not a first-order property; the usual
definition involves quantifying over all subsets.
Lattices
Lattices can be considered either as special sorts of partially ordered sets, with a signature consisting of one binary
relation symbol ≤, or as algebraic structures with a signature consisting of two binary operations ∧ and ∨. The two
approaches can be related by defining a≤ b to mean a∧b=a.
For two binary operations the axioms for a lattice are:
Commutative laws:
Associative laws:
Absorption laws:
For one relation ≤ the axioms are:
• Axioms stating ≤ is a partial order, as above.
•
•
(existence of c=a∧b)
(existence of c=a∨b)
First order properties include:
•
•
Completeness is not a first order property of lattice.
(distributive lattices)
(modular lattices)
List of first-order theories
Graphs
The signature of graphs has no constants or functions, and one binary relation symbol R, where R(x,y) is read as
"there is an edge from x to y".
The axioms for the theory of graphs are
• Symmetric: ∀x ∀y R(x,y)→ R(y,x)
• Anti-reflexive: ∀x ¬R(x,x) ("no loops")
The theory of random graphs has the following extra axioms for each positive integer n:
• For any two disjoint finite sets of size n, there is a point joined to all points of the first set and to no points of the
second set. (For each fixed n, it is easy to write this statement in the language of graphs.)
The theory of random graphs is ω categorical, complete, and decidable, and its countable model is called the Rado
graph. A statement in the language of graphs is true in this theory if and only if it is true with probability 1 for a
random graph on a countable number of points.
Boolean algebras
There are several different signatures and conventions used for Boolean algebras:
1. The signature has 2 constants, 0 and 1, and two binary functions ∧ and ∨ ("and" and "or"), and one unary
function ¬ ("not"). This is a little confusing as the functions use the same symbols as the propositional functions
of first-order logic.
2. In set theory, a common convention is that the language has 2 constants, 0 and 1, and two binary functions · and
+, and one unary function −. The three functions have the same interpretation as the functions in the first
convention. Unfortunately, this convention clashes badly with the next convention:
3. In algebra, the usual convention is that the language has 2 constants, 0 and 1, and two binary functions · and +.
The function · has the same meaning as ∧, but a+b means a∨b∧¬(a∧b). The reason for this is that the axioms for a
Boolean algebra are then just the axioms for a ring with 1 plus ∀x x2 = x. Unfortunately this clashes with the
standard convention in set theory given above.
The axioms are:
• The axioms for a distributive lattice (see above)
• ∀a∀b a∧¬a = 0, ∀a∀b a∨¬a = 1 (properties of negation)
• Some authors add the extra axiom ¬0=1, to exclude the trivial algebra with one element.
Tarski proved that the theory of Boolean algebras is decidable.
We write x ≤ y as an abbreviation for x ∧ y = x, and atom(x) as an abbreviation for ¬x = 0 ∧ ∀y y≤x → y = 0 ∨ y = x,
read as "x is an atom", in other words a non-zero element with nothing between it and 0. Here are some first-order
properties of Boolean algebras:
• Atomic: ∀x x=0 ∨ ∃y y≤x ∧ atom(y)
• Atomless: ∀x ¬atom(x)
The theory of atomless Boolean algebras is ω-categorical and complete.
For any Boolean algebra B, there are several invariants defined as follows.
• the ideal I(B) consists of elements that are the sum of an atomic and an atomless element.
• The quotient algebras Bi of B are defined inductively by B0=B, Bk+1 = Bk/I(Bk).
• The invariant m(B) is the smallest integer such that Bm+1 is trivial, or ∞ if no such integer exists.
• If m(B) is finite, the invariant n(B) is the number of atoms of Bm(B) if this number is finite, or ∞ if this number is
infinite.
• The invariant l(B) is 0 if Bm(B) is atomic or if m(B) is ∞, and 1 otherwise.
97
List of first-order theories
Then two Boolean algebras are elementarily equivalent if and only if their invariants l, m, and n are the same. In
other words, the values of these invariants classify the possible completions of the theory of Boolean algebras. So the
possible complete theories are:
• The trivial algebra (if this is allowed; sometimes 0≠1 is included as an axiom.)
• The theory with m=∞
• The theories with m a natural number, n a natural number or ∞, and l = 0 or 1 (with l = 0 if n=0).
Groups
The signature of group theory has one constant 1 (the identity), one function of arity 1 (the inverse) whose value on t
is denoted by t−1, and one function of arity 2, which is usually omitted from terms. For any integer n. tn is an
abbreviation for the obvious term for the nth power of t.
Groups are defined by the axioms
• Identity: ∀x 1x = x ∧ x1 = x
• Inverse: ∀x x−1x = 1 ∧ xx−1 = 1
• Associative: ∀x∀y∀z (xy)z = x(yz)
Some properties of groups that can be defined in the first-order language of groups are:
•
•
•
•
•
•
•
Abelian ∀x ∀y xy = yx.
Torsion free ∀x x2 = 1→x = 1, ∀x x3 = 1 → x = 1, ∀x x4 = 1 → x = 1, ...
Divisible ∀x ∃y y2 = x, ∀x ∃y y3 = x, ∀x ∃y y4 = x, ...
Infinite (as in identity theory)
Exponent n (for any fixed positive integer n) ∀x xn = 1
Nilpotent of class n (for any fixed positive integer n)
Solvable of class n (for any fixed positive integer n)
The theory of Abelian groups is decidable. The theory of Infinite divisible torsion-free abelian groups is
complete, as is the theory of Infinite abelian groups of exponent p (for p prime).
The theory of finite groups is the set of first-order statements in the language of groups that are true in all finite
groups (there are plenty of infinite models of this theory). It is not completely trivial to find any such statement that
is not true for all groups: one example is "given two elements of order 2, either they are conjugate or there is a
non-trivial element commuting with both of them".
The properties of being finite, or free, or simple, or torsion are not first-order. More precisely, the first-order theory
of all groups with one of these properties has models that do not have this property.
Rings and fields
The signature of (unital) rings has 2 constants 0 and 1, two binary functions + and ×, and, optionally, one unary
inverse functions − −1.
Rings Axioms: Addition makes the ring into an abelian group, multiplication is associative and has an identity 1, and
multiplication is left and right distributive.
Commutative rings The axioms for rings plus ∀x ∀y xy=yx.
Fields The axioms for commutative rings plus ∀x ∃y xy=1 and ¬ 1=0. Many of the examples given here have only
universal, or algebraic axioms. The class of structures satisfying such a theory has the property of being closed
under substructure. For example, a subset of a group closed under the group actions of multiplication and inverse is
again a group. Since the signature of fields does not usually include multiplicative and additive inverse, the axioms
for inverses are not universal, and therefore a substructure of a field closed under addition and multiplication is not
always a field. This can be remedied by adding unary inverse functions to the language.
98
List of first-order theories
99
For any positive integer n the property that all equations of degree n have a root can be expressed by a single
first-order sentence:
• ∀ a1 ∀ a2... ∀ an ∃x (...((x+a1)x +a2)x+...)x+an = 0
Perfect fields The axioms for fields, plus axioms for each prime number p stating that if p 1 = 0 (i.e. the field has
characteristic p), then every field element has a pth root.
Algebraically closed fields of characteristic p The axioms for fields, plus for every positive n the axiom that all
polynomials of degree n have a root, plus axioms fixing the characteristic. The classical examples of complete
theories. Categorical in all uncountable cardinals. The theory ACFp has a universal domain property, in the sense
that every structure N satisfying the universal axioms of ACFp is a substructure of a sufficiently large algebraically
closed field
, and additionally any two such embeddings N → M induce an automorphism of M.
Finite fields. The theory of finite fields is the set of all first-order statements that are true in all finite fields.
Significant examples of such statements can, for example, be given by applying the Chevalley–Warning theorem,
over the prime fields. The name is a little misleading as the theory has plenty of infinite models. Ax proved that the
theory is decidable.
Formally real fields These are fields with the axiom
• For every positive n, the axiom ∀ a1 ∀ a2... ∀ an a1a1+a2a2+ ...+anan=0 → a1=0∨a2=0∨ ... ∨an=0 (0 is not a
non-trivial sum of squares).
Real closed fields Axioms:
• ∀x ∃y x=yy ∨ x+yy=0.
• For every odd positive n, the axiom stating that every polynomial of degree n has a root.
• For every positive n, the axiom ∀ a1 ∀ a2... ∀ an a1a1+a2a2+ ...+anan=0 → a1=0∨a2=0∨ ... ∨an=0 (0 is not a
non-trivial sum of squares).
The theory of real closed fields is decidable (Tarski) and therefore complete.
p-adic fields: Ax & Kochen (1965) showed that the theory of p-adic fields is decidable and gave a set of axioms for
it.
Geometry
Axioms for various systems of geometry usually use a typed language, with the different types corresponding to
different geometric objects such as points, lines, circles, planes, and so on. The signature will often consist of binary
incidence relations between objects of different types; for example, the relation that a point lies on a line. The
signature may have more complicated relations; for example ordered geometry might have a ternary "betweenness"
relation for 3 points, which says whether one lies between two others, or a "congruence" relation between 2 pairs of
points.
Some examples of axiomatized systems of geometry include ordered geometry, absolute geometry, affine geometry,
Euclidean geometry, projective geometry, and hyperbolic geometry. For each of these geometries there are many
different and inequivalent systems of axioms for various dimensions. Some of these axiom systems include
"completeness" axioms that are not first order.
As a typical example, the axioms for projective geometry use 2 types, points and lines, and a binary incidence
relation between points and lines. If point and line variables are indicated by small and capital letter, and a incident
to A is written as aA, then one set of axioms is
•
•
(There is a line through any 2 distinct points a,b ...)
(... which is unique)
•
(Veblen's
axiom: if ab and cd lie on intersecting lines, then so do ac and bd.)
List of first-order theories
•
100
(Every line has at least 3 points)
Euclid did not state all the axioms for Euclidean geometry explicitly, and the first complete list was given by Hilbert
in Hilbert's axioms. This is not a first order axiomatization as one of Hilbert's axioms is a second order completeness
axiom. Tarski's axioms are a first order axiomatization of Euclidean geometry. Tarski showed this axiom system is
complete and decidable by relating it to the complete and decidable theory of real closed fields.
Differential algebra
• The theory DF of differential fields.
The signature is that of fields (0, 1, +, -, ×) together with a unary function ∂, the derivation. The axioms are those for
fields together with
For this theory one can add the condition that the characteristic is p, a prime or zero, to get the theory DFp of
differential fields of characteristic p (and similarly with the other theories below).
If K is a differential field then the field of constants
The theory of differentially
perfect fields is the theory of differential fields together with the condition that the field of constants is perfect; in
other words for each prime p it has the axiom:
(There is little point in demanding that the whole field should be perfect field, because in non-zero characteristic this
implies the differential is 0.) For technical reasons to do with quantifier elimination it is sometimes more convenient
to force the constant field to be perfect by adding a new symbol r to the signature with the axioms
• The theory of DCF differentially closed fields is the theory of differentially perfect fields with axioms saying
that such that if f and g are differential polynomials and the separant of f is nonzero and g≠0 and f has order
greater than that of g, then there is some x in the field with f(x)=0 and g(x)≠0.
Addition
The theory of the natural numbers with a successor function has signature consisting of a constant 0 and a unary
function S ("successor": S(x) is interpreted as x+1), and has axioms:
1. ∀x ¬ Sx = 0
2. ∀x∀y Sx = Sy → x = y
3. Let P(x) be a first-order formula with a single free variable x. Then the following formula is an axiom:
(P(0) ∧ ∀x(P(x)→P(Sx))) → ∀y P(y).
The last axiom (induction) can be replaced by the axioms
• For each integer n>0, the axiom ∀x SSS...Sx ≠ x (with n copies of S)
• ∀x ¬ x = 0 → ∃y Sy = x
The theory of the natural numbers with a successor function is complete and decidable, and is κ-categorical for
uncountable κ but not for countable κ.
Presburger arithmetic is the theory of the natural numbers under addition, with signature consisting of a constant 0,
a unary function S, and a binary function +. It is complete and decidable. The axioms are
1. ∀x ¬ Sx = 0
2. ∀x∀y Sx = Sy → x = y
List of first-order theories
101
3. ∀x x + 0 = x
4. ∀x∀y x + Sy = S(x + y)
5. Let P(x) be a first-order formula with a single free variable x. Then the following formula is an axiom:
(P(0) ∧ ∀x(P(x)→P(Sx))) → ∀y P(y).
Arithmetic
Many of the first order theories described above can be extended to complete recursively enumerable consistent
theories. This is no longer true for most of the following theories; they can usually encode both multiplication and
addition of natural numbers, and this gives them enough power to encode themselves, which implies that Gödel's
incompleteness theorem applies and the theories can no longer be both complete and recursively enumerable (unless
they are inconsistent).
The signature of a theory of arithmetic has:
• The constant 0;
• The unary function, the successor function, here denoted by prefix S, or by prefix σ or postfix ′ elsewhere;
• Two binary functions, denoted by infix + and ×, called "addition" and "multiplication."
Some authors take the signature to contain a constant 1 instead of the function S, then define S in the obvious way as
St = 1 + t.
Robinson arithmetic (also called Q). Axioms (1) and (2) govern the distinguished element 0. (3) assures that S is an
injection. Axioms (4) and (5) are the standard recursive definition of addition; (6) and (7) do the same for
multiplication. Robinson arithmetic can be thought of as Peano arithmetic without induction. Q is a weak theory for
which Gödel's incompleteness theorem holds. Axioms:
1.
2.
3.
4.
5.
6.
7.
∀x ¬ Sx = 0
∀x ¬ x = 0 → ∃y Sy = x
∀x∀y Sx = Sy → x = y
∀x x + 0 = x
∀x∀y x + Sy = S(x + y)
∀x x × 0 = 0
∀x∀y x × Sy = (x × y) + x.
IΣn is first order Peano arithmetic with induction restricted to Σn formulas (for n = 0, 1, 2, ...). The theory IΣ0 is
often denoted by IΔ0. This is a series of more and more powerful fragments of Peano arithmetic. The case n = 1 has
about the same strength as primitive recursive arithmetic (PRA). Exponential function arithmetic (EFA) is IΣ0
with an axiom stating that xy exists for all x and y (with the usual properties).
First order Peano arithmetic, PA. The "standard" theory of arithmetic. The axioms are the axioms of Robinson
arithmetic above, together with the axiom scheme of induction:
•
for any formula φ in the language of PA. φ may contain free
variables other than x.
Kurt Gödel's 1931 paper proved that PA is incomplete, and has no consistent recursively enumerable completions.
Complete arithmetic (also known as true arithmetic) is the theory of the standard model of arithmetic, the natural
numbers N. It is complete but does not have a recursively enumerable set of axioms.
List of first-order theories
Second order arithmetic
Second-order arithmetic can refer to a first order theory (in spite of the name) with two types of variables, thought
of as varying over integers and subsets of the integers. (There is also a theory of arithmetic in second order logic that
is called second order arithmetic. It has only one model, unlike the corresponding theory in first order logic, which is
incomplete.) The signature will typically be the signature 0, S, +, × of arithmetic, together with a membership
relation ∈ between integers and subsets (though there are numerous minor variations). The axioms are those of
Robinson arithmetic, together with axiom schemes of induction and comprehension.
There are many different subtheories of second order arithmetic that differ in which formulas are allowed in the
induction and comprehension schemes. In order of increasing strength, five of the most common systems are
•
•
•
•
•
, Recursive Comprehension
, Weak König's lemma
, Arithmetical comprehension
, Arithmetical Transfinite Recursion
,
comprehension
These are defined in detail in the articles on second order arithmetic and reverse mathematics.
Set theories
The usual signature of set theory has one binary relation ∈, no constants, and no functions. Some of the theories
below are "class theories" which have two sorts of object, sets and classes. There are three common ways of
handling this in first-order logic:
1. Use first-order logic with two types.
2. Use ordinary first-order logic, but add a new unary predicate "Set", where "Set(t)" means informally "t is a set".
3. Use ordinary first-order logic, and instead of adding a new predicate to the language, treat "Set(t)" as an
abbreviation for "∃y t∈y"
Some first order set theories include:
• Weak theories lacking powersets:
•
•
•
•
•
•
•
•
•
• S' (Tarski, Mostowksi, and Robinson, 1953); (finitely axiomatizable)
• General set theory;
• Kripke-Platek set theory;
Zermelo set theory;
Ackermann set theory
Zermelo-Fraenkel set theory;
Von Neumann-Bernays-Gödel set theory; (finitely axiomatizable)
Morse–Kelley set theory;
Tarski–Grothendieck set theory;
New Foundations; (finitely axiomatizable)
Scott-Potter set theory
Positive set theory
Some extra first order axioms that can be added to one of these (usually ZF) include:
• axiom of choice, axiom of dependent choice
• Generalized continuum hypothesis
• Martin's axiom (usually together with the negation of the continuum hypothesis), Martin's maximum
• ◊ and ♣
• Axiom of constructibility (V=L)
102
List of first-order theories
• proper forcing axiom
• analytic determinacy, projective determinacy, Axiom of determinacy
• Many large cardinal axioms
References
• Ax, James; Kochen, Simon (1965), "Diophantine problems over local fields. II. A complete set of axioms for
p-adic number theory.", Amer. J. Math. (The Johns Hopkins University Press) 87 (3): 631–648,
doi:10.2307/2373066, JSTOR 2373066, MR0184931
• Chang, C.C.; Keisler, H. Jerome (1989), Model Theory (3 ed.), Elsevier, ISBN 0-7204-0692-7
• Hodges, Wilfrid (1997), A shorter model theory, Cambridge University Press, ISBN 0-521-58713-1
• Marker, David (2002), Model Theory: An Introduction, Graduate Texts in Mathematics, 217, Springer,
ISBN 0-387-98760-6
Pole splitting
Pole splitting is a phenomenon exploited in some forms of frequency compensation used in an electronic amplifier.
When a capacitor is introduced between the input and output sides of the amplifier with the intention of moving the
pole lowest in frequency (usually an input pole) to lower frequencies, pole splitting causes the pole next in frequency
(usually an output pole) to move to a higher frequency. This pole movement increases the stability of the amplifier
and improves its step response at the cost of decreased speed.[1] [2] [3] [4]
Example of pole splitting
This example shows that introduction of the capacitor
referred to as CC in the amplifier of Figure 1 has two
results: first it causes the lowest frequency pole of the
amplifier to move still lower in frequency and second,
it causes the higher pole to move higher in frequency.[5]
The amplifier of Figure 1 has a low frequency pole due
to the added input resistance Ri and capacitance Ci,
with the time constant Ci ( RA // Ri ). This pole is moved
Figure 1: Operational amplifier with compensation capacitor CC
down in frequency by the Miller effect. The amplifier is
between input and output; notice the amplifier has both input
given a high frequency output pole by addition of the
impedance Ri and output impedance Ro.
load resistance RL and capacitance CL, with the time
constant CL ( Ro // RL ). The upward movement of the high-frequency pole occurs because the Miller-amplified
compensation capacitor CC alters the frequency dependence of the output voltage divider.
The first objective, to show the lowest pole moves down in frequency, is established using the same approach as the
Miller's theorem article. Following the procedure described in the article on Miller's theorem, the circuit of Figure 1
is
transformed
to
that
of
Figure
103
Pole splitting
104
2, which is electrically equivalent to Figure 1.
Application of Kirchhoff's current law to the input side
of Figure 2 determines the input voltage to the ideal op
amp as a function of the applied signal voltage
,
namely,
Figure 2: Operational amplifier with compensation capacitor
transformed using Miller's theorem to replace the compensation
capacitor with a Miller capacitor at the input and a
frequency-dependent current source at the output.
which exhibits a roll-off with frequency beginning at f1 where
which introduces notation
for the time constant of the lowest pole. This frequency is lower than the initial low
frequency of the amplifier, which for CC = 0 F is
.
Turning to the second objective, showing the higher pole moves still higher in frequency, it is necessary to look at
the output side of the circuit, which contributes a second factor to the overall gain, and additional frequency
dependence. The voltage
is determined by the gain of the ideal op amp inside the amplifier as
Using this relation and applying Kirchhoff's current law to the output side of the circuit determines the load voltage
as a function of the voltage at the input to the ideal op amp as:
This expression is combined with the gain factor found earlier for the input side of the circuit to obtain the overall
gain as
This gain formula appears to show a simple two-pole response with two time constants. (It also exhibits a zero in the
numerator but, assuming the amplifier gain Av is large, this zero is important only at frequencies too high to matter in
this discussion , so the numerator can be approximated as unity.) However, although the amplifier does have a
two-pole behavior, the two time-constants are more complicated than the above expression suggests because the
Miller capacitance contains a buried frequency dependence that has no importance at low frequencies, but has
considerable effect at high frequencies. That is, assuming the output R-C product, CL ( Ro // RL ), corresponds to a
frequency well above the low frequency pole, the accurate form of the Miller capacitance must be used, rather than
Pole splitting
105
the Miller approximation. According to the article on Miller effect, the Miller capacitance is given by
(For a positive Miller capacitance, Av is negative.) Upon substitution of this result into the gain expression and
collecting terms, the gain is rewritten as:
with Dω given by a quadratic in ω, namely:
Every quadratic has two factors, and this expression looks simpler if it is rewritten as
where and are combinations of the capacitances and resistances in the formula for Dω.[6] They correspond to
the time constants of the two poles of the amplifier. One or the other time constant is the longest; suppose is the
longest time constant, corresponding to the lowest pole, and suppose
>> . (Good step response requires
>>
. See Selection of CC below.)
At low frequencies near the lowest pole of this amplifier, ordinarily the linear term in ω is more important than the
quadratic term, so the low frequency behavior of Dω is:
where now CM is redefined using the Miller approximation as
which is simply the previous Miller capacitance evaluated at low frequencies. On this basis
is determined,
provided
>> . Because CM is large, the time constant is much larger than its original value of Ci ( RA // Ri
).[7]
At high frequencies the quadratic term becomes important. Assuming the above result for
is valid, the second
time constant, the position of the high frequency pole, is found from the quadratic term in Dω as
Substituting in this expression the quadratic coefficient corresponding to the product
for , an estimate for the position of the second pole is found:
along with the estimate
and because CM is large, it seems is reduced in size from its original value CL ( Ro // RL ); that is, the higher pole
has moved still higher in frequency because of CC.[8]
In short, introduction of capacitor CC moved the low pole lower and the high pole higher, so the term pole splitting
seems a good description.
Pole splitting
106
Selection of CC
What value is a good choice for CC? For
general purpose use, traditional design
(often called dominant-pole or single-pole
compensation) requires the amplifier gain to
drop at 20 dB/decade from the corner
frequency down to 0 dB gain, or even
lower.[9] [10] With this design the amplifier
is stable and has near-optimal step response
even as a unity gain voltage buffer. A more
aggressive
technique
is
two-pole
compensation.[11] [12]
Figure 3: Idealized Bode plot for a two pole amplifier design. Gain drops from first
pole at f1 at 20 dB / decade down to second pole at f2 where the slope increases to
40 dB / decade.
The way to position f2 to obtain the design is
shown in Figure 3. At the lowest pole f1, the Bode gain plot breaks slope to fall at 20 dB/decade. The aim is to
maintain the 20 dB/decade slope all the way down to zero dB, and taking the ratio of the desired drop in gain (in dB)
of 20 log10 Av to the required change in frequency (on a log frequency scale[13] ) of ( log10 f2 − log10 f1 ) = log10 ( f2 /
f1 ) the slope of the segment between f1 and f2 is:
Slope per decade of frequency
which is 20 dB/decade provided f2 = Av f1 . If f2 is not this large, the second break in the Bode plot that occurs at the
second pole interrupts the plot before the gain drops to 0 dB with consequent lower stability and degraded step
response.
Figure 3 shows that to obtain the correct gain dependence on frequency, the second pole is at least a factor Av higher
in frequency than the first pole. The gain is reduced a bit by the voltage dividers at the input and output of the
amplifier, so with corrections to Av for the voltage dividers at input and output the pole-ratio condition for good step
response becomes:
Using the approximations for the time
constants developed above,
Figure 4: Miller capacitance at low frequencies CM (top) and compensation capacitor CC
(bottom) as a function of gain using Excel. Capacitance units are pF.
or
Pole splitting
107
which provides a quadratic equation to determine an appropriate value for CC. Figure 4 shows an example using this
equation. At low values of gain this example amplifier satisfies the pole-ratio condition without compensation (that
is, in Figure 4 the compensation capacitor CC is small at low gain), but as gain increases, a compensation capacitance
rapidly becomes necessary (that is, in Figure 4 the compensation capacitor CC increases rapidly with gain) because
the necessary pole ratio increases with gain. For still larger gain, the necessary CC drops with increasing gain
because the Miller amplification of CC, which increases with gain (see the Miller equation ), allows a smaller value
for CC.
To provide more safety margin for design uncertainties, often Av is increased to two or three times Av on the right
side of this equation.[14] See Sansen[4] or Huijsing[10] and article on step response.
Slew rate
The above is a small-signal analysis. However, when large signals are used, the need to charge and discharge the
compensation capacitor adversely affects the amplifier slew rate; in particular, the response to an input ramp signal is
limited by the need to charge CC.
References and notes
[1] That is, the rise time is selected to be the fastest possible consistent with low overshoot and ringing.
[2] C. Toumazu, Moschytz GS & Gilbert B (Editors) (2007). Trade-offs in analog circuit design: the designer's companion (http:/ / books.
google. com/ books?id=VoBIOvirkiMC& pg=PA272& lpg=PA272& dq="pole+ splitting"& source=web& ots=MC083mOWhv&
sig=duZQKaGECaAH80qDj-YNMdRd8nA). New York/Berlin/Dordrecht: Springer. pp. 272–275. ISBN 1402070373. .
[3] Marc T. Thompson (2006). Intuitive analog circuit design: a problem-solving approach using design case studies (http:/ / books. google.
com/ books?id=1Tyzjmf0DI8C& pg=PA200& dq=pole+ splitting+ analog+ amplifier& lr=& as_brr=0&
sig=gmvG9dtlK48hcqpvf3NwwqcF2Hk). Amsterdam: Elsevier Newnes. pp. 200. ISBN 0750677864. .
[4] Wally M. C. Sansen (2006). Analog design essentials (http:/ / worldcat. org/ isbn/ 0-387-25746-2). New York; Berlin: Springer. pp. §097, p.
266 et seq. ISBN 0-387-25746-2. .
[5] Although this example appears very specific, the associated mathematical analysis is very much used in circuit design.
[6] The sum of the time constants is the coefficient of the term linear in jω and the product of the time constants is the coefficient of the quadratic
term in (jω)2.
[7] The expression for
differs a little from ( CM+Ci ) ( RA // Ri ) as found initially for f1, but the difference is minor assuming the load
capacitance is not so large that it controls the low frequency response instead of the Miller capacitance.
[8] As an aside, the higher the high-frequency pole is made in frequency, the more likely it becomes for a real amplifier that other poles (not
considered in this analysis) play a part.
[9] A.S. Sedra and K.C. Smith (2004). Microelectronic circuits (http:/ / worldcat. org/ isbn/ 0-19-514251-9) (Fifth Edition ed.). New York:
Oxford University Press. pp. 849 and Example 8.6, p. 853. ISBN 0-19-514251-9. .
[10] Huijsing, Johan H. (2001). Operational amplifiers: theory and design (http:/ / books. google. com/ books?id=tiuV_agzk_EC& pg=PA102&
dq=isbn:0792372840& sig=d-oEw_n992coA6bU0h6gkoJzoUo#PPA206,M1). Boston, MA: Kluwer Academic. pp. §6.2, pp.205–206 and
Figure 6.2.1. ISBN 0-7923-7284-0. .
[11] Feucht, Dennis: Two-pole compensation (http:/ / www. analogzone. com/ col_0719. pdf)
[12] Self, Douglas (2006). Audio power amplifier design handbook (http:/ / books. google. com/ books?id=BRQZppvawWwC& pg=PA191&
lpg=PA191& dq="two+ pole+ compensation"& source=web& ots=qsxRG-z1Xl& sig=41uVzeYZW3vi3BndJORUNHNZqPY#PPA191,M1).
Oxford: Newnes. pp. 191–193. ISBN 0750680725. .
[13] That is, the frequency is plotted in powers of ten, as 1, 10, 102 etc.
[14] A factor of two results in the maximally flat or Butterworth design for a two-pole amplifier. However, real amplifiers have more than two
poles, and a factor greater than two often is necessary.
Pole splitting
108
External links
• Bode Plots (http://en.wikibooks.org/wiki/Circuit_Theory/Bode_Plots) in the Circuit Theory Wikibook
• Bode Plots (http://en.wikibooks.org/wiki/Control_Systems/Bode_Plots) in the Control Systems Wikibook
Damping ratio
In engineering, the damping ratio is a dimensionless measure describing how
oscillations in a system decay after a disturbance. Many systems exhibit oscillatory
behavior when they are disturbed from their position of static equilibrium. A mass
suspended from a spring, for example, might, if pulled and released, bounce up and
down. On each bounce, the system is "trying" to return to its equilibrium position, but
overshoots it. Sometimes losses (eg frictional) damp the system and can cause the
oscillations to gradually decay in amplitude towards zero. The damping ratio is a
measure of describing how rapidly the oscillations decay from one bounce to the next.
The behaviour of oscillating systems is often of interest in a diverse range of
disciplines that include control engineering, mechanical engineering, structural
engineering and electrical engineering. The physical quantity that is oscillating varies
greatly, and could be the swaying of a tall building in the wind, or the speed of an
electric motor, but a normalised, or non-dimensionalised approach can be convenient
in describing common aspects of behavior.
Oscillation modes
• Where the spring–mass system is completely lossless, the mass would oscillate
indefinitely, with each bounce of equal height to the last. This hypothetical case is
called undamped.
• If the system contained high losses, for example if the spring–mass experiment
were conducted in a viscous fluid, the mass could slowly return to its rest position
without ever overshooting. This case is called overdamped.
• Commonly, the mass tends to overshoot its starting position, and then return,
overshooting again. With each overshoot, some energy in the system is dissipated,
and the oscillations die towards zero. This case is called underdamped.
Underdamped spring–mass
system with ζ<1
• Between the overdamped and underdamped cases, there exists a certain level of damping at which the system will
just fail to overshoot and will not make a single oscillation. The key difference between critical damping and
overdamping is that, in critical damping, the system returns to equilibrium in the minimum amount of time. This
case is called critical damping.
Damping ratio
109
Definition
The damping ratio is a parameter, usually
denoted by ζ (zeta),[1] that characterizes the
frequency response of a second order
ordinary differential equation. It is
particularly important in the study of control
theory. It is also important in the harmonic
oscillator.
The damping ratio provides a mathematical
means of expressing the level of damping in
a system relative to critical damping. For a
damped harmonic oscillator with mass m,
damping coefficient c, and spring constant k,
it can be defined as the ratio of the damping
coefficient in the system's differential
equation to the critical damping coefficient:
The effect of varying damping ratio on a second-order system.
where the system differential equation is
and the corresponding critical damping coefficient is
The damping ratio is dimensionless, being the ratio of two coefficients of identical units.
Derivation
Using the natural frequency of the simple harmonic oscillator
and the definition of the damping ratio
above, we can rewrite this as:
This equation can be solved with the ansatz
where C and s are both complex constants. That ansatz assumes a solution that is oscillatory and/or decaying
exponentially. Using it in the ODE gives a condition on the frequency of the damped oscillations,
• Overdamped:If s is a real number, then the solution is simply a decaying exponential with no oscillation. This
case occurs for
, and is referred to as overdamped.
• Underdamped:If s is a complex number, then the solution is a decaying exponential combined with an
oscillatory portion that looks like
. This case occurs for
, and is referred to as
underdamped. (The case where
case the solution looks like
corresponds to the undamped simple harmonic oscillator, and in that
, as expected.)
Damping ratio
• Critically damped:The case where
110
is the border between the overdamped and underdamped cases, and is
referred to as critically damped. This turns out to be a desirable outcome in many cases where engineering
design of a damped oscillator is required (e.g., a door closing mechanism).
Q factor and decay rate
The factors Q, damping ratio ζ, and exponential decay rate α are related such that[2]
When a second-order system has
(that is, when the system is underdamped), it has two complex conjugate
poles that each have a real part of ; that is, the decay rate parameter represents the rate of exponential decay of
the oscillations. A lower damping ratio implies a lower decay rate, and so very underdamped systems oscillate for
long times.[3] For example, a high quality tuning fork, which has a very low damping ratio, has an oscillation that
lasts a long time, decaying very slowly after being struck by a hammer.
Logarithmic decrement
The damping ratio is also related to the logarithmic decrement
for underdamped vibrations via the relation
This relation is only meaningful for underdamped systems because the logarithmic decrement is defined as the
natural log of the ratio of any two successive amplitudes, and only underdamped systems exhibit oscillation.
References
[1] Alciatore, David G. (2007). Introduction to Mechatronics and Measurement Systems (3rd ed.). McGraw Hill. ISBN 978-0-07-296305-2.
[2] William McC. Siebert. Circuits, Signals, and Systems. MIT Press.
[3] Ming Rao and Haiming Qiu (1993). Process control engineering: a textbook for chemical, mechanical and electrical engineers (http:/ /
books. google. com/ books?id=NOpmEHNRH98C& pg=PA96). CRC Press. p. 96. ISBN 9782881246289. .
Second-order logic
Second-order logic
In logic and mathematics second-order logic is an extension of first-order logic, which itself is an extension of
propositional logic.[1] Second-order logic is in turn extended by higher-order logic and type theory.
First-order logic uses only variables that range over individuals (elements of the domain of discourse); second-order
logic has these variables as well as additional variables that range over sets of individuals. For example, the
second-order sentence
says that for every set P of individuals and every individual x,
either x is in P or it is not (this is the principle of bivalence). Second-order logic also includes variables quantifying
over functions, and other variables as explained in the section Syntax below. Both first-order and second-order logic
use the idea of a domain of discourse (often called simply the "domain" or the "universe"). The domain is a set of
individual elements which can be quantified over.
Expressive power
Second-order logic is more expressive than first-order logic. For example, if the domain is the set of all real
numbers, one can assert in first-order logic the existence of an additive inverse of each real number by writing ∀x ∃y
(x + y = 0) but one needs second-order logic to assert the least-upper-bound property for sets of real numbers, which
states that every bounded, nonempty set of real numbers has a supremum. If the domain is the set of all real numbers,
the following second-order sentence expresses the least upper bound property:
In second-order logic, it is possible to write formal sentences which say "the domain is finite" or "the domain is of
countable cardinality." To say that the domain is finite, use the sentence that says that every surjective function from
the domain to itself is injective. To say that the domain has countable cardinality, use the sentence that says that
there is a bijection between every two infinite subsets of the domain. It follows from the compactness theorem and
the upward Löwenheim–Skolem theorem that it is not possible to characterize finiteness or countability,
respectively, in first-order logic.
Syntax
The syntax of second-order logic tells which expressions are well formed formulas. In addition to the syntax of
first-order logic, second-order logic includes many new sorts (sometimes called types) of variables. These are:
• A sort of variables that range over sets of individuals. If S is a variable of this sort and t is a first-order term then
the expression t ∈ S (also written S(t) or St) is an atomic formula. Sets of individuals can also be viewed as unary
relations on the domain.
• For each natural number k there is a sort of variable that ranges over all k-ary relations on the individuals. If R is
such a k-ary relation variable and t1,..., tk are first-order terms then the expression R(t1,...,tk) is an atomic formula.
• For each natural number k there is a sort of variable that ranges over functions that take k elements of the domain
and return a single element of the domain. If f is such a k-ary function symbol and t1,...,tk are first-order terms then
the expression f(t1,...,tk) is a first-order term.
For each of the sorts of variable just defined, it is permissible to build up formulas by using universal and/or
existential quantifiers. Thus there are many sorts of quantifiers, two for each sort of variable.
A sentence in second-order logic, as in first-order logic, is a well-formed formula with no free variables (of any
sort).
In monadic second-order logic (MSOL), only variables for subsets of the domain are added. The second-order logic
with all the sorts of variables just described is sometimes called full second-order logic to distinguish it from the
monadic version.
111
Second-order logic
Just as in first-order logic, second-order logic may include non-logical symbols in a particular second-order
language. These are restricted, however, in that all terms that they form must be either first-order terms (which can
be substituted for a first-order variable) or second-order terms (which can be substituted for a second-order variable
of an appropriate sort).
Semantics
The semantics of second-order logic establish the meaning of each sentence. Unlike first-order logic, which has only
one standard semantics, there are two different semantics that are commonly used for second-order logic: standard
semantics and Henkin semantics. In each of these semantics, the interpretations of the first-order quantifiers and the
logical connectives are the same as in first-order logic. Only the ranges of quantifiers over second-order variables
differ in the two types of semantics.
In standard semantics, also called full semantics, the quantifiers range over all sets or functions of the appropriate
sort. Thus once the domain of the first-order variables is established, the meaning of the remaining quantifiers is
fixed. It is these semantics that give second-order logic its expressive power, and they will be assumed for the
remainder of this article.
In Henkin semantics, each sort of second-order variable has a particular domain of its own to range over, which may
be a proper subset of all sets or functions of that sort. Leon Henkin (1950) defined these semantics and proved that
Gödel's completeness theorem and compactness theorem, which hold for first-order logic, carry over to second-order
logic with Henkin semantics. This is because Henkin semantics are almost identical to many-sorted first-order
semantics, where additional sorts of variables are added to simulate the new variables of second-order logic.
Second-order logic with Henkin semantics is not more expressive than first-order logic. Henkin semantics are
commonly used in the study of second-order arithmetic.
Deductive systems
A deductive system for a logic is a set of inference rules and logical axioms that determine which sequences of
formulas constitute valid proofs. Several deductive systems can be used for second-order logic, although none can be
complete for the standard semantics (see below). Each of these systems is sound, which means any sentence they can
be used to prove is logically valid in the appropriate semantics.
The weakest deductive system that can be used consists of a standard deductive system for first-order logic (such as
natural deduction) augmented with substitution rules for second-order terms.[2] This deductive system is commonly
used in the study of second-order arithmetic.
The deductive systems considered by Shapiro (1991) and Henkin (1950) add to the augmented first-order deductive
scheme both comprehension axioms and choice axioms. These axioms are sound for standard second-order
semantics. They are sound for Henkin semantics if only Henkin models that satisfy the comprehension and choice
axioms are considered.[3]
Non-reducibility to first-order logic
One might attempt to reduce the second-order theory of the real numbers, with full second-order semantics, to the
first-order theory in the following way. First expand the domain from the set of all real numbers to a two-sorted
domain, with the second sort containing all sets of real numbers. Add a new binary predicate to the language: the
membership relation. Then sentences that were second-order become first-order, with the formerly second-order
quantifiers ranging over the second sort instead. This reduction can be attempted in a one-sorted theory by adding
unary predicates that tell whether an element is a number or a set, and taking the domain to be the union of the set of
real numbers and the power set of the real numbers.
112
Second-order logic
But notice that the domain was asserted to include all sets of real numbers. That requirement cannot be reduced to a
first-order sentence, as the Löwenheim-Skolem theorem shows. That theorem implies that there is some countably
infinite subset of the real numbers, whose members we will call internal numbers, and some countably infinite
collection of sets of internal numbers, whose members we will call "internal sets", such that the domain consisting of
internal numbers and internal sets satisfies exactly the same first-order sentences satisfied as the domain of
real-numbers-and-sets-of-real-numbers. In particular, it satisfies a sort of least-upper-bound axiom that says, in
effect:
Every nonempty internal set that has an internal upper bound has a least internal upper bound.
Countability of the set of all internal numbers (in conjunction with the fact that those form a densely ordered set)
implies that that set does not satisfy the full least-upper-bound axiom. Countability of the set of all internal sets
implies that it is not the set of all subsets of the set of all internal numbers (since Cantor's theorem implies that the
set of all subsets of a countably infinite set is an uncountably infinite set). This construction is closely related to
Skolem's paradox.
Thus the first-order theory of real numbers and sets of real numbers has many models, some of which are countable.
The second-order theory of the real numbers has only one model, however. This follows from the classical theorem
that there is only one Archimedean complete ordered field, along with the fact that all the axioms of an Archimedean
complete ordered field are expressible in second-order logic. This shows that the second-order theory of the real
numbers cannot be reduced to a first-order theory, in the sense that the second-order theory of the real numbers has
only one model but the corresponding first-order theory has many models.
There are more extreme examples showing that second-order logic with standard semantics is more expressive than
first-order logic. There is a finite second-order theory whose only model is the real numbers if the continuum
hypothesis holds and which has no model if the continuum hypothesis does not hold (cf. Shapiro 2000 p. 105). This
theory consists of a finite theory characterizing the real numbers as a complete Archimedean ordered field plus an
axiom saying that the domain is of the first uncountable cardinality. This example illustrates that the question of
whether a sentence in second-order logic is consistent is extremely subtle.
Additional limitations of second order logic are described in the next section.
Metalogical results
It is a corollary of Gödel's incompleteness theorem that there is no deductive system (that is, no notion of
provability) for second-order formulas that simultaneously satisfies these three desired attributes:[4]
• (Soundness) Every provable second-order sentence is universally valid, i.e., true in all domains under standard
semantics.
• (Completeness) Every universally valid second-order formula, under standard semantics, is provable.
• (Effectiveness) There is a proof-checking algorithm that can correctly decide whether a given sequence of
symbols is a valid proof or not.
This corollary is sometimes expressed by saying that second-order logic does not admit a complete proof theory. In
this respect second-order logic with standard semantics differs from first-order logic; Quine (1970, pp. 90–91)
pointed to the lack of a complete proof system as a reason for thinking of second-order logic as not logic, properly
speaking.
As mentioned above, Henkin proved that the standard deductive system for first-order logic is sound, complete, and
effective for second-order logic with Henkin semantics, and the deductive system with comprehension and choice
principles is sound, complete, and effective for Henkin semantics using only models that satisfy these principles.
113
Second-order logic
History and disputed value
Predicate logic was primarily introduced to the mathematical community by C. S. Peirce, who coined the term
second-order logic and whose notation is most similar to the modern form (Putnam 1982). However, today most
students of logic are more familiar with the works of Frege, who actually published his work several years prior to
Peirce but whose works remained in obscurity until Bertrand Russell and Alfred North Whitehead made them
famous. Frege used different variables to distinguish quantification over objects from quantification over properties
and sets; but he did not see himself as doing two different kinds of logic. After the discovery of Russell's paradox it
was realized that something was wrong with his system. Eventually logicians found that restricting Frege's logic in
various ways—to what is now called first-order logic—eliminated this problem: sets and properties cannot be
quantified over in first-order-logic alone. The now-standard hierarchy of orders of logics dates from this time.
It was found that set theory could be formulated as an axiomatized system within the apparatus of first-order logic (at
the cost of several kinds of completeness, but nothing so bad as Russell's paradox), and this was done (see
Zermelo-Fraenkel set theory), as sets are vital for mathematics. Arithmetic, mereology, and a variety of other
powerful logical theories could be formulated axiomatically without appeal to any more logical apparatus than
first-order quantification, and this, along with Gödel and Skolem's adherence to first-order logic, led to a general
decline in work in second (or any higher) order logic.
This rejection was actively advanced by some logicians, most notably W. V. Quine. Quine advanced the view that in
predicate-language sentences like Fx the "x" is to be thought of as a variable or name denoting an object and hence
can be quantified over, as in "For all things, it is the case that . . ." but the "F" is to be thought of as an abbreviation
for an incomplete sentence, not the name of an object (not even of an abstract object like a property). For example, it
might mean " . . . is a dog." But it makes no sense to think we can quantify over something like this. (Such a position
is quite consistent with Frege's own arguments on the concept-object distinction). So to use a predicate as a variable
is to have it occupy the place of a name which only individual variables should occupy. This reasoning has been
rejected by Boolos.
In recent years second-order logic has made something of a recovery, buoyed by George Boolos' interpretation of
second-order quantification as plural quantification over the same domain of objects as first-order quantification
(Boolos 1984). Boolos furthermore points to the claimed nonfirstorderizability of sentences such as "Some critics
admire only each other" and "Some of Fianchetto's men went into the warehouse unaccompanied by anyone else"
which he argues can only be expressed by the full force of second-order quantification. However, generalized
quantification and partially ordered, or branching, quantification may suffice to express a certain class of purportedly
nonfirstorderizable sentences as well and it does not appeal to second-order quantification.
Applications to complexity
The expressive power of various forms of second-order logic on finite structures is intimately tied to computational
complexity theory. The field of descriptive complexity studies which computational complexity classes can be
characterized by the power of the logic needed to express languages (sets of finite strings) in them. A string
w = w1···wn in a finite alphabet A can be represented by a finite structure with domain D = {1,...,n}, unary predicates
Pa for each a ∈ A, satisfied by those indices i such that wi = a, and additional predicates which serve to uniquely
identify which index is which (typically, one takes the graph of the successor function on D or the order relation <,
possibly with other arithmetic predicates). Conversely, the table of any finite structure can be encoded by a finite
string.
With this identification, we have the following characterizations of variants of second-order logic over finite
structures:
• NP is the set of languages definable by existential, second-order formulas (Fagin's theorem, 1974).
• co-NP is the set of languages definable by universal, second-order formulas.
114
Second-order logic
• PH is the set of languages definable by second-order formulas.
• PSPACE is the set of languages definable by second-order formulas with an added transitive closure operator.
• EXPTIME is the set of languages definable by second-order formulas with an added least fixed point operator.
Relationships among these classes directly impact the relative expressiveness of the logics over finite structures; for
example, if PH = PSPACE, then adding a transitive closure operator to second-order logic would not make it any
more expressive over finite structures.
Notes
[1]
[2]
[3]
[4]
Shapiro (1991) and Hinman (2005) give complete introductions to the subject, with full definitions.
Such a system is used without comment by Hinman (2005).
These are the models originally studied by Henkin (1950).
The proof of this corollary is that a sound, complete, and effective deduction system for standard semantics could be used to produce a
recursively enumerable completion of Peano arithmetic, which Gödel's theorem shows cannot exist.
References
• Andrews, Peter (2002). An Introduction to Mathematical Logic and Type Theory: To Truth Through Proof (2nd
ed.). Kluwer Academic Publishers.
• Boolos, George (1984). "To Be Is To Be a Value of a Variable (or to Be Some Values of Some Variables)".
Journal of Philosophy 81 (8): 430–50. doi:10.2307/2026308. JSTOR 2026308.. Reprinted in Boolos, Logic, Logic
and Logic, 1998.
• Henkin, L. (1950). "Completeness in the theory of types". Journal of Symbolic Logic 15 (2): 81–91.
doi:10.2307/2266967. JSTOR 2266967.
• Hinman, P. (2005). Fundamentals of Mathematical Logic. A K Peters. ISBN 1-56881-262-0.
• Putnam, Hilary (1982). "Peirce the Logician". Historia Mathematica 9 (3): 290–301.
doi:10.1016/0315-0860(82)90123-9.. Reprinted in Putnam, Hilary (1990), Realism with a Human Face, Harvard
University Press, pp. 252–260.
• W.V. Quine (1970). Philosophy of Logic. Prentice-Hall.
• Rossberg, M. (2004). "First-Order Logic, Second-Order Logic, and Completeness" (http://homepages.uconn.
edu/~mar08022/papers/RossbergCompleteness.pdf). In V. Hendricks et al., eds.. First-order logic revisited.
Berlin: Logos-Verlag.
• Shapiro, S. (2000). Foundations without Foundationalism: A Case for Second-order Logic. Oxford University
Press. ISBN 0-19-825029-0.
• Vaananen, J. (2001). "Second-Order Logic and Foundations of Mathematics" (http://www.math.ucla.edu/~asl/
bsl/0704/0704-003.ps). Bulletin of Symbolic Logic 7 (4): 504–520. doi:10.2307/2687796. JSTOR 2687796.
115
Impulse response
Impulse response
In signal processing, the impulse response,
or impulse response function (IRF), of a
dynamic system is its output when presented
with a brief input signal, called an impulse.
More generally, an impulse response refers
to the reaction of any dynamic system in
response to some external change. In both
cases, the impulse response describes the
reaction of the system as a function of time
(or possibly as a function of some other
independent variable that parameterizes the
dynamic behavior of the system).
The Impulse response from a simple audio system. Showing the original impulse,
For example, the dynamic system might be a
the response after high frequency boosting, and the response after low frequency
planetary system in orbit around a star; the
boosting.
external influence in this case might be
another massive object arriving from
elsewhere in the galaxy; the impulse response is the change in the motion of the planetary system caused by
interaction with the new object.
In all these cases, the 'dynamic system' and its 'impulse response' may refer to actual physical objects, or to a
mathematical system of equations describing these objects.
Mathematical considerations
Mathematically, how the impulse is described depends on whether the system is modeled in discrete or continuous
time. The impulse can be modeled as a Dirac delta function for continuous-time systems, or as the Kronecker delta
for discrete-time systems. The Dirac delta represents the limiting case of a pulse made very short in time while
maintaining its area or integral (thus giving an infinitely high peak). While this is impossible in any real system, it is
a useful idealisation. In Fourier analysis theory, such an impulse comprises equal portions of all possible excitation
frequencies, which makes it a convenient test probe.
Any system in a large class known as linear, time-invariant (LTI) is completely characterized by its impulse
response. That is, for any input function, the output function can be calculated in terms of the input and the impulse
response. (See LTI system theory.) The impulse response of a linear transformation is the image of Dirac's delta
function under the transformation, analogous to the fundamental solution of a partial differential operator.
The Laplace transform of the impulse response function is known as the transfer function. It is usually easier to
analyze systems using transfer functions as opposed to impulse response functions. The Laplace transform of a
system's output may be determined by the multiplication of the transfer function with the input function in the
complex plane, also known as the frequency domain. An inverse Laplace transform of this result will yield the
output function in the time domain.
To determine an output function directly in the time domain requires the convolution of the input function with the
impulse response function. This requires the use of integrals, and is usually more difficult than simply multiplying
two functions in the frequency domain.
The impulse response, considered as a Green's function, can be thought of as an "influence function:" how a point of
input influences output.
116
Impulse response
Practical applications
In practical systems, it is not possible to produce a perfect impulse to serve as input for testing; therefore, a brief
pulse is sometimes used as an approximation of an impulse. Provided that the pulse is short enough compared to the
impulse response, the result will be close to the true, theoretical, impulse response. In many systems, however,
driving with a very short strong pulse may drive the system into a nonlinear regime, so instead the system is driven
with a pseudo-random sequence, and the impulse response is computed from the input and output signals.[1]
Loudspeakers
An application that demonstrates this idea was the development of impulse response loudspeaker testing in the
1970s. Loudspeakers suffer from phase inaccuracy, a defect unlike other measured properties such as frequency
response. Phase inaccuracy is caused by (slightly) delayed frequencies/octaves that are mainly the result of passive
cross overs (especially higher order filters) but are also caused by resonance, energy storage in the cone, the internal
volume, or the enclosure panels vibrating. Measuring the impulse response, which is a direct plot of this
"time-smearing," provided a tool for use in reducing resonances by the use of improved materials for cones and
enclosures, as well as changes to the speaker crossover. The need to limit input amplitude to maintain the linearity of
the system led to the use of inputs such as pseudo-random maximum length sequences, and to the use of computer
processing to derive the impulse response.[2]
Digital filtering
Impulse response is a very important concept in the design of digital filters for audio processing, because digital
filters can differ from 'real' filters in often having a pre-echo, which the ear is not accustomed to.
Electronic processing
Impulse response analysis is a major facet of radar, ultrasound imaging, and many areas of digital signal processing.
An interesting example would be broadband internet connections. DSL/Broadband services use adaptive equalisation
techniques to help compensate for signal distortion and interference introduced by the copper phone lines used to
deliver the service.
Control systems
In control theory the impulse response is the response of a system to a Dirac delta input. This proves useful in the
analysis of dynamic systems: the Laplace transform of the delta function is 1, so the impulse response is equivalent
to the inverse Laplace transform of the system's transfer function.
Acoustic and audio applications
In acoustic and audio applications, impulse responses enable the acoustic characteristics of a location, such as a
concert hall, to be captured. Various commercial packages are available containing impulse responses from specific
locations, ranging from small rooms to large concert halls. These impulse responses can then be utilized in
convolution reverb applications to enable the acoustic characteristics of a particular location to be applied to target
audio.[3]
117
Impulse response
118
Economics
In economics, and especially in contemporary macroeconomic modeling, impulse response functions describe how
the economy reacts over time to exogenous impulses, which economists usually call 'shocks', and are often modeled
in the context of a vector autoregression. Impulses that are often treated as exogenous from a macroeconomic point
of view include changes in government spending, tax rates, and other fiscal policy parameters; changes in the
monetary base or other monetary policy parameters; changes in productivity or other technological parameters; and
changes in preferences, such as the degree of impatience. Impulse response functions describe the reaction of
endogenous macroeconomic variables such as output, consumption, investment, and employment at the time of the
shock and over subsequent points in time.[4] [5]
References
[1] F. Alton Everest (2000). Master Handbook of Acoustics (http:/ / books. google. com/ books?id=sgwg1Vwm9VUC& pg=RA1-PA510&
dq="impulse+ response"+ loudspeaker+ testing) (Fourth ed. ed.). McGraw-Hill Professional. ISBN 0071360972. .
[2] Monitor: Proceedings of the Institution of Radio and Electronics Engineers Australia (1976) (http:/ / books. google. com/
books?id=q6w7AAAAMAAJ& q="impulse+ response"+ "loudspeaker+ testing"& dq="impulse+ response"+ "loudspeaker+ testing"&
pgis=1)
[3] http:/ / www. acoustics. hut. fi/ projects/ poririrs/ the Concert Hall Impulse Responses from Pori, Finland
[4] Helmut Lütkepohl (2008), 'Impulse response function'. The New Palgrave Dictionary of Economics, 2nd. ed.
[5] James Hamilton (1994), Time Series Analysis, Chapter 1, page 5. Princeton University Press.
LTI system theory
Linear time-invariant system theory, commonly known as LTI system theory, comes from applied mathematics
and has direct applications in NMR spectroscopy, seismology, circuits, signal processing, control theory, and other
technical areas. It investigates the response of a linear and time-invariant system to an arbitrary input signal.
Trajectories of these systems are commonly measured and tracked as they move through time (e.g., an acoustic
waveform), but in applications like image processing and field theory, the LTI systems also have trajectories in
spatial dimensions. Thus these systems are also called linear translation-invariant to give the theory the most
general reach. In the case of generic discrete-time (i.e., sampled) systems, linear shift-invariant is the corresponding
term. A good example of LTI systems are electrical circuits that can be made up of resistors, capacitors and
inductors.[1]
Overview
The defining properties of any LTI system are linearity and time invariance.
• Linearity means that the relationship between the input and the output of the system is a linear map: If input
produces response
and input
produces response
then the scaled and summed input
produces the scaled and summed response
where
and
scalars. It follows that this can be extended to an arbitrary number of terms, and so for real numbers
,
Input produces output In particular,
Input produces output (Eq.1)
are real
LTI system theory
119
where
and
are scalars and inputs that vary over a continuum indexed by . Thus if an input function
can be represented by a continuum of input functions, combined "linearly", as shown, then the corresponding
output function can be represented by the corresponding continuum of output functions, scaled and summed in
the same way.
• Time invariance means that whether we apply an input to the system now or T seconds from now, the output will
be identical except for a time delay of the T seconds. That is, if the output due to input
is
, then the
output due to input
is
. Hence, the system is time invariant because the output does not
depend on the particular time the input is applied.
The fundamental result in LTI system theory is that any LTI system can be characterized entirely by a single
function called the system's impulse response. The output of the system is simply the convolution of the input to the
system with the system's impulse response. This method of analysis is often called the time domain point-of-view.
The same result is true of discrete-time linear shift-invariant systems in which signals are discrete-time samples, and
convolution is defined on sequences.
Equivalently, any LTI system can be
characterized in the frequency domain
by the system's transfer function, which
is the Laplace transform of the system's
impulse response (or Z transform in the
case of discrete-time systems). As a
result of the properties of these
transforms, the output of the system in
the frequency domain is the product of
the transfer function and the transform
of the input. In other words,
convolution in the time domain is
equivalent to multiplication in the
frequency domain.
For all LTI systems, the eigenfunctions,
Relationship between the time domain and the frequency domain
and the basis functions of the
transforms, are complex exponentials.
This is, if the input to a system is the complex waveform
for some complex amplitude
and complex
frequency
, the output will be some complex constant times the input, say
. The ratio
is the transfer function at frequency
for some new complex amplitude
.
Because sinusoids are a sum of complex exponentials with complex-conjugate frequencies, if the input to the system
is a sinusoid, then the output of the system will also be a sinusoid, perhaps with a different amplitude and a different
phase, but always with the same frequency. LTI systems cannot produce frequency components that are not in the
input.
LTI system theory is good at describing many important systems. Most LTI systems are considered "easy" to
analyze, at least compared to the time-varying and/or nonlinear case. Any system that can be modeled as a linear
homogeneous differential equation with constant coefficients is an LTI system. Examples of such systems are
electrical circuits made up of resistors, inductors, and capacitors (RLC circuits). Ideal spring–mass–damper systems
are also LTI systems, and are mathematically equivalent to RLC circuits.
Most LTI system concepts are similar between the continuous-time and discrete-time (linear shift-invariant) cases. In
image processing, the time variable is replaced with two space variables, and the notion of time invariance is
replaced by two-dimensional shift invariance. When analyzing filter banks and MIMO systems, it is often useful to
LTI system theory
120
consider vectors of signals.
A linear system that is not time-invariant can be solved using other approaches such as the Green function method.
The same method must be used when the initial conditions of the problem are not null.
Continuous-time systems
Impulse response and convolution
The behavior of a linear, continuous-time, time-invariant system with input signal x(t) and output signal y(t) is
described by the convolution integral,[2] :
(using commutativity)
where
is the system's response to an impulse: of the input function
The weighting function is
is therefore proportional to a weighted average
simply shifted by amount
weighting function emphasizes different parts of the input function. When
depends only on values of prior to time
As
is zero for all negative
with variable
and the system is said to be causal.
To understand why the convolution produces the output of an LTI system, let the notation
function
changes, the
and constant
And let the shorter notation
continuous-time system transforms an input function,
into an output function,
represent
represent the
Then a
And in general, every value
of the output can depend on every value of the input. This concept is represented by:
where
is the transformation operator for time
that occurred near time
system is uninteresting.
For a linear system,
In a typical system,
Unless the transform itself changes with
depends most heavily on the values of
the output function is just constant, and the
must satisfy Eq.1:
(Eq.2)
And the time-invariance requirement is:
(Eq.3)
In this notation, we can write the impulse response as Similarly:
(using Eq.3)
Substituting this result into the convolution integral:
LTI system theory
121
which has the form of the right side of Eq.2 for the case
and
Eq.2 then allows this continuation:
In summary, the input function,
can be represented by a continuum of time-shifted impulse functions,
combined "linearly", as shown at Eq.1. The system's linearity property allows the system's response to be
represented by the corresponding continuum of impulse responses, combined in the same way. And the
time-invariance property allows that combination to be represented by the convolution integral.
The mathematical operations above have a simple graphical simulation.[3]
Exponentials as eigenfunctions
An eigenfunction is a function for which the output of the operator is a scaled version of the same function. That is,
,
where f is the eigenfunction and
The exponential functions
is the eigenvalue, a constant.
, where
, are eigenfunctions of a linear, time-invariant operator. A
simple proof illustrates this concept. Suppose the input is
response
. The output of the system with impulse
is then
which, by the commutative property of convolution, is equivalent to
where the scalar
is dependent only on the parameter s.
So the system's response is a scaled version of the input. In particular, for any
product of the input
and the constant
corresponding eigenvalue is
.
. Hence,
, the system output is the
is an eigenfunction of an LTI system, and the
LTI system theory
122
Fourier and Laplace transforms
The eigenfunction property of exponentials is very useful for both analysis and insight into LTI systems. The
Laplace transform
is exactly the way to get the eigenvalues from the impulse response. Of particular interest are pure sinusoids (i.e.,
exponential functions of the form
where
and
). These are generally called
complex exponentials even though the argument is purely imaginary. The Fourier transform
gives the eigenvalues for pure complex sinusoids. Both of
and
are called the system function, system
response, or transfer function.
The Laplace transform is usually used in the context of one-sided signals, i.e. signals that are zero for all values of t
less than some value. Usually, this "start time" is set to zero, for convenience and without loss of generality, with the
transform integral being taken from zero to infinity (the transform shown above with lower limit of integration of
negative infinity is formally known as the bilateral Laplace transform).
The Fourier transform is used for analyzing systems that process signals that are infinite in extent, such as modulated
sinusoids, even though it cannot be directly applied to input and output signals that are not square integrable. The
Laplace transform actually works directly for these signals if they are zero before a start time, even if they are not
square integrable, for stable systems. The Fourier transform is often applied to spectra of infinite signals via the
Wiener–Khinchin theorem even when Fourier transforms of the signals do not exist.
Due to the convolution property of both of these transforms, the convolution that gives the output of the system can
be transformed to a multiplication in the transform domain, given signals for which the transforms exist
Not only is it often easier to do the transforms, multiplication, and inverse transform than the original convolution,
but one can also gain insight into the behavior of the system from the system response. One can look at the modulus
of the system function |H(s)| to see whether the input
is passed (let through) the system or rejected or
attenuated by the system (not let through).
Examples
• A simple example of an LTI operator is the derivative.
•
•
(i.e., it is linear)
(i.e., it is time invariant)
When the Laplace transform of the derivative is taken, it transforms to a simple multiplication by the Laplace
variable s.
That the derivative has such a simple Laplace transform partly explains the utility of the transform.
• Another simple LTI operator is an averaging operator
By the linearity of integration,
LTI system theory
123
it is linear. Additionally, because
it is time invariant. In fact,
can be written as a convolution with the boxcar function
. That is,
where the boxcar function
Important system properties
Some of the most important properties of a system are causality and stability. Causality is a necessity if the
independent variable is time, but not all systems have time as an independent variable. For example, a system that
processes still images does not need to be causal. Non-causal systems can be built and can be useful in many
circumstances. Even non-real systems can be built and are very useful in many contexts.
Causality
A system is causal if the output depends only on present and past inputs. A necessary and sufficient condition for
causality is
where
is the impulse response. It is not possible in general to determine causality from the Laplace transform,
because the inverse transform is not unique. When a region of convergence is specified, then causality can be
determined.
Stability
A system is bounded-input, bounded-output stable (BIBO stable) if, for every bounded input, the output is finite.
Mathematically, if every input satisfying
leads to an output satisfying
(that is, a finite maximum absolute value of
implies a finite maximum absolute value of
system is stable. A necessary and sufficient condition is that
norm):
), then the
1
, the impulse response, is in L (has a finite L1
LTI system theory
124
In the frequency domain, the region of convergence must contain the imaginary axis
.
As an example, the ideal low-pass filter with impulse response equal to a sinc function is not BIBO stable, because
the sinc function does not have a finite L1 norm. Thus, for some bounded input, the output of the ideal low-pass filter
is unbounded. In particular, if the input is zero for
and equal to a sinusoid at the cut-off frequency for
, then the output will be unbounded for all times other than the zero crossings.
Discrete-time systems
Almost everything in continuous-time systems has a counterpart in discrete-time systems.
Discrete-time systems from continuous-time systems
In many contexts, a discrete time (DT) system is really part of a larger continuous time (CT) system. For example, a
digital recording system takes an analog sound, digitizes it, possibly processes the digital signals, and plays back an
analog sound for people to listen to.
Formally, the DT signals studied are almost always uniformly sampled versions of CT signals. If
is a CT
signal, then an analog to digital converter will transform it to the DT signal:
where T is the sampling period. It is very important to limit the range of frequencies in the input signal for faithful
representation in the DT signal, since then the sampling theorem guarantees that no information about the CT signal
is lost. A DT signal can only contain a frequency range of
; other frequencies are aliased to the same range.
Impulse response and convolution
Let
represent the sequence
And let the shorter notation
.
represent
A discrete system transforms an input sequence,
into an output sequence,
In general, every element of
the output can depend on every element of the input. Representing the transformation operator by
, we can write:
Note that unless the transform itself changes with n, the output sequence is just constant, and the system is
uninteresting. (Thus the subscript, n.) In a typical system, y[n] depends most heavily on the elements of x whose
indices are near n.
For the special case of the Kronecker delta function,
For a linear system,
the output sequence is the impulse response:
must satisfy:
(Eq.4)
And the time-invariance requirement is:
(Eq.5)
In such a system, the impulse response,
characterizes the system completely. I.e., for any input sequence, the
output sequence can be calculated in terms of the input and the impulse response. To see how that is done, consider
the identity:
LTI system theory
125
which expresses
in terms of a sum of weighted delta functions.
Therefore:
where we have invoked Eq.4 for the case
and
And because of Eq.5, we may write:
Therefore:
(commutativity)
which is the familiar discrete convolution formula. The operator
can therefore be interpreted as proportional to a
weighted average of the function x[k]. The weighting function is h[-k], simply shifted by amount n. As n changes,
the weighting function emphasizes different parts of the input function. Equivalently, the system's response to an
impulse at n=0 is a "time" reversed copy of the unshifted weighting function. When h[k] is zero for all negative k,
the system is said to be causal.
Exponentials as eigenfunctions
An eigenfunction is a function for which the output of the operator is the same function, just scaled by some amount.
In symbols,
,
where f is the eigenfunction and
is the eigenvalue, a constant.
The exponential functions
, where
is the sampling interval, and
Suppose the input is
, are eigenfunctions of a linear, time-invariant operator.
. A simple proof illustrates this concept.
. The output of the system with impulse response
which is equivalent to the following by the commutative property of convolution
where
is dependent only on the parameter z.
is then
LTI system theory
So
126
is an eigenfunction of an LTI system because the system response is the same as the input times the constant
.
Z and discrete-time Fourier transforms
The eigenfunction property of exponentials is very useful for both analysis and insight into LTI systems. The Z
transform
is exactly the way to get the eigenvalues from the impulse response. Of particular interest are pure sinusoids, i.e.
exponentials of the form
, where
. These can also be written as
with
. These are
generally called complex exponentials even though the argument is purely imaginary. The Discrete-time Fourier
transform (DTFT)
gives the eigenvalues of pure sinusoids. Both of
and
are
called the system function, system response, or transfer function'.
The Z transform is usually used in the context of one-sided signals, i.e. signals that are zero for all values of t less
than some value. Usually, this "start time" is set to zero, for convenience and without loss of generality. The Fourier
transform is used for analyzing signals that are infinite in extent.
Due to the convolution property of both of these transforms, the convolution that gives the output of the system can
be transformed to a multiplication in the transform domain. That is,
Just as with the Laplace transform transfer function in continuous-time system analysis, the Z transform makes it
easier to analyze systems and gain insight into their behavior. One can look at the modulus of the system function
|H(z)| to see whether the input
is passed (let through) by the system, or rejected or attenuated by the system (not
let through).
Examples
• A simple example of an LTI operator is the delay operator
•
•
.
(i.e., it is linear)
(i.e., it is time invariant)
The Z transform of the delay operator is a simple multiplication by z-1. That is,
• Another simple LTI operator is the averaging operator
.
Because of the linearity of sums,
and so it is linear. Because,
LTI system theory
127
it is also time invariant.
Important system properties
The input-output characteristics of discrete-time LTI system are completely described by its impulse response
.
Some of the most important properties of a system are causality and stability. Unlike CT systems, non-causal DT
systems can be realized. It is trivial to make an acausal FIR system causal by adding delays. It is even possible to
make acausal IIR systems.[4] Non-stable systems can be built and can be useful in many circumstances. Even
non-real systems can be built and are very useful in many contexts.
Causality
A discrete-time LTI system is causal if the current value of the output depends on only the current value and past
values of the input.,[5] A necessary and sufficient condition for causality is
where
is the impulse response. It is not possible in general to determine causality from the Z transform,
because the inverse transform is not unique. When a region of convergence is specified, then causality can be
determined.
Stability
A system is bounded input, bounded output stable (BIBO stable) if, for every bounded input, the output is finite.
Mathematically, if
implies that
(that is, if bounded input implies bounded output, in the sense that the maximum absolute values of
are finite), then the system is stable. A necessary and sufficient condition is that
and
, the impulse response,
satisfies
In the frequency domain, the region of convergence must contain the unit circle (i.e., the locus satisfying
for complex z).
LTI system theory
Notes
[1]
[2]
[3]
[4]
[5]
Hespanha 2009, p. 78.
Crutchfield[web]
Crutchfield
Vaidyanathan,1995
Phillips 2007, p. 508.
References
• Phillips, C.l., Parr, J.M., & Riskin, E.A (2007). Signals, systems and Transforms. Prentice Hall.
ISBN 0-13-041207-4.
• Hespanha,J.P. (2009). Linear System Theory. Princeton university press. ISBN 0691140219.
• Crutchfield, Steve (October 12, 2010), "The Joy of Convolution" (http://www.jhu.edu/signals/convolve/
index.html), Johns Hopkins University, retrieved November 21, 2010
• Vaidyanathan, P. P.; Chen, T. (May 1995). "Role of anticausal inverses in multirate filter banks — Part I: system
theoretic fundamentals". IEEE Trans. Signal Proc. 43 (5): 1090. doi:10.1109/78.382395.
Further reading
• Porat, Boaz (1997). A Course in Digital Signal Processing. New York: John Wiley. ISBN 9780471149613.
• Vaidyanathan, P. P.; Chen, T. (May 1995). "Role of anticausal inverses in multirate filter banks — Part I: system
theoretic fundamentals". IEEE Trans. Signal Proc. 43 (5): 1090. doi:10.1109/78.382395.
External links
• ECE 209: Review of Circuits as LTI Systems (http://www.tedpavlic.com/teaching/osu/ece209/support/
circuits_sys_review.pdf) – Short primer on the mathematical analysis of (electrical) LTI systems.
• ECE 209: Sources of Phase Shift (http://www.tedpavlic.com/teaching/osu/ece209/lab3_opamp_FO/
lab3_opamp_FO_phase_shift.pdf) – Gives an intuitive explanation of the source of phase shift in two common
electrical LTI systems.
• JHU 520.214 Signals and Systems course notes (http://www.ece.jhu.edu/~cooper/courses/214/
signalsandsystemsnotes.pdf). An encapsulated course on LTI system theory. Adequate for self teaching.
128
RouthHurwitz stability criterion
129
Routh–Hurwitz stability criterion
The Routh–Hurwitz stability criterion is a necessary and sufficient method to establish the stability of a
single-input, single-output (SISO), linear time invariant (LTI) control system. More generally, given a polynomial,
some calculations using only the coefficients of that polynomial can lead to the conclusion that it is not stable. For
the discrete case, see the Jury test equivalent.
The criterion establishes a systematic way to show that the linearized equations of motion of a system have only
stable solutions exp(pt), that is where all p have negative real parts. It can be performed using either polynomial
divisions or determinant calculus.
The criterion is derived through the use of the Euclidean algorithm and Sturm's theorem in evaluating Cauchy
indices.
Using Euclid's algorithm
The criterion is related to Routh–Hurwitz theorem. Indeed, from the statement of that theorem, we have
where:
• p is the number of roots of the polynomial ƒ(z) with negative Real Part;
• q is the number of roots of the polynomial ƒ(z) with positive Real Part (let us remind ourselves that ƒ is supposed
to have no roots lying on the imaginary line);
• w(x) is the number of variations of the generalized Sturm chain obtained from
and
(by successive
Euclidean divisions) where
for a real y.
By the fundamental theorem of algebra, each polynomial of degree n must have n roots in the complex plane (i.e.,
for an ƒ with no roots on the imaginary line, p + q = n). Thus, we have the condition that ƒ is a (Hurwitz) stable
polynomial if and only if p − q = n (the proof is given below). Using the Routh–Hurwitz theorem, we can replace the
condition on p and q by a condition on the generalized Sturm chain, which will give in turn a condition on the
coefficients of ƒ.
Using matrices
Let f(z) be a complex polynomial. The process is as follows:
1.
2.
3.
4.
5.
Compute the polynomials
and
such that
where y is a real number.
Compute the Sylvester matrix associated to
and
.
Rearrange each row in such a way that an odd row and the following one have the same number of leading zeros.
Compute each principal minor of that matrix.
If at least one of the minors is negative (or zero), then the polynomial f is not stable.
Example
• Let
(for the sake of simplicity we take real coefficients) where
(to avoid a root
in zero so that we can use the Routh–Hurwitz theorem). First, we have to calculate the real polynomials
and
:
Next, we divide those polynomials to obtain the generalized Sturm chain:
•
•
yields
yields
and the Euclidean division stops.
Notice that we had to suppose b different from zero in the first division. The generalized Sturm chain is in this case
. Putting
, the sign of
is the opposite sign of
RouthHurwitz stability criterion
130
a and the sign of by is the sign of b. When we put
, the sign of the first element of the chain is again the opposite sign
of a and the sign of by is the opposite sign of b. Finally, -c has always the opposite sign of c.
Suppose now that f is Hurwitz-stable. This means that
properties of the function w, this is the same as
(the degree of f). By the
and
. Thus, a, b and c must have the
same sign. We have thus found the necessary condition of stability for polynomials of degree 2.
Routh–Hurwitz criterion for second, third, and fourth-order polynomials
• For a second-order polynomial,
, all the roots are in the left half plane (and the
system with characteristic equation
• For a third-order polynomial
is stable) if all the coefficients satisfy
.
, all the coefficients must satisfy
, and
• For a fourth-order polynomial
satisfy
, and
, all the coefficients must
and
Higher-order example
A tabular method can be used to determine the stability when the roots of a higher order characteristic polynomial
are difficult to obtain. For an nth-degree polynomial
•
the table has n + 1 rows and the following structure:
where the elements
and
can be computed as follows:
•
•
When completed, the number of sign changes in the first column will be the number of non-negative poles.
Consider a system with a characteristic polynomial
•
We have the following table:
RouthHurwitz stability criterion
131
1
2
3 0
4
5
6 0
0.75 1.5 0 0
−3
6
3
0
6
0
0
In the first column, there are two sign changes (0.75 → −3, and −3 → 3), thus there are two non-negative roots
where the system is unstable. " Sometimes the presence of poles on the imaginary axis creates a situation of marginal
stability. In that case the coefficients of the "Routh Array" become zero and thus further solution of the polynomial
for finding changes in sign is not possible. Then another approach comes into play. The row of polynomial which is
just above the row containing the zeroes is called "Auxiliary Polynomial".
•
We have the following table:
1 8
20 16
2 12 16 0
2 12 16 0
0 0
In such a case the Auxiliary polynomial is
0
0
which is again equal to zero. The next step
is to differentiate the above equation which yields the following polynomial.
. The
coefficients of the row containing zero now become "8" and "24". The process of Routh array is proceeded using
these values which yield two points on the imaginary axis. These two points on the imaginary axis are the prime
cause of marginal stability.[1]
References
[1] Saeed, Syed Hasan (2008). Automatic Control Systems. Delhi: Katson Publishers. pp. 206, 207. ISBN 978-81-906919-2-5.
• Hurwitz, A. (1964). "‘On the conditions under which an equation has only roots with negative real parts". Selected
Papers on Mathematical Trends in Control Theory.
• Routh, E. J. (1877). A Treatise on the Stability of a Given State of Motion: Particularly Steady Motion. Macmillan
and co..
• Gantmacher, F. R. (1959). "Applications of the Theory of Matrices". Interscience, New York 641 (9): 1–8.
• Pippard, A. B.; Dicke, R. H. (1986). "Response and Stability, An Introduction to the Physical Theory" (http://
link.aip.org/link/?AJPIAS/54/1052/1). American Journal of Physics 54: 1052. doi:10.1119/1.14826.
Retrieved 2008-05-07.
• Weisstein, Eric W. "Routh-Hurwitz Theorem." (http://mathworld.wolfram.com/Routh-HurwitzTheorem.
html). MathWorld--A Wolfram Web Resource.
• Richard C. Dorf, Robert H. Bishop (2001). Modern Control Systems, 9th Edition. Prentice Hall.
ISBN 0-13-030660-6.
RouthHurwitz stability criterion
External links
• A MATLAB script implementing the Routh-Hurwitz test (http://www.mathworks.com/matlabcentral/
fileexchange/25956-routh-hurwitz-stability-test)
132
133
Feedback properties
Control theory
Control theory is an interdisciplinary
branch of engineering and mathematics
that deals with the behavior of
dynamical systems. The external input
of a system is called the reference.
When one or more output variables of
The concept of the feedback loop to control the dynamic behavior of the system: this is
a system need to follow a certain
negative feedback, because the sensed value is subtracted from the desired value to create
reference over time, a controller
the error signal, which is amplified by the controller.
manipulates the inputs to a system to
obtain the desired effect on the output of the system.
The usual objective of control theory is to calculate solutions for the proper corrective action from the controller that
result in system stability, that is, the system will hold the set point and not oscillate around it.
The input and ouput of the system are related to each other by what is known as a transfer function (also known as
the system function or network function). The transfer function is a mathematical representation, in terms of spatial
or temporal frequency, of the relation between the input and output of a linear time-invariant system.
Extensive use is usually made of a diagrammatic style known as the block diagram.
Overview
Control theory is
• a theory that deals with influencing the behavior of dynamical systems
• an interdisciplinary subfield of science, which originated in engineering and mathematics, and evolved into use by
the social sciences, like psychology, sociology, criminology and in financial system.
Control systems can be thought of as having four functions; Measure, Compare, Compute, and Correct. These four
functions are completed by five elements; Detector, Transducer, Transmitter, Controller, and Final Control Element.
The measuring function is completed by the detector, transducer and transmitter. In practical applications these three
elements are typically contained in one unit. A standard example is a Resistance thermometer. The compare and
compute functions are completed within the controller which may be completed electronically through a
Proportional Control, PI Controller, PID Controller, Bistable, Hysteretic control or Programmable logic controller.
The correct function is completed with a final control element. The final control element changes an input or output
in the control system which affect the manipulated or controlled variable.
Control theory
134
An example
Consider a car's cruise control, which is a device designed to maintain vehicle speed at a constant desired or
reference speed provided by the driver. The controller is the cruise control, the plant is the car, and the system is the
car and the cruise control. The system output is the car's speed, and the control itself is the engine's throttle position
which determines how much power the engine generates.
A primitive way to implement cruise control is simply to lock the throttle position when the driver engages cruise
control. However, if the cruise control is engaged on a stretch of flat road, then the car will travel slower going uphill
and faster when going downhill. This type of controller is called an open-loop controller because no measurement of
the system output (the car's speed) is used to alter the control (the throttle position.) As a result, the controller can not
compensate for changes acting on the car, like a change in the slope of the road.
In a closed-loop control system, a sensor monitors the system output (the car's speed) and feeds the data to a
controller which adjusts the control (the throttle position) as necessary to maintain the desired system output (match
the car's speed to the reference speed.) Now when the car goes uphill the decrease in speed is measured, and the
throttle position changed to increase engine power, speeding the vehicle. Feedback from measuring the car's speed
has allowed the controller to dynamically compensate for changes to the car's speed. It is from this feedback that the
paradigm of the control loop arises: the control affects the system output, which in turn is measured and looped back
to alter the control.
History
Although control systems of various types date back to antiquity, a
more formal analysis of the field began with a dynamics analysis of the
centrifugal governor, conducted by the physicist James Clerk Maxwell
in 1868 entitled On Governors.[1] This described and analyzed the
phenomenon of "hunting", in which lags in the system can lead to
overcompensation and unstable behavior. This generated a flurry of
interest in the topic, during which Maxwell's classmate Edward John
Routh generalized the results of Maxwell for the general class of linear
systems.[2] Independently, Adolf Hurwitz analyzed system stability
using differential equations in 1877, resulting in what is now known as
the Routh-Hurwitz theorem.[3] [4]
A notable application of dynamic control was in the area of manned
flight. The Wright brothers made their first successful test flights on
December 17, 1903 and were distinguished by their ability to control
their flights for substantial periods (more so than the ability to produce
lift from an airfoil, which was known). Control of the airplane was
necessary for safe flight.
Centrifugal governor in a Boulton & Watt engine
of 1788
By World War II, control theory was an important part of fire-control systems, guidance systems and electronics.
Sometimes mechanical methods are used to improve the stability of systems. For example, ship stabilizers are fins
mounted beneath the waterline and emerging laterally. In contemporary vessels, they may be gyroscopically
controlled active fins, which have the capacity to change their angle of attack to counteract roll caused by wind or
waves acting on the ship.
The Sidewinder missile uses small control surfaces placed at the rear of the missile with spinning disks on their outer
surface; these are known as rollerons. Airflow over the disk spins them to a high speed. If the missile starts to roll,
the gyroscopic force of the disk drives the control surface into the airflow, cancelling the motion. Thus the
Control theory
Sidewinder team replaced a potentially complex control system with a simple mechanical solution.
The Space Race also depended on accurate spacecraft control. However, control theory also saw an increasing use in
fields such as economics.
People in systems and control
Many active and historical figures made significant contribution to control theory, including, for example:
• Alexander Lyapunov (1857–1918) in the 1890s marks the beginning of stability theory.
• Harold S. Black (1898–1983), invented the concept of negative feedback amplifiers in 1927. He managed to
develop stable negative feedback amplifiers in the 1930s.
• Harry Nyquist (1889–1976), developed the Nyquist stability criterion for feedback systems in the 1930s.
• Richard Bellman (1920–1984), developed dynamic programming since the 1940s.
• Andrey Kolmogorov (1903–1987) co-developed the Wiener-Kolmogorov filter (1941).
• Norbert Wiener (1894–1964) co-developed the Wiener-Kolmogorov filter and coined the term cybernetics in the
1940s.
• John R. Ragazzini (1912–1988) introduced digital control and the z-transform in the 1950s.
• Lev Pontryagin (1908–1988) introduced the maximum principle and the bang-bang principle.
Classical control theory
To avoid the problems of the open-loop controller, control theory introduces feedback. A closed-loop controller uses
feedback to control states or outputs of a dynamical system. Its name comes from the information path in the system:
process inputs (e.g., voltage applied to an electric motor) have an effect on the process outputs (e.g., speed or torque
of the motor), which is measured with sensors and processed by the controller; the result (the control signal) is used
as input to the process, closing the loop.
Closed-loop controllers have the following advantages over open-loop controllers:
• disturbance rejection (such as unmeasured friction in a motor)
• guaranteed performance even with model uncertainties, when the model structure does not match perfectly the
real process and the model parameters are not exact
• unstable processes can be stabilized
• reduced sensitivity to parameter variations
• improved reference tracking performance
In some systems, closed-loop and open-loop control are used simultaneously. In such systems, the open-loop control
is termed feedforward and serves to further improve reference tracking performance.
A common closed-loop controller architecture is the PID controller.
Closed-loop transfer function
The output of the system y(t) is fed back through a sensor measurement F to the reference value r(t). The controller
C then takes the error e (difference) between the reference and the output to change the inputs u to the system under
control P. This is shown in the figure. This kind of controller is a closed-loop controller or feedback controller.
This is called a single-input-single-output (SISO) control system; MIMO (i.e., Multi-Input-Multi-Output) systems,
with more than one input/output, are common. In such cases variables are represented through vectors instead of
simple scalar values. For some distributed parameter systems the vectors may be infinite-dimensional (typically
functions).
135
Control theory
136
If we assume the controller C, the plant P, and the sensor F are linear and time-invariant (i.e., elements of their
transfer function C(s), P(s), and F(s) do not depend on time), the systems above can be analysed using the Laplace
transform on the variables. This gives the following relations:
Solving for Y(s) in terms of R(s) gives:
The expression
is referred to as the closed-loop transfer function of the system.
The numerator is the forward (open-loop) gain from r to y, and the denominator is one plus the gain in going around
the feedback loop, the so-called loop gain. If
, i.e., it has a large norm with each value of s, and
if
, then Y(s) is approximately equal to R(s) and the output closely tracks the reference input.
PID controller
The PID controller is probably the most-used feedback control design. PID is an acronym for
Proportional-Integral-Derivative, referring to the three terms operating on the error signal to produce a control
signal. If u(t) is the control signal sent to the system, y(t) is the measured output and r(t) is the desired output, and
tracking error
, a PID controller has the general form
The desired closed loop dynamics is obtained by adjusting the three parameters
,
and
, often
iteratively by "tuning" and without specific knowledge of a plant model. Stability can often be ensured using only
the proportional term. The integral term permits the rejection of a step disturbance (often a striking specification in
process control). The derivative term is used to provide damping or shaping of the response. PID controllers are the
most well established class of control systems: however, they cannot be used in several more complicated cases,
especially if MIMO systems are considered.
Applying Laplace transformation results in the transformed PID controller equation
with the PID controller transfer function
Control theory
Modern control theory
In contrast to the frequency domain analysis of the classical control theory, modern control theory utilizes the
time-domain state space representation, a mathematical model of a physical system as a set of input, output and state
variables related by first-order differential equations. To abstract from the number of inputs, outputs and states, the
variables are expressed as vectors and the differential and algebraic equations are written in matrix form (the latter
only being possible when the dynamical system is linear). The state space representation (also known as the
"time-domain approach") provides a convenient and compact way to model and analyze systems with multiple inputs
and outputs. With inputs and outputs, we would otherwise have to write down Laplace transforms to encode all the
information about a system. Unlike the frequency domain approach, the use of the state space representation is not
limited to systems with linear components and zero initial conditions. "State space" refers to the space whose axes
are the state variables. The state of the system can be represented as a vector within that space.[5]
Topics in control theory
Stability
The stability of a general dynamical system with no input can be described with Lyapunov stability criteria. A linear
system that takes an input is called bounded-input bounded-output (BIBO) stable if its output will stay bounded for
any bounded input. Stability for nonlinear systems that take an input is input-to-state stability (ISS), which combines
Lyapunov stability and a notion similar to BIBO stability. For simplicity, the following descriptions focus on
continuous-time and discrete-time linear systems.
Mathematically, this means that for a causal linear system to be stable all of the poles of its transfer function must
satisfy some criteria depending on whether a continuous or discrete time analysis is used:
• In continuous time, the Laplace transform is used to obtain the transfer function. A system is stable if the poles of
this transfer function lie strictly in the open left half of the complex plane (i.e. the real part of all the poles is less
than zero).
• In discrete time the Z-transform is used. A system is stable if the poles of this transfer function lie strictly inside
the unit circle. i.e. the magnitude of the poles is less than one).
When the appropriate conditions above are satisfied a system is said to be asymptotically stable: the variables of an
asymptotically stable control system always decrease from their initial value and do not show permanent oscillations.
Permanent oscillations occur when a pole has a real part exactly equal to zero (in the continuous time case) or a
modulus equal to one (in the discrete time case). If a simply stable system response neither decays nor grows over
time, and has no oscillations, it is marginally stable: in this case the system transfer function has non-repeated poles
at complex plane origin (i.e. their real and complex component is zero in the continuous time case). Oscillations are
present when poles with real part equal to zero have an imaginary part not equal to zero.
Differences between the two cases are not a contradiction. The Laplace transform is in Cartesian coordinates and the
Z-transform is in circular coordinates, and it can be shown that:
• the negative-real part in the Laplace domain can map onto the interior of the unit circle
• the positive-real part in the Laplace domain can map onto the exterior of the unit circle
If a system in question has an impulse response of
then the Z-transform (see this example), is given by
137
Control theory
which has a pole in
138
(zero imaginary part). This system is BIBO (asymptotically) stable since the pole is
inside the unit circle.
However, if the impulse response was
then the Z-transform is
which has a pole at
and is not BIBO stable since the pole has a modulus strictly greater than one.
Numerous tools exist for the analysis of the poles of a system. These include graphical systems like the root locus,
Bode plots or the Nyquist plots.
Mechanical changes can make equipment (and control systems) more stable. Sailors add ballast to improve the
stability of ships. Cruise ships use antiroll fins that extend transversely from the side of the ship for perhaps 30 feet
(10 m) and are continuously rotated about their axes to develop forces that oppose the roll.
Controllability and observability
Controllability and observability are main issues in the analysis of a system before deciding the best control strategy
to be applied, or whether it is even possible to control or stabilize the system. Controllability is related to the
possibility of forcing the system into a particular state by using an appropriate control signal. If a state is not
controllable, then no signal will ever be able to control the state. If a state is not controllable, but its dynamics are
stable, then the state is termed Stabilizable. Observability instead is related to the possibility of "observing", through
output measurements, the state of a system. If a state is not observable, the controller will never be able to determine
the behaviour of an unobservable state and hence cannot use it to stabilize the system. However, similar to the
stabilizability condition above, if a state cannot be observed it might still be detectable.
From a geometrical point of view, looking at the states of each variable of the system to be controlled, every "bad"
state of these variables must be controllable and observable to ensure a good behaviour in the closed-loop system.
That is, if one of the eigenvalues of the system is not both controllable and observable, this part of the dynamics will
remain untouched in the closed-loop system. If such an eigenvalue is not stable, the dynamics of this eigenvalue will
be present in the closed-loop system which therefore will be unstable. Unobservable poles are not present in the
transfer function realization of a state-space representation, which is why sometimes the latter is preferred in
dynamical systems analysis.
Solutions to problems of uncontrollable or unobservable system include adding actuators and sensors.
Control specification
Several different control strategies have been devised in the past years. These vary from extremely general ones (PID
controller), to others devoted to very particular classes of systems (especially robotics or aircraft cruise control).
A control problem can have several specifications. Stability, of course, is always present: the controller must ensure
that the closed-loop system is stable, regardless of the open-loop stability. A poor choice of controller can even
worsen the stability of the open-loop system, which must normally be avoided. Sometimes it would be desired to
obtain particular dynamics in the closed loop: i.e. that the poles have
, where is a fixed value
strictly greater than zero, instead of simply asking that
.
Another typical specification is the rejection of a step disturbance; including an integrator in the open-loop chain (i.e.
directly before the system under control) easily achieves this. Other classes of disturbances need different types of
sub-systems to be included.
Control theory
Other "classical" control theory specifications regard the time-response of the closed-loop system: these include the
rise time (the time needed by the control system to reach the desired value after a perturbation), peak overshoot (the
highest value reached by the response before reaching the desired value) and others (settling time, quarter-decay).
Frequency domain specifications are usually related to robustness (see after).
Modern performance assessments use some variation of integrated tracking error (IAE,ISA,CQI).
Model identification and robustness
A control system must always have some robustness property. A robust controller is such that its properties do not
change much if applied to a system slightly different from the mathematical one used for its synthesis. This
specification is important: no real physical system truly behaves like the series of differential equations used to
represent it mathematically. Typically a simpler mathematical model is chosen in order to simplify calculations,
otherwise the true system dynamics can be so complicated that a complete model is impossible.
System identification
The process of determining the equations that govern the model's dynamics is called system identification. This can
be done off-line: for example, executing a series of measures from which to calculate an approximated mathematical
model, typically its transfer function or matrix. Such identification from the output, however, cannot take account of
unobservable dynamics. Sometimes the model is built directly starting from known physical equations: for example,
in the case of a mass-spring-damper system we know that
. Even assuming that a
"complete" model is used in designing the controller, all the parameters included in these equations (called "nominal
parameters") are never known with absolute precision; the control system will have to behave correctly even when
connected to physical system with true parameter values away from nominal.
Some advanced control techniques include an "on-line" identification process (see later). The parameters of the
model are calculated ("identified") while the controller itself is running: in this way, if a drastic variation of the
parameters ensues (for example, if the robot's arm releases a weight), the controller will adjust itself consequently in
order to ensure the correct performance.
Analysis
Analysis of the robustness of a SISO (single input single output) control system can be performed in the frequency
domain, considering the system's transfer function and using Nyquist and Bode diagrams. Topics include gain and
phase margin and amplitude margin. For MIMO (multi input multi output) and, in general, more complicated control
systems one must consider the theoretical results devised for each control technique (see next section): i.e., if
particular robustness qualities are needed, the engineer must shift his attention to a control technique by including
them in its properties.
Constraints
A particular robustness issue is the requirement for a control system to perform properly in the presence of input and
state constraints. In the physical world every signal is limited. It could happen that a controller will send control
signals that cannot be followed by the physical system: for example, trying to rotate a valve at excessive speed. This
can produce undesired behavior of the closed-loop system, or even damage or break actuators or other subsystems.
Specific control techniques are available to solve the problem: model predictive control (see later), and anti-wind up
systems. The latter consists of an additional control block that ensures that the control signal never exceeds a given
threshold.
139
Control theory
System classifications
Linear systems control
For MIMO systems, pole placement can be performed mathematically using a state space representation of the
open-loop system and calculating a feedback matrix assigning poles in the desired positions. In complicated systems
this can require computer-assisted calculation capabilities, and cannot always ensure robustness. Furthermore, all
system states are not in general measured and so observers must be included and incorporated in pole placement
design.
Nonlinear systems control
Processes in industries like robotics and the aerospace industry typically have strong nonlinear dynamics. In control
theory it is sometimes possible to linearize such classes of systems and apply linear techniques, but in many cases it
can be necessary to devise from scratch theories permitting control of nonlinear systems. These, e.g., feedback
linearization, backstepping, sliding mode control, trajectory linearization control normally take advantage of results
based on Lyapunov's theory. Differential geometry has been widely used as a tool for generalizing well-known linear
control concepts to the non-linear case, as well as showing the subtleties that make it a more challenging problem.
Decentralized systems
When the system is controlled by multiple controllers, the problem is one of decentralized control. Decentralization
is helpful in many ways, for instance, it helps control systems operate over a larger geographical area. The agents in
decentralized control systems can interact using communication channels and coordinate their actions.
Main control strategies
Every control system must guarantee first the stability of the closed-loop behavior. For linear systems, this can be
obtained by directly placing the poles. Non-linear control systems use specific theories (normally based on
Aleksandr Lyapunov's Theory) to ensure stability without regard to the inner dynamics of the system. The possibility
to fulfill different specifications varies from the model considered and the control strategy chosen. Here a summary
list of the main control techniques is shown:
Adaptive control
Adaptive control uses on-line identification of the process parameters, or modification of controller gains,
thereby obtaining strong robustness properties. Adaptive controls were applied for the first time in the
aerospace industry in the 1950s, and have found particular success in that field.
Hierarchical control
A Hierarchical control system is a type of Control System in which a set of devices and governing software is
arranged in a hierarchical tree. When the links in the tree are implemented by a computer network, then that
hierarchical control system is also a form of Networked control system.
Intelligent control
Intelligent control uses various AI computing approaches like neural networks, Bayesian probability, fuzzy
logic, machine learning, evolutionary computation and genetic algorithms to control a dynamic system.
Optimal control
Optimal control is a particular control technique in which the control signal optimizes a certain "cost index":
for example, in the case of a satellite, the jet thrusts needed to bring it to desired trajectory that consume the
least amount of fuel. Two optimal control design methods have been widely used in industrial applications, as
it has been shown they can guarantee closed-loop stability. These are Model Predictive Control (MPC) and
140
Control theory
Linear-Quadratic-Gaussian control (LQG). The first can more explicitly take into account constraints on the
signals in the system, which is an important feature in many industrial processes. However, the "optimal
control" structure in MPC is only a means to achieve such a result, as it does not optimize a true performance
index of the closed-loop control system. Together with PID controllers, MPC systems are the most widely
used control technique in process control.
Robust control
Robust control deals explicitly with uncertainty in its approach to controller design. Controllers designed using
robust control methods tend to be able to cope with small differences between the true system and the nominal
model used for design. The early methods of Bode and others were fairly robust; the state-space methods
invented in the 1960s and 1970s were sometimes found to lack robustness. A modern example of a robust
control technique is H-infinity loop-shaping developed by Duncan McFarlane and Keith Glover of Cambridge
University, United Kingdom. Robust methods aim to achieve robust performance and/or stability in the
presence of small modeling errors.
Stochastic control
Stochastic control deals with control design with uncertainty in the model. In typical stochastic control
problems, it is assumed that there exist random noise and disturbances in the model and the controller, and the
control design must take into account these random deviations.
References
[1] Maxwell, J.C. (1867). "On Governors". Proceedings of the Royal Society of London 16: 270–283. doi:10.1098/rspl.1867.0055.
JSTOR 112510.
[2] Routh, E.J.; Fuller, A.T. (1975). Stability of motion. Taylor & Francis.
[3] Routh, E.J. (1877). A Treatise on the Stability of a Given State of Motion, Particularly Steady Motion: Particularly Steady Motion. Macmillan
and co..
[4] Hurwitz, A. (1964). "On The Conditions Under Which An Equation Has Only Roots With Negative Real Parts". Selected Papers on
Mathematical Trends in Control Theory.
[5] Donald M Wiberg. State space & linear systems. Schaum's outline series. McGraw Hill. ISBN 0070700966.
Further reading
• Levine, William S., ed (1996). The Control Handbook. New York: CRC Press. ISBN 978-0-849-38570-4.
• Karl J. Åström and Richard M. Murray (2008). Feedback Systems: An Introduction for Scientists and Engineers.
(http://www.cds.caltech.edu/~murray/books/AM08/pdf/am08-complete_22Feb09.pdf). Princeton
University Press. ISBN 0691135762.
• Christopher Kilian (2005). Modern Control Technology. Thompson Delmar Learning. ISBN 1-4018-5806-6.
• Vannevar Bush (1929). Operational Circuit Analysis. John Wiley and Sons, Inc..
• Robert F. Stengel (1994). Optimal Control and Estimation. Dover Publications. ISBN 0-486-68200-5, ISBN
978-0-486-68200-6.
• Franklin et al. (2002). Feedback Control of Dynamic Systems (4 ed.). New Jersey: Prentice Hall.
ISBN 0-13-032393-4.
• Joseph L. Hellerstein, Dawn M. Tilbury, and Sujay Parekh (2004). Feedback Control of Computing Systems. John
Wiley and Sons. ISBN 0-47-126637-X, ISBN 978-0-471-26637-2.
• Diederich Hinrichsen and Anthony J. Pritchard (2005). Mathematical Systems Theory I - Modelling, State Space
Analysis, Stability and Robustness. Springer. ISBN 0-978-3-540-44125-0.
• Andrei, Neculai (2005). Modern Control Theory - A historical Perspective (http://camo.ici.ro/neculai/history.
pdf). Retrieved 2007-10-10.
• Sontag, Eduardo (1998). Mathematical Control Theory: Deterministic Finite Dimensional Systems. Second
Edition (http://www.math.rutgers.edu/~sontag/FTP_DIR/sontag_mathematical_control_theory_springer98.
141
Control theory
pdf). Springer. ISBN 0-387-984895.
• Goodwin, Graham (2001). Control System Design. Prentice Hall. ISBN 0-13-958653-9.
For Chemical Engineering
• Luyben, William (1989). Process Modeling, Simulation, and Control for Chemical Engineers. Mc Graw Hill.
ISBN 0-07-039159-9.
External links
• Control Tutorials for Matlab (http://www.engin.umich.edu/class/ctms/) - A set of worked through control
examples solved by several different methods.
• Control Tuning and Best Practices (http://www.controlguru.com)
Control system
A control system is a device, or set of devices to manage, command, direct or regulate the behavior of other devices
or system.
There are two common classes of control systems, with many variations and combinations: logic or sequential
controls, and feedback or linear controls. There is also fuzzy logic, which attempts to combine some of the design
simplicity of logic with the utility of linear control. Some devices or systems are inherently not controllable.
Overview
The term "control system" may be applied to the essentially manual controls that allow an operator, for example, to
close and open a hydraulic press, perhaps including logic so that it cannot be moved unless safety guards are in
place.
An automatic sequential control system may trigger a series of mechanical actuators in the correct sequence to
perform a task. For example various electric and pneumatic transducers may fold and glue a cardboard box, fill it
with product and then seal it in an automatic packaging machine.
In the case of linear feedback systems, a control loop, including sensors, control algorithms and actuators, is
arranged in such a fashion as to try to regulate a variable at a setpoint or reference value. An example of this may
increase the fuel supply to a furnace when a measured temperature drops. PID controllers are common and effective
in cases such as this. Control systems that include some sensing of the results they are trying to achieve are making
use of feedback and so can, to some extent, adapt to varying circumstances. Open-loop control systems do not make
use of feedback, and run only in pre-arranged ways.
Logic control
Logic control systems for industrial and commercial machinery were historically implemented at mains voltage
using interconnected relays, designed using ladder logic. Today, most such systems are constructed with
programmable logic controllers (PLCs) or microcontrollers. The notation of ladder logic is still in use as a
programming idiom for PLCs.[1]
Logic controllers may respond to switches, light sensors, pressure switches, etc., and can cause the machinery to start
and stop various operations. Logic systems are used to sequence mechanical operations in many applications. PLC
software can be written in many different ways – ladder diagrams, SFC – sequential function charts or in language
terms known as statement lists.[2]
Examples include elevators, washing machines and other systems with interrelated stop-go operations.
142
Control system
Logic systems are quite easy to design, and can handle very complex operations. Some aspects of logic system
design make use of Boolean logic.
On–off control
For example, a thermostat is a simple negative-feedback control: when the temperature (the "process variable" or
PV) goes below a set point (SP), the heater is switched on. Another example could be a pressure switch on an air
compressor: when the pressure (PV) drops below the threshold (SP), the pump is powered. Refrigerators and vacuum
pumps contain similar mechanisms operating in reverse, but still providing negative feedback to correct errors.
Simple on–off feedback control systems like these are cheap and effective. In some cases, like the simple
compressor example, they may represent a good design choice.
In most applications of on–off feedback control, some consideration needs to be given to other costs, such as wear
and tear of control valves and maybe other start-up costs when power is reapplied each time the PV drops. Therefore,
practical on–off control systems are designed to include hysteresis, usually in the form of a deadband, a region
around the setpoint value in which no control action occurs. The width of deadband may be adjustable or
programmable.
Linear control
Linear control systems use linear negative feedback to produce a control signal mathematically based on other
variables, with a view to maintaining the controlled process within an acceptable operating range.
The output from a linear control system into the controlled process may be in the form of a directly variable signal,
such as a valve that may be 0 or 100% open or anywhere in between. Sometimes this is not feasible and so, after
calculating the current required corrective signal, a linear control system may repeatedly switch an actuator, such as
a pump, motor or heater, fully on and then fully off again, regulating the duty cycle using pulse-width modulation.
Proportional control
When controlling the temperature of an industrial furnace, it is usually better to control the opening of the fuel valve
in proportion to the current needs of the furnace. This helps avoid thermal shocks and applies heat more effectively.
Proportional negative-feedback systems are based on the difference between the required set point (SP) and process
value (PV). This difference is called the error. Power is applied in direct proportion to the current measured error, in
the correct sense so as to tend to reduce the error (and so avoid positive feedback). The amount of corrective action
that is applied for a given error is set by the gain or sensitivity of the control system.
At low gains, only a small corrective action is applied when errors are detected: the system may be safe and stable,
but may be sluggish in response to changing conditions; errors will remain uncorrected for relatively long periods of
time: it is over-damped. If the proportional gain is increased, such systems become more responsive and errors are
dealt with more quickly. There is an optimal value for the gain setting when the overall system is said to be critically
damped. Increases in loop gain beyond this point will lead to oscillations in the PV; such a system is under-damped.
Under-damped furnace example
In the furnace example, suppose the temperature is increasing towards a set point at which, say, 50% of the available
power will be required for steady-state. At low temperatures, 100% of available power is applied. When the PV is
within, say 10° of the SP the heat input begins to be reduced by the proportional controller. (Note that this implies a
20° "proportional band" (PB) from full to no power input, evenly spread around the setpoint value). At the setpoint
the controller will be applying 50% power as required, but stray stored heat within the heater sub-system and in the
walls of the furnace will keep the measured temperature rising beyond what is required. At 10° above SP, we reach
the top of the proportional band (PB) and no power is applied, but the temperature may continue to rise even further
143
Control system
before beginning to fall back. Eventually as the PV falls back into the PB, heat is applied again, but now the heater
and the furnace walls are too cool and the temperature falls too low before its fall is arrested, so that the oscillations
continue.
Over-damped furnace example
The temperature oscillations that an under-damped furnace control system produces are unacceptable for many
reasons, including the waste of fuel and time (each oscillation cycle may take many minutes), as well as the
likelihood of seriously overheating both the furnace and its contents.
Suppose that the gain of the control system is reduced drastically and it is restarted. As the temperature approaches,
say 30° below SP (60° proportional band or PB now), the heat input begins to be reduced, the rate of heating of the
furnace has time to slow and, as the heat is still further reduced, it eventually is brought up to set point, just as 50%
power input is reached and the furnace is operating as required. There was some wasted time while the furnace crept
to its final temperature using only 52% then 51% of available power, but at least no harm was done. By carefully
increasing the gain (i.e. reducing the width of the PB) this over-damped and sluggish behavior can be improved until
the system is critically damped for this SP temperature. Doing this is known as 'tuning' the control system. A
well-tuned proportional furnace temperature control system will usually be more effective than on-off control, but
will still respond more slowly than the furnace could under skillful manual control.
PID control
Further information: PID controller
Apart from sluggish performance to avoid oscillations, another problem with proportional-only control is that power
application is always in direct proportion to the error. In the example above we assumed that the set temperature
could be maintained with 50% power. What happens if the furnace is required in a different application where a
higher set temperature will require 80% power to maintain it? If the gain was finally set to a 50° PB, then 80% power
will not be applied unless the furnace is 15° below setpoint, so for this other application the operators will have to
remember always to set the setpoint temperature 15° higher than actually needed. This 15° figure is not completely
constant either: it will depend on the surrounding ambient temperature, as well as other factors that affect heat loss
from or absorption within the furnace.
To resolve these two problems, many feedback control schemes include mathematical extensions to improve
performance. The most common extensions lead to proportional-integral-derivative control, or PID control
(pronounced pee-eye-dee).
Derivative action
The derivative part is concerned with the rate-of-change of the error with time: If the measured variable approaches
the setpoint rapidly, then the actuator is backed off early to allow it to coast to the required level; conversely if the
measured value begins to move rapidly away from the setpoint, extra effort is applied—in proportion to that
rapidity—to try to maintain it.
Derivative action makes a control system behave much more intelligently. On systems like the temperature of a
furnace, or perhaps the motion-control of a heavy item like a gun or camera on a moving vehicle, the derivative
action of a well-tuned PID controller can allow it to reach and maintain a setpoint better than most skilled human
operators could.
If derivative action is over-applied, it can lead to oscillations too. An example would be a PV that increased rapidly
towards SP, then halted early and seemed to "shy away" from the setpoint before rising towards it again.
144
Control system
Integral action
The integral term magnifies the effect of long-term steady-state errors, applying ever-increasing effort until they
reduce to zero. In the example of the furnace above working at various temperatures, if the heat being applied does
not bring the furnace up to setpoint, for whatever reason, integral action increasingly moves the proportional band
relative to the setpoint until the PV error is reduced to zero and the setpoint is achieved.
Other techniques
It is possible to filter the PV or error signal. Doing so can reduce the response of the system to undesirable
frequencies, to help reduce instability or oscillations. Some feedback systems will oscillate at just one frequency. By
filtering out that frequency, more "stiff" feedback can be applied, making the system more responsive without
shaking itself apart.
Feedback systems can be combined. In cascade control, one control loop applies control algorithms to a measured
variable against a setpoint, but then provides a varying setpoint to another control loop rather than affecting process
variables directly. If a system has several different measured variables to be controlled, separate control systems will
be present for each of them.
Control engineering in many applications produces control systems that are more complex than PID control.
Examples of such fields include fly-by-wire aircraft control systems, chemical plants, and oil refineries. Model
predictive control systems are designed using specialized computer-aided-design software and empirical
mathematical models of the system to be controlled.
Fuzzy logic
Further information: Fuzzy logic
Fuzzy logic is an attempt to get the easy design of logic controllers and yet control continuously-varying systems.
Basically, a measurement in a fuzzy logic system can be partly true, that is if yes is 1 and no is 0, a fuzzy
measurement can be between 0 and 1.
The rules of the system are written in natural language and translated into fuzzy logic. For example, the design for a
furnace would start with: "If the temperature is too high, reduce the fuel to the furnace. If the temperature is too low,
increase the fuel to the furnace."
Measurements from the real world (such as the temperature of a furnace) are converted to values between 0 and 1 by
seeing where they fall on a triangle. Usually the tip of the triangle is the maximum possible value which translates to
"1."
Fuzzy logic, then, modifies Boolean logic to be arithmetical. Usually the "not" operation is "output = 1 - input," the
"and" operation is "output = input.1 multiplied by input.2," and "or" is "output = 1 - ((1 - input.1) multiplied by (1 input.2))". This reduces to Boolean arithmetic if values are restricted to 0 and 1, instead of allowed to range in the
unit interval [0,1].
The last step is to "defuzzify" an output. Basically, the fuzzy calculations make a value between zero and one. That
number is used to select a value on a line whose slope and height converts the fuzzy value to a real-world output
number. The number then controls real machinery.
If the triangles are defined correctly and rules are right the result can be a good control system.
When a robust fuzzy design is reduced into a single, quick calculation, it begins to resemble a conventional feedback
loop solution and it might appear that the fuzzy design was unnecessary. However, the fuzzy logic paradigm may
provide scalability for large control systems where conventional methods become unwieldy or costly to derive.
Fuzzy electronics is an electronic technology that uses fuzzy logic instead of the two-value logic more commonly
used in digital electronics.
145
Control system
Physical implementations
Since modern small microprocessors are so cheap (often less than $1 US), it's very common to implement control
systems, including feedback loops, with computers, often in an embedded system. The feedback controls are
simulated by having the computer make periodic measurements and then calculating from this stream of
measurements (see digital signal processing, sampled data systems).
Computers emulate logic devices by making measurements of switch inputs, calculating a logic function from these
measurements and then sending the results out to electronically-controlled switches.
Logic systems and feedback controllers are usually implemented with programmable logic controllers which are
devices available from electrical supply houses. They include a little computer and a simplified system for
programming. Most often they are programmed with personal computers.
Logic controllers have also been constructed from relays, hydraulic and pneumatic devices, and electronics using
both transistors and vacuum tubes (feedback controllers can also be constructed in this manner).
References
[1] Kuphaldt, Tony R.. "Chapter 6 LADDER LOGIC" (http:/ / openbookproject. net/ / electricCircuits/ Digital/ DIGI_6. html). Lessons In
Electric Circuits -- Volume IV. . Retrieved 22 September 2010.
[2] Brady, Ian. "Programmable logic controllers - benefits and applications" (http:/ / www. optimacs. com/ reports/ PLC Report. pdf). PLCs. .
Retrieved 5 December 2011.
External links
• Semiautonomous Flight Direction - Reference unmannedaircraft.org (http://www.unmannedaircraft.org/f/
pat6460810.pdf)
Controller (control theory)
In control theory, a controller is a device which monitors and affects the operational conditions of a given
dynamical system. The operational conditions are typically referred to as output variables of the system which can be
affected by adjusting certain input variables. For example, the heating system of a house can be equipped with a
thermostat (controller) for sensing air temperature (output variable) which can turn on or off a furnace or heater
when the air temperature becomes too low or too high.
In this example, the thermostat is the controller and directs the activities of the heater. The heater is the processor
that warms the air inside the house to the desired temperature (setpoint). The air temperature reading inside the house
is the feedback. And finally, the house is the environment in which the heating system operates.
The notion of controllers can be extended to more complex systems. In the natural world, individual organisms also
appear to be equipped with controllers that assure the homeostasis necessary for survival of each individual. Both
human-made and natural systems exhibit collective behaviors amongst individuals in which the controllers seek
some form of equilibrium.
Types of Controlling System
In control theory there are two basic types of control. These are feedback and feed-forward. The input to a feedback
controller is the same as what it is trying to control - the controlled variable is "fed back" into the controller. The
thermostat of a house is an example of a feedback controller. This controller relies on measuring the controlled
variable, in this case the temperature of the house, and then adjusting the output, whether or not the heater is on.
However, feedback control usually results in intermediate periods where the controlled variable is not at the desired
146
Controller (control theory)
147
set-point. With the thermostat example, if the door of the house were opened on a cold day, the house would cool
down. After it fell below the desired temperature (set-point), the heater would kick on, but there would be a period
when the house was colder than desired.
Feed-forward control can avoid the slowness of feedback control. With feed-forward control, the disturbances are
measured and accounted for before they have time to affect the system. In the house example, a feed-forward system
may measure the fact that the door is opened and automatically turn on the heater before the house can get too cold.
The difficulty with feed-forward control is that the effect of the disturbances on the system must be accurately
predicted, and there must not be any unmeasured disturbances. For instance, if a window were opened that was not
being measured, the feed-forward-controlled thermostat might still let the house cool down.
To achieve the benefits of feedback control (controlling unknown disturbances and not having to know exactly how
a system will respond to disturbances) and the benefits of feed-forward control (responding to disturbances before
they can affect the system), there are combination's of feedback and feed-forward that can be used.
Some examples of where feedback and feed-forward control can be used together are dead-time compensation, and
inverse response compensation. Dead-time compensation is used to control devices that take a long time to show
any change to a change in input, for example, change in composition of flow through a long pipe. A dead-time
compensation control uses an element (also called a Smith predictor) to predict how changes made now by the
controller will affect the controlled variable in the future. The controlled variable is also measured and used in
feedback control. Inverse response compensation involves controlling systems where a change at first affects the
measured variable one way but later affects it in the opposite way. An example would be eating candy. At first it will
give you lots of energy, but later you will be very tired. As can be imagined, it is difficult to control this system with
feedback alone, therefore a predictive feed-forward element is necessary to predict the reverse effect that a change
will have in the future. ''''
Types of controllers
Most control valve systems in the past were implemented using mechanical systems or solid state electronics.
Pneumatics were often utilized to transmit information and control using pressure. However, most modern control
systems in industrial settings now rely on computers for the controller. Obviously it is much easier to implement
complex control algorithms on a computer than using a mechanical system.
For feedback controllers there are a few simple types. The most simple is like the thermostat that just turns the heat
on if the temperature falls below a certain value and off it exceeds a certain value (on-off control).
Another simple type of controller is a proportional controller. With this type of controller, the controller output
(control action) is proportional to the error in the measured variable.
In feedback control, it is standard to define the error as the difference between the desired value (setpoint)
the current value (measured)
and
. If the error is large, then the control action is large. Mathematically:
where
represents the control action (controller output),
represents the error,
represents the controller's gain, and
represents the steady state control action (bias) necessary to maintain the variable at the steady state when
there is no error.
It is important that the control action
counteracts the change in the controlled variable
There are then two cases depending on the sign of the process gain.
(negative feedback).
Controller (control theory)
148
In the first case the process gain is positive, so an increase in the controlled variable (measurement)
decrease in the control action
(reverse-acting control). In this case the controller gain
standard definition of the error already contains a negative sign for
is positive, because the
.
In the second case the process gain is negative, so an increase in the controlled variable (measurement)
an increase in the control action
requires a
(direct-acting control). In this case the controller gain
A typical example of a reverse-acting system is control of temperature (
requires
is negative.
) by use of steam (
). In this case the
process gain is positive, so if the temperature increases, the steam flow must be decreased to maintain the desired
temperature. Conversely, a typical example of a direct-acting system is control of temperature using cooling water.
In this case the process gain is negative, so if the temperature increases, the cooling water flow must be increased to
maintain the desired temperature.
Although proportional control is simple to understand, it has drawbacks. The largest problem is that for most systems
it will never entirely remove error. This is because when error is 0 the controller only provides the steady state
control action so the system will settle back to the original steady state (which is probably not the new set point that
we want the system to be at). To get the system to operate near the new steady state, the controller gain, Kc, must be
very large so the controller will produce the required output when only a very small error is present. Having large
gains can lead to system instability or can require physical impossibilities like infinitely large valves.
Alternates to proportional control are proportional-integral (PI) control and proportional-integral-derivative (PID)
control. PID control is commonly used to implement closed-loop control.
Open-loop control can be used in systems sufficiently well-characterized as to predict what outputs will necessarily
achieve the desired states. For example, the rotational velocity of an electric motor may be well enough characterized
for the supplied voltage to make feedback unnecessary.
The drawback of open-loop control is that it requires perfect knowledge of the system (i.e. one knows exactly what
inputs to give in order to get the desired output), and it assumes there are no disturbances to the system.
References
Process control
Process control
Process control is a statistics and engineering discipline that deals with architectures, mechanisms and algorithms
for maintaining the output of a specific process within a desired range. See also control theory.
Background
Process control is extensively used in industry and enables mass production of continuous processes such as oil
refining, paper manufacturing, chemicals, power plants and many other industries. Process control enables
automation, with which a small staff of operating personnel can operate a complex process from a central control
room.
For example, heating up the temperature in a room is a process that has the specific, desired outcome to reach and
maintain a defined temperature (e.g. 20°C), kept constant over time. Here, the temperature is the controlled
variable. At the same time, it is the input variable since it is measured by a thermometer and used to decide
whether to heat or not to heat. The desired temperature (20°C) is the setpoint. The state of the heater (e.g. the setting
of the valve allowing hot water to flow through it) is called the manipulated variable since it is subject to control
actions.
A commonly used control device called a programmable logic controller, or a PLC, is used to read a set of digital
and analog inputs, apply a set of logic statements, and generate a set of analog and digital outputs. Using the example
in the previous paragraph, the room temperature would be an input to the PLC. The logical statements would
compare the setpoint to the input temperature and determine whether more or less heating was necessary to keep the
temperature constant. A PLC output would then either open or close the hot water valve, an incremental amount,
depending on whether more or less hot water was needed. Larger more complex systems can be controlled by a
Distributed Control System (DCS) or SCADA system.
Types of control systems
In practice, process control systems can be characterized as one or more of the following forms:
• Discrete – Found in many manufacturing, motion and packaging applications. Robotic assembly, such as that
found in automotive production, can be characterized as discrete process control. Most discrete manufacturing
involves the production of discrete pieces of product, such as metal stamping.
• Batch – Some applications require that specific quantities of raw materials be combined in specific ways for
particular durations to produce an intermediate or end result. One example is the production of adhesives and
glues, which normally require the mixing of raw materials in a heated vessel for a period of time to form a
quantity of end product. Other important examples are the production of food, beverages and medicine. Batch
processes are generally used to produce a relatively low to intermediate quantity of product per year (a few
pounds to millions of pounds).
• Continuous – Often, a physical system is represented through variables that are smooth and uninterrupted in time.
The control of the water temperature in a heating jacket, for example, is an example of continuous process
control. Some important continuous processes are the production of fuels, chemicals and plastics. Continuous
processes in manufacturing are used to produce very large quantities of product per year (millions to billions of
pounds).
Applications having elements of discrete, batch and continuous process control are often called hybrid applications.
149
Process control
Statistical Process Control
Statistical Process Control (SPC) is an effective method of monitoring a process through the use of control charts.
Much of its power lies in the ability to monitor both the current center of a process and the process's variation about
that center. By collecting data from samples at various points within the process, variations in the process that may
affect the quality of the end product or service can be detected and corrected, thus reducing waste as well as the
likelihood that problems will be passed on to the customer. It has an emphasis on early detection and prevention of
problems.
Multivariable Process Control is a type of Statistical Process Control where a set of variables (manipulated variables
and control variables) are identified and the joint variations within this set are captured byng a step test. The
Dynamics captured in the model curves are used to control the plant
Examples
A thermostat is a simple example for a closed control loop: It constantly measures the current temperature and
controls the heater's valve setting to increase or decrease the room temperature according to the user-defined setting.
A simple method switches the heater either completely on, or completely off, and an overshoot and undershoot of the
controlled temperature must be expected. A more expensive method varies the amount of heat provided by the heater
depending on the difference between the required temperature (the "setpoint") and the actual temperature. This
minimizes over/undershoot.
An anti-lock braking system (ABS) is a more complex example, consisting of multiple inputs, conditions and
outputs.
External links
•
•
•
•
Jobs in Process Control [1]
The Michigan Chemical Engineering Process Dynamics and Controls Open Textbook [2]
PID Control Theory and Best Practices [3]
Process Control & Instrumentation [4]
References
[1]
[2]
[3]
[4]
http:/ / controljob. com
http:/ / controls. engin. umich. edu/ wiki
http:/ / www. controlguru. com
http:/ / www. flotech. com. sg
150
151
Three-term control
Proportionality (mathematics)
In mathematics, two variable quantities are proportional if one
of them is always the product of the other and a constant quantity,
called the coefficient of proportionality or proportionality constant.
In other words, x and y are proportional if the ratio
is constant.
We also say that one of the quantities is proportional to the other.
For example, if the speed of an object is constant, it travels a distance
that is proportional to the travel time.
y is directly proportional to x.
If a linear function transforms 0, a and b into 0, c and d, and if the product a b c d is not zero, we say a and b are
proportional to c and d. An equality of two ratios such as where no term is zero, is called a proportion.
Geometric illustration
The two rectangles with stripes are similar, the ratios of their dimensions are horizontally
written within the image. The duplication scale of a striped triangle is obliquely written,
in a proportion obtained by inverting two terms of another proportion horizontally
written.
Proportionality (mathematics)
152
When the duplication of a given rectangle preserves its shape, the ratio of the large dimension to the small dimension
is a constant number in all the copies, and in the original rectangle. The largest rectangle of the drawing is similar
to one or the other rectangle with stripes. From their width to their height, the coefficient is A ratio of their dimensions horizontally written within the image, at the top or the bottom, determines the common
shape of the three similar rectangles.
The common diagonal of the similar rectangles divides each rectangle into two superposable triangles, with two
different kinds of stripes. The four striped triangles and the two striped rectangles have a common vertex: the center
of an homothetic transformation with a negative ratio – k or , that transforms one triangle and its stripes into
another triangle with the same stripes, enlarged or reduced. The duplication scale of a striped triangle is the
proportionality constant between the corresponding sides lengths of the triangles, equal to a positive ratio obliquely
written within the image:
or In the proportion
, the terms a and d are called the extremes, while b and c are the means, because a and d
are the extreme terms of the list (a, b, c, d), while b and c are in the middle of the list. From any proportion, we get
another proportion by inverting the extremes or the means. And the product of the extremes equals the product
of the means. Within the image, a double arrow indicates two inverted terms of the first proportion.
Consider dividing the largest rectangle in two triangles, cutting along the diagonal. If we remove two triangles from
either half rectangle, we get one of the plain gray rectangles. Above and below this diagonal, the areas of the two
biggest triangles of the drawing are equal, because these triangles are superposable. Above and below the subtracted
areas are equal for the same reason. Therefore, the two plain gray rectangles have the same area: a d = b c.
Symbol
The mathematical symbol '∝' is used to indicate that two values are proportional. For example, A ∝ B.
In Unicode this is symbol U+221D.
Direct proportionality
Given two variables x and y, y is (directly) proportional to x (x and y vary directly, or x and y are in direct variation)
if there is a non-zero constant k such that
The relation is often denoted
and the constant ratio
is called the proportionality constant or constant of proportionality.
Proportionality (mathematics)
Examples
• If an object travels at a constant speed, then the distance traveled is proportional to the time spent traveling, with
the speed being the constant of proportionality.
• The circumference of a circle is proportional to its diameter, with the constant of proportionality equal to π.
• On a map drawn to scale, the distance between any two points on the map is proportional to the distance between
the two locations that the points represent, with the constant of proportionality being the scale of the map.
• The force acting on a certain object due to gravity is proportional to the object's mass; the constant of
proportionality between the mass and the force is known as gravitational acceleration.
Properties
Since
is equivalent to
it follows that if y is proportional to x, with (nonzero) proportionality constant k, then x is also proportional to y with
proportionality constant 1/k.
If y is proportional to x, then the graph of y as a function of x will be a straight line passing through the origin with
the slope of the line equal to the constant of proportionality: it corresponds to linear growth.
Inverse proportionality
The concept of inverse proportionality can be contrasted against direct proportionality. Consider two variables said
to be "inversely proportional" to each other. If all other variables are held constant, the magnitude or absolute value
of one inversely proportional variable will decrease if the other variable increases, while their product (the constant
of proportionality k) is always the same.
Formally, two variables are inversely proportional (or varying inversely, or in inverse variation, or in inverse
proportion or in reciprocal proportion) if one of the variables is directly proportional with the multiplicative
inverse (reciprocal) of the other, or equivalently if their product is a constant. It follows that the variable y is
inversely proportional to the variable x if there exists a non-zero constant k such that
The constant can be found by multiplying the original x variable and the original y variable.
As an example, the time taken for a journey is inversely proportional to the speed of travel; the time needed to dig a
hole is (approximately) inversely proportional to the number of people digging.
The graph of two variables varying inversely on the Cartesian coordinate plane is a hyperbola. The product of the X
and Y values of each point on the curve will equal the constant of proportionality (k). Since neither x nor y can equal
zero (if k is non-zero), the graph will never cross either axis.
153
Proportionality (mathematics)
154
Hyperbolic coordinates
The concepts of direct and inverse proportion lead to the location of points in the Cartesian plane by hyperbolic
coordinates; the two coordinates correspond to the constant of direct proportionality that locates a point on a ray and
the constant of inverse proportionality that locates a point on a hyperbola.
Exponential and logarithmic proportionality
A variable y is exponentially proportional to a variable x, if y is directly proportional to the exponential function of
x, that is if there exist non-zero constants k and a
Likewise, a variable y is logarithmically proportional to a variable x, if y is directly proportional to the logarithm of
x, that is if there exist non-zero constants k and a
Experimental determination
To determine experimentally whether two physical quantities are directly proportional, one performs several
measurements and plots the resulting data points in a Cartesian coordinate system. If the points lie on or close to a
straight line that passes through the origin (0, 0), then the two variables are probably proportional, with the
proportionality constant given by the line's slope.
Integral
Integration is an important concept in mathematics and, together with
its inverse, differentiation, is one of the two main operations in
calculus. Given a function ƒ of a real variable x and an interval [a, b] of
the real line, the definite integral
A definite integral of a function can be
represented as the signed area of the region
bounded by its graph.
is defined informally to be the net signed area of the region in the xy-plane bounded by the graph of ƒ, the x-axis, and
the vertical lines x = a and x = b.
The term integral may also refer to the notion of antiderivative, a function F whose derivative is the given function ƒ.
In this case, it is called an indefinite integral and is written:
Integral
The integrals discussed in this article are termed definite integrals.
The principles of integration were formulated independently by Isaac Newton and Gottfried Leibniz in the late 17th
century. Through the fundamental theorem of calculus, which they independently developed, integration is
connected with differentiation: if ƒ is a continuous real-valued function defined on a closed interval [a, b], then, once
an antiderivative F of ƒ is known, the definite integral of ƒ over that interval is given by
Integrals and derivatives became the basic tools of calculus, with numerous applications in science and engineering.
The founders of the calculus thought of the integral as an infinite sum of rectangles of infinitesimal width. A
rigorous mathematical definition of the integral was given by Bernhard Riemann. It is based on a limiting procedure
which approximates the area of a curvilinear region by breaking the region into thin vertical slabs. Beginning in the
nineteenth century, more sophisticated notions of integrals began to appear, where the type of the function as well as
the domain over which the integration is performed has been generalised. A line integral is defined for functions of
two or three variables, and the interval of integration [a, b] is replaced by a certain curve connecting two points on
the plane or in the space. In a surface integral, the curve is replaced by a piece of a surface in the three-dimensional
space. Integrals of differential forms play a fundamental role in modern differential geometry. These generalizations
of integrals first arose from the needs of physics, and they play an important role in the formulation of many physical
laws, notably those of electrodynamics. There are many modern concepts of integration, among these, the most
common is based on the abstract mathematical theory known as Lebesgue integration, developed by Henri Lebesgue.
History
Pre-calculus integration
Integration can be traced as far back as ancient Egypt ca. 1800 BC, with the Moscow Mathematical Papyrus
demonstrating knowledge of a formula for the volume of a pyramidal frustum. The first documented systematic
technique capable of determining integrals is the method of exhaustion of the ancient Greek astronomer Eudoxus
(ca. 370 BC), which sought to find areas and volumes by breaking them up into an infinite number of shapes for
which the area or volume was known. This method was further developed and employed by Archimedes in the 3rd
century BC and used to calculate areas for parabolas and an approximation to the area of a circle. Similar methods
were independently developed in China around the 3rd century AD by Liu Hui, who used it to find the area of the
circle. This method was later used in the 5th century by Chinese father-and-son mathematicians Zu Chongzhi and Zu
Geng to find the volume of a sphere.[1]
The next major step in integral calculus came from the Abbasid Caliphate when the 11th century mathematician Ibn
al-Haytham (known as Alhazen in Europe) devised what is now known as "Alhazen's problem", which leads to an
equation of the fourth degree, in his Book of Optics. While solving this problem, he applied mathematical induction
to find the formula for sums of fourth powers, by a method that can be generalized to sums of arbitrary natural
powers; then he used this formula to find the volume of a paraboloid (in modern terminology, he integrated a
polynomial of degree 4).[2] Some ideas of integral calculus are also found in the Siddhanta Shiromani, a 12th century
astronomy text by Indian mathematician Bhāskara II.
The next significant advances in integral calculus did not begin to appear until the 16th century. At this time the
work of Cavalieri with his method of indivisibles, and work by Fermat, began to lay the foundations of modern
calculus, with Cavalieri computing the integrals of xn up to degree n = 9 in Cavalieri's quadrature formula. Further
steps were made in the early 17th century by Barrow and Torricelli, who provided the first hints of a connection
between integration and differentiation. Barrow provided the first proof of the fundamental theorem of calculus.
155
Integral
Wallis generalized Cavalieri's method, computing integrals of x to a general power, including negative powers and
fractional powers.
At around the same time, there was also a great deal of work being done by Japanese mathematicians, particularly by
Seki Kōwa.[3] He made a number of contributions, namely in methods of determining areas of figures using
integrals, extending the method of exhaustion.
Newton and Leibniz
The major advance in integration came in the 17th century with the independent discovery of the fundamental
theorem of calculus by Newton and Leibniz. The theorem demonstrates a connection between integration and
differentiation. This connection, combined with the comparative ease of differentiation, can be exploited to calculate
integrals. In particular, the fundamental theorem of calculus allows one to solve a much broader class of problems.
Equal in importance is the comprehensive mathematical framework that both Newton and Leibniz developed. Given
the name infinitesimal calculus, it allowed for precise analysis of functions within continuous domains. This
framework eventually became modern calculus, whose notation for integrals is drawn directly from the work of
Leibniz.
Formalizing integrals
While Newton and Leibniz provided a systematic approach to integration, their work lacked a degree of rigour.
Bishop Berkeley memorably attacked the vanishing increments used by Newton, calling them "ghosts of departed
quantities". Calculus acquired a firmer footing with the development of limits. Integration was first rigorously
formalized, using limits, by Riemann. Although all bounded piecewise continuous functions are Riemann integrable
on a bounded interval, subsequently more general functions were considered – particularly in the context of Fourier
analysis – to which Riemann's definition does not apply, and Lebesgue formulated a different definition of integral,
founded in measure theory (a subfield of real analysis). Other definitions of integral, extending Riemann's and
Lebesgue's approaches, were proposed. These approaches based on the real number system are the ones most
common today, but alternative approaches exist, such as a definition of integral as the standard part of an infinite
Riemann sum, based on the hyperreal number system.
Notation
Isaac Newton used a small vertical bar above a variable to indicate integration, or placed the variable inside a box.
The vertical bar was easily confused with or
, which Newton used to indicate differentiation, and the box
notation was difficult for printers to reproduce, so these notations were not widely adopted.
The modern notation for the indefinite integral was introduced by Gottfried Leibniz in 1675 (Burton 1988, p. 359;
Leibniz 1899, p. 154). He adapted the integral symbol, ∫, from the letter ſ (long s), standing for summa (written as
ſumma; Latin for "sum" or "total"). The modern notation for the definite integral, with limits above and below the
integral sign, was first used by Joseph Fourier in Mémoires of the French Academy around 1819–20, reprinted in his
book of 1822 (Cajori 1929, pp. 249–250; Fourier 1822, §231).
Terminology and notation
If a function has an integral, it is said to be integrable. The function for which the integral is calculated is called the
integrand. The region over which a function is being integrated is called the domain of integration. Usually this
domain will be an interval, in which case it is enough to give the limits of that interval, which are called the limits of
integration. If the integral does not have a domain of integration, it is considered indefinite (one with a domain is
considered definite). In general, the integrand may be a function of more than one variable, and the domain of
integration may be an area, volume, a higher dimensional region, or even an abstract space that does not have a
geometric structure in any usual sense (such as a sample space in probability theory).
156
Integral
157
The simplest case, the integral of a real-valued function f of one real variable x on the interval [a, b], is denoted by
The ∫ sign represents integration; a and b are the lower limit and upper limit, respectively, of integration, defining
the domain of integration; f is the integrand, to be evaluated as x varies over the interval [a,b]; and dx is the variable
of integration. In correct mathematical typography, the dx is separated from the integrand by a space (as shown).
Some authors use an upright d (that is, dx instead of dx).
The variable of integration dx has different interpretations depending on the theory being used. For example, it can
be seen as strictly a notation indicating that x is a dummy variable of integration, as a reflection of the weights in the
Riemann sum, a measure (in Lebesgue integration and its extensions), an infinitesimal (in non-standard analysis) or
as an independent mathematical quantity: a differential form. More complicated cases may vary the notation slightly.
In the modern Arabic mathematical notation, which aims at pre-university levels of education in the Arab world and
is written from right to left, a reflected integral symbol
is used (W3C 2006).
Introduction
Integrals appear in many practical situations. If a swimming pool is rectangular with a flat bottom, then from its
length, width, and depth we can easily determine the volume of water it can contain (to fill it), the area of its surface
(to cover it), and the length of its edge (to rope it). But if it is oval with a rounded bottom, all of these quantities call
for integrals. Practical approximations may suffice for such trivial examples, but precision engineering (of any
discipline) requires exact and rigorous values for these elements.
To start off, consider the curve y = f(x) between x = 0 and x = 1, with
f(x) = √x. We ask:
What is the area under the function f, in the interval from 0 to 1?
and call this (yet unknown) area the integral of f. The notation for this
integral will be
Approximations to integral of √x from 0 to 1,
with ■ 5 right samples (above) and ■ 12 left
samples (below)
As a first approximation, look at the unit square given by the sides x = 0 to x = 1 and y = f(0) = 0 and y = f(1) = 1. Its
area is exactly 1. As it is, the true value of the integral must be somewhat less. Decreasing the width of the
approximation rectangles shall give a better result; so cross the interval in five steps, using the approximation points
0, 1⁄5, 2⁄5, and so on to 1. Fit a box for each step using the right end height of each curve piece, thus √1⁄5, √2⁄5, and so
on to √1 = 1. Summing the areas of these rectangles, we get a better approximation for the sought integral, namely
Notice that we are taking a sum of finitely many function values of f, multiplied with the differences of two
subsequent approximation points. We can easily see that the approximation is still too large. Using more steps
Integral
produces a closer approximation, but will never be exact: replacing the 5 subintervals by twelve as depicted, we will
get an approximate value for the area of 0.6203, which is too small. The key idea is the transition from adding
finitely many differences of approximation points multiplied by their respective function values to using infinitely
many fine, or infinitesimal steps.
As for the actual calculation of integrals, the fundamental theorem of calculus, due to Newton and Leibniz, is the
fundamental link between the operations of differentiating and integrating. Applied to the square root curve, f(x) =
x1/2, it says to look at the antiderivative F(x) = 2⁄3x3/2, and simply take F(1) − F(0), where 0 and 1 are the boundaries
of the interval [0,1]. So the exact value of the area under the curve is computed formally as
(This is a case of a general rule, that for f(x) = xq, with q ≠ −1, the related function, the so-called antiderivative is
F(x) = (xq+1)/(q + 1).)
The notation
conceives the integral as a weighted sum, denoted by the elongated s, of function values, f(x), multiplied by
infinitesimal step widths, the so-called differentials, denoted by dx. The multiplication sign is usually omitted.
Historically, after the failure of early efforts to rigorously interpret infinitesimals, Riemann formally defined
integrals as a limit of weighted sums, so that the dx suggested the limit of a difference (namely, the interval width).
Shortcomings of Riemann's dependence on intervals and continuity motivated newer definitions, especially the
Lebesgue integral, which is founded on an ability to extend the idea of "measure" in much more flexible ways. Thus
the notation
refers to a weighted sum in which the function values are partitioned, with μ measuring the weight to be assigned to
each value. Here A denotes the region of integration.
Differential geometry, with its "calculus on manifolds", gives the familiar notation yet another interpretation. Now
f(x) and dx become a differential form, ω = f(x) dx, a new differential operator d, known as the exterior derivative is
introduced, and the fundamental theorem becomes the more general Stokes' theorem,
from which Green's theorem, the divergence theorem, and the fundamental theorem of calculus follow.
More recently, infinitesimals have reappeared with rigor, through modern innovations such as non-standard analysis.
Not only do these methods vindicate the intuitions of the pioneers; they also lead to new mathematics.
Although there are differences between these conceptions of integral, there is considerable overlap. Thus, the area of
the surface of the oval swimming pool can be handled as a geometric ellipse, a sum of infinitesimals, a Riemann
integral, a Lebesgue integral, or as a manifold with a differential form. The calculated result will be the same for all.
158
Integral
159
Formal definitions
There are many ways of formally defining an integral, not all of which are equivalent. The differences exist mostly
to deal with differing special cases which may not be integrable under other definitions, but also occasionally for
pedagogical reasons. The most commonly used definitions of integral are Riemann integrals and Lebesgue integrals.
Riemann integral
The Riemann integral is defined in terms of Riemann sums of
functions with respect to tagged partitions of an interval. Let [a,b] be a
closed interval of the real line; then a tagged partition of [a,b] is a
finite sequence
Integral approached as Riemann sum based on
tagged partition, with irregular sampling positions
and widths (max in red). True value is 3.76;
estimate is 3.648.
This partitions the interval [a,b] into n sub-intervals
[xi−1, xi] indexed by i, each of which is "tagged" with a
distinguished point ti ∈ [xi−1, xi]. A Riemann sum of a
function f with respect to such a tagged partition is
defined as
Riemann sums converging as intervals halve, whether sampled at
■ right, ■ minimum, ■ maximum, or ■ left.
thus each term of the sum is the area of a rectangle with height equal to the function value at the distinguished point
of the given sub-interval, and width the same as the sub-interval width. Let Δi = xi−xi−1 be the width of sub-interval
Integral
160
i; then the mesh of such a tagged partition is the width of the largest sub-interval formed by the partition,
maxi=1…n Δi. The Riemann integral of a function f over the interval [a,b] is equal to S if:
For all ε > 0 there exists δ > 0 such that, for any tagged partition [a,b] with mesh less than δ, we have
When the chosen tags give the maximum (respectively, minimum) value of each interval, the Riemann sum becomes
an upper (respectively, lower) Darboux sum, suggesting the close connection between the Riemann integral and the
Darboux integral.
Lebesgue integral
The Riemann integral is not defined for a wide range of
functions and situations of importance in applications
(and of interest in theory). For example, the Riemann
integral can easily integrate density to find the mass of
a steel beam, but cannot accommodate a steel ball
resting on it. This motivates other definitions, under
which a broader assortment of functions are integrable
(Rudin 1987). The Lebesgue integral, in particular,
achieves great flexibility by directing attention to the
weights in the weighted sum.
Riemann-Darboux's integration (blue) and Lebesgue integration
The definition of the Lebesgue integral thus begins
(red).
with a measure, μ. In the simplest case, the Lebesgue
measure μ(A) of an interval A = [a,b] is its width, b − a,
so that the Lebesgue integral agrees with the (proper) Riemann integral when both exist. In more complicated cases,
the sets being measured can be highly fragmented, with no continuity and no resemblance to intervals.
To exploit this flexibility, Lebesgue integrals reverse the approach to the weighted sum. As Folland (1984, p. 56)
puts it, "To compute the Riemann integral of f, one partitions the domain [a,b] into subintervals", while in the
Lebesgue integral, "one is in effect partitioning the range of f".
One common approach first defines the integral of the indicator function of a measurable set A by:
.
This extends by linearity to a measurable simple function s, which attains only a finite number, n, of distinct
non-negative values:
(where the image of Ai under the simple function s is the constant value ai). Thus if E is a measurable set one defines
Then for any non-negative measurable function f one defines
Integral
that is, the integral of f is set to be the supremum of all the integrals of simple functions that are less than or equal to
f. A general measurable function f is split into its positive and negative values by defining
Finally, f is Lebesgue integrable if
and then the integral is defined by
When the measure space on which the functions are defined is also a locally compact topological space (as is the
case with the real numbers R), measures compatible with the topology in a suitable sense (Radon measures, of which
the Lebesgue measure is an example) and integral with respect to them can be defined differently, starting from the
integrals of continuous functions with compact support. More precisely, the compactly supported functions form a
vector space that carries a natural topology, and a (Radon) measure can be defined as any continuous linear
functional on this space; the value of a measure at a compactly supported function is then also by definition the
integral of the function. One then proceeds to expand the measure (the integral) to more general functions by
continuity, and defines the measure of a set as the integral of its indicator function. This is the approach taken by
Bourbaki (2004) and a certain number of other authors. For details see Radon measures.
Other integrals
Although the Riemann and Lebesgue integrals are the most widely used definitions of the integral, a number of
others exist, including:
• The Riemann–Stieltjes integral, an extension of the Riemann integral.
• The Lebesgue-Stieltjes integral, further developed by Johann Radon, which generalizes the Riemann–Stieltjes and
Lebesgue integrals.
• The Daniell integral, which subsumes the Lebesgue integral and Lebesgue-Stieltjes integral without the
dependence on measures.
• The Henstock-Kurzweil integral, variously defined by Arnaud Denjoy, Oskar Perron, and (most elegantly, as the
gauge integral) Jaroslav Kurzweil, and developed by Ralph Henstock.
• The Itō integral and Stratonovich integral, which define integration with respect to semimartingales such as
Brownian motion.
• The Young integral, which is a kind of Riemann-Stieltjes integral with respect to certain functions of unbounded
variation.
• The rough path integral defined for functions equipped with some additional "rough path" structure, generalizing
stochastic integration against both semimartingales and processes such as the fractional Brownian motion.
161
Integral
162
Properties
Linearity
• The collection of Riemann integrable functions on a closed interval [a, b] forms a vector space under the
operations of pointwise addition and multiplication by a scalar, and the operation of integration
is a linear functional on this vector space. Thus, firstly, the collection of integrable functions is closed under
taking linear combinations; and, secondly, the integral of a linear combination is the linear combination of the
integrals,
• Similarly, the set of real-valued Lebesgue integrable functions on a given measure space E with measure μ is
closed under taking linear combinations and hence form a vector space, and the Lebesgue integral
is a linear functional on this vector space, so that
• More generally, consider the vector space of all measurable functions on a measure space (E,μ), taking values in a
locally compact complete topological vector space V over a locally compact topological field K, f : E → V. Then
one may define an abstract integration map assigning to each function f an element of V or the symbol ∞,
that is compatible with linear combinations. In this situation the linearity holds for the subspace of functions
whose integral is an element of V (i.e. "finite"). The most important special cases arise when K is R, C, or a
finite extension of the field Qp of p-adic numbers, and V is a finite-dimensional vector space over K, and when
K=C and V is a complex Hilbert space.
Linearity, together with some natural continuity properties and normalisation for a certain class of "simple"
functions, may be used to give an alternative definition of the integral. This is the approach of Daniell for the case of
real-valued functions on a set X, generalized by Nicolas Bourbaki to functions with values in a locally compact
topological vector space. See (Hildebrandt 1953) for an axiomatic characterisation of the integral.
Inequalities for integrals
A number of general inequalities hold for Riemann-integrable functions defined on a closed and bounded interval [a,
b] and can be generalized to other notions of integral (Lebesgue and Daniell).
• Upper and lower bounds. An integrable function f on [a, b], is necessarily bounded on that interval. Thus there
are real numbers m and M so that m ≤ f (x) ≤ M for all x in [a, b]. Since the lower and upper sums of f over [a, b]
are therefore bounded by, respectively, m(b − a) and M(b − a), it follows that
• Inequalities between functions. If f(x) ≤ g(x) for each x in [a, b] then each of the upper and lower sums of f is
bounded above by the upper and lower sums, respectively, of g. Thus
Integral
163
This is a generalization of the above inequalities, as M(b − a) is the integral of the constant function with value
M over [a, b].
In addition, if the inequality between functions is strict, then the inequality between integrals is also strict.
That is, if f(x) < g(x) for each x in [a, b], then
• Subintervals. If [c, d] is a subinterval of [a, b] and f(x) is non-negative for all x, then
• Products and absolute values of functions. If f and g are two functions then we may consider their pointwise
products and powers, and absolute values:
If f is Riemann-integrable on [a, b] then the same is true for |f|, and
Moreover, if f and g are both Riemann-integrable then f 2, g 2, and fg are also Riemann-integrable, and
This inequality, known as the Cauchy–Schwarz inequality, plays a prominent role in Hilbert space theory,
where the left hand side is interpreted as the inner product of two square-integrable functions f and g on the
interval [a, b].
• Hölder's inequality. Suppose that p and q are two real numbers, 1 ≤ p, q ≤ ∞ with 1/p + 1/q = 1, and f and g are
two Riemann-integrable functions. Then the functions |f|p and |g|q are also integrable and the following Hölder's
inequality holds:
For p = q = 2, Hölder's inequality becomes the Cauchy–Schwarz inequality.
• Minkowski inequality. Suppose that p ≥ 1 is a real number and f and g are Riemann-integrable functions. Then |f|p,
|g|p and |f + g|p are also Riemann integrable and the following Minkowski inequality holds:
An analogue of this inequality for Lebesgue integral is used in construction of Lp spaces.
Conventions
In this section f is a real-valued Riemann-integrable function. The integral
over an interval [a, b] is defined if a < b. This means that the upper and lower sums of the function f are evaluated on
a partition a = x0 ≤ x1 ≤ . . . ≤ xn = b whose values xi are increasing. Geometrically, this signifies that integration
takes place "left to right", evaluating f within intervals [x i , x i +1] where an interval with a higher index lies to the
right of one with a lower index. The values a and b, the end-points of the interval, are called the limits of integration
of f. Integrals can also be defined if a > b:
• Reversing limits of integration. If a > b then define
Integral
164
This, with a = b, implies:
• Integrals over intervals of length zero. If a is a real number then
The first convention is necessary in consideration of taking integrals over subintervals of [a, b]; the second says that
an integral taken over a degenerate interval, or a point, should be zero. One reason for the first convention is that the
integrability of f on an interval [a, b] implies that f is integrable on any subinterval [c, d], but in particular integrals
have the property that:
• Additivity of integration on intervals. If c is any element of [a, b], then
With the first convention the resulting relation
is then well-defined for any cyclic permutation of a, b, and c.
Instead of viewing the above as conventions, one can also adopt the point of view that integration is performed of
differential forms on oriented manifolds only. If M is such an oriented m-dimensional manifold, and M is the same
manifold with opposed orientation and ω is an m-form, then one has:
These conventions correspond to interpreting the integrand as a differential form, integrated over a chain. In measure
theory, by contrast, one interprets the integrand as a function f with respect to a measure
and integrates over a
subset A, without any notion of orientation; one writes
to indicate integration over a subset
A. This is a minor distinction in one dimension, but becomes subtler on higher dimensional manifolds; see
Differential form: Relation with measures for details.
Fundamental theorem of calculus
The fundamental theorem of calculus is the statement that differentiation and integration are inverse operations: if a
continuous function is first integrated and then differentiated, the original function is retrieved. An important
consequence, sometimes called the second fundamental theorem of calculus, allows one to compute integrals by
using an antiderivative of the function to be integrated.
Statements of theorems
• Fundamental theorem of calculus. Let ƒ be a continuous real-valued function defined on a closed interval [a, b].
Let F be the function defined, for all x in [a, b], by
Then, F is continuous on [a, b], differentiable on the open interval (a, b), and
for all x in (a, b).
Integral
165
• Second fundamental theorem of calculus. Let ƒ be a real-valued function defined on a closed interval [a, b] that
admits an antiderivative g on [a, b]. That is, ƒ and g are functions such that for all x in [a, b],
If ƒ is integrable on [a, b] then
Extensions
Improper integrals
A "proper" Riemann integral assumes the integrand is defined and
finite on a closed and bounded interval, bracketed by the limits of
integration. An improper integral occurs when one or more of these
conditions is not satisfied. In some cases such integrals may be defined
by considering the limit of a sequence of proper Riemann integrals on
progressively larger intervals.
If the interval is unbounded, for instance at its upper end, then the
improper integral is the limit as that endpoint goes to infinity.
The improper integral
has unbounded intervals for both domain and
range.
If the integrand is only defined or finite on a half-open interval, for instance (a,b], then again a limit may provide a
finite result.
That is, the improper integral is the limit of proper integrals as one endpoint of the interval of integration approaches
either a specified real number, or ∞, or −∞. In more complicated cases, limits are required at both endpoints, or at
interior points.
Consider, for example, the function
integrated from 0 to ∞ (shown right). At the lower bound, as x goes to
0 the function goes to ∞, and the upper bound is itself ∞, though the function goes to 0. Thus this is a doubly
improper integral. Integrated, say, from 1 to 3, an ordinary Riemann sum suffices to produce a result of . To
integrate from 1 to ∞, a Riemann sum is not possible. However, any finite upper bound, say t (with t > 1), gives a
well-defined result,
. This has a finite limit as t goes to infinity, namely . Similarly, the integral
from 1⁄3 to 1 allows a Riemann sum as well, coincidentally again producing . Replacing 1⁄3 by an arbitrary positive
value s (with s < 1) is equally safe, giving
. This, too, has a finite limit as s goes to zero,
namely . Combining the limits of the two fragments, the result of this improper integral is
Integral
166
This process does not guarantee success; a limit may fail to exist, or may be unbounded. For example, over the
bounded interval 0 to 1 the integral of does not converge; and over the unbounded interval 1 to ∞ the integral of
does not converge.
It may also happen that an integrand is unbounded at an interior point,
in which case the integral must be split at that point, and the limit
integrals on both sides must exist and must be bounded. Thus
The improper integral
is unbounded internally, but both left and right
limits exist.
But the similar integral
cannot be assigned a value in this way, as the integrals above and below zero do not independently converge.
(However, see Cauchy principal value.)
Integral
167
Multiple integration
Integrals can be taken over regions other than intervals. In general, an
integral over a set E of a function f is written:
Double integral as volume under a surface.
Here x need not be a real number, but can be another suitable quantity, for instance, a vector in R3. Fubini's theorem
shows that such integrals can be rewritten as an iterated integral. In other words, the integral can be calculated by
integrating one coordinate at a time.
Just as the definite integral of a positive function of one variable represents the area of the region between the graph
of the function and the x-axis, the double integral of a positive function of two variables represents the volume of the
region between the surface defined by the function and the plane which contains its domain. (The same volume can
be obtained via the triple integral — the integral of a function in three variables — of the constant function f(x, y, z)
= 1 over the above mentioned region between the surface and the plane.) If the number of variables is higher, then
the integral represents a hypervolume, a volume of a solid of more than three dimensions that cannot be graphed.
For example, the volume of the cuboid of sides 4 × 6 × 5 may be obtained in two ways:
• By the double integral
of the function f(x, y) = 5 calculated in the region D in the xy-plane which is the base of the cuboid. For
example, if a rectangular base of such a cuboid is given via the xy inequalities 3 ≤ x ≤ 7, 4 ≤ y ≤ 10, our above
double integral now reads
From here, integration is conducted with respect to either x or y first; in this example, integration is first done
with respect to x as the interval corresponding to x is the inner integral. Once the first integration is completed
via the
method or otherwise, the result is again integrated with respect to the other variable.
The result will equate to the volume under the surface.
• By the triple integral
of the constant function 1 calculated on the cuboid itself.
Integral
168
Line integrals
The concept of an integral can be extended to more general domains of
integration, such as curved lines and surfaces. Such integrals are
known as line integrals and surface integrals respectively. These have
important applications in physics, as when dealing with vector fields.
A line integral (sometimes called a path integral) is an integral where
the function to be integrated is evaluated along a curve. Various
different line integrals are in use. In the case of a closed curve it is also
called a contour integral.
The function to be integrated may be a scalar field or a vector field.
The value of the line integral is the sum of values of the field at all
A line integral sums together elements along a
points on the curve, weighted by some scalar function on the curve
curve.
(commonly arc length or, for a vector field, the scalar product of the
vector field with a differential vector in the curve). This weighting
distinguishes the line integral from simpler integrals defined on intervals. Many simple formulas in physics have
natural continuous analogs in terms of line integrals; for example, the fact that work is equal to force, F, multiplied
by displacement, s, may be expressed (in terms of vector quantities) as:
For an object moving along a path in a vector field
such as an electric field or gravitational field, the total work
done by the field on the object is obtained by summing up the differential work done in moving from
to
. This gives the line integral
Surface integrals
A surface integral is a definite integral taken over a surface (which
may be a curved set in space); it can be thought of as the double
integral analog of the line integral. The function to be integrated may
be a scalar field or a vector field. The value of the surface integral is
the sum of the field at all points on the surface. This can be achieved
by splitting the surface into surface elements, which provide the
partitioning for Riemann sums.
For an example of applications of surface integrals, consider a vector
The definition of surface integral relies on
field v on a surface S; that is, for each point x in S, v(x) is a vector.
splitting the surface into small surface elements.
Imagine that we have a fluid flowing through S, such that v(x)
determines the velocity of the fluid at x. The flux is defined as the
quantity of fluid flowing through S in unit amount of time. To find the flux, we need to take the dot product of v with
the unit surface normal to S at each point, which will give us a scalar field, which we integrate over the surface:
The fluid flux in this example may be from a physical fluid such as water or air, or from electrical or magnetic flux.
Thus surface integrals have applications in physics, particularly with the classical theory of electromagnetism.
Integral
Integrals of differential forms
A differential form is a mathematical concept in the fields of multivariable calculus, differential topology and
tensors. The modern notation for the differential form, as well as the idea of the differential forms as being the
wedge products of exterior derivatives forming an exterior algebra, was introduced by Élie Cartan.
We initially work in an open set in Rn. A 0-form is defined to be a smooth function f. When we integrate a function f
over an m-dimensional subspace S of Rn, we write it as
(The superscripts are indices, not exponents.) We can consider dx1 through dxn to be formal objects themselves,
rather than tags appended to make integrals look like Riemann sums. Alternatively, we can view them as covectors,
and thus a measure of "density" (hence integrable in a general sense). We call the dx1, …,dxn basic 1-forms.
We define the wedge product, "∧", a bilinear "multiplication" operator on these elements, with the alternating
property that
for all indices a. Note that alternation along with linearity and associativity implies dxb∧dxa = −dxa∧dxb. This also
ensures that the result of the wedge product has an orientation.
We define the set of all these products to be basic 2-forms, and similarly we define the set of products of the form
dxa∧dxb∧dxc to be basic 3-forms. A general k-form is then a weighted sum of basic k-forms, where the weights are
the smooth functions f. Together these form a vector space with basic k-forms as the basis vectors, and 0-forms
(smooth functions) as the field of scalars. The wedge product then extends to k-forms in the natural way. Over Rn at
most n covectors can be linearly independent, thus a k-form with k > n will always be zero, by the alternating
property.
In addition to the wedge product, there is also the exterior derivative operator d. This operator maps k-forms to
(k+1)-forms. For a k-form ω = f dxa over Rn, we define the action of d by:
with extension to general k-forms occurring linearly.
This more general approach allows for a more natural coordinate-free approach to integration on manifolds. It also
allows for a natural generalisation of the fundamental theorem of calculus, called Stokes' theorem, which we may
state as
where ω is a general k-form, and ∂Ω denotes the boundary of the region Ω. Thus, in the case that ω is a 0-form and
Ω is a closed interval of the real line, this reduces to the fundamental theorem of calculus. In the case that ω is a
1-form and Ω is a two-dimensional region in the plane, the theorem reduces to Green's theorem. Similarly, using
2-forms, and 3-forms and Hodge duality, we can arrive at Stokes' theorem and the divergence theorem. In this way
we can see that differential forms provide a powerful unifying view of integration.
169
Integral
170
Summations
The discrete equivalent of integration is summation. Summations and integrals can be put on the same foundations
using the theory of Lebesgue integrals or time scale calculus.
Methods
Computing integrals
The most basic technique for computing definite integrals of one real variable is based on the fundamental theorem
of calculus. Let f(x) be the function of x to be integrated over a given interval [a, b]. Then, find an antiderivative of f;
that is, a function F such that F' = f on the interval. Provided the integrand and integral have no singularities on the
path of integration, by the fundamental theorem of calculus,
The integral is not actually the antiderivative, but the fundamental theorem provides a way to use antiderivatives to
evaluate definite integrals.
The most difficult step is usually to find the antiderivative of f. It is rarely possible to glance at a function and write
down its antiderivative. More often, it is necessary to use one of the many techniques that have been developed to
evaluate integrals. Most of these techniques rewrite one integral as a different one which is hopefully more tractable.
Techniques include:
•
•
•
•
•
•
•
•
•
•
Integration by substitution
Integration by parts
Changing the order of integration
Integration by trigonometric substitution
Integration by partial fractions
Integration by reduction formulae
Integration using parametric derivatives
Integration using Euler's formula
Differentiation under the integral sign
Contour integration
Alternate methods exist to compute more complex integrals. Many nonelementary integrals can be expanded in a
Taylor series and integrated term by term. Occasionally, the resulting infinite series can be summed analytically. The
method of convolution using Meijer G-functions can also be used, assuming that the integrand can be written as a
product of Meijer G-functions. There are also many less common ways of calculating definite integrals; for instance,
Parseval's identity can be used to transform an integral over a rectangular region into an infinite sum. Occasionally,
an integral can be evaluated by a trick; for an example of this, see Gaussian integral.
Computations of volumes of solids of revolution can usually be done with disk integration or shell integration.
Specific results which have been worked out by various techniques are collected in the list of integrals.
Symbolic algorithms
Many problems in mathematics, physics, and engineering involve integration where an explicit formula for the
integral is desired. Extensive tables of integrals have been compiled and published over the years for this purpose.
With the spread of computers, many professionals, educators, and students have turned to computer algebra systems
that are specifically designed to perform difficult or tedious tasks, including integration. Symbolic integration has
been one of the motivations for the development of the first such systems, like Macsyma.
A major mathematical difficulty in symbolic integration is that in many cases, a closed formula for the antiderivative
of a rather simple-looking function does not exist. For instance, it is known that the antiderivatives of the functions
exp ( x2), xx and sin x /x cannot be expressed in the closed form involving only rational and exponential functions,
Integral
logarithm, trigonometric and inverse trigonometric functions, and the operations of multiplication and composition;
in other words, none of the three given functions is integrable in elementary functions, which are the functions which
may be built from rational functions, roots of a polynomial, logarithm, and exponential functions. Risch algorithm
provides a general criterion to determine whether the antiderivative of an elementary function is elementary, and, if it
is, to compute it. Unfortunately, it turns out that functions with closed expressions of antiderivatives are the
exception rather than the rule. Consequently, computerized algebra systems have no hope of being able to find an
antiderivative for a randomly constructed elementary function. On the positive side, if the 'building blocks' for
antiderivatives are fixed in advance, it may be still be possible to decide whether the antiderivative of a given
function can be expressed using these blocks and operations of multiplication and composition, and to find the
symbolic answer whenever it exists. The Risch algorithm, implemented in Mathematica and other computer algebra
systems, does just that for functions and antiderivatives built from rational functions, radicals, logarithm, and
exponential functions.
Some special integrands occur often enough to warrant special study. In particular, it may be useful to have, in the
set of antiderivatives, the special functions of physics (like the Legendre functions, the hypergeometric function, the
Gamma function, the Incomplete Gamma function and so on - see Symbolic integration for more details). Extending
the Risch's algorithm to include such functions is possible but challenging and has been an active research subject.
More recently a new approach has emerged, using D-finite function, which are the solutions of linear differential
equations with polynomial coefficients. Most of the elementary and special functions are D-finite and the integral of
a D-finite function is also a D-finite function. This provide an algorithm to express the antiderivative of a D-finite
function as the solution of a differential equation.
This theory allows also to compute a definite integrals of a D-function as the sum of a series given by the first
coefficients and an algorithm to compute any coefficient.[4]
Numerical quadrature
The integrals encountered in a basic calculus course are deliberately chosen for simplicity; those found in real
applications are not always so accommodating. Some integrals cannot be found exactly, some require special
functions which themselves are a challenge to compute, and others are so complex that finding the exact answer is
too slow. This motivates the study and application of numerical methods for approximating integrals, which today
use floating-point arithmetic on digital electronic computers. Many of the ideas arose much earlier, for hand
calculations; but the speed of general-purpose computers like the ENIAC created a need for improvements.
The goals of numerical integration are accuracy, reliability, efficiency, and generality. Sophisticated methods can
vastly outperform a naive method by all four measures (Dahlquist & Björck 2008; Kahaner, Moler & Nash 1989;
Stoer & Bulirsch 2002). Consider, for example, the integral
which has the exact answer 94⁄25 = 3.76. (In ordinary practice the answer is not known in advance, so an important
task — not explored here — is to decide when an approximation is good enough.) A “calculus book” approach
divides the integration range into, say, 16 equal pieces, and computes function values.
171
Integral
172
Spaced function values
x
−2.00
f(x) 2.22800
x
f(x)
−1.50
−1.00
−0.50
0.00
0.50
1.00
1.50
2.00
2.45663 2.67200 2.32475 0.64400 −0.92575 −0.94000 −0.16963 0.83600
−1.75
−1.25
−0.75
−0.25
0.25
0.75
1.25
1.75
2.33041 2.58562 2.62934 1.64019 −0.32444 −1.09159 −0.60387 0.31734
Using the left end of each piece, the rectangle method sums 16
function values and multiplies by the step width, h, here 0.25, to get an
approximate value of 3.94325 for the integral. The accuracy is not
impressive, but calculus formally uses pieces of infinitesimal width, so
initially this may seem little cause for concern. Indeed, repeatedly
doubling the number of steps eventually produces an approximation of
3.76001. However, 218 pieces are required, a great computational
expense for such little accuracy; and a reach for greater accuracy can
force steps so small that arithmetic precision becomes an obstacle.
A better approach replaces the horizontal tops of the rectangles with
slanted tops touching the function at the ends of each piece. This
Numerical quadrature methods: ■ Rectangle,
trapezium rule is almost as easy to calculate; it sums all 17 function
■ Trapezoid, ■ Romberg, ■ Gauss
values, but weights the first and last by one half, and again multiplies
by the step width. This immediately improves the approximation to
3.76925, which is noticeably more accurate. Furthermore, only 210 pieces are needed to achieve 3.76000,
substantially less computation than the rectangle method for comparable accuracy.
Romberg's method builds on the trapezoid method to great effect. First, the step lengths are halved incrementally,
giving trapezoid approximations denoted by T(h0), T(h1), and so on, where hk+1 is half of hk. For each new step size,
only half the new function values need to be computed; the others carry over from the previous size (as shown in the
table above). But the really powerful idea is to interpolate a polynomial through the approximations, and extrapolate
to T(0). With this method a numerically exact answer here requires only four pieces (five function values)! The
Lagrange polynomial interpolating {hk,T(hk)}k=0…2 = {(4.00,6.128), (2.00,4.352), (1.00,3.908)} is 3.76+0.148h2,
producing the extrapolated value 3.76 at h = 0.
Gaussian quadrature often requires noticeably less work for superior accuracy. In this example, it can compute the
function values at just two x positions, ±2⁄√3, then double each value and sum to get the numerically exact answer.
The explanation for this dramatic success lies in error analysis, and a little luck. An n-point Gaussian method is exact
for polynomials of degree up to 2n−1. The function in this example is a degree 3 polynomial, plus a term that cancels
because the chosen endpoints are symmetric around zero. (Cancellation also benefits the Romberg method.)
Shifting the range left a little, so the integral is from −2.25 to 1.75, removes the symmetry. Nevertheless, the
trapezoid method is rather slow, the polynomial interpolation method of Romberg is acceptable, and the Gaussian
method requires the least work — if the number of points is known in advance. As well, rational interpolation can
use the same trapezoid evaluations as the Romberg method to greater effect.
Integral
173
Quadrature method cost comparison
Method
Points
Rel. Err.
Trapezoid
Romberg
Rational
Gauss
1048577
257
129
36
−5.3×10−13
−6.3×10−15
8.8×10−15
3.1×10−15
Value
In practice, each method must use extra evaluations to ensure an error bound on an unknown function; this tends to
offset some of the advantage of the pure Gaussian method, and motivates the popular Gauss–Kronrod quadrature
formulae. Symmetry can still be exploited by splitting this integral into two ranges, from −2.25 to −1.75 (no
symmetry), and from −1.75 to 1.75 (symmetry). More broadly, adaptive quadrature partitions a range into pieces
based on function properties, so that data points are concentrated where they are needed most.
Simpson's rule, named for Thomas Simpson (1710–1761), uses a parabolic curve to approximate integrals. In many
cases, it is more accurate than the trapezoidal rule and others. The rule states that
with an error of
The computation of higher-dimensional integrals (for example, volume calculations) makes important use of such
alternatives as Monte Carlo integration.
A calculus text is no substitute for numerical analysis, but the reverse is also true. Even the best adaptive numerical
code sometimes requires a user to help with the more demanding integrals. For example, improper integrals may
require a change of variable or methods that can avoid infinite function values, and known properties like symmetry
and periodicity may provide critical leverage.
Practical applications
Area under the curve (abbreviated AUC) is
frequently used in pharmacokinetics for
functions where the x-axis represents time
and the y-axis represents drug concentration.
For such functions, the area under the curve
usually correlates fairly well with the total
effect on the body that the drug will have. In
standard use, AUC is defined as either:
• AUC∞, the integral after a single dose
with a hypothetical infinite x-axis
• AUCτ, the integral in the time interval
between doses given regularly, and after
having reached steady state.
The time course of drug plasma concentrations over 96 hours following oral
administrations every 24 hours. Note that the AUC in steady state equals AUC∞
after the first dose.
Integral
Notes
[1] Shea, Marilyn (May 2007), Biography of Zu Chongzhi (http:/ / hua. umf. maine. edu/ China/ astronomy/ tianpage/ 0014ZuChongzhi9296bw.
html), University of Maine, , retrieved 9 January 2009
Katz, Victor J. (2004), A History of Mathematics, Brief Version, Addison-Wesley, pp. 125–126, ISBN 978-0-321-16193-2
[2] Victor J. Katz (1995), "Ideas of Calculus in Islam and India", Mathematics Magazine 68 (3): 163–174 [165–9 & 173–4]
[3] http:/ / www2. gol. com/ users/ coynerhm/ 0598rothman. html
[4] http:/ / algo. inria. fr/ chyzak/ mgfun. html
References
• Apostol, Tom M. (1967), Calculus, Vol. 1: One-Variable Calculus with an Introduction to Linear Algebra (2nd
ed.), Wiley, ISBN 978-0-471-00005-1
• Bourbaki, Nicolas (2004), Integration I, Springer Verlag, ISBN 3-540-41129-1. In particular chapters III and IV.
• Burton, David M. (2005), The History of Mathematics: An Introduction (6th ed.), McGraw-Hill, p. 359,
ISBN 978-0-07-305189-5
• Cajori, Florian (1929), A History Of Mathematical Notations Volume II (http://www.archive.org/details/
historyofmathema027671mbp), Open Court Publishing, pp. 247–252, ISBN 978-0-486-67766-8
• Dahlquist, Germund; Björck, Åke (2008), "Chapter 5: Numerical Integration" (http://www.mai.liu.se/~akbjo/
NMbook.html), Numerical Methods in Scientific Computing, Volume I, Philadelphia: SIAM
• Folland, Gerald B. (1984), Real Analysis: Modern Techniques and Their Applications (1st ed.), John Wiley &
Sons, ISBN 978-0-471-80958-6
• Fourier, Jean Baptiste Joseph (1822), Théorie analytique de la chaleur (http://books.google.com/
books?id=TDQJAAAAIAAJ), Chez Firmin Didot, père et fils, p. §231
Available in translation as Fourier, Joseph (1878), The analytical theory of heat (http://www.archive.org/
details/analyticaltheory00fourrich), Freeman, Alexander (trans.), Cambridge University Press, pp. 200–201
• Heath, T. L., ed. (2002), The Works of Archimedes (http://www.archive.org/details/
worksofarchimede029517mbp), Dover, ISBN 978-0-486-42084-4
(Originally published by Cambridge University Press, 1897, based on J. L. Heiberg's Greek version.)
• Hildebrandt, T. H. (1953), "Integration in abstract spaces" (http://projecteuclid.org/euclid.bams/1183517761),
Bulletin of the American Mathematical Society 59 (2): 111–139, ISSN 0273-0979
• Kahaner, David; Moler, Cleve; Nash, Stephen (1989), "Chapter 5: Numerical Quadrature", Numerical Methods
and Software, Prentice Hall, ISBN 978-0-13-627258-8
• Leibniz, Gottfried Wilhelm (1899), Gerhardt, Karl Immanuel, ed., Der Briefwechsel von Gottfried Wilhelm
Leibniz mit Mathematikern. Erster Band (http://name.umdl.umich.edu/AAX2762.0001.001), Berlin: Mayer
& Müller
• Miller, Jeff, Earliest Uses of Symbols of Calculus (http://jeff560.tripod.com/calculus.html), retrieved
2009-11-22
• O’Connor, J. J.; Robertson, E. F. (1996), A history of the calculus (http://www-history.mcs.st-andrews.ac.uk/
HistTopics/The_rise_of_calculus.html), retrieved 2007-07-09
• Rudin, Walter (1987), "Chapter 1: Abstract Integration", Real and Complex Analysis (International ed.),
McGraw-Hill, ISBN 978-0-07-100276-9
• Saks, Stanisław (1964), Theory of the integral (http://matwbn.icm.edu.pl/kstresc.php?tom=7&wyd=10&
jez=) (English translation by L. C. Young. With two additional notes by Stefan Banach. Second revised ed.), New
York: Dover
• Stoer, Josef; Bulirsch, Roland (2002), "Chapter 3: Topics in Integration", Introduction to Numerical Analysis (3rd
ed.), Springer, ISBN 978-0-387-95452-3.
• W3C (2006), Arabic mathematical notation (http://www.w3.org/TR/arabic-math/)
174
Integral
External links
• Riemann Sum (http://mathworld.wolfram.com/RiemannSum.html) by Wolfram Research
• Introduction to definite integrals (http://www.khanacademy.org/video/
introduction-to-definite-integrals?playlist=Calculus) by Khan Academy
Online books
• Keisler, H. Jerome, Elementary Calculus: An Approach Using Infinitesimals (http://www.math.wisc.edu/
~keisler/calc.html), University of Wisconsin
• Stroyan, K.D., A Brief Introduction to Infinitesimal Calculus (http://www.math.uiowa.edu/~stroyan/
InfsmlCalculus/InfsmlCalc.htm), University of Iowa
• Mauch, Sean, Sean's Applied Math Book (http://www.its.caltech.edu/~sean/book/unabridged.html), CIT, an
online textbook that includes a complete introduction to calculus
• Crowell, Benjamin, Calculus (http://www.lightandmatter.com/calc/), Fullerton College, an online textbook
• Garrett, Paul, Notes on First-Year Calculus (http://www.math.umn.edu/~garrett/calculus/)
• Hussain, Faraz, Understanding Calculus (http://www.understandingcalculus.com), an online textbook
• Kowalk, W.P., Integration Theory (http://einstein.informatik.uni-oldenburg.de/20910.html), University of
Oldenburg. A new concept to an old problem. Online textbook
• Sloughter, Dan, Difference Equations to Differential Equations (http://math.furman.edu/~dcs/book), an
introduction to calculus
• Numerical Methods of Integration (http://numericalmethods.eng.usf.edu/topics/integration.html) at Holistic
Numerical Methods Institute
• P.S. Wang, Evaluation of Definite Integrals by Symbolic Manipulation (http://www.lcs.mit.edu/publications/
specpub.php?id=660) (1972) - a cookbook of definite integral techniques
175
Derivative
176
Derivative
In calculus, a branch of mathematics, the derivative is a measure of
how a function changes as its input changes. Loosely speaking, a
derivative can be thought of as how much one quantity is changing in
response to changes in some other quantity; for example, the derivative
of the position of a moving object with respect to time is the object's
instantaneous velocity.
The derivative of a function at a chosen input value describes the best
linear approximation of the function near that input value. For a
real-valued function of a single real variable, the derivative at a point
equals the slope of the tangent line to the graph of the function at that
point. In higher dimensions, the derivative of a function at a point is a
linear transformation called the linearization.[1] A closely related
notion is the differential of a function.
The graph of a function, drawn in black, and a
tangent line to that function, drawn in red. The
slope of the tangent line is equal to the derivative
of the function at the marked point.
The process of finding a derivative is called differentiation. The reverse process is called antidifferentiation. The
fundamental theorem of calculus states that antidifferentiation is the same as integration. Differentiation and
integration constitute the two fundamental operations in single-variable calculus.
Differentiation and the derivative
Differentiation is a method to
compute the rate at which a dependent
output y changes with respect to the
change in the independent input x. This
rate of change is called the derivative
of y with respect to x. In more precise
language, the dependence of y upon x
means that y is a function of x. This
functional relationship is often denoted
y = ƒ(x), where ƒ denotes the function.
If x and y are real numbers, and if the
graph of y is plotted against x, the
derivative measures the slope of this
graph at each point.
(Click for larger image) At each point, the derivative of
is the slope
of a line that is tangent to the curve. The line is always tangent to the blue curve; its slope
is the derivative. Note derivative is positive where green, negative where red, and zero
where black.
The simplest case is when y is a linear
function of x, meaning that the graph
of y against x is a straight line. In this
case, y = ƒ(x) = m x + b, for real
numbers m and b, and the slope m is
given by
Derivative
177
where the symbol Δ (the uppercase form of the Greek letter Delta) is an abbreviation for "change in." This formula is
true because
y + Δy = ƒ(x+ Δx) = m (x + Δx) + b = m x + b + m Δx = y + mΔx.
It follows that Δy = m Δx.
This gives an exact value for the slope of a straight line. If the function ƒ is not linear (i.e. its graph is not a straight
line), however, then the change in y divided by the change in x varies: differentiation is a method to find an exact
value for this rate of change at any given value of x.
Figure 1. The tangent line at (x, ƒ(x))
Figure 2. The secant to curve y= ƒ(x) determined by points (x, ƒ(x)) and (x+h, ƒ(x+h))
Figure 3. The tangent line as limit of secants
The idea, illustrated by Figures 1-3, is to compute the rate of change as the limiting value of the ratio of the
differences Δy / Δx as Δx becomes infinitely small.
In Leibniz's notation, such an infinitesimal change in x is denoted by dx, and the derivative of y with respect to x is
written
suggesting the ratio of two infinitesimal quantities. (The above expression is read as "the derivative of y with respect
to x", "d y by d x", or "d y over d x". The oral form "d y d x" is often used conversationally, although it may lead to
confusion.)
Derivative
The most common approach[2] to turn this intuitive idea into a precise definition uses limits, but there are other
methods, such as non-standard analysis.[3]
Definition via difference quotients
Let ƒ be a real valued function. In classical geometry, the tangent line to the graph of the function ƒ at a real number a
was the unique line through the point (a, ƒ(a)) that did not meet the graph of ƒ transversally, meaning that the line did
not pass straight through the graph. The derivative of y with respect to x at a is, geometrically, the slope of the
tangent line to the graph of ƒ at a. The slope of the tangent line is very close to the slope of the line through (a, ƒ(a))
and a nearby point on the graph, for example (a + h, ƒ(a + h)). These lines are called secant lines. A value of h close
to zero gives a good approximation to the slope of the tangent line, and smaller values (in absolute value) of h will,
in general, give better approximations. The slope m of the secant line is the difference between the y values of these
points divided by the difference between the x values, that is,
This expression is Newton's difference quotient. The derivative is the value of the difference quotient as the secant
lines approach the tangent line. Formally, the derivative of the function ƒ at a is the limit
of the difference quotient as h approaches zero, if this limit exists. If the limit exists, then ƒ is differentiable at a.
Here ƒ′ (a) is one of several common notations for the derivative (see below).
Equivalently, the derivative satisfies the property that
which has the intuitive interpretation (see Figure 1) that the tangent line to ƒ at a gives the best linear approximation
to ƒ near a (i.e., for small h). This interpretation is the easiest to generalize to other settings (see below).
Substituting 0 for h in the difference quotient causes division by zero, so the slope of the tangent line cannot be
found directly using this method. Instead, define Q(h) to be the difference quotient as a function of h:
Q(h) is the slope of the secant line between (a, ƒ(a)) and (a + h, ƒ(a + h)). If ƒ is a continuous function, meaning that
its graph is an unbroken curve with no gaps, then Q is a continuous function away from h = 0. If the limit
exists, meaning that there is a way of choosing a value for Q(0) that makes the graph of Q a
continuous function, then the function ƒ is differentiable at a, and its derivative at a equals Q(0).
In practice, the existence of a continuous extension of the difference quotient Q(h) to h = 0 is shown by modifying
the numerator to cancel h in the denominator. This process can be long and tedious for complicated functions, and
many shortcuts are commonly used to simplify the process.
178
Derivative
179
Example
The squaring function ƒ(x) = x² is differentiable at x = 3, and its derivative there is 6. This result is established by
calculating the limit as h approaches zero of the difference quotient of ƒ(3):
The last expression shows that the difference quotient equals 6 + h when h ≠ 0 and is undefined when h = 0, because
of the definition of the difference quotient. However, the definition of the limit says the difference quotient does not
need to be defined when h = 0. The limit is the result of letting h go to zero, meaning it is the value that 6 + h tends
to as h becomes very small:
Hence the slope of the graph of the squaring function at the point (3, 9) is 6, and so its derivative at x = 3 is ƒ '(3) = 6.
More generally, a similar computation shows that the derivative of the squaring function at x = a is ƒ '(a) = 2a.
Continuity and differentiability
If y = ƒ(x) is differentiable at a, then ƒ must also be continuous at a. As
an example, choose a point a and let ƒ be the step function that returns
a value, say 1, for all x less than a, and returns a different value, say 10,
for all x greater than or equal to a. ƒ cannot have a derivative at a. If h
is negative, then a + h is on the low part of the step, so the secant line
from a to a + h is very steep, and as h tends to zero the slope tends to
infinity. If h is positive, then a + h is on the high part of the step, so the
secant line from a to a + h has slope zero. Consequently the secant
lines do not approach any single slope, so the limit of the difference
quotient does not exist.[4]
However, even if a function is continuous at a point, it may not be
differentiable there. For example, the absolute value function y = |x| is
continuous at x = 0, but it is not differentiable there. If h is positive,
then the slope of the secant line from 0 to h is one, whereas if h is
negative, then the slope of the secant line from 0 to h is negative one.
This can be seen graphically as a "kink" or a "cusp" in the graph at x =
0. Even a function with a smooth graph is not differentiable at a point
where its tangent is vertical: For instance, the function y = 3√x is not
differentiable at x = 0.
In summary: for a function ƒ to have a derivative it is necessary for the
function ƒ to be continuous, but continuity alone is not sufficient.
This function does not have a derivative at the
marked point, as the function is not continuous
there.
The absolute value function is continuous, but
fails to be differentiable at x = 0 since the tangent
slopes do not approach the same value from the
left as they do from the right.
Most functions that occur in practice have derivatives at all points or at
almost every point. Early in the history of calculus, many mathematicians assumed that a continuous function was
differentiable at most points. Under mild conditions, for example if the function is a monotone function or a
Lipschitz function, this is true. However, in 1872 Weierstrass found the first example of a function that is continuous
everywhere but differentiable nowhere. This example is now known as the Weierstrass function. In 1931, Stefan
Banach proved that the set of functions that have a derivative at some point is a meager set in the space of all
Derivative
continuous functions.[5] Informally, this means that hardly any continuous functions have a derivative at even one
point.
The derivative as a function
Let ƒ be a function that has a derivative at every point a in the domain of ƒ. Because every point a has a derivative,
there is a function that sends the point a to the derivative of ƒ at a. This function is written f′(x) and is called the
derivative function or the derivative of ƒ. The derivative of ƒ collects all the derivatives of ƒ at all the points in the
domain of ƒ.
Sometimes ƒ has a derivative at most, but not all, points of its domain. The function whose value at a equals f′(a)
whenever f′(a) is defined and elsewhere is undefined is also called the derivative of ƒ. It is still a function, but its
domain is strictly smaller than the domain of ƒ.
Using this idea, differentiation becomes a function of functions: The derivative is an operator whose domain is the
set of all functions that have derivatives at every point of their domain and whose range is a set of functions. If we
denote this operator by D, then D(ƒ) is the function f′(x). Since D(ƒ) is a function, it can be evaluated at a point a. By
the definition of the derivative function, D(ƒ)(a) = f′(a).
For comparison, consider the doubling function ƒ(x) =2x; ƒ is a real-valued function of a real number, meaning that it
takes numbers as inputs and has numbers as outputs:
The operator D, however, is not defined on individual numbers. It is only defined on functions:
Because the output of D is a function, the output of D can be evaluated at a point. For instance, when D is applied to
the squaring function,
D outputs the doubling function,
which we named ƒ(x). This output function can then be evaluated to get ƒ(1) = 2, ƒ(2) = 4, and so on.
Higher derivatives
Let ƒ be a differentiable function, and let f′(x) be its derivative. The derivative of f′(x) (if it has one) is written f′′(x)
and is called the second derivative of ƒ. Similarly, the derivative of a second derivative, if it exists, is written f′′′(x)
and is called the third derivative of ƒ. These repeated derivatives are called higher-order derivatives.
If x(t) represents the position of an object at time t, then the higher-order derivatives of x have physical
interpretations. The second derivative of x is the derivative of x′(t), the velocity, and by definition this is the object's
acceleration. The third derivative of x is defined to be the jerk, and the fourth derivative is defined to be the jounce.
A function ƒ need not have a derivative, for example, if it is not continuous. Similarly, even if ƒ does have a
derivative, it may not have a second derivative. For example, let
Calculation shows that ƒ is a differentiable function whose derivative is
180
Derivative
f′(x) is twice the absolute value function, and it does not have a derivative at zero. Similar examples show that a
function can have k derivatives for any non-negative integer k but no (k + 1)-order derivative. A function that has k
successive derivatives is called k times differentiable. If in addition the kth derivative is continuous, then the
function is said to be of differentiability class Ck. (This is a stronger condition than having k derivatives. For an
example, see differentiability class.) A function that has infinitely many derivatives is called infinitely
differentiable or smooth.
On the real line, every polynomial function is infinitely differentiable. By standard differentiation rules, if a
polynomial of degree n is differentiated n times, then it becomes a constant function. All of its subsequent
derivatives are identically zero. In particular, they exist, so polynomials are smooth functions.
The derivatives of a function ƒ at a point x provide polynomial approximations to that function near x. For example,
if ƒ is twice differentiable, then
in the sense that
If ƒ is infinitely differentiable, then this is the beginning of the Taylor series for ƒ.
Inflection point
A point where the second derivative of a function changes sign is called an inflection point.[6] At an inflection point,
the second derivative may be zero, as in the case of the inflection point x=0 of the function y=x3, or it may fail to
exist, as in the case of the inflection point x=0 of the function y=x1/3. At an inflection point, a function switches from
being a convex function to being a concave function or vice versa.
Notations for differentiation
Leibniz's notation
The notation for derivatives introduced by Gottfried Leibniz is one of the earliest. It is still commonly used when the
equation y = ƒ(x) is viewed as a functional relationship between dependent and independent variables. Then the first
derivative is denoted by
and was once thought of as an infinitesimal quotient. Higher derivatives are expressed using the notation
for the nth derivative of y = ƒ(x) (with respect to x). These are abbreviations for multiple applications of the
derivative operator. For example,
With Leibniz's notation, we can write the derivative of y at the point x = a in two different ways:
181
Derivative
182
Leibniz's notation allows one to specify the variable for differentiation (in the denominator). This is especially
relevant for partial differentiation. It also makes the chain rule easy to remember:[7]
Lagrange's notation
Sometimes referred to as prime notation,[8] one of the most common modern notations for differentiation is due to
Joseph-Louis Lagrange and uses the prime mark, so that the derivative of a function ƒ(x) is denoted ƒ′(x) or simply ƒ′.
Similarly, the second and third derivatives are denoted
and
To denote the number of derivatives beyond this point, some authors use Roman numerals in superscript, whereas
others place the number in parentheses:
or
The latter notation generalizes to yield the notation ƒ (n) for the nth derivative of ƒ — this notation is most useful
when we wish to talk about the derivative as being a function itself, as in this case the Leibniz notation can become
cumbersome.
Newton's notation
Newton's notation for differentiation, also called the dot notation, places a dot over the function name to represent a
time derivative. If y = ƒ(t), then
and
denote, respectively, the first and second derivatives of y with respect to t. This notation is used exclusively for time
derivatives, meaning that the independent variable of the function represents time. It is very common in physics and
in mathematical disciplines connected with physics such as differential equations. While the notation becomes
unmanageable for high-order derivatives, in practice only very few derivatives are needed.
Euler's notation
Euler's notation uses a differential operator D, which is applied to a function ƒ to give the first derivative Df. The
second derivative is denoted D2ƒ, and the nth derivative is denoted Dnƒ.
If y = ƒ(x) is a dependent variable, then often the subscript x is attached to the D to clarify the independent variable x.
Euler's notation is then written
or
,
although this subscript is often omitted when the variable x is understood, for instance when this is the only variable
present in the expression.
Euler's notation is useful for stating and solving linear differential equations.
Derivative
183
Computing the derivative
The derivative of a function can, in principle, be computed from the definition by considering the difference
quotient, and computing its limit. In practice, once the derivatives of a few simple functions are known, the
derivatives of other functions are more easily computed using rules for obtaining derivatives of more complicated
functions from simpler ones.
Derivatives of elementary functions
Most derivative computations eventually require taking the derivative of some common functions. The following
incomplete list gives some of the most frequently used functions of a single real variable and their derivatives.
• Derivatives of powers: if
where r is any real number, then
wherever this function is defined. For example, if
, then
and the derivative function is defined only for positive x, not for x = 0. When r = 0, this rule implies that f′(x) is zero
for x ≠ 0, which is almost the constant rule (stated below).
• Exponential and logarithmic functions:
• Trigonometric functions:
• Inverse trigonometric functions:
Derivative
184
Rules for finding the derivative
In many cases, complicated limit calculations by direct application of Newton's difference quotient can be avoided
using differentiation rules. Some of the most basic rules are the following.
• Constant rule: if ƒ(x) is constant, then
• Sum rule:
for all functions ƒ and g and all real numbers a and b.
• Product rule:
for all functions ƒ and g.
• Quotient rule:
for all functions ƒ and g where g ≠ 0.
• Chain rule: If
, then
Example computation
The derivative of
is
Here the second term was computed using the chain rule and third using the product rule. The known derivatives of
the elementary functions x2, x4, sin(x), ln(x) and exp(x) = ex, as well as the constant 7, were also used.
Derivatives in higher dimensions
Derivatives of vector valued functions
A vector-valued function y(t) of a real variable sends real numbers to vectors in some vector space Rn. A
vector-valued function can be split up into its coordinate functions y1(t), y2(t), …, yn(t), meaning that y(t) = (y1(t), ...,
yn(t)). This includes, for example, parametric curves in R2 or R3. The coordinate functions are real valued functions,
so the above definition of derivative applies to them. The derivative of y(t) is defined to be the vector, called the
tangent vector, whose coordinates are the derivatives of the coordinate functions. That is,
Equivalently,
if the limit exists. The subtraction in the numerator is subtraction of vectors, not scalars. If the derivative of y exists
for every value of t, then y′ is another vector valued function.
If e1, …, en is the standard basis for Rn, then y(t) can also be written as y1(t)e1 + … + yn(t)en. If we assume that the
derivative of a vector-valued function retains the linearity property, then the derivative of y(t) must be
Derivative
because each of the basis vectors is a constant.
This generalization is useful, for example, if y(t) is the position vector of a particle at time t; then the derivative y′(t)
is the velocity vector of the particle at time t.
Partial derivatives
Suppose that ƒ is a function that depends on more than one variable. For instance,
ƒ can be reinterpreted as a family of functions of one variable indexed by the other variables:
In other words, every value of x chooses a function, denoted fx, which is a function of one real number.[9] That is,
Once a value of x is chosen, say a, then f(x,y) determines a function fa that sends y to a² + ay + y²:
In this expression, a is a constant, not a variable, so fa is a function of only one real variable. Consequently the
definition of the derivative for a function of one variable applies:
The above procedure can be performed for any choice of a. Assembling the derivatives together into a function gives
a function that describes the variation of ƒ in the y direction:
This is the partial derivative of ƒ with respect to y. Here ∂ is a rounded d called the partial derivative symbol. To
distinguish it from the letter d, ∂ is sometimes pronounced "der", "del", or "partial" instead of "dee".
In general, the partial derivative of a function ƒ(x1, …, xn) in the direction xi at the point (a1 …, an) is defined to be:
In the above difference quotient, all the variables except xi are held fixed. That choice of fixed values determines a
function of one variable
and, by definition,
In other words, the different choices of a index a family of one-variable functions just as in the example above. This
expression also shows that the computation of partial derivatives reduces to the computation of one-variable
derivatives.
An important example of a function of several variables is the case of a scalar-valued function ƒ(x1,...xn) on a domain
in Euclidean space Rn (e.g., on R² or R³). In this case ƒ has a partial derivative ∂ƒ/∂xj with respect to each variable xj.
At the point a, these partial derivatives define the vector
This vector is called the gradient of ƒ at a. If ƒ is differentiable at every point in some domain, then the gradient is a
vector-valued function ∇ƒ that takes the point a to the vector ∇f(a). Consequently the gradient determines a vector
185
Derivative
field.
Directional derivatives
If ƒ is a real-valued function on Rn, then the partial derivatives of ƒ measure its variation in the direction of the
coordinate axes. For example, if ƒ is a function of x and y, then its partial derivatives measure the variation in ƒ in the
x direction and the y direction. They do not, however, directly measure the variation of ƒ in any other direction, such
as along the diagonal line y = x. These are measured using directional derivatives. Choose a vector
The directional derivative of ƒ in the direction of v at the point x is the limit
In some cases it may be easier to compute or estimate the directional derivative after changing the length of the
vector. Often this is done to turn the problem into the computation of a directional derivative in the direction of a
unit vector. To see how this works, suppose that v = λu. Substitute h = k/λ into the difference quotient. The
difference quotient becomes:
This is λ times the difference quotient for the directional derivative of f with respect to u. Furthermore, taking the
limit as h tends to zero is the same as taking the limit as k tends to zero because h and k are multiples of each other.
Therefore Dv(ƒ) = λDu(ƒ). Because of this rescaling property, directional derivatives are frequently considered only
for unit vectors.
If all the partial derivatives of ƒ exist and are continuous at x, then they determine the directional derivative of ƒ in the
direction v by the formula:
This is a consequence of the definition of the total derivative. It follows that the directional derivative is linear in v,
meaning that Dv + w(ƒ) = Dv(ƒ) + Dw(ƒ).
The same definition also works when ƒ is a function with values in Rm. The above definition is applied to each
component of the vectors. In this case, the directional derivative is a vector in Rm.
Total derivative, total differential and Jacobian matrix
When ƒ is a function from an open subset of Rn to Rm, then the directional derivative of ƒ in a chosen direction is the
best linear approximation to ƒ at that point and in that direction. But when n > 1, no single directional derivative can
give a complete picture of the behavior of ƒ. The total derivative, also called the (total) differential, gives a complete
picture by considering all directions at once. That is, for any vector v starting at a, the linear approximation formula
holds:
Just like the single-variable derivative, ƒ ′(a) is chosen so that the error in this approximation is as small as possible.
If n and m are both one, then the derivative ƒ ′(a) is a number and the expression ƒ ′(a)v is the product of two
numbers. But in higher dimensions, it is impossible for ƒ ′(a) to be a number. If it were a number, then ƒ ′(a)v would
be a vector in Rn while the other terms would be vectors in Rm, and therefore the formula would not make sense. For
the linear approximation formula to make sense, ƒ ′(a) must be a function that sends vectors in Rn to vectors in Rm,
and ƒ ′(a)v must denote this function evaluated at v.
To determine what kind of function it is, notice that the linear approximation formula can be rewritten as
186
Derivative
Notice that if we choose another vector w, then this approximate equation determines another approximate equation
by substituting w for v. It determines a third approximate equation by substituting both w for v and a + v for a. By
subtracting these two new equations, we get
If we assume that v is small and that the derivative varies continuously in a, then ƒ ′(a + v) is approximately equal to
ƒ ′(a), and therefore the right-hand side is approximately zero. The left-hand side can be rewritten in a different way
using the linear approximation formula with v + w substituted for v. The linear approximation formula implies:
This suggests that ƒ ′(a) is a linear transformation from the vector space Rn to the vector space Rm. In fact, it is
possible to make this a precise derivation by measuring the error in the approximations. Assume that the error in
these linear approximation formula is bounded by a constant times ||v||, where the constant is independent of v but
depends continuously on a. Then, after adding an appropriate error term, all of the above approximate equalities can
be rephrased as inequalities. In particular, ƒ ′(a) is a linear transformation up to a small error term. In the limit as v
and w tend to zero, it must therefore be a linear transformation. Since we define the total derivative by taking a limit
as v goes to zero, ƒ ′(a) must be a linear transformation.
In one variable, the fact that the derivative is the best linear approximation is expressed by the fact that it is the limit
of difference quotients. However, the usual difference quotient does not make sense in higher dimensions because it
is not usually possible to divide vectors. In particular, the numerator and denominator of the difference quotient are
not even in the same vector space: The numerator lies in the codomain Rm while the denominator lies in the domain
Rn. Furthermore, the derivative is a linear transformation, a different type of object from both the numerator and
denominator. To make precise the idea that ƒ ′ (a) is the best linear approximation, it is necessary to adapt a different
formula for the one-variable derivative in which these problems disappear. If ƒ : R → R, then the usual definition of
the derivative may be manipulated to show that the derivative of ƒ at a is the unique number ƒ ′(a) such that
This is equivalent to
because the limit of a function tends to zero if and only if the limit of the absolute value of the function tends to zero.
This last formula can be adapted to the many-variable situation by replacing the absolute values with norms.
The definition of the total derivative of ƒ at a, therefore, is that it is the unique linear transformation ƒ ′(a) : Rn → Rm
such that
Here h is a vector in Rn, so the norm in the denominator is the standard length on Rn. However, ƒ′(a)h is a vector in
Rm, and the norm in the numerator is the standard length on Rm. If v is a vector starting at a, then ƒ ′(a)v is called the
pushforward of v by ƒ and is sometimes written ƒ*v.
If the total derivative exists at a, then all the partial derivatives and directional derivatives of ƒ exist at a, and for all
v, ƒ ′(a)v is the directional derivative of ƒ in the direction v. If we write ƒ using coordinate functions, so that ƒ = (ƒ1, ƒ2,
..., ƒm), then the total derivative can be expressed using the partial derivatives as a matrix. This matrix is called the
Jacobian matrix of ƒ at a:
187
Derivative
188
The existence of the total derivative ƒ′(a) is strictly stronger than the existence of all the partial derivatives, but if the
partial derivatives exist and are continuous, then the total derivative exists, is given by the Jacobian, and depends
continuously on a.
The definition of the total derivative subsumes the definition of the derivative in one variable. That is, if ƒ is a
real-valued function of a real variable, then the total derivative exists if and only if the usual derivative exists. The
Jacobian matrix reduces to a 1×1 matrix whose only entry is the derivative ƒ′(x). This 1×1 matrix satisfies the
property that ƒ(a + h) − ƒ(a) − ƒ ′(a)h is approximately zero, in other words that
Up to changing variables, this is the statement that the function
is the best linear
approximation to ƒ at a.
The total derivative of a function does not give another function in the same way as the one-variable case. This is
because the total derivative of a multivariable function has to record much more information than the derivative of a
single-variable function. Instead, the total derivative gives a function from the tangent bundle of the source to the
tangent bundle of the target.
The natural analog of second, third, and higher-order total derivatives is not a linear transformation, is not a function
on the tangent bundle, and is not built by repeatedly taking the total derivative. The analog of a higher-order
derivative, called a jet, cannot be a linear transformation because higher-order derivatives reflect subtle geometric
information, such as concavity, which cannot be described in terms of linear data such as vectors. It cannot be a
function on the tangent bundle because the tangent bundle only has room for the base space and the directional
derivatives. Because jets capture higher-order information, they take as arguments additional coordinates
representing higher-order changes in direction. The space determined by these additional coordinates is called the jet
bundle. The relation between the total derivative and the partial derivatives of a function is paralleled in the relation
between the kth order jet of a function and its partial derivatives of order less than or equal to k.
Generalizations
The concept of a derivative can be extended to many other settings. The common thread is that the derivative of a
function at a point serves as a linear approximation of the function at that point.
• An important generalization of the derivative concerns complex functions of complex variables, such as functions
from (a domain in) the complex numbers C to C. The notion of the derivative of such a function is obtained by
replacing real variables with complex variables in the definition. If C is identified with R² by writing a complex
number z as x + i y, then a differentiable function from C to C is certainly differentiable as a function from R² to
R² (in the sense that its partial derivatives all exist), but the converse is not true in general: the complex derivative
only exists if the real derivative is complex linear and this imposes relations between the partial derivatives called
the Cauchy Riemann equations — see holomorphic functions.
• Another generalization concerns functions between differentiable or smooth manifolds. Intuitively speaking such
a manifold M is a space that can be approximated near each point x by a vector space called its tangent space: the
prototypical example is a smooth surface in R³. The derivative (or differential) of a (differentiable) map ƒ: M → N
between manifolds, at a point x in M, is then a linear map from the tangent space of M at x to the tangent space of
N at ƒ(x). The derivative function becomes a map between the tangent bundles of M and N. This definition is
fundamental in differential geometry and has many uses — see pushforward (differential) and pullback
(differential geometry).
• Differentiation can also be defined for maps between infinite dimensional vector spaces such as Banach spaces
and Fréchet spaces. There is a generalization both of the directional derivative, called the Gâteaux derivative, and
Derivative
•
•
•
•
of the differential, called the Fréchet derivative.
One deficiency of the classical derivative is that not very many functions are differentiable. Nevertheless, there is
a way of extending the notion of the derivative so that all continuous functions and many other functions can be
differentiated using a concept known as the weak derivative. The idea is to embed the continuous functions in a
larger space called the space of distributions and only require that a function is differentiable "on average".
The properties of the derivative have inspired the introduction and study of many similar objects in algebra and
topology — see, for example, differential algebra.
The discrete equivalent of differentiation is finite differences. The study of differential calculus is unified with the
calculus of finite differences in time scale calculus.
Also see arithmetic derivative.
Notes
[1] Differential calculus, as discussed in this article, is a very well-established mathematical discipline for which there are many sources. Almost
all of the material in this article can be found in Apostol 1967, Apostol 1969, and Spivak 1994.
[2] Spivak 1994, chapter 10.
[3] See Differential (infinitesimal) for an overview. Further approaches include the Radon–Nikodym theorem, and the universal derivation (see
Kähler differential).
[4] Despite this, it is still possible to take the derivative in the sense of distributions. The result is nine times the Dirac measure centered at a.
[5] Banach, S. (1931), "Uber die Baire'sche Kategorie gewisser Funktionenmengen", Studia. Math. (3): 174–179.. Cited by Hewitt, E and
Stromberg, K (1963), Real and abstract analysis, Springer-Verlag, Theorem 17.8
[6] Apostol 1967, §4.18
[7] In the formulation of calculus in terms of limits, the du symbol has been assigned various meanings by various authors. Some authors do not
assign a meaning to du by itself, but only as part of the symbol du/dx. Others define "dx" as an independent variable, and define du by
du = dx•ƒ′ (x). In non-standard analysis du is defined as an infinitesimal. It is also interpreted as the exterior derivative du of a function u. See
differential (infinitesimal) for further information.
[8] (http:/ / web. mit. edu/ wwmath/ calculus/ differentiation/ notation. html)
[9] This can also be expressed as the adjointness between the product space and function space constructions.
References
Print
• Anton, Howard; Bivens, Irl; Davis, Stephen (February 2, 2005), Calculus: Early Transcendentals Single and
Multivariable (8th ed.), New York: Wiley, ISBN 978-0-471-47244-5
• Apostol, Tom M. (June 1967), Calculus, Vol. 1: One-Variable Calculus with an Introduction to Linear Algebra, 1
(2nd ed.), Wiley, ISBN 978-0-471-00005-1
• Apostol, Tom M. (June 1969), Calculus, Vol. 2: Multi-Variable Calculus and Linear Algebra with Applications, 1
(2nd ed.), Wiley, ISBN 978-0-471-00007-5
• Courant, Richard; John, Fritz (December 22, 1998), Introduction to Calculus and Analysis, Vol. 1,
Springer-Verlag, ISBN 978-3-540-65058-4
• Eves, Howard (January 2, 1990), An Introduction to the History of Mathematics (6th ed.), Brooks Cole,
ISBN 978-0-03-029558-4
• Larson, Ron; Hostetler, Robert P.; Edwards, Bruce H. (February 28, 2006), Calculus: Early Transcendental
Functions (4th ed.), Houghton Mifflin Company, ISBN 978-0-618-60624-5
• Spivak, Michael (September 1994), Calculus (3rd ed.), Publish or Perish, ISBN 978-0-914098-89-8
• Stewart, James (December 24, 2002), Calculus (5th ed.), Brooks Cole, ISBN 978-0-534-39339-7
• Thompson, Silvanus P. (September 8, 1998), Calculus Made Easy (Revised, Updated, Expanded ed.), New York:
St. Martin's Press, ISBN 978-0-312-18548-0
189
Derivative
Online books
• Crowell, Benjamin (2003), Calculus (http://www.lightandmatter.com/calc/)
• Garrett, Paul (2004), Notes on First-Year Calculus (http://www.math.umn.edu/~garrett/calculus/), University
of Minnesota
• Hussain, Faraz (2006), Understanding Calculus (http://www.understandingcalculus.com/)
• Keisler, H. Jerome (2000), Elementary Calculus: An Approach Using Infinitesimals (http://www.math.wisc.
edu/~keisler/calc.html)
• Mauch, Sean (2004), Unabridged Version of Sean's Applied Math Book (http://www.its.caltech.edu/~sean/
book/unabridged.html)
• Sloughter, Dan (2000), Difference Equations to Differential Equations (http://synechism.org/drupal/de2de/)
• Strang, Gilbert (1991), Calculus (http://ocw.mit.edu/ans7870/resources/Strang/strangtext.htm)
• Stroyan, Keith D. (1997), A Brief Introduction to Infinitesimal Calculus (http://www.math.uiowa.edu/
~stroyan/InfsmlCalculus/InfsmlCalc.htm)
• Wikibooks, Calculus (http://en.wikibooks.org/wiki/Calculus)
Web pages
• Khan Academy: Derivative lesson 1 (http://www.khanacademy.org/video/
calculus--derivatives-1--new-hd-version?playlist=Calculus)
• Weisstein, Eric W. " Derivative. (http://mathworld.wolfram.com/Derivative.html)" From MathWorld
• Derivatives of Trigonometric functions (http://www.ugrad.math.ubc.ca/coursedoc/math100/notes/
derivative/trig2.html), UBC
• Solved Problems in Derivatives (http://calculus.solved-problems.com/category/derivative/)
190
191
Control methods
OLE for process control
OLE for Process Control (OPC), which stands for Object Linking and Embedding (OLE) for Process Control, is
the original name for a standards specification developed in 1996 by an industrial automation industry task force.
The standard specifies the communication of real-time plant data between control devices from different
manufacturers.
After the initial release, the OPC Foundation was created to maintain the standard. Since then, standards have been
added and names have been changed. As of June, 2006, "OPC is a series of standards specifications". (Seven current
standards and two emerging standards.) "The first standard (originally called simply the OPC Specification"), is
"now called the Data Access Specification", or (later on the same page) "OPC Data Access", or OPC Data Access
Specification. [1]
While OPC originally stood for "OLE for Process Control", the official stance of the OPC Foundation is that OPC is
no longer an acronym and the technology is simply known as "OPC". One of the reasons behind this is while OPC is
heavily used within the process industries, it can be, and is, widely used in discrete manufacturing as well. Hence,
OPC is known for more than just its applications within process control.
Origin and uses
The OPC Specification was based on the OLE, COM, and DCOM technologies developed by Microsoft for the
Microsoft Windows operating system family. The specification defined a standard set of objects, interfaces and
methods for use in process control and manufacturing automation applications to facilitate interoperability. The most
common OPC specification is OPC Data Access, which is used to read and write real-time data. When vendors refer
to OPC generically, they typically mean OPC Data Access. OPC DA itself has gone through 3 major revisions since
its inception. Versions are backwards compatible, in that a version 3 OPC Server can still be accessed by a version 1
OPC Client, since the specifications add functionality but still require the older version to be implemented as well.
However, a Client could be written that does not support the older functions since everything can be done using the
newer ones, so a DA 3 compatible Client will not necessarily work with a DA 1.0 Server.
OPC was designed to provide a common bridge for Windows based software applications and process control
hardware. Standards define consistent methods of accessing field data from plant floor devices. This method remains
the same regardless of the type and source of data. An OPC Server for one hardware device provides the same
methods for an OPC Client to access its data as any and every other OPC Server for that same and any other
hardware device. The aim was to reduce the amount of duplicated effort required from hardware manufacturers and
their software partners, and from the SCADA and other HMI producers in order to interface the two. Once a
hardware manufacturer had developed their OPC Server for the new hardware device their work was done to allow
any 'top end' to access their device, and once the SCADA producer had developed their OPC Client their work was
done to allow access to any hardware, existing or yet to be created, with an OPC compliant server.
OPC servers provide a method for many different software packages (so long as it is an OPC Client) to access data
from a process control device, such as a PLC or DCS. Traditionally, any time a package needed access to data from a
device, a custom interface, or driver, had to be written. The purpose of OPC is to define a common interface that is
written once and then reused by any business, SCADA, HMI, or custom software packages.
OLE for process control
There is nothing in the OPC Specs. to restrict the Server to providing access to a process control device. OPC
Servers can be written for anything from getting the internal temperature of a microprocessor to the current
temperature in Monument Valley.
Once an OPC server is written for a particular device, it can be reused by any application that is able to act as an
OPC client. OPC servers use Microsoft’s OLE technology (also known as the Component Object Model, or COM) to
communicate with clients. COM technology permits a standard for real-time information exchange between software
applications and process hardware to be defined.
It is important to note that some OPC specifications are published, others are available only to member of the OPC
Foundation. So whilst no company "owns" OPC and anyone can develop an OPC server, whether or not they are a
member of the OPC Foundation, non-members will not necessarily be using the latest specifications. Anyone can
integrate OPC products, and there is no pre-requisite for the system integrator to belong to any organization. It is
therefore up to each company that requires OPC products to ensure that their products are certified and that their
system integrators have the necessary training.
Future
The OPC Unified Architecture (UA) has been specified and is being tested and implemented through its Early
Adopters program. It can be implemented with Java, Microsoft .NET, or C, eliminating the need to use a Microsoft
Windows based platform of earlier OPC versions. UA combines the functionality of the existing OPC interfaces with
new technologies such as XML and Web Services to deliver higher level MES and ERP support.
On September 16, 2010, The OPC Foundation and the MTConnect Institute announced a cooperation to ensure
interoperability and consistency between the two standards.[2]
References
[1] What is OPC? (http:/ / www. opcfoundation. org/ Default. aspx/ 01_about/ 01_whatis. asp?MID=AboutOPC) opcfoundation.org (http:/ /
www. opcfoundation. org/ )
[2] "OPC Foundation and MTConnect Institute Announce a Memorandum of Understanding" (http:/ / www. opcfoundation. org/ Default. aspx/
02_news/ 02_news_display. asp?id=845& MID=News). OPC Foundation. 2010-10-21. . Retrieved 2010-10-26.
External links
•
•
•
•
•
•
OPC Foundation (http://www.opcfoundation.org/)
OPC Tutorials (http://matrikonopc.ca/resources/opc-tutorials.aspx)
OPC Programmers' Connection (http://www.opcconnect.com)
FatRat Library - free OPC server developers toolkit (http://code.google.com/p/frl/)
OpenOPC - Open Source OPC client development in Python (http://openopc.sourceforge.net)
OPC Unified Architecture Address Space e-book (http://www.commsvr.com/UAModelDesigner/Index.aspx)
192
Nonlinear control
Nonlinear control
Nonlinear control is the area of control engineering specifically involved with systems that are nonlinear,
time-variant, or both. Many well-established analysis and design techniques exist for LTI systems (e.g., root-locus,
Bode plot, Nyquist criterion, state-feedback, pole placement); however, one or both of the controller and the system
under control in a general control system may not be an LTI system, and so these methods cannot necessarily be
applied directly. Nonlinear control theory studies how to apply existing linear methods to these more general control
systems. Additionally, it provides novel control methods that cannot be analyzed using LTI system theory. Even
when LTI system theory can be used for the analysis and design of a controller, a nonlinear controller can have
attractive characteristics (e.g., simpler implementation, increased speed, or decreased control energy); however,
nonlinear control theory usually requires more rigorous mathematical analysis to justify its conclusions.
Properties of nonlinear systems
Some properties of nonlinear dynamic systems are
• They do not follow the principle of superposition (linearity and homogeneity).
• They may have multiple isolated equilibrium points.
• They may exhibit properties such as limit-cycle, bifurcation, chaos.
• Finite escape time: Solutions of nonlinear systems may not exist for all times.
Analysis and control of nonlinear systems
There are several well-developed techniques for analyzing nonlinear feedback systems:
•
•
•
•
•
•
•
•
Describing function method
Phase plane method
Lyapunov stability analysis
Singular perturbation method
Popov criterion (described in The Lur'e Problem below)
Center manifold theorem
Small-gain theorem
Passivity analysis
Control design techniques for nonlinear systems also exist. These can be subdivided into techniques which attempt to
treat the system as a linear system in a limited range of operation and use (well-known) linear design techniques for
each region:
• Gain scheduling
Those that attempt to introduce auxiliary nonlinear feedback in such a way that the system can be treated as linear
for purposes of control design:
• Feedback linearization
And Lyapunov based methods:
•
•
•
•
Lyapunov redesign
Nonlinear damping
Backstepping
Sliding mode control
193
Nonlinear control
194
Nonlinear feedback analysis – The Lur'e problem
An early nonlinear feedback system
analysis problem was formulated by A.
I. Lur'e. Control systems described by
the Lur'e problem have a forward path
that is linear and time-invariant, and a
feedback path that contains a
memory-less, possibly time-varying,
static nonlinearity.
Lur'e problem block diagram
The linear part can be characterized by four matrices (A,B,C,D), while the nonlinear part is Φ(y) with
(a sector nonlinearity).
Absolute stability problem
Consider:
1. (A,B) is controllable and (C,A) is observable
2. two real numbers a, b with a<b, defining a sector for function Φ
The problem is to derive conditions involving only the transfer matrix H(s) and {a,b} such that x=0 is a globally
uniformly asymptotically stable equilibrium of the system. This is known as the Lur'e problem.
There are two main theorems concerning the problem:
• The Circle criterion
• The Popov criterion.
Popov criterion
The sub-class of Lur'e systems studied by Popov is described by:
where x ∈ Rn, ξ,u,y are scalars and A,b,c,d have commensurate dimensions. The nonlinear element Φ: R → R is a
time-invariant nonlinearity belonging to open sector (0, ∞). This means that
Φ(0) = 0, y Φ(y) > 0, ∀ y ≠ 0;
The transfer function from u to y is given by
Theorem: Consider the system (1)-(2) and suppose
1. A is Hurwitz
2. (A,b) is controllable
3. (A,c) is observable
4. d>0 and
5. Φ ∈ (0,∞)
Nonlinear control
195
then the system is globally asymptotically stable if there exists a number r>0 such that
infω ∈ R Re[(1+jωr)h(jω)] > 0 .
Things to be noted:
• The Popov criterion is applicable only to autonomous systems
• The system studied by Popov has a pole at the origin and there is no direct pass-through from input to output
• The nonlinearity Φ must satisfy an open sector condition
Theoretical results in nonlinear control
Frobenius Theorem
The Frobenius theorem is a deep result in Differential Geometry. When applied to Nonlinear Control, it says the
following: Given a system of the form
where
,
integral curves of
are vector fields belonging to a distribution
are restricted to a manifold of dimension
and
if span(
are control functions, the
and
is an involutive
distribution.
Further reading
• A. I. Lur'e and V. N. Postnikov, "On the theory of stability of control systems," Applied mathematics and
mechanics, 8(3), 1944, (in Russian).
• M. Vidyasagar, Nonlinear Systems Analysis, 2nd edition, Prentice Hall, Englewood Cliffs, New Jersey 07632.
• A. Isidori, Nonlinear Control Systems, 3rd edition, Springer Verlag, London, 1995.
• H. K. Khalil, Nonlinear Systems, 3rd edition, Prentice Hall, Upper Saddle River, New Jersey, 2002. ISBN
0130673897
• B. Brogliato, R. Lozano, B. Maschke, O. Egeland, "Dissipative Systems Analysis and Control", Springer Verlag,
London, 2nd edition, 2007.
Proportional control
196
Proportional control
A proportional control system is a type of linear feedback control
system. Two classic mechanical examples are the toilet bowl float
proportioning valve and the fly-ball governor.
The proportional control system is more complex than an on-off
control system like a bi-metallic domestic thermostat, but simpler than
a proportional-integral-derivative (PID) control system used in
something like an automobile cruise control. On-off control will work
where the overall system has a relatively long response time, but will
result in instability if the system being controlled has a rapid response
time. Proportional control overcomes this by modulating the output to
the controlling device, such as a continuously variable valve.
The fly-ball governor is a classic example of
proportional control.
An analogy to on-off control is driving a car by applying either full
power or no power and varying the duty cycle, to control speed. The power would be on until the target speed is
reached, and then the power would be removed, so the car reduces speed. When the speed falls below the target, with
a certain hysteresis, full power would again be applied. It can be seen that this looks like pulse-width modulation, but
would obviously result in poor control and large variations in speed. The more powerful the engine; the greater the
instability, the heavier the car; the greater the stability. Stability may be expressed as correlating to the
power-to-weight ratio of the vehicle.
Proportional control is how most drivers control the speed of a car. If the car is at target speed and the speed
increases slightly, the power is reduced slightly, or in proportion to the error (the actual versus target speed), so that
the car reduces speed gradually and reaches the target point with very little, if any, "overshoot", so the result is much
smoother control than on-off control.
Further refinements like PID control would help compensate for additional variables like hills, where the amount of
power needed for a given speed change would vary, which would be accounted for by the integral function of the
PID control.
Proportional Control Theory
In the proportional control algorithm, the controller output is proportional to the error signal, which is the difference
between the set point and the process variable. In other words, the output of a proportional controller is the
multiplication product of the error signal and the proportional gain.
This can be mathematically expressed as
where
•
: Output of the proportional controller
•
: Proportional gain
•
: Instantaneous process error at time 't'.
• SP: Set point
• PV: Process variable
Proportional control
197
External links
• Proportional control compared to ON-OFF or bang-bang control [1]
References
[1] http:/ / www. fourmilab. ch/ hackdiet/ www/ subsection1_2_3_0_5. html
Optimal control
Optimal control theory, an extension of the calculus of variations, is a mathematical optimization method for
deriving control policies. The method is largely due to the work of Lev Pontryagin and his collaborators in the Soviet
Union[1] and Richard Bellman in the United States.
General method
Optimal control deals with the problem of finding a control law for a given system such that a certain optimality
criterion is achieved. A control problem includes a cost functional that is a function of state and control variables. An
optimal control is a set of differential equations describing the paths of the control variables that minimize the cost
functional. The optimal control can be derived using Pontryagin's maximum principle (a necessary condition also
known as Pontryagin's minimum principle or simply Pontryagin's Principle[2] ), or by solving the
Hamilton-Jacobi-Bellman equation (a sufficient condition).
We begin with a simple example. Consider a car traveling on a straight line through a hilly road. The question is,
how should the driver press the accelerator pedal in order to minimize the total traveling time? Clearly in this
example, the term control law refers specifically to the way in which the driver presses the accelerator and shifts the
gears. The "system" consists of both the car and the road, and the optimality criterion is the minimization of the total
traveling time. Control problems usually include ancillary constraints. For example the amount of available fuel
might be limited, the accelerator pedal cannot be pushed through the floor of the car, speed limits, etc.
A proper cost functional is a mathematical expression giving the traveling time as a function of the speed,
geometrical considerations, and initial conditions of the system. It is often the case that the constraints are
interchangeable with the cost functional.
Another optimal control problem is to find the way to drive the car so as to minimize its fuel consumption, given that
it must complete a given course in a time not exceeding some amount. Yet another control problem is to minimize
the total monetary cost of completing the trip, given assumed monetary prices for time and fuel.
A more abstract framework goes as follows. Minimize the continuous-time cost functional
subject to the first-order dynamic constraints
the algebraic path constraints
and the boundary conditions
where
is the state,
initial time, and
is the control,
is the independent variable (generally speaking, time),
is the terminal time. The terms
and
is the
are called the endpoint cost and Lagrangian,
Optimal control
198
respectively. Furthermore, it is noted that the path constraints are in general inequality constraints and thus may not
be active (i.e., equal to zero) at the optimal solution. It is also noted that the optimal control problem as stated above
may have multiple solutions (i.e., the solution may not be unique). Thus, it is most often the case that any solution
the optimal control problem is locally minimizing.
Linear quadratic control
A special case of the general nonlinear optimal control problem given in the previous section is the linear quadratic
(LQ) optimal control problem. The LQ problem is stated as follows. Minimize the quadratic continuous-time cost
functional
Subject to the linear first-order dynamic constraints
and the initial condition
A particular form of the LQ problem that arises in many control system problems is that of the linear quadratic
regulator (LQR) where all of the matrices (i.e.,
,
,
, and
) are constant, the initial time is arbitrarily
set to zero, and the terminal time is taken in the limit
(this last assumption is what is known as infinite
horizon). The LQR problem is stated as follows. Minimize the infinite horizon quadratic continuous-time cost
functional
Subject to the linear time-invariant first-order dynamic constraints
and the initial condition
In the finite-horizon case the matrices are restricted in that
and
are positive semi-definite and positive
definite, respectively. In the infinite-horizon case, however, the matrices
and
are not only
positive-semidefinite and positive-definite, respectively, but are also constant. These additional restrictions on
and
in the infinite-horizon case are enforced to ensure that the cost functional remains positive. Furthermore, in
order to ensure that the cost function is bounded, the additional restriction is imposed that the pair
is
controllable. Note that the LQ or LQR cost functional can be thought of physically as attempting to minimize the
control energy (measured as a quadratic form).
The infinite horizon problem (i.e., LQR) may seem overly restrictive and essentially useless because it assumes that
the operator is driving the system to zero-state and hence driving the output of the system to zero. This is indeed
correct. However the problem of driving the output to a desired nonzero level can be solved after the zero output one
is. In fact, it can be proved that this secondary LQR problem can be solved in a very straightforward manner. It has
been shown in classical optimal control theory that the LQ (or LQR) optimal control has the feedback form
where
and
is a properly dimensioned matrix, given as
is the solution of the differential Riccati equation. The differential Riccati equation is given as
Optimal control
199
For the finite horizon LQ problem, the Riccati equation is integrated backward in time using the terminal boundary
condition
For the infinite horizon LQR problem, the differential Riccati equation is replaced with the algebraic Riccati
equation (ARE) given as
Understanding that the ARE arises from infinite horizon problem, the matrices
,
,
, and
are all
constant. It is noted that there are in general multiple solutions to the algebraic Riccati equation and the positive
definite (or positive semi-definite) solution is the one that is used to compute the feedback gain. The LQ (LQR)
problem was elegantly solved by Rudolf Kalman.[3]
Numerical methods for optimal control
Optimal control problems are generally nonlinear and therefore, generally do not have analytic solutions (e.g., like
the linear-quadratic optimal control problem). As a result, it is necessary to employ numerical methods to solve
optimal control problems. In the early years of optimal control (circa 1950s to 1980s) the favored approach for
solving optimal control problems was that of indirect methods. In an indirect method, the calculus of variations is
employed to obtain the first-order optimality conditions. These conditions result in a two-point (or, in the case of a
complex problem, a multi-point) boundary-value problem. This boundary-value problem actually has a special
structure because it arises from taking the derivative of a Hamiltonian. Thus, the resulting dynamical system is a
Hamiltonian system of the form
where
is the augmented Hamiltonian and in an indirect method, the boundary-value problem is solved (using the
appropriate boundary or transversality conditions). The beauty of using an indirect method is that the state and
adjoint (i.e.,
) are solved for and the resulting solution is readily verified to be an extremal trajectory. The
disadvantage of indirect methods is that the boundary-value problem is often extremely difficult to solve
(particularly for problems that span large time intervals or problems with interior point constraints). A well-known
software program that implements indirect methods is BNDSCO.[4]
The approach that has risen to prominence in numerical optimal control over the past two decades (i.e., from the
1980s to the present) is that of so called direct methods. In a direct method, the state and/or control are approximated
using an appropriate function approximation (e.g., polynomial approximation or piecewise constant
parameterization). Simultaneously, the cost functional is approximated as a cost function. Then, the coefficients of
the function approximations are treated as optimization variables and the problem is "transcribed" to a nonlinear
optimization problem of the form:
Minimize
subject to the algebraic constraints
Depending upon the type of direct method employed, the size of the nonlinear optimization problem can be quite
small (e.g., as in a direct shooting or quasilinearization method) or may be quite large (e.g., a direct collocation
method[5] ). In the latter case (i.e., a collocation method), the nonlinear optimization problem may be literally
Optimal control
200
thousands to tens of thousands of variables and constraints. Given the size of many NLPs arising from a direct
method, it may appear somewhat counter-intuitive that solving the nonlinear optimization problem is easier than
solving the boundary-value problem. It is, however, the fact that the NLP is easier to solve than the boundary-value
problem. The reason for the relative ease of computation, particularly of a direct collocation method, is that the NLP
is sparse and many well-known software programs exist (e.g., SNOPT[6] ) to solve large sparse NLPs. As a result,
the range of problems that can be solved via direct methods (particularly direct collocation methods which are very
popular these days) is significantly larger than the range of problems that can be solved via indirect methods. In fact,
direct methods have become so popular these days that many people have written elaborate software programs that
employ these methods. In particular, many such programs written in FORTRAN include DIRCOL,[7] SOCS,[8]
OTIS,[9] GESOP/ASTOS[10] and DITAN.[11] In recent years, due to the advent of the MATLAB programming
language, optimal control software in MATLAB has become more common. Examples of academically developed
MATLAB software tools implementing direct methods include RIOTS [12],[13] DIDO,[14] DIRECT,[15] and GPOPS
[16] [17]
,
while an example of an industry developed MATLAB tool is PROPT.[18] These software tools have
increased significantly the opportunity for people to explore complex optimal control problems both for academic
research and industrial-strength problems. Finally, it is noted that general-purpose MATLAB optimization
environments such as TOMLAB have made coding complex optimal control problems significantly easier than was
previously possible in languages such as C and FORTRAN.
Discrete-time optimal control
The examples thus far have shown continuous time systems and control solutions. In fact, as optimal control
solutions are now often implemented digitally, contemporary control theory is now primarily concerned with discrete
time systems and solutions. The Theory of Consistent Approximations[19] provides conditions under which solutions
to a series of increasingly accurate discretized optimal control problem converge to the solution of the original,
continuous-time problem. Not all discretization methods have this property, even seemingly obvious ones. For
instance, using a variable step-size routine to integrate the problem's dynamic equations may generate a gradient
which does not converge to zero (or point in the right direction) as the solution is approached. The direct method
RIOTS [20] is based on the Theory of Consistent Approximation.
Examples
A common solution strategy in many optimal control problems is to solve for the costate (sometimes called the
shadow price)
. The costate summarizes in one number the marginal value of expanding or contracting the
state variable next turn. The marginal value is not only the gains accruing to it next turn but associated with the
duration of the program. It is nice when
can be solved analytically, but usually the most one can do is describe
it sufficiently well that the intuition can grasp the character of the solution and an equation solver can solve
numerically for the values.
Having obtained
, the turn-t optimal value for the control can usually be solved as a differential equation
conditional on knowledge of
. Again it is infrequent, especially in continuous-time problems, that one obtains
the value of the control or the state explicitly. Usually the strategy is to solve for thresholds and regions that
characterize the optimal control and use a numerical solver to isolate the actual choice values in time.
Finite time
Consider the problem of a mine owner who must decide at what rate to extract ore from his mine. He owns rights to
the ore from date to date . At date there is
ore in the ground, and the instantaneous stock of ore
declines at the rate the mine owner extracts it u(t). The mine owner extracts ore at cost
constant price
. He does not value the ore remaining in the ground at time
and sells ore at a
(there is no "scrap value"). He
chooses the rate of extraction in time u(t) to maximize profits over the period of ownership with no time discounting.
Optimal control
201
1. Discrete-time version
The manager maximizes profit
2. Continuous-time version
:
The manager maximizes profit
:
subject to the law of evolution for the state variable
subject to the law of evolution for the state variable
Form the Hamiltonian and differentiate:
Form the Hamiltonian and differentiate:
As the mine owner does not value the ore remaining at time
,
Using the above equations, it is easy to solve for the
series
and using the initial and turn-T conditions, the
explicitly, giving
.
and
As the mine owner does not value the ore remaining at time
,
Using the above equations, it is easy to solve for the differential
equations governing
and
series can be solved
and using the initial and turn-T conditions, the functions can be solved
numerically.
References
[1] L. S. Pontryagin, 1962. The Mathematical Theory of Optimal Processes.
[2] I. M. Ross, 2009. A Primer on Pontryagin's Principle in Optimal Control, Collegiate Publishers. ISBN 978-0-9843571-0-9.
[3] Kalman, Rudolf. A new approach to linear filtering and prediction problems. Transactions of the ASME, Journal of Basic Engineering,
82:34–45, 1960
[4] Oberle, H. J. and Grimm, W., "BNDSCO-A Program for the Numerical Solution of Optimal Control Problems," Institute for Flight Systems
Dynamics, DLR, Oberpfaffenhofen, 1989
[5] Betts, J. T., Practical Methods for Optimal Control Using Nonlinear Programming, SIAM Press, Philadelphia, Pennsylvania, 2001
[6] Gill, P. E., Murray, W. M., and Saunders, M. A., User's Manual for SNOPT Version 7: Software for Large-Scale Nonlinear Programming,
University of California, San Diego Report, 24 April 2007
[7] von Stryk, O., User's Guide for DIRCOL (version 2.1): A Direct Collocation Method for the Numerical Solution of Optimal Control
Problems, Fachgebiet Simulation und Systemoptimierung (SIM), Technische Universität Darmstadt (2000, Version of November 1999).
[8] Betts, J.T. and Huffman, W. P., Sparse Optimal Control Software, SOCS, Boeing Information and Support Services, Seattle, Washington,
July 1997
[9] Hargraves, C. R. and Paris, S. W., "Direct Trajectory Optimization Using Nonlinear Programming and Collocation", Journal of Guidance,
Control, and Dynamics, Vol. 10, No. 4., 1987, pp. 338-342
[10] Gath, P.F., Well, K.H., "Trajectory Optimization Using a Combination of Direct Multiple Shooting and Collocation", AIAA 2001-4047,
AIAA Guidance, Navigation, and Control Conference, Montréal, Québec, Canada, 6–9 August 2001
[11] Vasile M., Bernelli-Zazzera F., Fornasari N., Masarati P., "Design of Interplanetary and Lunar Missions Combining Low-Thrust and Gravity
Assists", Final Report of the ESA/ESOC Study Contract No. 14126/00/D/CS, September 2002
[12] http:/ / www. schwartz-home. com/ RIOTS/
[13] Schwartz, Adam, Theory and Implementation of Methods based on Runge-Kutta Integration for Solving Optimal Control Problems,
University of California at Berkeley, PhD Dissertation, 1996.
[14] Ross, I. M. and Fahroo, F., User's Manual for DIDO: A MATLAB Package for Dynamic Optimization, Dept. of Aeronautics and
Astronautics, Naval Postgraduate School Technical Report, 2002
[15] Williams, P., User's Guide to DIRECT, Version 2.00, Melbourne, Australia, 2008
[16] http:/ / gpops. sourceforge. net
[17] Rao, A. V., Benson, D. A., Huntington, G. T., Francolin, C., Darby, C. L., and Patterson, M. A., User's Manual for GPOPS: A MATLAB
Package for Dynamic Optimization Using the Gauss Pseudospectral Method, University of Florida Report, August 2008.
Optimal control
[18] Rutquist, P. and Edvall, M. M, PROPT - MATLAB Optimal Control Software," 1260 S.E. Bishop Blvd Ste E, Pullman, WA 99163, USA:
Tomlab Optimization, Inc.
[19] E. Polak, On the use of consistent approximations in the solution of semi-infinite optimization and optimal control problems Math. Prog. 62
pp. 385-415 (1993).
[20] http:/ / www. schwartz-home. com/ RIOTS
Further reading
Books
• Athans, M. A. and Falb, P. L., Optimal Control, McGraw-Hill, New York, 1966.
• Becerra, V.M., 2008, Optimal control (http://www.scholarpedia.org/article/Optimal_control). Scholarpedia
(http://www.scholarpedia.org), 3(1):5354
• Bryson, A. E., 1969. Applied Optimal Control: Optimization, Estimation, & Control.
• Evans, L.C., An Introduction to Optimal Control Theory ( available free online (http://math.berkeley.edu/
~evans/control.course.pdf))
• Ross, I. M. A Primer on Pontryagin's Principle in Optimal Control, Collegiate Publishers, 2009. ISBN
978-0-9843571-0-9. (http://www.ElissarGlobal.com free chapter available online)
• H. O. Fattorini and S. S. Sritharan, "Existence of Optimal Controls for Viscous Flow Problems," Proceedings of
the Royal Society of London Series A, Vol. 439, 1992, pp. 81–102.
• H. O. Fattorini and S. S. Sritharan, "Necessary and Sufficient Conditions for Optimal Controls in Viscous Flow,"
Proceedings of the Royal Society of Edinburgh, Series A, Vol. 124A, 1994, pp. 211–251.
• H. O. Fattorini and S. S. Sritharan, "Optimal chattering controls for viscous flow," Nonlinear analysis, Theory,
Methods and Applications, Vol. 25, No. 8, pp. 763–797, 1995.
• H. O. Fattorini and S. S. Sritharan,"Optimal control problems with state constraints in fluid mechanics and
combustion," Applied Mathematics and Optimization, Vol. 38(2), 1998, pp. 159–192.
• Kirk, D. E., 2004. Optimal Control Theory: An Introduction.
• Lebedev, L. P., and Cloud, M. J., 2003. The Calculus of Variations and Functional Analysis with Optimal Control
and Applications in Mechanics. World Scientific. Especially chpt. 2.
• Lewis, F. L., and Syrmos, V. L., 19nn. Optimal Control, 2nd ed. John Wiley & Sons.
• S. S. Sritharan, "Optimal Control of Viscous Flow", SIAM, 1998. (http://www.ec-securehost.com/SIAM/ot59.
html)
• S. S. Sritharan, "An Optimal Control Problem in Exterior Hydrodynamics," Proceedings of the Royal Society of
Edinburgh, Series 121A, 1992, pp. 5–32.
• S. S. Sritharan,"Deterministic and stochastic control of viscous flow with linear, monotone and hyper viscosities,"
Applied Mathematics and Optimization, Vol. 41(2), pp. 255–308, 2000.
• Stengel, R. F., 1994. Optimal Control and Estimation. (http://www.princeton.edu/~stengel/OptConEst.html)
Dover.
• Sethi, S. P., and Thompson, G. L., 2000. Optimal Control Theory: Applications to Management Science and
Economics, 2nd edition, Springer (ISBN 0387280928 and ISBN 0792386086). Slides are available at http://
www.utdallas.edu/~sethi/OPRE7320presentation.html
• Sontag, Eduardo D. Mathematical Control Theory: Deterministic Finite Dimensional Systems. Second Edition.
Springer. (ISBN 0-387-984895) ( available free online (http://www.math.rutgers.edu/~sontag/FTP_DIR/
sontag_mathematical_control_theory_springer98.pdf))
• Brogan, William L. 1990. Modern Control Theory. ISBN 0135897637
• Bryson, A.E.; Ho, Y.C. (1975). Applied optimal control. Washington, DC: Hemisphere.
Journals
• Optimal Control Applications and Methods. (http://www3.interscience.wiley.com/cgi-bin/jhome/
2133?CRETRY=1&SRETRY=0) John Wiley & Sons, Inc.
202
Optimal control
• SIAM Journal of Control and Optimization.
External links
• ACADO Toolkit - Open Source Toolkit for Automatic Control and Dynamic Optimization (C++, MATLAB
interface available) (http://www.acadotoolkit.org)
• Dr. Benoît CHACHUAT: Automatic Control Laboratory (http://lawww.epfl.ch/page4234.html) - Nonlinear
Programming, Calculus of Variations and Optimal Control.
• Flood: An open source neural networks C++ library (http://www.cimne.com/flood)
• GPOPS - Open Source Pseudospectral Optimal Control Solver in MATLAB (http://gpops.sourceforge.net)
• Elmer G. Wiens: Optimal Control (http://www.egwald.ca/optimalcontrol/index.php) - Applications of
Optimal Control Theory Using the Pontryagin Maximum Principle with interactive models.
• PPOPT - MATLAB Optimal Control Software (http://tomdyn.com/)
• PSOPT - Open Source Pseudospectral Optimal Control Solver in C++ (https://sites.google.com/a/psopt.org/
psopt/)
• GESOP - Graphical Environment for Simulation and OPtimization (http://www.astos.de/products/gesop)
Robust control
Robust control is a branch of control theory that explicitly deals with uncertainty in its approach to controller
design. Robust control methods are designed to function properly so long as uncertain parameters or disturbances are
within some (typically compact) set. Robust methods aim to achieve robust performance and/or stability in the
presence of bounded modelling errors.
The early methods of Bode and others were fairly robust; the state-space methods invented in the 1960s and 1970s
were sometimes found to lack robustness,[1] prompting research to improve them. This was the start of the theory of
Robust Control, which took shape in the 1980s and 1990s and is still active today.
In contrast with an adaptive control policy, a robust control policy is static; rather than adapting to measurements of
variations, the controller is designed to work assuming that certain variables will be unknown but, for example,
bounded.[2] [3]
When is a control method said to be robust?
Informally, a controller designed for a particular set of parameters is said to be robust if it would also work well
under a different set of assumptions. High-gain feedback is a simple example of a robust control method; with
sufficiently high gain, the effect of any parameter variations will be negligible. High-gain feedback is the principle
that allows simplified models of operational amplifiers and emitter-degenerated bipolar transistors to be used in a
variety of different settings. This idea was already well understood by Bode and Black in 1927.
The modern theory of robust control
The theory of robust control began in the late 1970s and early 1980s and soon developed a number of techniques for
dealing with bounded system uncertainty.[4] [5]
Probably the most important example of a robust control technique is H-infinity loop-shaping, which was developed
by Duncan McFarlane and Keith Glover of Cambridge University; this method minimizes the sensitivity of a system
over its frequency spectrum, and this guarantees that the system will not greatly deviate from expected trajectories
when disturbances enter the system.
Another example is LQG/LTR, which was developed to overcome the robustness problems of LQG control.
203
Robust control
References
[1]
[2]
[3]
[4]
[5]
M. Athans, Editorial on the LQG problem, IEEE Trans. Autom. Control 16 (1971), no. 6, 528.
J. Ackermann: Robuste Regelung. Springer-Verlag, 1993. (Section 1.5) In German; an english version is also available
Manfred Morari : Homepage (http:/ / control. ee. ethz. ch/ ~morari/ )
Safonov: editorial (http:/ / routh. usc. edu/ pub/ safonov/ safo97d. pdf)
Kemin Zhou: Essentials of Robust Control (http:/ / www. ee. lsu. edu/ kemin/ essentials. htm)
Further reading
• Ray, L.R.; Stengel, R.F. (1991). "Stochastic Robustness of Linear-Time-Invariant Control Systems" (http://
www.princeton.edu/~stengel/RayTAC1991.pdf). IEEE Trans. Automatic Control 36 (1): 82–87.
doi:10.1109/9.62270.
• V. Barbu and S. S. Sritharan (1998). "H-infinity Control Theory of Fluid Dynamics" (http://www.nps.edu/
Academics/Schools/GSEAS/SRI/R19.pdf). Proceedings of the Royal Royal Society of London, Ser. A. 545:
3009–3033.
• Dullerud, G.E.; Paganini, F. (2000). A Course in Robust Control Theory: A Convex Approach. Springer Verlag
New York. ISBN 0-387-98945-5.
• Bhattacharya, Apellat,Keel (2000). Robust Control-The Parametric Approach. Prentice Hall PTR.
ISBN 0-13-781576-X.
• Zhou, Kemin; Doyle C., John (1999). Essentials of Robust Control. Prentice Hall. ISBN 0-13-525833-2.
• Morari, Manfred; Zafiriou, Evanghelos (1989). Robust Process Control Theory. Prentice Hall.
ISBN 0-13-782153-0.
• Mahmoud S., Magdi; Munro, Neil (1989). Robust Control and Filtering for Time-Delay Systems. Marcel Dekker
Inc.. ISBN 0-8247-0327-8.
• Calafiore, G.; Dabbene, F. (ed.) (2006). Probabilistic and Randomized Methods for Design under Uncertainty.
Springer Verlag London Ltd.. ISBN 978-1-84628-094-8.
204
Adaptive control
Adaptive control
Adaptive control is the control method used by a controller which must adapt to a controlled system with
parameters which vary, or are initially uncertain. For example, as an aircraft flies, its mass will slowly decrease as a
result of fuel consumption; a control law is needed that adapts itself to such changing conditions. Adaptive control is
different from robust control in that it does not need a priori information about the bounds on these uncertain or
time-varying parameters; robust control guarantees that if the changes are within given bounds the control law need
not be changed, while adaptive control is concerned with control law changes themselves.
Parameter estimation
The foundation of adaptive control is parameter estimation. Common methods of estimation include recursive least
squares and gradient descent. Both of these methods provide update laws which are used to modify estimates in real
time (i.e., as the system operates). Lyapunov stability is used to derive these update laws and show convergence
criterion (typically persistent excitation). Projection (mathematics) and normalization are commonly used to improve
the robustness of estimation algorithms.
Classification of adaptive control techniques
In general one should distinguish between:
1. Feedforward Adaptive Control
2. Feedback Adaptive Control
as well as between
1. Direct Methods and
2. Indirect Methods
Direct methods are ones wherein the estimated parameters are those directly used in the adaptive controller. In
contrast, indirect methods are those in which the estimated parameters are used to calculate required controller
parameters[1]
There are several broad categories of feedback adaptive control (classification can vary):
• Dual Adaptive Controllers [based on Dual control theory]
• Optimal Dual Controllers [difficult to design]
• Suboptimal Dual Controllers
• Nondual Adaptive Controllers
•
•
•
•
•
Adaptive Pole Placement
Extremum Seeking Controllers
Iterative learning control
Gain scheduling
Model Reference Adaptive Controllers (MRACs) [incorporate a reference model defining desired closed loop
performance]
205
Adaptive control
206
• Gradient Optimization
MRACs [use local rule for
adjusting params when
performance differs from
reference. Ex.: "MIT rule".]
• Stability Optimized MRACs
• Model Identification Adaptive
Controllers (MIACs) [perform
System identification while the
system is running]
MRAC
• Cautious Adaptive Controllers
[use current SI to modify
control law, allowing for SI
uncertainty]
• Certainty Equivalent Adaptive
Controllers [take current SI to
be the true system, assume no
uncertainty]
• Nonparametric Adaptive
Controllers
• Parametric Adaptive Controllers
MIAC
• Explicit Parameter Adaptive Controllers
• Implicit Parameter Adaptive Controllers
Some special topics in adaptive control can be introduced as well:
1.
2.
3.
4.
5.
Adaptive Control Based on Discrete-Time Process Identification
Adaptive Control Based on the Model Reference Technique
Adaptive Control based on Continuous-Time Process Models
Adaptive Control of Multivariable Processes
Adaptive Control of Nonlinear Processes
Applications
When designing adaptive control systems, special consideration is necessary of convergence and robustness issues.
Lyapunov stability is typically used to derive control adaptation laws and show convergence.
Typical applications of adaptive control are (in general):
• Self-tuning of subsequently fixed linear controllers during the implementation phase for one operating point;
• Self-tuning of subsequently fixed robust controllers during the implementation phase for whole range of operating
points;
• Self-tuning of fixed controllers on request if the process behaviour changes due to ageing, drift, wear etc.;
• Adaptive control of linear controllers for nonlinear or time-varying processes;
• Adaptive control or self-tuning control of nonlinear controllers for nonlinear processes;
• Adaptive control or self-tuning control of multivariable controllers for multivariable processes (MIMO systems);
Usually these methods adapt the controllers to both the process statics and dynamics. In special cases the adaptation
can be limited to the static behavior alone, leading to adaptive control based on characteristic curves for the
steady-states or to extremum value control, optimizing the steady state. Hence, there are several ways to apply
adaptive control algorithms.
Adaptive control
References
[1] Astrom, Karl (2008). Adaptive Control. Dover. pp. 25–26.
Further reading
• K. J. Astrom and B. Wittenmark, Adaptive Control, Addison-Wesley, 1989, 2d ed. 1994.
External links
• Shankar Sastry and Marc Bodson, Adaptive Control: Stability, Convergence, and Robustness, Prentice-Hall,
1989-1994 (book) (http://www.ece.utah.edu/~bodson/acscr/)
• K. Sevcik: Tutorial on Model Reference Adaptive Control (Drexel University) (http://www.pages.drexel.edu/
~kws23/tutorials/MRAC/MRAC.html)
Stochastic control
Stochastic control is a subfield of control theory which deals with the existence of uncertainty in the data. The
designer assumes, in a Bayesian probability-driven fashion, that a random noise with known probability distribution
affects the state evolution and the observation of the controllers. Stochastic control aims to design the optimal
controller that performs the desired control task with minimum average cost despite the presence of these noises.[1]
An extremely well studied formulation in stochastic control is that of linear-quadratic-Gaussian problem. Here the
model is linear, and the objective function is the expected value of a quadratic form, and the additive disturbances
are distributed in a Gaussian manner. A basic result for discrete time centralized systems is the certainty
equivalence property:[2] that the optimal control solution in this case is the same as would be obtained in the
absence of the additive disturbances. This property is applicable to all systems that are merely linear and quadratic
(LQ), and the Gaussian assumption allows for the optimal control laws, that are based on the certainty-equivalence
property, to be linear functions of the observations of the controllers.
This property fails to hold for decentralized control, as was demonstrated by Witsenhausen in the celebrated
Witsenhausen's counterexample.
Any deviation from the above assumptions—a nonlinear state equation, a non-quadratic objective function, or noise
in the multiplicative parameters of the model—would cause the certainty equivalence property not to hold. In the
discrete-time case with uncertainty about the parameter values in the transition matrix and/or the control response
matrix of the state equation, but still with a linear state equation and quadratic objective function, a matrix Riccati
equation can still be obtained for iterating to each period's solution.[3] [2] ch.13 The discrete-time case of a
non-quadratic loss function but only additive disturbances can also be handled, albeit with more complications.[4]
References
[1] Definition from Answers.com (http:/ / www. answers. com/ topic/ stochastic-control-theory?cat=technology)
[2] Chow, Gregory P., Analysis and Control of Dynamic Economic Systems, Wiley, 1976.
[3] Turnovsky, Stephen, "Optimal stabilization policies for stochastic linear systems: The case of correlated multiplicative and additive
disturbances," Review of Economic Studies 43(1), 1976, 191-94.
[4] Mitchell, Douglas W., "Tractable risk sensitive control bassed on approximate expected utility," Economic Modelling, April 1990, 161-164.
207
Neural network
Neural network
The term neural network was traditionally used to refer to a network
or circuit of biological neurons.[1] The modern usage of the term often
refers to artificial neural networks, which are composed of artificial
neurons or nodes. Thus the term has two distinct usages:
1. Biological neural networks are made up of real biological neurons
that are connected or functionally related in a nervous system. In the
field of neuroscience, they are often identified as groups of neurons
that perform a specific physiological function in laboratory analysis.
2. Artificial neural networks are composed of interconnecting artificial
neurons (programming constructs that mimic the properties of
biological neurons). Artificial neural networks may either be used to
gain an understanding of biological neural networks, or for solving
artificial intelligence problems without necessarily creating a model
of a real biological system. The real, biological nervous system is
highly complex: artificial neural network algorithms attempt to
Simplified view of a feedforward artificial neural
abstract this complexity and focus on what may hypothetically
network
matter most from an information processing point of view. Good
performance (e.g. as measured by good predictive ability, low generalization error), or performance mimicking
animal or human error patterns, can then be used as one source of evidence towards supporting the hypothesis that
the abstraction really captured something important from the point of view of information processing in the brain.
Another incentive for these abstractions is to reduce the amount of computation required to simulate artificial
neural networks, so as to allow one to experiment with larger networks and train them on larger data sets.
This article focuses on the relationship between the two concepts; for detailed coverage of the two different concepts
refer to the separate articles: biological neural network and artificial neural network.
Overview
A biological neural network is composed of a group or groups of chemically connected or functionally associated
neurons. A single neuron may be connected to many other neurons and the total number of neurons and connections
in a network may be extensive. Connections, called synapses, are usually formed from axons to dendrites, though
dendrodendritic microcircuits[2] and other connections are possible. Apart from the electrical signaling, there are
other forms of signaling that arise from neurotransmitter diffusion, which have an effect on electrical signaling. As
such, neural networks are extremely complex.
Artificial intelligence and cognitive modeling try to simulate some properties of biological neural networks. While
similar in their techniques, the former has the aim of solving particular tasks, while the latter aims to build
mathematical models of biological neural systems.
In the artificial intelligence field, artificial neural networks have been applied successfully to speech recognition,
image analysis and adaptive control, in order to construct software agents (in computer and video games) or
autonomous robots. Most of the currently employed artificial neural networks for artificial intelligence are based on
statistical estimations, classification optimization and control theory.
The cognitive modelling field involves the physical or mathematical modeling of the behavior of neural systems;
ranging from the individual neural level (e.g. modeling the spike response curves of neurons to a stimulus), through
the neural cluster level (e.g. modelling the release and effects of dopamine in the basal ganglia) to the complete
208
Neural network
209
organism (e.g. behavioral modelling of the organism's response to stimuli). Artificial intelligence, cognitive
modelling, and neural networks are information processing paradigms inspired by the way biological neural systems
process data.
History of the neural network analogy
In the brain, spontaneous order appears to arise out of decentralized networks of simple units (neurons).
Neural network theory has served both to better identify how the neurons in the brain function and to provide the
basis for efforts to create artificial intelligence. The preliminary theoretical base for contemporary neural networks
was independently proposed by Alexander Bain[3] (1873) and William James[4] (1890). In their work, both thoughts
and body activity resulted from interactions among neurons within the brain.
For Bain,[3] every activity led to the firing of a certain set of neurons. When activities were repeated, the connections
between those neurons strengthened. According to his theory, this repetition was what led to the formation of
memory. The general scientific community at the time was skeptical of Bain’s[3] theory because it required what
appeared to be an inordinate number of neural connections within the brain. It is now apparent that the brain is
exceedingly complex and that the same brain “wiring” can handle multiple problems and inputs.
James’s[4] theory was similar to Bain’s,[3] however, he suggested that memories and actions resulted from electrical
currents flowing among the neurons in the brain. His model, by focusing on the flow of electrical currents, did not
require individual neural connections for each memory or action.
C. S. Sherrington[5] (1898) conducted experiments to test James’s theory. He ran electrical currents down the spinal
cords of rats. However, instead of the demonstrating an increase in electrical current as projected by James,
Sherrington found that the electrical current strength decreased as the testing continued over time. Importantly, this
work led to the discovery of the concept of habituation.
McCullouch and Pitts[6] (1943) created a computational model for neural networks based on mathematics and
algorithms. They called this model threshold logic. The model paved the way for neural network research to split
into two distinct approaches. One approach focused on biological processes in the brain and the other focused on the
application of neural networks to artificial intelligence.
In the late 1940s psychologist Donald Hebb[7] created a hypothesis of learning based on the mechanism of neural
plasticity that is now known as Hebbian learning. Hebbian learning is considered to be a 'typical' unsupervised
learning rule and its later variants were early models for long term potentiation. These ideas started being applied to
computational models in 1948 with Turing's B-type machines.
Farley and Clark[8] (1954) first used computational machines, then called calculators, to simulate a Hebbian network
at MIT. Other neural network computational machines were created by Rochester, Holland, Habit, and Duda[9]
(1956).
Rosenblatt[10] (1958) created the perceptron, an algorithm for pattern recognition based on a two-layer learning
computer network using simple addition and subtraction. With mathematical notation, Rosenblatt also described
circuitry not in the basic perceptron, such as the exclusive-or circuit, a circuit whose mathematical computation
could not be processed until after the backpropogation algorithm was created by Werbos[11] (1975).
The perceptron is essentially a linear classifier for classifying data
and an output function
specified by parameters
. Its parameters are adapted with an ad-hoc rule similar to
stochastic steepest gradient descent. Because the inner product is a linear operator in the input space, the perceptron
can only perfectly classify a set of data for which different classes are linearly separable in the input space, while it
often fails completely for non-separable data. While the development of the algorithm initially generated some
enthusiasm, partly because of its apparent relation to biological mechanisms, the later discovery of this inadequacy
caused such models to be abandoned until the introduction of non-linear models into the field.
Neural network
Neural network research stagnated after the publication of research of machine learning research by Minsky and
Papert[12] (1969). They discovered two key issues with the computational machines that processed neural networks.
The first issue was that single-layer neural networks were incapable of processing the exclusive-or circuit. The
second significant issue was that computers were not sophisticated enough to effectively handle the long run time
required by large neural networks. Neural network research slowed until computers achieved greater processing
power. Also key in later advances was the backpropogation algorithm which effectively solved the exclusive-or
problem (Werbos 1975).[11]
The cognitron (1975) designed by Kunihiko Fukushima[13] was an early multilayered neural network with a training
algorithm. The actual structure of the network and the methods used to set the interconnection weights change from
one neural strategy to another, each with its advantages and disadvantages. Networks can propagate information in
one direction only, or they can bounce back and forth until self-activation at a node occurs and the network settles on
a final state. The ability for bi-directional flow of inputs between neurons/nodes was produced with the Hopfield's
network (1982), and specialization of these node layers for specific purposes was introduced through the first hybrid
network.
The parallel distributed processing of the mid-1980s became popular under the name connectionism. The text by
Rummelhart and McClelland[14] (1986) provided a full exposition of the use connectionism in computers to simulate
neural processes.
The rediscovery of the backpropagation algorithm was probably the main reason behind the repopularisation of
neural networks after the publication of "Learning Internal Representations by Error Propagation" in 1986 (Though
backpropagation itself dates from 1969). The original network utilized multiple layers of weight-sum units of the
type
, where was a sigmoid function or logistic function such as used in logistic regression.
Training was done by a form of stochastic gradient descent. The employment of the chain rule of differentiation in
deriving the appropriate parameter updates results in an algorithm that seems to 'backpropagate errors', hence the
nomenclature. However it is essentially a form of gradient descent. Determining the optimal parameters in a model
of this type is not trivial, and local numerical optimization methods such as gradient descent can be sensitive to
initialization because of the presence of local minima of the training criterion. In recent times, networks with the
same architecture as the backpropagation network are referred to as multilayer perceptrons. This name does not
impose any limitations on the type of algorithm used for learning.
The backpropagation network generated much enthusiasm at the time and there was much controversy about whether
such learning could be implemented in the brain or not, partly because a mechanism for reverse signaling was not
obvious at the time, but most importantly because there was no plausible source for the 'teaching' or 'target' signal.
However, since 2006, several unsupervised learning procedures have been proposed for neural networks with one or
more layers, using so-called deep learning algorithms. These algorithms can be used to learn intermediate
representations, with or without a target signal, that capture the salient features of the distribution of sensory signals
arriving at each layer of the neural network.
210
Neural network
The brain, neural networks and computers
Neural networks, as used in artificial intelligence, have traditionally
been viewed as simplified models of neural processing in the brain,
even though the relation between this model and brain biological
architecture is debated, as little is known about how the brain actually
works.
A subject of current research in theoretical neuroscience is the question
surrounding the degree of complexity and the properties that individual
neural elements should have to reproduce something resembling
animal intelligence.
Historically, computers evolved from the von Neumann architecture,
which is based on sequential processing and execution of explicit
Computer simulation of the branching
instructions. On the other hand, the origins of neural networks are
architecture of the dendrites of pyramidal
based on efforts to model information processing in biological systems,
[15]
neurons.
which may rely largely on parallel processing as well as implicit
instructions based on recognition of patterns of 'sensory' input from
external sources. In other words, at its very heart a neural network is a complex statistical processor (as opposed to
being tasked to sequentially process and execute).
Neural coding is concerned with how sensory and other information is represented in the brain by neurons. The main
goal of studying neural coding is to characterize the relationship between the stimulus and the individual or ensemble
neuronal responses and the relationship among electrical activity of the neurons in the ensemble.[16] It is thought that
neurons can encode both digital and analog information.[17]
Neural networks and artificial intelligence
A neural network (NN), in the case of artificial neurons called artificial neural network (ANN) or simulated neural
network (SNN), is an interconnected group of natural or artificial neurons that uses a mathematical or computational
model for information processing based on a connectionistic approach to computation. In most cases an ANN is an
adaptive system that changes its structure based on external or internal information that flows through the network.
In more practical terms neural networks are non-linear statistical data modeling or decision making tools. They can
be used to model complex relationships between inputs and outputs or to find patterns in data.
However, the paradigm of neural networks - i.e., implicit, not explicit , learning is stressed - seems more to
correspond to some kind of natural intelligence than to the traditional symbol-based Artificial Intelligence, which
would stress, instead, rule-based learning.
Background
An artificial neural network involves a network of simple processing elements (artificial neurons) which can exhibit
complex global behavior, determined by the connections between the processing elements and element parameters.
Artificial neurons were first proposed in 1943 by Warren McCulloch, a neurophysiologist, and Walter Pitts, a
logician, who first collaborated at the University of Chicago.[18]
One classical type of artificial neural network is the recurrent Hopfield net.
In a neural network model simple nodes (which can be called by a number of names, including "neurons",
"neurodes", "Processing Elements" (PE) and "units"), are connected together to form a network of nodes — hence
the term "neural network". While a neural network does not have to be adaptive per se, its practical use comes with
algorithms designed to alter the strength (weights) of the connections in the network to produce a desired signal
flow.
211
Neural network
In modern software implementations of artificial neural networks the approach inspired by biology has more or less
been abandoned for a more practical approach based on statistics and signal processing. In some of these systems,
neural networks, or parts of neural networks (such as artificial neurons), are used as components in larger systems
that combine both adaptive and non-adaptive elements.
The concept of a neural network appears to have first been proposed by Alan Turing in his 1948 paper "Intelligent
Machinery".
Applications of natural and of artificial neural networks
The utility of artificial neural network models lies in the fact that they can be used to infer a function from
observations and also to use it. Unsupervised neural networks can also be used to learn representations of the input
that capture the salient characteristics of the input distribution, e.g., see the Boltzmann machine (1983), and more
recently, deep learning algorithms, which can implicitly learn the distribution function of the observed data.
Learning in neural networks is particularly useful in applications where the complexity of the data or task makes the
design of such functions by hand impractical.
The tasks to which artificial neural networks are applied tend to fall within the following broad categories:
• Function approximation, or regression analysis, including time series prediction and modeling.
• Classification, including pattern and sequence recognition, novelty detection and sequential decision making.
• Data processing, including filtering, clustering, blind signal separation and compression.
Application areas of ANNs include system identification and control (vehicle control, process control), game-playing
and decision making (backgammon, chess, racing), pattern recognition (radar systems, face identification, object
recognition), sequence recognition (gesture, speech, handwritten text recognition), medical diagnosis, financial
applications, data mining (or knowledge discovery in databases, "KDD"), visualization and e-mail spam filtering.
Neural networks and neuroscience
Theoretical and computational neuroscience is the field concerned with the theoretical analysis and computational
modeling of biological neural systems. Since neural systems are intimately related to cognitive processes and
behaviour, the field is closely related to cognitive and behavioural modeling.
The aim of the field is to create models of biological neural systems in order to understand how biological systems
work. To gain this understanding, neuroscientists strive to make a link between observed biological processes (data),
biologically plausible mechanisms for neural processing and learning (biological neural network models) and theory
(statistical learning theory and information theory).
Types of models
Many models are used in the field, each defined at a different level of abstraction and trying to model different
aspects of neural systems. They range from models of the short-term behaviour of individual neurons, through
models of how the dynamics of neural circuitry arise from interactions between individual neurons, to models of
how behaviour can arise from abstract neural modules that represent complete subsystems. These include models of
the long-term and short-term plasticity of neural systems and its relation to learning and memory, from the individual
neuron to the system level.
212
Neural network
Current research
While initially research had been concerned mostly with the electrical characteristics of neurons, a particularly
important part of the investigation in recent years has been the exploration of the role of neuromodulators such as
dopamine, acetylcholine, and serotonin on behaviour and learning.
Biophysical models, such as BCM theory, have been important in understanding mechanisms for synaptic plasticity,
and have had applications in both computer science and neuroscience. Research is ongoing in understanding the
computational algorithms used in the brain, with some recent biological evidence for radial basis networks and
neural backpropagation as mechanisms for processing data.
Computational devices have been created in CMOS for both biophysical simulation and neuromorphic computing.
More recent efforts show promise for creating nanodevices[19] for very large scale principal components analyses
and convolution. If successful, these efforts could usher in a new era of neural computing[20] that is a step beyond
digital computing, because it depends on learning rather than programming and because it is fundamentally analog
rather than digital even though the first instantiations may in fact be with CMOS digital devices.
Architecture
The basic architecture consists of three types of neuron layers: input, hidden, and output. In feed-forward networks,
the signal flow is from input to output units, strictly in a feed-forward direction. The data processing can extend over
multiple layers of units, but no feedback connections are present. Recurrent networks contain feedback connections.
Contrary to feed-forward networks, the dynamical properties of the network are important. In some cases, the
activation values of the units undergo a relaxation process such that the network will evolve to a stable state in which
these activations do not change anymore.
In other applications, the changes of the activation values of the output neurons are significant, such that the
dynamical behavior constitutes the output of the network. Other neural network architectures include adaptive
resonance theory maps and competitive networks.
Criticism
A common criticism of neural networks, particularly in robotics, is that they require a large diversity of training for
real-world operation. This is not surprising, since any learning machine needs sufficient representative examples in
order to capture the underlying structure that allows it to generalize to new cases. Dean Pomerleau, in his research
presented in the paper "Knowledge-based Training of Artificial Neural Networks for Autonomous Robot Driving,"
uses a neural network to train a robotic vehicle to drive on multiple types of roads (single lane, multi-lane, dirt, etc.).
A large amount of his research is devoted to (1) extrapolating multiple training scenarios from a single training
experience, and (2) preserving past training diversity so that the system does not become overtrained (if, for
example, it is presented with a series of right turns – it should not learn to always turn right). These issues are
common in neural networks that must decide from amongst a wide variety of responses, but can be dealt with in
several ways, for example by randomly shuffling the training examples, by using a numerical optimization algorithm
that does not take too large steps when changing the network connections following an example, or by grouping
examples in so-called mini-batches.
A. K. Dewdney, a former Scientific American columnist, wrote in 1997, "Although neural nets do solve a few toy
problems, their powers of computation are so limited that I am surprised anyone takes them seriously as a general
problem-solving tool." (Dewdney, p. 82)
Arguments for Dewdney's position are that to implement large and effective software neural networks, much
processing and storage resources need to be committed. While the brain has hardware tailored to the task of
processing signals through a graph of neurons, simulating even a most simplified form on Von Neumann technology
may compel a NN designer to fill many millions of database rows for its connections - which can consume vast
213
Neural network
amounts of computer memory and hard disk space. Furthermore, the designer of NN systems will often need to
simulate the transmission of signals through many of these connections and their associated neurons - which must
often be matched with incredible amounts of CPU processing power and time. While neural networks often yield
effective programs, they too often do so at the cost of efficiency (they tend to consume considerable amounts of time
and money).
Arguments against Dewdney's position are that neural nets have been successfully used to solve many complex and
diverse tasks, ranging from autonomously flying aircraft [21] to detecting credit card fraud [22].
Technology writer Roger Bridgman commented on Dewdney's statements about neural nets:
Neural networks, for instance, are in the dock not only because they have been hyped to high heaven, (what
hasn't?) but also because you could create a successful net without understanding how it worked: the bunch of
numbers that captures its behaviour would in all probability be "an opaque, unreadable table...valueless as a
scientific resource".
In spite of his emphatic declaration that science is not technology, Dewdney seems here to pillory neural nets
as bad science when most of those devising them are just trying to be good engineers. An unreadable table that
a useful machine could read would still be well worth having.[23]
In response to this kind of criticism, one should note that although it is true that analyzing what has been learned by
an artificial neural network is difficult, it is much easier to do so than to analyze what has been learned by a
biological neural network. Furthermore, researchers involved in exploring learning algorithms for neural networks
are gradually uncovering generic principles which allow a learning machine to be successful. For example, Bengio
and LeCun (2007) wrote an article regarding local vs non-local learning, as well as shallow vs deep architecture [24].
Some other criticisms came from believers of hybrid models (combining neural networks and symbolic approaches).
They advocate the intermix of these two approaches and believe that hybrid models can better capture the
mechanisms of the human mind (Sun and Bookman 1990).
References
[1] J. J. HOPFIELD Neural networks and physical systems with emergent collective computational abilities. Proc. NatL Acad. Sci. USA Vol. 79,
pp. 2554-2558, April 1982 Biophysics (http:/ / www. pnas. org/ content/ 79/ 8/ 2554. full. pdf)
[2] Arbib, p.666
[3] Bain (1873). Mind and Body: The Theories of Their Relation. New York: D. Appleton and Company.
[4] James (1890). The Principles of Psychology. New York: H. Holt and Company.
[5] Sherrington, C.S.. "Experiments in Examination of the Peripheral Distribution of the Fibers of the Posterior Roots of Some Spinal Nerves".
Proceedings of the Royal Society of London 190: 45–186.
[6] McCullock, Warren; Walter Pitts (1943). "A Logical Calculus of Ideas Immanent in Nervous Activity". Bulletin of Mathematical Biophysics
5 (4): 115–133. doi:10.1007/BF02478259.
[7] Hebb, Donald (1949). The Organization of Behavior. New York: Wiley.
[8] Farley, B; W.A. Clark (1954). "Simulation of Self-Organizing Systems by Digital Computer". IRE Transactions on Information Theory 4 (4):
76–84. doi:10.1109/TIT.1954.1057468.
[9] Rochester, N.; J.H. Holland, L.H. Habit, and W.L. Duda (1956). "Tests on a cell assembly theory of the action of the brain, using a large
digital computer". IRE Transactions on Information Theory 2 (3): 80–93. doi:10.1109/TIT.1956.1056810.
[10] Rosenblatt, F. (1958). "The Perceptron: A Probalistic Model For Information Storage And Organization In The Brain". Psychological
Review 65 (6): 386–408. doi:10.1037/h0042519. PMID 13602029.
[11] Werbos, P.J. (1975). Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences.
[12] Minsky, M.; S. Papert (1969). An Introduction to Computational Geometry. MIT Press. ISBN 0262630222.
[13] Fukushima, Kunihiko (1975). "Cognitron: A self-organizing multilayered neural network". Biological Cybernetics 20 (3–4): 121–136.
doi:10.1007/BF00342633. PMID 1203338.
[14] Rummelhart, D.E; James McClelland (1986). Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Cambridge:
MIT Press.
[15] "PLoS Computational Biology Issue Image". PLoS Computational Biology 6 (8): ev06.ei08. 2010. doi:10.1371/image.pcbi.v06.i08.
[16] Brown EN, Kass RE, Mitra PP. (2004). "Multiple neural spike train data analysis: state-of-the-art and future challenges". Nature
Neuroscience 7 (5): 456–61. doi:10.1038/nn1228. PMID 15114358.
214
Neural network
[17] Spike arrival times: A highly efficient coding scheme for neural networks (http:/ / pop. cerco. ups-tlse. fr/ fr_vers/ documents/
thorpe_sj_90_91. pdf), SJ Thorpe - Parallel processing in neural systems, 1990
[18] McCulloch, Warren; Pitts, Walter, "A Logical Calculus of Ideas Immanent in Nervous Activity", 1943, Bulletin of Mathematical Biophysics
5:115-133.
[19] Yang, J. J.; Pickett, M. D.; Li, X. M.; Ohlberg, D. A. A.; Stewart, D. R.; Williams, R. S. Nat. Nanotechnol. 2008, 3, 429–433.
[20] Strukov, D. B.; Snider, G. S.; Stewart, D. R.; Williams, R. S. Nature 2008, 453, 80–83.
[21] http:/ / www. nasa. gov/ centers/ dryden/ news/ NewsReleases/ 2003/ 03-49. html
[22] http:/ / www. visa. ca/ en/ about/ visabenefits/ innovation. cfm
[23] Roger Bridgman's defence of neural networks (http:/ / members. fortunecity. com/ templarseries/ popper. html)
[24] http:/ / www. iro. umontreal. ca/ ~lisa/ publications2/ index. php/ publications/ show/ 4
Further reading
• Arbib, Michael A. (Ed.) (1995). The Handbook of Brain Theory and Neural Networks.
• Alspector, U.S. Patent 4874963 (http://www.google.com/patents?vid=4874963) "Neuromorphic learning
networks". October 17, 1989.
• Agre, Philip E. (1997). Computation and Human Experience. Cambridge University Press. ISBN 0-521-38603-9.,
p. 80
• Yaneer Bar-Yam (2003). Dynamics of Complex Systems, Chapter 2 (http://necsi.edu/publications/dcs/
Bar-YamChap2.pdf).
• Yaneer Bar-Yam (2003). Dynamics of Complex Systems, Chapter 3 (http://necsi.edu/publications/dcs/
Bar-YamChap3.pdf).
• Yaneer Bar-Yam (2005). Making Things Work (http://necsi.edu/publications/mtw/). See chapter 3.
• Bertsekas, Dimitri P. (1999). Nonlinear Programming. ISBN 1886529000.
• Bertsekas, Dimitri P. & Tsitsiklis, John N. (1996). Neuro-dynamic Programming. ISBN 1886529108.
• Bhadeshia H. K. D. H. (1992). "Neural Networks in Materials Science" (http://www.msm.cam.ac.uk/
phase-trans/abstracts/neural.review.pdf). ISIJ International 39 (10): 966–979.
doi:10.2355/isijinternational.39.966.
• Boyd, Stephen & Vandenberghe, Lieven (2004). Convex Optimization (http://www.stanford.edu/~boyd/
cvxbook/).
• Dewdney, A. K. (1997). Yes, We Have No Neutrons: An Eye-Opening Tour through the Twists and Turns of Bad
Science. Wiley, 192 pp. ISBN 0471108065. See chapter 5.
• Egmont-Petersen, M., de Ridder, D., Handels, H. (2002). "Image processing with neural networks - a review".
Pattern Recognition 35 (10): 2279–2301. doi:10.1016/S0031-3203(01)00178-9.
• Fukushima, K. (1975). "Cognitron: A Self-Organizing Multilayered Neural Network". Biological Cybernetics 20
(3–4): 121–136. doi:10.1007/BF00342633. PMID 1203338.
• Frank, Michael J. (2005). "Dynamic Dopamine Modulation in the Basal Ganglia: A Neurocomputational Account
of Cognitive Deficits in Medicated and Non-medicated Parkinsonism". Journal of Cognitive Neuroscience 17 (1):
51–72. doi:10.1162/0898929052880093. PMID 15701239.
• Gardner, E.J., & Derrida, B. (1988). "Optimal storage properties of neural network models". Journal of Physics a
21: 271–284. doi:10.1088/0305-4470/21/1/031.
• Hadzibeganovic, Tarik & Cannas, Sergio A. (2009). "A Tsallis' statistics based neural network model for novel
word learning". Physica A: Statistical Mechanics and its Applications 388 (5): 732–746.
doi:10.1016/j.physa.2008.10.042.
• Krauth, W., & Mezard, M. (1989). "Storage capacity of memory with binary couplings". Journal de Physique 50
(20): 3057–3066. doi:10.1051/jphys:0198900500200305700.
• Maass, W., & Markram, H. (2002). "On the computational power of recurrent circuits of spiking neurons" (http://
www.igi.tugraz.at/maass/publications.html). Journal of Computer and System Sciences 69 (4): 593–616.
doi:10.1016/j.jcss.2004.04.001.
215
Neural network
• MacKay, David (2003). Information Theory, Inference, and Learning Algorithms (http://www.inference.phy.
cam.ac.uk/mackay/itprnn/book.html).
• Mandic, D. & Chambers, J. (2001). Recurrent Neural Networks for Prediction: Architectures, Learning
algorithms and Stability. Wiley. ISBN 0471495174.
• Minsky, M. & Papert, S. (1969). An Introduction to Computational Geometry. MIT Press. ISBN 0262630222.
• Muller, P. & Insua, D.R. (1995). "Issues in Bayesian Analysis of Neural Network Models". Neural Computation
10 (3): 571–592. PMID 9527841.
• Reilly, D.L., Cooper, L.N. & Elbaum, C. (1982). "A Neural Model for Category Learning". Biological
Cybernetics 45: 35–41. doi:10.1007/BF00387211.
• Rosenblatt, F. (1962). Principles of Neurodynamics. Spartan Books.
• Sun, R. & Bookman,L. (eds.) (1994.). Computational Architectures Integrating Neural and Symbolic Processes..
Kluwer Academic Publishers, Needham, MA..
• Sutton, Richard S. & Barto, Andrew G. (1998). Reinforcement Learning : An introduction (http://www.cs.
ualberta.ca/~sutton/book/the-book.html).
• Van den Bergh, F. Engelbrecht, AP. Cooperative Learning in Neural Networks using Particle Swarm Optimizers.
CIRG 2000.
• Wilkes, A.L. & Wade, N.J. (1997). "Bain on Neural Networks". Brain and Cognition 33 (3): 295–305.
doi:10.1006/brcg.1997.0869. PMID 9126397.
• Wasserman, P.D. (1989). Neural computing theory and practice. Van Nostrand Reinhold. ISBN 0442207433.
• Jeffrey T. Spooner, Manfredi Maggiore, Raul Ord onez, and Kevin M. Passino, Stable Adaptive Control and
Estimation for Nonlinear Systems: Neural and Fuzzy Approximator Techniques, John Wiley and Sons, NY, 2002.
• Peter Dayan, L.F. Abbott. Theoretical Neuroscience. MIT Press. ISBN 0262041995.
• Wulfram Gerstner, Werner Kistler. Spiking Neuron Models:Single Neurons, Populations, Plasticity. Cambridge
University Press. ISBN 0521890799.
• Steeb, W-H (2008). The Nonlinear Workbook: Chaos, Fractals, Neural Networks, Genetic Algorithms, Gene
Expression Programming, Support Vector Machine, Wavelets, Hidden Markov Models, Fuzzy Logic with C++,
Java and SymbolicC++ Programs: 4th edition. World Scientific Publishing. ISBN 981-281-852-9.
External links
• A Brief Introduction to Neural Networks (D. Kriesel) (http://www.dkriesel.com/en/science/neural_networks)
- Illustrated, bilingual manuscript about artificial neural networks; Topics so far: Perceptrons, Backpropagation,
Radial Basis Functions, Recurrent Neural Networks, Self Organizing Maps, Hopfield Networks.
• LearnArtificialNeuralNetworks (http://www.learnartificialneuralnetworks.com/robotcontrol.html) - Robot
control and neural networks
• Review of Neural Networks in Materials Science (http://www.msm.cam.ac.uk/phase-trans/abstracts/neural.
review.html)
• Artificial Neural Networks Tutorial in three languages (Univ. Politécnica de Madrid) (http://www.gc.ssr.upm.
es/inves/neural/ann1/anntutorial.html)
• Introduction to Neural Networks and Knowledge Modeling (http://www.makhfi.com/tutorial/introduction.
htm)
• Another introduction to ANN (http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html)
• Next Generation of Neural Networks (http://youtube.com/watch?v=AyzOUbkUf3M) - Google Tech Talks
• Performance of Neural Networks (http://www.msm.cam.ac.uk/phase-trans/2009/performance.html)
• Neural Networks and Information (http://www.msm.cam.ac.uk/phase-trans/2009/
review_Bhadeshia_SADM.pdf)
• PMML Representation (http://www.dmg.org/v4-0/NeuralNetwork.html) - Standard way to represent neural
networks
216
Fuzzy logic
217
Fuzzy logic
Fuzzy logic is a form of many-valued logic; it deals with reasoning that is approximate rather than fixed and exact.
In contrast with traditional logic theory, where binary sets have two-valued logic: true or false, fuzzy logic variables
may have a truth value that ranges in degree between 0 and 1. Fuzzy logic has been extended to handle the concept
of partial truth, where the truth value may range between completely true and completely false.[1] Furthermore, when
linguistic variables are used, these degrees may be managed by specific functions.
Fuzzy logic began with the 1965 proposal of fuzzy set theory by Lotfi Zadeh.[2]
many fields, from control theory to artificial intelligence.
[3]
Fuzzy logic has been applied to
Overview
The reasoning in fuzzy logic is similar to human reasoning. It allows for approximate values and inferences as well
as incomplete or ambiguous data (fuzzy data) as opposed to only relying on crisp data (binary yes/no choices). Fuzzy
logic is able to process incomplete data and provide approximate solutions to problems other methods find difficult
to solve. Terminology used in fuzzy logic not used in other methods are: very high, increasing, somewhat decreased,
reasonable and very low. [4]
Degrees of truth
Fuzzy logic and probabilistic logic are mathematically similar – both have truth values ranging between 0 and 1 –
but conceptually distinct, due to different interpretations—see interpretations of probability theory. Fuzzy logic
corresponds to "degrees of truth", while probabilistic logic corresponds to "probability, likelihood"; as these differ,
fuzzy logic and probabilistic logic yield different models of the same real-world situations.
Both degrees of truth and probabilities range between 0 and 1 and hence may seem similar at first. For example, let a
100 ml glass contain 30 ml of water. Then we may consider two concepts: Empty and Full. The meaning of each of
them can be represented by a certain fuzzy set. Then one might define the glass as being 0.7 empty and 0.3 full. Note
that the concept of emptiness would be subjective and thus would depend on the observer or designer. Another
designer might equally well design a set membership function where the glass would be considered full for all values
down to 50 ml. It is essential to realize that fuzzy logic uses truth degrees as a mathematical model of the vagueness
phenomenon while probability is a mathematical model of ignorance.
Applying truth values
A basic application might characterize subranges of a continuous variable. For instance, a temperature measurement
for anti-lock brakes might have several separate membership functions defining particular temperature ranges needed
to control the brakes properly. Each function maps the same temperature value to a truth value in the 0 to 1 range.
These truth values can then be used to determine how the brakes should be controlled.
Fuzzy logic temperature
Fuzzy logic
In this image, the meaning of the expressions cold, warm, and hot is represented by functions mapping a temperature
scale. A point on that scale has three "truth values"—one for each of the three functions. The vertical line in the
image represents a particular temperature that the three arrows (truth values) gauge. Since the red arrow points to
zero, this temperature may be interpreted as "not hot". The orange arrow (pointing at 0.2) may describe it as "slightly
warm" and the blue arrow (pointing at 0.8) "fairly cold".
Linguistic variables
While variables in mathematics usually take numerical values, in fuzzy logic applications, the non-numeric linguistic
variables are often used to facilitate the expression of rules and facts.[5]
A linguistic variable such as age may have a value such as young or its antonym old. However, the great utility of
linguistic variables is that they can be modified via linguistic hedges applied to primary terms. The linguistic hedges
can be associated with certain functions.
Example
Fuzzy set theory defines fuzzy operators on fuzzy sets. The problem in applying this is that the appropriate fuzzy
operator may not be known. For this reason, fuzzy logic usually uses IF-THEN rules, or constructs that are
equivalent, such as fuzzy associative matrices.
Rules are usually expressed in the form:
IF variable IS property THEN action
For example, a simple temperature regulator that uses a fan might look like this:
IF temperature IS very cold THEN stop fan
IF temperature IS cold THEN turn down fan
IF temperature IS normal THEN maintain level
IF temperature IS hot THEN speed up fan
There is no "ELSE" – all of the rules are evaluated, because the temperature might be "cold" and "normal" at the
same time to different degrees.
The AND, OR, and NOT operators of boolean logic exist in fuzzy logic, usually defined as the minimum, maximum,
and complement; when they are defined this way, they are called the Zadeh operators. So for the fuzzy variables x
and y:
NOT x = (1 - truth(x))
x AND y = minimum(truth(x), truth(y))
x OR y = maximum(truth(x), truth(y))
There are also other operators, more linguistic in nature, called hedges that can be applied. These are generally
adverbs such as "very", or "somewhat", which modify the meaning of a set using a mathematical formula.
218
Fuzzy logic
Logical analysis
In mathematical logic, there are several formal systems of "fuzzy logic"; most of them belong among so-called
t-norm fuzzy logics.
Propositional fuzzy logics
The most important propositional fuzzy logics are:
• Monoidal t-norm-based propositional fuzzy logic MTL is an axiomatization of logic where conjunction is defined
by a left continuous t-norm, and implication is defined as the residuum of the t-norm. Its models correspond to
MTL-algebras that are prelinear commutative bounded integral residuated lattices.
• Basic propositional fuzzy logic BL is an extension of MTL logic where conjunction is defined by a continuous
t-norm, and implication is also defined as the residuum of the t-norm. Its models correspond to BL-algebras.
• Łukasiewicz fuzzy logic is the extension of basic fuzzy logic BL where standard conjunction is the Łukasiewicz
t-norm. It has the axioms of basic fuzzy logic plus an axiom of double negation, and its models correspond to
MV-algebras.
• Gödel fuzzy logic is the extension of basic fuzzy logic BL where conjunction is Gödel t-norm. It has the axioms
of BL plus an axiom of idempotence of conjunction, and its models are called G-algebras.
• Product fuzzy logic is the extension of basic fuzzy logic BL where conjunction is product t-norm. It has the
axioms of BL plus another axiom for cancellativity of conjunction, and its models are called product algebras.
• Fuzzy logic with evaluated syntax (sometimes also called Pavelka's logic), denoted by EVŁ, is a further
generalization of mathematical fuzzy logic. While the above kinds of fuzzy logic have traditional syntax and
many-valued semantics, in EVŁ is evaluated also syntax. This means that each formula has an evaluation.
Axiomatization of EVŁ stems from Łukasziewicz fuzzy logic. A generalization of classical Gödel completeness
theorem is provable in EVŁ.
Predicate fuzzy logics
These extend the above-mentioned fuzzy logics by adding universal and existential quantifiers in a manner similar to
the way that predicate logic is created from propositional logic. The semantics of the universal (resp. existential)
quantifier in t-norm fuzzy logics is the infimum (resp. supremum) of the truth degrees of the instances of the
quantified subformula.
Decidability issues for fuzzy logic
The notions of a "decidable subset" and "recursively enumerable subset" are basic ones for classical mathematics and
classical logic. Then, the question of a suitable extension of such concepts to fuzzy set theory arises. A first proposal
in such a direction was made by E.S. Santos by the notions of fuzzy Turing machine, Markov normal fuzzy algorithm
and fuzzy program (see Santos 1970). Successively, L. Biacino and G. Gerla showed that such a definition is not
adequate and therefore proposed the following one. Ü denotes the set of rational numbers in [0,1]. A fuzzy subset s :
S
[0,1] of a set S is recursively enumerable if a recursive map h : S×N
Ü exists such that, for every x in S, the
function h(x,n) is increasing with respect to n and s(x) = lim h(x,n). We say that s is decidable if both s and its
complement –s are recursively enumerable. An extension of such a theory to the general case of the L-subsets is
proposed in Gerla 2006. The proposed definitions are well related with fuzzy logic. Indeed, the following theorem
holds true (provided that the deduction apparatus of the fuzzy logic satisfies some obvious effectiveness property).
Theorem. Any axiomatizable fuzzy theory is recursively enumerable. In particular, the fuzzy set of logically true
formulas is recursively enumerable in spite of the fact that the crisp set of valid formulas is not recursively
enumerable, in general. Moreover, any axiomatizable and complete theory is decidable.
219
Fuzzy logic
It is an open question to give supports for a Church thesis for fuzzy logic claiming that the proposed notion of
recursive enumerability for fuzzy subsets is the adequate one. To this aim, further investigations on the notions of
fuzzy grammar and fuzzy Turing machine should be necessary (see for example Wiedermann's paper). Another open
question is to start from this notion to find an extension of Gödel’s theorems to fuzzy logic.
Fuzzy databases
Once fuzzy relations are defined, it is possible to develop fuzzy relational databases. The first fuzzy relational
database, FRDB, appeared in Maria Zemankova's dissertation. Later, some other models arose like the Buckles-Petry
model, the Prade-Testemale Model, the Umano-Fukami model or the GEFRED model by J.M. Medina, M.A. Vila et
al. In the context of fuzzy databases, some fuzzy querying languages have been defined, highlighting the SQLf by P.
Bosc et al. and the FSQL by J. Galindo et al. These languages define some structures in order to include fuzzy
aspects in the SQL statements, like fuzzy conditions, fuzzy comparators, fuzzy constants, fuzzy constraints, fuzzy
thresholds, linguistic labels and so on.
Comparison to probability
Fuzzy logic and probability are different ways of expressing uncertainty. While both fuzzy logic and probability
theory can be used to represent subjective belief, fuzzy set theory uses the concept of fuzzy set membership (i.e.,
how much a variable is in a set), and probability theory uses the concept of subjective probability (i.e., how probable
do I think that a variable is in a set). While this distinction is mostly philosophical, the fuzzy-logic-derived
possibility measure is inherently different from the probability measure, hence they are not directly equivalent.
However, many statisticians are persuaded by the work of Bruno de Finetti that only one kind of mathematical
uncertainty is needed and thus fuzzy logic is unnecessary. On the other hand, Bart Kosko argues that probability is a
subtheory of fuzzy logic, as probability only handles one kind of uncertainty. He also claims to have proven a
derivation of Bayes' theorem from the concept of fuzzy subsethood. Lotfi Zadeh argues that fuzzy logic is different
in character from probability, and is not a replacement for it. He fuzzified probability to fuzzy probability and also
generalized it to what is called possibility theory. (cf.[6] )
Notes
[1] Novák, V., Perfilieva, I. and Močkoř, J. (1999) Mathematical principles of fuzzy logic Dodrecht: Kluwer Academic. ISBN 0-7923-8595-0
[2] "Fuzzy Logic" (http:/ / plato. stanford. edu/ entries/ logic-fuzzy/ ). Stanford Encyclopedia of Philosophy. Stanford University. 2006-07-23. .
Retrieved 2008-09-29.
[3] Zadeh, L.A. (1965). "Fuzzy sets", Information and Control 8 (3): 338–353.
[4] James A. O'Brien; George M. Marakas (2011). Management Information Systesm (10th ed.). New York: McGraw Hill. pp. 431.
[5] Zadeh, L. A. et al. 1996 Fuzzy Sets, Fuzzy Logic, Fuzzy Systems, World Scientific Press, ISBN 9810224214
[6] Novák, V. Are fuzzy sets a reasonable tool for modeling vague phenomena?, Fuzzy Sets and Systems 156 (2005) 341—348.
Bibliography
• Von Altrock, Constantin (1995). Fuzzy logic and NeuroFuzzy applications explained. Upper Saddle River, NJ:
Prentice Hall PTR. ISBN 0-13-368465-2.
• Arabacioglu, B. C. (2010). "Using fuzzy inference system for architectural space analysis" (http://www.
sciencedirect.com/science/article/pii/S1568494609002014). Applied Soft Computing 10 (3): 926–937.
• Biacino, L.; Gerla, G. (2002). "Fuzzy logic, continuity and effectiveness". Archive for Mathematical Logic 41 (7):
643–667. doi:10.1007/s001530100128. ISSN 0933-5846.
• Cox, Earl (1994). The fuzzy systems handbook: a practitioner's guide to building, using, maintaining fuzzy
systems. Boston: AP Professional. ISBN 0-12-194270-8.
• Gerla, Giangiacomo (2006). "Effectiveness and Multivalued Logics". Journal of Symbolic Logic 71 (1): 137–162.
doi:10.2178/jsl/1140641166. ISSN 0022-4812.
220
Fuzzy logic
• Hájek, Petr (1998). Metamathematics of fuzzy logic. Dordrecht: Kluwer. ISBN 0792352386.
• Hájek, Petr (1995). "Fuzzy logic and arithmetical hierarchy". Fuzzy Sets and Systems 3 (8): 359–363.
doi:10.1016/0165-0114(94)00299-M. ISSN 0165-0114.
• Halpern, Joseph Y. (2003). Reasoning about uncertainty. Cambridge, Mass: MIT Press. ISBN 0-262-08320-5.
• Höppner, Frank; Klawonn, F.; Kruse, R.; Runkler, T. (1999). Fuzzy cluster analysis: methods for classification,
data analysis and image recognition. New York: John Wiley. ISBN 0-471-98864-2.
• Ibrahim, Ahmad M. (1997). Introduction to Applied Fuzzy Electronics. Englewood Cliffs, N.J: Prentice Hall.
ISBN 0-13-206400-6.
• Klir, George J.; Folger, Tina A. (1988). Fuzzy sets, uncertainty, and information. Englewood Cliffs, N.J: Prentice
Hall. ISBN 0-13-345984-5.
• Klir, George J.; St Clair, Ute H.; Yuan, Bo (1997). Fuzzy set theory: foundations and applications. Englewood
Cliffs, NJ: Prentice Hall. ISBN 0133410587.
• Klir, George J.; Yuan, Bo (1995). Fuzzy sets and fuzzy logic: theory and applications. Upper Saddle River, NJ:
Prentice Hall PTR. ISBN 0-13-101171-5.
• Kosko, Bart (1993). Fuzzy thinking: the new science of fuzzy logic. New York: Hyperion. ISBN 0-7868-8021-X.
• Kosko, Bart; Isaka, Satoru (July 1993). "Fuzzy Logic". Scientific American 269 (1): 76–81.
doi:10.1038/scientificamerican0793-76.
• Montagna, F. (2001). "Three complexity problems in quantified fuzzy logic". Studia Logica 68 (1): 143–152.
doi:10.1023/A:1011958407631. ISSN 0039-3215.
• Mundici, Daniele; Cignoli, Roberto; D'Ottaviano, Itala M. L. (1999). Algebraic foundations of many-valued
reasoning. Dodrecht: Kluwer Academic. ISBN 0-7923-6009-5.
• Novák, Vilém (1989). Fuzzy Sets and Their Applications. Bristol: Adam Hilger. ISBN 0-85274-583-4.
• Novák, Vilém (2005). "On fuzzy type theory". Fuzzy Sets and Systems 149 (2): 235–273.
doi:10.1016/j.fss.2004.03.027.
• Novák, Vilém; Perfilieva, Irina; Močkoř, Jiří (1999). Mathematical principles of fuzzy logic. Dordrecht: Kluwer
Academic. ISBN 0-7923-8595-0.
• Onses, Richard (1996). Second Order Experton: A new Tool for Changing Paradigms in Country Risk
Calculation. ISBN 8477195587.
• Onses, Richard (1994). Détermination de l´incertitude inhérente aux investissements en Amérique Latine sur la
base de la théorie des sous ensembles flous. Barcelona. ISBN 8447508811.
• Passino, Kevin M.; Yurkovich, Stephen (1998). Fuzzy control. Boston: Addison-Wesley. ISBN 020118074X.
• Pedrycz, Witold; Gomide, Fernando (2007). Fuzzy systems engineering: Toward Human-Centerd Computing.
Hoboken: Wiley-Interscience. ISBN 978047178857-7.
• Pu, Pao Ming; Liu, Ying Ming (1980). "Fuzzy topology. I. Neighborhood structure of a fuzzy point and
Moore-Smith convergence". Journal of Mathematical Analysis and Applications 76 (2): 571–599.
doi:10.1016/0022-247X(80)90048-7. ISSN 0022-247X
• Santos, Eugene S. (1970). "Fuzzy Algorithms". Information and Control 17 (4): 326–339.
doi:10.1016/S0019-9958(70)80032-8.
• Scarpellini, Bruno (1962). "Die Nichaxiomatisierbarkeit des unendlichwertigen Prädikatenkalküls von
Łukasiewicz". Journal of Symbolic Logic (Association for Symbolic Logic) 27 (2): 159–170.
doi:10.2307/2964111. ISSN 0022-4812. JSTOR 2964111.
• Steeb, Willi-Hans (2008). The Nonlinear Workbook: Chaos, Fractals, Cellular Automata, Neural Networks,
Genetic Algorithms, Gene Expression Programming, Support Vector Machine, Wavelets, Hidden Markov Models,
Fuzzy Logic with C++, Java and SymbolicC++ Programs: 4edition. World Scientific. ISBN 981-281-852-9.
• Wiedermann, J. (2004). "Characterizing the super-Turing computing power and efficiency of classical fuzzy
Turing machines". Theor. Comput. Sci. 317 (1-3): 61–69. doi:10.1016/j.tcs.2003.12.004.
221
Fuzzy logic
• Yager, Ronald R.; Filev, Dimitar P. (1994). Essentials of fuzzy modeling and control. New York: Wiley.
ISBN 0-471-01761-2.
• Van Pelt, Miles (2008). Fuzzy Logic Applied to Daily Life. Seattle, WA: No No No No Press.
ISBN 0-252-16341-9.
• Wilkinson, R.H. (1963). "A method of generating functions of several variables using analog diode logic". IEEE
Transactions on Electronic Computers 12 (2): 112–129. doi:10.1109/PGEC.1963.263419.
• Zadeh, L.A. (1968). "Fuzzy algorithms". Information and Control 12 (2): 94–102.
doi:10.1016/S0019-9958(68)90211-8. ISSN 0019-9958.
• Zadeh, L.A. (1965). "Fuzzy sets". Information and Control 8 (3): 338–353.
doi:10.1016/S0019-9958(65)90241-X. ISSN 0019-9958.
• Zemankova-Leech, M. (1983). Fuzzy Relational Data Bases. Ph. D. Dissertation. Florida State University.
• Zimmermann, H. (2001). Fuzzy set theory and its applications. Boston: Kluwer Academic Publishers.
ISBN 0-7923-7435-5.
External links
Additional articles
•
•
•
•
•
Formal fuzzy logic (http://en.citizendium.org/wiki/Formal_fuzzy_logic) - article at Citizendium
Fuzzy Logic (http://www.scholarpedia.org/article/Fuzzy_Logic) - article at Scholarpedia
Modeling With Words (http://www.scholarpedia.org/article/Modeling_with_words) - article at Scholarpedia
Fuzzy logic (http://plato.stanford.edu/entries/logic-fuzzy/) - article at Stanford Encyclopedia of Philosophy
Fuzzy Math (http://blog.peltarion.com/2006/10/25/fuzzy-math-part-1-the-theory) - Beginner level
introduction to Fuzzy Logic.
• Fuzzy Logic and the Internet of Things: I-o-T (http://www.i-o-t.org/post/WEB_3)
Links pages
• Web page about FSQL (http://www.lcc.uma.es/~ppgg/FSQL/): References and links about FSQL
Applications
• Research conference paper providing a way to establish a fuzzy workload plan while dealing with project
planning problem under uncertainty (http://www.enim.fr/mosim2010/articles/279.pdf)
• Research article that describes how industrial foresight could be integrated into capital budgeting with intelligent
agents and Fuzzy Logic (http://econpapers.repec.org/paper/amrwpaper/398.htm)
• A doctoral dissertation describing how Fuzzy Logic can be applied in profitability analysis of very large industrial
investments (http://econpapers.repec.org/paper/pramprapa/4328.htm)
• A method for asset valuation that uses fuzzy logic and fuzzy numbers for real option valuation (http://users.abo.
fi/mcollan/fuzzypayoff.html)
222
Expert system
Expert system
In artificial intelligence, an expert system is a computer system that emulates the decision-making ability of a
human expert.[1] Expert systems are designed to solve complex problems by reasoning about knowledge, like an
expert, and not by following the procedure of a developer as is the case in conventional programming.[2] [3] [4] The
first expert systems were created in the 1970s and then proliferated in the 1980s.[5] Expert systems were among the
first truly successful forms of AI software.[6] [7] [8] [9] [10] [11]
An expert system has a unique structure, different from traditional programs. It is divided into two parts, one fixed,
independent of the expert system: the inference engine, and one variable: the knowledge base. To run an expert
system, the engine reasons about the knowledge base like a human.[12] In the 80's a third part appeared: a dialog
interface to communicate with users.[13] This ability to conduct a conversation with users was later called
"conversational".[14] [15]
History
Expert systems were introduced by researchers in the Stanford Heuristic Programming Project, including the "father
of expert systems" Edward Feigenbaum, with the Dendral and Mycin systems. Principal contributors to the
technology were Bruce Buchanan, Edward Shortliffe, Randall Davis, William vanMelle, Carli Scott and others at
Stanford. Expert systems were among the first truly successful forms of AI software.[6] [7] [8] [9] [10] [11]
Research is also very active in France, where researchers focus on the automation of reasoning and logic engines.
The French Prolog computer language, designed in 1972, marks a real advance over expert systems like Dendral or
Mycin: it is a shell,[16] that's to say a software structure ready to receive any expert system and to run it. It integrates
an engine using First-Order logic, with rules and facts. It's a tool for mass production of expert systems and was the
first operational declarative language,[17] later becoming the best selling IA language in the world.[18] However
Prolog is not particularly user friendly and is an order of logic away from human logic.[19] [20] [21]
In the 1980s, expert systems proliferated as they were recognized as a practical tool for solving real-world problems.
Universities offered expert system courses and two thirds of the Fortune 1000 companies applied the technology in
daily business activities.[5] [22] Interest was international with the Fifth Generation Computer Systems project in
Japan and increased research funding in Europe. Growth in the field continued into the 1990s.
The development of expert systems was aided by the development of the symbolic processing languages Lisp and
Prolog. To avoid re-inventing the wheel, expert system shells were created that had more specialized features for
building large expert systems.[23]
In 1981 the first IBM PC was introduced, with MS-DOS operating system. Its low price started to multiply users and
opened a new market for computing and expert systems. In the 80's the image of IA was very good and people
believed it would succeed within a short time[15]. Many companies began to market expert systems shells from
universities, renamed "generators" because they added to the shell a tool for writing rules in plain language and thus,
theoretically, allowed to write expert systems without a programming language nor any other software[16]. The best
known: Guru (USA) inspired by Mycin[17][18], Personal Consultant Plus (USA)[19][20], Nexpert Object
(developed by Neuron Data, company founded in California by three French)[21][22], Genesia (developed by French
public company Electricité de France and marketed by Steria)[23], VP Expert (USA)[24]. But eventually the tools
were only used in research projects. They did not penetrate the business market, showing that AI technology was not
mature.
In 1986, a new expert system generator for PCs appeared on the market, derived from the French academic research:
Intelligence Service,[24] [25] sold by GSI-TECSI software company. This software showed a radical innovation: it
used propositional logic ("Zeroth order logic") to execute expert systems, reasoning on a knowledge base written
with everyday language rules, producing explanations and detecting logic contradictions between the facts. It was the
223
Expert system
first tool showing the AI defined by Edward Feigenbaum in his book about the Japanese Fifth Generation, Artificial
Intelligence and Japan's Computer Challenge to the World [26] (1983): "The machines will have reasoning power:
they will automatically engineer vast amounts of knowledge to serve whatever purpose humans propose, from
medical diagnosis to product design, from management decisions to education", "The reasoning animal has, perhaps
inevitably, fashioned the reasoning machine", "the reasoning power of these machines matches or exceeds the
reasoning power of the humans who instructed them and, in some cases, the reasoning power of any human
performing such tasks". Intelligence Service was in fact "Pandora" (1985),[27] a software developed for their thesis
by two academic students of Jean-Louis Laurière,[28] one of the most famous and prolific French AI researcher.[29]
Unfortunately, as this software was not developed by his own IT developers, GSI-TECSI was unable to make it
evolve. Sales became scarce and marketing stopped after a few years.
Software architecture
The rule base or knowledge base
In expert system technology, the knowledge base is expressed with natural language rules IF ... THEN ... For
examples :
• "IF it is living THEN it is mortal"
• "IF his age = known THEN his year of birth = date of today - his age in years"
• "IF the identity of the germ is not known with certainty AND the germ is gram-positive AND the morphology of
the organism is "rod" AND the germ is aerobic THEN there is a strong probability (0.8) that the germ is of type
enterobacteriacae"[30]
This formulation has the advantage of speaking in everyday language which is very rare in computer science (a
classic program is coded). Rules express the knowledge to be exploited by the expert system. There exists other
formulations of rules, which are not in everyday language, understandable only to computer scientists. Each rule
style is adapted to an engine style. The whole problem of expert systems is to collect this knowledge, usually
unconscious, from the experts. There are methods but almost all are usable only by computer scientists.
The inference engine
The inference engine is a computer program designed to produce a reasoning on rules. In order to produce a
reasoning, it is based on logic. There are several kinds of logic: propositional logic, predicates of order 1 or more,
epistemic logic, modal logic, temporal logic, fuzzy logic, etc. Except propositional logic, all are complex and can
only be understood by mathematicians, logicians or computer scientists. Propositional logic is the basic human logic,
that expressed in the syllogism. The expert system that uses that logic are also called zeroth-order expert system.
With logic, the engine is able to generate new information from the knowledge contained in the rule base and data to
be processed.
The engine has two ways to run: batch or conversational. In batch, expert system has all the necessary data to
process from the beginning. For the user, the program works as a classical program: he provides data and receives
results immediately. Reasoning is invisible. The conversational becomes necessary when the developer knows he
can't ask the user all the necessary data at the start, the problem being too complex. The software must "invent" the
way to solve the problem, request the user missing data, gradually, approaching the goal as quickly as possible. The
result gives the impression of a dialogue led by an expert. To guide a dialogue, the engine may have several levels of
sophistication : "forward chaining", "backward chaining" and "mixed chaining". Forward chaining is the questioning
of an expert who has no idea of the solution and investigates progressively (e.g. fault diagnosis). In backward
chaining, the engine has an idea of the target (e.g. is it okay or not? Or: there is danger but what is the level?). It
starts from the goal in hopes of finding the solution as soon as possible. In mixed chaining the engine has an idea of
the goal but it is not enough: it deduces in forward chaining from previous user responses all that is possible before
224
Expert system
asking the next question. So, quite often, he deduces the answer to the next question before asking it.
A strong interest in using logic is that this kind of software is able to give to user clear explanation of what it is doing
(the "Why?") and what it has deduced (the "How?" ). Better yet, thanks to logic the most sophisticated expert system
are able to detect contradictions[31] into user information or in the knowledge and can explain them clearly, revealing
at the same time the expert knowledge and his way of thinking.
Advantages
Expert systems offer many advantages for users when compared to traditional programs because they operate like a
human brain.
Quick availability and opportunity to program itself
As the rule base is in everyday language (the engine is untouchable), expert system can be written much faster than a
conventional program, by users or experts, bypassing professional developers and avoiding the need to explain the
subject.
Ability to exploit a considerable amount of knowledge
The expert system uses a rule base, unlike conventional programs, which means that the volume of knowledge to
program is not a major concern. Whether the rule base has 10 rules or 10 000, the engine operation is the same.
Reliability
The reliability of an expert system is the same as the reliability of a database, i.e. good, higher than that of a classical
program.
Scalability
Evolving an expert system is to add, modify or delete rules. Since the rules are written in plain language, it is easy to
identify those to be removed or modified.
Pedagogy
The engines that are run by a true logic are able to explain to the user in plain language why they ask a question and
how they arrived at each deduction. In doing so, they show knowledge of the expert contained in the expert system.
So, user can learn this knowledge in its context. Moreover, they can communicate their deductions step by step. So,
the user has information about their problem even before the final answer of the expert system.
Preservation and improvement of knowledge
Valuable knowledge can disappear with the death, resignation or retirement of an expert. Recorded in an expert
system, it becomes eternal. To develop an expert system is to interview an expert and make the system aware of their
knowledge. In doing so, it reflects and enhances it.
New areas neglected by conventional computing
Automating a vast knowledge, the developer may meet a classic problem: "combinatorial explosion" commonly
known as "information overload" that greatly complicates his work and results in a complex and time consuming
program. The reasoning expert system does not encounter that problem since the engine automatically loads
combinatorics between rules. This ability can address areas where combinatorics are enormous: highly interactive or
conversational applications, fault diagnosis, decision support in complex systems, educational software, logic
simulation of machines or systems, constantly changing software.
225
Expert system
Disadvantages
The expert system has a major flaw which explains its low success although the principle has existed for 70 years:
knowledge collection and interpretation into rules, the knowledge engineering. Most developers have no method to
perform this task. They work "manually" what opens to many possibilities for errors. Expert knowledge is not well
understood, there is a lack of rules, rules are contradictory, some are poorly written and unusable. Worse, they most
often use an engine unable to reasoning. Result: the expert system works badly and the project is abandoned.[32]
Correct development methodology can mitigate these problems. There exists software to interview the expert step by
step which automatically write the rules and simultaneously run the expert system before his eyes, performing a
consistency of the rules control.[33] [34] [35] So expert and users can check the quality of the software before it is
finished.
Many expert systems are also penalized by the logic used. Most of the logics operate on "variables" facts, i.e. whose
value changes several times during one reasoning, considered as a property belonging to a more powerful logic.
Exactly like in classical computing, a way of programming where developers are in fact comfortable. This is the case
of Mycin, Dendral, fuzzy logic, predicate logic (Prolog), symbolic logic, mathematical logic, etc.. Propositional logic
uses only not variable facts.[36] It turns out that in human mind, the facts used must remain invariable as long as the
brain reasons on them. This makes possible detection of contradictions and production of explanations, two ways of
controlling consistency of the knowledge.[37] [38] That is why expert systems using variable facts, more
understandable for IT developers so the most numerous, are less easy to develop, less clear to users, less reliable and
why they don't produce explanation or contradiction detection.
Application field
Expert systems address areas where combinatorics is enormous:
•
•
•
•
•
•
•
highly interactive or conversational applications, IVR, voice server, chatterbot
fault diagnosis, medical diagnosis
decision support in complex systems, process control, interactive user guide
educational and tutorial software
logic simulation of machines or systems
knowledge management
constantly changing software.
They can also be used in software engineering for rapid prototyping applications (RAD). Indeed, the expert system
quickly developed in front of the expert shows him if the future application should be programmed.
Indeed, any program contains expert knowledge and classic programming always begins with an expert interview. A
program written in the form of expert system receives all the specific benefits of expert system, among others things
it can be developed by anyone without computer training and without programming languages. But this solution has
a defect: expert system runs slower than a traditional program because he consistently "thinks" when in fact a classic
software just follows paths traced by the programmer.
Examples of applications
Expert systems are designed to facilitate tasks in the fields of accounting, medicine, process control, financial
service, production, human resources, among others. Typically, the problem area is complex enough that a more
simple traditional algorithm cannot provide a proper solution. The foundation of a successful expert system depends
on a series of technical procedures and development that may be designed by technicians and related experts. As
such, expert systems do not typically provide a definitive answer, but provide probabilistic recommendations.
An example of the application of expert systems in the financial field is expert systems for mortgages. Loan
departments are interested in expert systems for mortgages because of the growing cost of labour, which makes the
226
Expert system
handling and acceptance of relatively small loans less profitable. They also see a possibility for standardized,
efficient handling of mortgage loan by applying expert systems, appreciating that for the acceptance of mortgages
there are hard and fast rules which do not always exist with other types of loans. Another common application in the
financial area for expert systems are in trading recommendations in various marketplaces. These markets involve
numerous variables and human emotions which may be impossible to deterministically characterize, thus expert
systems based on the rules of thumb from experts and simulation data are used. Expert system of this type can range
from ones providing regional retail recommendations, like Wishabi, to ones used to assist monetary decisions by
financial institutions and governments.
Another 1970s and 1980s application of expert systems, which we today would simply call AI, was in computer
games. For example, the computer baseball games Earl Weaver Baseball and Tony La Russa Baseball each had
highly detailed simulations of the game strategies of those two baseball managers. When a human played the game
against the computer, the computer queried the Earl Weaver or Tony La Russa Expert System for a decision on what
strategy to follow. Even those choices where some randomness was part of the natural system (such as when to
throw a surprise pitch-out to try to trick a runner trying to steal a base) were decided based on probabilities supplied
by Weaver or La Russa. Today we would simply say that "the game's AI provided the opposing manager's strategy".
Knowledge engineering
The building, maintaining and development of expert systems is known as knowledge engineering.[39] Knowledge
engineering is a "discipline that involves integrating knowledge into computer systems in order to solve complex
problems normally requiring a high level of human expertise".[40]
There are generally three individuals having an interaction in an expert system. Primary among these is the end-user,
the individual who uses the system for its problem solving assistance. In the construction and maintenance of the
system there are two other roles: the problem domain expert who builds the system and supplies the knowledge base,
and a knowledge engineer who assists the experts in determining the representation of their knowledge, enters this
knowledge into an explanation module and who defines the inference technique required to solve the problem.
Usually the knowledge engineer will represent the problem solving activity in the form of rules. When these rules are
created from domain expertise, the knowledge base stores the rules of the expert system.
References
[1] Jackson, Peter (1998), Introduction To Expert Systems (3 ed.), Addison Wesley, p. 2, ISBN 978-0-201-87686-4
[2] Nwigbo Stella and Agbo Okechuku Chuks (http:/ / www. hrmars. com/ admin/ pics/ 261. pdf), School of Science Education, Expert system: a
catalyst in educational development in Nigeria: "The ability of this system to explain the reasoning process through back-traces (...) provides
an additional feature that conventional programming does not handle"
[3] Regina Barzilay, Daryl McCullough, Owen Rambow, Jonathan DeCristofaro, Tanya Korelsky, Benoit Lavoie (http:/ / www. cogentex. com/
papers/ explanation-iwnlg98. pdf): "A new approach to expert system explanations"
[4] Conventional programming (http:/ / www. pcmag. com/ encyclopedia_term/ 0,2542,t=conventional+ programming& i=40325,00. asp)
[5] Cornelius T. Leondes (2002). Expert systems: the technology of knowledge management and decision making for the 21st century. pp. 1–22.
ISBN 978-0-12-443880-4.
[6] ACM 1998, I.2.1
[7] Russell & Norvig 2003, pp. 22−24
[8] Luger & Stubblefield 2004, pp. 227–331
[9] Nilsson 1998, chpt. 17.4
[10] McCorduck 2004, pp. 327–335, 434–435
[11] Crevier 1993, pp. 145–62, 197−203
[12] Nwigbo Stella and Agbo Okechuku Chuks (http:/ / www. hrmars. com/ admin/ pics/ 261. pdf), School of Science Education, Expert system:
a catalyst in educational development in Nigeria: "Knowledge-based systems collect the small fragments of human know-how into a
knowledge-base which is used to reason through a problem, using the knowledge that is appropriated"
[13] Koch, C. G.; Isle, B. A.; Butler, A. W. (1988). "Intelligent user interface for expert systems applied to power plant maintenance and
troubleshooting". IEEE Transactions on Energy Conversion 3: 71. doi:10.1109/60.4202.
227
Expert system
[14] McTear, M. F. (2002). "Spoken dialogue technology: Enabling the conversational user interface". ACM Computing Surveys 34: 90.
doi:10.1145/505282.505285.
[15] Lowgren, J. (1992). "The Ignatius environment: Supporting the design and development of expert-system user interfaces". IEEE Expert 7
(4): 49–57. doi:10.1109/64.153464.
[16] George F. Luger and William A. Stubblefield, Benjamin/Cummings Publishers, Rule Based Expert System Shell: example of code using the
Prolog rule based expert system shell
[17] A. MICHIELS (http:/ / promethee. philo. ulg. ac. be/ engdep1/ download/ prolog/ htm_docs/ prolog. htm), Université de Liège, Belgique:
"PROLOG, the first declarative language
[18] Carnegie Mellon University's AI Web Site (http:/ / www. prenhall. com/ divisions/ bp/ app/ turban/ dss/ html/ chap16. html): "Prolog was
the most popular AI language in Japan and probably in Europe"
[19] Ivana Berković, Biljana Radulović and Petar Hotomski (http:/ / www. proceedings2007. imcsit. org/ pliks/ 33. pdf), University of Novi Sad,
2007, Extensions of Deductive Concept in Logic Programing and Some Applications: "the defects of PROLOG-system: the expansion
concerning Horn clauses, escaping negation treatment as definite failure"
[20] "Software developed in Prolog has been criticized for having a high performance penalty compared to conventional programming
languages"
[21] Dr. Nikolai Bezroukov (http:/ / www. softpanorama. org/ Lang/ prolog. shtml), Softpanorama: "I think that most people exposed to Prolog
remember strongly the initial disappointment. Language was/is so hyped but all you can see initially are pretty trivial examples that are solved
by complex, obscure notation that lacks real expressive power: some of simple examples can be expressed no less concisely is many other
languages"
[22] Durkin, J. Expert Systems: Catalog of Applications. Intelligent Computer Systems, Inc., Akron, OH, 1993.
[23] page 21. Giarratano & Riley, 3rd ed.
[24] Flamant B. and Girard G (http:/ / cat. inist. fr/ ?aModele=afficheN& cpsidt=7001328)., GSI-TECSI, Intelligence Service: build your own
expert system : "Intelligence Service is a development environment for expert systems that requires no experience of classic programming that
offers to everyone the opportunity to develop its own expert system"
[25] Bertrand Savatier (http:/ / www. tree-logic. com/ Articles/ 01, seul IS fait de l'IA, par un universitaire. jpg), Le Monde Informatique,
November 23, 1987: "Expert systems accessible to all"
[26] http:/ / www. atarimagazines. com/ creative/ v10n8/ 103_The_fifth_generation_Jap. php
[27] Jean-Philippe de Lespinay (http:/ / www. tree-logic. com/ scienceetvie. htm), Science et Vie, "From total zero to Zero Plus [logic]", May
1991
[28] Death of Jean-Louis Laurière (http:/ / www. lip6. fr/ actualite/ information-fiche. php?RECORD_KEY(informations)=id&
id(informations)=18)
[29] Journey " In honor of Jean-Louis Laurière (http:/ / www. lip6. fr/ Laboratoire/ 2006-03-22/ 2006-03-22-Affiche. pdf)", Université Pierre et
Marie Curie in Paris (March 22, 2006)
[30] Mycin rule
[31] Nabil Arman (http:/ / www. ccis2k. org/ iajit/ PDF/ vol. 4,no. 1/ 9-Nabil. pdf), Polytechnic University of Palestine, January 2007, Fault
Detection in Dynamic Rule Bases Using Spanning Trees and Disjoin Sets: ""
[32] Kenneth Laudon, Jane Laudon, Eric Fimbel, "Management Information Systems: Managing the Digital Firm", Business & Economics, 2010
edition, chapter 11-3.5: The implementation of a large number of expert systems requires the deployment of considerable development efforts,
lengthy and expensive. Hiring and training a larger number of experts may be less expensive than building an expert system .(...) Some expert
systems, particularly the largest, are so complex that over years, the costs of curative and adaptive maintenance become as high as the cost of
development.
[33] Systèmes Experts, April 15, 1990, Miao, authentic expert system generator of fault diagnosis: "MIAO can explain, again in [plain]
language, all of his logical approach: why he is asking such a question and how it came to such a conclusion. And that because he is
constantly reasoning and not because an IT developer programmed in advance all the possible explanations."
[34] Olivier Rafal (http:/ / www. tree-logic. com/ Articles/ LMI 2001 cf Maieutique. jpg), Le Monde Informatique, Programming for all (T.Rex
generator): "This software allows to develop a conversational application (...) leading to a self-learning" (i.e. thanks to the automatic
explanations)
[35] French Technology Survey (http:/ / www. tree-logic. com/ Articles/ Maïeutica par FTS (91). jpg), MAIEUTICA, An Expert System
Generator which writes its own rules, July 1991: "checking the coherence of the knowledge", "it can detect contradictions", "it react
appropriately to changes of minds"
[36] RGU: School of Computing (http:/ / www. comp. rgu. ac. uk/ docs/ info/ index. php), More Complex Inference: "propositional logic, where
variables are not allowed".
[37] Ong K. and Lee R.M, Texas University-Austin, A logic model for maintaining consistency of bureaucratic policies, 1993: "Inconsistencies
can be detected if any of the integrity constraints is proven false, and an explanation can be provided based on the proof tree. A more general
inference mechanism is presented based on the theory of abduction for checking potential inconsistency of policies"
[38] Carl G. Hempel and Paul Oppenheim (http:/ / people. cohums. ohio-state. edu/ tennant9/ hempel_oppenheim_PS1948. pdf), Philosophy of
Science, Studies in the Logic of Explanation, 1948: "The sentences constituting the explanans must be true"
[39] Kendal, S.L.; Creen, M. (2007), An introduction to knowledge engineering, London: Springer, ISBN 978-1-84628-475-5, OCLC 70987401
228
Expert system
[40] Feigenbaum, Edward A.; McCorduck, Pamela (1983), The fifth generation (1st ed.), Reading, MA: Addison-Wesley,
ISBN 978-0-201-11519-2, OCLC 9324691
Bibliography
Textbooks
• Darlington, Keith (2000). The Essence of Expert Systems. Pearson Education. ISBN 978-0-13-022774-4.
• Ignizio, James (1991). Introduction to Expert Systems. McGraw-Hill Companies. ISBN 978-0-07-909785-9.
• Giarratano, Joseph C. and Riley, Gary; Gary Riley (2005). Expert Systems, Principles and Programming. Course
Technology Ptr. ISBN 978-0-534-38447-0.
• Jackson, Peter (1998). Introduction to Expert Systems. Addison Wesley. ISBN 978-0-201-87686-4.
• Walker, Adrian et al. (1990). Knowledge Systems and Prolog. Addison-Wesley. ISBN 978-0-201-52424-6.
• Naylor, Chris. (1983). Build your own Expert System. Sigma Technical Press. ISBN 978-0-905-10441-6.
History of AI
• Crevier, Daniel (1993), AI: The Tumultuous Search for Artificial Intelligence, New York, NY: BasicBooks, ISBN
0-465-02997-3
• McCorduck, Pamela (2004), Machines Who Think (http://www.pamelamc.com/html/machines_who_think.
html) (2nd ed.), Natick, MA: A. K. Peters, Ltd., ISBN 1-56881-205-1
• Luger, George; William A. Stubblefield (2004). Artificial Intelligence: Structures and Strategies for Complex
Problem Solving (http://www.cs.unm.edu/~luger/ai-final/tocfull.html) (5th ed.). The Benjamin/Cummings
Publishing Company, Inc.. ISBN 978-0-8053-4780-7.
• Nilsson, Nils (1998-04-01). Artificial Intelligence: A New Synthesis. Morgan Kaufmann Publishers.
ISBN 978-1-55860-467-4.
• Russell, Stuart J.; Norvig, Peter (2003), Artificial Intelligence: A Modern Approach (http://aima.cs.berkeley.
edu/) (2nd ed.), Upper Saddle River, New Jersey: Prentice Hall, ISBN 0-13-790395-2
• Winston, Patrick Henry (1984-04). Artificial Intelligence. Addison-Wesley. ISBN 978-0-201-08259-3.
Other
• "ACM Computing Classification System: Artificial intelligence" (http://www.acm.org/class/1998/I.2.html).
ACM. 1998. Retrieved 2007-08-30.
• Jean-Philippe de Lespinay, Admiroutes, December 2008: Reasoning AI (http://www.admiroutes.asso.fr/
larevue/2008/93/lespinay.htm)
• Automates Intelligents, 2009: Conversational and Call centers (http://www.automatesintelligents.com/
interviews/2009/lespinay.html)
• US patent 4763277 (http://worldwide.espacenet.com/textdoc?DB=EPODOC&IDX=US4763277), Ashford,
Thomas J. et al., "Method for obtaining information in an expert system", published 1988-08-09, issued
1988-08-09
229
Expert system
External links
• Artificial Intelligence (http://www.dmoz.org/Computers/Artificial_Intelligence//) at the Open Directory
Project
• Expert System tutorial on Code Project (http://www.codeproject.com/KB/recipes/ArtificialAdvice-1.aspx)
230
231
The Root-Locus Design Method
Root locus
Root locus analysis is a graphical method for examining how the roots of a system change with variation of a certain
system parameter, commonly the gain of a feedback system. This is a technique used in the field of control systems
developed by Walter R. Evans.
Uses
In addition to determining the stability of the system, the root locus can be used to design for the damping ratio and
natural frequency of a feedback system. Lines of constant damping ratio can be drawn radially from the origin and
lines of constant natural frequency can be drawn as arcs whose center points coincide with the origin. By selecting a
point along the root locus that coincides with a desired damping ratio and natural frequency a gain, K, can be
calculated and implemented in the controller. More elaborate techniques of controller design using the root locus are
available in most control textbooks: for instance, lag, lead, PI, PD and PID controllers can be designed
approximately with this technique.
The definition of the damping ratio and natural frequency presumes that the overall feedback system is well
approximated by a second order system, that is, the system has a dominant pair of poles. This often doesn't happen
and so it's good practice to simulate the final design to check if the project goals are satisfied.
Example
Suppose there is a plant (process) with
a transfer function expression P(s), and
a forward controller with both an
adjustable gain K and output
expression C(s) as shown in the block
diagram below.
RL = root locus
Root locus
232
A unity feedback loop is constructed to complete this feedback system. For this system, the overall transfer function
is given by
Thus the closed-loop poles (roots of the characteristic equation) of the transfer function are the solutions to the
equation 1+ KC(s)P(s) = 0. The principal feature of this equation is that roots may be found wherever KCP = -1. The
variability of K, the gain for the controller, removes amplitude from the equation, meaning the complex valued
evaluation of the polynomial in s C(s)P(s) needs to have net phase of 180 deg, wherever there is a closed loop pole.
The geometrical construction adds angle contributions from the vectors extending from each of the poles of KC to a
prospective closed loop root (pole) and subtracts the angle contributions from similar vectors extending from the
zeros, requiring the sum be 180. The vector formulation arises from the fact that each polynomial term in the
factored CP,(s-a) for example, represents the vector from a which is one of the roots, to s which is the prospective
closed loop pole we are seeking. Thus the entire polynomial is the product of these terms, and according to vector
mathematics the angles add (or subtract, for terms in the denominator) and lengths multiply (or divide). So to test a
point for inclusion on the root locus, all you do is add the angles to all the open loop poles and zeros. Indeed a form
of protractor, the "spirule" was once used to draw exact root loci.
From the function T(s), we can also see that the zeros of the open loop system (CP) are also the zeros of the closed
loop system. It is important to note that the root locus only gives the location of closed loop poles as the gain K is
varied, given the open loop transfer function. The zeros of a system can not be moved.
Using a few basic rules, the root locus method can plot the overall shape of the path (locus) traversed by the roots as
the value of K varies. The plot of the root locus then gives an idea of the stability and dynamics of this feedback
system for different values of k.[1] [2]
Sketching root locus
• Mark open-loop poles and zeros
• Mark real axis portion to the left of an odd number of poles and zeros
• Find asymptotes
Let P be the number of poles and Z be the number of zeros:
number of asymptotes
The asymptotes intersect the real axis at
where
and depart at angle
is the sum of all the locations of the poles, and
given by:
is the sum of all the locations of the explicit zeros.
Root locus
• Phase condition on test point to find angle of deparature
• Compute breakaway/break-in points
The breakaway points are located at the roots of the following equation:
Once you solve for z, the real roots give you the breakaway/reentry points. Complex roots correspond to a lack of
breakaway/reentry.
The break-away (break-in) points are obtained by solving a polynomial equation
z-plane versus s-plane
The root locus can also be computed in the z-plane, the discrete counterpart of the s-plane. An equation (z = esT)
maps continuous s-plane poles (not zeros) into the z-domain, where T is the sampling period. The stable, left half
s-plane maps into the interior of the unit circle of the z-plane, with the s-plane origin equating to |z| = 1
(because e0 = 1). A diagonal line of constant damping in the s-plane maps around a spiral from (1,0) in the z plane as
it curves in toward the origin. Note also that the Nyquist aliasing criteria is expressed graphically in the z-plane by
the x-axis, where (wnT = π). The line of constant damping just described spirals in indefinitely but in sampled data
systems, frequency content is aliased down to lower frequencies by integral multiples of the Nyquist frequency. That
is, the sampled response appears as a lower frequency and better damped as well since the root in the z-plane maps
equally well to the first loop of a different, better damped spiral curve of constant damping. Many other interesting
and relevant mapping properties can be described, not least that z-plane controllers, having the property that they
may be directly implemented from the z-plane transfer function (zero/pole ratio of polynomialls), can be imagined
graphically on a z-plane plot of the open loop transfer function, and immediately analyzed utilizing root locus.
Since root locus is a graphical angle technique, root locus rules work the same in the z and s planes.
The idea of a root locus can be applied to many systems where a single parameter K is varied. For example, it is
useful to sweep any system parameter for which the exact value is uncertain, in order to determine its behavior.
References
[1] Evans, W. R. (January 1948), "Graphical Analysis of Control Systems", Trans. AIEE 67 (1): 547–551, doi:10.1109/T-AIEE.1948.5059708,
ISSN 0096-3860
[2] Evans, W. R. (January 1950), "Control Systems Synthesis by Root Locus Method", Trans. AIEE 69 (1): 66–69,
doi:10.1109/T-AIEE.1950.5060121, ISSN 0096-3860
• Ash, R. H.; Ash, G. H. (October 1968), "Numerical Computation of Root Loci Using the Newton-Raphson
Technique", IEEE Trans. Automatic Control 13 (5), doi:10.1109/TAC.1968.1098980
• Williamson, S. E. (May 1968), "Design Data to assist the Plotting of Root Loci (Part I)", Control Magazine 12
(119): 404–407
• Williamson, S. E. (June 1968), "Design Data to assist the Plotting of Root Loci (Part II)", Control Magazine 12
(120): 556–559
• Williamson, S. E. (July 1968), "Design Data to assist the Plotting of Root Loci (Part III)", Control Magazine 12
(121): 645–647
• Williamson, S. E. (May 15, 1969), "Computer Program to Obtain the Time Response of Sampled Data Systems",
IEE Electronics Letters 5 (10): 209–210, doi:10.1049/el:19690159
• Williamson, S. E. (July 1969), "Accurate Root Locus Plotting Including the Effects of Pure Time Delay" (http://
ieeexplore.ieee.org/Xplore/login.jsp?url=http://ieeexplore.ieee.org/iel5/5247218/5248781/05249527.
pdf?tp=&arnumber=5249527&punumber=5247218&authDecision=-203), Proc. IEE 116 (7): 1269–1271,
doi:10.1049/piee.1969.0235
233
Root locus
External links
• Wikibooks: Control Systems/Root Locus (http://en.wikibooks.org/wiki/Control_Systems/Root_Locus)
• Carnegie Mellon / University of Michigan Tutorial (http://www.engin.umich.edu/group/ctm/rlocus/rlocus.
html)
• Excellent examples. Start with example 5 and proceed backwards through 4 to 1. Also visit the main page (http://
www.swarthmore.edu/NatSci/echeeve1/Ref/LPSA/Root_Locus/RLocusExamples.html#ex5)
• The root-locus method: Drawing by hand techniques (http://www.atp.ruhr-uni-bochum.de/rt1/syscontrol/
node46.html)
• "RootLocs": A free multi-featured root-locus plotter for Mac and Windows platforms (http://www.coppice.
myzen.co.uk)
• "Root Locus": A free root-locus plotter/analyzer for Windows (http://web.archive.org/web/20091027092528/
http://geocities.com/aseldawy/root_locus.html)
• Root Locus at ControlTheoryPro.com (http://wikis.controltheorypro.com/index.php?title=Root_Locus)
• Root Locus Analysis of Control Systems (http://www.roymech.co.uk/Related/Control/root_locus.html)
• Wechsler, E. R. (January-March 1983), Root Locus Algorithms for Programmable Pocket Calculators (http://
ipnpr.jpl.nasa.gov/progress_report/42-73/73F.PDF), NASA, pp. 60–64, TDA Progress Report 42-73
Compensation (engineering)
In engineering, compensation is planning for side effects or other unintended issues in a design. The design of an
invention can itself also be to compensate for some other existing issue or exception.
One example is in a voltage-controlled crystal oscillator (VCXO), which is normally affected not only by voltage,
but to a lesser extent by temperature. A temperature-compensated version (a TCVCXO) is designed so that heat
buildup within the enclosure of a transmitter or other such device will not alter the piezoelectric effect, thereby
causing frequency drift.
Another example is motion compensation on digital cameras and video cameras, which keep a picture steady and not
blurry.
Other examples in electrical engineering include:
• A constant voltage device compensates for low or high voltage in an electrical circuit, keeping its output the same
within a given range of input.
• Error correction compensates for data corruption.
• Gray coding compensates for errors on rotary encoders and linear encoders.
• Debouncing compensates for jitter in an electrical switch (see Contact Bounce section in Switch article).
• A resistor or inductor compensates for negative resistance in gas-discharge lighting.
• Frequency compensation is used in feedback control systems to avert oscillations.
There are also examples in civil engineering:
• Expansion joints in sidewalks, buildings, and bridges compensate for expansion and contraction.
• Various devices between a structure and its Foundation compensate for earthquake movements, either actively or
passively.
234
Compensation (engineering)
Railways
• On railways, steep gradients on sharp curves have an equivalent gradient slightly steeper than the stated value.
This is because of extra friction of the wheel on the rails, and because wagons stretch out on the chord rather than
on the arc, and may thus cause trains to stall. To compensate, the gradient of ruling grades is slightly reduced
from say 1 in 50 to 1 in 53.
Frequency compensation
In electrical engineering, frequency compensation is a technique used in amplifiers, and especially in amplifiers
employing negative feedback. It usually has two primary goals: To avoid the unintentional creation of positive
feedback, which will cause the amplifier to oscillate, and to control overshoot and ringing in the amplifier's step
response.
Explanation
Most amplifiers use negative feedback
to trade gain for other desirable
properties, such as decreased distortion
or improved noise reduction. Ideally,
the phase characteristic of an
amplifier's frequency response would
be linear; however, device limitations
make this goal physically unattainable.
More particularly, capacitances within
the amplifier's gain stages cause the
Figure 1: Step response of a two-pole amplifier for various degrees of compensation.
output signal to lag behind the input
Parameter ζ is set by a compensation capacitor: smaller ζ results in faster response, but
more ringing and overshoot.
signal by 90° for each pole they
[1]
create. If the sum of these phase lags
reaches 360°, the output signal will be in phase with the input signal. Feeding back any portion of this output signal
to the input when the gain of the amplifier is sufficient will cause the amplifier to oscillate. This is because the
feedback signal will reinforce the input signal. That is, the feedback is then positive rather than negative.
Frequency compensation is implemented to avoid this result.
Another goal of frequency compensation is to control the step response of an amplifier circuit as shown in Figure 1.
For example, if a step in voltage is input to a voltage amplifier, ideally a step in output voltage would occur.
However, the output is not ideal because of the frequency response of the amplifier, and ringing occurs. Several
figures of merit to describe the adequacy of step response are in common use. One is the rise time of the output,
which ideally would be short. A second is the time for the output to lock into its final value, which again should be
short. The success in reaching this lock-in at final value is described by overshoot (how far the response exceeds
final value) and settling time (how long the output swings back and forth about its final value). These various
measures of the step response usually conflict with one another, requiring optimization methods.
Frequency compensation is implemented to optimize step response, one method being pole splitting.
235
Frequency compensation
Use in operational amplifiers
Because operational amplifiers are so ubiquitous and are designed to be used with feedback, the following discussion
will be limited to frequency compensation of these devices.
It should be expected that the outputs of even the simplest operational amplifiers will have at least two poles. An
unfortunate consequence of this is that at some critical frequency, the phase of the amplifier's output = -180°
compared to the phase of its input signal. The amplifier will oscillate if it has a gain of one or more at this critical
frequency. This is because (a) the feedback is implemented through the use of an inverting input that adds an
additional -180° to the output phase making the total phase shift -360° and (b) the gain is sufficient to induce
oscillation.
A more precise statement of this is the following: An operational amplifier will oscillate at the frequency at which its
open loop gain equals its closed loop gain if, at that frequency,
1. The open loop gain of the amplifier is ≥ 1 and
2. The difference between the phase of the open loop signal and phase response of the network creating the
closed loop output = -180°. Mathematically,
ΦOL – ΦCLnet = -180°
Practice
Frequency compensation is implemented by modifying the gain and phase characteristics of the amplifier's open loop
output or of its feedback network, or both, in such a way as to avoid the conditions leading to oscillation. This is
usually done by the internal or external use of resistance-capacitance networks.
Dominant-pole compensation
The method most commonly used is called dominant-pole compensation, which is a form of lag compensation. A
pole placed at an appropriate low frequency in the open-loop response reduces the gain of the amplifier to one (0 dB)
for a frequency at or just below the location of the next highest frequency pole. The lowest frequency pole is called
the dominant pole because it dominates the effect of all of the higher frequency poles. The result is that the
difference between the open loop output phase and the phase response of a feedback network having no reactive
elements never falls below −180° while the amplifier has a gain of one or more, ensuring stability.
Dominant-pole compensation can be implemented for general purpose operational amplifiers by adding an
integrating capacitance to the stage that provides the bulk of the amplifier's gain. This capacitor creates a pole that is
set at a frequency low enough to reduce the gain to one (0 dB) at or just below the frequency where the pole next
highest in frequency is located. The result is a phase margin of ≈ 45°, depending on the proximity of still higher
poles.[2] This margin is sufficient to prevent oscillation in the most commonly used feedback configurations. In
addition, dominant-pole compensation allows control of overshoot and ringing in the amplifier step response, which
can be a more demanding requirement than the simple need for stability.
Though simple and effective, this kind of conservative dominant pole compensation has two drawbacks:
1. It reduces the bandwidth of the amplifier, thereby reducing available open loop gain at higher frequencies.
This, in turn, reduces the amount of feedback available for distortion correction, etc. at higher frequencies.
2. It reduces the amplifier's slew rate. This reduction results from the time it takes the finite current driving the
compensated stage to charge the compensating capacitor. The result is the inability of the amplifier to
reproduce high amplitude, rapidly changing signals accurately.
A strong example of this can be found at http://www.ece.utah.edu/~harrison/ece3110/Compensation.pdf
Often, the implementation of dominant-pole compensation results in the phenomenon of Pole splitting. This results
in the lowest frequency pole of the uncompensated amplifier "moving" to an even lower frequency to become the
236
Frequency compensation
dominant pole, and the higher-frequency pole of the uncompensated amplifier "moving" to a higher frequency.
Other methods
Some other compensation methods are: lead compensation, lead–lag compensation and feed-forward compensation.
Lead compensation. Whereas dominant pole compensation places or moves poles in the open loop response,
lead compensation places a zero[3] in the open loop response to cancel one of the existing poles.
Lead–lag compensation places both a zero and a pole in the open loop response, with the pole usually being at
an open loop gain of less than one.
Feed-forward compensation uses a capacitor to bypass a stage in the amplifier at high frequencies, thereby
eliminating the pole that stage creates.
The purpose of these three methods is to allow greater open loop bandwidth while still maintaining amplifier closed
loop stability. They are often used to compensate high gain, wide bandwidth amplifiers.
Footnotes
[1] In this context, a pole is the point in a frequency response curve where the amplitude decreases by 3db due to an integrating resistance and
capacitive reactance. Ultimately, each pole will result in a phase lag of 90°, i.e., the output signal will lag behind the input signal by 90° at this
point. For the mathematical concept of a pole, see, Pole (complex analysis).
[2] The dominant pole produces a phase shift approximating -90° from approx. 10 times the pole frequency to a frequency a factor of ten below
the next higher pole position. The next higher pole, in turn, adds another -45° for a frequency at its location for a total of -135° (neglecting the
still higher poles).
[3] In this context, a zero is the point in a frequency response curve where the amplitude increases by 3db due to a differentiating resistance and
capacitive reactance. Ultimately, each zero will result in a phase lead of 90°, i.e., the phase of the output signal will be 90° ahead of the phase
of the input signal at this point. For the mathematical concept of a zero, see, Zero (complex analysis).
Error detection and correction
In information theory and coding theory with applications in computer science and telecommunication, error
detection and correction or error control are techniques that enable reliable delivery of digital data over unreliable
communication channels. Many communication channels are subject to channel noise, and thus errors may be
introduced during transmission from the source to a receiver. Error detection techniques allow detecting such errors,
while error correction enables reconstruction of the original data.
Definitions
The general definitions of the terms are as follows:
• Error detection is the detection of errors caused by noise or other impairments during transmission from the
transmitter to the receiver.[1]
• Error correction is the detection of errors and reconstruction of the original, error-free data.
Introduction
The general idea for achieving error detection and correction is to add some redundancy (i.e., some extra data) to a
message, which receivers can use to check consistency of the delivered message, and to recover data determined to
be erroneous. Error-detection and correction schemes can be either systematic or non-systematic: In a systematic
scheme, the transmitter sends the original data, and attaches a fixed number of check bits (or parity data), which are
derived from the data bits by some deterministic algorithm. If only error detection is required, a receiver can simply
apply the same algorithm to the received data bits and compare its output with the received check bits; if the values
237
Error detection and correction
do not match, an error has occurred at some point during the transmission. In a system that uses a non-systematic
code, the original message is transformed into an encoded message that has at least as many bits as the original
message.
Good error control performance requires the scheme to be selected based on the characteristics of the communication
channel. Common channel models include memory-less models where errors occur randomly and with a certain
probability, and dynamic models where errors occur primarily in bursts. Consequently, error-detecting and
correcting codes can be generally distinguished between random-error-detecting/correcting and
burst-error-detecting/correcting. Some codes can also be suitable for a mixture of random errors and burst errors.
If the channel capacity cannot be determined, or is highly varying, an error-detection scheme may be combined with
a system for retransmissions of erroneous data. This is known as automatic repeat request (ARQ), and is most
notably used in the Internet. An alternate approach for error control is hybrid automatic repeat request (HARQ),
which is a combination of ARQ and error-correction coding.
Implementation
Error correction may generally be realized in two different ways:
• Automatic repeat request (ARQ) (sometimes also referred to as backward error correction): This is an error
control technique whereby an error detection scheme is combined with requests for retransmission of erroneous
data. Every block of data received is checked using the error detection code used, and if the check fails,
retransmission of the data is requested – this may be done repeatedly, until the data can be verified.
• Forward error correction (FEC): The sender encodes the data using an error-correcting code (ECC) prior to
transmission. The additional information (redundancy) added by the code is used by the receiver to recover the
original data. In general, the reconstructed data is what is deemed the "most likely" original data.
ARQ and FEC may be combined, such that minor errors are corrected without retransmission, and major errors are
corrected via a request for retransmission: this is called hybrid automatic repeat-request (HARQ).
Error detection schemes
Error detection is most commonly realized using a suitable hash function (or checksum algorithm). A hash function
adds a fixed-length tag to a message, which enables receivers to verify the delivered message by recomputing the tag
and comparing it with the one provided.
There exists a vast variety of different hash function designs. However, some are of particularly widespread use
because of either their simplicity or their suitability for detecting certain kinds of errors (e.g., the cyclic redundancy
check's performance in detecting burst errors).
Random-error-correcting codes based on minimum distance coding can provide a suitable alternative to hash
functions when a strict guarantee on the minimum number of errors to be detected is desired. Repetition codes,
described below, are special cases of error-correcting codes: although rather inefficient, they find applications for
both error correction and detection due to their simplicity.
Repetition codes
A repetition code is a coding scheme that repeats the bits across a channel to achieve error-free communication.
Given a stream of data to be transmitted, the data is divided into blocks of bits. Each block is transmitted some
predetermined number of times. For example, to send the bit pattern "1011", the four-bit block can be repeated three
times, thus producing "1011 1011 1011". However, if this twelve-bit pattern was received as "1010 1011 1011" –
where the first block is unlike the other two – it can be determined that an error has occurred.
Repetition codes are very inefficient, and can be susceptible to problems if the error occurs in exactly the same place
for each group (e.g., "1010 1010 1010" in the previous example would be detected as correct). The advantage of
238
Error detection and correction
repetition codes is that they are extremely simple, and are in fact used in some transmissions of numbers stations.
Parity bits
A parity bit is a bit that is added to a group of source bits to ensure that the number of set bits (i.e., bits with value
1) in the outcome is even or odd. It is a very simple scheme that can be used to detect single or any other odd number
(i.e., three, five, etc.) of errors in the output. An even number of flipped bits will make the parity bit appear correct
even though the data is erroneous.
Extensions and variations on the parity bit mechanism are horizontal redundancy checks, vertical redundancy
checks, and "double," "dual," or "diagonal" parity (used in RAID-DP).
Checksums
A checksum of a message is a modular arithmetic sum of message code words of a fixed word length (e.g., byte
values). The sum may be negated by means of a ones'-complement operation prior to transmission to detect errors
resulting in all-zero messages.
Checksum schemes include parity bits, check digits, and longitudinal redundancy checks. Some checksum schemes,
such as the Luhn algorithm and the Verhoeff algorithm, are specifically designed to detect errors commonly
introduced by humans in writing down or remembering identification numbers.
Cyclic redundancy checks (CRCs)
A cyclic redundancy check (CRC) is a single-burst-error-detecting cyclic code and non-secure hash function
designed to detect accidental changes to digital data in computer networks. It is characterized by specification of a
so-called generator polynomial, which is used as the divisor in a polynomial long division over a finite field, taking
the input data as the dividend, and where the remainder becomes the result.
Cyclic codes have favorable properties in that they are well suited for detecting burst errors. CRCs are particularly
easy to implement in hardware, and are therefore commonly used in digital networks and storage devices such as
hard disk drives.
Even parity is a special case of a cyclic redundancy check, where the single-bit CRC is generated by the divisor x +
1.
Cryptographic hash functions
A cryptographic hash function can provide strong assurances about data integrity, provided that changes of the
data are only accidental (i.e., due to transmission errors). Any modification to the data will likely be detected through
a mismatching hash value. Furthermore, given some hash value, it is infeasible to find some input data (other than
the one given) that will yield the same hash value. Message authentication codes, also called keyed cryptographic
hash functions, provide additional protection against intentional modification by an attacker.
Error-correcting codes
Any error-correcting code can be used for error detection. A code with minimum Hamming distance, d, can detect up
to d − 1 errors in a code word. Using minimum-distance-based error-correcting codes for error detection can be
suitable if a strict limit on the minimum number of errors to be detected is desired.
Codes with minimum Hamming distance d = 2 are degenerate cases of error-correcting codes, and can be used to
detect single errors. The parity bit is an example of a single-error-detecting code.
The Berger code is an early example of a unidirectional error(-correcting) code that can detect any number of errors
on an asymmetric channel, provided that only transitions of cleared bits to set bits or set bits to cleared bits can
occur.
239
Error detection and correction
Error correction
Automatic repeat request
Automatic Repeat reQuest (ARQ) is an error control method for data transmission that makes use of error-detection
codes, acknowledgment and/or negative acknowledgment messages, and timeouts to achieve reliable data
transmission. An acknowledgment is a message sent by the receiver to indicate that it has correctly received a data
frame.
Usually, when the transmitter does not receive the acknowledgment before the timeout occurs (i.e., within a
reasonable amount of time after sending the data frame), it retransmits the frame until it is either correctly received
or the error persists beyond a predetermined number of retransmissions.
Three types of ARQ protocols are Stop-and-wait ARQ, Go-Back-N ARQ, and Selective Repeat ARQ.
ARQ is appropriate if the communication channel has varying or unknown capacity, such as is the case on the
Internet. However, ARQ requires the availability of a back channel, results in possibly increased latency due to
retransmissions, and requires the maintenance of buffers and timers for retransmissions, which in the case of network
congestion can put a strain on the server and overall network capacity.[2]
Error-correcting code
An error-correcting code (ECC) or forward error correction (FEC) code is a system of adding redundant data, or
parity data, to a message, such that it can be recovered by a receiver even when a number of errors (up to the
capability of the code being used) were introduced, either during the process of transmission, or on storage. Since the
receiver does not have to ask the sender for retransmission of the data, a back-channel is not required in forward
error correction, and it is therefore suitable for simplex communication such as broadcasting. Error-correcting codes
are frequently used in lower-layer communication, as well as for reliable storage in media such as CDs, DVDs, hard
disks, and RAM.
Error-correcting codes are usually distinguished between convolutional codes and block codes:
• Convolutional codes are processed on a bit-by-bit basis. They are particularly suitable for implementation in
hardware, and the Viterbi decoder allows optimal decoding.
• Block codes are processed on a block-by-block basis. Early examples of block codes are repetition codes,
Hamming codes and multidimensional parity-check codes. They were followed by a number of efficient codes,
Reed-Solomon codes being the most notable due to their current widespread use. Turbo codes and low-density
parity-check codes (LDPC) are relatively new constructions that can provide almost optimal efficiency.
Shannon's theorem is an important theorem in forward error correction, and describes the maximum information rate
at which reliable communication is possible over a channel that has a certain error probability or signal-to-noise ratio
(SNR). This strict upper limit is expressed in terms of the channel capacity. More specifically, the theorem says that
there exist codes such that with increasing encoding length the probability of error on a discrete memoryless channel
can be made arbitrarily small, provided that the code rate is smaller than the channel capacity. The code rate is
defined as the fraction k/n of k source symbols and n encoded symbols.
The actual maximum code rate allowed depends on the error-correcting code used, and may be lower. This is
because Shannon's proof was only of existential nature, and did not show how to construct codes which are both
optimal and have efficient encoding and decoding algorithms.
240
Error detection and correction
Hybrid schemes
Hybrid ARQ is a combination of ARQ and forward error correction. There are two basic approaches:[2]
• Messages are always transmitted with FEC parity data (and error-detection redundancy). A receiver decodes a
message using the parity information, and requests retransmission using ARQ only if the parity data was not
sufficient for successful decoding (identified through a failed integrity check).
• Messages are transmitted without parity data (only with error-detection information). If a receiver detects an
error, it requests FEC information from the transmitter using ARQ, and uses it to reconstruct the original message.
The latter approach is particularly attractive on an erasure channel when using a rateless erasure code.
Applications
Applications that require low latency (such as telephone conversations) cannot use Automatic Repeat reQuest
(ARQ); they must use Forward Error Correction (FEC). By the time an ARQ system discovers an error and
re-transmits it, the re-sent data will arrive too late to be any good.
Applications where the transmitter immediately forgets the information as soon as it is sent (such as most television
cameras) cannot use ARQ; they must use FEC because when an error occurs, the original data is no longer available.
(This is also why FEC is used in data storage systems such as RAID and distributed data store).
Applications that use ARQ must have a return channel. Applications that have no return channel cannot use ARQ.
Applications that require extremely low error rates (such as digital money transfers) must use ARQ.
Internet
In a typical TCP/IP stack, error control is performed at multiple levels:
• Each Ethernet frame carries a CRC-32 checksum. Frames received with incorrect checksums are discarded by the
receiver hardware.
• The IPv4 header contains a checksum protecting the contents of the header. Packets with mismatching checksums
are dropped within the network or at the receiver.
• The checksum was omitted from the IPv6 header in order to minimize processing costs in network routing and
because current link layer technology is assumed to provide sufficient error detection (see also RFC 3819).
• UDP has an optional checksum covering the payload and addressing information from the UDP and IP headers.
Packets with incorrect checksums are discarded by the operating system network stack. The checksum is optional
under IPv4, only, because the Data-Link layer checksum may already provide the desired level of error
protection.
• TCP provides a checksum for protecting the payload and addressing information from the TCP and IP headers.
Packets with incorrect checksums are discarded within the network stack, and eventually get retransmitted using
ARQ, either explicitly (such as through triple-ack) or implicitly due to a timeout.
Deep-space telecommunications
Development of error-correction codes was tightly coupled with the history of deep-space missions due to the
extreme dilution of signal power over interplanetary distances, and the limited power availability aboard space
probes. Whereas early missions sent their data uncoded, starting from 1968 digital error correction was implemented
in the form of (sub-optimally decoded) convolutional codes and Reed-Muller codes.[3] The Reed-Muller code was
well suited to the noise the spacecraft was subject to (approximately matching a bell curve), and was implemented at
the Mariner spacecraft for missions between 1969 and 1977.
The Voyager 1 and Voyager 2 missions, which started in 1977, were designed to deliver color imaging amongst
scientific information of Jupiter and Saturn.[4] This resulted in increased coding requirements, and thus the spacecraft
241
Error detection and correction
were supported by (optimally Viterbi-decoded) convolutional codes that could be concatenated with an outer Golay
(24,12,8) code. The Voyager 2 probe additionally supported an implementation of a Reed-Solomon code: the
concatenated Reed-Solomon-Viterbi (RSV) code allowed for very powerful error correction, and enabled the
spacecraft's extended journey to Uranus and Neptune.
The CCSDS currently recommends usage of error correction codes with performance similar to the Voyager 2 RSV
code as a minimum. Concatenated codes are increasingly falling out of favor with space missions, and are replaced
by more powerful codes such as Turbo codes or LDPC codes.
The different kinds of deep space and orbital missions that are conducted suggest that trying to find a "one size fits
all" error correction system will be an ongoing problem for some time to come. For missions close to earth the nature
of the channel noise is different from that of a spacecraft on an interplanetary mission experiences. Additionally, as a
spacecraft increases its distance from earth, the problem of correcting for noise gets larger.
Satellite broadcasting (DVB)
The demand for satellite transponder bandwidth continues to grow, fueled by the desire to deliver television
(including new channels and High Definition TV) and IP data. Transponder availability and bandwidth constraints
have limited this growth, because transponder capacity is determined by the selected modulation scheme and
Forward error correction (FEC) rate.
Overview
• QPSK coupled with traditional Reed Solomon and Viterbi codes have been used for nearly 20 years for the
delivery of digital satellite TV.
• Higher order modulation schemes such as 8PSK, 16QAM and 32QAM have enabled the satellite industry to
increase transponder efficiency by several orders of magnitude.
• This increase in the information rate in a transponder comes at the expense of an increase in the carrier power to
meet the threshold requirement for existing antennas.
• Tests conducted using the latest chipsets demonstrate that the performance achieved by using Turbo Codes may
be even lower than the 0.8 dB figure assumed in early designs.
Data storage
Error detection and correction codes are often used to improve the reliability of data storage media.
A "parity track" was present on the first magnetic tape data storage in 1951. The "Optimal Rectangular Code" used
in group code recording tapes not only detects but also corrects single-bit errors.
Some file formats, particularly archive formats, include a checksum (most often CRC32) to detect corruption and
truncation and can employ redundancy and/or parity files to recover portions of corrupted data.
Reed Solomon codes are used in compact discs to correct errors caused by scratches.
Modern hard drives use CRC codes to detect and Reed-Solomon codes to correct minor errors in sector reads, and to
recover data from sectors that have "gone bad" and store that data in the spare sectors.[5]
RAID systems use a variety of error correction techniques, to correct errors when a hard drive completely fails.
242
Error detection and correction
Error-correcting memory
DRAM memory may provide increased protection against soft errors by relying on error correcting codes. Such
error-correcting memory, known as ECC or EDAC-protected memory, is particularly desirable for high fault-tolerant
applications, such as servers, as well as deep-space applications due to increased radiation.
Error-correcting memory controllers traditionally use Hamming codes, although some use triple modular
redundancy.
Interleaving allows distributing the effect of a single cosmic ray potentially upsetting multiple physically
neighboring bits across multiple words by associating neighboring bits to different words. As long as a single event
upset (SEU) does not exceed the error threshold (e.g., a single error) in any particular word between accesses, it can
be corrected (e.g., by a single-bit error system may be maintained.[6]
Notes
[1] See also Encyclopedia of Microcomputers: Volume 6 (http:/ / books. google. com. br/ books?id=ajpoqYqFrkQC& lpg=PA343& dq=Error
detection and correction is& as_brr=3& client=firefox-a& pg=PA343#v=snippet& q="error detection is"& f=false), p. 343.
[2] A. J. McAuley, Reliable Broadband Communication Using a Burst Erasure Correcting Code, ACM SIGCOMM, 1990.
[3] K. Andrews et al., The Development of Turbo and LDPC Codes for Deep-Space Applications, Proceedings of the IEEE, Vol. 95, No. 11, Nov.
2007.
[4] W. Huffman, V. Pless, Fundamentals of error-correcting codes, Cambridge University Press, ISBN 9780521782807, 2003.
[5] My Hard Drive Died | Scott A. Moulton (http:/ / www. myharddrivedied. com/ presentations_whitepaper. html)
[6] "Using StrongArm SA-1110 in the On-Board Computer of Nanosatellite" (http:/ / www. apmcsta. org/ File/ doc/ Conferences/ 6th meeting/
Chen Zhenyu. doc). Tsinghua Space Center, Tsinghua University, Beijing. . Retrieved 2009-02-16.
References
• Shu Lin, Daniel J. Costello, Jr. (1983). Error Control Coding: Fundamentals and Applications. Prentice Hall.
ISBN 0-13-283796-X.
External links
• The on-line textbook: Information Theory, Inference, and Learning Algorithms (http://www.inference.phy.
cam.ac.uk/mackay/itila/), by David MacKay, contains chapters on elementary error-correcting codes; on the
theoretical limits of error-correction; and on the latest state-of-the-art error-correcting codes, including
low-density parity-check codes, turbo codes, and fountain codes.
• Compute parameters of linear codes (http://ipsit.bu.edu/comp.html) - an on-line interface for generating and
computing parameters (e.g. minimum distance, covering radius) of linear error-correcting codes.
• ECC Page (http://www.eccpage.com/)
243
Linear encoder
244
Linear encoder
A linear encoder is a sensor,
transducer or readhead paired with a
scale that encodes position. The sensor
reads the scale in order to convert the
encoded position into an analog or
digital signal, which can then be
decoded into position by a digital
readout (DRO) or motion controller.
The encoder can be either incremental
or absolute. Motion can be determined
by change in position over time. Linear
encoder technologies include optical,
magnetic, inductive, capacitive and
eddy current. Optical technologies
include shadow, self imaging and
interferometric. Linear encoders are
used in metrology instruments, motion
systems and high precision machining
tools ranging from digital calipers and
coordinate measuring machines to
stages, CNC Mills, manufacturing
gantry tables and semiconductor
steppers.
Typical linear optical encoders.
Physical principle
Linear encoders are transducers that
exploit many different physical
properties in order to encode position:
Visualization of magnetic structures of a linear encoder (Recorded with MagView).
Scale / reference based
Optical
Optical linear encoders [1], [2] dominate the high resolution market and may employ shuttering / Moiré, diffraction
or holographic principles. Typical incremental scale periods vary from hundreds down to sub-micrometre and
following interpolation can provide resolutions as fine as a nanometre. Light sources used include infrared LEDs,
visible LEDs, miniature light-bulbs and laser diodes.
Magnetic
Magnetic linear encoders [3] employ either active (magnetized) or passive (variable reluctance) scales and position
may be sensed using sense-coils, Hall Effect or magnetoresistive readheads. With coarser scale periods than optical
encoders (typically a few hundred micrometers to several millimeters) resolutions in the order of a micrometer are
the norm.
Linear encoder
Capacitive
Capacitive linear encoders work by sensing the capacitance between a reader and scale. Typical applications are
digital calipers. One of the disadvantage is the sensitivity to uneven dirt, which can locally change the relative
permittivity.
Inductive
Inductive technology is robust to contaminants, allowing calipers and other measurement tools that are coolant-proof
[4]. A popular application of the inductive measuring principle is the Inductosyn [5]. In effect it is a resolver
unwound into a linear system. The Spherosyn encoder [6] is based on the principle of electromagnetic induction and
uses coils to sense nickel-chrome ball-bearings mounted within a tube.
Eddy current
US Patent 3820110, "Eddy current type digital encoder and position reference", gives an example of this type of
encoder, which uses a scale coded with high and low permeability, non-magnetic materials, which is detected and
decoded by monitoring changes in inductance of an AC circuit that includes an inductive coil sensor. Maxon [7]
makes an example (rotary encoder) product (the MILE encoder).
Without scales
Optical image sensor
The sensors are based on an image correlation method. The Sensor takes subsequent pictures from the surface being
measured and compares the images for displacement [8]. Resolutions down to 1 nm are possible. [9]
Applications
There are two main areas of application for linear encoders:
Measurement
Measurement application include coordinate-measuring machines (CMM), laser scanners, calipers, gear
measurement [10], tension testers [11] and Digital read outs (DROs).
Motion systems
Servo controlled motion systems employ linear encoder so as to provide accurate, high-speed movement. Typical
applications include robotics, machine tools, pick-and-place PCB assembly equipment; semiconductors handling and
test equipment, wire bonders, printers and digital presses.[12]
245
Linear encoder
Output Signal Formats
Incremental signals
Linear encoders can have analog or digital outputs.
Analog
The industry standard, analog output for linear encoders
is sine and cosine quadrature signals. These are usually
transmitted differentially so as to improve noise
immunity. An early industry standard was 12 µA
peak-peak current signals but more recently this has
been replaced with 1V peak to peak voltage signals.
Compared to digital transmission, the analog signals'
lower bandwidth helps to minimise emc emissions.
Quadrature sine/cosine signals can be monitored easily
by using an oscilloscope in XY mode to display a
circular Lissajous Figure. Highest accuracy signals are
obtained if the Lissajous Figure is circular (no gain or
phase error) and perfectly centred. Modern encoder
systems employ circuitry to trim these error mechanisms
automatically. The overall accuracy of the linear encoder
is a combination of the scale accuracy and errors
introduced by the readhead. Scale contributions to the
error budget include linearity and slope (scaling factor
error). Readhead error mechanisms are usually described
as cyclic error or sub-divisional error (SDE) as they
repeat every scale period. The largest contributor to
readhead inaccuracy is signal offset, followed by signal
imbalance (ellipticity) and phase error (the quadrature
The sine and cosine outputs.
signals not being exactly 90° apart). Overall signal size
does not affect encoder accuracy, however,
signal-to-noise and jitter performance may degrade with smaller signals. Automatic signal compensation
mechanisms can include automatic offset compensation (AOC), automatic balance compensation (ABC) and
automatic gain control (AGC). Phase is more difficult to compensate dynamically and is usually applied as one time
compensation during installation or calibration. Other forms of inaccuracy include signal distortion (frequently
harmonic distortion of the sine/cosine signals).
246
Linear encoder
Digital
Many linear encoders interpolate the
analogue sine/cosine signals in order to
sub-divide the scale period, providing
a higher measurement resolution. The
output of the interpolation process is
quadrature squarewaves – the distance
between edges of the two channels
being the resolution of the encoder.
The A and B quadrature channels
The reference mark or index pulse will
also be processed digitally and will be a pulse, usually one to four units-of-resolution wide.
The major advantage of encoders with built-in interpolation and digital signal transmission is improved noise
immunity. However, the high frequency, fast edge speed signals may produce more emc emissions.
Incremental encoders with built-in digital processing make it possible to transmit position to any subsequent
electronics such as a position counter.
Absolute reference signals
As well as analog or digital incremental output signals, linear encoders can provide absolute reference or positioning
signals.
Reference mark
Most incremental, linear encoders can produce an index or reference mark pulse providing a datum position along
the scale for use at power-up or following a loss of power. This index signal must be able to identify position within
one, unique period of the scale. The reference mark may comprise a single feature on the scale, an autocorrelator
pattern (typically a Barker code) or a chirp pattern.
Distance coded reference marks (DCRM) are placed onto the scale in a unique pattern allowing a minimal movement
(typically moving past two reference marks) to define the readhead's position. Multiple, equally spaced reference
marks may also be placed onto the scale such that following installation, the desired marker can be selected - usually
via a magnet or optically.
Absolute code
With suitably encoded scales (multitrack, vernier, digital code, or pseudo-random code) an encoder can determine its
position without movement or needing to find a reference position. Such absolute encoders also communicate using
serial communication protocols. Many of these protocols are proprietary – Fanuc, Mitsubishi, EnDat, DriveCliq,
Panasonic, Yaskawa – but open standards such as BiSS [13] are now appearing, which avoid tying users to a
particular supplier.
247
Linear encoder
Limit switches
Many linear encoders include built-in limit switches - either optical or magnetic. Two limit switches are frequently
included such that on power-up the controller can determine if the encoder is at an end-of-travel and in which
direction to drive the axis.
Physical arrangement / protection
Linear encoders may be either enclosed or open. Enclosed linear encoders are employed in dirty, hostile
environments such as machine-tools. They typically comprise an aluminium extrusion enclosing a glass or metal
scale. Flexible lip seals allow an internal, guided readhead to read the scale. Accuracy is limited due to the friction
and hysteresis imposed by this mechanical arrangement.
For the highest accuracy, lowest measurement hysteresis and lowest friction applications, open linear encoders are
used.
Linear encoders may use transmissive (glass) or reflective scales, employing Ronchi or phase gratings. Scale
materials include chrome on glass, metal (stainless steel, gold plated steel, Invar), ceramics (Zerodur) and plastics.
The scale may be self supporting, be thermally mastered to the substrate (via adhesive or adhesive tape) or track
mounted. Track mounting may allow the scale to maintain its own coefficient of thermal expansion and allows large
equipment to be broken down for shipment.
Encoder terms
Resolution
Repeatability
Hysteresis
Signal-to-noise ratio / Noise / Jitter
Lissajous Figure
Quadrature
Index / Reference Mark / Datum / Fiducial
Distance Coded Reference Marks (DCRM)
Book
• David S. Nyce: Linear Position Sensors: Theory and Application, New Jersey, John Wiley & Sons Inc. (2003)
• Walcher Hans: Position Sensing: Angle and Distance Measurement for Engineers, Butterworth Heinemann
(1994)
References
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
http:/ / www. microesys. com/ m2/ index. html
http:/ / www. renishaw. com/ en/ 6433. aspx
http:/ / www. rls. si/ default. asp?prod=LMencoders
http:/ / www. mitutoyo. com/ pdf/ ABS1813-293. pdf
http:/ / www. ruhle. com/ bar_scale. htm
http:/ / www. newall. com/ LEDs/ operation. htm
http:/ / www. maxonmotor. com/ downloads/ Flyer_EC6_MILE_e_03. 09. pdf
http:/ / www. intacton. com/ us/ products/ INTACTON/ OpticalMotionSensors_Context/
LengthSpeedSensors_Optical_Context_Technology_PopupBase. html
[9] http:/ / www. mitutoyo. com/ pdf/ 1976_MICSYS. pdf
[10] http:/ / www. wenzel-cmm. co. uk/ Industries. asp?SE=9
[11] http:/ / www. instron. co. uk/ wa/ product/ Tension-Testers. aspx
[12] http:/ / global. oce. com/ products/ productionprinting/ digitalpresses/ color/ default. aspx
[13] http:/ / www. biss-interface. com/
248
Gray code
249
Gray code
Gray code
by bit width
2-bit 4-bit
00
01
11
10
3-bit
000
001
011
010
110
111
101
100
0000
0001
0011
0010
0110
0111
0101
0100
1100
1101
1111
1110
1010
1011
1001
1000
The reflected binary code, also known as Gray code after Frank Gray, is a binary numeral system where two
successive values differ in only one bit. It is a non-weighted code.
The reflected binary code was originally designed to prevent spurious output from electromechanical switches.
Today, Gray codes are widely used to facilitate error correction in digital communications such as digital terrestrial
television and some cable TV systems.
Name
Bell Labs researcher Frank Gray introduced the term reflected binary
code in his 1947 patent application, remarking that the code had "as yet
no recognized name".[1] He derived the name from the fact that it "may
be built up from the conventional binary code by a sort of reflection
process".
The code was later named after Gray by others who used it. Two
different 1953 patent applications give "Gray code" as an alternative
name for the "reflected binary code";[2] [3] one of those also lists
"minimum error code" and "cyclic permutation code" among the
names.[3] A 1954 patent application refers to "the Bell Telephone Gray
code".[4]
Gray's patent introduces the term "reflected
binary code"
Gray code
250
Motivation
Many devices indicate position by closing and opening switches. If that device uses natural binary codes, these two
positions would be right next to each other:
...
011
100
...
The problem with natural binary codes is that, with real (mechanical) switches, it is very unlikely that switches will
change states exactly in synchrony. In the transition between the two states shown above, all three switches change
state. In the brief period while all are changing, the switches will read some spurious position. Even without
keybounce, the transition might look like 011 — 001 — 101 — 100. When the switches appear to be in position 001,
the observer cannot tell if that is the "real" position 001, or a transitional state between two other positions. If the
output feeds into a sequential system (possibly via combinational logic) then the sequential system may store a false
value.
The reflected binary code solves this problem by changing only one switch at a time, so there is never any ambiguity
of position,
Dec
0
1
2
3
4
5
6
7
Gray
000
001
011
010
110
111
101
100
Binary
000
001
010
011
100
101
110
111
Notice that state 7 can roll over to state 0 with only one switch change. This is called the "cyclic" property of a Gray
code. A good way to remember Gray coding is by being aware that the least significant bit follows a repetitive
pattern of 2. That is 11, 00, 11 etc. and the second digit follows a pattern of fours.
More formally, a Gray code is a code assigning to each of a contiguous set of integers, or to each member of a
circular list, a word of symbols such that each two adjacent code words differ by one symbol. These codes are also
known as single-distance codes, reflecting the Hamming distance of 1 between adjacent codes. There can be more
than one Gray code for a given word length, but the term was first applied to a particular binary code for the
non-negative integers, the binary-reflected Gray code, or BRGC, the three-bit version of which is shown above.
History and practical application
Reflected binary codes were applied to mathematical puzzles before they became known to engineers. The French
engineer Émile Baudot used Gray codes in telegraphy in 1878. He received the French Legion of Honor medal for
his work. The Gray code is sometimes attributed, incorrectly,[5] to Elisha Gray (in Principles of Pulse Code
Modulation, K. W. Cattermole,[6] for example).
Frank Gray, who became famous for inventing the signaling method that came to be used for compatible color
television, invented a method to convert analog signals to reflected binary code groups using vacuum tube-based
apparatus. The method and apparatus were patented in 1953 and the name of Gray stuck to the codes. The "PCM
tube" apparatus that Gray patented was made by Raymond W. Sears of Bell Labs, working with Gray and William
M. Goodall, who credited Gray for the idea of the reflected binary code.[7]
Gray code
251
Part of front page of Gray's patent, showing PCM tube (10) with reflected binary code in plate (15)
The use of his eponymous codes that Gray was most interested in was to minimize the effect of error in the
conversion of analog signals to digital; his codes are still used today for this purpose, and others.
Position encoders
Gray codes are used in position encoders (linear encoders and rotary
encoders), in preference to straightforward binary encoding. This
avoids the possibility that, when several bits change in the binary
representation of an angle, a misread will result from some of the bits
changing before others. Originally, the code pattern was electrically
conductive, supported (in a rotary encoder) by an insulating disk. Each
track had its own stationary metal spring contact; one more contact
made the connection to the pattern. That common contact was
connected by the pattern to whichever of the track contacts were
resting on the conductive pattern. However, sliding contacts wear out
and need maintenance, which favors optical encoders.
Rotary encoder for angle-measuring devices
Regardless of the care in aligning the contacts, and accuracy of the
marked in 3-bit binary-reflected Gray code
pattern, a natural-binary code would have errors at specific disk
(BRGC)
positions, because it is impossible to make all bits change at exactly the
same time as the disk rotates. The same is true of an optical encoder;
transitions between opaque and transparent cannot be made to happen simultaneously for certain exact positions.
Rotary encoders benefit from the cyclic nature of Gray codes, because consecutive positions of the sequence differ
by only one bit.
Gray code
Towers of Hanoi
The binary-reflected Gray code can also be used to serve as a solution
guide for the Towers of Hanoi problem, as well as the classical
Chinese rings puzzle, a sequential mechanical puzzle mechanism.[5] It
also forms a Hamiltonian cycle on a hypercube, where each bit is seen
as one dimension.
Genetic algorithms
Due to the Hamming distance properties of Gray codes, they are
sometimes used in genetic algorithms. They are very useful in this
field, since mutations in the code allow for mostly incremental
changes, but occasionally a single bit-change can cause a big leap and
lead to new properties.
Karnaugh maps
Gray codes are also used in labelling the axes of Karnaugh maps.[8]
Error correction
In modern digital communications, Gray codes play an important role
A Gray code absolute rotary encoder with 13
in error correction. For example, in a digital modulation scheme such
tracks. At the top can be seen the housing,
as QAM where data is typically transmitted in symbols of 4 bits or
interrupter disk, and light source; at the bottom
more, the signal's constellation diagram is arranged so that the bit
can be seen the sensing element and support
patterns conveyed by adjacent constellation points differ by only one
components.
bit. By combining this with forward error correction capable of
correcting single-bit errors, it is possible for a receiver to correct any transmission errors that cause a constellation
point to deviate into the area of an adjacent point. This makes the transmission system less susceptible to noise.
Communication between clock domains
Digital logic designers use Gray codes extensively for passing multi-bit count information between synchronous
logic that operates at different clock frequencies. The logic is considered operating in different "clock domains". It is
fundamental to the design of large chips that operate with many different clocking frequencies.
Gray code counters and arithmetic
A typical use of Gray code counters is building a FIFO (first-in, first-out) data buffer that has read and write ports
that exist in different clock domains. The input and output counters inside such a dual-port FIFO are often stored
using Gray code to prevent invalid transient states from being captured when the count crosses clock domains.[9] The
updated read and write pointers need to be passed between clock domains when they change, to be able to track
FIFO empty and full status in each domain. Each bit of the pointers is sampled non-deterministically for this clock
domain transfer. So for each bit, either the old value or the new value is propagated. Therefore, if more than one bit
in the multi-bit pointer is changing at the sampling point, a "wrong" binary value (neither new nor old) can be
propagated. By guaranteeing only one bit can be changing, Gray codes guarantee that the only possible sampled
values are the new or old multi-bit value. Typically Gray codes of power-of-two length are used.
252
Gray code
253
Sometimes digital buses in electronic systems are used to convey quantities that can only increase or decrease by one
at a time, for example the output of an event counter which is being passed between clock domains or to a
digital-to-analog converter. The advantage of Gray codes in these applications is that differences in the propagation
delays of the many wires that represent the bits of the code cannot cause the received value to go through states that
are out of the Gray code sequence. This is similar to the advantage of Gray codes in the construction of mechanical
encoders, however the source of the Gray code is an electronic counter in this case. The counter itself must count in
Gray code, or if the counter runs in binary then the output value from the counter must be reclocked after it has been
converted to Gray code, because when a value is converted from binary to Gray code, it is possible that differences
in the arrival times of the binary data bits into the binary-to-Gray conversion circuit will mean that the code could go
briefly through states that are wildly out of sequence. Adding a clocked register after the circuit that converts the
count value to Gray code may introduce a clock cycle of latency, so counting directly in Gray code may be
advantageous. A Gray code counter was patented in 1962 US3020481 [10], and there have been many others since. In
recent times a Gray code counter can be implemented as a state machine in Verilog. In order to produce the next
count value, it is necessary to have some combinational logic that will increment the current count value that is
stored in Gray code. Probably the most obvious way to increment a Gray code number is to convert it into ordinary
binary code, add one to it with a standard binary adder, and then convert the result back to Gray code. This approach
was discussed in a paper in 1996 [11] and then subsequently patented by someone else in 1998 US5754614 [12]. Other
methods of counting in Gray code are discussed in a report by R. W. Doran, including taking the output from the
first latches of the master-slave flip flops in a binary ripple counter.[13]
Perhaps the most common electronic counter with the "only one bit changes at a time" property is the Johnson
counter.
Constructing an n-bit Gray code
The binary-reflected Gray code list for n bits can be generated
recursively from the list for n−1 bits by reflecting the list (i.e. listing
the entries in reverse order), concatenating the original list with the
reversed list, prefixing the entries in the original list with a binary 0,
and then prefixing the entries in the reflected list with a binary 1. For
example, generating the n = 3 list from the n = 2 list:
The first few steps of the reflect-and-prefix
method.
Gray code
254
4-bit Gray code permutation
2-bit list:
00, 01, 11, 10
Reflected:
10, 11, 01, 00
Prefix old entries with 0:
000, 001, 011, 010,
Prefix new entries with 1:
Concatenated:
110, 111, 101, 100
000, 001, 011, 010, 110, 111, 101, 100
The one-bit Gray code is G1 = (0, 1). This can be thought of as built recursively as above from a zero-bit Gray code
G0 = { Λ } consisting of a single entry of zero length. This iterative process of generating Gn+1 from Gn makes the
following properties of the standard reflecting code clear:
• Gn is a permutation of the numbers 0, ... , 2n−1. (Each number appears exactly once in the list.)
• Gn is embedded as the first half of Gn+1.
• Therefore the coding is stable, in the sense that once a binary number appears in Gn it appears in the same
position in all longer lists; so it makes sense to talk about the reflective Gray code value of a number: G(m) = the
m-th reflecting Gray code, counting from 0.
• Each entry in Gn differs by only one bit from the previous entry. (The Hamming distance is 1.)
• The last entry in Gn differs by only one bit from the first entry. (The code is cyclic.)
These characteristics suggest a simple and fast method of translating a binary value into the corresponding Gray
code. Each bit is inverted if the next higher bit of the input value is set to one. This can be performed in parallel by a
bit-shift and exclusive-or operation if they are available: the nth Gray code is obtained by computing
A similar method can be used to perform the reverse translation, but the computation of each bit depends on the
computed value of the next higher bit so it cannot be performed in parallel. Assuming is the th gray-coded bit (
being the most significant bit), and
is the
reverse translation can be given recursively:
th binary-coded bit (
, and
being the most-significant bit), the
. Alternatively, decoding a Gray code
into a binary number can be described as a prefix sum of the bits in the Gray code, where each individual summation
operation in the prefix sum is performed modulo two.
To construct the binary-reflected Gray code iteratively, start with the code 0, and at step i find the bit position of the
least significant '1' in the binary representation of i - flip the bit at that position in the previous code to get the next
code. The bit positions start 0, 1, 0, 2, 0, 1, 0, 3, ... (sequence A007814 in OEIS). See find first set for efficient
Gray code
255
algorithms to compute these values.
Converting to and from Gray code
/*
The purpose of this function is to convert an unsigned
binary number to reflected binary Gray code.
*/
unsigned short binaryToGray(unsigned short num)
{
return (num>>1) ^ num;
}
/*
The purpose of this function is to convert a reflected binary
Gray code number to a binary number.
*/
unsigned short grayToBinary(unsigned short num)
{
unsigned short temp = num ^ (num>>8);
temp ^= (temp>>4);
temp ^= (temp>>2);
temp ^= (temp>>1);
return temp;
}
Special types of Gray codes
In practice, a "Gray code" almost always refers to a binary-reflected Gray code (BRGC). However, mathematicians
have discovered other kinds of Gray codes. Like BRGCs, each consists of a lists of words, where each word differs
from the next in only one digit (each word has a Hamming distance of 1 from the next word).
n-ary Gray code
There are many specialized types of Gray codes other than the binary-reflected Gray code. One such type of Gray
code is the n-ary Gray code, also known as a non-Boolean Gray code. As the name implies, this type of Gray code
uses non-Boolean values in its encodings.
For example, a 3-ary (ternary) Gray code would use the values {0, 1, 2}. The (n, k)-Gray code is the n-ary Gray code
with k digits.[14] The sequence of elements in the (3, 2)-Gray code is: {00, 01, 02, 12, 10, 11, 21, 22, 20}. The
(n, k)-Gray code may be constructed recursively, as the BRGC, or may be constructed iteratively. An algorithm to
iteratively generate the (N, k)-Gray code is presented (in C):
// parameters: value, base, digits
// Convert a value to a graycode with the given base and digits.
Iterating
// through a sequence of values would result in a sequence of
graycodes in
// which only one digit changes at a time.
Gray code
int baseN[digits];
per entry
int gray[digits];
int i;
256
// Stores the ordinary base-N number, one digit
// Stores the base-N graycode number
// The loop variable
// Put the normal baseN number into the baseN array. For base 10, 109
// would be stored as [9,0,1]
int tempvalue = value;
for(i = 0; i < digits; i++) {
baseN[i] = tempvalue % base;
tempvalue /= base;
}
// Convert the normal baseN number into the graycode equivalent. Note
that
// the loop starts at the most significant digit and goes down.
int shift = 0;
for(i = digits - 1; i >= 0; i--) {
// The gray digit gets shifted down equal to the sum of the
higher
// digits.
gray[i] = (baseN[i] - shift) % base;
shift += gray[i] - base; // - base to prevent negative in mod
operation
}
// EXAMPLES
// input: value = 1899, base = 10, digits = 4
// output: baseN[] = [9,9,8,1], gray[] = [0,1,7,1]
// input: value = 1900, base = 10, digits = 4
// output: baseN[] = [0,0,9,1], gray[] = [0,1,8,1]
Gray code
257
Ternary number -> ternary Gray code
0
1
2
10
11
12
20
21
22
100
101
102
110
111
112
120
121
122
200
201
202
210
211
212
220
221
222
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
000
001
002
012
010
011
021
022
020
120
121
122
102
100
101
111
112
110
210
211
212
222
220
221
201
202
200
There are other graycode algorithms for (n,k)-Gray codes. It is important to note that the (n,k)-Gray codes produced
by the above algorithm is always cyclical; some algorithms, such as that by Guan,[14] lack this property when k is
odd. On the other hand, while only one digit at a time changes with this method, it can change by wrapping (looping
from n-1 to 0). In Guan's algorithm, the count alternately rises and falls, so that the numeric difference between two
graycode digits is always one.
Gray codes are not uniquely defined, because a permutation of the columns of such a code is a Gray code too. The
above procedure produces a code in which the lower the significance of a digit, the more often it changes, making it
similar to normal counting methods.
Balanced Gray code
Although the binary reflected Gray code is useful in many scenarios, it is not optimal in certain cases because of a
lack of "uniformity".[15] In balanced Gray codes, we seek to make the number of changes in each coordinate position
as close as possible. To make this more precise, let G be a R-ary complete Gray cycle having transition sequence
; the transition counts (spectrum) of
are the collection of integers defined by
We say that a Gray code is uniform or uniformly balanced if its transition counts are all equal, in which case we have
for all k. Clearly, when
, such codes exist only if n is a power of 2. Otherwise, if n does not
divide
evenly, the best we can do is to construct well-balanced codes where every transition count is either
or
. Gray codes can also be exponentially balanced if all of their transition counts are adjacent
powers of two, and such codes exist for every power of two.[16]
We will now show a construction for well-balanced binary Gray codes which allows us to generate a n-digit
balanced Gray code for every n.[17] The main principle is to inductively construct a
-digit Gray code
Gray code
258
given an n-digit Gray code G in such a way that the balanced property is preserved. To do this, we consider
partitions of
into an even number L of non-empty blocks of the form
where
, and
(mod
). This partition induces a
-digit Gray code given
by
If we define the transition multiplicities
to be the number of times the digit
in position i changes between consecutive blocks in a partition, then for the
this partition the transition spectrum
-digit Gray code induced by
is
The delicate part of this construction is to find an adequate partitioning of a balanced n-digit Gray code such that the
code induced by it remains balanced. Uniform codes can be found when
and
, and this construction can be extended to the R-ary case as well.[17]
Monotonic Gray codes
Monotonic codes are useful in the theory of interconnection networks, especially for minimizing dilation for linear
arrays of processors.[18] If we define the weight of a binary string to be the number of 1's in the string, then although
we clearly cannot have a Gray code with strictly increasing weight, we may want to approximate this by having the
code run through two adjacent weights before reaching the next one.
We can formalize the concept of monotone Gray codes as follows: consider the partition of the hypercube
into levels of vertices that have equal weight, i.e.
for
. These levels satisfy
, and let
be the edges in
. Let
be the subgraph of
induced by
. A monotonic Gray code is then a Hamiltonian path
in
such that whenever
comes before
in the path, then
.
An elegant construction of monotonic n-digit Gray codes for any n is based on the idea of recursively building
subpaths
of length
or
otherwise. Here,
having edges in
.[18] We define
first few values of
whenever
, and
is a suitably defined permutation and
. These paths give rise to two monotonic n-digit Gray codes
The choice of
,
refers to the path P with its coordinates permuted by
and
given by
which ensures that these codes are indeed Gray codes turns out to be
are shown in the table below.
. The
Gray code
259
Subpaths in the Savage-Winkler algorithm
j=0
j=1
n=1
0, 1
n=2
00, 01
10, 11
n=3
000, 001
100, 110, 010, 011
j=2
j=3
101, 111
n = 4 0000, 0001 1000, 1100, 0100, 0110, 0010, 0011 1010, 1011, 1001, 1101, 0101, 0111 1110, 1111
These monotonic Gray codes can be efficiently implemented in such a way that each subsequent element can be
generated in O(n) time. The algorithm is most easily described using coroutines.
Monotonic codes have an interesting connection to the Lovász conjecture, which states that every connected
vertex-transitive graph contains a Hamiltonian path. The "middle-level" subgraph
is vertex-transitive
(that is, its automorphism group is transitive, so that each vertex has the same "local environment"" and cannot be
differentiated from the others, since we can relabel the coordinates as well as the binary digits to obtain an
automorphism) and the problem of finding a Hamiltonian path in this subgraph is called the "middle-levels
problem", which can provide insights into the more general conjecture. The question has been answered
affirmatively for
, and the preceding construction for monotonic codes ensures a Hamiltonian path of
length at least 0.839N where N is the number of vertices in the middle-level subgraph.[19]
Beckett–Gray code
Another type of Gray code, the Beckett–Gray code, is named for Irish playwright Samuel Beckett, who was
interested in symmetry. His play "Quad" features four actors and is divided into sixteen time periods. Each period
ends with one of the four actors entering or leaving the stage. The play begins with an empty stage, and Beckett
wanted each subset of actors to appear on stage exactly once.[20] Clearly the set of actors currently on stage can be
represented by a 4-bit binary Gray code. Beckett, however, placed an additional restriction on the script: he wished
the actors to enter and exit so that the actor who had been on stage the longest would always be the one to exit. The
actors could then be represented by a first in, first out queue, so that (of the actors onstage) the actor being dequeued
is always the one who was enqueued first.[20] Beckett was unable to find a Beckett–Gray code for his play, and
indeed, an exhaustive listing of all possible sequences reveals that no such code exists for n = 4. It is known today
that such codes do exist for n = 2, 5, 6, 7, and 8, and do not exist for n = 3 or 4. An example of an 8-bit Beckett–Gray
code can be found in Knuth's Art of Computer Programming.[5] According to Sawada and Wong, the search space
for n = 6 can be explored in 15 hours, and more than 9,500 solutions for the case n = 7 have been found.[21]
Snake-in-the-box codes
Snake-in-the-box codes, or snakes, are the sequences of nodes of induced paths in an n-dimensional hypercube
graph, and coil-in-the-box codes, or coils, are the sequences of nodes of induced cycles in a hypercube. Viewed as
Gray codes, these sequences have the property of being able to detect any single-bit coding error. Codes of this type
were first described by W. H. Kautz in the late 1950s;[22] since then, there has been much research on finding the
code with the largest possible number of codewords for a given hypercube dimension.
Gray code
Single-track Gray code
Yet another kind of Gray code is the single-track Gray code (STGC) developed by N. B. Spedding (NZ Patent
264738 - October 28, 1994)[23] and refined by Hiltgen, Paterson and Brandestini in "Single-track Gray codes"
(1996).[24] The STGC is a cyclical list of P unique binary encodings of length n such that two consecutive words
differ in exactly one position, and when the list is examined as a P x n matrix, each column is a cyclic shift of the
first column.[25]
The name comes from their use with rotary encoders, where a number of tracks are being sensed by contacts,
resulting for each in an output of 0 or 1. To reduce noise due to different contacts not switching at exactly the same
moment in time, one preferably sets up the tracks so that the data output by the contacts are in Gray code. To get
high angular accuracy, one needs lots of contacts; in order to achieve at least 1 degree accuracy, one needs at least
360 distinct positions per revolution, which requires a minimum of 9 bits of data, and thus the same number of
contacts.
If all contacts are placed at the same angular position, then 9 tracks are needed to get a standard BRGC with at least
1 degree accuracy. However, if the manufacturer moves a contact to a different angular position (but at the same
distance from the center shaft), then the corresponding "ring pattern" needs to be rotated the same angle to give the
same output. If the most significant bit (the inner ring in Figure 1) is rotated enough, it exactly matches the next ring
out. Since both rings are then identical, the inner ring can be cut out, and the sensor for that ring moved to the
remaining, identical ring (but offset at that angle from the other sensor on that ring). Those 2 sensors on a single ring
make a quadrature encoder. That reduces the number of tracks for a "1 degree resolution" angular encoder to 8
tracks. Reducing the number of tracks still further can't be done with BRGC.
For many years, Torsten Sillke and other mathematicians believed that it was impossible to encode position on a
single track such that consecutive positions differed at only a single sensor, except for the 2-sensor, 1-track
quadrature encoder. So for applications where 8 tracks were too bulky, people used single-track incremental
encoders (quadrature encoders) or 2-track "quadrature encoder + reference notch" encoders.
N. B. Spedding, however, registered a patent in 1994 with several examples showing that it was possible.[23] Hiltgen
and Paterson published a paper in 2001 exhibiting a single-track gray code with exactly 360 angular positions,
constructed using 9 sensors,[26] the same number as a BRGC with the same resolution. (It would be impossible to
discriminate that many positions with any fewer sensors.)
An STGC for P = 30 and n = 5 is reproduced here:
10000
10100
11100
11110
260
Gray code
11010
11000
01000
01010
01110
01111
01101
01100
00100
00101
00111
10111
10110
00110
00010
10010
10011
11011
01011
00011
00001
01001
11001
11101
10101
10001
Note that each column is a cyclic shift of the first column, and from any row to the next row only one bit changes.[27]
The single-track nature (like a code chain) is useful in the fabrication of these wheels (compared to BRGC), as only
one track is needed, thus reducing their cost and size. The Gray code nature is useful (compared to chain codes, also
called De Bruijn sequences), as only one sensor will change at any one time, so the uncertainty during a transition
between two discrete states will only be plus or minus one unit of angular measurement the device is capable of
resolving.[28]
Notes
[1] F. Gray. Pulse code communication, March 17, 1953 (filed Nov. 1947). U.S. Patent 2632058 (http:/ / www. google. com/
patents?vid=2632058)
[2] J. Breckman. Encoding Circuit, Jan 31, 1956 (filed Dec. 1953). U.S. Patent 2733432 (http:/ / www. google. com/ patents?vid=2733432)
[3] E. A. Ragland et al. Direction-Sensitive Binary Code Position Control System, Feb. 11, 1958 (filed Oct. 1953). U.S. Patent 2823345 (http:/ /
www. google. com/ patents?vid=2823345)
[4] S. Reiner et al. Automatic Rectification System, Jun 24, 1958 (filed Jan. 1954). U.S. Patent 2839974 (http:/ / www. google. com/
patents?vid=2839974)
[5] Knuth, Donald E. "Generating all n-tuples." The Art of Computer Programming, Volume 4A: Enumeration and Backtracking, pre-fascicle 2a,
October 15, 2004. (http:/ / www-cs-faculty. stanford. edu/ ~knuth/ fasc2a. ps. gz)
[6] Cattermole, K. W. (1969). Principles of Pulse Code Modulation. New York: American Elsevier. ISBN 0-444-19747-8.
[7] Goodall, W. M. (1951). "Television by Pulse Code Modulation". Bell Sys. Tech. J. 30: 33–49.
[8] Wakerly, John F (1994). Digital Design: Principles & Practices. New Jersey: Prentice Hall. pp. 222, 48–49. ISBN 0132114593. Note that the
two page sections taken together say that K-maps are labeled with Gray code. The first section says that they are labeled with a code that
changes only one bit between entries and the second section says that such a code is called Gray code.
261
Gray code
[9] "Synchronization in Digital Logic Circuits (http:/ / www. stanford. edu/ class/ ee183/ handouts_spr2003/ synchronization_pres. pdf) by Ryan
Donohue
[10] http:/ / www. google. com/ patents?vid=3020481
[11] Mehta, H.; Owens, R.M. & Irwin, M.J. (1996), Some issues in gray code addressing (http:/ / ieeexplore. ieee. org/ xpls/ abs_all. jsp?tp=&
arnumber=497616& isnumber=10625), in the Proceedings of the 6th Great Lakes Symposium on VLSI (GLSVLSI 96), IEEE Computer
Society,pp. 178
[12] http:/ / www. google. com/ patents?vid=5754614
[13] The Gray Code by R. W. Doran (http:/ / www. cs. auckland. ac. nz/ CDMTCS/ / researchreports/ 304bob. pdf)
[14] Guan, Dah-Jyh (1998). "Generalized Gray Codes with Applications" (http:/ / nr. stpi. org. tw/ ejournal/ ProceedingA/ v22n6/ 841-848. pdf)
(PDF). Proc. Natl. Sci. Counc. Repub. Of China (A) 22: 841–848. .
[15] Girish S. Bhat and Carla D. Savage (1996). "Balanced Gray codes" (http:/ / www. combinatorics. org/ Volume_3/ Abstracts/ v3i1r25. html).
The Electronic Journal of Combinatorics 3 (1): R25. .
[16] I. N Suparta (2005). "A simple proof for the existence of exponentially balanced Gray codes". Electron. J. Combin 12.
[17] M. Flahive and B. Bose (2007). "Balancing cyclic R-ary Gray codes". the electronic journal of combinatorics 14.
[18] C. D Savage and P. Winkler (1995). "Monotone Gray codes and the middle levels problem". Journal of Combinatorial Theory, Series A 70
(2): 230–248. doi:10.1016/0097-3165(95)90091-8. ISSN 0097-3165.
[19] C. D Savage (1997). Long cycles in the middle two levels of the Boolean lattice.
[20] Goddyn, Luis (1999). "MATH 343 Applied Discrete Math Supplementary Materials" (http:/ / www. math. sfu. ca/ ~goddyn/ Courses/ 343/
supMaterials. pdf) (PDF). Dept. of Math, Simon Fraser U. .
[21] Wong, J. (2007). "A Fast Algorithm to generate Beckett-Gray codes". Electronic Notes in Discrete Mathematics 29: 571–577.
doi:10.1016/j.endm.2007.07.091.
[22] Kautz, W. H. (1958). "Unit-distance error-checking codes". IRE Trans. Elect. Comput. 7: 177–180.
[23] A position encoder (http:/ / www. winzurf. co. nz/ Single_Track_Grey_Code_Patent/ Single_track_Grey_code_encoder_patent. pdf). 1994. .
[24] Hiltgen, Alain P.; Kenneth G. Paterson, Marco Brandestini (1996). "Single-Track Gray Codes" (http:/ / ieeexplore. ieee. org/ iel1/ 18/
11236/ 00532900. pdf) (PDF). IEEE Transactions on Information Theory 42 (5): 1555–1561. doi:10.1109/18.532900. .
[25] Etzion, Tuvi; Moshe Schwartz (1999). "The Structure of Single-Track Gray Codes" (http:/ / www. cs. technion. ac. il/ ~etzion/ PUB/ Gray2.
pdf) (PDF). IEEE Transactions on Information Theory 45 (7): 2383–2396. doi:10.1109/18.796379. .
[26] Hiltgen, Alain P.; Kenneth G. Paterson (2001). "Single-Track Circuit Codes" (http:/ / www. hpl. hp. com/ techreports/ 2000/ HPL-2000-81.
pdf) (PDF). IEEE Transactions on Information Theory 47 (6): 2587–2595. doi:10.1109/18.945274. .
[27] "Venn Diagram Survey — Symmetric Diagrams" (http:/ / www. combinatorics. org/ Surveys/ ds5/ VennSymmEJC. html). The Electronic
Journal of Combinatorics. 2001. .
[28] Alciatore, David G.; Michael B. Histand (1999). McGraw-Hill Education - Europe. ISBN 9780071314442. http:/ / mechatronics. colostate.
edu/ .
References
• Black, Paul E. Gray code (http://www.nist.gov/dads/HTML/graycode.html). 25 February 2004. NIST.
• Press, WH; Teukolsky, SA; Vetterling, WT; Flannery, BP (2007). "Section 22.3. Gray Codes" (http://apps.
nrbook.com/empanel/index.html#pg=1166). Numerical Recipes: The Art of Scientific Computing (3rd ed.).
New York: Cambridge University Press. ISBN 978-0-521-88068-8.
• Savage, Carla (1997). "A Survey of Combinatorial Gray Codes" (http://www.csc.ncsu.edu/faculty/savage/
AVAILABLE_FOR_MAILING/survey.ps). SIAM Rev. 39 (4): 605–629. doi:10.1137/S0036144595295272.
JSTOR 2132693.
• Wilf, Herbert S. (1989). "Chapters 1-3". Combinatorial algorithms: an update. SIAM. ISBN 0-89871-231-9.
262
Gray code
External links
• "Gray Code" demonstration (http://demonstrations.wolfram.com/BinaryGrayCode/) by Michael Schreiber,
Wolfram Demonstrations Project (with Mathematica implementation). 2007.
• NIST Dictionary of Algorithms and Data Structures: Gray code (http://www.nist.gov/dads/HTML/graycode.
html)
• Hitch Hiker's Guide to Evolutionary Computation, Q21: What are Gray codes, and why are they used? (http://
www.aip.de/~ast/EvolCompFAQ/Q21.htm), including C code to convert between binary and BRGC
• Subsets or Combinations (http://www.theory.cs.uvic.ca/~cos/gen/comb.html) Can generate BRGC strings
• "The Structure of Single-Track Gray Codes" (http://www.cs.technion.ac.il/users/wwwb/cgi-bin/tr-info.
cgi?1998/CS/CS0937) by Moshe Schwartz, Tuvi Etzion
• Single-Track Circuit Codes (http://www.hpl.hp.com/techreports/2000/HPL-2000-81.html) by Hiltgen, Alain
P.; Paterson, Kenneth G.
• Dragos A. Harabor uses Gray codes in a 3D digitizer (http://www.ugcs.caltech.edu/~dragos/3DP/coord.
html).
• single-track gray codes, binary chain codes ( Lancaster 1994 (http://tinaja.com/text/chain01.html)), and linear
feedback shift registers are all useful in finding one's absolute position on a single-track rotary encoder (or other
position sensor).
• Computing Binary Combinatorial Gray Codes Via Exhaustive Search With SAT Solvers (http://ieeexplore.ieee.
org/xpls/abs_all.jsp?isnumber=4475352&arnumber=4475394&count=44&index=39) by Zinovik, I.;
Kroening, D.; Chebiryak, Y.
• AMS Column: Gray codes (http://www.ams.org/featurecolumn/archive/gray.html)
• Optical Encoder Wheel Generator (http://www.bushytails.net/~randyg/encoder/encoderwheel.html)
• ProtoTalk.net - Understanding Quadrature Encoding (http://prototalk.net/forums/showthread.php?t=78) Covers quadrature encoding in more detail with a focus on robotic applications
263
Rotary encoder
264
Rotary encoder
A rotary encoder, also called a shaft encoder, is an
electro-mechanical device that converts the angular position or motion
of a shaft or axle to an analog or digital code. The output of
incremental encoders provides information about the motion of the
shaft which is typically further processed elsewhere into information
such as speed, distance, RPM and position. The output of absolute
encoders indicates the current position of the shaft, making them angle
transducers. Rotary encoders are used in many applications that require
precise shaft unlimited rotation—including industrial controls,
robotics, special purpose photographic lenses,[1] computer input
devices (such as optomechanical mice and trackballs), controlled stress
rheometers, and rotating radar platforms. There are two main types:
absolute and incremental (relative).
A Gray code absolute rotary encoder with 13
tracks. At the top can be seen the housing,
interrupter disk, and light source; at the bottom
can be seen the sensing element and support
components.
Absolute rotary encoder
Construction
Digital absolute encoders produce a unique digital code for each
distinct angle of the shaft. They come in two basic types: optical and
mechanical.
Mechanical absolute encoders
A metal disc containing a set of concentric rings of openings is fixed to
an insulating disc, which is rigidly fixed to the shaft. A row of sliding
contacts is fixed to a stationary object so that each contact wipes
against the metal disc at a different distance from the shaft. As the disc
rotates with the shaft, some of the contacts touch metal, while others
Absolute rotary encoder ROD 425
Rotary encoder
265
fall in the gaps where the metal has been cut out. The metal sheet is connected to a source of electric current, and
each contact is connected to a separate electrical sensor. The metal pattern is designed so that each possible position
of the axle creates a unique binary code in which some of the contacts are connected to the current source (i.e.
switched on) and others are not (i.e. switched off).
Optical absolute encoders
The optical encoder's disc is made of glass or plastic with transparent and opaque areas. A light source and photo
detector array reads the optical pattern that results from the disc's position at any one time.
This code can be read by a controlling device, such as a microprocessor or microcontroller to determine the angle of
the shaft.
The absolute analog type produces a unique dual analog code that can be translated into an absolute angle of the
shaft (by using a special algorithm).
Standard binary encoding
An example of a binary code, in an extremely simplified encoder with
only three contacts, is shown below.
Rotary encoder for angle-measuring devices
marked in 3-bit binary. The inner ring
corresponds to Contact 1 in the table. Black
sectors are "on". Zero degrees is on the right-hand
side, with angle increasing counterclockwise.
Standard Binary Encoding
Sector Contact 1 Contact 2 Contact 3
Angle
1
off
off
off
0° to 45°
2
off
off
ON
45° to 90°
3
off
ON
off
90° to 135°
4
off
ON
ON
135° to 180°
5
ON
off
off
180° to 225°
6
ON
off
ON
225° to 270°
7
ON
ON
off
270° to 315°
8
ON
ON
ON
315° to 360°
Rotary encoder
266
In general, where there are n contacts, the number of distinct positions of the shaft is 2n. In this example, n is 3, so
there are 2³ or 8 positions.
In the above example, the contacts produce a standard binary count as the disc rotates. However, this has the
drawback that if the disc stops between two adjacent sectors, or the contacts are not perfectly aligned, it can be
impossible to determine the angle of the shaft. To illustrate this problem, consider what happens when the shaft
angle changes from 179.9° to 180.1° (from sector 4 to sector 5). At some instant, according to the above table, the
contact pattern changes from off-on-on to on-off-off. However, this is not what happens in reality. In a practical
device, the contacts are never perfectly aligned, so each switches at a different moment. If contact 1 switches first,
followed by contact 3 and then contact 2, for example, the actual sequence of codes is:
off-on-on (starting position)
on-on-on (first, contact 1 switches on)
on-on-off (next, contact 3 switches off)
on-off-off (finally, contact 2 switches off)
Now look at the sectors corresponding to these codes in the table. In order, they are 4, 8, 7 and then 5. So, from the
sequence of codes produced, the shaft appears to have jumped from sector 4 to sector 8, then gone backwards to
sector 7, then backwards again to sector 5, which is where we expected to find it. In many situations, this behaviour
is undesirable and could cause the system to fail. For example, if the encoder were used in a robot arm, the controller
would think that the arm was in the wrong position, and try to correct the error by turning it through 180°, perhaps
causing damage to the arm.
Gray encoding
To avoid the above problem, Gray encoding is used. This is a system
of binary counting in which adjacent codes differ in only one position.
For the three-contact example given above, the Gray-coded version
would be as follows.
Rotary encoder for angle-measuring devices
marked in 3-bit binary-reflected Gray code
(BRGC). The inner ring corresponds to Contact 1
in the table. Black sectors are "on". Zero degrees
is on the right-hand side, with angle increasing
anticlockwise.
Rotary encoder
267
Gray Coding
Sector Contact 1 Contact 2 Contact 3
Angle
1
off
off
off
0° to 45°
2
off
off
ON
45° to 90°
3
off
ON
ON
90° to 135°
4
off
ON
off
135° to 180°
5
ON
ON
off
180° to 225°
6
ON
ON
ON
225° to 270°
7
ON
off
ON
270° to 315°
8
ON
off
off
315° to 360°
In this example, the transition from sector 4 to sector 5, like all other transitions, involves only one of the contacts
changing its state from on to off or vice versa. This means that the sequence of incorrect codes shown in the previous
illustration cannot happen.
Single-track Gray encoding
If the designer moves a contact to a different angular position (but at the same distance from the center shaft), then
the corresponding "ring pattern" needs to be rotated the same angle to give the same output. If the most significant
bit (the inner ring in Figure 1) is rotated enough, it exactly matches the next ring out. Since both rings are then
identical, the inner ring can be omitted, and the sensor for that ring moved to the remaining, identical ring (but offset
at that angle from the other sensor on that ring). Those two sensors on a single ring make a quadrature encoder.
For many years, Torsten Sillke [2] and other mathematicians believed that it was impossible to encode position on a
single track so that consecutive positions differed at only a single sensor, except for the two-sensor, one-track
quadrature encoder. However, in 1994 N. B. Spedding registered a patent (NZ Patent 264738) showing it was
possible with several examples. See Single-track Gray code for details.
Absolute encoder output formats
In commercial absolute encoders there are several formats for transmission of absolute encoder data, including
parallel binary, SSI, "BiSS" [3], ISI, Profibus, DeviceNet, CANopen, Endat and Hiperface, depending on the
manufacturer of the device.
Incremental rotary encoder
Encoder ROD 420
An incremental rotary encoder provides cyclical outputs (only) when
the encoder is rotated. They can be either mechanical or optical. The
mechanical type requires debouncing and is typically used as digital
potentiometers on equipment including consumer devices. Most
modern home and car stereos use mechanical rotary encoders for
volume control. Due to the fact the mechanical switches require
debouncing, the mechanical type are limited in the rotational speeds
they can handle. The incremental rotary encoder is the most widely
used of all rotary encoders due to its low cost and ability to provide
signals that can be easily interpreted to provide motion related
information such as velocity and RPM.
Rotary encoder
268
The fact that incremental encoders use only two sensors does not compromise their accuracy. One can find in the
market incremental encoders with up to 10,000 counts per revolution, or more.
There can be an optional third output: reference, which happens once every turn. This is used when there is the need
of an absolute reference, such as positioning systems.
The optical type is used when higher RPMs are encountered or a higher degree of precision is required.
Incremental encoders are used to track motion and can be used to determine position and velocity. This can be either
linear or rotary motion. Because the direction can be determined, very accurate measurements can be made.
They employ two outputs called A & B, which are called quadrature outputs, as they are 90 degrees out of phase.
The state diagram:
Phase
A
B
Phase
A
B
1
0
0
1
1
0
2
0
1
2
1
1
3
1
1
3
0
1
4
1
0
4
0
0
The two output wave forms are 90 degrees
out of phase, which is all that the quadrature
term means. These signals are decoded to
produce a count up pulse or a count down
pulse. For decoding in software, the A & B
outputs are read by software, either via an
interrupt on any edge or polling, and the
Two square waves in quadrature (clockwise rotation).
above table is used to decode the direction.
For example, if the last value was 00 and the
current value is 01, the device has moved one half step in the clockwise direction. The mechanical types would be
debounced first by requiring that the same (valid) value be read a certain number of times before recognizing a state
change.
On encoders with detents there are different ways to switch states. In some, both A and B are always open circuit at
the detents, and an entire 00 → 00 switching cycle occurs while transitioning from one detent to the next. Others
have detents of alternating 00 and 11 value, with staggered switching times during the transition between detents.
If the encoder is turning too fast, an invalid transition may occur, such as 00 → 11. There is no way to know which
way the encoder turned; if it was 00 → 01 → 11, or 00 → 10 → 11.
If the encoder is turning even faster, a backward count may occur. Example: consider the 00 → 01 → 11 → 10
transition (3 steps forward). If the encoder is turning too fast, the system might read only the 00 and then the 10,
which yields a 00 → 10 transition (1 step backward).
This same principle is used in ball mice to track whether the mouse is moving to the right/left or forward/backward.
Rotary encoders with a single output cannot be used to sense direction of motion. They are well-suited for systems
that measure rate-of-movement variables such as velocity and RPM. In certain applications they may be used to
measure distance of motion (e.g. feet of movement).
Rotary encoder
Incremental versus absolute encoder terminology
There seem to be some grey areas as to what constitutes an incremental encoder as opposed to an absolute encoder.
Traditional absolute encoders
Traditional absolute encoders have multiple code rings with various binary weightings which provide a data word
representing the absolute position of the encoder within one revolution. This type of encoder is often referred to as a
parallel absolute encoder. The distinguishing feature of the absolute encoder is that it reports the absolute position of
the encoder to the electronics immediately upon power-up with no need for indexing.[4]
Traditional incremental encoders
A traditional incremental encoder works differently by providing an A and a B pulse output that provide no usable
count information in their own right. Rather, the counting is done in the external electronics. The point where the
counting begins depends on the counter in the external electronics and not on the position of the encoder. To provide
useful position information, the encoder position must be referenced to the device to which it is attached, generally
using an index pulse. The distinguishing feature of the incremental encoder is that it reports an incremental change in
position of the encoder to the counting electronics.[4]
Battery backed incremental encoders
Some encoder manufacturers, such as Fanuc, have taken a different approach to this terminology. These
manufacturers use absolute as their terminology for incremental encoders with a battery backed up memory to store
count information and provide an absolute count immediately upon power up.[4]
Sine wave encoder
A variation on the Incremental encoder is the Sinewave Encoder. Instead of producing two quadrature square waves,
the outputs are quadrature sine waves (a Sine and a Cosine). By performing the arctangent function, arbitrary levels
of resolution can be achieved.
Use in industry
Encoders used on servomotors
Rotary encoders are often used to track the position of the motor shaft on permanent magnet brushless motors, which
are commonly used on CNC machines, robots, and other industrial equipment. Incremental (Quadrature) encoders
are used on Induction Motor type servomotors, but absolute encoders are used in Permanent Magnet Brushless
Motors, where applicable. In these applications, the feedback device (encoder) plays a vital role in ensuring that the
equipment operates properly. The encoder synchronizes the relative rotor magnet and stator winding positions to the
current provided by the drive. Maximum torque results if the current is applied to the windings when the rotor
magnets are in a particular position range relative to the stator windings. The motor will perform poorly or not at all
if this timing is not adjusted correctly. Improper encoder alignment on the motor can actually cause it to run
backwards sometimes resulting in a hazardous run away condition. Correct alignment is essential to proper operation
of these motors.[5]
269
Rotary encoder
270
Encoder technologies
Encoders may be implemented using a variety of technologies:
• Conductive tracks. A series of copper pads etched onto a PCB is
used to encode the information. Contact brushes sense the
conductive areas. This form of encoder is now rarely seen.
• Optical. This uses a light shining onto a photodiode through slits in
a metal or glass disc. Reflective versions also exist. This is one of
the most common technologies.
• Magnetic. Strips of magnetised material are placed on the rotating
disc and are sensed by a Hall-effect sensor or magnetoresistive
sensor. Hall effect sensors are also used to sense gear teeth directly,
without the need for a separate encoder disc.
References
[1] (http:/ / www. canon. com/ bctv/ faq/ rotary. html) Canon video camera lens, used
for zoom and aperture control
[2] http:/ / www. mathematik. uni-bielefeld. de/ ~sillke/ PROBLEMS/ gray
[3] http:/ / www. biss-interface. com
Hall-effect quadrature encoder, sensing gear teeth
on the driveshaft of a robot vehicle.
[4] TI-5000EX Serial/Incremental Encoder Test System User Manual (http:/ / www. mitchell-electronics. com/ downloads/ Catalog_PriceList/
TI5000EXManual. pdf), Mitchell Electronics, Inc.
[5] PM Brushless Servo Motor Feedback Commutation Series – Part 1 (http:/ / www. mitchell-electronics. com/ downloads/ AN5000-PD01.
pdf), Mitchell Electronics, Inc.
External links
• "Choosing a code wheel: A detailed look at how encoders work" (http://www.smallformfactors.com/articles/
id/?3039) article by Steve Trahey 2008-03-25 describes "rotary encoders".
• "Encoders provide a sense of place" (http://embedded.com/showArticle.jhtml?articleID=166400808) article by
Jack Ganssle 2005-07-19 describes "nonlinear encoders".
• "Robot Encoders" (http://www.societyofrobots.com/sensors_encoder.shtml).
• Introductory Tutorial (http://web.archive.org/web/20110717090211/http://www.embedded.com.au/pages/
Motor_Interface.html) on PWM and Quadrature Encoding
• ProtoTalk.net - Understanding Quadrature Encoding (http://prototalk.net/forums/showthread.php?t=78) Covers details of rotary and quadrature encoding with a focus on robotic applications
271
The Frequency-Response Design Method
Bode plot
A Bode plot is a graph of the transfer
function of a linear, time-invariant
system versus frequency, plotted with
a log-frequency axis, to show the
system's frequency response. It is
usually a combination of a Bode
magnitude plot, expressing the
magnitude of the frequency response
gain, and a Bode phase plot,
expressing the frequency response
phase shift.
Overview
Among
his
several
important
contributions to circuit theory and
control theory, engineer Hendrik Wade
Bode (1905–1982), while working at
Figure 1(a): The Bode plot for a first-order (one-pole) highpass filter; the straight-line
Bell Labs in the United States in the
approximations are labeled "Bode pole"; phase varies from 90° at low frequencies (due to
1930s, devised a simple but accurate
the contribution of the numerator, which is 90° at all frequencies) to 0° at high
frequencies (where the phase contribution of the denominator is −90° and cancels the
method for graphing gain and
contribution of the numerator).
phase-shift plots. These bear his name,
Bode gain plot and Bode phase plot
(pronounced Boh-dee in English, Bow-duh in Dutch).[1]
The magnitude axis of the Bode plot is usually expressed as decibels of power, that is by the 20 log rule: 20 times the
common (base 10) logarithm of the amplitude gain. With the magnitude gain being logarithmic, Bode plots make
multiplication of magnitudes a simple matter of adding distances on the graph (in decibels), since
Bode plot
272
Figure 1(b): The Bode plot for a first-order (one-pole) lowpass filter; the straight-line
approximations are labeled "Bode pole"; phase is 90° lower than for Figure 1(a) because
the phase contribution of the numerator is 0° at all frequencies.
A Bode phase plot is a graph of phase versus frequency, also plotted on a log-frequency axis, usually used in
conjunction with the magnitude plot, to evaluate how much a signal will be phase-shifted. For example a signal
described by: Asin(ωt) may be attenuated but also phase-shifted. If the system attenuates it by a factor x and phase
shifts it by −Φ the signal out of the system will be (A/x) sin(ωt − Φ). The phase shift Φ is generally a function of
frequency.
Phase can also be added directly from the graphical values, a fact that is mathematically clear when phase is seen as
the imaginary part of the complex logarithm of a complex gain.
In Figure 1(a), the Bode plots are shown for the one-pole highpass filter function:
where f is the frequency in Hz, and f1 is the pole position in Hz, f1 = 100 Hz in the figure. Using the rules for
complex numbers, the magnitude of this function is
while the phase is:
Care must be taken that the inverse tangent is set up to return degrees, not radians. On the Bode magnitude plot,
decibels are used, and the plotted magnitude is:
In Figure 1(b), the Bode plots are shown for the one-pole lowpass filter function:
Bode plot
273
Also shown in Figure 1(a) and 1(b) are the straight-line approximations to the Bode plots that are used in hand
analysis, and described later.
The magnitude and phase Bode plots can seldom be changed independently of each other — changing the amplitude
response of the system will most likely change the phase characteristics and vice versa. For minimum-phase systems
the phase and amplitude characteristics can be obtained from each other with the use of the Hilbert transform.
If the transfer function is a rational function with real poles and zeros, then the Bode plot can be approximated with
straight lines. These asymptotic approximations are called straight line Bode plots or uncorrected Bode plots and
are useful because they can be drawn by hand following a few simple rules. Simple plots can even be predicted
without drawing them.
The approximation can be taken further by correcting the value at each cutoff frequency. The plot is then called a
corrected Bode plot.
Rules for hand-made Bode plot
The premise of a Bode plot is that one can consider the log of a function in the form:
as a sum of the logs of its poles and zeros:
This idea is used explicitly in the method for drawing phase diagrams. The method for drawing amplitude plots
implicitly uses this idea, but since the log of the amplitude of each pole or zero always starts at zero and only has one
asymptote change (the straight lines), the method can be simplified.
Straight-line amplitude plot
Amplitude decibels is usually done using the
where
and
are constants,
version. Given a transfer function in the form
,
, and H is the transfer function:
• at every value of s where
(a zero), increase the slope of the line by
per decade.
• at every value of s where
(a pole), decrease the slope of the line by
per decade.
• The initial value of the graph depends on the boundaries. The initial point is found by putting the initial angular
frequency ω into the function and finding |H(jω)|.
• The initial slope of the function at the initial value depends on the number and order of zeros and poles that are at
values below the initial value, and are found using the first two rules.
To handle irreducible 2nd order polynomials,
can, in many cases, be approximated as
.
Note that zeros and poles happen when ω is equal to a certain
or
. This is because the function in question is
the magnitude of H(jω), and since it is a complex function,
is
a
zero
or
pole
involving
the
.
term
. Thus at any place where there
,
the
magnitude
of
that
term
is
Bode plot
274
Corrected amplitude plot
To correct a straight-line amplitude plot:
• at every zero, put a point
above the line,
• at every pole, put a point
below the line,
• draw a smooth curve through those points using the straight lines as asymptotes (lines which the curve
approaches).
Note that this correction method does not incorporate how to handle complex values of
or
. In the case of an
irreducible polynomial, the best way to correct the plot is to actually calculate the magnitude of the transfer function
at the pole or zero corresponding to the irreducible polynomial, and put that dot over or under the line at that pole or
zero.
Straight-line phase plot
Given a transfer function in the same form as above:
the idea is to draw separate plots for each pole and zero, then add them up. The actual phase curve is given by
.
To draw the phase plot, for each pole and zero:
• if A is positive, start line (with zero slope) at 0 degrees
• if A is negative, start line (with zero slope) at 180 degrees
• at every
(for stable zeros –
beginning one decade before
• at every
(E.g.:
(E.g.:
degrees per decade,
), decrease the slope by
degrees per decade,
)
(for stable poles –
beginning one decade before
), increase the slope by
)
• "unstable" (right half plane) poles and zeros (
• flatten the slope again when the phase has changed by
) have opposite behavior
degrees (for a zero) or
degrees (for a
pole),
• After plotting one line for each pole or zero, add the lines together to obtain the final phase plot; that is, the final
phase plot is the superposition of each earlier phase plot.
Example
A passive (unity pass band gain) lowpass RC filter, for instance has the following transfer function expressed in the
frequency domain:
From the transfer function it can be determined that the cutoff frequency point fc (in hertz) is at the frequency
or (equivalently) at
where
is the angular cutoff frequency in radians per second.
The transfer function in terms of the angular frequencies becomes:
Bode plot
275
The above equation is the normalized form of the transfer function. The Bode plot is shown in Figure 1(b) above,
and construction of the straight-line approximation is discussed next.
Magnitude plot
The magnitude (in decibels) of the transfer function above, (normalized and converted to angular frequency form),
given by the decibel gain expression
:
when plotted versus input frequency on a logarithmic scale, can be approximated by two lines and it forms the
asymptotic (approximate) magnitude Bode plot of the transfer function:
• for angular frequencies below
it is a horizontal line at 0 dB since at low frequencies the
term is small and
can be neglected, making the decibel gain equation above equal to zero,
• for angular frequencies above
it is a line with a slope of −20 dB per decade since at high frequencies the
term dominates and the decibel gain expression above simplifies to
which is a straight line with a
slope of −20 dB per decade.
These two lines meet at the corner frequency. From the plot, it can be seen that for frequencies well below the corner
frequency, the circuit has an attenuation of 0 dB, corresponding to a unity pass band gain, i.e. the amplitude of the
filter output equals the amplitude of the input. Frequencies above the corner frequency are attenuated – the higher
the frequency, the higher the attenuation.
Phase plot
The phase Bode plot is obtained by plotting the phase angle of the transfer function given by
versus
, where
and
lower than corner, the ratio
are the input and cutoff angular frequencies respectively. For input frequencies much
is small and therefore the phase angle is close to zero. As the ratio increases the
absolute value of the phase increases and becomes –45 degrees when
. As the ratio increases for input
frequencies much greater than the corner frequency, the phase angle asymptotically approaches −90 degrees. The
frequency scale for the phase plot is logarithmic.
Bode plot
276
Normalized plot
The horizontal frequency axis, in both the magnitude and phase plots, can be replaced by the normalized
(nondimensional) frequency ratio
. In such a case the plot is said to be normalized and units of the frequencies
are no longer used since all input frequencies are now expressed as multiples of the cutoff frequency
.
An example with pole and zero
Figures 2-5 further illustrate construction of Bode plots. This example with both a pole and a zero shows how to use
superposition. To begin, the components are presented separately.
Figure 2 shows the Bode magnitude plot for a zero and a low-pass pole, and compares the two with the Bode straight
line plots. The straight-line plots are horizontal up to the pole (zero) location and then drop (rise) at 20 dB/decade.
The second Figure 3 does the same for the phase. The phase plots are horizontal up to a frequency factor of ten
below the pole (zero) location and then drop (rise) at 45°/decade until the frequency is ten times higher than the pole
(zero) location. The plots then are again horizontal at higher frequencies at a final, total phase change of 90°.
Figure 4 and Figure 5 show how superposition (simple addition) of a pole and zero plot is done. The Bode straight
line plots again are compared with the exact plots. The zero has been moved to higher frequency than the pole to
make a more interesting example. Notice in Figure 4 that the 20 dB/decade drop of the pole is arrested by the 20
dB/decade rise of the zero resulting in a horizontal magnitude plot for frequencies above the zero location. Notice in
Figure 5 in the phase plot that the straight-line approximation is pretty approximate in the region where both pole
and zero affect the phase. Notice also in Figure 5 that the range of frequencies where the phase changes in the
straight line plot is limited to frequencies a factor of ten above and below the pole (zero) location. Where the phase
of the pole and the zero both are present, the straight-line phase plot is horizontal because the 45°/decade drop of the
pole is arrested by the overlapping 45°/decade rise of the zero in the limited range of frequencies where both are
active contributors to the phase.
Example with pole and zero
Figure 2: Bode magnitude plot for zero and low-pass pole; curves
labeled "Bode" are the straight-line Bode plots
Figure 3: Bode phase plot for zero and low-pass pole; curves labeled
"Bode" are the straight-line Bode plots
Bode plot
Figure 4: Bode magnitude plot for pole-zero combination; the location
of the zero is ten times higher than in Figures 2&3; curves labeled
"Bode" are the straight-line Bode plots
277
Figure 5: Bode phase plot for pole-zero combination; the location of
the zero is ten times higher than in Figures 2&3; curves labeled "Bode"
are the straight-line Bode plots
Gain margin and phase margin
Bode plots are used to assess the stability of negative feedback amplifiers by finding the gain and phase margins of
an amplifier. The notion of gain and phase margin is based upon the gain expression for a negative feedback
amplifier given by
where AFB is the gain of the amplifier with feedback (the closed-loop gain), β is the feedback factor and AOL is the
gain without feedback (the open-loop gain). The gain AOL is a complex function of frequency, with both magnitude
and phase.[2] Examination of this relation shows the possibility of infinite gain (interpreted as instability) if the
product βAOL = −1. (That is, the magnitude of βAOL is unity and its phase is −180°, the so-called Barkhausen
stability criterion). Bode plots are used to determine just how close an amplifier comes to satisfying this condition.
Key to this determination are two frequencies. The first, labeled here as f180, is the frequency where the open-loop
gain flips sign. The second, labeled here f0dB, is the frequency where the magnitude of the product | β AOL | = 1 (in
dB, magnitude 1 is 0 dB). That is, frequency f180 is determined by the condition:
where vertical bars denote the magnitude of a complex number (for example, | a + j b | = [ a2 + b2]1/2 ), and
frequency f0dB is determined by the condition:
One measure of proximity to instability is the gain margin. The Bode phase plot locates the frequency where the
phase of βAOL reaches −180°, denoted here as frequency f180. Using this frequency, the Bode magnitude plot finds
the magnitude of βAOL. If |βAOL|180 = 1, the amplifier is unstable, as mentioned. If |βAOL|180 < 1, instability does not
occur, and the separation in dB of the magnitude of |βAOL|180 from |βAOL| = 1 is called the gain margin. Because a
magnitude of one is 0 dB, the gain margin is simply one of the equivalent forms: 20 log10( |βAOL|180) = 20 log10(
|AOL|180) − 20 log10( 1 / β ).
Another equivalent measure of proximity to instability is the phase margin. The Bode magnitude plot locates the
frequency where the magnitude of |βAOL| reaches unity, denoted here as frequency f0dB. Using this frequency, the
Bode phase plot finds the phase of βAOL. If the phase of βAOL( f0dB) > −180°, the instability condition cannot be met
at any frequency (because its magnitude is going to be < 1 when f = f180), and the distance of the phase at f0dB in
degrees above −180° is called the phase margin.
If a simple yes or no on the stability issue is all that is needed, the amplifier is stable if f0dB < f180. This criterion is
sufficient to predict stability only for amplifiers satisfying some restrictions on their pole and zero positions
(minimum phase systems). Although these restrictions usually are met, if they are not another method must be used,
such as the Nyquist plot.[3] [4]
Bode plot
Examples using Bode plots
Figures 6 and 7 illustrate the gain behavior and terminology. For a three-pole amplifier, Figure 6 compares the Bode
plot for the gain without feedback (the open-loop gain) AOL with the gain with feedback AFB (the closed-loop gain).
See negative feedback amplifier for more detail.
In this example, AOL = 100 dB at low frequencies, and 1 / β = 58 dB. At low frequencies, AFB ≈ 58 dB as well.
Because the open-loop gain AOL is plotted and not the product β AOL, the condition AOL = 1 / β decides f0dB. The
feedback gain at low frequencies and for large AOL is AFB ≈ 1 / β (look at the formula for the feedback gain at the
beginning of this section for the case of large gain AOL), so an equivalent way to find f0dB is to look where the
feedback gain intersects the open-loop gain. (Frequency f0dB is needed later to find the phase margin.)
Near this crossover of the two gains at f0dB, the Barkhausen criteria are almost satisfied in this example, and the
feedback amplifier exhibits a massive peak in gain (it would be infinity if β AOL = −1). Beyond the unity gain
frequency f0dB, the open-loop gain is sufficiently small that AFB ≈ AOL (examine the formula at the beginning of this
section for the case of small AOL).
Figure 7 shows the corresponding phase comparison: the phase of the feedback amplifier is nearly zero out to the
frequency f180 where the open-loop gain has a phase of −180°. In this vicinity, the phase of the feedback amplifier
plunges abruptly downward to become almost the same as the phase of the open-loop amplifier. (Recall, AFB ≈ AOL
for small AOL.)
Comparing the labeled points in Figure 6 and Figure 7, it is seen that the unity gain frequency f0dB and the phase-flip
frequency f180 are very nearly equal in this amplifier, f180 ≈ f0dB ≈ 3.332 kHz, which means the gain margin and
phase margin are nearly zero. The amplifier is borderline stable.
Figures 8 and 9 illustrate the gain margin and phase margin for a different amount of feedback β. The feedback
factor is chosen smaller than in Figure 6 or 7, moving the condition | β AOL | = 1 to lower frequency. In this example,
1 / β = 77 dB, and at low frequencies AFB ≈ 77 dB as well.
Figure 8 shows the gain plot. From Figure 8, the intersection of 1 / β and AOL occurs at f0dB = 1 kHz. Notice that the
peak in the gain AFB near f0dB is almost gone.[5] [6]
Figure 9 is the phase plot. Using the value of f0dB = 1 kHz found above from the magnitude plot of Figure 8, the
open-loop phase at f0dB is −135°, which is a phase margin of 45° above −180°.
Using Figure 9, for a phase of −180° the value of f180 = 3.332 kHz (the same result as found earlier, of course[7] ).
The open-loop gain from Figure 8 at f180 is 58 dB, and 1 / β = 77 dB, so the gain margin is 19 dB.
Stability is not the sole criterion for amplifier response, and in many applications a more stringent demand than
stability is good step response. As a rule of thumb, good step response requires a phase margin of at least 45°, and
often a margin of over 70° is advocated, particularly where component variation due to manufacturing tolerances is
an issue.[8] See also the discussion of phase margin in the step response article.
278
Bode plot
279
Examples
Figure 6: Gain of feedback amplifier AFB in dB and corresponding
open-loop amplifier AOL. Parameter 1/β = 58 dB, and at low
frequencies AFB ≈ 58 dB as well. The gain margin in this amplifier is
nearly zero because | βAOL| = 1 occurs at almost f = f180°.
Figure 7: Phase of feedback amplifier °AFB in degrees and
corresponding open-loop amplifier °AOL. The phase margin in this
amplifier is nearly zero because the phase-flip occurs at almost the
unity gain frequency f = f0dB where | βAOL| = 1.
Figure 8: Gain of feedback amplifier AFB in dB and corresponding
open-loop amplifier AOL. In this example, 1 / β = 77 dB. The gain
margin in this amplifier is 19 dB.
Figure 9: Phase of feedback amplifier AFB in degrees and
corresponding open-loop amplifier AOL. The phase margin in this
amplifier is 45°.
Bode plotter
The Bode plotter is an electronic
instrument resembling an oscilloscope,
which produces a Bode diagram, or a
graph, of a circuit's voltage gain or
phase shift plotted against frequency in
a feedback control system or a filter.
An example of this is shown in Figure
10. It is extremely useful for analyzing
and testing filters and the stability of
feedback control systems, through the
measurement of corner (cutoff)
frequencies and gain and phase
margins.
Figure 10: Amplitude diagram of a 10th order Chebyshev filter plotted using a Bode
Plotter application. The chebyshev transfer function is defined by poles and zeros which
are added by clicking on a graphical complex diagram.
This is identical to the function
performed by a vector network analyzer, but the network analyzer is typically used at much higher frequencies.
For education/research purposes, plotting Bode diagrams for given transfer functions facilitates better understanding
and getting faster results (see external links).
Bode plot
280
Related plots
Two related plots that display the same data in different coordinate systems are the Nyquist plot and the Nichols plot.
These are parametric plots, with frequency as the input and magnitude and phase of the frequency response as the
output. The Nyquist plot displays these in polar coordinates, with magnitude mapping to radius and phase to
argument (angle). The Nichols plot displays these in rectangular coordinates, on the log scale.
Related Plots
A Nyquist plot.
A Nichols plot of the same response.
Notes
[1] Van Valkenburg, M. E. University of Illinois at Urbana-Champaign, "In memoriam: Hendrik W. Bode (1905-1982)", IEEE Transactions on
Automatic Control, Vol. AC-29, No 3., March 1984, pp. 193-194. Quote: "Something should be said about his name. To his colleagues at Bell
Laboratories and the generations of engineers that have followed, the pronunciation is boh-dee. The Bode family preferred that the original
Dutch be used as boh-dah."
[2] Ordinarily, as frequency increases the magnitude of the gain drops and the phase becomes more negative, although these are only trends and
may be reversed in particular frequency ranges. Unusual gain behavior can render the concepts of gain and phase margin inapplicable. Then
other methods such as the Nyquist plot have to be used to assess stability.
[3] Thomas H. Lee (2004). The design of CMOS radio-frequency integrated circuits (http:/ / worldcat. org/ isbn/ 0-521-83539-9) (Second Edition
ed.). Cambridge UK: Cambridge University Press. p. §14.6 pp. 451–453. ISBN 0-521-83539-9. .
[4] William S Levine (1996). The control handbook: the electrical engineering handbook series (http:/ / books. google. com/
books?id=2WQP5JGaJOgC& pg=RA1-PA163& lpg=RA1-PA163& dq=stability+ "minimum+ phase"& source=web& ots=P3fFTcyfzM&
sig=ad5DJ7EvVm6In_zhI0MlF_6vHDA) (Second Edition ed.). Boca Raton FL: CRC Press/IEEE Press. p. §10.1 p. 163. ISBN 0849385709. .
[5] The critical amount of feedback where the peak in the gain just disappears altogether is the maximally flat or Butterworth design.
[6] Willy M C Sansen (2006). Analog design essentials (http:/ / worldcat. org/ isbn/ 0-387-25746-2). Dordrecht, The Netherlands: Springer.
p. §0517-§0527 pp. 157–163. ISBN 0-387-25746-2. .
[7] The frequency where the open-loop gain flips sign f180 does not change with a change in feedback factor; it is a property of the open-loop
gain. The value of the gain at f180 also does not change with a change in β. Therefore, we could use the previous values from Figures 6 and 7.
However, for clarity the procedure is described using only Figures 8 and 9.
[8] Willy M C Sansen. §0526 p. 162 (http:/ / worldcat. org/ isbn/ 0-387-25746-2). ISBN 0-387-25746-2. .
Bode plot
References
External links
• Explanation of Bode plots with movies and examples (http://www.facstaff.bucknell.edu/mastascu/
eControlHTML/Freq/Freq5.html)
• How to draw piecewise asymptotic Bode plots (http://lpsa.swarthmore.edu/Bode/BodeHow.html)
• Summarized drawing rules (http://lims.mech.northwestern.edu/~lynch/courses/ME391/2003/bodesketching.
pdf) (PDF)
• Bode plot applet (http://www.uwm.edu/People/msw/BodePlot/) - Accepts transfer function coefficients as
input, and calculates magnitude and phase response
• Circuit analysis in electrochemistry (http://www.abc.chemistry.bsu.by/vi/fit.htm)
• Tim Green: Operational amplifier stability (http://www.en-genius.net/includes/files/acqt_013105.pdf)
Includes some Bode plot introduction
• Gnuplot code for generating Bode plot: DIN-A4 printing template (pdf)
Nyquist stability criterion
When designing a feedback control system,
it is generally necessary to determine
whether the closed-loop system will be
stable. An example of a destabilizing
feedback control system would be a car
steering system that overcompensates -- if
the car drifts in one direction, the control
system overcorrects in the opposite
direction, and even further back in the first,
until the car goes off the road. In contrast,
for a stable system the vehicle would
continue to track the control input. The
Nyquist stability criterion, named after
Harry Nyquist, is a graphical technique for
determining the stability of a system.
Because it only looks at the Nyquist plot of
the open loop systems, it can be applied
without explicitly computing the poles and
zeros of either the closed-loop or open-loop
The Nyquist plot for
.
system (although the number of each type of
right-half-plane singularities must be
known). As a result, it can be applied to systems defined by non-rational functions, such as systems with delays. In
contrast to Bode plots, it can handle transfer functions with right half-plane singularities. In addition, there is a
natural generalization to more complex systems with multiple inputs and multiple outputs, such as control systems
for airplanes.
While Nyquist is one of the most general stability tests, it is still restricted to linear, time-invariant systems.
Non-linear systems must use more complex stability criteria, such as Lyapunov. While Nyquist is a graphical
technique, it only provides a limited amount of intuition for why a system is stable or unstable, or how to modify an
281
Nyquist stability criterion
282
unstable system to be stable. Techniques like Bode plots, while less general, are sometimes a more useful design
tool.
Background
We consider a system whose open loop transfer function (OLTF) is
feedback
; when placed in a closed loop with
, the closed loop transfer function (CLTF) then becomes
. The case where H=1 is usually
taken, when investigating stability, and then the characteristic equation, used to predict stability, becomes
. Stability can be determined by examining the roots of this equation e.g. using the Routh array, but
this method is somewhat tedious. Conclusions can also be reached by examining the OLTF, using its Bode plots or,
as here, polar plot of the OLTF using the Nyquist criterion, as follows.
Any Laplace domain transfer function
can be expressed as the ratio of two polynomials
We define:
• Zero: the zeros of
are the roots of
• Pole: the poles of
are the roots of
Stability of
, and
.
is determined by its poles or simply the roots of the characteristic equation:
stability, the real part of every pole must be negative. If
around the open-loop transfer function
zeros of
. For
is formed by closing a negative unity feedback loop
, then the roots of the characteristic equation are also the
, or simply the roots of
.
Cauchy's argument principle
From complex analysis, specifically the argument principle, we know that a contour
drawn in the complex
plane, encompassing but not passing through any number of zeros and poles of a function
another plane (the
plane
plane) by the function
times, where
inside the contour
.
. The resulting contour
and
, can be mapped to
will encircle the origin of the
are respectively the number of zeros and poles of
. Note that we count encirclements in the
plane in the same sense as the contour
and that encirclements in the opposite direction are negative encirclements.
Instead of Cauchy's argument principle, the original paper by Harry Nyquist in 1932 uses a less elegant approach.
The approach explained here is similar to the approach used by Leroy MacColl (Fundamental theory of
servomechanisms 1945) or by Hendrik Bode (Network analysis and feedback amplifier design 1945), both of whom
also worked for Bell Laboratories. This approach appears in most modern textbooks on control theory.
The Nyquist criterion
We first construct The Nyquist Contour, a contour that encompasses the right-half of the complex plane:
• a path traveling up the
axis, from
• a semicircular arc, with radius
to
, that starts at
The Nyquist Contour mapped through the function
.
and travels clock-wise to
yields a plot of
.
in the complex plane. By
the Argument Principle, the number of clock-wise encirclements of the origin must be the number of zeros of
in the right-half complex plane minus the poles of
the contour is mapped through the open-loop transfer function
in the right-half complex plane. If instead,
, the result is the Nyquist Plot of
. By
counting the resulting contour's encirclements of -1, we find the difference between the number of poles and zeros in
Nyquist stability criterion
283
the right-half complex plane of
the poles of
are same as the poles of
Given a Nyquist contour
zeros of
. Recalling that the zeros of
, let
encircled by
are the poles of the closed-loop system, and noti
, we now state The Nyquist Criterion:
be the number of poles of
encircled by
. Alternatively, and more importantly,
loop system in the right half plane. The resultant contour in the
, and
be the number of
is the number of poles of the closed
-plane,
shall encircle (clock-wise) the
point
times such that
.
If the system is originally open-loop unstable, feedback is necessary to stabilize the system. Right-half-plane (RHP)
poles represent that instability. For closed-loop stability of a system, the number of closed-loop roots in the right half
of the s-plane must be zero. Hence, the number of counter-clockwise encirclements about
must be equal
to the number of open-loop poles in the RHP. Any clockwise encirclements of the critical point by the open-loop
frequency response (when judged from low frequency to high frequency) would indicate that the feedback control
system would be destabilizing if the loop were closed. (Using RHP zeros to "cancel out" RHP poles does not remove
the instability, but rather ensures that the system will remain unstable even in the presence of feedback, since the
closed-loop roots travel between open-loop poles and zeros in the presence of feedback. In fact, the RHP zero can
make the unstable pole unobservable and therefor not stabilizable through feedback.)
The Nyquist criterion for systems with poles on the imaginary axis
The above consideration was conducted with an assumption that the open-loop transfer function
have any pole on the imaginary axis (i.e. poles of the form
does not
). This results from the requirement of the
argument principle that the contour cannot pass through any pole of the mapping function. The most common case
are systems with integrators (poles at zero).
To be able to analyze systems with poles on the imaginary axis, the Nyquist Contour can be modified to avoid
passing through the point
. One way to do it is to construct a semicircular arc with radius
around
, that starts at
that the phasor
and travels anticlockwise to
travels along an arc of infinite radius by
. Such a modification implies
, where
is the multiplicity of the pole on the
imaginary axis.
Summary
• If the open-loop transfer function
discontinuity at
has a zero pole of multiplicity
, then the Nyquist plot has a
. During further analysis it should be assumed that the phasor travels
times clock-wise
along a semicircle of infinite radius. After applying this rule, the zero poles should be neglected, i.e. if there are
no other unstable poles, then the open-loop transfer function
should be considered stable.
• If the open-loop transfer function
is stable, then the closed-loop system is unstable for any encirclement of
the point -1.
• If the open-loop transfer function
is unstable, then there must be one counter clock-wise encirclement of -1
for each pole of
in the right-half of the complex plane.
• The number of surplus encirclements (greater than N+P) is exactly the number of unstable poles of the
closed-loop system
• However, if the graph happens to pass through the point
, then deciding upon even the marginal
stability of the system becomes difficult and the only conclusion that can be drawn from the graph is that there
exist zeros on the
axis.
Nyquist stability criterion
284
References
•
•
•
•
Faulkner, E.A. (1969): Introduction to the Theory of Linear Systems; Chapman & Hall; ISBN 0-412-09400-2
Pippard, A.B. (1985): Response & Stability; Cambridge University Press; ISBN 0-521-31994-3
Gessing, R. (2004): Control fundamentals; Silesian University of Technology; ISBN 83-7335-176-0
Franklin, G. (2002): Feedback Control of Dynamic Systems; Prentice Hall, ISBN 0-130-32393-4
Instability
In numerous fields of study, the component of instability within a
system is generally characterized by some of the outputs or internal
states growing without bounds. Not all systems that are not stable are
unstable; systems can also be marginally stable or exhibit limit cycle
behavior.
In control theory, a system is unstable if any of the roots of its
A ball on the top of a hill is an unstable situation.
characteristic equation has real part greater than zero (or if zero is a
repeated root). This is equivalent to any of the eigenvalues of the state
matrix having either real part greater than zero, or, for the eigenvalues on the imaginary axis, the algebraic
multiplicity being larger than the geometric multiplicity.
In structural engineering, a structure can become unstable when excessive load is applied. Beyond a certain
threshold, structural deflections magnify stresses, which in turn increases deflections. This can take the form of
buckling or crippling. The general field of study is called structural stability.
Atmospheric instability is a major component of all weather systems on Earth.
Fluid instabilities
Fluid instabilities occur in liquids, gases and plasmas, and are often
characterized by the shape that form; they are studied in fluid dynamics
and magnetohydrodynamics. Fluid instabilities include:
• Ballooning mode instability (some analogy to the Rayleigh–Taylor
instability); found in the magnetosphere
• Atmospheric instability
• Hydrodynamic instability or dynamic instability (atmospheric
dynamics)
• Inertial instability; baroclinic instability; symmetric
instability, conditional symmetric or convective symmetric
instability; barotropic instability; Helmholtz or shearing
instability; rotational instability
Hydrodynamics simulation of the
[1]
Rayleigh–Taylor instability
• Hydrostatic instability or static instability/vertical instability (parcel instability), thermodynamic instability
(atmospheric thermodynamics)
• Conditional or static instability, buoyant instability, latent instability, nonlocal static instability,
conditional-symmetric
Instability
285
instability; convective, potential, or thermal instability,
convective instability of the first and second kind; absolute or
mechanical instability
• Bénard instability
• Drift mirror instability
• Kelvin–Helmholtz instability (similar, but different from the
diocotron instability in plasmas)
• Rayleigh–Taylor instability
• Plateau-Rayleigh instability (similar to the Rayleigh–Taylor
instability)
• Richtmyer-Meshkov instability (similar to the Rayleigh–Taylor
instability)
Plasma instabilities
Plasma instabilities can be divided into two general groups (1)
hydrodynamic instabilities (2) kinetic instabilities. Plasma instabilities
are also categorised into different modes - see this paragraph in plasma
stability.
Unstable flow structure generated from the
collision of two impinging jets.
Instabilities of stellar systems
Galaxies and star clusters can be unstable, if small perturbations in the gravitational potential cause changes in the
density that reinforce the original perturbation. Such instabilities usually require that the motions of stars be highly
correlated, so that the perturbation is not "smeared out" by random motions. After the instability has run its course,
the system is typically "hotter" (the motions are more random) or rounder than before. Instabilities in stellar systems
include:
• Bar instability of rapidly-rotating disks
• Jeans instability
•
•
•
•
Firehose instability[2]
Gravothermal instability
Radial-orbit instability
Various instabilities in cold rotating disks
Notes
[1] Shengtai Li, Hui Li "Parallel AMR Code for Compressible MHD or HD Equations" (Los Alamos National Laboratory)
(http:/ / math. lanl. gov/ Research/ Highlights/ amrmhd. shtml)
[2] Merritt, D.; Sellwood, J. (1994), "Bending Instabilities of Stellar Systems", The Astrophysical Journal 425: 551–567,
Bibcode 1994ApJ...425..551M, doi:10.1086/174005
External links
• eFluids Fluid Flow Image Gallery (http://www.efluids.com/efluids/pages/gallery.htm)
Frequency response
286
Frequency response
Frequency response is the quantitative measure of the output spectrum of a system or device in response to a
stimulus, and is used to characterize the dynamics of the system. It is a measure of magnitude and phase of the
output as a function of frequency, in comparison to the input. In simplest terms, if a sine wave is injected into a
system at a given frequency, a linear system will respond at that same frequency with a certain magnitude and a
certain phase angle relative to the input. Also for a linear system, doubling the amplitude of the input will double the
amplitude of the output. In addition, if the system is time-invariant, then the frequency response also will not vary
with time.
Two applications of frequency response analysis are related but have different objectives. For an audio system, the
objective may be to reproduce the input signal with no distortion. That would require a uniform (flat) magnitude of
response up to the bandwidth limitation of the system, with the signal delayed by precisely the same amount of time
at all frequencies. That amount of time could be seconds, or weeks or months in the case of recorded media. In
contrast, for a feedback apparatus used to control a dynamical system, the objective is to give the closed-loop system
improved response as compared to the uncompensated system. The feedback generally needs to respond to system
dynamics within a very small number of cycles of oscillation (usually less than one full cycle), and with a definite
phase angle relative to the commanded control input. For feedback of sufficient amplification, getting the phase
angle wrong can lead to instability for an open-loop stable system, or failure to stabilize a system that is open-loop
unstable. Digital filters may be used for both audio systems and feedback control systems, but since the objectives
are different, generally the phase characteristics of the filters will be significantly different for the two applications.
Estimation and plotting
Estimating the frequency response for a
physical system generally involves exciting
the system with an input signal, measuring
both input and output time histories, and
comparing the two through a process such
as the Fast Fourier Transform (FFT). One
thing to keep in mind for the analysis is that
the frequency content of the input signal
must cover the frequency range of interest
or the results will not be valid for the
portion of the frequency range not covered.
The frequency response of a system can be
measured by applying a test signal, for
example:
Frequency response of a low pass filter with 6 dB per octave or 20 dB per decade
• applying an impulse to the system and
measuring its response (see impulse response)
• sweeping a constant-amplitude pure tone through the bandwidth of interest and measuring the output level and
phase shift relative to the input
• applying a signal with a wide frequency spectrum (for example digitally-generated maximum length sequence
noise, or analog filtered white noise equivalent, like pink noise), and calculating the impulse response by
deconvolution of this input signal and the output signal of the system.
The frequency response is characterized by the magnitude of the system's response, typically measured in decibels
(dB) or as a decimal, and the phase, measured in radians or degrees, versus frequency in radians/sec or Hertz (Hz).
Frequency response
These response measurements can be plotted in three ways: by plotting the magnitude and phase measurements on
two rectangular plots as functions of frequency to obtain a Bode plot; by plotting the magnitude and phase angle on a
single polar plot with frequency as a parameter to obtain a Nyquist plot; or by plotting magnitude and phase on a
single rectangular plot with frequency as a parameter to obtain a Nichols plot.
For audio systems with nearly uniform time delay at all frequencies, the magnitude versus frequency portion of the
Bode plot may be all that is of interest. For design of control systems, any of the three types of plots [Bode, Nyquist,
Nichols] can be used to infer closed-loop stability and stability margins (gain and phase margins) from the open-loop
frequency response, provided that for the Bode analysis the phase-versus-frequency plot is included.
Applications
In electronics this stimulus would be an input signal.[1] In the audible range it is usually referred to in connection
with electronic amplifiers, microphones and loudspeakers. Radio spectrum frequency response can refer to
measurements of coaxial cable, twisted-pair cable, video switching equipment, wireless communications devices,
and antenna systems. Infrasonic frequency response measurements include earthquakes and electroencephalography
(brain waves).
Frequency response requirements differ depending on the application.[2] In high fidelity audio, an amplifier requires
a frequency response of at least 20–20,000 Hz, with a tolerance as tight as ±0.1 dB in the mid-range frequencies
around 1000 Hz, however, in telephony, a frequency response of 400–4,000 Hz, with a tolerance of ±1 dB is
sufficient for intelligibility of speech.[2]
Frequency response curves are often used to indicate the accuracy of electronic components or systems.[1] When a
system or component reproduces all desired input signals with no emphasis or attenuation of a particular frequency
band, the system or component is said to be "flat", or to have a flat frequency response curve.[1]
Once a frequency response has been measured (e.g., as an impulse response), providing the system is linear and
time-invariant, its characteristic can be approximated with arbitrary accuracy by a digital filter. Similarly, if a system
is demonstrated to have a poor frequency response, a digital or analog filter can be applied to the signals prior to
their reproduction to compensate for these deficiencies.
References
Notes
[1] Stark, 2002, p. 51.
[2] Luther, 1999, p. 141.
Bibliography
• Luther, Arch C.; Inglis, Andrew F. Video engineering (http://books.google.com/books?id=VRailj6TKqUC),
McGraw-Hill, 1999. ISBN 0-07-135017-9
• Stark, Scott Hunter. Live Sound Reinforcement (http://books.google.com/books?id=7QOcDeGFx4UC),
Vallejo, California, Artistpro.com, 1996–2002. ISBN 0-918371-07-4
287
Frequency response
288
External links
• University of Michigan: Frequency Response Analysis and Design Tutorial (http://www.engin.umich.edu/
group/ctm/freq/freq.html)
• Smith, Julius O. III: Introduction to Digital Filters with Audio Applications (http://ccrma.stanford.edu/~jos/
filters/) has a nice chapter on Frequency Response (http://ccrma.stanford.edu/~jos/filters/
Frequency_Response_I.html)
Nyquist plot
A Nyquist plot is a parametric plot of a transfer function used in
automatic control and signal processing. The most common use of
Nyquist plots is for assessing the stability of a system with feedback. In
Cartesian coordinates, the real part of the transfer function is plotted on
the X axis. The imaginary part is plotted on the Y axis. The frequency
is swept as a parameter. Alternatively, in polar coordinates, the gain of
the transfer function is plotted as the radial coordinate, while the phase
of the transfer function is plotted as the angular coordinate. The
Nyquist plot is named after Harry Nyquist, a former engineer at Bell
Laboratories.
A Nyquist plot.
Uses
Assessment of the stability of a closed-loop negative feedback system is done by applying the Nyquist stability
criterion to the Nyquist plot of the open-loop system (i.e. the same system without its feedback loop). This method is
easily applicable even for systems with delays and other non-rational transfer functions, which may appear difficult
to analyze by means of other methods. Stability is determined by looking at the number of encirclements of the point
at (-1,0). Range of gains over which the system will be stable can be determined by looking at crossing of the real
axis.
The Nyquist plot can provide some information about the shape of the transfer function. For instance, the plot
provides information on the difference between the number of poles and zeros of the transfer function[1] by the angle
at which the curve approaches the origin.
When drawn by hand, a cartoon version of the Nyquist plot is sometimes used, which shows the shape of the curve,
but where coordinates are distorted to show more detail in regions of interest. When plotted computationally, one
needs to be careful to cover all frequencies of interest. This typically means that the parameter is swept
logarithmically, in order to cover a wide range of values.
Nyquist plot
289
References
[1] Nyquist Plots (http:/ / www. facstaff. bucknell. edu/ mastascu/ econtrolhtml/ Freq/ Freq6. html)
External links
• Applets with modifiable parameters (http://controlcan.homestead.com/files/idxpages.htm)
• EIS Spectrum Analyser - a freeware program for analysis and simulation of impedance spectra (http://www.abc.
chemistry.bsu.by/vi/analyser/)
Nichols plot
The Nichols plot is a plot used in signal processing and control design
[1] [2] [3]
.
Use in Control Design
Given a transfer function,
A Nichols plot.
with the closed-loop transfer function defined as,
the Nichols plots displays
versus
. Loci of constant
and
are overlaid to allow the designer to obtain the closed loop transfer function directly from the open
loop transfer function. Thus, the frequency is the parameter along the curve. This plot may be compared to the
Bode plot in which the two inter-related graphs versus
and
versus
) - are plotted. Note that the decibel convention shown above,
is not unique; another
convention sometimes used is
.
In feedback control design, the plot is useful for assessing the stability and robustness of a linear system. This
application of the Nichols plot is central to the Quantitative feedback theory (QFT) of Horowitz and Sidi, which is a
well known method for robust control system design.
Nichols plot
290
References
[1] Isaac M. Howowitz, Synthesis of Feedback Systems, Academic Press, 1963, Lib Congress 63-12033 p. 194-198
[2] Boris J. Lurie and Paul J. Enright, Classical Feedback Control, Marcel Dekker, 2000, ISBN 0-8247-0370-7 p. 10
[3] Allen Stubberud, Ivan Williams, and Joseph DeStefano, Shaums Outline Feedback and Control Systems, McGraw-Hill, 1995, ISBN-10:
0070170525 ch. 17
Bandwidth (signal processing)
Bandwidth is the difference between the
upper and lower frequencies in a contiguous
set of frequencies. It is typically measured in
hertz, and may sometimes refer to passband
bandwidth,
sometimes
to
baseband
bandwidth, depending on context. Passband
bandwidth is the difference between the
upper and lower cutoff frequencies of, for
example,
an
electronic
filter,
a
communication channel, or a signal
spectrum. In case of a low-pass filter or
baseband signal, the bandwidth is equal to
its upper cutoff frequency. The term
baseband bandwidth always refers to the
upper cutoff frequency, regardless of
whether the filter is bandpass or low-pass.
Baseband bandwidth. Here the bandwidth equals the upper frequency.
Bandwidth in hertz is a central concept in many fields, including electronics, information theory, radio
communications, signal processing, and spectroscopy. A key characteristic of bandwidth is that a band of a given
width can carry the same amount of information, regardless of where that band is located in the frequency spectrum
(assuming equivalent noise level). For example, a 5 kHz band can carry a telephone conversation whether that band
is at baseband (as in your POTS telephone line) or modulated to some higher (passband) frequency.
In computer networking and other digital fields, the term bandwidth often refers to a data rate measured in bits per
second, for example network throughput, sometimes denoted network bandwidth, data bandwidth or digital
bandwidth. The reason is that according to Hartley's law, the digital data rate limit (or channel capacity) of a physical
communication link is proportional to its bandwidth in hertz, sometimes denoted radio frequency (RF) bandwidth,
signal bandwidth, frequency bandwidth, spectral bandwidth or analog bandwidth. For bandwidth as a
computing term, less ambiguous terms are bit rate, throughput, maximum throughput, goodput or channel capacity.
Overview
Bandwidth is a key concept in many telephony applications. In radio communications, for example, bandwidth is the
frequency range occupied by a modulated carrier wave, whereas in optics it is the width of an individual spectral line
or the entire spectral range.
In many signal processing contexts, bandwidth is a valuable and limited resource. For example, an FM radio
receiver's tuner spans a limited range of frequencies. A government agency (such as the Federal Communications
Commission in the United States) may apportion the regionally available bandwidth to broadcast license holders so
that their signals do not mutually interfere. Each transmitter owns a slice of bandwidth, a valuable (if intangible)
commodity.
Bandwidth (signal processing)
For different applications there are different precise definitions. For example, one definition of bandwidth could be
the range of frequencies beyond which the frequency function is zero. This would correspond to the mathematical
notion of the support of a function (i.e., the total "length" of values for which the function is nonzero). A less strict
and more practically useful definition will refer to the frequencies where the frequency function is small. Small
could mean less than 3 dB below (i.e., power output < 1/2 or voltage output <
0.707 of) the maximum
value, or more rarely 10 dB below, or it could mean below a certain absolute value. As with any definition of the
width of a function, many definitions are suitable for different purposes.
Bandwidth typically refers to baseband bandwidth in the context of, for example, sampling theorem and Nyquist
sampling rate, while it refers to passband bandwidth in the context of Nyquist symbol rate or Shannon-Hartley
channel capacity for communication systems.
X-dB bandwidth
In some contexts, the signal bandwidth in
hertz refers to the frequency range in which
the signal's spectral density is nonzero or
above a small threshold value. That
definition is used in calculations of the
lowest sampling rate that will satisfy the
sampling theorem. Because this range of
non-zero amplitude may be very broad or
infinite, this definition is typically relaxed
so that the bandwidth is defined as the range
of frequencies in which the signal's spectral
density is above a certain threshold relative
to its maximum. Most commonly,
A graph of a bandpass filter's gain magnitude, illustrating the concept of −3 dB
bandwidth refers to the 3-dB bandwidth,
bandwidth at a gain of 0.707. The frequency axis of this symbolic diagram can be
that is, the frequency range within which the
linear or logarithmically scaled.
spectral density (in W/Hz or V2/Hz) is
above half its maximum value (or the spectral amplitude, in V or V/Hz, is more than 70.7% of its maximum); that is,
above −3 dB relative to the peak.[1]
The word bandwidth applies to signals as described above, but it could also apply to systems, for example filters or
communication channels. To say that a system has a certain bandwidth means that the system can process signals of
that bandwidth, or that the system reduces the bandwidth of a white noise input to that bandwidth.
The 3 dB bandwidth of an electronic filter or communication channel is the part of the system's frequency response
that lies within 3 dB of the response at its peak, which in the passband filter case is typically at or near its center
frequency, and in the lowpass filter is near 0 hertz. If the maximum gain is 0 dB, the 3 dB gain is the range where the
gain is more than -3dB, or the attenuation is less than + 3dB. This is also the range of frequencies where the
amplitude gain is above 70.7% of the maximum amplitude gain, and above half the maximum power gain. This same
"half power gain" convention is also used in spectral width, and more generally for extent of functions as full width
at half maximum (FWHM).
In electronic filter design, a filter specification may require that within the filter passband, the gain is nominally 0 dB
+/- a small number of dB, for example within the +/- 1 dB interval. In the stopband(s), the required attenuation in dB
is above a certain level, for example >100 dB. In a transition band the gain is not specified. In this case, the filter
bandwidth corresponds to the passband width, which in this example is the 1dB-bandwidth. If the filter shows
amplitude ripple within the passband, the x dB point refers to the point where the gain is x dB below the nominal
291
Bandwidth (signal processing)
292
passband gain rather than x dB below the maximum gain.
A commonly used quantity is fractional bandwidth. This is the bandwidth of a device divided by its center
frequency. E.g., a passband filter that has a bandwidth of 2 MHz with center frequency 10 MHz will have a
fractional bandwidth of 2/10, or 20%.
In communication systems, in calculations of the Shannon–Hartley channel capacity, bandwidth refers to the
3dB-bandwidth. In calculations of the maximum symbol rate, the Nyquist sampling rate, and maximum bit rate
according to the Hartley formula, the bandwidth refers to the frequency range within which the gain is non-zero, or
the gain in dB is below a very large value.
The fact that in equivalent baseband models of communication systems, the signal spectrum consists of both
negative and positive frequencies, can lead to confusion about bandwidth, since they are sometimes referred to only
by the positive half, and one will occasionally see expressions such as
, where
is the total bandwidth
(i.e. the maximum passband bandwidth of the carrier-modulated RF signal and the minimum passband bandwidth of
the physical passband channel), and
is the positive bandwidth (the baseband bandwidth of the equivalent channel
model). For instance, the baseband model of the signal would require a lowpass filter with cutoff frequency of at
least
to stay intact, and the physical passband channel would require a passband filter of at least
to stay
intact.
In signal processing and control theory the bandwidth is the frequency at which the closed-loop system gain drops
3 dB below peak.
In basic electric circuit theory, when studying band-pass and band-reject filters, the bandwidth represents the
distance between the two points in the frequency domain where the signal is
of the maximum signal amplitude
(half power).
Antenna systems
In the field of antennas, two different methods of expressing relative bandwidth are used for narrowband and
wideband antennas.[2] For either, a set of criteria is established to define the extents of the bandwidth, such as input
impedance, pattern, or polarization.
Percent bandwidth, usually used for narrowband antennas, is used defined as
.
The theoretical limit to percent bandwidth is 200%, which occurs for
.
Fractional bandwidth or Ratio bandwidth, usually used for wideband antennas, is defined as
typically presented in the form of
and is
. Fractional bandwidth is used for wideband antennas because of the
compression of the percent bandwidth that occurs mathematically with percent bandwidths above 100%, which
corresponds to a fractional bandwidth of 3:1.
.
Bandwidth (signal processing)
Photonics
In photonics, the term bandwidth occurs in a variety of meanings:
• the bandwidth of the output of some light source, e.g., an ASE source or a laser; the bandwidth of ultrashort
optical pulses can be particularly large
• the width of the frequency range that can be transmitted by some element, e.g. an optical fiber
• the gain bandwidth of an optical amplifier
• the width of the range of some other phenomenon (e.g., a reflection, the phase matching of a nonlinear process, or
some resonance)
• the maximum modulation frequency (or range of modulation frequencies) of an optical modulator
• the range of frequencies in which some measurement apparatus (e.g., a powermeter) can operate
• the data rate (e.g., in Gbit/s) achieved in an optical communication system; see bandwidth (computing).
A related concept is the spectral linewidth of the radiation emitted by excited atoms.
References
[1] Van Valkenburg, M. E.. Network Analysis (http:/ / www. amazon. com/ Network-Analysis-Mac-Van-Valkenburg/ dp/ 0136110959) (3rd
edition ed.). pp. 383–384. ISBN 0-13-611095-9. . Retrieved 2008-06-22.
[2] Stutzman, Warren L., and Gary A. Theiele. Antenna Theory and Design. 2nd Ed. New York: 1998. ISBN 0-471-02590-9
Lead–lag compensator
A lead–lag compensator is a component in a control system that improves an undesirable frequency response in a
feedback and control system. It is a fundamental building block in classical control theory.
Applications
Lead–lag compensators influence disciplines as varied as robotics, satellite control, automobile diagnostics, laser
frequency stabilization, and many more. They are an important building block in analog control systems, and can
also be used in digital control.
Given a control plant, desired specifications can be achieved using compensators. I, D, PI, PD, and PID, are
optimizing controllers which are used to improve system parameters (such as reducing steady state error, reducing
resonant peak, improving system response by reducing rise time). All these operations can be done by compensators
as well.
Theory
Both lead compensators and lag compensators introduce a pole–zero pair into the open loop transfer function. The
transfer function can be written in the Laplace domain as
where X is the input to the compensator, Y is the output, s is the complex Laplace transform variable, z is the zero
frequency and p is the pole frequency. The pole and zero are both typically negative. In a lead compensator, the pole
is left of the zero in the complex plane,
, while in a lag compensator
.
A lead-lag compensator consists of a lead compensator cascaded with a lag compensator. The overall transfer
function can be written as
293
Leadlag compensator
Typically
294
, where z1 and p1 are the zero and pole of the lead compensator and z2 and p2
are the zero and pole of the lag compensator. The lead compensator provides phase lead at high frequencies. This
shifts the poles to the left, which enhances the responsiveness and stability of the system. The lag compensator
provides phase lag at low frequencies which reduces the steady state error.
The precise locations of the poles and zeros depend on both the desired characteristics of the closed loop response
and the characteristics of the system being controlled. However, the pole and zero of the lag compensator should be
close together so as not to cause the poles to shift right, which could cause instability or slow convergence. Since
their purpose is to affect the low frequency behaviour, they should be near the origin.
Implementation
Both analog and digital control systems use lead-lag compensators. The technology used for the implementation is
different in each case, but the underlying principles are the same. The transfer function is rearranged so that the
output is expressed in terms of sums of terms involving the input, and integrals of the input and output. For example,
In analog control systems, where integrators are expensive, it is common to group terms together to minimize the
number of integrators required:
In analog control, the control signal is typically an electrical voltage or current (although other signals such as
hydraulic pressure can be used). In this case a lead-lag compensator will consist of a network of operational
amplifiers ("op-amps") connected as integrators and weighted adders. In digital control, the operations are performed
numerically.
The reason for expressing the transfer function as an integral equation is that differentiating signals amplify the noise
on the signal, since even very small amplitude noise has a high derivative if its frequency is high, while integrating a
signal averages out the noise. This makes implementations in terms of integrators the most numerically stable.
Intuitive explanation
To begin designing a lead-lag compensator, an engineer must consider whether the system needing correction can be
classified as a lead-network, a lag-network, or a combination of the two: a lead-lag network (hence the name
"lead-lag compensator"). The electrical response of this network to an input signal is expressed by the network's
Laplace-domain transfer function, a complex mathematical function which itself can be expressed as one of two
ways: as the current-gain ratio transfer function or as the voltage-gain ratio transfer function. Remember that a
complex function can be in general written as
, where
is the "Real Part" and
is the "Imaginary Part" of the single-variable function,,
The "phase angle" of the network is the argument of
.
; in the left half plane this is
. If
the phase angle is negative for all signal frequencies in the network then the network is classified as a "lag network".
If the phase angle is positive for all signal frequencies in the network then the network is classified as a "lead
network". If the total network phase angle has a combination of positive and negative phase as a function of
frequency then it is a "lead-lag network".
Depending upon the nominal operation design parameters of a system under an active feedback control, a lag or lead
network can cause instability and poor speed and response times.
Leadlag compensator
References
1. Nise, Norman S. (2004); Control Systems Engineering (4 ed.); Wiley & Sons; ISBN 0-471-44577-0
2. Horowitz, P. & Hill, W. (2001); The Art of Electronics (2 ed.); Cambridge University Press; ISBN 0-521-37095-7
3. Cathey, J.J. (1988); Electronic Devices and Circuits (Schaum's Outlines Series); McGraw-Hill ISBN
0-07-010274-0
External links
•
•
•
•
http://www.library.cmu.edu/ctms/ctms/extras/leadlag.htm
lead controller using Matlab [1]
Lead-Lag Frequency Response [2] at MathPages
Lead-Lag Algorithms [3] at MathPages
References
[1] http:/ / www. sccs. swarthmore. edu/ users/ 06/ adem/ engin/ e58/ lab6/ index. php
[2] http:/ / www. mathpages. com/ home/ kmath249/ kmath249. htm
[3] http:/ / www. mathpages. com/ home/ kmath198/ kmath198. htm
Static VAR compensator
A static var compensator (or SVC) is an electrical device for providing fast-acting reactive power on high-voltage
electricity transmission networks.[1] [2] SVCs are part of the Flexible AC transmission system device family,
regulating voltage and stabilising the system. The term "static" refers to the fact that the SVC has no moving parts
(other than circuit breakers and disconnects, which do not move under normal SVC operation), unlike the
Synchronous condenser, which is a rotating electrical machine. Prior to the invention of the SVC, power factor
compensation was the preserve of large rotating machines such as synchronous condensers.[3]
The SVC is an automated impedance matching device, designed to bring the system closer to unity power factor.
SVCs are used in two main situations:
• Connected to the power system, to regulate the transmission voltage ("Transmission SVC")
• Connected near large industrial loads, to improve power quality ("Industrial SVC")
In transmission applications, the SVC is used to regulate the grid voltage. If the power system's reactive load is
capacitive (leading), the SVC will use reactors (usually in the form of Thyristor-Controlled Reactors) to consume
vars from the system, lowering the system voltage. Under inductive (lagging) conditions, the capacitor banks are
automatically switched in, thus providing a higher system voltage.
In industrial applications, SVCs are typically placed near high and rapidly varying loads, such as arc furnaces, where
they can smooth flicker voltage.[1] [4]
295
Static VAR compensator
Description
Principle
Typically, an SVC comprises one or more banks of fixed or switched shunt capacitors or reactors, of which at least
one bank is switched by thyristors. Elements which may be used to make an SVC typically include:
•
•
•
•
Thyristor controlled reactor (TCR), where the reactor may be air- or iron-cored
Thyristor Switched Capacitor (TSC)
Harmonic filter(s)
Mechanically switched capacitors and/or reactors (switched by a circuit breaker)
One-line diagram of a typical SVC configuration; here employing a thyristor-controlled reactor, a thyristor switched
capacitor, a harmonic filter, a mechanically switched capacitor and a mechanically switched reactor
By means of phase angle modulation switched by the thyristors, the reactor may be variably switched into the circuit
and so provide a continuously variable Mvar injection (or absorption) to the electrical network.[2] In this
configuration, coarse voltage control is provided by the capacitors; the thyristor-controlled reactor is to provide
smooth control. Smoother control and more flexibility can be provided with thyristor-controlled capacitor
switching.[5]
The thyristors are electronically controlled. Thyristors, like all semiconductors, generate heat and deionized water is
commonly used to cool them.[3] Chopping reactive load into the circuit in this manner injects undesirable odd-order
harmonics and so banks of high-power filters are usually provided to smooth the waveform. Since the filters
themselves are capacitive, they also export Mvars to the power system.
More complex arrangements are practical where precise voltage regulation is required. Voltage regulation is
provided by means of a closed-loop controller.[5] Remote supervisory control and manual adjustment of the voltage
set-point are also common.
296
Static VAR compensator
Connection
Generally, static var compensation is not done at line voltage; a bank of transformers steps the transmission voltage
(for example, 230 kV) down to a much lower level (for example, 9.5 kV).[3] This reduces the size and number of
components needed in the SVC, although the conductors must be very large to handle the high currents associated
with the lower voltage. In some static var compensators for industrial applications such as electric arc furnaces,
where there may be an existing medium-voltage busbar present (for example at 33kV or 34.5kV), the static var
compensator may be directly connected in order to save the cost of the transformer.
The dynamic nature of the SVC lies in the use of thyristors connected in series and inverse-parallel, forming
"thyristor valves"). The disc-shaped semiconductors, usually several inches in diameter, are usually located indoors
in a "valve house".
Advantages
The main advantage of SVCs over simple mechanically-switched compensation schemes is their near-instantaneous
response to changes in the system voltage.[5] For this reason they are often operated at close to their zero-point in
order to maximise the reactive power correction they can rapidly provide when required.
They are, in general, cheaper, higher-capacity, faster and more reliable than dynamic compensation schemes such as
synchronous condensers [5] . However, static var compensators are more expensive than mechanically switched
capacitors, so many system operators use a combination of the two technologies (sometimes in the same
installation), using the static var compensator to provide support for fast changes and the mechanically switched
capacitors to provide steady-state Vars.
References
[1] De Kock, Jan; Strauss, Cobus (2004). Practical Power Distribution for Industry (http:/ / books. google. co. uk/ books?id=N8bJpt1wSd4C&
pg=PA74). Elsevier. pp. 74–75. ISBN 9780750663960. .
[2] Deb, Anjan K.. Power Line Ampacity System (http:/ / books. google. co. uk/ books?id=ebZHT8gzpksC& pg=PA169). CRC Press.
pp. 169–171. ISBN 9780849313066. .
[3] Ryan, H.M. (2001). High Voltage Engineering and Testing (http:/ / books. google. co. uk/ books?id=Jg1xA65n56oC& pg=PA160). IEE.
pp. 160–161. ISBN 9780852967751. .
[4] Arrillaga,, J.; Watson, N. R.. Power System Harmonics (http:/ / books. google. co. uk/ books?id=1h9aqRj4o8EC& pg=PA126). Wiley.
pp. 126. ISBN 9780470851296. .
[5] Padiyar, K. R. (1998). Analysis of Subsynchronous Resonance in Power Systems (http:/ / books. google. co. uk/ books?id=QMSELoMjsg0C&
pg=PA169). Springer. pp. 169–177. ISBN 9780792383192. .
297
PID controller
298
PID controller
A
proportional–integral–derivative
controller (PID controller) is a generic
control
loop
feedback
mechanism
(controller) widely used in industrial control
systems – a PID is the most commonly used
feedback controller. A PID controller
calculates an "error" value as the difference
between a measured process variable and a
desired setpoint. The controller attempts to
minimize the error by adjusting the process
control inputs.
A block diagram of a PID controller
The PID controller calculation (algorithm) involves three separate constant parameters, and is accordingly
sometimes called three-term control: the proportional, the integral and derivative values, denoted P, I, and D.
Heuristically, these values can be interpreted in terms of time: P depends on the present error, I on the accumulation
of past errors, and D is a prediction of future errors, based on current rate of change.[1] The weighted sum of these
three actions is used to adjust the process via a control element such as the position of a control valve, or the power
supplied to a heating element.
In the absence of knowledge of the underlying process, a PID controller is the best controller.[2] By tuning the three
parameters in the PID controller algorithm, the controller can provide control action designed for specific process
requirements. The response of the controller can be described in terms of the responsiveness of the controller to an
error, the degree to which the controller overshoots the setpoint and the degree of system oscillation. Note that the
use of the PID algorithm for control does not guarantee optimal control of the system or system stability.
Some applications may require using only one or two actions to provide the appropriate system control. This is
achieved by setting the other parameters to zero. A PID controller will be called a PI, PD, P or I controller in the
absence of the respective control actions. PI controllers are fairly common, since derivative action is sensitive to
measurement noise, whereas the absence of an integral term may prevent the system from reaching its target value
due to the control action.
Control loop basics
Further information: Control system
A familiar example of a control loop is the action taken when adjusting hot and cold faucets (valves) to maintain the
water at a desired temperature. This typically involves the mixing of two process streams, the hot and cold water.
The person touches the water to sense or measure its temperature. Based on this feedback they perform a control
action to adjust the hot and cold water valves until the process temperature stabilizes at the desired value.
The sensed water temperature is the process variable or process value (PV). The desired temperature is called the
setpoint (SP). The input to the process (the water valve position) is called the manipulated variable (MV). The
difference between the temperature measurement and the set point is the error (e) and quantifies whether the water is
too hot or too cold and by how much.
After measuring the temperature (PV), and then calculating the error, the controller decides when to change the tap
position (MV) and by how much. When the controller first turns the valve on, it may turn the hot valve only slightly
if warm water is desired, or it may open the valve all the way if very hot water is desired. This is an example of a
simple proportional control. In the event that hot water does not arrive quickly, the controller may try to speed-up
PID controller
299
the process by opening up the hot water valve more-and-more as time goes by. This is an example of an integral
control.
Making a change that is too large when the error is small is equivalent to a high gain controller and will lead to
overshoot. If the controller were to repeatedly make changes that were too large and repeatedly overshoot the target,
the output would oscillate around the setpoint in either a constant, growing, or decaying sinusoid. If the oscillations
increase with time then the system is unstable, whereas if they decrease the system is stable. If the oscillations
remain at a constant magnitude the system is marginally stable.
In the interest of achieving a gradual convergence at the desired temperature (SP), the controller may wish to damp
the anticipated future oscillations. So in order to compensate for this effect, the controller may elect to temper its
adjustments. This can be thought of as a derivative control method.
If a controller starts from a stable state at zero error (PV = SP), then further changes by the controller will be in
response to changes in other measured or unmeasured inputs to the process that impact on the process, and hence on
the PV. Variables that impact on the process other than the MV are known as disturbances. Generally controllers are
used to reject disturbances and/or implement setpoint changes. Changes in feedwater temperature constitute a
disturbance to the faucet temperature control process.
In theory, a controller can be used to control any process which has a measurable output (PV), a known ideal value
for that output (SP) and an input to the process (MV) that will affect the relevant PV. Controllers are used in industry
to regulate temperature, pressure, flow rate, chemical composition, speed and practically every other variable for
which a measurement exists.
PID controller theory
This section describes the parallel or non-interacting form of the PID controller. For other forms please see
the section Alternative nomenclature and PID forms.
The PID control scheme is named after its three correcting terms, whose sum constitutes the manipulated variable
(MV). The proportional, integral, and derivative terms are summed to calculate the output of the PID controller.
Defining
as the controller output, the final form of the PID algorithm is:
where
: Proportional gain, a tuning parameter
: Integral gain, a tuning parameter
: Derivative gain, a tuning parameter
: Error
: Time or instantaneous time (the present)
PID controller
300
Proportional term
The proportional term makes a change
to the output that is proportional to the
current error value. The proportional
response can be adjusted by multiplying
the error by a constant Kp, called the
proportional gain.
The proportional term is given by:
Plot of PV vs time, for three values of Kp (Ki and Kd held constant)
A high proportional gain results in a large change in the output for a given change in the error. If the proportional
gain is too high, the system can become unstable (see the section on loop tuning). In contrast, a small gain results in
a small output response to a large input error, and a less responsive or less sensitive controller. If the proportional
gain is too low, the control action may be too small when responding to system disturbances. Tuning theory and
industrial practice indicate that the proportional term should contribute the bulk of the output change.
Droop
A pure proportional controller will not always settle at its target value, but may retain a steady-state error.
Specifically, drift in the absence of control, such as cooling of a furnace towards room temperature, biases a pure
proportional controller. If the drift is downwards, as in cooling, then the bias will be below the set point, hence the
term "droop".
Droop is proportional to the process gain and inversely proportional to proportional gain. Specifically the
steady-state error is given by:
Droop is an inherent defect of purely proportional control. Droop may be mitigated by adding a compensating bias
term (setting the setpoint above the true desired value), or corrected by adding an integral term.
PID controller
301
Integral term
The contribution from the integral term
is proportional to both the magnitude of
the error and the duration of the error.
The integral in a PID controller is the
sum of the instantaneous error over time
and gives the accumulated offset that
should have been corrected previously.
The accumulated error is then
multiplied by the integral gain (
)
and added to the controller output.
The integral term is given by:
Plot of PV vs time, for three values of Ki (Kp and Kd held constant)
The integral term accelerates the movement of the process towards setpoint and eliminates the residual steady-state
error that occurs with a pure proportional controller. However, since the integral term responds to accumulated errors
from the past, it can cause the present value to overshoot the setpoint value (see the section on loop tuning).
Derivative term
The derivative of the process error is
calculated by determining the slope of
the error over time and multiplying this
rate of change by the derivative gain
. The magnitude of the
contribution of the derivative term to
the overall control action is termed the
derivative gain,
.
The derivative term is given by:
Plot of PV vs time, for three values of Kd (Kp and Ki held constant)
PID controller
The derivative term slows the rate of change of the controller output. Derivative control is used to reduce the
magnitude of the overshoot produced by the integral component and improve the combined controller-process
stability. However, the derivative term slows the transient response of the controller. Also, differentiation of a signal
amplifies noise and thus this term in the controller is highly sensitive to noise in the error term, and can cause a
process to become unstable if the noise and the derivative gain are sufficiently large. Hence an approximation to a
differentiator with a limited bandwidth is more commonly used. Such a circuit is known as a phase-lead
compensator.
Loop tuning
Tuning a control loop is the adjustment of its control parameters (gain/proportional band, integral gain/reset,
derivative gain/rate) to the optimum values for the desired control response. Stability (bounded oscillation) is a basic
requirement, but beyond that, different systems have different behavior, different applications have different
requirements, and requirements may conflict with one another.
PID tuning is a difficult problem, even though there are only three parameters and in principle is simple to describe,
because it must satisfy complex criteria within the limitations of PID control. There are accordingly various methods
for loop tuning, and more sophisticated techniques are the subject of patents; this section describes some traditional
manual methods for loop tuning.
Designing and tuning a PID controller appears to be conceptually intuitive, but can be hard in practice, if multiple
(and often conflicting) objectives such as short transient and high stability are to be achieved. Usually, initial designs
need to be adjusted repeatedly through computer simulations until the closed-loop system performs or compromises
as desired.
Some processes have a degree of non-linearity and so parameters that work well at full-load conditions don't work
when the process is starting up from no-load; this can be corrected by gain scheduling (using different parameters in
different operating regions). PID controllers often provide acceptable control using default tunings, but performance
can generally be improved by careful tuning, and performance may be unacceptable with poor tuning.
Stability
If the PID controller parameters (the gains of the proportional, integral and derivative terms) are chosen incorrectly,
the controlled process input can be unstable, i.e., its output diverges, with or without oscillation, and is limited only
by saturation or mechanical breakage. Instability is caused by excess gain, particularly in the presence of significant
lag.
Generally, stability of response is required and the process must not oscillate for any combination of process
conditions and setpoints, though sometimes marginal stability (bounded oscillation) is acceptable or desired.
Optimum behavior
The optimum behavior on a process change or setpoint change varies depending on the application.
Two basic requirements are regulation (disturbance rejection – staying at a given setpoint) and command tracking
(implementing setpoint changes) – these refer to how well the controlled variable tracks the desired value. Specific
criteria for command tracking include rise time and settling time. Some processes must not allow an overshoot of the
process variable beyond the setpoint if, for example, this would be unsafe. Other processes must minimize the
energy expended in reaching a new setpoint.
302
PID controller
303
Overview of methods
There are several methods for tuning a PID loop. The most effective methods generally involve the development of
some form of process model, then choosing P, I, and D based on the dynamic model parameters. Manual tuning
methods can be relatively inefficient, particularly if the loops have response times on the order of minutes or longer.
The choice of method will depend largely on whether or not the loop can be taken "offline" for tuning, and the
response time of the system. If the system can be taken offline, the best tuning method often involves subjecting the
system to a step change in input, measuring the output as a function of time, and using this response to determine the
control parameters.
Choosing a Tuning Method
Method
Advantages
Disadvantages
Manual Tuning
No math required. Online method.
Requires experienced personnel.
Ziegler–Nichols
Proven Method. Online method.
Process upset, some trial-and-error,
very aggressive tuning.
Software Tools
Consistent tuning. Online or offline method. May include valve and sensor analysis.
Allow simulation before downloading. Can support Non-Steady State (NSS) Tuning.
Some cost and training involved.
Cohen-Coon
Good process models.
Some math. Offline method. Only
good for first-order processes.
Manual tuning
If the system must remain online, one tuning method is to first set
and
values to zero. Increase the
until the output of the loop oscillates, then the
should be set to approximately half of that value for a "quarter
amplitude decay" type response. Then increase
until any offset is corrected in sufficient time for the process.
However, too much
will cause instability. Finally, increase
, if required, until the loop is acceptably quick
to reach its reference after a load disturbance. However, too much
will cause excessive response and overshoot.
A fast PID loop tuning usually overshoots slightly to reach the setpoint more quickly; however, some systems cannot
accept overshoot, in which case an over-damped closed-loop system is required, which will require a
significantly less than half that of the
setting causing oscillation.
Effects of increasing a parameter independently
Parameter
[3]
Rise time
Overshoot
Settling time
Steady-state error
Decrease
Increase
Small change
Decrease
Degrade
Increase
Increase
Decrease significantly
Degrade
Decrease
[4]
Minor decrease Minor decrease Minor decrease
No effect in theory
Stability
Improve if
small
setting
PID controller
304
Ziegler–Nichols method
Another heuristic tuning method is formally known as the Ziegler–Nichols method, introduced by John G. Ziegler
and Nathaniel B. Nichols in the 1940s. As in the method above, the
and
gains are first set to zero. The P
gain is increased until it reaches the ultimate gain,
the oscillation period
, at which the output of the loop starts to oscillate.
and
are used to set the gains as shown:
Ziegler–Nichols method
Control Type
P
PI
-
-
PID
These gains apply to the ideal, parallel form of the PID controller. When applied to the standard PID form, the
integral and derivative time parameters
and
are only dependent on the oscillation period
. Please see the
section "Alternative nomenclature and PID forms".
PID tuning software
Most modern industrial facilities no longer tune loops using the manual calculation methods shown above. Instead,
PID tuning and loop optimization software are used to ensure consistent results. These software packages will gather
the data, develop process models, and suggest optimal tuning. Some software packages can even develop tuning by
gathering data from reference changes.
Mathematical PID loop tuning induces an impulse in the system, and then uses the controlled system's frequency
response to design the PID loop values. In loops with response times of several minutes, mathematical loop tuning is
recommended, because trial and error can literally take days just to find a stable set of loop values. Optimal values
are harder to find. Some digital loop controllers offer a self-tuning feature in which very small setpoint changes are
sent to the process, allowing the controller itself to calculate optimal tuning values.
Other formulas are available to tune the loop according to different performance criteria. Many patented formulas are
now embedded within PID tuning software and hardware modules.
Advances in automated PID Loop Tuning software also deliver algorithms for tuning PID Loops in a dynamic or
Non-Steady State (NSS) scenario. The software will model the dynamics of a process, through a disturbance, and
calculate PID control parameters in response.
Modifications to the PID algorithm
The basic PID algorithm presents some challenges in control applications that have been addressed by minor
modifications to the PID form.
Integral windup
One common problem resulting from the ideal PID implementations is integral windup, where a large change in
setpoint occurs (say a positive change) and the integral term accumulates an error larger than the maximal value for
the regulation variable (windup), thus the system overshoots and continues to increase as this accumulated error is
unwound. This problem can be addressed by:
• Initializing the controller integral to a desired value
• Increasing the setpoint in a suitable ramp
• Disabling the integral function until the PV has entered the controllable region
• Limiting the time period over which the integral error is calculated
PID controller
• Preventing the integral term from accumulating above or below pre-determined bounds
Overshooting from known disturbances
For example, a PID loop is used to control the temperature of an electric resistance furnace, the system has
stabilized. Now the door is opened and something cold is put into the furnace the temperature drops below the
setpoint. The integral function of the controller tends to compensate this error by introducing another error in
the positive direction. This overshoot can be avoided by freezing of the integral function after the opening of
the door for the time the control loop typically needs to reheat the furnace.
Replacing the integral function by a model based part
Often the time-response of the system is approximately known. Then it is an advantage to simulate this
time-response with a model and to calculate some unknown parameter from the actual response of the system.
If for instance the system is an electrical furnace the response of the difference between furnace temperature
and ambient temperature to changes of the electrical power will be similar to that of a simple RC low-pass
filter multiplied by an unknown proportional coefficient. The actual electrical power supplied to the furnace is
delayed by a low-pass filter to simulate the response of the temperature of the furnace and then the actual
temperature minus the ambient temperature is divided by this low-pass filtered electrical power. Then, the
result is stabilized by another low-pass filter leading to an estimation of the proportional coefficient. With this
estimation, it is possible to calculate the required electrical power by dividing the set-point of the temperature
minus the ambient temperature by this coefficient. The result can then be used instead of the integral function.
This also achieves a control error of zero in the steady-state, but avoids integral windup and can give a
significantly improved control action compared to an optimized PID controller. This type of controller does
work properly in an open loop situation which causes integral windup with an integral function. This is an
advantage if, for example, the heating of a furnace has to be reduced for some time because of the failure of a
heating element, or if the controller is used as an advisory system to a human operator who may not switch it
to closed-loop operation. It may also be useful if the controller is inside a branch of a complex control system
that may be temporarily inactive.
Many PID loops control a mechanical device (for example, a valve). Mechanical maintenance can be a major cost
and wear leads to control degradation in the form of either stiction or a deadband in the mechanical response to an
input signal. The rate of mechanical wear is mainly a function of how often a device is activated to make a change.
Where wear is a significant concern, the PID loop may have an output deadband to reduce the frequency of
activation of the output (valve). This is accomplished by modifying the controller to hold its output steady if the
change would be small (within the defined deadband range). The calculated output must leave the deadband before
the actual output will change.
The proportional and derivative terms can produce excessive movement in the output when a system is subjected to
an instantaneous step increase in the error, such as a large setpoint change. In the case of the derivative term, this is
due to taking the derivative of the error, which is very large in the case of an instantaneous step change. As a result,
some PID algorithms incorporate the following modifications:
Derivative of the Process Variable
In this case the PID controller measures the derivative of the measured process variable (PV), rather than the
derivative of the error. This quantity is always continuous (i.e., never has a step change as a result of changed
setpoint). For this technique to be effective, the derivative of the PV must have the opposite sign of the
derivative of the error, in the case of negative feedback control.
Setpoint ramping
In this modification, the setpoint is gradually moved from its old value to a newly specified value using a
linear or first order differential ramp function. This avoids the discontinuity present in a simple step change.
Setpoint weighting
305
PID controller
Setpoint weighting uses different multipliers for the error depending on which element of the controller it is
used in. The error in the integral term must be the true control error to avoid steady-state control errors. This
affects the controller's setpoint response. These parameters do not affect the response to load disturbances and
measurement noise.
History
PID controllers date to 1890s governor design.[2] [5] PID controllers
were subsequently developed in automatic ship steering. One of the
earliest examples of a PID-type controller was developed by Elmer
Sperry in 1911,[6] while the first published theoretical analysis of a PID
controller was by Russian American engineer Nicolas Minorsky, in
(Minorsky 1922). Minorsky was designing automatic steering systems
for the US Navy, and based his analysis on observations of a
helmsman, observing that the helmsman controlled the ship not only
based on the current error, but also on past error and current rate of
PID theory developed by observing the action of
change;[7] this was then made mathematical by Minorsky. His goal was
helmsmen.
stability, not general control, which significantly simplified the
problem. While proportional control provides stability against small disturbances, it was insufficient for dealing with
a steady disturbance, notably a stiff gale (due to droop), which required adding the integral term. Finally, the
derivative term was added to improve control.
Trials were carried out on the USS New Mexico, with the controller controlling the angular velocity (not angle) of
the rudder. PI control yielded sustained yaw (angular error) of ±2°, while adding D yielded yaw of ±1/6°, better than
most helmsmen could achieve.[8]
The Navy ultimately did not adopt the system, due to resistance by personnel. Similar work was carried out and
published by several others in the 1930s.
Limitations of PID control
While PID controllers are applicable to many control problems, and often perform satisfactorily without any
improvements or even tuning, they can perform poorly in some applications, and do not in general provide optimal
control. The fundamental difficulty with PID control is that it is a feedback system, with constant parameters, and no
direct knowledge of the process, and thus overall performance is reactive and a compromise – while PID control is
the best controller with no model of the process,[2] better performance can be obtained by incorporating a model of
the process.
The most significant improvement is to incorporate feed-forward control with knowledge about the system, and
using the PID only to control error. Alternatively, PIDs can be modified in more minor ways, such as by changing
the parameters (either gain scheduling in different use cases or adaptively modifying them based on performance),
improving measurement (higher sampling rate, precision, and accuracy, and low-pass filtering if necessary), or
cascading multiple PID controllers.
PID controllers, when used alone, can give poor performance when the PID loop gains must be reduced so that the
control system does not overshoot, oscillate or hunt about the control setpoint value. They also have difficulties in
the presence of non-linearities, may trade-off regulation versus response time, do not react to changing process
behavior (say, the process changes after it has warmed up), and have lag in responding to large disturbances.
306
PID controller
Linearity
Another problem faced with PID controllers is that they are linear, and in particular symmetric. Thus, performance
of PID controllers in non-linear systems (such as HVAC systems) is variable. For example, in temperature control, a
common use case is active heating (via a heating element) but passive cooling (heating off, but no cooling), so
overshoot can only be corrected slowly – it cannot be forced downward. In this case the PID should be tuned to be
overdamped, to prevent or reduce overshoot, though this reduces performance (it increases settling time).
Noise in derivative
A problem with the derivative term is that small amounts of measurement or process noise can cause large amounts
of change in the output. It is often helpful to filter the measurements with a low-pass filter in order to remove
higher-frequency noise components. However, low-pass filtering and derivative control can cancel each other out, so
reducing noise by instrumentation means is a much better choice. Alternatively, a nonlinear median filter may be
used, which improves the filtering efficiency and practical performance.[9] In some case, the differential band can be
turned off in many systems with little loss of control. This is equivalent to using the PID controller as a PI controller.
Improvements
Feed-forward
The control system performance can be improved by combining the feedback (or closed-loop) control of a PID
controller with feed-forward (or open-loop) control. Knowledge about the system (such as the desired acceleration
and inertia) can be fed forward and combined with the PID output to improve the overall system performance. The
feed-forward value alone can often provide the major portion of the controller output. The PID controller can be used
primarily to respond to whatever difference or error remains between the setpoint (SP) and the actual value of the
process variable (PV). Since the feed-forward output is not affected by the process feedback, it can never cause the
control system to oscillate, thus improving the system response and stability.
For example, in most motion control systems, in order to accelerate a mechanical load under control, more force or
torque is required from the prime mover, motor, or actuator. If a velocity loop PID controller is being used to control
the speed of the load and command the force or torque being applied by the prime mover, then it is beneficial to take
the instantaneous acceleration desired for the load, scale that value appropriately and add it to the output of the PID
velocity loop controller. This means that whenever the load is being accelerated or decelerated, a proportional
amount of force is commanded from the prime mover regardless of the feedback value. The PID loop in this
situation uses the feedback information to change the combined output to reduce the remaining difference between
the process setpoint and the feedback value. Working together, the combined open-loop feed-forward controller and
closed-loop PID controller can provide a more responsive, stable and reliable control system.
Other improvements
In addition to feed-forward, PID controllers are often enhanced through methods such as PID gain scheduling
(changing parameters in different operating conditions), fuzzy logic or computational verb logic. [10] [11] Further
practical application issues can arise from instrumentation connected to the controller. A high enough sampling rate,
measurement precision, and measurement accuracy are required to achieve adequate control performance. Another
new method for improvement of PID controller is to increase the degree of freedom by using fractional order. The
order of the integrator and differentiator add increased flexibility to the controller.
307
PID controller
308
Cascade control
One distinctive advantage of PID controllers is that two PID controllers can be used together to yield better dynamic
performance. This is called cascaded PID control. In cascade control there are two PIDs arranged with one PID
controlling the set point of another. A PID controller acts as outer loop controller, which controls the primary
physical parameter, such as fluid level or velocity. The other controller acts as inner loop controller, which reads the
output of outer loop controller as set point, usually controlling a more rapid changing parameter, flowrate or
acceleration. It can be mathematically proven that the working frequency of the controller is increased and the time
constant of the object is reduced by using cascaded PID controller..
Physical implementation of PID control
In the early history of automatic process control the PID controller was implemented as a mechanical device. These
mechanical controllers used a lever, spring and a mass and were often energized by compressed air. These pneumatic
controllers were once the industry standard.
Electronic analog controllers can be made from a solid-state or tube amplifier, a capacitor and a resistance.
Electronic analog PID control loops were often found within more complex electronic systems, for example, the
head positioning of a disk drive, the power conditioning of a power supply, or even the movement-detection circuit
of a modern seismometer. Nowadays, electronic controllers have largely been replaced by digital controllers
implemented with microcontrollers or FPGAs.
Most modern PID controllers in industry are implemented in programmable logic controllers (PLCs) or as a
panel-mounted digital controller. Software implementations have the advantages that they are relatively cheap and
are flexible with respect to the implementation of the PID algorithm.
Variable voltages may be applied by the time proportioning form of Pulse-width modulation (PWM) – a cycle time
is fixed, and variation is achieved by varying the proportion of the time during this cycle that the controller outputs
+1 (or −1) instead of 0. On a digital system the possible proportions are discrete – e.g., increments of .1 second
within a 2 second cycle time yields 20 possible steps: percentage increments of 5% – so there is a discretization
error, but for high enough time resolution this yields satisfactory performance.
Alternative nomenclature and PID forms
Ideal versus standard PID form
The form of the PID controller most often encountered in industry, and the one most relevant to tuning algorithms is
the standard form. In this form the
gain is applied to the
, and
terms, yielding:
where
is the integral time
is the derivative time
In this standard form, the parameters have a clear physical meaning. In particular, the inner summation produces a
new single error value which is compensated for future and past errors. The addition of the proportional and
derivative components effectively predicts the error value at
seconds (or samples) in the future, assuming that
the loop control remains unchanged. The integral component adjusts the error value to compensate for the sum of all
past errors, with the intention of completely eliminating them in
seconds (or samples). The resulting
compensated single error value is scaled by the single gain
.
In the ideal parallel form, shown in the controller theory section
PID controller
the gain parameters are related to the parameters of the standard form through
309
and
. This
parallel form, where the parameters are treated as simple gains, is the most general and flexible form. However, it is
also the form where the parameters have the least physical interpretation and is generally reserved for theoretical
treatment of the PID controller. The standard form, despite being slightly more complex mathematically, is more
common in industry.
Basing derivative action on PV
In most commercial control systems, derivative action is based on PV rather than error. This is because the digitised
version of the algorithm produces a large unwanted spike when the SP is changed. If the SP is constant then changes
in PV will be the same as changes in error. Therefore this modification makes no difference to the way the controller
responds to process disturbances.
Basing proportional action on PV
Most commercial control systems offer the option of also basing the proportional action on PV. This means that only
the integral action responds to changes in SP. While at first this might seem to adversely affect the time that the
process will take to respond to the change, the controller may be retuned to give almost the same response - largely
by increasing
. The modification to the algorithm does not affect the way the controller responds to process
disturbances, but the change in tuning has a beneficial effect. Often the magnitude and duration of the disturbance
will be more than halved. Since most controllers have to deal frequently with process disturbances and relatively
rarely with SP changes, properly tuned the modified algorithm can dramatically improve process performance.
Tuning methods such as Ziegler-Nichols and Cohen-Coon will not be reliable when used with this algorithm.
King[12] describes an effective chart-based method.
Laplace form of the PID controller
Sometimes it is useful to write the PID regulator in Laplace transform form:
Having the PID controller written in Laplace form and having the transfer function of the controlled system makes it
easy to determine the closed-loop transfer function of the system.
PID controller
310
PID Pole Zero Cancellation
The PID equation can be written in this form:
When this form is used it is easy to determine the closed loop transfer function.
If
Then
This can be very useful to remove unstable poles
Series/interacting form
Another representation of the PID controller is the series, or interacting form
where the parameters are related to the parameters of the standard form through
,
, and
with
.
This form essentially consists of a PD and PI controller in series, and it made early (analog) controllers easier to
build. When the controllers later became digital, many kept using the interacting form.
Discrete implementation
The analysis for designing a digital implementation of a PID controller in a Microcontroller (MCU) or FPGA device
requires the standard form of the PID controller to be discretised.[13] Approximations for first-order derivatives are
made by backward finite differences. The integral term is discretised, with a sampling time
,as follows,
The derivative term is approximated as,
Thus, a velocity algorithm for implementation of the discretised PID controller in a MCU is obtained by
differentiating
, using the numerical definitions of the first and second derivative and solving for
and
finally obtaining:
PID controller
311
Pseudocode
Here is a simple software loop that implements the PID algorithm in its 'ideal, parallel' form:
previous_error = setpoint - process_feedback
integral = 0
start:
wait(dt)
error = setpoint - process_feedback
integral = integral + (error*dt)
derivative = (error - previous_error)/dt
output = (Kp*error) + (Ki*integral) + (Kd*derivative)
previous_error = error
goto start
PI controller
A PI Controller (proportional-integral
controller) is a special case of the PID
controller in which the derivative (D) of the
error is not used.
The controller output is given by
Basic block of a PI controller.
where
is the error or deviation of actual measured value (PV) from the set-point (SP).
.
A PI controller can be modelled easily in software such as Simulink using a "flow chart" box involving Laplace
operators:
where
= proportional gain
= integral gain
Setting a value for
is often a trade off between decreasing overshoot and increasing settling time.
The lack of derivative action may make the system more steady in the steady state in the case of noisy data. This is
because derivative action is more sensitive to higher-frequency terms in the inputs.
Without derivative action, a PI-controlled system is less responsive to real (non-noise) and relatively fast alterations
in state and so the system will be slower to reach setpoint and slower to respond to perturbations than a well-tuned
PID system may be.
PID controller
References
[1] Araki, M.. "PID Control" (http:/ / www. eolss. net/ ebooks/ Sample Chapters/ C18/ E6-43-03-03. pdf). .
[2] Bennett, Stuart (1993). A history of control engineering, 1930-1955. IET. p. p. 48 (http:/ / books. google. com/ books?id=VD_b81J3yFoC&
pg=PA48). ISBN 9-780863412998
[3] Ang, K.H., Chong, G.C.Y., and Li, Y. (2005). PID control system analysis, design, and technology, IEEE Trans Control Systems Tech, 13(4),
pp.559-576. http:/ / eprints. gla. ac. uk/ 3817/
[4] Jinghua Zhong (Spring 2006). PID Controller Tuning: A Short Tutorial (http:/ / saba. kntu. ac. ir/ eecd/ pcl/ download/ PIDtutorial. pdf). .
Retrieved 2011-04-04.
[5] Bennett, Stuart (November 1984). "Nicholas Minorsky and the automatic steering of ships" (http:/ / ieeexplore. ieee. org/ iel5/ 37/ 24267/
01104827. pdf?arnumber=1104827). IEEE Control Systems Magazine 4 (4): 10–15. doi:10.1109/MCS.1984.1104827. ISSN 0272-1708.
[6] "A Brief Building Automation History" (http:/ / www. building-automation-consultants. com/ building-automation-history. html). . Retrieved
2011-04-04.
[7] (Bennett 1993, p. 67 (http:/ / books. google. com/ books?id=VD_b81J3yFoC& pg=PA67))
[8] Bennett, Stuart (June 1986). A history of control engineering, 1800-1930. IET. pp. 142–148 (http:/ / books. google. com/
books?id=1gfKkqB_fTcC& pg=PA142). ISBN 978-0-86341047-5.
[9] Li, Y. and Ang, K.H. and Chong, G.C.Y. (2006) PID control system analysis and design - Problems, remedies, and future directions. IEEE
Control Systems Magazine, 26 (1). pp. 32-41. ISSN 0272-1708 (http:/ / eprints. gla. ac. uk/ 3815/ )
[10] Yang, T. (June 2005). "Architectures of Computational Verb Controllers: Towards a New Paradigm of Intelligent Control". International
Journal of Computational Cognition (Yang's Scientific Press) 3 (2): 74–101.
[11] Liang, Y.-L.() et al. (2009). "Controlling fuel annealer using computational verb PID controllers". Proceedings of the 3rd international
conference on Anti-Counterfeiting, security, and identification in communication (IEEE): 417–420.
[12] King, Myke. Process Control: A Practical Approach. Wiley, 2010, p. 52-78
[13] "Discrete PI and PID Controller Design and Analysis for Digital Implementation" (http:/ / www. scribd. com/ doc/ 19070283/
Discrete-PI-and-PID-Controller-Design-and-Analysis-for-Digital-Implementation). Scribd.com. . Retrieved 2011-04-04.
• Minorsky, Nicolas (1922). "Directional stability of automatically steered bodies". J. Amer. Soc. Naval Eng. 34
(2): 280–309
• Liptak, Bela (1995). Instrument Engineers' Handbook: Process Control. Radnor, Pennsylvania: Chilton Book
Company. pp. 20–29. ISBN 0-8019-8242-1.
• Tan, Kok Kiong; Wang Qing-Guo, Hang Chang Chieh (1999). Advances in PID Control. London, UK:
Springer-Verlag. ISBN 1-85233-138-0.
• King, Myke (2010). Process Control: A Practical Approach (http://eu.wiley.com/WileyCDA/WileyTitle/
productCd-0470975873.html). Chichester, UK: John Wiley & Sons Ltd.. ISBN 978-0-470-97587-9.
• Van, Doren, Vance J. (July 1, 2003). "Loop Tuning Fundamentals" (http://old.controleng.com/article/
268148-Loop_Tuning_Fundamentals.php.html). Control Engineering (Reed Business Information).
• Sellers, David. "An Overview of Proportional plus Integral plus Derivative Control and Suggestions for Its
Successful Application and Implementation" (http://web.archive.org/web/20070307161741/http://www.
peci.org/library/PECI_ControlOverview1_1002.pdf) (PDF). Archived from the original (http://www.peci.
org/library/PECI_ControlOverview1_1002.pdf) on March 7, 2007. Retrieved 2007-05-05.
• Graham, Ron (10/03/2005). "FAQ on PID controller tuning" (http://web.archive.org/web/20050206113949/
www.tcnj.edu/~rgraham/PID-tuning.html). Retrieved 2009-01-05.
312
PID controller
External links
PID tutorials
• PID Tutorial (http://www.engin.umich.edu/group/ctm/PID/PID.html)
• P.I.D. Without a PhD (http://igor.chudov.com/manuals/Servo-Tuning/PID-without-a-PhD.pdf): a beginner's
guide to PID loop theory with sample programming code
• What's All This P-I-D Stuff, Anyhow? (http://electronicdesign.com/article/analog-and-mixed-signal/
what-s-all-this-p-i-d-stuff-anyhow-6131.aspx) Article in Electronic Design
• Shows how to build a PID controller with basic electronic components (http://asl.epfl.ch/research/projects/
VtolIndoorFlying/rapports/rapportSemStauffer.pdf) (pg. 22)
• Virtual PID Controller Laboratory (http://www.pidlab.com)
• PID Design & Tuning (http://www.pidlab.com/en/pid-design-and-tuning)
• Online PID Tuning applet from University of Texas Control Group (http://www.che.utexas.edu/course/
che360/documents/tuner/Process_Tuner.html)
• example PID implementation in 16F887 microchip (http://simf1.blogspot.com/)
• PID Control with MATLAB and Simulink (http://www.mathworks.com/discovery/pid-control.html)
• Improving the Beginner’s PID – Introduction (http://brettbeauregard.com/blog/2011/04/
improving-the-beginners-pid-introduction)
Special topics and PID control applications
• Proven Methods and Best Practices for PID Control (http://www.controlguru.com/pages/table.html)
• PID Control Primer (http://www.embedded.com/story/OEG20020726S0044) Article in Embedded Systems
Programming
313
Article Sources and Contributors
Article Sources and Contributors
Systems analysis Source: http://en.wikipedia.org/w/index.php?oldid=467516213 Contributors: 1ForTheMoney, Aitias, Al.Ragsdale, Albamuth, AllTheCoolNamesAreAlreadyTaken,
Architectchao, Ask123, Beowulph, BigHaz, Bigs slb, Bobbyanalog, Bobo192, Brian48, Btyner, Burzmali, CMG, Cask05, Charleenmerced, Cyrus Grisham, DVdm, Deipnosophista, EagleFan, El
C, Erkan Yilmaz, Fooziex, Freakmighty, Goethean, Ian Tabi, Impaciente, Irmtraining, Jac16888, Jeffmartincg, Jesgab, Jheald, Johnbrownsbody, JonHarder, Jpbowen, Kenyon, Kingturtle, Kuru,
Larsobrien, Liridon, Logicalgregory, Luna Santin, Malcolmxl5, Maria C Mosak, Mark Renier, Materialscientist, Maurreen, Mcsharry, Mdd, Mnent, Mr Stephen, MrOllie, Mydogategodshat,
Mårten Berglund, NawlinWiki, Nbarth, Nick Green, Nono64, Octahedron80, Philip Trueman, Philosopher06, Piano non troppo, Pinethicket, Qst, R'n'B, RJBurkhart, Rajah, RattleMan, Remuel,
Riana, Rich Farmbrough, Ronz, S0aasdf2sf, Shreshth91, SimonClinch, Skittleys, Soresumakashi, Srikant.sharma, Stevertigo, Straker, Strangnet, Stypex, Sverdrup, Tawker, Tbackstr, Ted Wilkes,
The Missing Hour, The Rambling Man, Tide rolls, Toxickore, Velella, WereSpielChequers, Wiki Roxor, Willking1979, Wpociengel, X!, Yamamoto Ichiro, Zoara, 217 anonymous edits
Hardware architecture Source: http://en.wikipedia.org/w/index.php?oldid=456923882 Contributors: Alan Liefting, Basar, Bwilkins, CoderGnome, Dpv, EagleFan, FlyHigh, Frap, Giraffedata,
Hetar, J04n, JonHarder, Kubanczyk, Mboverload, MichaelBillington, Neparis, Normxxx, Overjive, Rettetast, Rror, Squids and Chips, TDS, Tribaal, Unused0030, YoavShapira, 29 anonymous
edits
Stability theory Source: http://en.wikipedia.org/w/index.php?oldid=444273131 Contributors: Arcfrk, Bird of paradox, BradLipovsky, Dmitri83, El C, Grafen, Gurch, Hans Adler, Infovarius,
Jbergquist, Linas, MathMartin, Mathmanta, Mathuvw, Mdd, Megus, Michael Hardy, Pleasantville, R.e.b., Sanfranman59, Sniedo, Spinningspark, TedPavlic, Tim1357, 14 anonymous edits
Control engineering Source: http://en.wikipedia.org/w/index.php?oldid=461797703 Contributors: AJStadlin, Ahmedsaieed, Allan McInnes, Alll, Americanhero, AndrewHowse, Andy Dingley,
Atique1619, Attilios, AutomationBooks, Billymac00, Caporaletti, Cburnett, Chmod007, Chzz, Cjlim, Control.optimization, ControlsGuru, Conversion script, Curtwulf, Cvgittings, Cyrius,
DRivera44, Darigan, DeNihiloNihil, Delldot, Dicklyon, Djgreenfield, Dthomsen8, Dúnadan, Edcolins, El C, Energon, Engelec, Firebladed, GB fan, Ghostal, Glenn, Idril, Iswantoumy,
Iterator12n, IvanLanin, JanHRichter, Jimmayjr, Jiuguang Wang, Jluedem, Jugander, JustAGal, KKoolstra, KarenW, Kdcarver, Krellis, Light current, MWOAP(AWB), Mani excel, Mdd,
Mduench, Michael Hardy, Mohammad1985, MrOllie, Nbarth, Ninly, Nmnogueira, Nposs, Oleg Alexandrov, Oli Filth, Orderud, Pan Dan, Pascalv, PaulGarner, Petedarnell, PeterWoolf, Phuzion,
PoccilScript, Politepunk, Pro crast in a tor, Rangoon11, Ranilb5, Rich Farmbrough, Rjpbi, Rrburke, Rs2, Saatwik Katiha, Sanguinity, Saty85, Savh, SchreyP, SeanMon411, Seb, Seh444,
Sigmundg, SimonMayer, Sitar Physics, Spalding, Spradlig, Style, Swrkoab, Sympatycznyfacet, Tdadamemd, Tobias Hoevekamp, Toffile, Triddle, User A1, Viral sachde, Wayne Slam,
WhatisFeelings?, WikHead, 148 anonymous edits
Signal processing Source: http://en.wikipedia.org/w/index.php?oldid=465772966 Contributors: 10metreh, Adoniscik, Alexkin, Alinja, Altenmann, Andrus Kallastu, Antandrus, Bact, Bethnim,
Biasoli, Borgx, Burhem, Cdworetzky, Cedars, Charles Matthews, Chester Markel, Christopherlin, Conversion script, Cookie90, Dicklyon, Dwachter, EagleFan, Emperorbma, Firefly322,
Funandtrvl, Geoeg, Giftlite, Glenn, Glrx, Grebaldar, Groovamos, GyroMagician, Helwr, Heron, Hezarfenn, Isheden, Ixfd64, Jamilsamara, Janki, Jennifer Levinson, Jimmaths, Johnuniq, Jovial4u,
Jusdafax, Kjkolb, Kvng, Light current, Llorenzi, Luckyz, MER-C, Mako098765, Mange01, Mingramh, Monig1, Muriel Gottrop, Nabla, Ninly, Northumbrian, Oicumayberight, Oleg Alexandrov,
OrgasGirl, Redgecko, Rememberway, Robert.Harker, Robsavoie, Rockyrackoon, Saddhiyama, SamShearman, Sandeeppalakkal, Sergio.ballestrero, Sfeldm2, Shyamal, Sl, Smack, Spinningspark,
The Photon, Tonto Kowalski, TransporterMan, Triddle, Vishalbhardwaj, Voidxor, Wiki contributor 21, Wikihitech, Yk Yk Yk, Yswismer, Zvika, Ömer Cengiz Çelebi, 75 anonymous edits
Feedback Source: http://en.wikipedia.org/w/index.php?oldid=465317525 Contributors: 0, 130.159.254.xxx, 16@r, 28bytes, 4twenty42o, A8UDI, ABoerma, Aapo Laitinen, Abmcdonald,
AbsolutDan, Achler, AdjustShift, Ahmedsaieed, Alan McBeth, Ale2006, Aliekens, Altenmann, Andre Engels, AndrewHowse, Anupmalavia, Anwormy, ArglebargleIV, Arman Cagle, Arthena,
Asinclair, Athkalani, Banjodog, Bart133, Barticus88, Belovedfreak, Bhuston, Blueyoshi321, Brews ohare, Brian787, Bushytails, Butseriouslyfolks, Bwhack, CH3374H, Cancun771,
CapitalLetterBeginning, Charles Matthews, Chjoaygame, Chuunen Baka, Circuit dreamer, Cliff smith, Constructive editor, Conti, Cutler, Darrik2, Davidrust, Dcoetzee, Dicklyon, Dino, Dlewis3,
DocWatson42, Dragana666, Drphilharmonic, E-Kartoffel, Eaefremov, Edudobay, Eekerz, El C, Elwikipedista, Erckvlp, Erianna, Euryalus, Examtester, Fjellstad, Fredbauder, Furrykef, Fyrael,
GeorgeMoney, Giftlite, Glane23, Glenn, Gloo, Gogo Dodo, Goto, Heron, Holgerson0, Hoof Hearted, Houtlijm, Huangdi, Id447, Intgr, Isourou, IstvanWolf, Jaeger5432, Jbergste, JenVan,
Jh51681, Jim.henderson, Jimmaciejewski, Jiuguang Wang, Jkeene, John, Johnuniq, Jpbowen, Jwdietrich2, KD5TVI, Kansas Sam, Keegan, Kenneth M Burke, Kingturtle, Klemen Kocjancic,
Ktpeil, Kwharris, Kzzl, L Kensington, Lachatdelarue, Lakinekaki, Lecochonbleu, Levineps, Lexor, Lightmouse, Linuxlad, Luna Santin, Mac, Mahanga, Masgatotkaca, Mausy5043, Mdd, Michael
Hardy, Mike Rosoft, Mintleaf, Mirwin, Mrs.arna11, Mschlindwein, Musicbox1987, Mwtoews, Nbarth, Neelix, Nick Green, Nigelj, Nikoladie, Nitsky416, Nixdorf, Nonstandard, Nuno Tavares,
Nvillega, Nzd, Nzzl, Oatmeal batman, Odie5533, Oobyduby, Ott, Owlgorithm, Patrick, Pengo, Pgreenfinch, Phanerozoic, Piano non troppo, Poccil, Pokipsy76, Pol098, Postglock, Poweroid,
Profero, Qwert, RFerreira, RainbowOfLight, ReCover, Reviewsnap, Rjwilmsi, Rod57, Rogerbrent, Ronz, Rurigok, Ryguasu, SEWilco, Sandshoe, Santryl, Shirulashem, SimonP, SoftwareLibre,
Sommers, Sophie means wisdom, Steve10345, Strife911, SusanLesch, Taxman, Thseamon, Torc2, Trevithj, Troy 07, Ttwaring, Underpants, Vonkje, Wayne Slam, WhatamIdoing, Wikiborg,
Williamoxley, Wji, Woohookitty, XJamRastafire, Xanzzibar, Zlerman, Zzuuzz, Мирослав Ћика, ‫ﮐﺎﺷﻒ ﻋﻘﯿﻞ‬, 297 anonymous edits
Negative feedback Source: http://en.wikipedia.org/w/index.php?oldid=469176142 Contributors: ABF, Abmcdonald, Achler, Aeluwas, Alex Liebsch, Alfred Centauri, AlsatianRain, Anthere,
Arcadian, BenFrantzDale, Bensaccount, Binksternet, Bn, Brews ohare, Brian0918, Calrosfing, Catgut, Chjoaygame, Chris fluppy, Coppertwig, CyrilB, DRosenbach, Darthsco, Davidmcnatt,
Dcoetzee, Dicklyon, Discospinster, Dominick, Epbr123, Evercat, Flyingidiot, GcSwRhIc, Giftlite, Giridhar, Gizziiusa, Glenn, Heybutch, Hooperbloob, IMSoP, J.delanoy, Jncraton, Johnuniq,
Jpbowen, Jwdietrich2, KD5TVI, Ktims, Ktpeil, Kuyabribri, Kyk, Leafyplant, Lexor, Lexw, Light current, Linuxlad, Lir, Makawity, Marshall Williams2, Matthew Yeager, Mdd, Michael Hardy,
Mitch994, Moondoll, Nathan Johnson, Nbarth, Nigelj, Nikoladie, Novangelis, OlEnglish, Oli Filth, Omegatron, Patrick, Pearle, Pebkac, Philip Trueman, Prashanthns, PrestonH, RJaguar3,
RUL3R, Rogerbrent, Rohitbd, Rspanton, Rustyfence, RyanGerbil10, Ryulong, Shell Kinney, Sheridp, SineWave, Snippingo, So God created Manchester, Soliloquial, Stepp-Wulf, Teapeat,
Tech77jp, The Utahraptor, Tide rolls, Tswsl1989, Two Bananas, Uthbrian, William Ortiz, Wlodzimierz, Wolfkeeper, XRDoDRX, Xcentaur, Yettie0711, Youre dreaming eh?, Yves-Laurent,
ZimZalaBim, Zundark, 226 anonymous edits
Positive feedback Source: http://en.wikipedia.org/w/index.php?oldid=469820378 Contributors: Ahoerstemeier, Allthingstoallpeople, Alta-Snowbird, Altenmann, Andareed, AndrewHowse,
Andy M. Wang, Angela, Anthere, Arcadian, Athkalani, BD2412, Beland, Bgwhite, Bhuston, Brendan19, C-randles, Cafzal, Calrosfing, Caltas, Cantus, Chjoaygame, Circuit dreamer, Cisba,
CyborgTosser, DavidLevinson, Dcoetzee, DeadEyeArrow, Denelson83, Denniskakai, Derek Ross, Djr32, Drrdudley, Dudecon, Eleassar777, Emiehling, Epbr123, Erkan Yilmaz, Every name is
taken12345, Ewlyahoocom, Fredrick day, Geologyguy, Giftlite, Ginsengbomb, Gizziiusa, Glenn, GliderMaven, Hammersoft, Harryboyles, Herenthere, Hulten, Id447, ImperfectlyInformed,
Indium, Jenks1987, Johnuniq, Jonathanstray, Jopienator, Joriki, Jpbowen, KD5TVI, Kakuchirana, Ktpeil, Lexor, LibLord, Light current, LilHelpa, Luigifan, Magwo, Mandarax, Mannafredo,
Maurice Carbonaro, Mdd, Member, Mild Bill Hiccup, Mluo2010, ModManDan, NESFreak92, NHRHS2010, NSR, Nascar1996, Nashhinton, Nathan Johnson, Nbarth, NeonMerlin, Nigelj,
Nihiltres, Nikoladie, Nimur, Nivix, Novangelis, Oli Filth, Onionmon, Oxymoron83, Patrick, Peripitus, Phanerozoic, Philmcrae, Pmarshal, Pstudier, R'n'B, RandyBurge, ReCover, Recognizance,
Richard001, RichardVeryard, Riking8, Rinconsoleao, Robertmacl, Robthepiper, Roger Mexico, Rogerbrent, Ryan Roos, RyanGerbil10, Scwlong, Semiwiki, Slakr, Sokari, Spalding,
Spinningspark, Sporkot, Squidonius, Superm401, Swarm, Teapeat, Thinboy00P, Thorsten1, USMCSoftLove, VandalCruncher, Vivio Testarossa, WMCEREBELLUM, Walk&check, William M.
Connolley, Wimt, Winderful1, Wolfkeeper, Wotnow, Wtmitchell, Wtshymanski, Xcentaur, Yill577, Yuckhil, Yworo, 212 anonymous edits
Systems design Source: http://en.wikipedia.org/w/index.php?oldid=452494858 Contributors: Ahoerstemeier, Allan McInnes, Anaxial, Andreas Kaufmann, Architectchao, Arthena, Avalon,
Beber, Contemplator, Danhiel, Dominic, Dudboi, Eastlaw, Esoltas, FlyHigh, Freakofnurture, FritzSolms, G1langos, Grafen, H.ehsaan, Hadal, Hari, Imjustmatthew, Imran, Jcw69, Jerryobject,
Joy, Jpbowen, K.Nevelsteen, Keilana, Lawynholds, LightAnkh, LouH68, Mannamalai, Maqingpei, Mdd, Miker9asim, Msh210, NSR77, Nav102, Nscheffey, Rjwilmsi, Ruud Koot, SE SME,
Shaun F, Shenme, Shirik, SimonMayer, Stefanson, Tee Owe, Tonyshan, Toxickore, West.andrew.g, Wikipelli, Xyzzyplugh, 82 anonymous edits
Requirement Source: http://en.wikipedia.org/w/index.php?oldid=466427773 Contributors: 16@r, 9Nak, Aaronbrick, Abhinashak, AbsolutDan, Al Lemos, Allan McInnes, Andreas Kaufmann,
Architectchao, Beetstra, Bigglesjames, BirgitteSB, Bloodshedder, Cassbeth, Cheesy UK, Chiswick Chap, Chocmah, Chris Howard, Danikolt, Davewild, Dekisugi, Dhulis, Discospinster,
Ellengott, Epbr123, Ewlyahoocom, Fernandoacorreia, Furrykef, H-b-g, HappySneezy, Herbee, Iansommerville, Ijansch, Ilia Kr., Indon, Iridescent, ItsZippy, Jadadad, Jamelan, Janiejones56,
JathanBiddlecomb, Jc3, Jpbowen, Katalaveno, KelleyCook, Khalid hassani, Kuru, Lakeworks, Lcg.wda, Lgalumbeck, MPerel, Magioladitis, Mark Renier, Martarius, Mdd, Michael Hardy,
Michig, Mjchonoles, Mwfnwa, Nasrani, PedroLamarao, Ph4g3, Pm master, Pnoble805, Project2501a, Qweedsa, RL THe Joker, Rbarroso, Rich Janis, Robert Weemeyer, Ruud Koot, Salsb,
Samiroy, Scmbwis, Shahrome, Shambhaviroy, Shanemcd, Starrymessenger, Strellean, Struway2, Tasma3197, Thoughtpod, ThreePD, TimBentley, Triddle, Useight, Vincehk, Wknight94,
Wmahan, 146 anonymous edits
Systems architecture Source: http://en.wikipedia.org/w/index.php?oldid=464190956 Contributors: Allan McInnes, Andreas Kaufmann, Architectchao, Basar, Brighterorange, Cask05,
CoderGnome, Coreurus corporation, Craigb81, Creacon, Daduzi, Deemery, DonFiresmith, Dpv, Dvavasour, EagleFan, Eric-Wester, Erkan Yilmaz, FlyHigh, Graham Berrisford, Hans Adler,
Hetar, Horatio Huxham, Ian Pitchford, Iskander s, J04n, JaGa, Jhbendz, Jpbowen, Latiligence, Leutha, Marcika, Maria C Mosak, Mattisse, Mausy5043, Mdd, Michael Hardy, MichaelBillington,
Nav102, Normxxx, Onlygains, PaulBoxley, Pegej, Poor Yorick, Psychohistorian, Ronz, Ruud Koot, SEI Publications, Scope creep, Squids and Chips, Tedickey, Tribaal, Walter Görlitz,
Willardsimmons, Workingboy, YoavShapira, 35 anonymous edits
Function model Source: http://en.wikipedia.org/w/index.php?oldid=459132849 Contributors: Architectchao, Banana Concoction, Bender235, Chatkan Bunnag, Cpt2slow, EagleFan, Grafen,
J04n, Jncraton, John of Reading, LittleHow, M-le-mot-dit, Mark Renier, Mdd, Mr pand, Pallaverm, R'n'B, Rayhennessey, TeunSpaans, Topfix, WikHead, Woohookitty, 雁太郎, 10 anonymous
edits
314
Article Sources and Contributors
Function block diagram Source: http://en.wikipedia.org/w/index.php?oldid=468331594 Contributors: BrianOfRugby, EagleFan, Feyyaz, Jncraton, KSchutte, Karol Langner, KathrynLybarger,
Lmatt, Mdd, Nameless23, R'n'B, SeventyThree, Slearl, Spalding, Swen, Vonkje, Zabadab, Zundark, 9 anonymous edits
Transfer function Source: http://en.wikipedia.org/w/index.php?oldid=469624290 Contributors: A multidimensional liar, Ale2006, Ap, Attilios, AxelBoldt, BenFrantzDale, Brad7777, Bryan
Derksen, Colin Marquardt, Conversion script, Crazycasta, Cyrius, D1ma5ad, Dbroadwell, Dysprosia, Elwikipedista, Fresheneesz, Giftlite, Grubber, H2g2bob, HappySophie, HeroTsai,
Hypergeek14, Ihatetoregister, Imjustmatthew, Jiuguang Wang, Jorge Stolfi, Josh1238, Jwdietrich2, Kku, L Kensington, Light current, Linas, Maitchy, Mastermeister, Mckee, Mebden, Melcombe,
Metacomet, Michael Hardy, Mschlindwein, Mwilde, Mwtoews, NathanHagen, Nejko, Oarih, Oleg Alexandrov, Oli Filth, Omegatron, Rbj, Rhetth, Ro8269, Roadrunner, Rwestafer, Sagaciousuk,
SamuelRiv, Smack, Spinningspark, Splash, Sterrys, TedPavlic, The Anome, Thermochap, Tijfo098, Tlotoxl, Tobias Hoevekamp, Tuos, Vonkje, Wallers, Wowo53, Xlation, Zfeinst, 107
anonymous edits
Convolution Source: http://en.wikipedia.org/w/index.php?oldid=467161674 Contributors: 16@r, 1exec1, AhmedFasih, Aitias, Ajb, Akella, Almwi, Anonymous Dissident, Antandrus, Anthony
Appleyard, Ap, Arcenciel, Armin Straub, Auclairde, AxelBoldt, Bdiscoe, Bdmy, Belizefan, Ben pcc, BenFrantzDale, Berland, BigHaz, BjarteSorensen, Bob K, Boing! said Zebedee, Bombshell,
Bracchesimo, Btyner, CRGreathouse, Carolus m, CaseInPoint, Cdnc, Charles Matthews, Chato, Chowbok, Coco, Coffee2theorems, Comech, Conversion script, Crisófilax, Cronholm144,
CryptoDerk, Cwkmail, Cyp, D1ma5ad, Daniele.tampieri, DaveWF, Dbfirs, December21st2012Freak, Dicklyon, DirkOliverTheis, Dmmaus, Dmn, Dpbsmith, Drew335, Dysprosia, Dzordzm,
Edward Z. Yang, Eleassar, Emc2, Emote, Esoth, Feraudyh, Flavio Guitian, Frozenport, Furrykef, GabrielEbner, Gareth Owen, Gene Ward Smith, Giftlite, Giuscarl, Greenec3, Greg newhouse,
H2g2bob, Hankwang, Headbomb, Helder.wiki, Hellbus, HenningThielemann, Hippasus, Hurricane111, Hyacinth, IMSoP, Inductiveload, Intelliproject, JSKau, Jeff G., JethroElfman, Jevinsweval,
Jschuler16, Jshadias, Justin545, Kallikanzarid, Karada, Kazkaskazkasako, Kelly Martin, Kenyon, Kielhorn, Kku, Konradek, Kurykh, Lamoidfl, Larvy, Lautarocarmona, Liotier, Lklundin, Loisel,
Looxix, Loren Rosen, Lovibond, Lwalt, MFH, Madmath789, MarcLevoy, Matteo.Bertini, Mckee, Mct mht, Melchoir, Michael Hardy, Mild Bill Hiccup, Minna Sora no Shita, Mircha, Mitrut,
Miym, Mmendis, Modster, Mojodaddy, Mormegil, Mpatel, MrOllie, Mschlindwein, Mwilde, N.MacInnes, Naddy, Nbarth, Neffk, Ngchen, Nuwewsco, Odedee, Oleg Alexandrov, Oli Filth,
Omegatron, Orderud, Ortho, Oski, P4lm0r3, PMajer, Patrick, Paul Rako, Perey, Phys, Pinethicket, Pion, PowerWill500, Rade Kutil, RapidR, Rlcook.unm, Robert K S, RobertStar20, Rogerbrent,
Rubybrian, Schneelocke, Seabhcan, SebastianHelm, Shanes, Silly rabbit, Slawekb, Smack the donkey, Smithwn, Stevenj, SuburbaniteFury, Sullivan.t.j, Sławomir Biały, Tarquin, Tblackma222,
TheObtuseAngleOfDoom, Thenub314, Tinos, Tomash, User A1, UtherSRG, Vuo, Waqas946, Wayne Slam, Wikomidia, WojciechSwiderski, Wwoods, Wxm29, Xe7al, Yaris678, Zoltan808,
Zvika, 294 anonymous edits
Dynamical system Source: http://en.wikipedia.org/w/index.php?oldid=468735819 Contributors: 0, 195.186.254.xxx, 7&6=thirteen, Aaronp808, Adam majewski, Aleksandar Guzijan, Aliotra,
Altenmann, Americanhero, AntOnTrack, Ap, Athkalani, AxelBoldt, Bluemoose, Brazzouk, CBM, CX, Caesium, Charles Matthews, Chetvorno, Chopchopwhitey, Complex01, Complexica, Cumi,
Cutler, Daniele.tampieri, Dhollm, Dinhxuanduyet, Dino, Dmharvey, Dratman, Duoduoduo, Dysprosia, EPM, El C, Epbr123, Epolk, Everyking, Evilphoenix, Filemon, Filur, Finn-Zoltan,
Frederick.d.abraham, Fredrik, Gandalf61, Giftlite, Graham87, Headbomb, Hesam7, Highlightened, Hve, Hydroli, Iamthedeus, Jabernal, Jay Gatsby, Jeff3000, Jeffrey Smith, JerrySteal, Jitse
Niesen, JocK, Jowa fan, Jugander, K-UNIT, KYPark, Karol Langner, Kayvan45622, Kenneth M Burke, Kotepho, Kzzl, LaHonda MBA, Lakinekaki, LeilaniLad, Levineps, Lightmouse, Linas,
LokiClock, ManiacK, Marj Tiefert, Math.geek3.1415926, MathMartin, Mathmanta, Mct mht, Mdd, Meersan, Michael Hardy, Milly.mortimer, MrOllie, Msh210, Myasuda, Neelix, Neruo, Nick
Number, Nn123645, Noeckel, Oleg Alexandrov, Orange Death, OrgasGirl, Paine Ellsworth, Patrickdepinguin, PetaRZ, Pgan002, Phys, PlatypeanArchcow, Profyorke, RDBury, RedWolf, Reddi,
Reinderien, Revolver, Rhetth, Rhythmiccycle, Rich Farmbrough, Rintrah, Rjwilmsi, Roesser, Rschwieb, SEIBasaurus, Sadi Carnot, Salgueiro, Salix alba, Sam Korn, Samuelbf85, Schmloof,
SilverSurfer314, Snoyes, Solace098, Sverdrup, Tedder, Template namespace initialisation script, The Anome, The wub, Tiled, Tobias Hoevekamp, Tomisti, Tommyjs, Torcini, Tosha, Ttrantow,
Volfy, Voretus, Waitati, WaysToEscape, Wclxlus, WhiteC, WillowW, XJamRastafire, XaosBits, Yellow octopus, Zsniew, 147 anonymous edits
Graph dynamical system Source: http://en.wikipedia.org/w/index.php?oldid=418085477 Contributors: Charvest, Docu, Eslip17, Harryboyles, Henning.Mortveit, Jim.belk, Jyoshimi, Mhym,
Michael Hardy, 3 anonymous edits
First-order logic Source: http://en.wikipedia.org/w/index.php?oldid=469501602 Contributors: .digamma, 34jjkky, Ahmed saeed, Ahouseholder, Alastair Haines, Almit39, Apolkhanov, Arthur
Rubin, AshtonBenson, AugPi, Avakar, AxelBoldt, Axl, Banazir, Be hajian, BenBaker, Bender235, Blaisorblade, Bookandcoffee, Borgx, Brick Thrower, CBM, CBM2, CRGreathouse, Cacadril,
Caesura, Carbo1200, Cdills, Chalst, Charles Matthews, Childoftv, Chinju, Classicalecon, Cloudyed, ConcernedScientist, Conversion script, Creidieki, Cronholm144, Crowne, Dan Gluck,
Danman3459, David Eppstein, David.Monniaux, Dbtfz, Dcoetzee, Dhruvbaldawa, Dhruvee, Diannaa, Djk3, Dkf11, Dpol, Dwheeler, Dysprosia, ELDRAS, Ejars, Ekspiulo, Eleuther,
Elwikipedista, EmilJ, English Subtitle, Eubulide, Exostor, Expensivehat, Filemon, Fl, Foxjwill, Francvs, Frecklefoot, Fredrik, Gf uip, Gherson2, Giftlite, Greenrd, Gregbard, Grim23, Gubbubu,
Hakeem.gadi, Hans Adler, Harburg, Henning Makholm, Heyitspeter, Hilverd, History2007, Histre, Hobsonlane, Hq3473, Ht686rg90, Iannigb, Igodard, Inquam, Iridescent, Ixfd64, JRSpriggs,
Jan1nad, Jay Gatsby, Jayme, Jleedev, Jod, Jon Awbrey, Jorend, Joriki, Jsnx, Juansempere, Julian Mendez, JulianMendez, Karl-Henner, Katieh5584, Kelly Martin, Kendrick Hang, Kgoarany, Kim
Bruning, Kku, Klausness, Kubigula, Kumioko, Kusunose, Kwertii, LBehounek, Lambiam, Lauri.pirttiaho, Lethe, Linas, Ljf255, Loadmaster, Looxix, Lord British, Lucidish, MarSch, Mark
Renier, Marner, Meloman, Mets501, Mhss, Michael Hardy, Michael Slone, Mike Fikes, Mike Segal, Mild Bill Hiccup, Minesweeper, Mistercupcake, Mormegil, Mpeisenbr, Msh210, Nahaj,
Nanmus, Nanobear, NavarroJ, NicDumZ, Nick Number, NoBu11, Norman Ramsey, Nortexoid, Nzrs0006, Obradovic Goran, Oleg Alexandrov, Omphaloscope, Onceler, Otto ter Haar, Palnot,
Patrick, Paul August, Pcap, Pdibner, Penumbra2000, Per Olofsson, Philogo, Phuzion, Physis, Phyte, Pkrecker, Pomte, R.e.b., RG2, RJaguar3, Randall Holmes, Randomblue, Reach Out to the
Truth, Rich Farmbrough, Richard L. Peterson, Rjwilmsi, Rlcolasanti, RobHar, Sampletalk, Saric, Siroxo, SixWingedSeraph, Slaniel, SouthLake, Spayrard, SpigotMap, Spug, Starlord, Stevertigo,
Subversive.sound, TechnoFaye, Tesseran, The Anome, The Tetrast, Tigranes Damaskinos, Tijfo098, Timwi, TitusCarus, Tizio, Tkuvho, ToastieIL, Tobias Bergemann, Trovatore, Turms,
Utcursch, Vanden, Velho, VictorAnyakin, Virago250, Voorlandt, Whitepaw, WikHead, Willhig, Wireless99, Xplat, Youandme, Zeno Gantner, Zero sharp, 265 ,.‫ דניאל ב‬anonymous edits
List of first-order theories Source: http://en.wikipedia.org/w/index.php?oldid=469791986 Contributors: Ben Standeven, Bernard the Varanid, Bwhack, CBM, Charles Matthews, DA3N,
Dmharvey, EmmetCaulfield, FlyHigh, GabrielEbner, Giftlite, Gregbard, Hans Adler, JackSchmidt, Knotwork, Lethe, Madmardigan53, Magmalex, ManoaChild, Mcpk, Mets501, Michael Hardy,
Ott2, R.e.b., Rjwilmsi, RobHar, Shellgirl, SomeStranger, SunCreator, Tesseran, Trovatore, Uncia, Wmahan, Ynhockey, ZeroOne, 12 anonymous edits
Pole splitting Source: http://en.wikipedia.org/w/index.php?oldid=356754552 Contributors: BillC, Brews ohare, Circeus, Giftlite, Michael Hardy, Nbarth, Spinningspark, Toffile, WakingLili, 6
anonymous edits
Damping ratio Source: http://en.wikipedia.org/w/index.php?oldid=470153952 Contributors: ASSami, AndrewDressel, Apoorvajsh, Barkman, BillC, Btyner, Cheeto81, Craigerv,
Daniele.tampieri, Dannya222, Dicklyon, EdJogg, Gene Nygaard, Giftlite, HappyCamper, Jgreeter, MarSch, Michael Hardy, Nbarth, Nmnogueira, Oleg Alexandrov, PigFlu Oink, Remuel,
Robobeg, Salsb, Sanpaz, Smit, SyntheticDNA, Teapeat, TedPavlic, Tlee123, Warut, Yeokaiwei, 41 anonymous edits
Second-order logic Source: http://en.wikipedia.org/w/index.php?oldid=468830873 Contributors: Aminorex, AshtonBenson, AugPi, BrideOfKripkenstein, CBM, Chalst, Charles Matthews,
Chinju, Chira, Chris the speller, Creidieki, David Eppstein, Dcoetzee, Delirium, DividedByNegativeZero, Dysprosia, E-boy, EdJohnston, EmilJ, Eric Yurken, Ericbodden, Giftlite, Glaucus,
Gregbard, Gruepig, Hairy Dude, Iannigb, Inego, JRSpriggs, Jamelan, Jason Quinn, Jkcmsal, Joriki, Julian Mendez, Kan8eDie, Knight Of The Woeful Countenance, Kumioko, LOL, Lipedia,
Lpgeffen, Mets501, Michael Hardy, Michaeljsouth, Mistercupcake, Modify, Nortexoid, Ntmatter, Oleg Alexandrov, Omnipaedista, Ott2, Postcrypto, Rich Farmbrough, Rjwilmsi, Root4(one),
Russell C. Sibley, Salgueiro, Sapphic, Sergey Dmitriev, Shellgirl, Tesseran, That Guy, From That Show!, The Tetrast, Wclark, Zero sharp, 44 anonymous edits
Impulse response Source: http://en.wikipedia.org/w/index.php?oldid=442978665 Contributors: Adoniscik, Ahoerstemeier, Aluvus, Anjosacruca, Beeline bc, BenFrantzDale,
Benjamin.friedrich, Binksternet, Bob K, Bobblewik, CodyARay, Colonies Chris, Dicklyon, Firefly322, Giftlite, Iain, Jag123, Jerryfishenden, Karatsobanis, Kvng, LRG, Lightmouse, Lindosland,
Magioladitis, Melcombe, Metacomet, Michael Hardy, Msa11usec, Mwilde, Nbarth, Nolaiz, Oleg Alexandrov, Oli Filth, PeterEmanuelRoos, Phase Theory, Rinconsoleao, Ro8269, Rs2, SUL,
Spectrogram, Stephenellwood99, Voyaging, Wolfkeeper, 62 anonymous edits
LTI system theory Source: http://en.wikipedia.org/w/index.php?oldid=465030887 Contributors: Batsmacks, BenFrantzDale, Bob K, Bodrell, CBM, Cbs228, Cihan, Cxw, DerHexer, Dicklyon,
Gbruin, Giftlite, Grebaldar, GreyCat, Infzy, Interferometrist, J04n, JABoye47x, Jamelan, Jpkotta, Juansempere, Kb9wte, Mako098765, Marlynboke, Mdd, Melcombe, Metacomet, Michael Hardy,
Mmeijeri, Mwilde, Natalya, Nejko, Nish-NJITWILL, Niteowlneils, No-body, Nuwewsco, Oli Filth, Onkelringelhuth, Rbj, Rsrikanth05, Schizodelight, Sonick, Splash, Sterrys, TedPavlic,
ThePianoMan, Toffile, Zvika, 62 anonymous edits
Routh–Hurwitz stability criterion Source: http://en.wikipedia.org/w/index.php?oldid=469091140 Contributors: Becritical, Billymac00, Bletchley, Brews ohare, Catslash, Charles Matthews,
Dicklyon, Djenicek, Duncharris, Ekotkie, FourBlades, Fweeky, Giftlite, Insertesla, Jitse Niesen, Jiuguang Wang, Julien Tuerlinckx, Kevinsane, Kmellem, Linuxlad, Mdd, Mdupont, Michael
Hardy, Nanasid, Novangelis, Olaf, Oleg Alexandrov, Pelotas, Spradlig, Tbsmith, Titoxd, User A1, Vivek7de, Zaxxonal, 48 anonymous edits
Control theory Source: http://en.wikipedia.org/w/index.php?oldid=469715053 Contributors: A Sengupta, Absurdburger, Adam Keller, Aleksandar Guzijan, Americanhero, Anakata, Ancheta
Wis, Ap, Arman Cagle, Aschwarm, Attilios, Aushulz, Bact, Barak, BenFrantzDale, Benbest, Billymac00, Bn, Bob12794, Boden Rodenberger, Brad7777, Brews ohare, Bullhaddha,
CRGreathouse, Canglesea, Capricorn umair, Cburnett, Ceyockey, Cha'yu, Chandu15, ChangChienFu, Chimin 07, Chongkian, ChristianSchluchter, Complexica, Constant314,
Control.optimization, Conversion script, Copeland.James.H, Coronafire, Cvgittings, D1ma5ad, D6, DARTH SIDIOUS 2, DCwom, Dan Polansky, Daniel Kellis, Daniel Sperry, David Pardy,
Dedmonds, Didier wkpd, Dinhxuanduyet, Dja25, DrJunge, Dudered, EUvin, EmreDuran, Engelec, Erik the Red 2, Erkan Yilmaz, Euyyn, EverGreg, Fenice, First Harmonic, Frederick12,
Frenchwhale, GRAHAMUK, Gaius Cornelius, Gandalf61, Gauge, GeneralAlgernon, GeorgeLouis, Giftlite, Glenn, Gnusbiz, Graham87, GregorB, Gwernol, Hammertime, Hazard, Heron, Holnet,
IPSOS, Ignacioerrico, Ignignot, Ikelos, Jamesontai, JanHRichter, Jbergquist, Jeff3000, Jimmaths, Jiuguang Wang, Jiy, Jmdaly, John43ny, JohnBlackburne, Jpbowen, Jpkotta, Jugander, Julien
Tuerlinckx, Jwdietrich2, Kdcarver, Khukri, Kku, Kurochka, Kurykh, Lexor, Lightmouse, Lommer, LuckyWizard, Magmi, Mani excel, Marek69, Markeditwiki, Marklar2007, Martin451, Mastlab,
315
Article Sources and Contributors
Mathmanta, Matt Gies, Mbbradford, Mdd, Michael Hardy, Mickraus, Mindmatrix, Mmeijeri, Mononoke, Mrdarrett, Nbarth, Neonil, Netlad, Oleg Alexandrov, Oli Filth, Onco p53, Pak21, PeR,
Phmoreno, PierreAbbat, Pmg, Poor Yorick, Poree, Ppntori, Pulkitgrover, Ralesk, Rawling, Regancy42, Revolver, Riceplaytexas, Rich Farmbrough, Rjwilmsi, Ro8269, Rod57, RoyKok,
Royalguard11, Rs2, S Roper, Sahodaran, Sattyam, Scott Dial, Seb, Seh444, Serrano24, Shadowjams, Sigmundg, Sigmundur, Simetrical, Simoneau, Siroxo, Someguy1221, Spalding, Spike
Wilbury, Sponge, Spradlig, StephenReed, Style, Suruena, Swrkoab, Tailboom22, Tamtamar, Taral, TedPavlic, The Anome, The Transhumanist, Thelctrclengnr, Theraft, Tide rolls, Tilin, Titoxd,
Tommy2010, Torzsmokus, Tupeliano, Tuxa, User A1, Vonkje, Vsh3r, Wavelength, Wknight94, Wmahan, Wojciech.ziniewicz, XieChengnuo, Yamamoto Ichiro, Ykhwong, Yuriybrisk, Zander
the Mander, ‫ﺳﻌﯽ‬, 312 anonymous edits
Control system Source: http://en.wikipedia.org/w/index.php?oldid=469294718 Contributors: 28421u2232nfenfcenc, Aasdasdas, Abarry, Ahmedsaieed, Alansohn, Andres, Angela, Anna
Lincoln, AugPi, Beaufanamus, Benbest, Biancasimone, Bruyninc, Buhrerf, Can't sleep, clown will eat me, Chrisdotb, Christophenstein, Chzz, Circuit dreamer, Crunchy Numbers, Cybercobra,
D1ma5ad, Darigan, Data8504, Daz 90, Dicklyon, Dmsar, Dougofborg, Edcolins, Engelec, Engineer Bob, Epbr123, Erkan Yilmaz, Excirial, Firien, Frappucino, Gary Cziko, Gepwiki, Giftlite,
Glenn, Groogle, Gurch, Haipa Doragon, Iautomation, Intelliw, JHunterJ, Jbrasla, Jiang, JoanneB, Joel Saks, Joeschmoe35, Jpbowen, Kanonkas, KarenW, Keegscee, Kku, Klilidiplomus,
Kmellem, KnightRider, Krellis, Krish matlab, Kyle01 is, L Kensington, L337p4wn, Lambiam, Light current, LittleOldMe, M@sk, MER-C, Masgatotkaca, Mbeychok, Mdd, Mentifisto,
Mindmatrix, Mishlai, MrOllie, My Username is..., Nbarth, Nick, Nigelj, Nirvana888, OmarNJITWILL, Palica, Passw0rd, PatrickJCollins, Pauldouglas72, Paulpw, PeaceNT, Petedarnell, Pnm,
Pyrophilliac, Radagast83, Rafmag, RazorICE, Rdhunkins, Reaper Eternal, Recurring dreams, Redvers, Rjwilmsi, Rob 52240, Roman12345, Rror, S Roper, SCH56, SchreyP, Shell Kinney, Softy,
Spalding, Spinningspark, Spradlig, Srinivasasha, StaticGull, StephenReed, Swrkoab, Tabletop, TedPavlic, The Thing That Should Not Be, Ttwaring, User A1, Vonkje, Vrenator, Wdl1961,
Wintonian, Xionbox, Yk Yk Yk, Zander the Mander, ‫ﺳﻌﯽ‬, 250 anonymous edits
Controller (control theory) Source: http://en.wikipedia.org/w/index.php?oldid=469506307 Contributors: Archelon, Attilios, Aushulz, Benbest, CALR, Caporaletti, Coppertwig, DJ LoPaTa,
Drtelnet, Duoduoduo, EagleFan, Eightkinarrows, Enboifre, Gaius Cornelius, Icairns, Joechao, Jpbowen, JustAGal, Jwdietrich2, K3rb, Marlysse, Mellery, Mikeblas, Palica, RJFJR, Robofish,
Robosorne, Rod57, TheTito, Think outside the box, Tom, ViM, Vonkje, ‫ﺳﻌﯽ‬, 45 anonymous edits
Process control Source: http://en.wikipedia.org/w/index.php?oldid=468297917 Contributors: Attilios, Aushulz, AutomationBooks, Ayrton Prost, Ben Ben, Billymac00, BjKa, Bob, COGDEN,
Caesura, Cflm001, Charles Matthews, Chemical Processing, Chrisportelli, Citect, Control.optimization, Cvgittings, Dysprosia, E Wing, Engelec, Ewlyahoocom, Finejon, Fubar Obfusco, GB fan,
Giftlite, Grandviewmedia, HaPi, Heron, J04n, Jerry, Jiuguang Wang, Jmuenzing, Jschwa1, Kiefer.Wolfowitz, Kwill, Materialscientist, Mdd, MegaSloth, Melcombe, Michael Hardy, Mojtaba
Cazi, Mschel, Mydogategodshat, N4AEE, Netlad, Nsaa, PeterWoolf, Phmoreno, Pimlottc, Pm master, Pugges, Pumpmeup, Ramanraj, Reconsider the static, Requestion, RichardVeryard,
Rifleman 82, Ripogenus77, Rlsheehan, RockMFR, Ronz, RoyKok, Sciurinæ, Scjessey, Scott white85, Search255, Smalljim, Spalding, Spallen, SteffiKl, Stevenknudsen, SuperMarioMan,
Supersteve04038, Swrkoab, Tagishsimon, Timwi, Tnxman307, Unregistered.coward, User A1, Uthbrian, VQuakr, Vdegroot, Weetoddid, WhiteOakTree, Wtshymanski, Zhenqinli, 96 anonymous
edits
Proportionality (mathematics) Source: http://en.wikipedia.org/w/index.php?oldid=469841540 Contributors: 28421u2232nfenfcenc, Aidepikiw0000, Alansohn, Alberto da Calvairate, Andres,
Aniten21, Asifkemalpasha, Audiosmurf, AugPi, Aughost, Auron Cloud, AxelBoldt, Baseball Watcher, Belajane123, Brandon, Btyner, CSTAR, Capitalist, Capricorn42, CarbonUnit,
Cbowman713, Charles Matthews, Chris the speller, Commander Keane, Constructive editor, Cybercobra, DARTH SIDIOUS 2, Dadude3320, David R. Ingham, Dcirovic, Dendodge, DerHexer,
Dhollm, Dicklyon, Dmyersturnbull, Ecemaml, Eighty, Enviroboy, Evan.oltmanns, Ewen, Excirial, Fdskjs, Finell, Funandtrvl, Fuzzform, Giftlite, Gigabyte, Hazard-SJ, Heron, Hq3473, In actu,
Inflector, Irbisgreif, Isheden, Iwilllearn2010, Ja 62, JaGa, Jacobusmaximus34, JerroldPease-Atlanta, Jerzy, Jncraton, Jusdafax, Jóna Þórunn, Kaihsu, Kjarda, Kozuch, Krellis, Kuddle143, La
goutte de pluie, Levineps, Luk, Marc Venot, Marek69, Mayfare, Miccoh, Michael Hardy, Mindlurker, Mpatel, Mutt20, Nbarth, Netalarm, Nicoli nicolivich, Noodle snacks, Omegatron,
Oxymoron83, Paste, Patrick, Paul August, Pfctdayelise, Pjvpjv, Pred, Pstanton, Randywombat, Rgdboer, Rjd0060, Sakkura, Scottish lad, Skater, Sohmc, Sonett72, SpaceFlight89, Spinningspark,
Spoon!, TBster2001, TejasDiscipulus2, The Pikachu Who Dared, The Thing That Should Not Be, Theoprakt, Tide rolls, Tobias Bergemann, Tom harrison, Tommy2010, Tompw, Traxs7, Tricore,
Viridae, WeijiBaikeBianji, WikHead, Willking1979, Yeungchunk, Zonuleofzinn, 百家姓之四, 217 anonymous edits
Integral Source: http://en.wikipedia.org/w/index.php?oldid=469216818 Contributors: 129.174.59.xxx, 4C, 6birc, A bit iffy, Ace Frahm, Acegikmo1, Admiral Norton, Adrruiz, Ahknor, Ais523,
Alansohn, Aleksander.adamowski, Alexius08, Alsandro, Aly89, Amahoney, AnOddName, Andrei Stroe, Andrew Moylan, Andrewcmcardle, Anonymous Dissident, Antillarum, Apokrif, Arcfrk,
ArnoldReinhold, Arunirde, AxelBoldt, Azuredu, [email protected], BarretB, Bdesham, Bdmy, Beefman, Bemoeial, BenFrantzDale, Benjaminwill, Benzi455, Berria, Bethnim, Bnitin, Bo
Jacoby, Bobo192, Bomac, Borb, Boreas231, Bovineone, Brad7777, Brews ohare, Brufydsy, Bsmntbombdood, Butko, CSTAR, CalebNoble, Calltech, Caltas, Calvin 1998, Capefeather, Caramdir,
Cardamon, Cassini83, Cat2020, Catgut, Cemenarist, Centrx, Cflm001, Chadernook, Chait1027, Charles Matthews, Chetvorno, Chinju, ChrisHodgesUK, Ciko, Closedmouth, Conversion script,
Crakkpot, Cronholm144, D.Lazard, DHN, Da nuke, DabMachine, Daryl Williams, Davewild, DavidCBryant, Dchristle, Den fjättrade ankan, Der Spion, Diberri, Dima373, Discospinster, Diza,
Djradon, Dmcq, Dmitrey, Doctormatt, Dojarca, Drdonzi, Dugwiki, Dysprosia, E2eamon, Eagleal, Edward, Edward Knave, Einsteins37, ElTchanggo, Electron9, Emaalt, Emet truth, Enochlau,
Eomund, Epbr123, Espressobongo, Euty, Evil saltine, Favonian, Ferengi, FerrousTigrus, Filemon, FilipeS, Fintor, FractalFusion, FrankTobia, Franklin.vp, Fraqtive42, Fredrik, Frokor, Fulvius,
Fyyer, Gadykozma, Gandalf61, Garo, Gary King, Geometry guy, Gesslein, Giftlite, Glaurung, Glenn, Gmcastil, Gnixon, Goethean, Greg Stevens, HairyFotr, Hajhouse, Hakeem.gadi,
Hakkasberra, Hal0920, Herald747, Heron, Hotstreets, Hrafeiro, Icairns, Ider88, Igny, Inter, Introareforcommonpublic, Iridescent, Isheden, Iulianu, Ivan Štambuk, JB Gnome, JForget, JKBlock,
JRSpriggs, Jackfork, Jagged 85, JakeVortex, Jakob.scholbach, Jalesh, Jdlambert, Jfgrcar, Jim.belk, Jitse Niesen, Johnlemartirao, JonMcLoone, JonezyKiDx, Jose77, Josh dos, Jugander, Jynus,
KSmrq, Kapoor Amit, Karada, Karimjb, Karol Langner, Katzmik, Kawautar, Keegan, Kevin Baas, KhoousesWiki, KieferSkunk, Kiensvay, King Bee, Kingpin13, Kirbytime, Knakts, Kntrabssi,
Kumioko, Kurykh, Kusunose, Kwantus, Kyle1278, LOL, Lambiam, Leland McInnes, Lethe, Levi.vaieua, LiDaobing, Light current, Lightdarkness, Lindberg G Williams Jr, Lir, Loisel, Loom91,
Luís Felipe Braga, MC10, MJBurrage, MONGO, Madmath789, Magioladitis, Marek69, MarkSweep, Materialscientist, Matqkks, Matsoftware, Maxvcore, Mcld, Mcorazao, Mecanismo, Melchoir,
Mets501, MiNombreDeGuerra, Michael Hardy, MightyBig, Mikael Häggström, Mike2vil, Mindmatrix, Minestrone Soup, Miquonranger03, Momo san, Mormegil, MrOllie, Ms2ger, Mtz1010,
MuZemike, Mìthrandir, Nbarth, Nickrds09, Nikai, Nitya Dharma, Nnedass, NonvocalScream, Obeattie, Oleg Alexandrov, OrgasGirl, Ourhomeplanet, Ozob, PMG, Paolo.dL, Patrick, Paul
August, Paul Matthews, Paxsimius, Pcb21, PericlesofAthens, PhilipMW, Phillip J, PhySusie, Physicistjedi, PiMaster3, Pie4all88, Planetary Chaos Redux, Plasticup, Point-set topologist,
Pooryorick, Programmar, Python eggs, Quuxplusone, RDBury, RJFJR, Raamin, Radomir, Rama's Arrow, Randomblue, Raven4x4x, Razorflame, RedWordSmith, Rich Farmbrough, Ringbang,
Rjwilmsi, Rklawton, Robbyjo, Roboquant, Rracecarr, Rubybrian, SJP, Saforrest, Salix alba, Sankalpdravid, Satori42, Schneelocke, Schoolscienceteacher, Scott MacLean, Sdornan, Seaphoto,
Seresin, Shadowjams, Shipmaster, Showgun45, Siddhant, Silly rabbit, Slakr, Sligocki, Smithpith, Snigbrook, Sodin, Somedaydoctor, Someguy1221, Spellcast, Splashkid2, Ssd, StaticGull,
Stca74, Stevertigo, Stlrams22, StradivariusTV, Stw, Super-real dance, Sushant gupta, Susurrus, Sławomir Biały, TJRC, TStein, Tabletop, Tbsmith, Template namespace initialisation script, The
Anome, Thegeneralguy, Thenub314, Theshadow27, Thiseye, Tjdw, Tkuvho, Tobby72, Tobias Bergemann, Tomyumgoong, Topology Expert, TroyBurm, Twp, Ukexpat, Urdutext, VKokielov,
Van helsing, Vanished User 0001, Velvetron, Viames, Vladislav Pogorelov, Vrenator, Waabu, Waltpohl, Wik, Wile E. Heresiarch, Wilsonater23456, Wimt, WinterSpw, Witchinghour,
Wknight94, Wood Thrush, Wordsoup, Wtshymanski, Xantharius, Yacht, Ybbor, Yosha, Youandme, Your Lord and Master, Yuyudevil, Zfr, Zoicon5, ZooFari, Zundark, 510 anonymous edits
Derivative Source: http://en.wikipedia.org/w/index.php?oldid=468104358 Contributors: 10p12a0195, 123Mike456Winston789, 127, 24.44.206.xxx, A. Parrot, A876, ABCD, AbcXyz, Acannas,
Ace Frahm, Ahmad87, Akshaysrinivasan, Alex Bakharev, Algebraist, AllTheThings, AllyUnion, Aly89, Andre Engels, Andrejj, Andres, Anonymous Dissident, Anwar saadat, ArglebargleIV, Art
LaPella, Arthur543, Ash4Math, AstroHurricane001, Audiovideo, AugPi, AvicAWB, Aw.rootbeer1, AxelBoldt, AySz88, Az1568, AzaToth, Azuredu, BJjake, Baccyak4H, Baprameya, Barak Sh,
Bcrowell, Bdesham, Bdmy, Ben b, BenFrantzDale, Bethnim, Bethpage89, Bhxho, BiT, Binary TSO, Blaze2010, Bo Jacoby, Bobianite, Bobo192, Bomac, Boothy443, Brian Tvedt, Bryan
Derksen, Bsadowski1, Bsmntbombdood, Buckyboy314, Burk, Buttseckks, CBM, CSTAR, Caesura, CanadianLinuxUser, Canthusus, Cataclysm12, Catgut, Celestianpower, Cenarium, Ceyockey,
ChP94, Charles Matthews, Charvest, Cheeser1, Chenyu, ChrKrabbe, Chrishmt0423, Chrisportelli, Chuck SMITH, Cj67, Cje, Cobaltcigs, Complex (de), Conversion script, Cooolsduudes,
Courcelles, Crisófilax, Cronholm144, Cyp, DGaw, DHN, Daniel.Cardenas, Darrel francis, DarrylNester, Den fjättrade ankan, Deor, DerHexer, Dialecticas, Dino, Dmcq, Dmharvey, Doctormatt,
Dojarca, Dolphin51, Dotancohen, DrBob, Drmies, DroEsperanto, Dtm142, DuKot, Duoduoduo, Dysprosia, ENIAC, EconoPhysicist, Elektron, Eman2129, Enigmaman, Enviroboy, Epbr123,
Eric119, Erik Sandberg, Error792, Essjay, Estudiarme, Evan2718281828, Everyking, Evil saltine, Evolauxia, Extransit, Ezh, Fazan1983, Fcelikler, Fintor, Foobarnix, Foxandpotatoes, Fredrik,
Fresheneesz, Frickylick, Fsiler, GTBacchus, Gabbe, Gabn1, Gagueci, Garrooney, Gary King, Geometry guy, Georgia guy, Gesslein, Giftlite, Gilliam, Ginkgo100, Glenn, Gmaxwell, Graham87,
Greg von greg, Guardian of Light, Gurch, Haham hanuka, Hashar, Headbomb, Heimstern, HenningThielemann, Hephaestos, Heron, Hoolycheeese, Hteen, Iamthedeus, Icairns, Ideyal, If I Am
Blocked I Will Cry, InkKznr, Innerproduct, IstvanWolf, Iulianu, Izno, JAn Dudík, JB82, JRSpriggs, Jacj, Jackzhp, Jacobolus, JamesBWatson, Janus Shadowsong, JeLuF, Jeff G., Jim.belk,
Jimothytrotter, Jitse Niesen, Jixzad123, Jketola, Joey-das-WBF, John254, JohnOwens, Johnuniq, Jon Harald Søby, Jonathan48, Josh dos, Jrtayloriv, Jshadias, Jsjsjs1111, KSmrq, Kaimbridge,
Karol Langner, Katzmik, Kbolino, Keilana, Keithcc, Kerdek, Keryk, Kevin Saff, King Bee, KleinKlio, KnowledgeOfSelf, Knutux, Kocher2006, Konradek, Korovyev, Kourtney88, Kozuch,
Kpennington82, Kukini, Kumioko, Kurykh, Kwantus, Lethe, Lewallen, Lightmouse, Livius3, Loisel, Loodog, Lorrybizzle314159, Lotje, Ltmboy, Lynxmb, MC10, MStankie, Mac, Mac Davis,
Madhero88, Maelin, Magister Mathematicae, MagnaMopus, Mandavi, Mani1, Manulinho72, MarSch, Marc Venot, Marek69, Markjoseph125, Marquez, Masterofpsi, Materialscientist,
MatrixFrog, Maurice Carbonaro, Mboverload, Mcorazao, Melchoir, Melongrower, Metacomet, Michael Hardy, Mike2vil, Mindmatrix, MisterSheik, Mo0, Mormegil, Mountain, Mouse20080706,
Mpatel, Mphilip1, Mr Ape, MrOllie, Ms2ger, Nahum Reduta, Najoj, Nandesuka, Nbarth, NeonMerlin, Netrapt, Newone, NickBush24, Nickrds09, No Guru, Nobaddude, Noel Bush, Nonagonal
Spider, ObserverFromAbove, OlEnglish, Oleg Alexandrov, Omegatron, Onebravemonkey, Orderud, Oscabat, Ozob, Palfrey, Paolo.dL, Pascal.Tesson, Patrick, Paul August, PaulSmith641,
Pbroks13, Pcb21, PeteX, Peterwhy, Phgao, Phillip J, Pinkadelica, Pizza Puzzle, Pmanderson, Polx, Populus, Potatoswatter, Poulter7777777, Powowatushtoosha, Programmar, Pseudolus42, Qef,
Quadrescence, Quondum, Qz, R3m0t, RDBury, RMFan1, Ralian, Ramcdpai, Rdsmith4, Realmestup, RedWolf, Revolver, Rhyshuw1, Ricardo sandoval, Rick Norwood, Ritchy, Rjwilmsi,
Robbjedi, Rosasco, Rossami, Rrenner, Ruud Koot, Rx5674, SJP, SQGibbon, Salix alba, Sanu.soysa, Saotomi5102, Saretakis, Scientific29, Scifitechie360, Sergiobgaspar, Shellgirl, Shellreef,
Sigmalmtd, Silly rabbit, Simetrical, Simfish, Skizzik, Sky Attacker, SkyMachine, Slakr, Sligocki, SlipperyHippo, Slysplace, Smithpith, Snaxe920, Sonett72, Spartan S58, Srleffler, Starr sam,
Stevertigo, Stomme, StradivariusTV, Stuart07, Stw, Sukolsak, SunCreator, Sushant gupta, Svick, Sławomir Biała, Sławomir Biały, THEN WHO WAS PHONE?, TStein, TakuyaMurata,
Tammojan, Tcncv, Tcnuk, Template namespace initialisation script, Terminaterjohn, Tewy, Thalesfernandes, Thariath1, The Anome, The Font, The Wordsmith, TheNightFly, TheRanger,
TheTito, Theda, Theunixgeek, ThirdParty, Tholme, Thomasmeeks, Tiddly Tom, Timothy Clemans, Timwi, Titoxd, Tkuvho, Tobias Bergemann, Tomaxer, Tosha, Trombe29, Turenar, Turgidson,
Ufnoise, Ukexpat, Ukikonno, Unint, Urdutext, User A1, Usuwiki, VKokielov, Vangos, Vanished User 0001, VectorPosse, Vevek, WatermelonPotion, Wavelength, Wik, Wikid77, Wikifixi,
316
Article Sources and Contributors
Wikithesource, Willking1979, Wisdom89, Wolfmankurd, WriterHound, X!, XP1, Xantharius, Yacht, Yazaq, Yk Yk Yk, Youssefsan, Zack wadghiri, Zchenyu, Zoicon5, Zundark, Περίεργος, 758
anonymous edits
OLE for process control Source: http://en.wikipedia.org/w/index.php?oldid=450498911 Contributors: A876, AdAstraRG, Aligilman, Alutov, Anrie Nord, Arotto, ArséniureDeGallium,
Automationgirl, BBird, Beetstra, Boostaldo, CALR, CL, Cp2020, Dpc1962, Harksaw, Honzinus, Hu12, Iamdaman2, Inductiveautomation, JonHarder, KenJackson.US, Khrasmussen, Kku,
Krackersk, KrakatoaKatie, Kuru, Langerwolfgang, Lastorset, Lcfenwick, Leirith, Lem1968, Limited Atonement, Merbabu, Mihju, Mikedi, Modster, Mr.opc, Ninokurtalj, Pawel Duda, Pdcook, Pg
msk, Plrk, Pnm, Poccil, Poniard, Rail, Ripogenus77, Roland Abt, SCH56, Sdddlt, SoManySpammers, Steveh7, Supersteve04038, Tabletop, Techtonik, Texture, The Utahraptor, Themfromspace,
Thingg, Thumperward, USMstudent09, Veinor, Vikramrc, Vivers, VladNick, Warren, Whoelie, Z10x, Zemke98, 118 anonymous edits
Nonlinear control Source: http://en.wikipedia.org/w/index.php?oldid=450610110 Contributors: Encyclops, Favonian, Jiuguang Wang, Jmdaly, Kwiki, Lambiam, Lovewarcoffee, Mild Bill
Hiccup, Mmeijeri, Neelix, Nick Number, Ohconfucius, Pakaraki, Pivs, Pmg, R'n'B, Ronz, Serra66, TedPavlic, Woohookitty, 18 anonymous edits
Proportional control Source: http://en.wikipedia.org/w/index.php?oldid=452386886 Contributors: Antandrus, Cheet, Dollars, Dougsim, Edward, Heron, Hytar, Javaboy.ms, Jeffthejiff,
Malcolma, Nbarth, Nillerdk, SiliconDioxide, Sonett72, Spalding, Zwarepeet, 12 anonymous edits
Optimal control Source: http://en.wikipedia.org/w/index.php?oldid=469477092 Contributors: Alansohn, Alschwartz123, Amillar, Arishth, Arsen.palestini2, Arthur Rubin, Attilios, Awhan,
BenFrantzDale, Billymac00, Bruyninc, Charles Matthews, Daniel5127, Delaszk, Dralexccf, Drusillo, Duoduoduo, EagleFan, Eisv, Encyclops, Fph, Gaius Cornelius, GavinTing, Giftlite, Hanshuo,
Hess88, Hqb, Ikaros321, Jamelan, Jcoffland, Jitse Niesen, Jiuguang Wang, Jmdaly, Johan.akesson modelon, John of Reading, Kimchi.sg, Mangogirl2, Mani excel, Mat cross, Mdd, Medvall,
Michael Hardy, Michaelnikolaou, Mindmatrix, Nbarth, Nima3, Nmilich, Ocuser, Ogo, Oleg Alexandrov, Oli Filth, Optctl, Oscarjquintana, PS User, PSContributor, PeR, Robbbb, SimonP,
Smartcat, Subodha0, Sudhanshu1, TedPavlic, Themfromspace, Trey Urqhart, Ummit, V madhu, Vinzklorthos, Vmbecerra, Wolfkeeper, YUL89YYZ, Zsniew, 69 anonymous edits
Robust control Source: http://en.wikipedia.org/w/index.php?oldid=461282929 Contributors: Aminrahimian, Anantsrao1988, CRGreathouse, Cretog8, Dja25, Encyclops, Hfst, Jmoini, Jugander,
LachlanA, Melcombe, Michael Hardy, Oleg Alexandrov, PhilDWraight, Rjwilmsi, Robbbb, Sniedo, Tanja-Else, TedPavlic, TheLightbringer, Tilin, Tribaal, Yakudza, 17 anonymous edits
Adaptive control Source: http://en.wikipedia.org/w/index.php?oldid=452054329 Contributors: Aushulz, Billymac00, Celithemis, Control.optimization, Encyclops, J04n, JForget, James086,
Jason Quinn, Jdpipe, Jiuguang Wang, Letranova, LilHelpa, Mdd, Misrom, Pekaje, Pjetter, Pol098, Rjpbi, Sonymontuno, Sympatycznyfacet, Taggard, Tomas e, Trebor, User A1, Vonkje,
Wizard191, Іванко1, ‫ ﺇﻣﺎﺭﺍﺗﻲ‬1971, 16 anonymous edits
Stochastic control Source: http://en.wikipedia.org/w/index.php?oldid=435585994 Contributors: Aryaniae, Chris the speller, Duoduoduo, Encyclops, Jiuguang Wang, Jonkerz, Melcombe,
Pulkitgrover, Raymondwinn, 1 anonymous edits
Neural network Source: http://en.wikipedia.org/w/index.php?oldid=469341914 Contributors: A tzoumerkioti, Aavindraa, Adams7, Adodge, Ahoerstemeier, Ahyeek, Alan Au, Alanbly,
Alansohn, Aleksey Vayner, AnAj, Andrew C. Wilson, Andrew4010, Anetode, Antandrus, Anthony Appleyard, AnthonyQBachler, Anthonyhcole, Antonio Lopez, Aomarks, Apmonitor,
Appljax123, Arcadian, ArielGold, Arru, Arthena, Arthur Rubin, Arun143123, Aupif, Avenue, Az1568, B9 hummingbird hovering, BSTRhino, Batch1, Benbread, Bethnim, BeverlyCrusher,
Bhadani, BlaiseFEgan, Bonnarj, Bpavel88, Brian0918, BrokenSegue, Brunton, Butros, CX, Caana, Caltas, Cameltrader, Canley, Cantus, Cap'n Refsmmat, Cburnett, Ceran, Chaosdruid,
CharlesGillingham, Chase me ladies, I'm the Cavalry, Chire, ChrisCooper1991, ChrisKennedy, Clarissaelsinger, Clementi, Coladie, CommodiCast, Cs007, D.Strand, DVdm, Daev, Dan Guan,
Dan100, Dank, Darkride, Daryakav, David R. Ingham, Daytona2, Ddtuttle, Delirium, Denisarona, Denn, Denoir, Deodar, Dicklyon, Diegotorquemada, Digfarenough, Dlohcierekim's sock,
Doczilla, Doyley, Driper, Dstorkey, Duncharris, Dysprosia, Dzkd, E. Ripley, Ecalot, Efbfweborg, Elsendero, Epsilon60198, EyrianAtWork, Fangz, Fbarl, Fenice, Ferdinandopo, Figma, Fippy
Darkpaw, Foobar, Francob, G.A.S, Gauss, Ghepeu, Giftlite, Gilhamto, Gioto, GoingBatty, Goyston, Gray1, Guntram, Gurch, Hebbian Learning, Herschel, HighKing, Hmains, Holme215,
Hongooi, Hu, Hu12, Icktoofay, Imersion, Imroy, Ines it, Iohannes Animosus, Itistoday, J.Davis314, Jacj, Jafeluv, James-davis, Jhedengren, Jiteshchougule, Jitse Niesen, John Broughton, John
Quiggin, JohnJBarton, JohnWilliams, Jon Awbrey, JonEAhlquist, JonHarder, Josephorourke, Jpbowen, Justaguy0479, Justin W Smith, Jwdietrich2, Jwmillerusa, Kaimiddleton, Karbinski,
Khoikhoi, Kl4m, Klaus, Kuchhadiyaa, Kuru, La goutte de pluie, LaggedOnUser, Lampsalot, Levin, LilHelpa, LindsayH, Looie496, Lordvolton, Luna Santin, MER-C, MSchmahl, MaBe, Maian,
Majoreditor, Male1979, Mamling, Mangst, Manop, Mareino, Margaret52, Mathaddins, Mathemajor, Matthew, Mattisse, Maxversace, Mbell, Mcasl, Mcsee, Mdd, Mehdiabbasi, Meier99, Mekem,
Melcombe, Michael C Price, Michael Hardy, Mietchen, MikeWren, Minimac, Mosquitopsu, Mr.winston, Murx, Mwanner, Mysid, Nguyengiap84, Nicholasnice, NickW557, Nikai, Nikoladie,
Ninly, Nomad, NotAnonymous0, Nunh-huh, Oleg Alexandrov, Olethros, Oli Filth, Omegatron, Onasraou, Opelio, Originalname37, Ostracon, Pakcw, Pankaj Sharma Computers, Passino,
Pavan206, Pavel Vozenilek, Pepperpot80, Pescadero, Peter.kabai, Pietro16, Pjetter, Prolog, PseudoSudo, Psyphen, Qwfp, Qwyrxian, R'n'B, RJASE1, RWillwerth, Random User 937494,
Raven1977, Rayc, Reddi, Reedbeta, Reedy, Requestion, RexNL, Riana, Rich Farmbrough, Richard001, Ritchy, Riturajrr, Rjwilmsi, Rmyeid, RobertMel, Romanm, Ronz, Royalguard11,
Rtcpenguin, Rwp, SDas, Salehjoon, SallyForth123, SamuelRiv, Satishsnandihalli, Schwnj, Seans Potato Business, Shadmanzafar, Shawndkc, Shinosin, Slakr, Sonyjose, SparkOfCreation,
Spazzm, Spudtater, Steven Zhang, Stevenj, Stoni, Strangerer, Strife911, Sunny256, Sunsetsky, Supten, Svea Kollavainen, Tarotcards, Tbutzon, Tectonicura, That Guy, From That Show!, The
sunder king, TheJosh, Themfromspace, Thomasmeeks, Thompsma, Titoxd, Tribaal, Trilobitealive, Tryptofish, Twri, User A1, Vendettax, Violetriga, Vrenator, Waggers, Wavelength, Wbrameld,
Wduch, Wernerseyfried, Wikiklrsc, Will316, Wingchi, Wknight8111, Wmahan, Wraithdart, X7q, Xanzzibar, Yoshua.Bengio, Zeiden, Zybler, Ömer Cengiz Çelebi, 537 anonymous edits
Fuzzy logic Source: http://en.wikipedia.org/w/index.php?oldid=469291037 Contributors: AK Auto, Abtin, Academic Challenger, Ace Frahm, Acer, Adrian, Ahoerstemeier, Aiyasamy, Ajensen,
Alarichus, Alca Isilon, Allmightyduck, Amitauti, Amram99, Andreas Mueller, Andres, AndrewHowse, Anidaane, Anonymous Dissident, Ap, Aperezbios, Arabani, ArchonMagnus, Arjun01,
Aronbeekman, Arthur Rubin, Atabəy, AugPi, Avoided, Aydos, Aylabug, Babytoys, Bairam, BenRayfield, BertSeghers, Betterusername, Bjankuloski06en, Blackmetalbaz, Blainster, BlaiseFEgan,
Boffob, Borgx, Brat32, Brentdax, C2math, CLW, CRGreathouse, Catchpole, Causa sui, Cedars, Cesarpermanente, Chalst, Charvest, Christian List, Chronulator, Ck lostsword, Clemwang,
Closedmouth, Cnoguera, Crasshopper, Crunchy Numbers, Cryptographic hash, Cybercobra, Damian Yerrick, Denoir, Dethomas, Dhollm, Diegomonselice, Dragonfiend, Drift chambers, Drwu82,
Duniyadnd, EdH, Elockid, Elwikipedista, Em3ryguy, Eric119, Eulenreich, EverettColdwell, Evert Mouw, Expensivehat, EyeSerene, False vacuum, Felipe Gonçalves Assis, Flaminchimp, Flewis,
Fratrep, Fullofstars, Furrykef, Fyyer, GManNickG, Gauss, Gbellocchi, George100, Gerhardvalentin, Gerla, Gerla314, GideonFubar, Giftlite, Gregbard, Grim23, Gryllida, Guard, Gurch,
Gurchzilla, Gyrofrog, Gökhan, H11, Hargle, Harry Wood, Heron, History2007, Hkhandan, Honglyshin, Hypertall, ISEGeek, Icairns, Ignacioerrico, Igoldste, Ihope127, Intgr, Ioverka, Iridescent,
Ixfd64, J.delanoy, J04n, JHTaler, Jack and Mannequin, Jadorno, Jaganath, Jbbyiringiro, Jchernia, Jcrada, Jeff Silvers, JesseHogan, Jim1138, JimBrule, Joriki, Junes, JustAnotherJoe,
K.Nevelsteen, KSmrq, Kadambarid, Kariteh, Katzmik, Kilmer-san, Kingmu, Klausness, Koavf, Kuru, Kzollman, L353a1, LBehounek, Labnoor, Lambiam, LanguidMandala, Lars Washington,
Lawrennd, Lbhales, Lese, Letranova, Leujohn, Loopy48, Lord Hawk, Loren Rosen, Lynxoid84, M.r.ebraahimi, MC MasterChef, MER-C, Maddie!, Malcolmxl5, Mani1, Manop, Marcus Beyer,
Mastercampbell, Materialscientist, Mathaddins, Maurice Carbonaro, Mdd, Mdotley, Megatronium, Melcombe, Mhss, Michael Hardy, Mkcmkc, Mladifilozof, Mneser, Moilleadóir, Mr. Billion,
Nbarth, Ndavies2, Nekura, Nortexoid, Ohka-, Oicumayberight, Oleg Alexandrov, Olethros, Oli Filth, Omegatron, Omicronpersei8, Oroso, Palfrey, Panadero45, Paper Luigi, Passino, Paul August,
PeterFisk, Peterdjones, Peyna, Pickles8, Pit, Pkrecker, Pownuk, Predictor, Ptroen, Quackor, Qwfp, R. S. Shaw, RTC, Rabend, RedHouse18, Repep, Requestion, Rgheck, Riyad parvez, Rjstott,
Rjwilmsi, Rohan Ghatak, Ronaldloui, Ronz, Ruakh, Rursus, Ruud Koot, S. Neuman, SAE1962, Sahelefarda, Saibo, Samohyl Jan, Saros136, Scimitar, Scriber, Sebastjanmm, Sebesta, Sector001,
Serpentdove, Sesanker, Shervink, Slashme, Smmurphy, Snespeca, SparkOfCreation, Srikeit, Srinivasasha, StephenReed, Stevertigo, SuzanneKn, Swagato Barman Roy, T2gurut2, T3hZ10n,
TankMiche, Tarquin, Teutonic Tamer, ThornsCru, Thumperward, Tide rolls, Topher385, Traroth, TreyHarris, Trovatore, Trusilver, Turnstep, Typochimp, Ultimatewisdom, Ululuca,
Vansskater692, Velho, Vendettax, Virtk0s, Vizier, Voyagerfan5761, Wavelength, Williamborg, Wireless friend, Woohookitty, Xaosflux, Xca777, Xezbeth, Yamamoto Ichiro, Zaphod
Beeblebrox, Zfr, Zoicon5, 499 anonymous edits
Expert system Source: http://en.wikipedia.org/w/index.php?oldid=468790751 Contributors: .mdk., 2help, 4twenty42o, 767Mobile, A5b, Abaddon314159, Abdull, AbsoluteFlatness, Acroterion,
Adrian.walker, Agutie, Ahoerstemeier, Alansohn, Aldarsior, Alksub, Alphachimp, Aluion, Amypro, Andrew Parke Yang Yu, AndyTheGrump, Angela, Anonymous Dissident, Anselmobattisti,
Arnehans, Arpabr, Arthur Rubin, Arved, Asheeshgoja, Barneca, Bdonlan, Beetstra, Beland, Bentogoa, BertSeghers, Bevan.coleman, BlackHatMentor, BlueNovember, Boing! said Zebedee,
Bookandcoffee, Boothy443, Brandonm0720, Brat32, CaliforniaDreaming, Calltech, Canley, Carlosliu, CatherineMunro, Caulde, CharlesGillingham, Chocolateboy, Chris 73, Chris the speller,
Christian75, Clemwang, Coll7, Control.optimization, Conversion script, Courcelles, Criffin, Crysb, Cse.buddhika, DARTH SIDIOUS 2, DXBari, Dan Gan, Davehi1, Dbfirs, DieSkurk,
Discospinster, Doc Tropics, Dottydotdot, Dreadstar, Drwu82, Dspradau, Dullfig, Dupz, EAi, EagleFan, Econrad, Ee79, Eh7, Ehheh, Elphington, Epbr123, Erkan Yilmaz, Eternal dragon, Excirial,
Favonian, Floris V, Frecklefoot, Freek Verkerk, Furrykef, Fvasconcellos, Fyyer, Fæ, Gadfium, Gh23, Gilliam, Gmstanley, GoingBatty, Graham king 3, Graymornings, Greenchilefrog, Greenrd,
Gtg204y, Gurch, Gwernol, Gökhan, H-ko, Harvey the rabbit, Hetar, Hingram, Hirzel, History2007, Hobartimus, Hoziron, Hubyrod, Hut 8.5, Ike9898, Iluvcapra, Imran, Informavores, Ingolfson,
IvanLanin, Ixfd64, JMSwtlk, JTN, Jahiegel, JamesBWatson, Jeff G., Jerome Charles Potts, Jez9999, Jim McKeeth, Jim15936, Jmaio2, Jmchauvet, Joachim.baumeister, Joanna Bryson,
JonHarder, Joydurgin, Joyous!, Jpbowen, Jtp15, Kaimiddleton, Karthikanandraj, Kelovy, Kenyon, Khalid hassani, Kingpin13, Komandobb, Kpjas, Kristensson, Kuru, Kvng, Kyle1278, Lastorset,
Lbhales, Lear's Fool, Lights, LittleOldMe old, Looxix, Louharris, Lradrama, Luciandrei, Luk, Luna Santin, Lupo, Lycurgus, MC MasterChef, MCrawford, MER-C, Madelinefelkins, Manop,
Mariannehgv, MarkJFAllen, Marudubshinki, Massic80, Materialscientist, MattBan, Maurreen, Mav, Mdd, Merbabu, Mi ai guy, Michael Hardy, MichaelBillington, Mikael Häggström, Mlaffs,
Mmernex, Mononomic, MonsterSRM, Mormegil, Mr. Lefty, Mrholybrain, Mrmatiko, Mrseibert, Muchness, Murdomck, Mwi33, Mydogategodshat, Mzucherman, Müslimix, Nabeth, Neilc, Nick
Number, Noodlez84, Oddbodz, OingoBoingo2, OldManInACoffeeCan, Oliver202, Ooz dot ie, OrgasGirl, Paddles, Pakaran, Pat grenier, Pavel Vozenilek, Pd75, Pearle, Perfectblue97, Pgr94,
Pharaoh of the Wizards, Philip Trueman, Piano non troppo, Pinethicket, Prakash Nadkarni, Predictor, Programmar, Prokopenya Viktor, Proofreader77, PseudoSudo, Quintote, Qwertyus, Qxz, R
Calvete, R'n'B, RDBury, RapidR, Reach Out to the Truth, Reconsider the static, Rich Farmbrough, Richardhenricks, RickK, Rmosler2100, Robert Kowalski, Ronz, Rspeer, Rursus, Ryulong,
SNowwis, Sam Korn, Sam42, Sander Säde, SarekOfVulcan, Sarnholm, Saswat012, SatuSuro, Savant13, Scott Dial, Semperf, Serdar XoXo, Sesu Prime, Shadowjams, Shaggyjacobs, Shepard,
Shizhao, SimonArlott, SimonLyall, Skinneal, Skyezx, Skysmith, SlackerMom, Snowolf, Solitude, Sortior, SpeedyGonsales, Spellmaster, Sprocc, Squidfish, Srleffler, Stephen Bain, Stevag,
Storkk, Supten, Swollib, Sylvain Mielot, Talk2xpert, TerrorTowers, TexMurphy, The Epopt, Thistheman, Tide rolls, Tim1357, Titov Anton, Tmg1165, Tobias Bergemann, Tom Pippens,
Topiarydan, Treyt021, Ulric1313, Useight, Van helsing, Vangelis12, VictorAnyakin, Vrenator, WaltBusterkeys, Wayiran, West Brom 4ever, WikiLaurent, Wikilibrarian, WillWare, Woohookitty,
317
Article Sources and Contributors
Wrathchild, X96lee15, Yonkie, Youblend2, Youssefsan, ZENDELBACH, Zoicon5, Zzuuzz, Δ, 662 anonymous edits
Root locus Source: http://en.wikipedia.org/w/index.php?oldid=456907370 Contributors: Adoniscik, AhmedHan, Alphachimp, Angela, AvicAWB, Biezl, Brews ohare, Burns512, Celieber,
Charles Matthews, Cuaxdon, Daniel fady, David Ayton, Dja25, Fredrik, Glrx, Gtnash, Hvn0413, J.delanoy, Kgartner, Kir360, Lpkeys, Marcosagliano, Markrkrebs, Masgatotkaca, Mdd,
Melhawar, Michael Hardy, Musically ut, OlavN, Oleg Alexandrov, Petrb, Quinwound, Raiker, Rasputin243, RickK, Rs2, Salamurai, Silverfish, Spradlig, Sss41, Topbanana, Trevor MacInnis,
Tuvas, Updatehelper, WhiteOak2006, Wigan25, Wj32, Xeno, Yamamoto Ichiro, Zfeinst, Zzxterry, 102 anonymous edits
Compensation (engineering) Source: http://en.wikipedia.org/w/index.php?oldid=416361288 Contributors: Bearcat, EagleFan, Fratrep, HGB, Longhair, Metrax, Nickjwebb, Pichpich, R'n'B,
Radiojon, Tabletop, 5 anonymous edits
Frequency compensation Source: http://en.wikipedia.org/w/index.php?oldid=434185903 Contributors: Anoneditor, Brews ohare, Cburnett, CyborgTosser, Gaius Cornelius, Glenn,
Hooperbloob, Light current, MichaelBillington, Nbarth, Omegatron, PhilKnight, TedPavlic, Toffile, Wdwd, 13 anonymous edits
Error detection and correction Source: http://en.wikipedia.org/w/index.php?oldid=470214539 Contributors: 0nlyth3truth, 4th-otaku, Abdull, Aboeing, Agentbla, Anon lynx, Antaeus Feldspar,
Apantomimehorse, Arthena, Arvindn, Auntof6, Avocade, Awmorp, Babybahamut, Beamishboy, Bekant, Bigs slb, Binary TSO, Bird of paradox, BitJam, Bob.v.R, Bobblewik, Bryan Derksen,
Bumm13, Caesura, Cantalamessa, Cerireid, Chekholko, Cihan, Conversion script, Copeland.James.H, Cryptonaut, Cunchem, D6, DGerman, DanielPharos, DavidCBryant, DavidCary,
Davidhorman, Deflective, Dogposter, Drhex, Dthede, Dysprosia, Eastlaw, Em3ryguy, Eric119, EugeneZelenko, EverGreg, Eyreland, Firien, Flying hazard, Francs2000, Gaius Cornelius,
GearCutter, Giftlite, Gmazeroff, Gnorthup, GringoCroco, Gwern, Helder.wiki, Icecats11, Inventio, Iridescent, Izab1, Jaidan, Jao, Jarble, Jcea, Jdseymour, Jerry, Jkl, Joey7643, Jorge Stolfi,
Kenyon, Linas, Lir, Lotje, Lowellian, Mandarax, Mange01, Margin1522, Martarius, Masterpiece2000, Materialscientist, Mathuranathan, Michael Hardy, Mitchan, Miterdale, Mmeijeri, Morten,
Music Sorter, MuthuKutty, Mysid, Nageh, Netpilot43556, Niceguyedc, Nihiltres, Nubiatech, Ojl, Oleg Alexandrov, Oli Filth, Phantomsteve, Phr, Pinethicket, Piotrus, Pnm, Poeloq, Pol098,
Pouya, PrologFan, Pudhed, R'n'B, Radagast83, Ranjithsutari, Rbrewer42, Reedy, Reetep, Requestion, RexNL, Rgray001, Ricky81682, Ring0, Rjstott, Ruud Koot, Ryan Roos, SDS, SNIyer12,
Sepia tone, Seven of Nine, Sikilai, Sixequalszero, Snowolf, Snoyes, Splash, Splintercellguy, Spoon!, Suffusion of Yellow, Supercoop, Swerdnaneb, T-rex, Taral, Technopilgrim, Teejayo,
Tejas81, The Anome, The Rationalist, The Thing That Should Not Be, Tobias Bergemann, Traal, Trachten, Ubunaki, Unco, Vipinhari, WatchAndObserve, Wavelength, WestwoodMatt,
Woohookitty, Ykhwong, ZeroOne, 231 anonymous edits
Linear encoder Source: http://en.wikipedia.org/w/index.php?oldid=463085769 Contributors: C mobile, CWA-FRB, Colonies Chris, Fiducial, Fresheneesz, Lightmouse, Magioladitis,
Malcolma, Matesy, Mechgyver, Mild Bill Hiccup, Nagle, Pt130135, R'n'B, Radiojon, 28 anonymous edits
Gray code Source: http://en.wikipedia.org/w/index.php?oldid=469363225 Contributors: A876, AJP, Acerperi, Ahoerstemeier, Ahseaton, AileTheAlien, Andy Dingley, Animagi1981, BAxelrod,
BD2412, Bender235, Bggoldie, Bobblehead, Bogdangiusca, BrotherE, Bubba73, Bunyk, Cburnett, Charles Matthews, Charvest, Cholling, Chris the speller, ChrisEich, CosineKitty, Cyp, David
Eppstein, David.Monniaux, DavidCary, Daxx wp, Dcoetzee, Deacs33, Demi, Denelson83, Dicklyon, Dina, Dispenser, Djfeldman, Dr0b3rts, Droll, Edratzer, El C, Elias, Elphion, Emvee,
Fcdesign, Fresheneesz, Gerbrant, Giftlite, Gnorthup, GoingBatty, Haham hanuka, Hariva, Heron, Hv, Iamfscked, Inductiveload, Iridescent, Isaac Dupree, Isiah Schwartz, JLaTondre, Jaiguru42,
Jan olieslagers, Jason Recliner, Esq., Jeff 113, Jim1138, Jjbeard, Johnmorgan, Johnuniq, Jos.koot, Jturkia, Kanesue, Kateshortforbob, Kjkjava, Law, Leonard G., LilHelpa, Linas, Lipedia,
MER-C, MSGJ, Machine Elf 1735, MarkSweep, MattGiuca, Matusz, Max711, Maxal, Maxalot, Mellum, Michael Hardy, Mike1024, Mild Bill Hiccup, MooMan1, MrOllie, MyrddinE, Nick
Pisarro, Jr., Nikevich, Nneonneo, Noe, Oashi, Ocatecir, Ohiosoil, Pacifier, Pakaran, Pgimeno, PierreAbbat, Piet Delport, Plamka, Pleasantville, Plugwash, Prashantgonarkar, PuerExMachina,
Qwyrxian, RTC, RayKiddy, Requestion, Rich Farmbrough, Ricky81682, Rjwilmsi, RobH (2004 account), Ronz, Sakurambo, SciCompTeacher, Sciyoshi, Seantellis, Seraphimblade, Shellreef,
Sjock, Smalljim, Snowolf, SuneJ, Suruena, Svick, Tedickey, TheNightFly, Tomo, TutterMouse, Vanish2, Vanka5, Wapcaplet, Wikkrockiana, Winzurf, Wwoods, XJamRastafire, Yahya
Abdal-Aziz, Yoshigev, Yworo, ZeroOne, Zeycus, 197 anonymous edits
Rotary encoder Source: http://en.wikipedia.org/w/index.php?oldid=465481711 Contributors: Ace Frahm, AlexHe34, Alhaiksandher, AliasFan4Lyfe, Aliasgherman, Americanhero, Artie p,
Attilios, Bazkaz, Beta16, Birdav, Bongwarrior, Calltech, Cat5nap, Cburnett, Coasterlover1994, CommonsDelinker, Cristan, Damian Yerrick, Dancter, Danim, David-bel, Dhollm, Dina, Dr0b3rts,
Dtgriscom, EUROFARAD, Erik9, Evanh, Gamekock, Halfgaar, Heron, Hooperbloob, IP83, Jaho, Jamesooders, Jcdonelson, JohnCD, Johncatsoulis, Julesd, Katana, Kithira, Kku, Mgros,
Mike1024, Nagle, North8000, Ocaasi, Oceano2012, Oli Filth, Omegatron, Onionmon, Overand, Phlake, RoySmith, Rqtfi, Rspanton, SBunce, Sagsaw, Sdec25, Seaphoto, Semiwiki, Shai-kun,
Shubinator, Sinaerem, Sintaku, Snibble, Srleffler, Sscott328, Steelerdon, Suruena, THEN WHO WAS PHONE?, TheDog, Therealpaulie, Tpikonen, Victor Korniyenko, WikHead, Wikkrockiana,
Wolfch, Wtshymanski, ‫חובבשירה‬, ‫ﻣﺎﻧﻲ‬, 136 anonymous edits
Bode plot Source: http://en.wikipedia.org/w/index.php?oldid=466646364 Contributors: A2Kafir, Andy.ackland, Anna Lincoln, Arcturus4669, Avicennasis, Avoided, BenFrantzDale,
Betacommand, Bobblehead, Brews ohare, Buffetrand, Chaosgate, Clam0p, Clemens vi, Dbtfz, Derdeib, Dicklyon, Dr.K., Drew335, Ec5618, ExportRadical, Falcorian, Fresheneesz, Gaius
Cornelius, Ganzor, Giftlite, Gmoose1, Graibeard, Gravix, Heron, Hooperbloob, Huntthetroll, Jamelan, Jeff P Harris, Jiuguang Wang, Jive Dadson, Joy, Jshen6, KPH2293, Kenyon, Kmellem,
Lambertch, Mdd, Mehran, Mellery, Michael Devore, Michael Hardy, Miguelgoldstein, Mivey6, Mr. PIM, Nabla, Nbarth, Niketmjoshi, Nitin.mehta, Nodlit, Omegatron, Patrick, Peter.scottie,
Phosphoricx, Poulpy, Private Pilot, Redheylin, Rehnn83, Requestion, Shadowjams, SimonP, Sommacal alfonso, Spradlig, Stw, Tim Starling, Toffile, Tomer shalev, User A1, Vukg, WakingLili,
Wtshymanski, Zoomzoom1, 114 anonymous edits
Nyquist stability criterion Source: http://en.wikipedia.org/w/index.php?oldid=467069219 Contributors: Aminrahimian, Attilios, Beta16, Brews ohare, Buffetrand, Charles Matthews, Cuaxdon,
Dicklyon, Duncharris, Ewlyahoocom, Giftlite, Haditya, Hvn0413, Ikiwaner, Jenblower, Jiuguang Wang, Jm1234567890, Krishnavedala, LachlanA, Linuxlad, Lupo, Lv131, Mailer diablo,
Matutano, Merovingian, Minhlong87, Musically ut, Nillerdk, Novangelis, Pharaoh of the Wizards, Phobulos, PoqVaUSA, Preben Holm, Smalljim, SunCreator, Toffile, Trusilver, Wknight8111,
XMxWx, Пика Пика, 58 anonymous edits
Instability Source: http://en.wikipedia.org/w/index.php?oldid=464134809 Contributors: Anticipation of a New Lover's Arrival, The, Art Carlson, Bluemoose, Chris the speller, Chuckiesdad,
Chyeburashka, Complexica, Crowsnest, Cyrus Grisham, David Kendall, David R Merritt, Djbaniel, Donarreiskoffer, Eassin, Evolauxia, Gasheadsteve, Glenn, Grm wnr, Headbomb,
Hornandsoccer, Iantresman, JIP, Jenesis, Jiuguang Wang, Johnwsimpson, Jujutacular, JzG, Kvng, Mais oui!, Mgmirkin, Mleconte, Mstrickl, Nherm, Nikai, Oleg Alexandrov, Open2universe,
Originalwana, Ottojack, Pearle, Petri Krohn, Rs2, Salih, Silverfish, The RedBurn, Tokamac, UlmPhysiker, Valentinian, Vsmith, WilyD, XJamRastafire, Ytrottier, 20 anonymous edits
Frequency response Source: http://en.wikipedia.org/w/index.php?oldid=468175898 Contributors: Andrei Stroe, Antaya, Askbojesen, BenFrantzDale, Binksternet, C6H3N3O3, Cburnett,
CosineKitty, Furrykef, GRAHAMUK, Giftlite, Glenn, H.sh8241, Headbomb, Hooperbloob, Jag123, Janyllp, Java13690, Jj137, Jorge Stolfi, Krishnavedala, Laurascudder, Light current,
Materialscientist, Mbrennwa, Metacomet, Mulad, Mwilde, Neelix, Nillerdk, Nnh, Oarih, Omegatron, Onco p53, PoqVaUSA, Redheylin, Someguy1221, Spinningspark, Steve Quinn, Tomasz Zuk,
Tomer shalev, Trumpeter96, Tsiaojian lee, Wolps, Yaris678, Zoicon5, 35 anonymous edits
Nyquist plot Source: http://en.wikipedia.org/w/index.php?oldid=468278645 Contributors: A. B., Attilios, Bazz, Brews ohare, Cuaxdon, CyrilB, Engelec, Flehmen, Giftlite, Hemanshu, Heron,
Hooperbloob, Jeff P Harris, Jiuguang Wang, John of Reading, JorisvS, Linuxlad, MarkusHagenlocher, Mdd, Mets501, Michael Hardy, Mpassman, Nbarth, Omegatron, Patrick, Pjvpjv, Pladask,
PoqVaUSA, Reconsider the static, Ro8269, Rod57, Salasks, So God created Manchester, Spiff, The Anome, Titan, Toffile, Trojancowboy, Vonkje, XMxWx, Yorrak, 36 anonymous edits
Nichols plot Source: http://en.wikipedia.org/w/index.php?oldid=464927680 Contributors: B4hand, Bazz, Blair Bonnett, Chthonicdaemon, Encyclops, Engelec, Giftlite, Hooperbloob, Mdd,
Michael Hardy, Peter17, The Anome, Tuvosi, 4 anonymous edits
Bandwidth (signal processing) Source: http://en.wikipedia.org/w/index.php?oldid=460610886 Contributors: 16@r, 198.211.186.xxx, 203.109.250.xxx, A3r0, Academic Challenger, Agi896,
Ahoerstemeier, Alfio, [email protected], Alinja, Andonic, Andre Engels, Andrewpmk, Andy120290, Archer3, Archimerged, Atlant, AxelBoldt, Bandwith, BenFrantzDale, Binksternet, Blobglob,
Bobblewik, Bobo192, Brews ohare, CONFIQ, Carbonite, Cart73, Cburnett, Christian Bienia, Cihan, Cmathas, Conversion script, Coredesat, CosineKitty, DVD R W, DarkArcher, Dbroadwell,
Delinquer, DerHexer, Dicklyon, Don4of4, Dysprosia, Epbr123, Escapepea, Falling Cloud, Fatsamsgrandslam, FreplySpang, FrozenUmbrella, Gbleem, Gene Nygaard, Giftlite, Gilliam,
Graham87, Green Giant, Green Heart 1985, GunnerJr, Gwernol, Henrikb4, Heron, Httpwebwitch, Ianharvey, Ianneub, Inductiveload, Jacobolus, Jamelan, Jasonrichmond, Jestar, Jhbdel,
Jim.henderson, JohnTechnologist, JonHarder, Jongleur100, Jordav, JordoCo, Kumioko, KyraVixen, Light current, Lightmouse, Lloydsmart, Lord Snoeckx, Madewokherd, Maglev2, Mange01,
Marek69, Mav, Mbell, Mejor Los Indios, Melaniemm, Metacomet, Michael Hardy, Mike Logghe, Mnmngb, Motmajor, Mrdvt92, Mrendo, Munford, MuthuKutty, N6gn, Nbarth, Nekkensj, Noisy,
Noosentaal, Nposs, Ohconfucius, Ojw, Oleg Alexandrov, Omegatron, Onceonthisisland, OverlordQ, Oysterguitarist, Patrick, PatrikR, Patstuart, PeterFlannery, Phe, Philip Trueman, Prenz, R'n'B,
Redheylin, Reinthal, RexNL, Ribilla, Richard001, Rintrah, Rmaarts, RobertG, RoyBoy, Rs2, Rwestafer, S415046, Saihtam, Scorpion, Sharewarepro, Shark96z, Shawn 357, Shawnc, Skicavs,
Smack, Smyth, Srleffler, Steve8675309, Stevonova, The Anome, Tiddly Tom, Tim Starling, Tins128, Titoxd, TonyTiger22, Vriullop, Vspengen, Waveguy, Weihao.chiu, West London Dweller,
Whiskers9, Wknight94, Wolfkeeper, Wtshymanski, X736e65616b, Yonatan, Yosri, Yudiweb, ZZninepluralZalpha, Zalgo, Zereshk, Zowie, 311 anonymous edits
Lead–lag compensator Source: http://en.wikipedia.org/w/index.php?oldid=451172418 Contributors: Ademkader, Chris Melton, Cmichael, Dicklyon, LachlanA, Michael Hardy, Ojw, Oli Filth,
Oxnard28, Razorflame, Rogerbrent, Shoez, Tariqabjotu, TedPavlic, Vonkje, XieChengnuo, 33 anonymous edits
Static VAR compensator Source: http://en.wikipedia.org/w/index.php?oldid=470109427 Contributors: Anders751004, ArséniureDeGallium, BillC, Clampower, Cmdrjameson, Desert Tripper,
Dhollm, Eassin, Fireglowe, Grafen, Iridescent, Karada, Linas, Mariraja2007, Paul Comfort, Rjwilmsi, Scampiandchips, Sladen, Solipsist, SpK, Tommy2010, Tony Fox, Uncle G, Volpix0,
Wtshymanski, 28 anonymous edits
318
Article Sources and Contributors
PID controller Source: http://en.wikipedia.org/w/index.php?oldid=469730997 Contributors: 2m man, A. Carty, A3RO, AaronMK, Abarry, Acdx, Ademkader, Aeolian, Alexius08, Alll, Altzinn,
Aluvus, AnitaS, Anna Lincoln, Attilios, Aturevsk, AutomationBooks, Bdijkstra, BeastRHIT, Ben pcc, BenFrantzDale, Benjamander, Bentogoa, Billymac00, Bobblewik, Bora Eryilmaz,
Bpavel88, BradBeattie, Briceb, Bsodmike, Buesser, CanisRufus, CapitalR, Carmichael95, Caslon, Charles Matthews, Chiklit, ChristianG2, Ciphergoth, Cometstyles, Computationalverb,
Control.optimization, Copeland.James.H, Coredesat, Crinoid, Crohnie, CyberneticSheepCZ, DARTH SIDIOUS 2, Dabozz88, Dangrayorg, DanielRigal, Dbaechtel, Dedmonds, Dethme0w,
Deville, Dhatfield, Dicklyon, Dinhxuanduyet, Dja25, Djgreenfield, Dmcq, Dominick, Dzkd, Ejwong, Elcobbola, Encyclops, Engelec, Epolk, FastIkarus, Femto, Fenn fukai, Frank Lofaro Jr.,
Frappucino, Fæ, GB fan, Gabelstaplerfahrer, Gaius Cornelius, GarrettSocling, Gene Nygaard, Giftlite, Glane23, GunnerJr, HK-90, Hankwang, Harriv, HenkvD, Heron, HexaChord, Hughjack,
Icairns, J.delanoy, JLLCuba, JamieStapleton, Jbcmf, Jdigangi, Je bonilla, Jeff G., Joel Saks, Jon Mansfield, JonDePlume, JonathonReinhart, Jrdioko, Karthikkumar411, Kbosak, Kerotan, Kku,
Kreuzer5, Kvng, LHOON, Laurentsavaete, Learnfpga, LiDaobing, LieAfterLie, Liu.Yunxia, Liyang, LyonL, Marsian, Masgatotkaca, Mausy5043, Mav, Mbbradford, Mbeychok, Mbutts,
Md2perpe, Meatmanek, Meestaplu, Michael Hardy, Mikiemike, Miles underwood, Mmeijeri, Mononoke, Motafa, Movedgood, MrOllie, Mustafa Al-mosawi, N2e, Nave.notnilc, Nbarth, Nigelj,
Nitin.mehta, NunoSoares, Oh Snap, Oli Filth, Omegatron, Orzetto, Othompson, Papa November, Patrick, Piano non troppo, Pko, Printer222, ProcessControlGuy, Profr ettore, Pteromys, Q
Science, Ray Van De Walker, Rdautel, Redbeanpaste, Relilles, Requestion, Rich Farmbrough, Robert - Northern VA, Rod57, Ronz, RoyKok, Rspadim, Russvdw, S.borchers, SD5, SMC89,
Saxbryn, Seanp, Seaphoto, Sectryan, Sesc, Shadowjams, Shanes, Shimei, Shriramrs31, Signalhead, SilverStar, Sinasgay, Sk wiki, Skorkmaz, Smzoha, Sonett72, Spalding, Spiel496, Spradlig,
Stephenb, Steveaa, Strait, Swrkoab, Teoryn, The Anome, Tremilux, Trusilver, Ummit, Unbitwise, Undead warrior, Unregistered.coward, User A1, Van helsing, Vary, Vivek.cd, Vizcarra,
Wahapure, WakingLili, Warniats, WikiDao, Wmahan, Wtshymanski, Xenodevil, XieChengnuo, Yasirniazkhan, Zephalis, Zerodamage, Zoomzoom1, 680 anonymous edits
319
Image Sources, Licenses and Contributors
Image Sources, Licenses and Contributors
Image:LOCKHEED F-117A NIGHT HAWK.png Source: http://en.wikipedia.org/w/index.php?title=File:LOCKHEED_F-117A_NIGHT_HAWK.png License: Public Domain Contributors:
Cobatfor, D-Kuru, Joshbaumgartner, Matthewedwards, Orlovic, Slomox, Stahlkocher, SvonHalenbach, 1 anonymous edits
Image:F-117A GBU-28.JPEG Source: http://en.wikipedia.org/w/index.php?title=File:F-117A_GBU-28.JPEG License: Public Domain Contributors: Original uploader was Koalorka at
en.wikipedia (Original text : MSGT EDWARD SNYDER)
Image:Schnittbild drehnatrieb 01.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Schnittbild_drehnatrieb_01.jpg License: Creative Commons Attribution-Sharealike 2.0
Contributors: HerbstrittMse
Image:insulin pump with infusion set.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Insulin_pump_with_infusion_set.jpg License: Public Domain Contributors: Original uploader
was Mbbradford at en.wikipedia
Image:SRH025-p40.jpg Source: http://en.wikipedia.org/w/index.php?title=File:SRH025-p40.jpg License: Public Domain Contributors: Original uploader was user:Megapixie at en.wikipedia
Image:Space Shuttle Columbia launching.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Space_Shuttle_Columbia_launching.jpg License: Public Domain Contributors: NASA
Image:Ideal feedback model.svg Source: http://en.wikipedia.org/w/index.php?title=File:Ideal_feedback_model.svg License: Public Domain Contributors: Me (Intgr)
Image:ACTH Negative Feedback.svg Source: http://en.wikipedia.org/w/index.php?title=File:ACTH_Negative_Feedback.svg License: Creative Commons Attribution 3.0 Contributors:
DRosenbach
File:Herdwick Stampede.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Herdwick_Stampede.jpg License: Creative Commons Attribution 2.0 Contributors: Andy Docker from
England
File:Birmingham Northern Rock bank run 2007.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Birmingham_Northern_Rock_bank_run_2007.jpg License: Creative Commons
Attribution-Sharealike 2.0 Contributors: 159753, Infrogmation, Man vyi, OSX, Yarl
Image:Positive feedback bistable switch.svg Source: http://en.wikipedia.org/w/index.php?title=File:Positive_feedback_bistable_switch.svg License: Public Domain Contributors: Naj-GMU,
Squidonius, THEN WHO WAS PHONE?, 1 anonymous edits
Image:Phanerozoic Biodiversity.svg Source: http://en.wikipedia.org/w/index.php?title=File:Phanerozoic_Biodiversity.svg License: GNU Free Documentation License Contributors: SVG
version by Albert Mestre
File:Regenerartive Receiver-S7300056.JPG Source: http://en.wikipedia.org/w/index.php?title=File:Regenerartive_Receiver-S7300056.JPG License: Public Domain Contributors: Ozguy89
Image:Smitt hysteresis graph.svg Source: http://en.wikipedia.org/w/index.php?title=File:Smitt_hysteresis_graph.svg License: Creative Commons Attribution-ShareAlike 3.0 Unported
Contributors: FDominec
Image:R-S mk2.gif Source: http://en.wikipedia.org/w/index.php?title=File:R-S_mk2.gif License: Creative Commons Attribution 2.0 Contributors: Napalm Llama
File:Technics SL-1210MK2.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Technics_SL-1210MK2.jpg License: GNU Free Documentation License Contributors: Cschirp.
Original uploader was Cschirp at de.wikipedia
File:Adam Savage HOPE.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Adam_Savage_HOPE.jpg License: Creative Commons Attribution-Sharealike 3.0 Contributors: Original
uploader was Porkrind at en.wikipedia
Image:IDEF Diagram Example.jpg Source: http://en.wikipedia.org/w/index.php?title=File:IDEF_Diagram_Example.jpg License: Public Domain Contributors: Defense Acquisition
University
Image:Fo2ufg823rhf832hfdorfg.JPG Source: http://en.wikipedia.org/w/index.php?title=File:Fo2ufg823rhf832hfdorfg.JPG License: Public Domain Contributors: DFD guru
Image:Functional Flow Block Diagram Format.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Functional_Flow_Block_Diagram_Format.jpg License: Public Domain
Contributors: Original uploader was Mdd at en.wikipedia
Image:IPO-S Model.gif Source: http://en.wikipedia.org/w/index.php?title=File:IPO-S_Model.gif License: Public Domain Contributors: DXBari
Image:N2 chart definition.JPG Source: http://en.wikipedia.org/w/index.php?title=File:N2_chart_definition.JPG License: Public Domain Contributors: NASA
Image:SADT.svg Source: http://en.wikipedia.org/w/index.php?title=File:SADT.svg License: Public Domain Contributors: UlrichAAB
Image:BPMN-AProcesswithNormalFlow.jpg Source: http://en.wikipedia.org/w/index.php?title=File:BPMN-AProcesswithNormalFlow.jpg License: Public Domain Contributors: Tttt1234
Image:FEA BRM Hierachy.JPG Source: http://en.wikipedia.org/w/index.php?title=File:FEA_BRM_Hierachy.JPG License: Public Domain Contributors: doi.gov
Image:PD-icon.svg Source: http://en.wikipedia.org/w/index.php?title=File:PD-icon.svg License: Public Domain Contributors: Alex.muller, Anomie, Anonymous Dissident, CBM, MBisanz,
Quadell, Rocket000, Strangerer, Timotheus Canens, 1 anonymous edits
Image:Functional block diagram of the attitude control and maneuvering electronics system .jpg Source:
http://en.wikipedia.org/w/index.php?title=File:Functional_block_diagram_of_the_attitude_control_and_maneuvering_electronics_system_.jpg License: Public Domain Contributors:
McDonnell, "Project Gemini Familiarization Charts,"
File:Convolution of box signal with itself2.gif Source: http://en.wikipedia.org/w/index.php?title=File:Convolution_of_box_signal_with_itself2.gif License: Creative Commons
Attribution-Sharealike 3.0 Contributors: Convolution_of_box_signal_with_itself.gif: Brian Amberg derivative work: Tinos (talk)
File:Convolution of spiky function with box2.gif Source: http://en.wikipedia.org/w/index.php?title=File:Convolution_of_spiky_function_with_box2.gif License: Creative Commons
Attribution-Sharealike 3.0 Contributors: Convolution_of_spiky_function_with_box.gif: Brian Amberg derivative work: Tinos (talk)
File:Convolution3.PNG Source: http://en.wikipedia.org/w/index.php?title=File:Convolution3.PNG License: Public Domain Contributors: Original uploader was w:en:User:EmoteEmote at
en.wikipedia Later version(s) were uploaded by w:en:User:McLoafMcLoaf at en.wikipedia.
File:Lorenz attractor yb.svg Source: http://en.wikipedia.org/w/index.php?title=File:Lorenz_attractor_yb.svg License: Creative Commons Attribution-Sharealike 2.5 Contributors:
User:Dschwen, User:Wikimol
File:LinearFields.png Source: http://en.wikipedia.org/w/index.php?title=File:LinearFields.png License: Creative Commons Attribution 2.5 Contributors: XaosBits
Image:circ-4-nor.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Circ-4-nor.jpg License: Creative Commons Attribution 3.0 Contributors: Henning.Mortveit
Image:circ-4-nor-1234.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Circ-4-nor-1234.jpg License: Creative Commons Attribution 3.0 Contributors: Henning.Mortveit
File:Prop-tableau-4.svg Source: http://en.wikipedia.org/w/index.php?title=File:Prop-tableau-4.svg License: GNU Free Documentation License Contributors: Original uploader was Tizio at
en.wikipedia. Later version(s) were uploaded by RobHar at en.wikipedia.
Image:Pole Splitting Example.png Source: http://en.wikipedia.org/w/index.php?title=File:Pole_Splitting_Example.png License: GNU Free Documentation License Contributors: Brews ohare
Image:Pole Splitting Example with Miller Transform.png Source: http://en.wikipedia.org/w/index.php?title=File:Pole_Splitting_Example_with_Miller_Transform.png License: Creative
Commons Attribution-Share Alike Contributors: Brews ohare
Image:Two-pole Bode magnitude plot.png Source: http://en.wikipedia.org/w/index.php?title=File:Two-pole_Bode_magnitude_plot.png License: Creative Commons Attribution-Share Alike
Contributors: Brews ohare
Image:Compensation capacitance.png Source: http://en.wikipedia.org/w/index.php?title=File:Compensation_capacitance.png License: GNU Free Documentation License Contributors:
Brews ohare
Image:Damped spring.gif Source: http://en.wikipedia.org/w/index.php?title=File:Damped_spring.gif License: Public Domain Contributors: Oleg Alexandrov
Image:2nd Order Damping Ratios.svg Source: http://en.wikipedia.org/w/index.php?title=File:2nd_Order_Damping_Ratios.svg License: Public Domain Contributors: Inductiveload
Image:Impulse.png Source: http://en.wikipedia.org/w/index.php?title=File:Impulse.png License: Public domain Contributors: w:User:IainIain 09:41, 7 June 2006 (UTC)
Image:LTI.png Source: http://en.wikipedia.org/w/index.php?title=File:LTI.png License: Public Domain Contributors: Flappiefh, Joelholdsworth, Ma-Lik, Maksim
File:Feedback loop with descriptions.svg Source: http://en.wikipedia.org/w/index.php?title=File:Feedback_loop_with_descriptions.svg License: Creative Commons Attribution-Share Alike
Contributors: Myself
File:Boulton and Watt centrifugal governor-MJ.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Boulton_and_Watt_centrifugal_governor-MJ.jpg License: Creative Commons
Attribution 3.0 Contributors: Dr. Mirko Junge
File:simple feedback control loop2.png Source: http://en.wikipedia.org/w/index.php?title=File:Simple_feedback_control_loop2.png License: unknown Contributors: Corona
320
Image Sources, Licenses and Contributors
Image:Variables proporcionals.png Source: http://en.wikipedia.org/w/index.php?title=File:Variables_proporcionals.png License: GNU Free Documentation License Contributors: M. Romero
Schmidkte
Image:Academ_homothetic_rectangles.svg Source: http://en.wikipedia.org/w/index.php?title=File:Academ_homothetic_rectangles.svg License: Creative Commons Attribution-Sharealike 3.0
Contributors: Yves Baelde
File:Integral example.svg Source: http://en.wikipedia.org/w/index.php?title=File:Integral_example.svg License: Creative Commons Attribution-ShareAlike 3.0 Unported Contributors: KSmrq
File:ArabicIntegralSign.svg Source: http://en.wikipedia.org/w/index.php?title=File:ArabicIntegralSign.svg License: Public Domain Contributors: File:Integral approximations.svg Source: http://en.wikipedia.org/w/index.php?title=File:Integral_approximations.svg License: Creative Commons Attribution-ShareAlike 3.0 Unported
Contributors: KSmrq
File:Integral Riemann sum.png Source: http://en.wikipedia.org/w/index.php?title=File:Integral_Riemann_sum.png License: Creative Commons Attribution-ShareAlike 3.0 Unported
Contributors: KSmrq
File:Riemann sum convergence.png Source: http://en.wikipedia.org/w/index.php?title=File:Riemann_sum_convergence.png License: Creative Commons Attribution-ShareAlike 3.0 Unported
Contributors: KSmrq
Image:RandLintegrals.png Source: http://en.wikipedia.org/w/index.php?title=File:RandLintegrals.png License: Creative Commons Attribution-ShareAlike 3.0 Unported Contributors: Aaron
Rotenberg, Darapti, Dubhe
File:Improper integral.svg Source: http://en.wikipedia.org/w/index.php?title=File:Improper_integral.svg License: Creative Commons Attribution-ShareAlike 3.0 Unported Contributors:
KSmrq
File:Improper integral unbounded internally.svg Source: http://en.wikipedia.org/w/index.php?title=File:Improper_integral_unbounded_internally.svg License: Creative Commons
Attribution-ShareAlike 3.0 Unported Contributors: KSmrq
File:Volume under surface.png Source: http://en.wikipedia.org/w/index.php?title=File:Volume_under_surface.png License: Public Domain Contributors: Oleg Alexandrov
File:Line-Integral.gif Source: http://en.wikipedia.org/w/index.php?title=File:Line-Integral.gif License: GNU Free Documentation License Contributors: Cronholm144, Darapti, McZusatz,
Nandhp, SkiDragon
File:Surface integral illustration.png Source: http://en.wikipedia.org/w/index.php?title=File:Surface_integral_illustration.png License: Public Domain Contributors: Darapti, Jahobr, Oleg
Alexandrov, WikipediaMaster
File:Numerical quadrature 4up.png Source: http://en.wikipedia.org/w/index.php?title=File:Numerical_quadrature_4up.png License: Creative Commons Attribution-ShareAlike 3.0 Unported
Contributors: KSmrq
File:Linear PK Example.png Source: http://en.wikipedia.org/w/index.php?title=File:Linear_PK_Example.png License: Creative Commons Attribution 3.0 Contributors: Alfie↑↓©
File:Tangent to a curve.svg Source: http://en.wikipedia.org/w/index.php?title=File:Tangent_to_a_curve.svg License: Public Domain Contributors: Original uploader was Jacj at en.wikipedia
Later versions were uploaded by Oleg Alexandrov at en.wikipedia.
File:Graph of sliding derivative line.gif Source: http://en.wikipedia.org/w/index.php?title=File:Graph_of_sliding_derivative_line.gif License: Public Domain Contributors: en:User:Dino
Image:Tangent-calculus.svg Source: http://en.wikipedia.org/w/index.php?title=File:Tangent-calculus.svg License: GNU Free Documentation License Contributors: derivative work: Pbroks13
(talk) Tangent-calculus.png: Rhythm
Image:Secant-calculus.svg Source: http://en.wikipedia.org/w/index.php?title=File:Secant-calculus.svg License: GNU Free Documentation License Contributors: derivative work: Pbroks13
(talk) Secant-calculus.png: Shizhao
Image:Lim-secant.svg Source: http://en.wikipedia.org/w/index.php?title=File:Lim-secant.svg License: GNU Free Documentation License Contributors: derivative work: Pbroks13 (talk)
Lim-secant.png: Original uploader was CSTAR at en.wikipedia
File:Right-continuous.svg Source: http://en.wikipedia.org/w/index.php?title=File:Right-continuous.svg License: Public Domain Contributors: w:User:JacjJacj
File:Absolute value.svg Source: http://en.wikipedia.org/w/index.php?title=File:Absolute_value.svg License: GNU Free Documentation License Contributors: This hand-written SVG version
by Qef Original bitmap version Image:Absolute_value.png by Ævar Arnfjörð Bjarmason
Image:Lur'e_Problem_Block.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Lur'e_Problem_Block.jpg License: Creative Commons Attribution 3.0 Contributors: Pivs
File:Centrifugal governor.png Source: http://en.wikipedia.org/w/index.php?title=File:Centrifugal_governor.png License: Public Domain Contributors: Dicklyon, Glenn, Guam, Liftarn, Mdd,
MdeVicente, Pieter Kuiper, Tano4595, WikipediaMaster, 2 anonymous edits
Image:MRAC.svg Source: http://en.wikipedia.org/w/index.php?title=File:MRAC.svg License: Public Domain Contributors: Original uploader was Pekaje at en.wikipedia
Image:MIAC.svg Source: http://en.wikipedia.org/w/index.php?title=File:MIAC.svg License: Public Domain Contributors: Original uploader was Pekaje at en.wikipedia
Image:Neural network example.svg Source: http://en.wikipedia.org/w/index.php?title=File:Neural_network_example.svg License: Public Domain Contributors: User:Wiso
File:Forest of synthetic pyramidal dendrites grown using Cajal's laws of neuronal branching.png Source:
http://en.wikipedia.org/w/index.php?title=File:Forest_of_synthetic_pyramidal_dendrites_grown_using_Cajal's_laws_of_neuronal_branching.png License: Creative Commons Attribution 2.5
Contributors: Mietchen, NeverDoING
Image:Fuzzy logic temperature en.svg Source: http://en.wikipedia.org/w/index.php?title=File:Fuzzy_logic_temperature_en.svg License: Creative Commons Attribution-ShareAlike 3.0
Unported Contributors: fullofstars
Image:RL&ZARL-(1 2)-(1 3 5 1).png Source: http://en.wikipedia.org/w/index.php?title=File:RL&ZARL-(1_2)-(1_3_5_1).png License: Public Domain Contributors: Mintz l
File:Root-locus-example-diagram.png Source: http://en.wikipedia.org/w/index.php?title=File:Root-locus-example-diagram.png License: Public Domain Contributors: Sss41
Image:Step response.png Source: http://en.wikipedia.org/w/index.php?title=File:Step_response.png License: GNU Free Documentation License Contributors: Brews ohare
File:Optical_Encoders.png Source: http://en.wikipedia.org/w/index.php?title=File:Optical_Encoders.png License: Creative Commons Attribution-Share Alike Contributors: Fiducial
File:Visualisierung der magnetischen Struktur eines Linearencoders (Aufnahme mit MagView).jpg Source:
http://en.wikipedia.org/w/index.php?title=File:Visualisierung_der_magnetischen_Struktur_eines_Linearencoders_(Aufnahme_mit_MagView).jpg License: Creative Commons
Attribution-Sharealike 3.0 Contributors: User:Matesy
File:Circular Lissajous.gif Source: http://en.wikipedia.org/w/index.php?title=File:Circular_Lissajous.gif License: Creative Commons Attribution-Sharealike 3.0 Contributors: Fiducial
File:Quadrature Diagram.svg Source: http://en.wikipedia.org/w/index.php?title=File:Quadrature_Diagram.svg License: Public Domain Contributors: Sagsaw
Image:Reflected binary Gray 2632058.png Source: http://en.wikipedia.org/w/index.php?title=File:Reflected_binary_Gray_2632058.png License: Public Domain Contributors: Bulwersator,
Dicklyon, Liftarn, Tassedethe
Image:US02632058 Gray.png Source: http://en.wikipedia.org/w/index.php?title=File:US02632058_Gray.png License: Public Domain Contributors: Original uploader was Dicklyon at
en.wikipedia
Image:Encoder Disc (3-Bit).svg Source: http://en.wikipedia.org/w/index.php?title=File:Encoder_Disc_(3-Bit).svg License: Public Domain Contributors: jjbeard
File:Gray code rotary encoder 13-track opened.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Gray_code_rotary_encoder_13-track_opened.jpg License: Public Domain
Contributors: Mike1024
File:Binary-reflected Gray code construction.svg Source: http://en.wikipedia.org/w/index.php?title=File:Binary-reflected_Gray_code_construction.svg License: Public Domain Contributors:
Inductiveload
File:Gray code permutation matrix 16.svg Source: http://en.wikipedia.org/w/index.php?title=File:Gray_code_permutation_matrix_16.svg License: Public Domain Contributors: Lipedia
Image:Enkelspoors-Graycode.svg Source: http://en.wikipedia.org/w/index.php?title=File:Enkelspoors-Graycode.svg License: Creative Commons Attribution-Sharealike 2.5 Contributors:
Gerbrant
Image:Rotatory EnDat Encoder.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Rotatory_EnDat_Encoder.jpg License: GNU Free Documentation License Contributors: IP83
Image:Encoder disc (3-Bit binary).svg Source: http://en.wikipedia.org/w/index.php?title=File:Encoder_disc_(3-Bit_binary).svg License: Creative Commons Attribution-ShareAlike 3.0
Unported Contributors: en:User:Cburnett
File:ROD420 HEIDENHAIN.jpg Source: http://en.wikipedia.org/w/index.php?title=File:ROD420_HEIDENHAIN.jpg License: Creative Commons Attribution 3.0 Contributors: Victor
Korniyenko
Image:Quadrature Diagram.svg Source: http://en.wikipedia.org/w/index.php?title=File:Quadrature_Diagram.svg License: Public Domain Contributors: Sagsaw
321
Image Sources, Licenses and Contributors
Image:Odometergearcropped.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Odometergearcropped.jpg License: GNU Free Documentation License Contributors: Original
uploader was Nagle at en.wikipedia
Image:Bode High-Pass.PNG Source: http://en.wikipedia.org/w/index.php?title=File:Bode_High-Pass.PNG License: GNU Free Documentation License Contributors: Brews ohare
Image:Bode Low-Pass.PNG Source: http://en.wikipedia.org/w/index.php?title=File:Bode_Low-Pass.PNG License: Creative Commons Attribution-Share Alike Contributors: Brews ohare
Image:Bode Low Pass Magnitude Plot.PNG Source: http://en.wikipedia.org/w/index.php?title=File:Bode_Low_Pass_Magnitude_Plot.PNG License: GNU Free Documentation License
Contributors: Brews ohare
Image:Bode Low Pass Phase Plot.PNG Source: http://en.wikipedia.org/w/index.php?title=File:Bode_Low_Pass_Phase_Plot.PNG License: GNU Free Documentation License Contributors:
Brews ohare
Image:Bode Pole-Zero Magnitude Plot.PNG Source: http://en.wikipedia.org/w/index.php?title=File:Bode_Pole-Zero_Magnitude_Plot.PNG License: GNU Free Documentation License
Contributors: Brews ohare
Image:Bode Pole-Zero Phase Plot.PNG Source: http://en.wikipedia.org/w/index.php?title=File:Bode_Pole-Zero_Phase_Plot.PNG License: GNU Free Documentation License Contributors:
Brews ohare
Image:Magnitude of feedback amplifier.PNG Source: http://en.wikipedia.org/w/index.php?title=File:Magnitude_of_feedback_amplifier.PNG License: Creative Commons Attribution-Share
Alike Contributors: Brews ohare
Image:Phase of feedback amplifier.PNG Source: http://en.wikipedia.org/w/index.php?title=File:Phase_of_feedback_amplifier.PNG License: GNU Free Documentation License Contributors:
Brews ohare
Image:Gain Margin.PNG Source: http://en.wikipedia.org/w/index.php?title=File:Gain_Margin.PNG License: GNU Free Documentation License Contributors: Brews ohare
Image:Phase Margin.PNG Source: http://en.wikipedia.org/w/index.php?title=File:Phase_Margin.PNG License: GNU Free Documentation License Contributors: Brews ohare
Image:Bodeplot.png Source: http://en.wikipedia.org/w/index.php?title=File:Bodeplot.png License: Public Domain Contributors: derivative work: Stw (talk) Bodeplot.JPG: Vukg
Image:Nyquist.svg Source: http://en.wikipedia.org/w/index.php?title=File:Nyquist.svg License: Public Domain Contributors: Engelec
Image:Nichols.svg Source: http://en.wikipedia.org/w/index.php?title=File:Nichols.svg License: Public domain Contributors: Engelec (talk)
Image:Nyquist example.svg Source: http://en.wikipedia.org/w/index.php?title=File:Nyquist_example.svg License: Public Domain Contributors: Utkarsh Upadhyay,--Krishnavedala (talk)
20:21, 3 June 2011 (UTC) (python version)
Image:Unstable3.svg Source: http://en.wikipedia.org/w/index.php?title=File:Unstable3.svg License: GNU Free Documentation License Contributors: Ma-Lik, Pokipsy76
Image:HD-Rayleigh-Taylor.gif Source: http://en.wikipedia.org/w/index.php?title=File:HD-Rayleigh-Taylor.gif License: Public Domain Contributors: Shengtai Li, Hui Li
Image:Fluid Instability.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Fluid_Instability.jpg License: Creative Commons Attribution-Sharealike 3.0 Contributors: Sungjune Jung
Image:Butterworth response.svg Source: http://en.wikipedia.org/w/index.php?title=File:Butterworth_response.svg License: Creative Commons Attribution-Sharealike 3.0,2.5,2.0,1.0
Contributors: derivative work: Krishnavedala (talk) Butterworth_response.png: Omegatron
Image:Baseband.svg Source: http://en.wikipedia.org/w/index.php?title=File:Baseband.svg License: Public Domain Contributors: Jhbdel (talk)
Image:Bandwidth 2.svg Source: http://en.wikipedia.org/w/index.php?title=File:Bandwidth_2.svg License: Public Domain Contributors: Inductiveload
File:Static VAR Compensator 2a.png Source: http://en.wikipedia.org/w/index.php?title=File:Static_VAR_Compensator_2a.png License: Creative Commons Attribution-Sharealike 3.0
Contributors: User:Clampower
Image:PID en.svg Source: http://en.wikipedia.org/w/index.php?title=File:PID_en.svg License: Creative Commons Attribution-Sharealike 3.0 Contributors: Zerodamage
Image:Change with Kp.png Source: http://en.wikipedia.org/w/index.php?title=File:Change_with_Kp.png License: Public Domain Contributors: Skorkmaz
Image:Change with Ki.png Source: http://en.wikipedia.org/w/index.php?title=File:Change_with_Ki.png License: Public Domain Contributors: Skorkmaz
Image:Change with Kd.png Source: http://en.wikipedia.org/w/index.php?title=File:Change_with_Kd.png License: unknown Contributors: Skorkmaz
File:Scross helmsman.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Scross_helmsman.jpg License: Creative Commons Attribution-Sharealike 2.5 Contributors: Original uploader
was Fishdecoy at en.wikipedia
Image:PI controller.png Source: http://en.wikipedia.org/w/index.php?title=File:PI_controller.png License: Public Domain Contributors: Email4mobile (talk)Email4mobile
322
License
License
Creative Commons Attribution-Share Alike 3.0 Unported
//creativecommons.org/licenses/by-sa/3.0/
323