Knowledge-based agents

Games & Agents 2012-2013
Games & Agents
Lecture 2
Agents and Intelligence
Jan Broersen
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
The subject of today
I will give a very broad overview of different ideas from AI about
endowing agents with intelligence.
First: a movie about how the game-industry looks at intelligence
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
The AI in Killzone 2
[email protected]
Tuesday, May 28, 13
Intelligence
Games & Agents 2012-2013
•
In the trailer:
– (Agent) intelligence: (1) enemies pin you down and rush up to
you, (2) you do not see the same thing twice, (3) the
environment is destructible (4) the AI responds to changes
in the environment, (5) lean and peak system
– Multi-agent intelligence: buddies in the game that tell you to
take cover, saving your skin
•
In AI:
– Intelligence = intelligent behavior (we endorse Turing’s test)
– Intelligent behavior = selecting the ‘intelligent choices’ from
all the ones that are feasible and performing them
– Agent theory: intelligence is a function from percepts to
actions; the function can either be programmed or learned
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
Simple reflex agents
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
Simple Reflex Agent
[email protected]
Tuesday, May 28, 13
Decision tree
Games & Agents 2012-2013
Conditionalization of actions is
of the simplest kind
In the tree: decisions at the
squares depend only on
observations at these squares.
Cycle: sense, act
[email protected]
Tuesday, May 28, 13
Are these used in computer games?
Games & Agents 2012-2013
Yes: these are the scripted agents
Where do agent actions in scripts conditionalize on?
Very simple information like the player being nearby
However, intelligent choices should
1.
2.
3.
4.
be conditional on the state of the environment
be conditional on the state of other agents
be conditional on internal states
adapt after earlier experiences
[email protected]
Tuesday, May 28, 13
Shortcomings simple reflex agents
Games & Agents 2012-2013
(1) No conditionalization on appropriate circumstances
(2) No situation awareness
(3) No looking ahead for decision making
=> agents need a model of the world, that is, we have to give them
knowledge
=> agents need a goal in life, that is, we have to give them
motivations (desires, intentions, obligations, etc.)
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
Knowledge-based agents
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
Reflex + State Agent
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
Decision tree
Decisions conditionalize not
directly on the percepts, but
on the agent’s model of the
world (recalls what happened
before and what it has done)
How the world is, may now be
derived instead of observed
Allows for ‘reasoning’ before
acting
Cycle: sense, reason, act
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
Knowledge-based agents
Perceived as a major break-through in the late 50ies, just after the
birth of artificial intelligence as a discipline (maybe the same
brake-through still has to be realized in game-AI..)
Model not only knowledge, but also lack thereof (uncertainty)
Logic based approach 1: epistemic logic
Logic based approach 2: non-monotonic reasoning
Bayesian approach: beliefs as probabilities
[email protected]
Tuesday, May 28, 13
Are these used in computer games?
Games & Agents 2012-2013
Hardly (do you know of examples?)
However, if we agree the cognitive SOAR architecture gives us
knowledge based agents, then we do have an interesting
example.
Paper: [It knows what you’re going to do: adding anticipation to a
Quakebot, John Laird]
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
A cognitive architecture: SOAR
Close to ACT-R
SOAR is claimed to be:
a candidate unified theory of cognition (Newell, 1990),
and
an architecture for intelligent behaviour
Decisions are never precompiled into uninterruptible
sequences => continuous (and concurrent) interaction
with the environment and with multiple internal
processes.
[email protected]
Tuesday, May 28, 13
The SOAR Architecture
Games & Agents 2012-2013
SOAR is an ‘AI engine’ for making and executing decisions
Basic objects of decision: operators (we would call them actions)
Operators are either environmental actions, internal actions,
complex actions, or abstract goals.
SOAR is completely rule-based
So, far, very much like 3APL/2APL
[email protected]
Tuesday, May 28, 13
The SOAR Architecture
Long-Term Memories
Games & Agents 2012-2013
Procedural/Skill
Semantic
Learning
Rule
Learning
Short-Term Memory
Control
Procedure
Action
Perception
[email protected]
Tuesday, May 28, 13
Semantic/Concept
Body
Episodic
Episodic
Learning
Games & Agents 2012-2013
But, unlike in 3APL/2APL:
There is no deliberation or control language! Also there is no
predefined order in which to apply rules: the system is
completely data-driven.
The current state (‘working memory’) consists of perceptions,
elaborations on it, and everything else derived by earlier rule
applications.
SOAR continually and concurrently proposes, selects and applies
operator rules to the current state.
Abstract (goal) operators that cannot be executed directly generate
sub-states for which additional operators are proposed, selected
and applied (remember HTN-planning)
[email protected]
Tuesday, May 28, 13
Adding anticipation to a Quakebot
Games & Agents 2012-2013
•
•
•
Central idea: enable a Quakebot to project itself in its
enemies to predict its behavior and to take advantage of
that.
Similar to a 1 ply deep minimax or alfa-beta algorithm
(Chess, game theory) where the idea is also to use the
same evaluation function for opponents.
Add three new sets of rules to SOAR bots:
1. For deciding when to start predicting
2. For generating expected opponent behavior
3. For proposing when to stop predicting and for how to
take advantage of the prediction (e.g. ambush)
[email protected]
Tuesday, May 28, 13
When to start predicting? (task 1)
Games & Agents 2012-2013
Not always..
Decided by a special set of rules
Which are up to the Quakebot designer..
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
Generating expected behavior (task 2)
First the quake bot generates an expected internal state of its
opponent, by applying special rules on his perceptions of the
other (his position, health, armor level, current weapon).
Then he applies his own behavioral rules on this state, thereby
projecting himself in the other!
The behavioral projection works on the generated internal state and
projected environment, which is less computational intensive
than the ‘real thing’.
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
Predictions, and how they are used (task 3)
Predictions should not go on forever: special rules decide when to
stop. For instance if an ambush is not possible due to
environmental conditions or the timing.
Predictions also stop when they encounter too much uncertainty
(impasses in the prediction!).
In the paper’s set-up the technique is only used to hunt, ambush and
deny power-ups.
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
Limitations
Anticipations is restricted to situations deemed worthwhile by the
programmer.
So, tactics are not ‘discovered’ but preprogrammed by the Quake
bot designer.
Alternative: always do predictions, and add general evaluation
functions and planning ⇒ too expensive, computationally.
[email protected]
Tuesday, May 28, 13
Conclusions SOAR application to Quake
Games & Agents 2012-2013
SOAR quake bots use about 750 rules.
Adding anticipation requires only 4 rules for deciding when to start
predicting, 16 to generate an approximate opponent internal
state, 5 to use the prediction.
So adding anticipation turns out to be quite easy, from a
programming point of view.
But, computational much more intensive.
[email protected]
Tuesday, May 28, 13
Shortcomings of knowledge-based agents
Games & Agents 2012-2013
Goals are implicit in the set of rules
Agents decide without explicitly looking ahead in the tree
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
Goal-based Agent
[email protected]
Tuesday, May 28, 13
Deliberation cycles for the different agent types
Games & Agents 2012-2013
Simple reflex agents: sense, act
Reflex agents with state: sense, reason, act
Goal-based agents: sense, update beliefs, select goals, select
intentions, etc..., act
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
The tree view: planning
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
Are these agents used in computer games?
Yes: path-planning using A*
Knowledge: the network of waypoints
Goal: finding a shortest route
However, this is a very specific type of planning, because actions
(movements) and goals (wanting to be somewhere else)
themselves are very specific
Knowledge about waypoints is simply assumed (and not learned
through observation)
Because the problem is so specific, we might even consider to
calculate most of it offline for a given fixed environment..
[email protected]
Tuesday, May 28, 13
Planning
Games & Agents 2012-2013
See course “intelligent systems”
State space planning versus plan space planning (HTN planning)
Planning under uncertainty (conformant planning, contingency
planning, etc.)
Main focus of research: controlling complexity, heuristics
[email protected]
Tuesday, May 28, 13
BDI-theory
Games & Agents 2012-2013
Prominent theory: Belief-Desire-Intention
Issue: how do all three depend on each other?
Example: commitment strategies [Cohen and Levesque]: you keep
an intention as long as you belief the associated desire to be
attainable
Example: realism: you do not desire things you believe to be true
already; you do not believe things only because they match your
desire (wishful thinking)
[email protected]
Tuesday, May 28, 13
3APL/2APL
Games & Agents 2012-2013
Inspired by BDI
Goal-base, Belief-base, and Plan-base (intentions)
A language to specify the deliberation cycle
Intuitions from Prolog
[email protected]
Tuesday, May 28, 13
Shortcomings
Games & Agents 2012-2013
Knowledge is not ‘fine grained’
Goals are not ‘fine grained’
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
Utility-based agents
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
Utility-based agent
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
Maximal expected utility
• Cycle: sense, find optimal
tactic given beliefs,
utilities, act
• Reasoning is now in terms
of utilities, chances and
look aheads: MEU, MDPs,
PoMDPs
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
MDPs and PoMDPs
Again see the theory explained in the course “intelligent
systems” (Policies, Bellman equations, value iteration, policy
iteration, etc.)
I do not know of any example from computer games where this
agent theory is applied (do you?)
[email protected]
Tuesday, May 28, 13
Bayesian update
Games & Agents 2012-2013
Bayes' theorem adjusts probabilities given new evidence in the following way:
Where H represents a specific hypothesis, which may or may not be some null
hypothesis.
P(H) is called the prior probability of H that was inferred before new evidence,
E, became available.
P(E | H) is called the conditional probability of seeing the evidence E if the
hypothesis H happens to be true. It is also called a likelihood function when it
is considered as a function of H for fixed E.
P(E) is called the marginal probability of E: the a priori probability of
witnessing the new evidence E under all possible hypotheses.
P(H | E) is called the posterior probability of H given E.
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
The three prisoners
Three prisoners A, B and C are told they have a 2/3 chance of being
executed.
At a certain point the decision is made by the director of the prison.
Then the director visits prisoner A. A asks: will B or C be
executed? The director answers: B
What is A’s chance of survival?
Answer: unclear!
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
The three doors
The Monty Hall dilemma
Conditionalizing on extra information
The chances for getting the car when not changing are 1/3. When
changing, the chances are 2/3! (1/3 for initially correct and
changing to the wrong door and 2/3 for initially wrong after
which changing guarantees the good outcome).
[email protected]
Tuesday, May 28, 13
Shortcomings
Games & Agents 2012-2013
• However, No learning (like all of the previous)
• Or, is there?
• Any update of the model of the world can be seen as a
learning action
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
Learning Agent
Can be combined with any of the previous
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
Trends in Neural Networks
There has always been the “black box” criticism towards neural
networks (I call it “the tweaking problem”)
Current trend: add structure to neural networks
How should such structure look like?
Inspiration from biology?
NEAT: the structure of the neural network evolves determined by a
genetic algorithm
Liquid state machines
Neuro-symbolic computing
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
Rationality versus emotion
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
Some functions of emotions
Frijda: emotions are (search) heuristics.
Emotions may thus be said to guide planning.
E.g. happiness may keep an agent on the right track, because it
makes him selecting similar choices to reinforce his happiness.
E.g. anger or distress may guide an agent into selecting choices he
would otherwise not consider. Those choices maybe the solution
to his problem.
A thought: A* is an optimistic algorithm. Because of that it is
complete (always finds a solution if it exists) and optimal (it will
always find the ‘best’ solution). Optimism seems to be related to
positive emotions...
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
Emotions and rationality
What is the relation between rationality and emotion? Both are
involved in selecting (intelligent) decisions.
For instance, rationally you estimate it is better to kill your enemy,
or he will probably kill you. But, emotionally you feel extremely
bad about killing.
Or, rationally you estimate it is good to punish your child,
emotionally you detest it.
There is an example case of a man without emotion due to brain
damage. He kept an average IQ, but could no longer decide on
simple things like what to wear.
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
The Multi-Agent paradigm
[email protected]
Tuesday, May 28, 13
Computing perspective
Games & Agents 2012-2013
Multi-agent systems constitute a new computational paradigm
based on:
Distribution of computational power
Interconnection and communication
Autonomy and Intelligence
Delegation and distribution of control
Anthropomorphic concepts
[email protected]
Tuesday, May 28, 13
Software engineering perspective
Games & Agents 2012-2013
Multi-agent systems are a new software engineering paradigm.
Analysis ⇒ Design ⇒ Implementation ⇒ Test
The aim is to provide high-level abstraction, to model and develop complex
systems
70’s: Structural analysis methodology
80’s / 90’s: Object-oriented methodology
90’s / 00’s: Agent-oriented methodology
[email protected]
Tuesday, May 28, 13
Artificial intelligence perspective
Games & Agents 2012-2013
Multi-agent systems and Artificial Intelligence do not coincide.
Artificial Intelligence
Planning
Learning
Vision
Language understanding
Multi-agent Systems
Understand and model social intelligence and emergent behavior
Interaction and Communication
Social concepts: obligation, norms, responsibilities, etc.
Optimal solutions can be obtained by co-ordination and co-operation
Simulation can verify social and economic theory
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
Characteristics of multi-agent systems
Multi-agent systems consist of a number of interacting autonomous agents.
Multi-agent systems are designed to achieve some global goal.
Agents need special abilities to cooperate, coordinate, and negotiate to achieve
their objectives.
Multi-agent systems are specified in terms of high-level abstract concepts such as
role, permission, responsibility, and interaction.
Applications for multi-agent systems are, for example, power management
systems, transportation systems, and auctions.
[email protected]
Tuesday, May 28, 13
Autonomous agents
Games & Agents 2012-2013
Autonomous agents are intelligent agents from a multi-agent perspective.
Autonomous agents are active, social, and adaptable computer systems situated in
some dynamic environment that autonomously perform actions to achieve
their objectives.
1.
2.
3.
4.
Reactivity: responding to changes in the environment
Pro-activity (deliberativeness): goal-directed behavior
Social awareness: interaction and communication
Rationality: maximization of utilities
•
•
Agents decide which action to perform based on their internal state.
The internal state of agents can be specified in terms of high-level abstract
concepts such as belief, desire, goal, intention, plan, and action.
[email protected]
Tuesday, May 28, 13
Games & Agents 2012-2013
The End
[email protected]
Tuesday, May 28, 13