2. Rationality and Performance Measures

Source: AIMA 4th Ed, Chapter 2 (Section 2.2), physical PDF pp. 102–107

Introduction

Once we know what an agent is, the immediate question is: what makes one agent better than another? Section 2.2 answers this by defining rationality formally. The answer is not “acts like a human” or “always picks the optimal action” — it is something more carefully calibrated: acting to maximize expected performance given available information.

2.2.1 Performance Measures

Consequentialism

AI evaluates agents by their consequences. This philosophical stance — called consequentialism — says we evaluate behavior by what it produces, not by the process used or the intentions behind it.

An agent placed in an environment: 1. Receives percepts 2. Generates a sequence of actions 3. Those actions cause the environment to pass through a sequence of states

If the sequence of states is desirable, the agent has performed well. Desirability is captured by a performance measure that evaluates any given sequence of environment states.

Performance Measure

A performance measure is the objective criterion for evaluating agent success — it assigns a score to sequences of environment states.

Key design principle (from the text, stated in italics):

As a general rule, it is better to design performance measures according to what one actually wants to be achieved in the environment, rather than according to how one thinks the agent should behave.

Example — vacuum agent: - Bad performance measure: “amount of dirt sucked up in a single 8-hour shift.” A rational agent maximizing this will clean the floor, dump dirt back, clean again. - Better: “one point per clean square per time step (with penalty for electricity consumed).” This rewards the state of having a clean floor, not the act of cleaning.

The King Midas Problem: Always specifying performance measures correctly is hard. If you specify the wrong measure, a perfectly rational agent will optimize the wrong thing — you get exactly what you asked for, not what you wanted. This motivates the value alignment problem discussed in Chapter 1.

Machines vs. Humans

For humans, the performance measure is in the agent’s own mind (desires, preferences). For machines, it is initially in the mind of the designer. Some designs represent the performance measure explicitly; others have it entirely implicit. Either way, the machine may need to learn more about the true performance measure over time (see Chapters 16, 18, 22).

2.2.2 Rationality

The Four Determinants of Rationality

What is rational at any given time depends on exactly four things:

  1. The performance measure that defines the criterion of success
  2. The agent’s prior knowledge of the environment
  3. The actions that the agent can perform
  4. The agent’s percept sequence to date

Formal Definition of a Rational Agent

For each possible percept sequence, a rational agent should select an action that is expected to maximize its performance measure, given the evidence provided by the percept sequence and whatever built-in knowledge the agent has.

This is the definition to memorize for quals. Note what it says and what it does not say: - It says “expected to maximize” — not “actually maximizes” (no crystal ball required) - It says “given the evidence” — rationality is relative to available information - It does not require human-level intelligence; a thermostat can be rational

Vacuum Agent — Is It Rational?

Given the following setup: - Performance: +1 per clean square per time step, over 1000 steps - Geography known, dirt distribution and initial location unknown - Actions: Right, Left, Suck - Sensors: location and cleanliness of current square, correctly perceived

Under these conditions, the simple reflex vacuum agent (suck if dirty, else move) is rational — its expected performance is at least as good as any other agent’s.

Under different conditions it may be irrational. If clean squares can’t become dirty again, the agent wastefully oscillates. A rational agent would stop once the floor is clean. The key lesson: rationality is always relative to the performance measure, prior knowledge, and percept sequence.

2.2.3 Omniscience, Learning, and Autonomy

Rationality vs. Omniscience

Omniscience = knowing the actual outcome of every action.

Rationality does not require omniscience. Consider the example: you rationally cross the street because no traffic is visible. A cargo door falls from a passing airliner and kills you. Were you irrational? No — your obituary would not read “Idiot attempts to cross street.”

The key distinction: - Rationality maximizes expected performance (given available information) - Omniscience maximizes actual performance (which requires knowing the future)

Requiring actual optimality would make it impossible to design any rational agent — you’d need a time machine.

Information Gathering and Exploration

Rationality does require gathering information when doing so improves expected performance. Examples: - Looking both ways before crossing a street (the “looking” action modifies future percepts) - Exploration by a vacuum agent in an initially unknown environment

Doing actions specifically to gather information for future decisions is called information gathering. It is a rational strategy, not a detour from rationality.

Learning

A rational agent must also learn from what it perceives. Initial configuration may encode prior knowledge, but as the agent gains experience, that knowledge can be augmented.

Extreme case: if the environment is completely known a priori and completely predictable, the agent need not perceive or learn at all — it simply executes the optimal plan. Such agents are fragile:

Dung beetle example: Evolution hard-coded the behavior “fetch dung ball to plug nest.” If the ball is removed mid-task, the beetle continues its pantomime indefinitely — it cannot learn that the plan is failing.

Sphex wasp example: Similarly, the sphex wasp loops endlessly on a check-and-drag sequence if the caterpillar is moved each time it checks. No ability to learn that the innate plan is failing.

Autonomy

To the extent an agent relies on the designer’s prior knowledge rather than its own percepts and learning, the agent lacks autonomy.

Definition: A rational agent should be autonomous — it should learn to compensate for partial or incorrect prior knowledge.

Practical note: complete autonomy from the start is unreasonable. With no experience, an agent would have to act randomly. Like evolution providing animals enough built-in reflexes to survive long enough to learn, it is reasonable to provide a new agent with some initial knowledge plus the ability to learn. After sufficient experience, the agent’s behavior can become effectively independent of prior knowledge.

The payoff: incorporating learning allows a single rational agent design to succeed across a vast variety of environments — it does not need to be re-engineered for each new environment.

Key Equations and Formal Notation

There is no single “rationality equation” in Section 2.2, but the formal statement is precise enough to be used as a definition:

Rational action at time t:
    a* = argmax_a E[performance | percept_sequence_1..t, a, prior_knowledge]

where the expectation is over possible outcomes of action a given the agent’s beliefs about the environment state.

Summary Table

Concept Definition Key nuance
Performance measure Score function over environment state sequences Design for desired outcomes, not agent behavior
Rationality Maximize expected performance given percepts + knowledge Not omniscience; not perfection
Omniscience Know actual outcomes Impossible; not required for rationality
Information gathering Acting to improve future percepts Part of rationality, not a detour
Learning Updating knowledge from experience Required for robust rationality
Autonomy Relying on own percepts/learning vs. designer’s knowledge Good agents become autonomous over time

Common Exam Pitfalls

  1. Rationality ≠ optimality. A rational agent maximizes expected performance; an optimal agent maximizes actual performance. The latter is impossible to guarantee.
  2. Performance measure must evaluate state sequences, not actions. Rewarding the act of cleaning (not the state of cleanliness) invites Goodhart’s Law pathologies.
  3. Rationality is always relative. There is no absolute “rational action” — it depends on the performance measure, prior knowledge, available actions, and current percepts.
  4. Autonomy is a spectrum. New agents need some initial knowledge; fully autonomous agents rely entirely on learning. Both extremes are pathological in practice.

Cross-References