es might be inferred from the actions of each side (for example, offers made or responses to offers proposed). Incomplete information is expressed as uncertainty regarding the utility preferences of the opponent, and it is assumed there is a finite set of different negotiator types. These types are associated with different additive utility functions (for example, one type might have a long-term orientation regarding the final agreement, while the other type might have a more constrained orientation). Lastly, the negotiation is conducted once with each opponent.

As for incomplete information, the QOAgent tackles the problem by applying a simple Bayesian update mechanism, which, after each action tries to infer which utility best suits the opponent (when receiving an offer or when receiving a response to an offer). For the decision-making process, the approach used by the QOAgent is more of a qualitative approach. 36 While the QOAgent’s model applies utility functions, it is based on a non-classical decision-making method, rather than focusing on maximizing the expected utility. The QOAgent uses the maximin function and the qualitative valuation of offers. Using these methods the QOAgent generates offers and decides whether to accept or reject proposals it has received.

Lin et al. 24 tested the QOAgent in several distinct domains and their results

show that the QOAgent reaches more agreements and plays more effectively than its human counterparts, when the effectiveness is measured by the score of the individual utility. They also show that the sum of utilities is higher in negotiations when the QOAgent is involved, as compared to human-human negotiations. Thus, they assert, it is indeed possible to build an automated agent that can negotiate successfully with humans. However, it is also important to state that their agent has certain limitations. They assume there is a finite set of different agent types and thus their agent cannot generate a dynamic model (and perhaps a more accurate one) of the opponent. In addition, they have not shown whether their agent can also maintain high scores when matched with other automated agents, which is an important characteristic of open environment negotiations. Moreover, the QOAgent does not scale well when numerous offers are proposed, which can cause its performance to deteriorate.

Finally, we conclude with a description of a more complex type of agent that incorporates many features, far beyond the negotiation strategy itself.

 

The Virtual Human Agent Kenny et al. 19 describe work on virtual humans used for interpersonal training for skills, such as: negotiation, leadership, interviewing, and cultural

main contributions of each agent.

Agent

Diplomat

AutONA

Cliff-Edge

Colored-Trails

Guessing Heuristic

QOAgent

Virtual Human

main contribution Changing the agent’s personality heuristics Non-deterministic behavior / randomization Tactics and heuristics incorporating data from past interactions Concession mechanism Virtual learning incorporating data from past interactions Gender-sensitive approach Non-deterministic behavior / randomization (implicitly) incorporating data from past interactions Machine learning Generic agent / domain independent Concession mechanism Generic agent / domain independent Qualitative decision making Non-deterministic behavior / randomization Tactics and heuristics Cognitive architecture

training. To achieve this they require a large amount of research in many fields (such as, knowledge representation, cognitive and emotional modeling, natural language processing, among others). Their intelligent agent is based on the Soar Cognitive Architecture, which is a symbolic reasoning system used to make decisions.

Traum et al. discuss the negotiation strategies of the virtual human agent in more detail. 37 In their paper they describe a set of strategies implemented by the agent (for example, when to act aggressively if it seems that the current outcome will incur a negative utility, or when to find the appropriate issue on which to currently negotiate). The strategy chosen each time is influenced by several factors: the control the agent has over the negotiations, the estimated utility of an outcome and the estimated best utility of an outcome, the trust the agent bestows the opponent and the commitment of all agents to the given issues. The virtual agent also tries to model the opponent by reasoning about its mental state.

Traum et al. tested their agents in several negotiation scenarios. One of these scenarios is a simulation for soldiers that practice and conduct bilateral engagements with virtual humans, and in situations in which culture plays an important role. In this case, the different actions can be selected from a menu that includes appropriate questions based on the history of the simulation thus far. The second domain requires trainees to communicate with an embodied virtual human doctor to negotiate and convince him to move a clinic, located in a middle of a war zone, out of harm’s way (see Figure 2). Their prototypes are continuously tested with cadets and civilians. Traum et al. are more concerned with the system as a whole and thus they do not provide insights with respect to the proficiency of their automated negotiator. Regarding the environment, they state that the subjects enjoy using the system for negotiations and that it also allows them to learn from their mistakes.

Traum also report some of the existing limitations of their system. Currently, the virtual agent cannot consider arbitrary offers made by a human negotiator. In addition, more strategies are required to better cover the