Summary

This paper is about the challenges seen as of 2004 about automation components effectively working with humans. In modern research parlance we would likely call the automation components AI. Some automation components whose behavior is well understood and intuitive to humans might not be called AI though and this shows an example of how we move the goal post once the technological tooling is performing well.

Something as old as human history.

Central premise: A Basic Compact

The basic compact is something in any set of actions carried out by > 1 humans would see as plausible

1. agree amongst the humans that they intend to work together on a task
2. behave in mutually predictable ways
3. be mutually directable
4. maintain common ground
5. Goal alignment

Some subtler points

1. Humans need to signal their dropout behavior if they intend to leave the compact
2. If there behavior is unpredictable the participant should have a signalling mechanism for this unpredictable state.
3. Mutual detection and repair of faulty knowledge, beliefs, and assumptions.

The Ten Challenges

1. Fulfill the requirements of the basic contract to engage in common-grounding activities
2. Participants can adequately model other participants intentions and actions vis-a-vis join activity state and evolution.
3. Human-Agent team members must be mutually predictable.
4. Agents must directable, e.g. agent autonomy is something configurable.
5. Status, intention, and signaling of an agent must be communicated in an obvious way to teammates (humans and AI)
6. Agents must be able to observe and interpret pertinent signals of status and intent.
7. Agents must be able to engage in goal negotation.
8. Support technology for planning and autonomy must enable collaboration in all forms human-human, humain-AI, AI-AI.
9. Attention management must be both controlled individually and also shared in the team.
10. All team members-humans and AIs-must help control costs of coordinated activity.

One challenge the authors mention is passing but is very relevant is the need to align shared and non-shared goals with short and long term time horizons. There is nothing about time horizons in the Ten challenges per-se and this is a limitation of the enumeration but not of the paper itself because the time horizon is mentioned as an issue.

Purpose

The authors present the challegnes as both a blueprint for reseach and development as well as cautionary tales for designers and as a practicable basis for human-AI systems.

The paper is available here.