lar situation, and that can look ahead and predict both supply and demand trends in the near future, in order to prepare for future reductions in available supply, or to make the most effective use of supply when it is available.

The design of such intelligent systems is challenged by the complexity of the domains in which they are deployed. For example, within a home, demand reduction may involve shifting the time of use of a number of electrical appliances, each with their own individual constraints (for example, lighting cannot be shifted, a washing machine can be shifted by a day or two, while a dishwasher may be shiftable by a few hours24). Similarly, both heating (given that this will likely be electrified through the use of efficient heat pumps) and cooling loads can be shifted as long as the comfort and temperature preferences of the householders are met. To be effective in this, it may also be necessary for such systems to learn the thermal properties of the home in which they are deployed, as well as the local weather conditions, and the way in which these local conditions impact on the heat loss, or gain, of the home. Crucially, these approaches will have to take into account the fact that each individual householder will have his or her own preferences, and these preferences must either be explicitly elicited, or learned. Since these preferences are likely to exhibit change over time, and depend on the current activities of the householder and local weather conditions, in computational terms this translates into an online learning and scheduling problem under uncertainty.

Similarly, commercial and industrial consumers will be constrained by existing contracts and commercial considerations (for example, a factory may have to deliver products within certain deadlines, while a data center has to be available to its customers 24 hours a day), and must balance demand reduction against these additional factors. Large industrial consumers of electricity with significant heating, cooling, or pumping loads may have considerable flexibility regarding when they actually consume electricity as long as some overarching constraints are satisfied.e e During the 2000 California electricity crisis, which saw extremely high spot prices, several

it will be important
to design simulation
systems that
can accurately
represent both
the grid and
the reaction of
consumers, in
order to predict
the emergent
properties of the
system under a
range of different
conditions and
worst-case
scenarios.

However, to do so in a responsive way requires that the usage optimization algorithm that is deployed is able to model and predict both the prices within the grid, and also the industrial processes themselves (similar to the home heating scenario where a thermal model of the home must be learned). Furthermore, in both settings, it will be essential that the householders and business owners are able to understand the consequences of the automated actions that are taken, and are happy to delegate control to an intelligent device or software agent. In this respect, it will be important to define the adjustable autonomy of such systems; to what extent should the agent automatically decide to shift devices to run at certain times, and when should it ask for confirmation from the user. 33

The development of these autonomous technologies raises the prospect that such systems will be widely deployed in possibly millions of homes, each individually reacting to prices and to the preferences of householders. Defining the convergence properties (that is, how the aggregate demand profile will respond to price signals) of such a complex system will be central to the definition of what constitutes safe and efficient behaviors for the grid. In particular, it will be necessary to ensure that neither significant inefficiencies, nor excessive volatility ensue from these autonomous systems converging to poor equilibria (or not converging at all). Hence, it will be important to design simulation systems that can accurately represent both the grid and the reaction of consumers, in order to predict the emergent properties of the system under a range of different conditions (for example, weather patterns or social activities) and worst-case scenarios (some generators fail or lines trip).

Against this background, recent work has begun to research the use of autonomous agents, representing individual consumers, that interact through markets, 10, 40 and individually learn to optimize their use of electrical loads or storage devices in a number bauxite smelters realized there was greater profit to be had in reselling electricity they had bought in long-term forward contracts, than in using it themselves to produce aluminium. 3