Application of Hybrid Agents to Smart Energy Management of a Prosumer Node

We outline a solution to the problem of intelligent control of energy consumption of a smart building system by a prosumer planning agent that acts on the base of the knowledge of the system state and of a prediction of future states. Predictions are obtained by using a synthetic model of the system as obtained with a machine learning approach. We present case studies simulations implementing different instantiations of agents that control an air conditioner according to temperature set points dynamically chosen by the user. The agents are able of energy saving while trying to keep indoor temperature within a given comfort interval.


I. INTRODUCTION
nergy management of modern smart buildings requires distributed intelligent control in order to save energy [21]. A smart building with energy management capability is a node which is part of a smart grid of energy; such kind of energy node can be conceived of as a prosumer, i.e. producer and consumer of energy in different forms. An energy efficient building already has its passive intrinsic energy efficiency which is embedded by design in its structure and materials, however further improvements in energy efficiency can be achieved by adopting a control methodology inspired to intelligent agents. An intelligent control should be intrinsically dynamic taking into account real-time requirements in response to the building dynamical thermo-physical behavior, considering that it is immersed in a dynamical environment where weather events can change its energy footprint. In other words a proper control that aims at achieving energetic optimality must rely on a detailed model of the thermal behavior of the building areas in the range of observable values of environmental variables. Such a dynamical model, more complex than the one used during the design of the building, is seldom available and its analytic construction can be highly costly and deemed to be imprecise due to physical complexity of building system and its environment; on the other hand, current techniques in machine learning and model synthesis, when properly fed with sample data, allow an inexpensive run-time prediction of the system behavior much like predictions that an analytical model could give, that can be valuable in planning a heuristical control of the system aimed at specific optimization goals.
In this work we outline a possible implementation of the prosumer agent architecture that is able of a predictive dynamic control of a smart building that, when applied to an air conditioner controller, achieves a better energy efficiency over the usual thermostatic control as implemented in commercial HVACs. The agent is equipped with an intelligent control scheme based on a learning phase that constructs a synthetic model by sampling the system in different environmental and controlled situations. The synthetic model could be used for local forecasting of the system state in order to plan an efficient control.
Our agents architecture includes a perception layer -that incorporates the prosumer forecaster-and a reasoning layerthe prosumer planner. The perception layer has the job of transforming the sensors array signals into abstract predicates, i.e. fluents, that can be handled by the reasoning layer.
The application that we consider falls into the realms of Distributed Intelligent Control Systems that can be described in terms of Complex Event Processing (CEP) [3], where the need for some degree of autonomy is crucial in order to enable components to dynamically respond to ever-changing circumstances while trying to achieve overarching objectives, and properly handle many events. In fact, each definite area of the building -e.g., the dining room, the bathroom, the hall, etc.can be managed by a hybrid agent which plays the role of the prosumer, in communication with its siblings of the other areas, in order to accomplish the performance goals. Therefore, the energy management of the building can be represented in terms of a Multi-Agent Systems (MAS), where each agent deals with the others in terms of planning and forecasting, as in [2].
The trends extracted from load predictions, energy consumption, and system variables made by the forecaster feed the symbolic interpretation needed for a rule-based logical agent, described in the DALI language. From the planner point of view, this abstraction becomes a fluent in an event calculus context, i.e. events to be handled proactively by the DALI MAS. The latter should also take into account external requirements and user preferences, in order to achieve the performance goals, which are defined a priori by the user management.

II. THE PROSUMER NODE
In Fig. 1 an average daily power consumption profile of a prosumer node is shown, where energy load can be obtained by integration. There are times of the day when energy is in surplus, and times were it is in deficiency. An ideal energy manager should level its energy with the given goals, limiting peaks and avoiding surplus periods, and founding its decision  on an accurate dynamical behavioral model of the underlying building infrastructure in its environment. A smart building connected to a smart energy grid can be considered at the same time a consumer and a producer of energy. So, a energy prosumer node, as defined by [11], should have local energy sources which are independent from the grid. Usually, these sources include sustainable energy typese.g. solar thermal and PV, wind, geothermic, etc.but also conventional fossil fuel based sources, as for instance continuity groups and diesel generators that need to be managed appropriately. The energy prosumer definition should also take into account the local consumption profile and the locally available energy metering tools. An intelligent energy manager should also be present in order to improve the energy efficiency of the prosumer node.
The intelligent prosumer manager should derive its behavior from the capability to dynamically predict the energy consumption and production given the intrinsic time constant of the building itselfe.g. 15 minutes as a typical thermal inertia of a modern energy efficient building.
In Fig. 2, a general block diagram of a prosumer node is shown. The building energy status is given at each time instant by: external weather conditions, internal user behavior, internal signals and settings in terms of comfort, i.e. internal temperature and reference point. Also the internal sensors set should include what a modern smart building can be equipped with, such as energy load sensors at the power plugs of electrical equipments or at the electrical cabinet for each floor. In this way, a complete energy consumption profile of the building can be obtained at each sampling time. The prosumer controller will then implement a first local control loop that should keep at equilibrium the goals given by the prosumer planner, having an impact on the energy flows by means of the actuators, i.e. heaters, water chillers, air conditioners, windows controls, etc., together with other controllable energy generators or accumulators. The external sensors array should return the state of the most important environmental signals with an impact to the energy profile of the building, i.e. external temperature, solar radiation, humidity, etc.. The resolution and the distribution of the external sensors at which such signals should be sampled depends on the structural configuration of the building. Diffused solar radiation has a major impact on the energy demand of the prosumer node, also can depend on neighborhood buildings for the shading account.
The prosumer planner gets the whole set of signals, internal and external, as well as possible weather models of the area, so as to feed the prosumer forecaster. Then, the prosumer forecaster dynamically produces an estimate of the near future energy requirements (i.e. at the next time sample) of the prosumer node, as well as possibly other environment and system variables, relevant for plan generation and for goal adherence. It can also give the best estimates of internal climate signals, i.e. temperatures, so to that the prosumer planner can reason about comfort evolution and take decisions about the best action to perform.

III. THE ENERGY MANAGEMENT PROBLEM
The energy management problem at a hand could be summarized as in the following. Planning goals could be targeted to possibly conflicting functions such as to minimise energy, cost and CO2 emission, and to keep comfort within a specific interval. Possible conflict between goals might arise due to the different type of resources available like grid, photovoltaic panels, wind rotors or fossil fuel.
Let us define the global goal set such as:

G = {EnergySaving, CostReduction, EmissionReduction, HighestComfort }
These goals are selected by the human energy manager who defines the current general energy policy of the building. Let us consider a prosumer node with a wide range of possible energy actions A, each with a different energy/cost/emission profile, as, for instance:

A={Ep , Ec , Ea , Eb , Es }
where subscripts stand for {produce, consume, accumulate, buy, sell} energy. The planner agent might choose an energy generation sequence among feasible ones, as constrained by the given global goals, user preferences, and other local requirements.
Thus, given the input data set I = Ie + Ii as the union of the set of external sensors/variables Ie array and the set of internal sensors/variables array Ii , the prosumer forecaster F can extract relevant variables values from the model f constructed by the machine learning module, behaving as a function F=f(Ie , Ii). The prosumer planner would then generate a plan P as a sequence of possible actions taken from the predefined set A of possible energy related actions, given the prosumer forecaster estimates: P=p(F)=a1(q1)...an(qn), with ai∈ A and qi∈ Real represents the quantity relative to the action ai. The building energy management goals are processed by the he overall planner, whose results are broadcast, via a FIPA compliant middle layer, to every active prosumer node (supervised by a local agent) managed by the MAS. The agents are in charge of adapting the planner results to the specific situation they are responsible for. As an example, we can consider the case when the global goal at a given time is EnergySaving, and the state goals for the prosumer controller, as example the air conditioner, is the set point at which the thermal zone temperature should be, within the user comfort interval. A domotic7 technology could embed a planner agent which send control signals to the local controller of a specific area that would co-impose the goal to its local control loop, that is the temperature set point.
The DALI [6] knowledge base with preferences of a prosumer planner agent can be expressed as following: This very concise planner agent knowledge base is interpreted as follows. The local planner, relative to a single thermal zone, has three possible actions: increase the temperature set point of the air conditioner, decrease it or standby (hold). There are two preferences, expressed by the rules with connective ' > '. Each rule states that the leftmost action is preferred. The preference rule applies whenever its body (the part after the ':-') holds, and preferences apply to feasible actions. An action is feasible if its preconditions (if 7 home automation any) are verified. Preconditions are expressed by rules with connective ':<'. A basic precondition for an action to be feasible is that the action has been enabled by the overall planner (e.g., the planner might not enable temperature decrease if too much energy is being consumed at the moment). If the perception layer, in other words the prosumer forecaster agent, detects that the external environmental signals of temperature and solar irradiance shows the same variation trend, then the planner agent would prefer to increase the temperature set point so to temporally save energy, waiting for a better thermal equilibrium between the building and its environment.
Also, in order to further save energy, it would prefer not doing anything rather than reducing the set point, i.e., decreasing the room temperature, unless user comfort requirements are no more satisfied. In this latter case, being this a precondition to all action, it would definitively reduce the temperature set point until the system is in equilibrium and user comfort requirement is met. An other important precondition is the plannerHasEnabled predicate. It allows to configure a hierarchy of planner agents: a global planner agent associated to the whole building and many other sub-agents associated to each thermal zone, or rooms. In this way the knowledge base of the global planner can enable or disable the possible actions of the local planners, depending on global energy reasoning goals, like energy availability or special energy needs.
The above rule is re-evaluated periodically, at a certain (customizable) frequency. This mechanism (which is a generalization of the DALI internal event construct) makes the agent proactive, i.e., capable of autonomously operating on its environment. The frequency will be customized according to the kind of appliances that are being controlled and to the granularity of results that one needs to obtain about temperature and energy consumption.
This kind of multi-agent system based control allows the temperature to be kept within user preferences, overcoming the control scheme of a simple thermostat, which is based only on a kind of "infinite energy supply" hypothesis in order to keep the temperature constant, a policy that is no longer feasible in modern buildings.

IV. CASE STUDY
In a preliminary work [23] we argue that the prosumer forecaster could be implemented with any suitable machine learning approach, where we proposed and experimented using a neural net to predict energy consumption that showed good performance, given the relative smoothness of the response of the system to changes.
In this work we present an experimental simulation of a single instance of a prosumer node as depicted in Fig. 3. In this experiment no real planning is involved yet. Here plans are single actions to achieve a single given goal, so we could collapse the prosumer planner and the prosumer node since the agent acts as a simpler consumption forecaster agent.
As shown in [24] a way to extract information from a large set of measured data is the so-called knowledge discovery in databases (KDD). In this work we applied the KDD method to the thermal measurements taken in the laboratory so that the prosumer agent could embeds a model of the environment, room, and HVAC8 system. Such model was constructed using the Eureqa symbolic regression program [18,19] fed with real data sampled in the Roio's HVAC thermal laboratory, as later explained in the experimental setup section. The derived analytical model is capable to give a good estimate of the internal temperature and HVAC electrical consumption, given the external weather condition, as shown in Fig.5.  Target variables that constitute the forecaster model are energy consumption in function of all the remaining system variables and indoor temperature in function of the remaining system variables, i.e. temperatures sensors across the room. Results after running Eureqa on real data are depicted in Fig.  4. Fig. 5 depicts the data fitting of the Eureqa's model data.
The models that we obtained were used both for running simulation of different controlling agents and for predicting variables values in near future as needed by some agents controlling strategies. We run two simulations with four different agents. The first set of four simulates the behavior of simple heuristic strategies. The second set simulate the same agents when their strategies are improved with some predictive ability. 8 heating, ventilation and air conditioning

A. Non-predictive control
With the aim of validating the synthetic models, and of showing that better energy managements than usual thermostatic control are possible, we used the model for simulating the behavior of the system when controlled by four simple controlling agents using different strategies with which they would act in setting the set point of the air conditioner. The agents behaviors we analyzed are: A) The picky agent: It acts by keeping the thermostatic set point of the air conditioner as set by the user's dynamical choice. This agent is picky and strict -and in fact implemented as null-as it relies only on the air conditioner hardware and the user's setting  [18,19], in function of external weather conditions and user set point.
B) The cheap agent: It acts by keeping the set point at the extreme of the comfort interval, in this case defined as the set point chosen by the user + 3° Celsius. This agent is cheap as it tries to save energy by keeping temperature at the highest possible point within the defined comfort interval as chosen by the user.
C) The parsimonious agent: It acts by increasing the set point to the highest possible within the comfort interval, behaving as the cheap agent, if both outside temperature and solar radiation are increasing, or decreasing, among the out of thermal equilibrium condition principle. It is parsimonious as it accepts to relax the strict adherence to the set point chosen by the user when it expects a heat wave that would be expensive to combat or when is coming a natural cooling period. On the other hands, when trends have different sign, the set point correction is zero.
D) The heuristic agent: It acts according to a rule that has been heuristically constructed after an human guided analysis of the behavior of the system trends, observing real data and simulated scenarios. It's a variation over the parsimonious agent behavior that can be summarized as follows: given the trends of both external temperature and solar irradiance by means of filtered discrete derivatives, their respective amplitudes are summed and the sign of the result Rt is considered. If Rt is positive the comfort is relaxed. Rt can be positive either if both temperature and solar are increasing, either if one of them has a prevalent increasing trend.  In Fig. 6 we show the results of the four simulations and in Table 1 we report the total energy consumption and the estimated discomfort in the simulated time period. Discomfort is measured both as number of samples outside the comfort interval and as integral of inside temperature distance from the set point chosen by the user. As we see, the heuristic agents C and D achieve an energy consumption in between the range from the maximum consumption of the strict agent A and the minimum of the cheap agent B, with a comfort in the range of the maximum achieved by A and the minimum achieved by B.

B. Predictive control
The second experimentation was obtained by providing all agents with a rule that would decrease the set point of the air conditioner if they would predict an indoor temperature beyond the comfort interval. For implementing this rule the agents use the constructed model in two ways. First they use it to predict energy consumption in the following time period given other variables, and in particular actual controlled set point. External environment variables are predicted with different models: temperature Text is derived from the daily temperature distribution function from ASHRAE (American Society of Heating, Refrigerating and Air-Conditioning Engineers) manual [22], in function of the moving estimate of maximum and minimum temperature of the day. Solar irradiance S has been with Eureqa symbolic regression program fitting it against time of the year, with samples taken over the previous 2 weeks, so the get the best approximation of future solar irradiance taking into account the local shading near the thermal laboratory at Roio's hill.
Text and S are again used to predict indoor temperature in the following time period together with the predicted energy consumption, measured in kWatts over the sampling period.  Fig.7 we show the results of the simulations and in Table  2 we report the total energy consumption and the estimated discomfort in the simulated time period. Also, so as to get an objective prove of the more efficient functioning of the predictive agents we computed the total comfort improvement over their non-predictive version. As we see, predictive ability does not improve performance of the picky and the cheap agent but it does improve performance of the parsimonious and the heuristic agent. In fact they both achieve about the same discomfort time as the picky agent with a sensible energy saving. Moreover both the predictive versions of the parsimonious and heuristic agent improve total comfort over their non-predictive versions, with a relative smaller increase in energy consumption.

V. EXPERIMENTAL SETUP
The thermal laboratory is located at Roio's Monteluco hill, coordinates 42.33 N 13.37 E, 1100 from sea level. The room is 8x8x3 mt, with a small window ( 1msq ) on the east wall and a wooden door on the south wall. The HVAC system is made by a Mitzubishi MSZ-FD25VA inverter thermal pump and an external chiller MUZ-FD25VA. Temperatures are measured by means of 3 dry bulb PT100 thermo-resistors sensors, at different distance from the air conditioner, connected to a Delta-T logger DL2e.
The thermal control is supervised with a domotic network. The HVAC consumption is measured with toroidal sensors over power lines. The set point is set to the HVAC by means of a IR-remote emulator. The sampling frequency is set at 30 minutes, during which temperatures are averaged and instantaneous HVAC power level is accumulated. There is also an external weather station Geoves Micro 3 that measures solar irradiance, ambient temperatures, wind direction/amplitude and humidity.

A. Active Logic Programming
In the field of Active Logic Programming, we have chosen to use DALI [4,6] for its unique features in respect of the other development frameworks that are available. The DALI semantics is fully defined in [4], while the operational semantics of the interpreter is described in [8]. A DALI agent is a logic program that contains a particular kind of rules, reactive rules, aimed at interacting with an external environment. The environment is perceived in the form of external events, that can be exogenous events, observations, or messages by other agents. In response, the agent can perform actions, send messages, adopt goals, etc. The reactive and proactive behavior of the agent is triggered by several kinds of events: external events, internal, present and past events. When an event arrives to the agent from its "external world", the agent can perceive it and decide to react. However, when an agent perceives an event from the "external world", it doesn't necessarily react to it immediately: it has the possibility of reasoning about the event, before (or instead of) triggering a reaction. Furthermore, internal events make a DALI agent proactive.
In particular, [5] proposes an extension so to manage complex reaction in rule-based logical agents using a DALIlike syntax which associates a set of preference rules, that can express the performance goals directly into the language.
The literature in MAS is wide and large; see [14] for a survey. However, there are some points in contact with different solutions, both at the domain and at a design and development levels.
3APL [9,10] is a programming language and a platform specifically tailored to implement MAS, where the agents are designed in terms of "data structure" or "mental attitudes" (beliefs, goals, plans and reasoning rules) and "deliberation process"implemented as programming instructions. Based on its beliefs, an agent can reach its goals by planning its actions thanks to the deliberation cycle, that can lead also to the check and revision of its mental attitudes. Although 3APL is based on the concepts of rules and planning, it lacks to implement the idea of event, unlike for instance DALI. 3APL has been used mostly to realize application for virtual training (e.g. [17]).
A similar approach is known as KGP (Knowledge, Goals and Plans), which is based on logic programming with priorities, taking beliefs, desires and intentions as a starting point, but adding reasoning capabilities, state transitions and controlsee [15]. Furthermore, it is implemented directly in computational logic. KGP could be a valid alternative candidate compared to DALI.
METATEM (and its extension Current MERATEM) [1] is a language based on first-order linear temporal logic, and thus it is suitable for temporal planning and temporal knowledge representation. At the basis it has concepts such as beliefs, intentions, goals and plans, but the very fundamental rules are of the form "past and present formula implies present and future formula" [16]. So, like DALI, at the center of METATEM there are the concept of time and the idea that the past determines the present and the future, but it has no different classes of events and METATEM agents are just reactive agents, unlike DALI. METATEM can have a wide range of applications (e.g. patient monitoring, fault tolerance system, process control etc.) [13,12], but as far as the authors know, it has not been used for the energy management of a building.

B. Energy Saving and Air Conditioning
From a classical control point of view, the work in [20] developed cost efficient control strategies to achieve optimal energy and acceptable comfort conditions. The importance of agents based intelligent control systems for indoor building environments, coordinated through MAS, in the context of an energy efficient building has been reviewed in [21], were an evolutionary algorithm shapes the fuzzy knowledge base of a multilevel hierarchical MAS, to take the overall user comfort, including thermal, visual and air ventilation, under control.

VII. CONCLUSION
The experimental simulations demonstrate feasibility and the effectiveness of predictive agent based control towards providing a range of different behaviors to adhere to a given goal. This preliminary results calls for embedding predictive capabilities in planners as outlined Stefania Costantini is full professor of computer science at the DISIM Università dell'Aquila, Italy. She graduated in computer science at the Università di Pisa in 1983. Her research focus is on computational logic, logic programming and artificial intelligence, artificial agents, non-monotonic reasoning. Her contributions extend from meta-logic programming, reflective prolog language to answer set programming (ASP) with resources and preferences and logical agents, with the DALI language. She leads the intelligent agents and artificial intelligence research group at University of L'Aquila.
Giovanni De Gasperis Ph.D. is researcher and assistant professor at the DISIM Università dell'Aquila, Italy. He graduated in electronics engineering at the Università degli Studi dell'Aquila in 1991 where also earned a doctorate in electronics engineering in 1995 with a thesis about microfabrication of dielectrophoresis traps. He was a PostDoctoral Fellow at MD Anderson Cancer Center, Houston, TX, between 1995 and 1998 doing research on computer vision applied to automation of electro-rotation measurements. His research focus is on the application of artificial intelligence and machine learning to the smart energy management problem and autonomous systems. He is an active developer of the DALI intelligent agents framework, built on Prolog.

Mario Gimenez De Lorenzo graduated in Electrical
Engineering and is currently doing a PhD on Electrical Engineering at the DIIIE Università dell'Aquila. His research activity includes application of home automation into design of energy efficient buildings and electrical plants. Since young, he is an Information Technologies (IT) enthusiast so as to have earned several experiences as independent consultant, and teacher of introductory IT courses.