Emotional model for a multi-robot system with emergent behavior

Received Oct 28, 2019 Revised May 23, 2020 Accepted Jun 8, 2020 This article describes an emotional model for a general-purpose robot operating in a multi-robot system with emergent behavior. The model considers four basic emotions: anger, rejection, sadness and joy, plus a neutral emotional state, which affect the behavior of the robot, both individually and collectively. The emotional state of each robot in the system is constructed through the conjunction of a series of factors related to their individual and collective actions, which are: safety, load, acting and interaction, which serve as input to an emotional process that results in an index of satisfaction of the robot that establishes the emotional state in which it is in a certain moment. The emotional state of a robot influences its interactions with the other robots and with the environment, that is, it determines its emergent behavior in the system. This paper presents the design of this model, and establishes some considerations for its implementation.


INTRODUCTION
Emotions are psycho and physiological reactions that occur in individuals and that condition their action in certain situations. Emotions condition the body how it perceives a stimulus of the environment and how it acts in function of it. The emotional spectrum in humans is very broad, and affects positively or negatively their performance in the environment.
Multiple works have been developed in the area of emotions for robots that include model proposals, hardware implementations, and human-robot interactions based on emotions, among others. In [1], emotions are included in a multi-robot system with the object of generating intelligent behaviors between robots that cooperate in the execution of a specific task. In [2] is presented a hardware design for the implementation of emotions in a robotic agent. In [3], the authors propose an emotional model that considers multiple factors that affect emotions: stimuli, cognitive and non-cognitive factors, personality, the subconscious of the agent, among others. [4] describes an emotional model for a pet robot, in order to adapt it to have a natural behavior. In [5] is presented the study of emotions for robots in two domains: the first related to the way in which robots relate to humans, and the second related to the robot's adaptation to the environment and how emotions can improve its performance. In general, the emotions in the robots are represented in discrete states that control the actions of these during their operation. This research proposes an emotional model that considers four basic emotions, which influence the behavior of a general-purpose robot, through the direct involvement in its processes of perception and performance, as well as in its decision making. The model is implemented in an architecture for multi-robot systems with emergent behavior. The emotions influence the interactions that occur between robots and between robots and the environment. Even though they are not able to recognize the emotions of others, their responses to interactions with other robots in the system or with the environment vary according to their emotional state. In this way, the current emotional state in a robot defines its emergent behavior in the system.
The main difference with previous works is that we propose a general emotional model, which is used in order to generate emergent behaviors in the robots of a multi robot system. In our case, the emotions of the robots describe their current functional states, and they are used by the multi robot system in order to define the actions of the robots.
This article is organized in 4 sections. The first one describes the proposed emotional model. The second refers to its implementation. The next section presents the results of the tests carried out. To finalize, the conclusions of the research are presented.

EMOTIONAL MODEL FOR AN ARCHITECTURE FOR MULTI-ROBOT SYSTEMS WITH EMERGENT BEHAVIOR
In [6][7][8][9] is presented an architecture for multi-robot systems with emergent behavior (called AMEB), which is structured in three levels: one individual, one collective and another for the knowledge and learning management (see Figure 1). At the individual level of AMEB, a behavioral module is implemented in [10], which aims to manage the behavior of the robot. This behavioral module requires an emotional model, which considers the set of next emotions ε = {anger, rejection, neutral, sadness, joy}, represented in a one dimensional space proposed in [11][12][13], where the X axis represents the satisfaction or dissatisfaction state of the robot in the interval [-1,1] (see Figure 2), and the emotional spectrum ranges from highly negative emotions like anger, to highly positive emotions such as joy, plus a neutral or non-emotion. Three sub-intervals are defined based on the relationship between emotions and individual behaviors [11][12][13]: an interval related to reactive behaviors, one related to cognitive behaviors, and a third interval related to collective behaviors. The level of satisfaction of the robot is influenced by a series of factors that define the internal state of the robot, and that are related to the operation of the robot and its performance in the environment [10]: -Battery state (BS): is represented by the energy level of the robot's battery at an instant t. where: nc t = level of charge in an instant t, cb = Battery charge capacity.
-Operating state (OS): represents the level of performance of the robot. It is given by the actual time of action (ta) of the robot versus the total system performance time (tt).
-Security state (SS): defined by the number of collisions (c) (caused by other robots or obstacles) and failures (f) versus the total system performance time (tt).
-Interaction state (IS): it is defined by the social or interaction capacity of the robot. It is measured by the number of messages sent (me) or received (mr) versus the total system performance time (tt).
The robot perceives stimuli that influence the state of the mentioned parameters. This activates the emotional process of the robot, which generates a satisfaction index (SI) that defines the current emotion in the robot and the type of behavior associated (see Figure 3). In [11][12][13][14][15] are considered three types of behavior: imitative, cognitive and reactive, which are related to a specific emotional state. In our proposal, the same assumptions are made: negative emotions predispose the individual to problem solving through a process that goes from the local to the collective, while the positive emotions lead to global approaches, ranging from the collective to the individual. Table 1 presents the emotions considered in this model, their classification, and the type of associated behavior.  As seen in the Table 1, highly positive emotions, such as joy, predispose the individual to imitate the behavior of other team members at the moment of deciding, while the slightly positive, such as sadness, lead to a process where the cognitive aspect predominates over the imitative aspect. Also, slightly negative emotions lead the individual to a process of internal reflection for decision making, and finally, a highly negative emotion as anger, leads to behaviors that are clearly reactive, where the individual seeks to achieve their survival by reacting to the stimuli of the environment without previous reasoning. The emotions affect the parameters of the robot; for example, if the robot is sad, then its ability of feeling will be affected; or if the robot is happy, then its speed of movement will be greater, which will affect its perform at a given time.
The types of behavior are defined below. a. Imitative: behaviors where an individual seeks to behave in a similar way to another, for example: persecution. b. Cognitive: behaviors where there is a reasoning process in order to decision-make, by the individual, example: shelter search. c. Reactive: behaviors where the individual seeks to survive, for example: obstacle avoidance.
The inclusion of emotions in the multi-robot system seeks to improve its adaptability to the dynamics of the environment, as well as facilitate the emergence in the system, by modifying the way of performing the behaviors that each individual is able to execute. The conjunction of individual behaviors defines the overall behavior of the system, which by its nature cannot be predicted a priori [16].

IMPLEMENTATION OF THE MODEL
The model is implemented in the AMEB behavioral module described in [10], which is structured in four layers (see Figure 4), in which the processes involved in the emotional model are organized: a) reactive layer: manages the reactive behaviors of the robot, which are generated by a stimulus -reaction process; b) cognitive layer: deliberative behaviors are managed in the robot, based on their local knowledge; c) social layer: it exploits the collective knowledge in the processes of decision making of the robot, basically, it manages the way that the robot interacts with other robots in the system, as well as its associated behaviors; d) affective layer: is responsible for managing the emotional process of the robot. The phases involved in the emotional model are described below, based on the phases described in [11]. a. Classifier In the proposed model, we define four basic emotions and a neutral or non-emotional state, considering positive and negative emotions, which lead to individual or collective behaviors. In this phase the types of emotion that the robot can activate at a certain time are defined. b. Behavior model The reactive, cognitive and social layers implement a set of behaviors of the robot in the environment [10]. These behaviors are constructed by the conjunction of two or more basic robot skills, which are closely related to their hardware and software capabilities, some of these skills are: -Move: The robot moves forward or backward, it can be for a certain time or a certain distance. From these skills are built behaviors of greater complexity, which define the way the robot acts, among which are: -Explore: Through this behavior the robot is able to explore the environment at random. -Obstacle avoidance: allows the robot to evade obstacles, which can be fixed or moving objects.  These behaviors are associated, as mentioned above, to a specific emotional state, for which the following rules are defined [10]: Rule 1: If <emotional state> is highly positive. then <imitative_behavior> Rule 2: If <emotional state> is positive, then <imitative_behavior> Rule 3: If <emotional state> is slightly positive, then <cognitive_behavior_priority> Rule 4: If <emotional state> is neutral, then <behavioral> Rule 5: If <emotional state> is slightly negative, then <cognitive_behavior_priority> Rule 6: If <emotional state> is negative, then <cognitive-behavior_priority> Rule 7: If <emotional state> is highly negative, then <reactive_priority> a. Emotional configurator Determining the emotion of the robot at an instant t is a fundamental stage of the emotional process. It depends on the level of satisfaction of the robot, which is given by the factors described in the previous section. Below, it is described this process. In this phase, the satisfaction index is calculated using a fuzzy system composed of four input variables and one output variable. The input variables, as has been explained in the previous section, represent the internal state of the robot, and are related to operating parameters. The output variable is related to the satisfaction status of the robot, and is associated with an emotion: -Input variables: BS, OS, SS, IS. -Output variable: SI.
The input variables have three fuzzy sets: {low, normal, high} and the output variable has five fuzzy sets: {anger, reject, neutral, sadness, joy}. Some of the rules that govern the system are: Rule 1: If BS is low and OS is low and SS is high and IS is low, then SI is sadness. Rule 2: If BS is low and OS is low and SS is normal and IS is low, then SI is sadness. Rule 3: If BS is low and OS is normal and SS is normal and IS is low, then SI is sadness. Rule 4: If BS is low and OS is normal and SS is normal and IS is normal, then SI is rejected. Figure 5 shows an example of the implementation of the fuzzy model in MATLAB®. For the case of the Figure 5, for the set of input {BS: high,OS: normal, SS: low, IS: high} was obtained the next output: {joy}. This, according to our model describes the following: -Battery Charging: optimal. -Operation state: normal, the robot has been executing some task during a good part of the operating period of the global system. -Safety factor: Low, indicating that there are few or no collisions and/or blockages. -Interaction factor: high, the robot has interacted with the other team members. The robot at that moment is satisfied and the active emotion is joy. Figure 6 shows the relationship between the battery state (BS) and state of operation (OS) variables with the other system variables, and how they influence the output variable (SI). It can be observed, for In this phase is modified the current behavior of the robot, as a consequence of the previous phase. This modification involves the association of an emotion to a behavior, as is shown in Table 1, according to the rules established in phase 2. The robot's performance is affected by the current emotion in two specific aspects.
-The selected behavior (imitative, cognitive or reactive) of the set of behaviors of the robot.
-The performance parameters that are modified by the satisfaction index of the robot, which influence the way in that the behavior is executed. The performance parameters that can be modified are related to the characteristics of the robots to be managed by the architecture [14]: -velocity of displacement range of detection of objects in front range of detection of marks on the ground decision-making capacity (time to decide). What implies that for the same behavior, different results will be obtained. For example: if the robot is in a chase and its current emotion is joy, then it will move faster and its sensory abilities will work better that if the robot has as current emotion the sadness.

.1. Scenarios
Different tests were carried out, in order to verify the emotional model through the variation of the parameters in the implemented fuzzy system. In these tests, the aim is to verify the emotional model in the robots, not focusing on the overall behavior of the system. The interactions between the robots and of them with the environment are simulated, as well as the factors that alter their internal state. The experimental scenarios are the following: -Determination of the current emotional state according to the current operational state of the robots -Analysis of the emotional behavior of a robot according to the events in the environment. a. First scenario We present the result of the test for the first scenario, where randomly were generated values for the parameters that define the state of the robot, and verified that the active emotion was the expected. Table 2 summarizes the results obtained in the first scenario, in which the emotional state of a robot was verified, according to the factors that influence it. The BS, OS, SS and IS columns present the values of the factors that define the internal state of the robot. As mentioned above, they have three fuzzy sets {low, medium, high}. The SI column presents the value of the satisfaction index, which has five fuzzy sets {anger, reject, neutral, sadness, joy}, such that its result is related to an emotional state. For example, a robot model whose internal state is {BS: high, OS: high, SS: low, IS: high} has a high SI, so it is happy at that moment. The fuzzy system establishes the expected emotions, according to the input variables about the states of the robot in a given a time t. The value of the index of satisfaction in each case shows how the level of intensity of the same emotion can vary. This could be used to affect the behavior of the individual at the time of performing a behavior, in other words, be more precise. Thus, the robots could present different levels of joy, and according to it, act different. Figure 7 shows different values for the same emotion in different robots in the first scenario. Below is presented the results of the second scenario, where the dynamics of a robot's emotional state during its operation are analyzed, for which the events that cause changes in its state are monitored: collisions/failures (C/F), the effective time of action (OT), interactions (I) and battery charge (BC). The robot interacts with the environment for a period of time t, and the samplings are carried out every 10 units of time. The simulation results are presented below, for which the first 100 samples carried out during time t are taken into consideration. Figure 8 shows the first results for the first group of samples, in a context where there is a set of negative factors at this time in the environment, such as collisions, isolation of the robots, etc. Figure 8 shows that, for t = 0, the robot is in a neutral state. During the beginning of its performance, the robot presents an abrupt tendency towards a highly negative emotion, this is motivated by the fact that the received stimuli were negative (high number of collisions, few interactions, etc.). During its operation, it begins to change its state to a highly positive emotional state, then the emotional state begins to decline, and it stabilizes in a slightly negative state. This is due to the occurrence of factors over time that negatively influence the state of the robot.  the discharge rate was similar, in addition to a high number of collisions (C). This should cause that the emotional state should be highly negative emotion. With the passage of time, the social capacity of the robot should increase steadily, the collisions should increase slowly stabilizing, the discharge rate should low, and also the number of stops. These events should influence that the robot maintains a positive emotional state at the end (high interactions, safe operation and stable battery charge (BC)). Figure 9. Events that affect the status of the robot during the second test of the second scenario Figure 10. Results for the second test for the second scenario.
In Figure 10, it is observed that at time zero events occur that lead the robot to a highly negative emotional state, and then, a process begins that increases its state of satisfaction to stabilize in a state highly positive. This is explained by the events that occur that affect the state of the robot (see Figure 9), which corroborates that our emotional model considers the events in the context to determine the emotional state of the robot.
The previous tests show the emotional process that occurs in a robot during a period of time, and how the events that occur in the environment define its emotional state. According to the current emotional state, the behavior of the robot can be different (see section 3). If the robot has a positive emotional state, then it will have an imitative behavior, as, for example, to explore; if the robot has a highly negative emotion, then it will have a reactive behavior, for example, to avoid obstacles, and so for the rest of the emotional states. In this way is generated an emerging behavior in a robot, based on its current emotional state, which can change during its operation.

Comparison with previous works
In this section, a qualitative comparison of the proposed model with previous works is carried out, for which the following is taken into consideration (see Table 3): technique used, features that define the emotion, and the application of the emotional model. In [17], a system is implemented to identify the emotional state of a human being. For this, the signal coming from an electrocardiogram (ECG) is analyzed and a series of characteristics are extracted, which are processed in a hybrid system based on a back propagation neural network and a crossbar array. The set of identified emotions by the system are: {angry, fear, grief, happy, and calm}. Our work uses a fuzzy system with the goal of identifying the emotional state of a robot according to 4 factors that define its internal state. The identified emotion is not shown to the environment, but it is used individually to regulate the performance of the robot in the system. In both cases, the signal of the ECG and our 4 factors considered, are affected by the interaction of the individual/robot with the environment.
In [18], a neural network is used to identify emotions based on a two-dimensional model that encompasses a broad emotional spectrum, supported by the levels of pleasant-miserable, and arousal-sleepy of the individual, the system is designed for humans. In our model, a simplified one-dimensional spectrum is used, based on the level of satisfaction and dissatisfaction of the individual/robot.
The work presented in [19] implements in a mobile robot an emotional model that involves three aspects: emotion (taken as a stimulus), feeling (considered as a short-term state) and mood (long-term state). They regulate, as in our work, the behavior of the robot, affecting its ability to perceive and act. The robot interacts with the environment, the information received is processed by a neural network, and the result is the emotion that the robot will express during its interaction with the human beings. In our case, we use a simplified emotional spectrum {happy, sad, fearful, angry}, due to the robot's processing capabilities.
In [20], a fuzzy system is used for the recognition of human emotions from signals generated by an electroencephalogram (EEG), in order to be applied it in human-robot interaction systems. The characteristics used as inputs to the system are extracted of the analysis of the signal. In [21] is described a system for the recognition of emotions, also for the processes that occur in the human-robot interaction. The system allows identifying the next set of emotions {sadness, angry, surprise, disgust, fear, neutral, happiness} in a human, from his facial gestures. In this model is present, as in our approach, the neutral state.
According to the Table 3, only our approach uses the emotions to regulate the behavior of the robots. The other approaches use the emotions for the human-computer interactions, they propose emotion recognition mechanisms for the human beings in order to guide the interaction of the robots with them. Our approach proposes an emotional model for the robots that determines their interactions and their influences on the environment, which are fundamental elements to generate an emergent behavior in a system [12,16]. Our model is very robust because defines the emotions of a robot based on its internal states, which characterize its real capabilities to interact in a given moment.

CONCLUSION
We have proposed an emotional model for an architecture of multi-robot systems with emergent behavior. The model simplifies the emotional spectrum based on four basic emotions: joy, sadness, rejection and anger. The model is implemented using a fuzzy model and is used for the generation of emergent behavior. Previous works have focused on the recognition of emotions, especially applied to human-robot interaction. In this work, the approach is oriented towards the ability to identify emotion as an individual state, which in principle affects the behavior of the robot. This influences the global behavior of the system, but not because individuals are able to recognize the emotion in the other, but because of the individual behavior that is generated in itself, which influences the global state. There is a robot-robot interaction explicitly directed by the current emotion in each robot.
The inclusion of emotions in the robot allows improving its adaptability to the dynamics of the environment, and facilitates the emergence and self-organization in a multi-robot system. Emotions influence the way that the robot acts according to the stimuli received, and can regulate its behavior over time. For example, if negative stimuli prevail during its performance, then the robot will take a certain emotional state for a certain time that will cause that it adapts itself to the environment. In this way, it will act according to its emotional state. For example, if the robot begins to collide constantly, then the robot will become cautious because its process of decision making will be cognitive due to its emotional state is sadness.
The expansion of the emotional spectrum and its implementation in robots with greater capacities, including the ability to express their emotional state, to allow other robots to recognize their state, are proposed as future works. Also, the analysis of the effects of the emotional state of the robots in the emergent behaviors in the system, must be carried out. The emotional states of the robots influence their interactions and with the environment, which should generate emergent behaviors in the system.