This chapter proposes a multi-agent organization model for self-adaptive software to examine the autonomous components and their self-adaptation that can be occurred at either the fine-grain behavior layer of a software agent or the coarse-grain organization layer of the roles that the agent plays. The authors design two-layer self-adaptation mechanisms and combine them with reinforcement learning together to tackle the uncertainty issues of self-adaptation, which enables software agents to make decisions on self-adaptation by learning at run-time to deal with various unanticipated changes. The reinforcement learning algorithms supporting fine-grain and coarse-grain adaptation mechanisms are designed. In order to support the development of self-adaptive software, the software architecture for individual agents, the development process and the software framework are proposed. A sample is developed in detail to illustrate our method and experiments are conducted to evaluate the effectiveness and efficiency of the proposed approach.
