The service-oriented computing paradigm is based on the assumption that existing services can be put together in order to obtain new composite services. This chapter focuses on how peer-to-peer architectures based on multi-agent systems can be used to build highly dynamic and reconfigurable infrastructure that support dynamic composition of grid services. The chapter starts by providing an overview of key technologies for SOC. It then introduces dynamic service composition and challenges of composing grid services. The authors further motivate for Multi-agent system approach in SOC and why it becomes important in service composition. They then present our research effort, AIDSEC, an agent-based infrastructure for dynamic service composition, describing its architecture, implementation and comparison with some related work in the literature. In addition, the chapter raises some emerging trends in SOC and the particular challenges they pose to service composition. They conclude by suggesting that a solution based on multi-agent system is required for composing services that possess capabilities of autonomy, reliability, flexibility, and robustness.