MapReduce and YARN API
Apache Hadoop includes Java APIs for different functions on a HDFS file system like creation of a file, renaming, deletion, and to set read-write permissions for directories. This can be done on a single and cluster of systems. In addition, REST (REpresentational State Transfer) APIs is a collection of web services to provide interoperability between a single system and an interconnected distributed network. REST is chosen for its speedy performance, scalability, simplicity, and reliability. YARN REST and MapReduce REST APIs are briefly discussed in this chapter. YARN web service REST API includes URI resources through which the cluster information, nodes, and application information can be accessed. YARN is comprised of Resource manager, node manager, and timeline REST APIs. The application has HTTP request as resource and the response can be in the form XML or JSON. The request URI, response status, header, and body are defined in actual format. Similarly, the REST API is used for MapReduce that comprises the details about the jobs running with the information such as number of tasks, counters, and attempts. Hence, the REST APIs on YARN and resource manager create small modules as a response when a resource is requested. An outline of the research and growth of REST APIs is included in this chapter.