In existed distributed edge extraction method based on MapReduce, the inappropriate dataset split algorithms leaded to the loss problem of image features in result. We presented a distributed computing platform called Split Process Cluster (SPC) to resolve this problem. In SPC, the images are partitioned with the resilient image pyramid model (RIP), a multi-layer and redundant data structure we presented earlier, to ensure the integrity of original image features. And SPC packages the image data to the form of Key-Value pairs, which could be processed through Hadoop, and reduces the results with density-based spatial clustering of applications with noise (DBSCAN) algorithm. Compared to traditional method, the extraction rate of image feature by using SPC has been improved, which indicates that using SPC is an efficient way to improve the extraction rate of distributed edge extraction.
On the feasibility of dynamic rescheduling on the Intel Distributed Computing Platform
