Running MapReduce jobs on AWS-EMR from Eclipse

2019-05-13 18:22发布

I have the WordCount MapReduce example in Eclipse. I exported it to Jar, and copied it to S3. I then ran it on AWS-EMR. Successfully.

Then, I read this article - It shows how to use AWS-EMR Api to run MapReduce jobs. It still assumes your MapReduce code is packaged in a Jar.

I would like to know if there is a way to run MapReduce code from Eclipse directly on AWS-EMR, without having to export it to a Jar.

2楼-- · 2019-05-13 18:44

I haven't found a way to do this (for mapreduce jobs written in Java). I believe there is no Eclipse utility or plugin that does this. You could maybe write a script of some kind and incorporate it in your build process to push out the Jar to your EMR cluster. But there is no single utility/tool that will compile, bundle, deploy and execute your mapreduce job to EMR.

登录 后发表回答