Returning Candidate?

Big Data Engineer II

Big Data Engineer II

Job ID 
US-CA-Palo Alto
Posted Date 
Company Services, Inc.
Position Category 
Software Development
Recruiting Team 

Job Description

Amazon EMR ( is an AWS service that makes it easy for customers to run their big data workloads. EMR supports well-known big data platforms like Hadoop and Spark, and multiple applications that are part of this ecosystem, like Hive, Presto, Pig and Hue.

The EMR team is looking for experienced, passionate and talented engineers to innovate in the rapidly growing area of big data and integrate the latest in big data technology applications into a massively scaled distributed service.

Utilize your domain expertise to tackles the complexities of build, integration and innovation while providing a reliable, quality stream of releases for today and emerging applications (Spark, Tez, Presto, Flink, etc.) in the big data space.

· Are passionate about distributed processing and massive scaled applications
· Enjoy using and participating in development of open source applications
· Enjoy working with the latest software technologies and paradigms
· Are eager to be an early adopter of emerging big data applications
· Savor the challenges of making diverse technologies work through the entire stack
· Are comfortable with continuous integration workflows and tools like git, jenkins and gradle
· Are comfortable with Java and digging into any programming language in order to understand the internals of an application
· Are at ease with Linux troubleshooting and bash scripting and can do wonders with a single command
· Are comfortable with OS-specific packaging such as rpm/dpkg

Basic Qualifications

· BS Computer Science or other relevant technical degree and/or related experience.
· Experience with Java or Scala build systems: maven, ant, sbt and gradle.
· Java or similar language development experience.
· Strong understanding of distributed systems and distributed computation.
· Deployment automation experience with scripting, chef, puppet, etc.
· Linux and RPM packaging experience.
· Experience with Hadoop, Spark or related projects
· Experience with virtualized environments and cloud services such as AWS
· Ability to communicate comfortably, at different levels, with different stakeholders
· Excellent communication and collaboration skills
· Worked on applications or web services deployed in at scale

Preferred Qualifications

· Experience with Apache Hadoop ecosystem applications: Hadoop, Hive, Oozie, Presto, Hue, Spark, Zeppelin and more!
· Strong Java development experience.
· Experience building and operating at scale on AWS
· Commits or contribution via code or technical guidance to Apache Hadoop, Spark or related big data projects.
· Significant contribution to development for applications or services with a large user base