As a Big Data Backend Engineer, you build out the data science techniques inside DataRobot utilizing distributed frameworks such as Spark and Hadoop. You will build scalable solutions across the entire machine learning process: loading, cleaning and transforming data, training and tuning predictive modeling jobs, and ensuring consistent and fast predictions. You will also be responsible for designing and implementing features from start to finish, including clean and easy to use APIs, automated tests, and deployment infrastructure.

The ideal candidate can bring new ideas from concept to implementation, write quality, testable code, and participate in design/development discussions.

Main Requirements

• 5+ years of experience in Python or a JVM based language, such as Java or Scala
• 3+ years of experience in architecting and developing distributed systems
• Some experience with data processing using at least one of the following tools:
• Pandas
• Numpy/Scipy
• Scikit-learn
• Spark
• In the interview process, you will be evaluated on your performance in a number of coding and design scenarios - be prepared to think!

Desired Skills

• Ability to communicate about technical topics
• Willingness to learn about new technologies
• Experience in some or all of the following:
• System/performance evaluation such as profiling process memory/cpu/io/network usage and language-specific debugging tools
• Document-oriented databases such as MongoDB
• Messaging services such as ZMQ or RabbitMQ
• Hadoop services such as Yarn, Spark and HDFS
• Docker