Manager, Data Engineer - Scala
We are building an amazing data playground. A highly scalable distributed platform that takes all of our data (petabytes of it), and makes it available to analysts and systems so that we can continue doing what we’ve always done – use data to make banking better.
As a Manager, Senior Data engineer on the Data Intelligence team, you will contribute to building a fast data and machine learning platform scalable to solve diverse business problems. We envision, create, deploy, and maintain full stack technology solutions powered by streaming big data, state of the art machine learning, micro-service architecture, and intuitive visualizations in the cloud.
At Capital One, we have seas of big data and rivers of fast data. To manage this, we are working with a number of cutting edge machine learning technologies, and are actively developing more. We are highly technical with strong backgrounds in what we do. Our use cases range from cyber threat prevention to predicting environment outages to enable our always on 24/7 services. We have the highest executive support and direct impact on our customer experience and bottom line.
On any given day you will:
Transform complex analytical models into scalable, production-ready solutions
Develop applications from ground up using a modern technology stack such as Scala, Spark, Postgres and NoSQL
Lead and develop sustainable data driven solutions with current new gen data technologies to meet the needs of our organization and business customers
Master new technologies rapidly as needed to progress varied initiatives
Provide technical guidance to team members
What we have:
- A startup mindset with the backing of a top 10 bank
- Monthly Innovation-Days dedicated to test driving cutting edge technologies - Flexible work schedules
- Convenient office locations
- Generous salary and merit-based pay incentives
- Your choice of equipment (MacBook/PC, iPhone/Android Device)
- Bachelor’s Degree or military experience
- 3+ years in coding in Scala, Python or Java
- At least 3 years experience with big data technologies (Cassandra, Accumulo, HBase, Spark, Hadoop, HDFS, AVRO, MongoDB, Zookeeper or similar)
- Master's Degree
- 3+ years with Agile engineering practices
- 3+ years in-depth experience with the Hadoop stack (MapReduce, Pig, Hive, Hbase)
- 3+ years experience with NoSQL implementation (Mongo, Cassandra, etc. a plus)
- 3+ years experience with Relational Database Systems and SQL
- 3+ years experience with Linux including basic commands and shell scripting
- Familiarity with data science tools and concepts
- Experience in industry, i.e. code committer, published, white paper, etc.