As a Master Data Engineer you will provide engineering leadership to create and enhance data solutions enabling seamless integration and flow of data across the data ecosystem. Additionally you will provide senior level technical consulting to peer data engineers during design and development for highly complex and critical data applications. This role will be responsible for Data Application Solution Engineering which will include design and development of data based and application solutions. Leveraging open source tools such as Java, Tez, NiFi, Sqoop, Hive, Pig, Python, etc. Additionally you will work on real time processing solutions using tools such as Spark, Apex, Storm and Kafka. You will deploy application code and analytical models using CI/CD tools and techniques and provide support for deployed data applications and analytical models.
* Break down complex requirements into workable technical solutions to solve business problems
* Build high quality software for large scale and highly available systems
* Utilize multiple development languages/tools such as Python, SPARK, HBase, Hive, Microsoft R, Java to build prototypes and evaluate results for effectiveness and feasibility
* Evaluate complex features and prepare application specifications
* Actively contribute to the Enterprise Decision & Data Management department's developer community to support technical initiatives and provide input into best practices
* Demonstrate a high-level of initiative, ability to work independently and a willingness to grow in a challenging environment
* Provide strong technical and team leadership
* Demonstrate effective verbal and written communication skills
* Promote a risk-aware culture, ensure efficient and effective risk and compliance management practices by adhering to required standards and processes
* Deep knowledge and very strong in SQL, and Relational Databases and strong in Unix / Shell scripting
* Work heavily within the Hadoop ecosystem to ingest and transform data from multiple sources including RDBMS, files, real time streaming and migrate data from Teradata to Hadoop.
* Deep understanding of the Hadoop technology stack, preferably the HortonWorks distribution
* Decision making abilities while gathering information and then put your decisions into action.
* Provide support for deployed data applications and analytical models by being a trusted advisor to Data Scientists and other data consumers by identifying data problems and guiding issue resolution with partner Data Engineers and source data providers.
* Provide subject matter expertise in the analysis, preparation of specifications and plans for the development of data processes.
* Passionate learner who enjoys education through class room training and self-discovery on a variety of emerging technologies
* Excellent written and verbal communication, presentation and professional speaking skills
* Influencing skills/ability. Must be able to work with different levels of management and all business areas.
#LI-KE We are an Equal Opportunity Employer and do not discriminate against any employee or applicant for employment because of race, color, sex, age, national origin, religion, sexual orientation, gender identity, status as a veteran, and basis of disability or any other federal, state or local protected class.
A little about us:
Discover is one of the most recognized brands in U.S. financial services. We’re a direct banking and payment services company built on a legacy of innovation and customer service.