Senior / Principal Hadoop Administrator
San Francisco , California
February 08, 2017
About the Job
At Oracle Utilities (Opower), we're applying cutting-edge computer science to one of humanity's greatest challenges: Energy. Our utility customers in the U.S. and abroad give us energy usage data for tens of millions of their customers, which we then analyze and aggregate using state-of-the-art tools such as Hadoop and Spark. If you are a top-notch system administrator looking for a fast-paced place to work while being surrounded by talented, driven peers then Oracle Utilities is the place for you.
  • Take the lead in ongoing administration of Hadoop infrastructure
  • Work with the data platform team to optimize cluster usage and ensure timely execution of business-critical workloads
  • Install Hadoop updates, patches, and version upgrades as required
  • Perform routine cluster maintenance, such as provisioning new nodes and performing HDFS backups and restores
  • Routine cluster monitoring and troubleshooting
  • Configuration and monitoring of job isolation, security, and resource queues
  • Develop scripts and tools to automate common Hadoop administration tasks
  • Work with the devops and data infrastructure teams to identify areas of the Hadoop infrastructure that can be improved
  • Participate in an on-call rotation with 24x7 availability requirements
About You
  • 5 or more years in a typical system administration role, performing system monitoring, storage capacity management, performance tuning, and system infrastructure development.
  • Minimum 1 year of experience in deploying and administering a large Hadoop cluster. Ability to isolate and troubleshoot Hadoop service issues using a combination of system and Hadoop logs and monitoring/alerting systems. Experience with Cloudera Manager is a plus.
  • Preferred experience administering Hadoop 2 Clusters to include YARN and YARN-based applications such as Spark.
  • Detailed knowledge of basic OS administration tasks such as configuring PAM authentication, disk quota, ulimit, etc. and managing security patches.
  • Experience configuring and using cluster-wide monitoring tools such as OpenTSDB, Splunk, and Grafana to diagnose cluster issues and propose operational enhancements.
  • Experience administering a data warehouse infrastructure based on Apache Hive. Management of Hive UDF's and PySpark dependencies a plus.
  • Experience coordinating rolling OS-level changes with cluster administration tool such as Cloudera Manager.
  • Experience with a configuration management tool such as Puppet or Chef.
  • Experience with a scripting language such as Ruby or Python
  • Experience managing search infrastructure such as Elasticsearch is a plus.

Analyze, design develop, troubleshoot and debug software programs for commercial or end user applications. Writes code, completes programming and performs testing and debugging of applications.

As a member of the software engineering division, you will analyze and integrate external customer specifications. Specify, design and implement modest changes to existing software architecture. Build new products and development tools. Build and execute unit tests and unit test plans. Review integration and regression test plans created by QA. Communicate with QA and porting engineering to discuss major changes to functionality.

Work is non-routine and very complex, involving the application of advanced technical/business skills in area of specialization. Leading contributor individually and as a team member, providing direction and mentoring to others. BS or MS degree or equivalent experience relevant to functional area. 7 years of software engineering or related experience.

Oracle will consider for employment qualified applicants with criminal histories in a manner consistent with the requirements of San Francisco's Fair Chance Ordinance.

Oracle is an Equal Employment Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability and protected veterans status or any other characteristic protected by law.

A little about us:
Oracle is shifting the complexity from IT, moving it out of the enterprise by engineering hardware and software to work together—in the cloud.

Know someone who would be interested in this job? Share it with your network.