Principal Software Engineer
Are you excited about the possibility of changing the world with Big Data by working on the largest Object Storage service in the world? Come join the Azure Storage team in building HDFS support directly in Azure Storage and revolutionize the way Cloud Storage supports Big Data!
The Azure Storage team is currently chartered with building, managing, and running the Persistent Cloud Storage for Microsoft Azure Cloud. Azure Storage consists of a Storage Platform which currently powers five different storage services – Blob Storage, Table Storage (NoSql Key Value store), Queue Storage, File Storage (SMB based File Storage) and Disk Storage. We are one of the foundational services in the Azure Cloud. This service hosts data from some of the largest companies in the world plus supports all of Microsoft’s largest online businesses including Xbox, One Drive, Skype and Bing, and has scaled to store exabytes of customer data with the roadmap looking even better in terms of growth. We are building a new foundational Storage service with HDFS support.
You can read more about the Service architecture here: (https://blogs.msdn.microsoft.com/windowsazurestorage/2011/11/20/sosp-paper-windows-azure-storage-a-highly-available-cloud-storage-service-with-strong-consistency/)
and about the current storage offerings here:
Storage is divided into three main layers. This position is with the Partition Layer team, which forms the heart of the system and provides: transaction support, querying capability, indexing, geo replication, partitioning, multi tenancy, and many other capabilities. The work for this position will cross many layers of the Azure Storage system and Azure as a whole, presenting unique engineering challenges for which we need top-notch talent. Right now is an exciting time for the team as we are working on expanding the range of supported scenarios for the service while working on the next version architecture of the storage service to support the next 100X growth. If you are excited about large scale data management problems, this is a unique opportunity to have an impact in a team unlike any other.
The specific role for this position is to build HDFS support in Azure Storage. The new HDFS support will be used to support Big Data workloads natively on top of Azure Storage, including Hadoop, Map Reduce, Hive, HBase, and more. If you have prior experience in working on HDFS or Big Data, this is your opportunity to build the system from ground up. People with prior experience in Security, especially building Kerberos/OAuth based system, are also a great fit as secure Hadoop is one of the biggest emerging area in the Big Data world and we intend to invest heavily in that area.
To be successful, you must thrive on solving problems related with data durability, availability and concurrency issues in a multi-tenant environment. You must have passion for storage and distributed systems. You will be challenged on a daily basis as you build the largest storage solution Microsoft has.
An ideal candidate will bring:
• Strong distributed systems knowledge
• Hands on development experience including designing and writing code
• Proven ability to plan, schedule, and deliver high quality software
• Passion for customers
• Excellent collaboration skills
• Big Data experience a plus, especially in the Hadoop and/or HPC ecosystem
• 7 years of building scalable system software and/or services in multithreaded C/C++
• BS/MS/PhD in Computer Science, Computer Engineering, or Electrical Engineering
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, gender, sexual orientation, gender identity or expression, religion, national origin, marital status, age, disability, veteran status, genetic information, or any other protected status.
Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings:
Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.