Principal Cloud and Storage Engineer
West Chester , Pennsylvania
October 24, 2017

Comcast's Technology & Product organization works at the intersection of media and technology. Our innovative teams are continually developing and delivering products that transform the customer experience. From creating apps like TVGo to new features such as the Talking Guide on the X1 platform, we work every day to make a positive impact through innovation in the pursuit of building amazing products that are enjoyable, easy to use and accessible across all platforms. The team also develops and supports our evolving network architecture, including next-generation consumer systems and technologies, infrastructure and engineering, network integration and management tools, and technical standards.

If you are an individual who is excited by the prospect of being part of a fast moving company that consistently leads across the entire spectrum of entertainment content delivery, then you want to join Comcast. The members of the Systems Reliability Engineering team provide Comcast customers with an incredible array of products and services within the private and public cloud offerings. The ever-changing technology landscape constantly challenges us to grow our portfolio and to maintain an innovative approach to the management of the hardware and software life cycle. Our team atmosphere is one of comprehensive cooperation across all technology disciplines. We pride ourselves in our approach to problem management and solutions implementation.

Working at Comcast provides great opportunities for individuals to grow both professionally and personally. Training offered by our Comcast University runs the gamut from purely technical courses to very advanced leadership and management opportunities. We offer flexibility and fully support work home life balance for all of our team members. As a member of the Systems Reliability Engineering team, you will provide the full spectrum of engineering support for the enterprise level compute and storage effort. This includes (but is not limited to) hardware and software upgrades, new deployments, decommissions, automation, performance and capacity management. You will be an agent of change and actively influence our continual process improvement, to include the associated technical procedure documentation. We are a team that thrives on big challenges, results, quality, and agility of purpose.

You will be part of the team, but have a high level of independence in your project design and execution efforts. Your day-to-day requirements will be coordinated through our storage team manager who will provide overall guidance and support. We have very limited travel requirements throughout the year. The specific technologies supporting these services are VMware; Cisco UCS blade technology; EMC VMAX, VNX, Data Domain, and Scale IO; Hitachi VSP and G1000 arrays; Dell, HP, and Brocade SAN switches.

We have very exciting times ahead of us; our roadmap for 2018 is changing to meet our new technology landscape. We are very excited to work through technologies such as Containers as a Service (CaaS), Metal as a Service (MaaS), Data Protection as a service, and commodity compute for our virtual infrastructure. As a member of the team, you will play a key role in the operational design, engineering, deployment, and implementation of all aspects of the new technologies. You will have the opportunity to work with incredibly skilled individuals at all levels of the design and engineering process who look forward to the sharing knowledge and experiences.

Overview of Responsibilities:

  • Lead virtual technical teams to design, engineer, implement, and support technology deployments
  • Liaison with vendors to review technology roadmaps
  • Partner with our business units to assist in solution design
  • Provide Level 3 support to internal team members and on call engineer as needed
  • Mentor Engineers 3 and Engineers 4
  • Participate in project teams focused on enhancing the Comcast Cloud infrastructure.
  • Proactive performance and capacity management.
  • Deliver solutions based on pre-defined, negotiated timelines.
  • Create and maintain technical and procedural documentation.

Here are some of the specific technologies we currently use:

  • EMC Storage and NAS Arrays (configuration/implementation/decommission and allocation)
    • VNX/Unity
    • Xtreme I/O
    • SIO
    • VMAX 20K and 200K
    • Unisphere and symcli experience
    • SRDF or other methods for host level migration
  • HDS Storage and NAS Arrays (configuration/implementation/decommission and allocation)
    • HNAS
    • VSP/G1000/G1500
    • Hitachi Command Suite and Storage Navigator
  • NetApp Storage arrays
  • Cisco UCS blades and Fi switches
  • VMware virtualization software
  • Brocade
    • Network Advisor GUI
    • Switch level CLI

Skills & Requirements

5+ years' experience in a 24×7 high-availability production environment

10+ years Information Technology experience as a Windows, Linux, or storage admin

3+ years' experience Vmware Vsphere in large environments

3+ years' experience with virtualization technologies such as OpenStack, and VMWare

5+ years' experience enterprise storage technologies, such as EMC VMAX and Hitachi VSP

5+ years' experience Cisco UCS blade and Fabric Interconnect technology

Experience with Software Defined Storage

Experience with scripting languages such as Powershell, Linux shell, or Perl

Experience with PaaS technology such as Pivotal Cloud Foundry or docker

Experience with CaaS technology

Experience with MaaS technology

Experience with HAproxy on Linux

Experience configuring and troubleshooting SAN and Brocade switches

In depth understanding of TCP/IP LAN/WAN networking technologies and troubleshooting techniques

Experience with relational and NoSQL database technologies such as Oracle, MySQL, Cassandra and CouchDB.

A background in automating the management of a data center environment

Experience with hardware or software based firewalls, load balancers and proxy servers

Experience with intrusion detection systems, network, and server security hardening

Experience in monitoring, metrics collection, and reporting

Excellent organizational skills and oral and written communication skills

Ability to work with minimal supervision, making decisions based upon priorities, schedules and an understanding of business initiatives.

Critical attention to detail, thoroughness and documentation

Providing rotational on call support

General skills

  • Documentation
    • Ability to document new processes and procedures
    • Good communication towards customers and peers
    • Forward thinking on what needs to be done to keep the environment going
      • For example, keeping array code levels consistent, schedule upgrades and update it
      • Mgmt software the team uses needs to be updated, schedule and update it
  • Environmental
    • Ability to work on tight timelines in a dynamically changing environment
    • Daily use of BASH or Python or GO (GO being a huge plus)
    • Reporting and presentation tools sets
    • Familiar with InfluxDB/Grafana or similar
    • Building dashboards for metric presentation
    • Understanding of AWS products a plus
    • Any Linux SA background would be great, especially from a host based migration perspective
    • Ability to work within UCS and VMWare environment (If they can work their way around the console and can find ESX server WWPN's, then we are good)
    • Since going Network based storage, having a good Network background would be very helpful.
    • Displays leadership qualities both technically and professionally
    • Is a subject matter expert in their realm of influence and has industry-wide connections; is aware of industry trends and innovations and brings information back to Comcast

Comcast is an EOE/Veterans/Disabled/LGBT employer

A little about us:
Comcast brings together the best in media + technology. We drive innovation to create the world's best entertainment and online experiences.

Know someone who would be interested in this job? Share it with your network.