Scope:As an Cloud Engineer, you will be responsible to design, deploy, configure, and maintain large-scale Apache Flink and Apache Beam clusters on Kubernetes within the Azure cloud platform. You will collaborate with data engineers, data scientists, and other stakeholders to ensure these clusters meet performance, scalability, and availability requirements in a Kubernetes and Azure environment.What you'll do:Provision and manage Azure resources required for Apache Flink and Apache Beam clusters, including Azure Kubernetes Service (AKS) clusters, Storage, and Networking components.Collaborate with architecture teams to design Flink and Beam clusters optimized for Kubernetes and Azure.Define Kubernetes configurations, including pods, services, and ingress controllers.Deploy, configure, and maintain Apache Flink and Apache Beam clusters on Kubernetes.Implement automation scripts, Helm charts, and Kubernetes Operators for cluster deployments and updates.Set up Azure Monitor and Log Analytics to monitor the performance and health of Flink and Beam clusters.Create custom monitoring and alerting solutions using Azure Monitor.Configure auto-scaling and resource optimization strategies to handle varying workloads efficiently.Implement Azure Autoscale for dynamic resource allocation.Provide training to team members on managing these clusters within the Azure environment.Collaborate with data engineering, data science, and Azure cloud teams to align cluster architecture with business objectives.Communicate effectively with stakeholders regarding the status, issues, and enhancements of the Azure-based Flink and Beam clusters.What we are looking for:Bachelor's or higher degree in Computer Science or a related field.5+ years of experience building high performance distributed systems, solving large scale engineering challenges.Proven experience in setting up and managing Apache Flink and Apache Beam clusters on Kubernetes at scale.Strong understanding of Kubernetes concepts, including pod management, ingress controllers, and stateful sets.Proficiency in Azure AKS and Azure Kubernetes integration.Kubernetes certifications (e.g., Certified Kubernetes Administrator) are a plus.Experience with DevOps practices and tools for Kubernetes.Excellent problem-solving and communication skills.Certification in Apache Flink and Apache Beam or related technologies.Familiarity with other Azure services and cloud-native technologies.Knowledge of stream processing and big data technologies.Our ValuesIf you want to know the heart of a company, take a look at their values. Ours unite us. They are what drive our success – and the success of our customers. Does your heart beat like ours? Find out here: Core ValuesDiversity, Inclusion, Value & Equality (DIVE) is our strategy for fostering an inclusive environment we can be proud of. Check out Blue Yonder's inaugural Diversity Report which outlines our commitment to change, and our video celebrating the differences in all of us in the words of some of our associates from around the world.All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability or protected veteran status.
View Original Job Posting