We are seeking experienced Senior System Software Engineers adept at Apache Spark to join our team. Data scientists spend a considerable amount of time exploring data and iterating over machine learning (ML) experiments. Every hour of compute required to sort through datasets, extract features and fit ML algorithms impedes an efficient business workflow. NVIDIA believes that data science workflows can benefit tremendously from being accelerated, to enable data scientists to explore many more and larger datasets to drive towards their business goals, faster and more efficiently.At NVIDIA, we are passionate about working on hard problems that have an impact. You will need to have previous experience working with Apache Spark applications, implementing big data applications for a variety of customers, programming skills, and familiarity with open source big data processing frameworks. You should be comfortable working with interdisciplinary teams. You will work with an engineering team accelerating Apache Spark with GPUs using CUDA and open source libraries. This is an opportunity to work with a team that is developing the Spark RAPIDS open source library to accelerate Spark applications. This is a strategic investment for NVIDIA. The code is being adopted by multiple cloud service providers and Apache Spark distributions.What you'll be doing:Designing and developing a world-class GPU accelerated Apache Spark service.Creating a collection of micro-services to provide the ability to run Spark applications on Kubernetes or other platforms.Implementing the REST API and its client libraries to simplify customer adoptions.Customizing the open-source projects to meet the project requirements.Deploying and verifying the solution on CSPs or on-prem Kubernetes environments.Engaging open source communities, including Apache Spark and RAPIDS, for technical discussions and contributionsWhat we need to see:8+ years of experience in software development.5+ years hands on experience with web service design and developmentBS/MS/PhD in computer science or a related fieldFamiliarity with the REST service frameworks like Spring BootFamiliarity with the modern data open source ecosystem (Apache Spark, Apache Kyuubi, Apache Zookeeper, gRPC, etc)Experience building performance and reliable service APIsExperience working on public and private cloud platformsPrior experience supporting enterprise customersKnowledge of SQL, Python and Scala/Java, Kubernetes and Helm chartsExcellence at communicating, presenting and explaining technical topicsWays to stand out from the crowd: Working experience with Apache Spark services: Databricks, AMS EMR, GCP Dataproc, Azure Synapse Analytics. Contributions to major open source projects such as Apache Spark, Apache Kyuubi, Apache Ranger, Apache Iceberg, and Delta Lake. Development experience of Apache Spark Data Sources and connectors. Development experience with Spark Client interfaces/tools like Jupyter, Zeppelin, Spark Connect, and BI tools.Basic ML/DL experience with PyTorch, TensorFlow, Spark ML and XGBoost.We are an AA/EEO/Disabled employer and with highly competitive salaries and a comprehensive benefits package. NVIDIA is widely considered to be one of the technology world’s most desirable employers. We have some of the most forward-thinking and hardworking people in the world working for us. Are you creative and autonomous? Do you love a challenge? If so, contact us!
View Original Job Posting