Principal Data Engineer

Job Requisition ID #25WD93165Position OverviewAutodesk seeks a strong principal engineer and technical leader within the Data Acquisition team, designated with designing new robust and resilient data pipelines for analytical abstracts, enrichment, and metadata about design content. These datasets form the basis for data selection and understanding for various AI/ML projects at Autodesk.This role collaborates with Product Managers, Program/Project managers, Analysts, Platform Architecture, and the Engineering Team to ensure data pipelines and delivery uses modern, scalable, reliable methods. Communication skills are critical, and you must be comfortable distilling complex technical information to a non-technical audience as a technical leader.We work with a group of highly collaborative individuals who are excited to solve challenges, and implement cloud based, resilient, available, reliable solutions. We are comfortable with several languages and paradigms and aim to choose the technology which is most appropriate for the task. We endeavor to automate our tests and deployments end to end. We seek to mitigate existing issues and balance tech debt with the delivery of value to Autodesk. If you can identify with these traits, we are looking forward to working with you.Responsibilities Designing, architecting, solutioning, and implementing data pipelines for enterprise consumptionMapping and automating data flows, from multiple sourcesDevelop workflows to run Apache Spark applications using Airflow and AWS EMR; may also work on Snowflake applications using Airflow and dbtDeploy solutions via automated CI/CD processes to execute data pipelines running in batchDeveloping, executing, and documenting software solutions to address complex data collection, processing, transformation, and reporting issuesInterpreting and translating business needs into technical requirementsTroubleshooting code level problems quickly and efficiently as neededParticipating in code and document reviewsMinimum QualificationsStrong experience in several of the following: Pytho / PySpark, Spark / Spark SQL, Apache Iceberg/Airflow (Astronomer is a plus), Amazon EMR, Lambda, S3, Snowflake / Snowflake SQL, dbt, Fivetran, Hive, HadoopExtensive experience with a strong focus across layers of data engineering and pipelinesStrong experience in distributed and cloud architecture patternsAbility to provide detailed architecture artifacts (L1 / L2 diagrams, sequence diagrams, etc.)Firm understanding of programming paradigms, their usage, as well as data structureEffective communication and collaboration skills, understanding how to tailor your message to your audienceAbility to fully develop a feature across all data and/or application layers from inceptionExperience building and deploying data applications with AWSExperience implementing CI/CD and familiarity TDD/BDD. Preferably concrete examples of eachStrong experience designing and using data lakes and data warehousesPowerful sense of ownership with a bias for action when challenges ariseAbility to work independently and collaboratively across an organization.Desire to create, investigate and follow best practicesFamiliarity with AI/Machine Learning workflows and technologies is a plusEagerness to learn and share knowledge with a good attitudeLearn MoreAbout AutodeskWelcome to Autodesk! Amazing things are created every day with our software – from the greenest buildings and cleanest cars to the smartest factories and biggest hit movies. We help innovators turn their ideas into reality, transforming not only how things are made, but what can be made.We take great pride in our culture here at Autodesk – it’s at the core of everything we do. Our culture guides the way we work and treat each other, informs how we connect with customers and partners, and defines how we show up in the world.When you’re an Autodesker, you can do meaningful work that helps build a better world designed and made for all. Ready to shape the world and your future? Join us!Salary transparencySalary is one part of Autodesk’s competitive compensation package. Offers are based on the candidate’s experience and geographic location. In addition to base salaries, our compensation package may include annual cash bonuses, commissions for sales roles, stock grants, and a comprehensive benefits package.Diversity & BelongingWe take pride in cultivating a culture of belonging where everyone can thrive. Learn more here: https://www.autodesk.com/company/diversity-and-belongingAre you an existing contractor or consultant with Autodesk? Please search for open jobs and apply internally (not on this external site).