Software Engineer II (Storage Services)

Company: Affirm
Company: Affirm
Location: Remote US
Posted on: 2024-03-02 01:01
Affirm is reinventing credit to make it more honest and friendly, giving consumers the flexibility to buy now and pay later without any hidden fees or compounding interest. The vision for the Analytics Storage @ Affirm team is to provide a reliable, scalable, secure, self-serve and Affirm-optimized storage and compute solution to enable ML, Product, and Financial Engineering success. Our team is responsible for building and maintaining a Analytics Storage platform that serves as a backbone for processing various business critical batch ETL workloads and analytics activities at Affirm through a wide range of scheduled and on-demand compute solutions built on the cloud. As a Software Engineer in the Analytics Storage team, you will be responsible for managing, optimizing, and ensuring the reliability and performance of our data lake. The ideal candidate will have extensive experience in database administration, a strong understanding of Apache Iceberg, and the ability to work closely with data engineers and analysts to support data-driven decision-making. What you’ll do Implement, manage, and monitor databases in our Apache Iceberg-powered data lake environment to ensure high levels of data availability and performance. Work with data engineering teams to design and implement scalable database schemas and optimize data storage and retrieval processes. Perform regular database maintenance tasks such as backups, indexing, and performance tuning to ensure data integrity and efficiency. Develop and implement data security measures, including access controls and encryption, to protect sensitive information. Collaborate with data analysts and business teams to understand data requirements and ensure the database meets business needs. Troubleshoot and resolve database-related issues in a timely manner. Stay current with emerging technologies and advancements in Lakehouse architectures, specifically Apache Iceberg, to recommend and implement improvements to our data infrastructure. Document database architectures, procedures, and processes for internal use and compliance purposes. What we look for Knowledge of SQL: Proven experience as a Database Administrator, with a strong preference for experience in managing data lakes and using Apache Iceberg. Deep understanding of database principles, architecture, and data modeling techniques. Python:. 1-3 years of general python experience Spark: Extensive experience with writing Spark SQL and working with DataFrames Experience with debugging Spark applications via metrics, history server, etc Understanding of  shuffling and re-partitioning concepts Understanding of off-heap and on-heap memory usage in Spark nderstands joins in a distributed context; eg sort-merge vs broadcast joins (nice to have)  Kubernetes: knowledgeable running Spark on Kubernetes, cluster experience using kubectl to inspect issues experience with Docker Experience with cloud data services (AWS, Azure, GCP) and familiarity with data lake ecosystems. Compensation & Benefits We offer a competitive package, with some highlights listed below:  Flexible Spending Wallets for tech, food and lifestyle Generous time off policies  Away Days - wellness days to take off work and recharge Learning & Development programs Parental leave Robust health benefits Employee Resource & Community Groups Pay Grade - USA29 Employees new to Affirm or promoted into a new role, typically begin in the min to mid range. USA base pay range (CA, WA, NY, NJ, CT) per year: Min: $138,800 Mid: $173,500 Max: $208,200 USA base pay range (all other U.S. states) per year: Min: $124,900 Mid: $156,100 Max: $187,300 #LI-Remote Affirm is proud to be a remote-first company! The majority of our roles are remote and you can work almost anywhere within the country of employment. Affirmers in proximal roles have the flexibility to work remotely, but will occasionally be required to work out of their assigned Affirm office. A limited number of roles remain office-based due to the nature of their job responsibilities. We’re extremely proud to offer competitive benefits that are anchored to our core value of people come first. Some key highlights of our benefits package include:  Health care coverage - Affirm covers all premiums for all levels of coverage for you and your dependents  Flexible Spending Wallets - generous stipends for spending on Technology, Food, various Lifestyle needs, and family forming expenses Time off - competitive vacation and holiday schedules allowing you to take time off to rest and recharge ESPP - An employee stock purchase plan enabling you to buy shares of Affirm at a discount We believe It’s On Us to provide an inclusive interview experience for all, including people with disabilities. We are happy to provide reasonable accommodations to candidates in need of individualized support during the hiring process. [For U.S. positions that could be performed in Los Angeles or San Francisco] Pursuant to the San Francisco Fair Chance Ordinance and Los Angeles Fair Chance Initiative for Hiring Ordinance, Affirm will consider for employment qualified applicants with arrest and conviction records. By clicking "Submit Application," you acknowledge that you have read the Affirm Employment Privacy Policy for applicants within the United States, the EU Employee Notice Regarding Use of Personal Data (Poland) for applicants applying from Poland, the EU Employee Notice Regarding Use of Personal Data (Spain) for applicants applying from Spain, or the Affirm U.K. Limited Employee Notice Regarding Use of Personal Data for applicants applying from the United Kingdom, and hereby freely and unambiguously give informed consent to the collection, processing, use, and storage of your personal information as described therein.
View Original Job Posting