Staff Software Engineer, ML Infrastructure

The Difference You Will Make:

At Airbnb, our mission is to create a world where anyone can belong anywhere. We use Machine Learning & Artificial Intelligence extensively across our product to create a more connected, empowered, and safer global community and enable an intelligent & worry-free travel experience.

As a Staff Software Engineer on our Feature Platform, a cornerstone for building robust and high-performing machine learning models, you will play a crucial role in creating and optimizing scalable batch and streaming computing systems that empower our state-of-the-art ML models. Your work will drive the efficiency and effectiveness of our AI and ML efforts, ensuring that we can harness the full potential of our data to generate insights and predictions that matter.

You will leverage cutting-edge frameworks, innovative algorithms, and advanced hardware to build and optimize the backbones of data-centric AI. This includes, but is not limited to, developing and advancing our Open-Sourced Chronon project, and ensuring AI/ML observability. Your contributions will directly impact the accuracy, scalability, and speed of our ML models, driving business outcomes and enabling us to stay ahead in a competitive landscape.

A Typical Day: 

A typical day in this role is dynamic, creative, and collaborative. You will have opportunities to work with diverse AI/ML practitioners across the company to address a wide variety of data processing, feature engineering, and online serving challenges:

• Collaboration and Alignment: You will work closely with AI/ML teams to align on requirements and incorporate the latest advancements in machine learning, including feature engineering. This involves understanding their needs, translating them into technical specifications, and ensuring that the solutions you build meet these requirements.
• Development and Optimization: You will spend a significant portion of your day coding, debugging, and testing new features and enhancements. This includes working with large datasets, implementing efficient data processing pipelines, and ensuring the seamless integration of batch and streaming systems. You will also be responsible for deploying and monitoring these systems, using your expertise to troubleshoot data quality issues and ensure high availability and reliability.
• Quality and Performance Analysis: You will dive deep into data quality and performance metrics, identifying root causes and bottlenecks, and brainstorming mitigation and optimization strategies. This involves using advanced analytics and monitoring tools to ensure that our systems are running efficiently and effectively.
• Technical Excellence: You will maintain high standards of technical quality, including architecture soundness, code quality, and performance. You will participate in architecture design discussions, code reviews, and tech transformations, contributing to various tech excellence initiatives to ensure our systems are robust and scalable.
• Open Source Contribution: You will contribute to the growth of a healthy open-source community by managing major improvement proposals, performing pull request reviews, and contributing to the Chronon open source repository. This involves both administrative duties and active development to enhance the community and the project.
• Continuous Learning: You will stay updated with the latest trends and advancements in AI and ML to continuously push the boundaries of what is possible in feature engineering and AI/ML infrastructure. This includes attending conferences, participating in workshops, and reading the latest research papers.
• Technical Leadership: You will inspire and guide your peers, including teammates and partners, towards achieving excellence in their respective roles. This involves providing best practices in both software engineering and AI/ML workflows, mentoring junior engineers, and fostering a culture of continuous improvement and innovation within the team.

Your Expertise:

• Strong Software Engineering Background: You have a robust foundation in software engineering, with a specific focus on building and optimizing large-scale, reliable, and maintainable data processing systems. Your expertise ensures the creation of systems that can handle vast amounts of data efficiently and robustly.
• Batch and Streaming Technologies: You possess extensive experience with both batch and streaming technologies, such as Spark, Kafka, Spark Streaming, and Flink. You are proficient in programming languages like Scala, Python, or Java, enabling you to develop and optimize complex data processing pipelines.
• Machine Learning Frameworks: You have a deep understanding of machine learning frameworks and libraries, including TensorFlow, PyTorch, or similar. This knowledge allows you to effectively integrate advanced machine learning techniques into our feature engineering platform.
• Distributed Algorithms and Cloud Infrastructure: You have a solid grasp of distributed algorithms and data structures. You are adept at leveraging cloud infrastructure and hardware accelerators, such as Data Warehouses, Airflow, Kubernetes, KV stores, and GPUs, to enhance computational efficiency and achieve high-quality platform capabilities.
• End-to-End Machine Learning Platforms (A Plus): You have a proven track record in building end-to-end machine learning platforms. This experience is highly desirable plus as it demonstrates your ability to oversee the entire lifecycle of machine learning projects, from data ingestion to model deployment and monitoring.
• Hands-on Model Development and Deployment (A Plus): Hands-on experience in developing and deploying machine learning models is a significant plus. Your practical experience in this area will enable you to contribute effectively to the continuous improvement of our ML feature engineering platform, ensuring that it meets the highest standards of performance and reliability.

Your Location:

This position is US - Remote Eligible. The role may include occasional work at an Airbnb office or attendance at offsites, as agreed to with your manager. While the position is Remote Eligible, you must live in a state where Airbnb, Inc. has a registered entity. Click here for the up-to-date list of excluded states. This list is continuously evolving, so please check back with us if the state you live in is on the exclusion list. If your position is employed by another Airbnb entity, your recruiter will inform you what states you are eligible to work from.

Our Commitment To Inclusion & Belonging:

Airbnb is committed to working with the broadest talent pool possible. We believe diverse ideas foster innovation and engagement, and allow us to attract creatively-led people, and to develop the best products, services and solutions. All qualified individuals are encouraged to apply.

We strive to also provide a disability inclusive application and interview process. If you are a candidate with a disability and require reasonable accommodation in order to submit an application, please contact us at: [email protected]. Please include your full name, the role you’re applying for and the accommodation necessary to assist you with the recruiting process. 

We ask that you only reach out to us if you are a candidate whose disability prevents you from being able to complete our online application.