Senior Software Engineer, Big Data Systems

About the Role:

With such a large catalog of content, data and machine learning are fundamental to Tubi's success, and Tubi's Data Platform is the cornerstone of all data and ML use. At Tubi, you will join a stellar team of engineers with a passion for solving the most challenging problems using cutting-edge technology. In this Senior Data Engineering role, you will be expected to be a hands-on leader, leading by example as you and your team build out systems to handle data in real-time on a massive scale. You will build and maintain a cutting-edge cloud ecosystem at Tubi using modern Infrastructure as Code (IaC) principles to support computing and storage needs for big data processing. You will drive best practices in security, deployment, CI/CD, system observability, ensuring continuous improvement in automation and developer efficiency. In short, you will enable Tubi to be truly data-driven.

Responsibilities: 

• Handle the collection and processing of large scale raw data
• Develop and manage big data infrastructure on Kubernetes, ensuring availability, scalability, resiliency and security
• Manage, and effectively integrate home grown infrastructures with cloud provider services like AWS and Databricks
• Builds and improves infrastructure for data extraction, transformation, loading and cleaning from diverse sources using APIs, SQL and AWS technologies
• Improving data quality by building tools, processes and pipelines to enforce, check and manage data quality at a large scale
• Implement CI/CD pipelines for data operations ensuring efficient and smooth deployment of data models and applications

Your Background:

• 5+ years of experience building scalable batch and streaming data pipelines (Spark or Flink)
• 3+ years of experience designing and implementing pipelines for ETL and cleaning of data from a wide variety of sources using APIs, SQL, Spark and AWS technologies
• Strong knowledge of Kubernetes, Streaming, distributed databases, and cloud storage (e.g., S3)
• Strong experience in JVM language (Scala is not required, but preferred)

Preferred Qualifications:

• Experience building and managing message queues like Kafka and Kinesis
• Experience with Spark and Flink cluster development and maintenance
• Experience with AWS ecosystems
• Have good communication skills and can clearly communicate thoughts and ideas with junior engineers and senior management