At 3Pillar, our focus is on leveraging cutting-edge technologies that revolutionize industries by enabling data driven decision making. As a Senior Data Engineer, you will hold a crucial position within our dynamic team, actively contributing to thrilling projects that reshape data analytics for our clients, providing them with a competitive advantage in their respective industries.
Responsibilities
6+ Years of relevant industry experience in Data Engineering working with large scale data driven systems.
Experience designing data schemas and fine-tuning SQL queries around large, complex data sets. (Expert)
Experience with AWS
Design, build and maintain data infrastructure that powers both batch and realtime processing of billions of records a day.
Improve the data quality and reliability of data pipelines through monitoring, validation and failure detection.
Design, build and maintain a central data cataloging system to ease integration and discovery of datasets.
Develop data pipelines that provide fast, optimized, and robust end-to-end solutions.
Automate manual processes and create a platform in favor of self-service data consumption.
Deploy and configure components to production environments.
Participate in an on-call schedule to provide emergency incident support.
Mentor and train teammates on design and operation of data platforms.
Stay current with industry trends, making recommendations as needed to help the company excel.