Senior Data Engineer

Job Summary:

The Senior Data Engineer + Generative AI will design, develop, and manage advanced data solutions with a strong focus on natural language processing (NLP) and generative AI technologies. This role requires an experienced engineer with a robust background in data pipeline architecture, cloud platforms, and big data technologies. The ideal candidate will have 5-8 years of relevant experience and will be proficient in implementing and managing scalable data infrastructures, as well as applying NLP techniques and generative AI models. Excellent communication skills and the ability to build effective working relationships are essential.

Key Responsibilities:

1) Data Pipeline Architecture

• Design, develop, and maintain scalable data pipelines.
• Ensure high data quality and perform data cleansing tasks.

2) Database Management: Utilize advanced SQL skills for database management and optimization.

3) Cloud Platforms: Implement solutions deployment on any of AWS, GCP, or Azure.

4) Programming: Develop robust applications using Python, Java, or Scala.

5) Big Data Technologies: Work with Hadoop and Spark for data processing.

6) Version Control: Collaborate using Git for version control.

7) Performance Tuning: Optimize system performance and ensure efficient data processing.

8) System Architecture: Design and implement system architectures.

9) Agile/Scrum: Participate in Agile/Scrum methodologies for project management.

10) Apply NLP techniques such as tokenization, word embeddings, language modeling, sequence labeling, and text generation.

11) Utilize LangChain for data processing, LLMs for data embedding, Vector databases, and prompt engineering.

Preferred Skills

1) AI Frameworks: Proficiency in TensorFlow, PyTorch, or Hugging Face Transformers for model development and experimentation.

2) Machine Learning

• Basic understanding of machine learning concepts.
• Experience with ML and AI techniques such as multivariate/logistic regression models, cluster analysis, predictive modeling, neural networks, deep learning, pricing models, decision trees, and ensemble methods.

3) Data Visualization: Proficiency with tools like Tableau and PowerBI.

4) Data Warehousing: Experience with Redshift or BigQuery.

5) Scripting: Knowledge of Bash or Perl for scripting.

6) Workflow Orchestration: Familiarity with Airflow or Luigi.

7) NoSQL Databases: Experience with Cassandra or MongoDB.

8) Real-time Data Processing: Knowledge of Kafka or Flink.

9) API Development: Skills in API development and integration.

10) DevOps: Understanding of DevOps practices.

11) Containerization: Experience with Docker and Kubernetes.

12) Business Intelligence: Familiarity with business intelligence tools.

Qualifications

• 5-8 years of experience in data engineering, with a focus on cloud platforms, big data technologies, and NLP.
• In-depth knowledge of data pipeline architecture, database management, and performance tuning.
• Strong understanding of NLP fundamentals and generative AI techniques.
• Excellent problem-solving skills and the ability to troubleshoot complex issues.
• Strong communication and collaboration skills.

Education

• Bachelor's or Master’s degree in Computer Science, Information Technology, or a related field.

WHO WE ARE

Credera has received numerous state and national awards! Please check out our employer awards at: Credera Awards.

Our data capabilities help our clients gain a competitive advantage by transforming raw data into insights that enable actionable decision-making. Our business-driven approach to envisioning and implementing innovative solutions helps organizations tackle their market, increase customer engagement, and grow revenue profitably via advanced technology.

Our data capabilities serve our clients across a broad spectrum of areas, specializing in Data Strategy, Modern Data Architecture, Data Visualization, Business Insights, Data Analytics, and Artificial Intelligence.

HOW WE WORK

• We work in small (typically 2-8 person) project teams that collaborate closely with clients to solve their toughest challenges.
• We employ principles of agile development to achieve high velocity and quality of delivery.
• We offer several channels for collaboration that make it easy to reach out to leaders within the practice.
• We adhere to Credera’s core values of integrity, excellence, professionalism, and humility at all times.

BASIC QUALIFICATIONS

• Thrive in a fast-paced, dynamic, client-facing role where delivering solid work products to exceed high expectations is a measure of success.
• Contribute in a team-oriented environment.
• Prioritize multiple tasks in order to consistently meet deadlines.
• Creatively solve problems in an analytical environment.
• Adapt to new environments, people, technologies, and processes.
• Excel in leadership, communication, and interpersonal skills.
• Establish strong work relationships with clients and team members.
• Generate ideas and understand different points of view.

LEARN MORE

Credera is a global consulting firm that combines transformational consulting capabilities, deep industry knowledge, and AI and technology expertise to deliver valuable customer experiences and accelerated growth across a broad range of industries worldwide. Our one-of-a-kind global boutique approach means we provide our clients with tailored solutions unique to their organization that can scale due to our extensive footprint.

As a values-led organization, our mission is to make an extraordinary impact on our clients, our people, and our community. We believe it is this approach that has allowed us to work with and transform the most influential brands and organizations in the world, from strategy through to execution.

More information is available at www.credera.com. We are part of the OPMG Group of Companies, a division of Omnicom Group Inc.

NOTE

Credera will never ask for money up front and will not use apps such as Facebook Messenger, WhatsApp, or Google Hangouts for communicating with you. You should be very wary of, and carefully scrutinize, any job opportunity that asks for money prior to starting and/or one where all communications take place exclusively via chat.