Posted On Apr 16, 2024
Job Description
Staff Data Engineer
Job Description
Summary
We are looking for a data engineer with at least 6 years of experience to join our team and help us build and maintain data pipelines, data warehouses, and data platforms. You will be responsible for designing, developing, testing, and deploying scalable and reliable data solutions that support our business needs and data analytics goals.
Responsibilities
- Design and implement data pipelines using various tools and technologies such as Python, SQL, Spark, Kafka, Airflow, etc.
- Optimize and monitor data pipelines for performance, reliability, and quality.
- Build and maintain data warehouses and data marts using cloud services such as AWS, GCP, or Azure.
- Develop and maintain data platforms that enable data ingestion, processing, storage, and access for various data sources and formats such as structured, semi-structured, and unstructured data.
- Collaborate with data analysts, data scientists, and other stakeholders to understand data requirements and deliver data solutions that meet their expectations.
- Ensure data security, privacy, and compliance with data governance standards and best practices.
- Document data architectures, data models, data dictionaries, and data quality metrics.
- Research and evaluate new data technologies and trends and provide recommendations for data engineering improvements.
- Bachelor-s degree in Computer Science, Engineering, Mathematics, or related field.
- At least 6 years of experience as a data engineer or in a similar role.
- Proficient in Python, SQL, and other programming languages for data engineering.
- Experience with data pipeline tools and frameworks such as Spark, Kafka, Airflow, etc.
- Experience with cloud services and platforms such as AWS, GCP, or Azure.
- Experience with data warehouse and data lake technologies such as Redshift, BigQuery, Snowflake, etc.
- Experience with data modeling, data quality, and data governance concepts and practices.
- Strong analytical and problem-solving skills.
- Excellent communication and collaboration skills.
- Ability to work independently and as part of a team.
All product names, logos, and brands are property of their respective owners.