Data Engineer

Summary

1872 Consulting is seeking a Data Engineer to develop data pipelines that integrate various data sources and ensure data quality. The role involves designing, developing, and launching data pipelines, collaborating with teams, and supporting the development process.

Responsibilities

• Design, develop, and launch efficient and reliable data pipelines to move and transform data to enable downstream consumption and insights
• Refine code to ensure performance and reliability of data pipelines and data processing
• Write unit tests, and automate functional and integration tests
• Collaborate with other data engineers, analysts, product managers, and customers to implement technical solutions to business problems
• Support the development process by participating in regular Agile/Scrum activities, code reviews, and architectural reviews
• Implement DevOps patterns in conjunction with our DevOps/SREs
• Diagnose and solve issues in existing data pipelines, and use learnings to build their successors

Skills

• 2+ years of experience with Data Engineering in cloud-based environments
• Demonstrated hands-on Engineering experience delivering large scale data, analytics, reporting, or machine learning products/solutions
• Strong SQL experience
• Python experience
• Experience with the issues of performance, availability, scalability, reliability, and maintainability of data pipelines
• Experience with Spark and PySpark
• Data modeling experience
• DevOps experience

Qualifications

Must Haves

• 2+ years of experience with Data Engineering in cloud-based environments
• Demonstrated hands-on Engineering experience delivering large scale data, analytics, reporting, or machine learning products/solutions
• Strong SQL experience
• Python experience
• Experience with the issues of performance, availability, scalability, reliability, and maintainability of data pipelines

Nice to Haves

• Experience with Spark and PySpark
• Data modeling experience
• DevOps experience