Data Transformation Engineer

Summary

Confie is a company focused on building next-generation Enterprise Data Solutions. The Data Transformation Engineer will work under the guidance of the Director of Enterprise Architecture to develop data solutions on Snowflake, including building data pipelines and ensuring data quality for critical business insights.

Responsibilities

• Design and develop data pipelines, ELT workflows to populate the cloud Data Lake and Data Warehouse on Snowflake with transformation tools (e.g., Coalesce, WhereScape, Azure Data Factory) and replication tools (e.g., Fivetran, Airbyte, etc.)
• Translate business requirements into technical specifications; establish and define details, definitions, and requirements of applications, components, and enhancements
• Design and develop robust and scalable data pipelines to support data integrations using Snowflake, Coalesce, Python, Airflow, and Fivetran
• Design and develop Snowflake data objects (tables, views, stored procedures, udf's, etc.)
• Implement ELT (Extract, Load, Transform) processes using Snowflake's features, such as SnowPipe, Streams, and Tasks
• Perform data cleaning, analysis, and integration using Python
• Ability to work with multiple data sources and types (structured/semi-structured/unstructured)
• Design and implement efficient data models and schemas within Snowflake to support reporting, analytics, and business intelligence needs
• Monitor and optimize query performance and resource utilization within Snowflake using query profilling, query optimization techniques, and workload management features
• Identify and resolve performance bottlenecks and optimize data warehouse configurations for maximum efficiency
• Implement processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it
• Monitor data pipelines for timely and accurate completion
• Stay up-to-date with industry trends and advancements in data engineering, continuously improving the team's technical knowledge and skills
• On-call support

Skills

• 4+ years of professional experience in data engineering, designing and implementing data pipelines, and building data infrastructure
• 4+ years of strong experience required in Snowflake data cloud and ETL development, including Snowflake procedures, udf's in Python and SQL, streams, tasks, Snowpipe, and working with semi-structured data
• 3+ years of strong experience with Python programming and extensively used frameworks/packages like Snowpark, pandas, numpy, and requests for Data Analysis and integration
• Solid understanding of data warehousing concepts, dimensional modeling, and data integration techniques
• 2+ years of strong experience with data integration and transformation tools like Coalesce, WhereScape, and Azure Data Factory
• Bachelor's degree in Computer Science, Engineering, or a related field
• Experience with data quality and observability concepts is a plus
• Snowflake Advanced Certification is a plus
• Additional certifications related to data platforms are a plus

Qualifications

Must Haves

• 4+ years of professional experience in data engineering, designing and implementing data pipelines, and building data infrastructure
• 4+ years of strong experience required in Snowflake data cloud and ETL development, including Snowflake procedures, udf's in Python and SQL, streams, tasks, Snowpipe, and working with semi-structured data
• 3+ years of strong experience with Python programming and extensively used frameworks/packages like Snowpark, pandas, numpy, and requests for Data Analysis and integration
• Solid understanding of data warehousing concepts, dimensional modeling, and data integration techniques
• 2+ years of strong experience with data integration and transformation tools like Coalesce, WhereScape, and Azure Data Factory
• Bachelor's degree in Computer Science, Engineering, or a related field

Nice to Haves

• Experience with data quality and observability concepts is a plus
• Snowflake Advanced Certification is a plus
• Additional certifications related to data platforms are a plus

Benefits

• Comprehensive benefits package including medical, dental, vision, and life insurance
• Performance-based bonuses to reward your contributions
• Paid time off to recharge and maintain a healthy work-life balance
• Flexible work options, including remote and hybrid opportunities, if eligible
• Retirement Plan (401k) with company-matched contributions
• Education Advancement, for employees and qualified dependents, via the Confie Enablement Scholarship Fund
• Fitness Reimbursement – up to $15/month for gym memberships
• Inclusive workplace through a strong commitment to Diversity, Equity, and Inclusion
• Employee Assistance Program – confidential support for personal or professional challenges, at no cost
• Extra Perks – optional plans for disability, hospital indemnity, health advocate program, universal life, critical illness, accident insurance, and even pet insurance