Data Engineer

Summary

Abacus is seeking a Data Engineer to develop, maintain, and support data pipelines within a modern Azure-based data platform. The role focuses on Python and SQL Server ETL development, data integration, pipeline monitoring, troubleshooting, documentation, and collaboration in regulated environments.

Responsibilities

• Develop and maintain ETL pipelines using Python and SQL Server for data ingestion, transformation, and integration from various data sources, including structured files and RESTful APIs
• Write, test, and debug Python code for data processing, automation, and basic integrations following established standards and best practices
• Create and maintain T-SQL queries, views, and stored procedures to support business logic and reporting requirements
• Assist in building and operating data workflows using Azure Data Factory, Azure SQL, and Azure Blob Storage
• Support the monitoring of data pipelines and help troubleshoot data quality issues, pipeline failures, and performance problems
• Follow defined data quality, security, and compliance procedures in regulated environments such as healthcare and financial services
• Collaborate with data analysts, software engineers, and DevOps teams to understand data requirements and upstream systems
• Participate in code reviews, implement feedback, and continuously improve coding and engineering practices
• Create and maintain clear documentation for data pipelines, logic, and operational processes

Skills

• 1–4 years of professional experience in Data Engineering, Software Engineering, or a related technical role
• Strong hands-on proficiency in Python, including writing functions, handling errors, and debugging code
• Experience working with relational databases, preferably SQL Server
• Strong working knowledge of SQL, including joins, aggregations, subqueries, and basic performance considerations
• Exposure to integrating or consuming data from RESTful APIs or external data sources
• Familiarity with Git or similar version control systems
• Strong analytical and problem-solving skills with the ability to learn quickly
• Exposure to Azure data services such as Azure Data Factory, Azure SQL, or Blob Storage
• Basic understanding of ETL concepts, data validation, and pipeline monitoring
• Familiarity with data modeling fundamentals, including tables, keys, and relationships
• Awareness of data security, privacy, and compliance standards such as HIPAA and SOC2
• Ability to design and support scalable ETL pipelines using Python and SQL Server for data ingestion, transformation, and integration from diverse sources
• Ability to develop and optimize T-SQL stored procedures to support business logic and reporting needs
• Ability to support secure and efficient data workflows using Azure Data Factory, Azure SQL, and Azure Functions
• Ability to help ensure data quality, lineage, and compliance requirements are maintained
• Ability to collaborate with software engineering, data analytics, security, and DevOps teams
• Ability to monitor and troubleshoot pipeline failures and data discrepancies
• Ability to participate in code reviews and continuously improve engineering practices
• Ability to work effectively in a collaborative, fast-paced environment
• Good written and verbal communication skills
• Bachelor's Degree
• Experience with additional Azure services such as Azure Synapse, Azure DevOps, and Azure Functions
• Understanding of data modeling and warehousing concepts
• Exposure to cybersecurity data, SIEM tools, or SOC operations
• Knowledge of .NET Framework and C#-based APIs, particularly in data consumption contexts
• Background in MSP/MSSP environments or consulting
• Familiarity with Power BI or other data visualization tools
• Experience with Power BI or other data visualization tools
• Basic understanding of application systems or APIs built using .NET or similar frameworks

Qualifications

Must Haves

• 1–4 years of professional experience in Data Engineering, Software Engineering, or a related technical role
• Strong hands-on proficiency in Python, including writing functions, handling errors, and debugging code
• Experience working with relational databases, preferably SQL Server
• Strong working knowledge of SQL, including joins, aggregations, subqueries, and basic performance considerations
• Exposure to integrating or consuming data from RESTful APIs or external data sources
• Familiarity with Git or similar version control systems
• Strong analytical and problem-solving skills with the ability to learn quickly
• Exposure to Azure data services such as Azure Data Factory, Azure SQL, or Blob Storage
• Basic understanding of ETL concepts, data validation, and pipeline monitoring
• Familiarity with data modeling fundamentals, including tables, keys, and relationships
• Awareness of data security, privacy, and compliance standards such as HIPAA and SOC2
• Ability to design and support scalable ETL pipelines using Python and SQL Server for data ingestion, transformation, and integration from diverse sources
• Ability to develop and optimize T-SQL stored procedures to support business logic and reporting needs
• Ability to support secure and efficient data workflows using Azure Data Factory, Azure SQL, and Azure Functions
• Ability to help ensure data quality, lineage, and compliance requirements are maintained
• Ability to collaborate with software engineering, data analytics, security, and DevOps teams
• Ability to monitor and troubleshoot pipeline failures and data discrepancies
• Ability to participate in code reviews and continuously improve engineering practices
• Ability to work effectively in a collaborative, fast-paced environment
• Good written and verbal communication skills

Nice to Haves

• Bachelor's Degree
• Experience with additional Azure services such as Azure Synapse, Azure DevOps, and Azure Functions
• Understanding of data modeling and warehousing concepts
• Exposure to cybersecurity data, SIEM tools, or SOC operations
• Knowledge of .NET Framework and C#-based APIs, particularly in data consumption contexts
• Background in MSP/MSSP environments or consulting
• Familiarity with Power BI or other data visualization tools
• Experience with Power BI or other data visualization tools
• Basic understanding of application systems or APIs built using .NET or similar frameworks

Benefits

• Remote in US
• Opportunities for hands-on learning and growth under the guidance of senior data engineers