Data Engineer (Contingent)

Summary

Wilcore Technologies, Inc. is hiring a Data Engineer to support an enterprise cloud-based data and analytics platform for a federal health agency. The role focuses on scalable data ingestion, shared-service data engineering, data quality, observability, governance, federated query tooling, and solution consulting for analytics, reporting, and machine learning.

Responsibilities

• Ingest New and Migrated Sources: Ingest new and migrated data sources into the platform using fit-for-purpose patterns, selected per source characteristics, operational constraints, and approved architecture decisions
• Provide Shared-Service Data Engineering: Transform ingested data into consumption-ready models, including semantic views, conformed dimensions, and performance-oriented partitioning for analytics, reporting, and machine learning
• Operate Delta Lake Patterns and Dedicated Pool: Operate and improve the platform's delta lake patterns and dedicated pool usage, ensuring parity of data and models migrated from other platforms before customer release
• Implement Definition-of-Done Quality Checks: Implement quality checks for each domain: equivalence and parity tests against systems of record and, where applicable, prior platforms; freshness SLAs and anomaly detection tied to pipeline events, publishing pass and fail results and residual risk notes in release artifacts
• Instrument Observability: Wire pipelines to platform monitoring and defect tracking, with playbooks for alerting, triage, rollback, and hot-fix
• Document Ingestion and Modeling Choices: Document pattern, orchestration, lineage, and tuning guidance, and keep links to approvals and decisions current through decision tickets and Architecture Decision Records
• Configure Federated Query Tooling: Configure and implement federated query tooling in alignment with the policies and targets set by the data governance workstream
• Provide Solution Consulting and Provisioning: Assess customer data, analytics, and operational requirements and recommend fit-for-purpose data architecture and ETL solutions. Guide the provisioning of required data infrastructure, tooling, and pipelines aligned with approved enterprise standards
• Define and Monitor KPIs: Define and monitor performance and cost KPIs such as timed loads, parallelism, data quality, and storage and egress for chosen patterns, and publish tuning recommendations for domain teams, documented in a Data Quality and Ingestion Dashboard

Skills

• Experience and expertise with Azure Data Factory, Databricks, and Kafka is required for this role
• Experience with Unity Catalog for unified governance, security, lineage, and data and AI asset management
• Familiarity with Collibra and Immuta, or comparable enterprise data catalog and policy-based access control tooling
• Expertise in the disciplines and technical areas related to the data ingestion, engineering, and quality tasks described above
• Applicants must be authorized to work in the United States
• Must be able to obtain and maintain a federal Public Trust (Tier 2 / Moderate Risk) background investigation
• Must reside and perform all work within the continental United States
• Production system monitoring and support coverage runs 7:00 a.m. to 8:00 p.m. Eastern Time, with participation in a 24/7 on-call escalation process for urgent issues
• Direct experience working in a federal agency environment

Qualifications

Must Haves

• Experience and expertise with Azure Data Factory, Databricks, and Kafka is required for this role
• Experience with Unity Catalog for unified governance, security, lineage, and data and AI asset management
• Familiarity with Collibra and Immuta, or comparable enterprise data catalog and policy-based access control tooling
• Expertise in the disciplines and technical areas related to the data ingestion, engineering, and quality tasks described above
• Applicants must be authorized to work in the United States
• Must be able to obtain and maintain a federal Public Trust (Tier 2 / Moderate Risk) background investigation
• Must reside and perform all work within the continental United States
• Production system monitoring and support coverage runs 7:00 a.m. to 8:00 p.m. Eastern Time, with participation in a 24/7 on-call escalation process for urgent issues

Nice to Haves

• Direct experience working in a federal agency environment