Databricks Engineer

Summary

Blu Omega is seeking a Databricks Engineer to support a federal program focused on data analytics and mission-critical government operations. The role designs, builds, operates, and optimizes scalable Databricks data solutions, including data pipelines, cloud-based platforms, CI/CD integrations, data quality frameworks, and production workloads.

Responsibilities

• Design, develop, and maintain scalable Databricks data pipelines using PySpark, Spark SQL, Delta Lake, and Unity Catalog to deliver high-quality, mission-critical data assets
• Build and maintain automated ingestion, transformation, and processing workflows supporting analytics, reporting, and operational decision-making
• Operate and optimize Databricks clusters, jobs, and workflows ensuring reliability, performance, and cost efficiency
• Integrate Databricks with enterprise CI/CD pipelines and SDLC tooling for automated deployments, source control, and testing
• Implement data quality, validation, and observability frameworks to enhance data accuracy and trustworthiness
• Monitor and troubleshoot data pipelines, platform performance, and production workloads to identify and resolve issues proactively
• Collaborate with stakeholders and cross-functional teams to translate operational requirements into technical solutions
• Develop and maintain technical documentation related to data pipelines, architecture, and operational processes

Skills

• 3+ years of experience in data engineering, analytics, or related technical roles
• 2+ years of hands-on experience with the Databricks platform
• Strong experience developing data solutions using PySpark and Spark SQL
• Experience working with Databricks clusters, jobs, workflows, Delta Lake, and Unity Catalog in production environments
• Experience designing and maintaining automated data pipelines and transformation workflows
• Experience integrating Databricks with CI/CD pipelines and enterprise SDLC tooling
• Working knowledge of Python for data engineering, automation, and ML workflows
• Knowledge of cloud platforms such as AWS, Microsoft Azure, or GCP and associated data services
• Understanding of data quality, validation, monitoring, and production support practices
• Strong analytical and troubleshooting skills
• Effective collaboration skills with technical and non-technical teams
• Ability to obtain and maintain a Public Trust or Suitability/Fitness determination
• Bachelor's degree in computer science, data science, engineering, or related field
• Experience supporting federal government or regulated industries
• Experience developing or operationalizing machine learning solutions within Databricks
• Experience with MLflow for machine learning lifecycle management
• Knowledge of streaming frameworks, real-time data processing, and observability tooling
• Experience implementing data governance, lineage, and access control practices
• Databricks certification or cloud platform certification from AWS, Azure, or GCP

Qualifications

Must Haves

• 3+ years of experience in data engineering, analytics, or related technical roles
• 2+ years of hands-on experience with the Databricks platform
• Strong experience developing data solutions using PySpark and Spark SQL
• Experience working with Databricks clusters, jobs, workflows, Delta Lake, and Unity Catalog in production environments
• Experience designing and maintaining automated data pipelines and transformation workflows
• Experience integrating Databricks with CI/CD pipelines and enterprise SDLC tooling
• Working knowledge of Python for data engineering, automation, and ML workflows
• Knowledge of cloud platforms such as AWS, Microsoft Azure, or GCP and associated data services
• Understanding of data quality, validation, monitoring, and production support practices
• Strong analytical and troubleshooting skills
• Effective collaboration skills with technical and non-technical teams
• Ability to obtain and maintain a Public Trust or Suitability/Fitness determination
• Bachelor's degree in computer science, data science, engineering, or related field

Nice to Haves

• Experience supporting federal government or regulated industries
• Experience developing or operationalizing machine learning solutions within Databricks
• Experience with MLflow for machine learning lifecycle management
• Knowledge of streaming frameworks, real-time data processing, and observability tooling
• Experience implementing data governance, lineage, and access control practices
• Databricks certification or cloud platform certification from AWS, Azure, or GCP

Benefits

• Medical, Dental, and Vision coverage through national providers
• 401(k) with company match (eligible after 6 months; vesting applies)
• Company-paid Life and AD&D insurance with additional voluntary options
• Short-term and long-term disability options
• Employee Assistance Program (EAP) with 24/7 confidential support and mental health resources
• Telehealth and virtual care options
• Pet insurance, legal services, and identity theft protection options
• Paid Time Off (PTO) for eligible employees
• Paid federal holidays for salaried employees
• Access to wellness programs, discounts, and lifestyle benefits
• Benefits availability may vary based on role and employment status