Tredence Inc.
Posted 45 days agoVerified live 19h ago
Databricks Developer
Brief overview
Remote
UndergradOr in progress
4+ yrsMinimum
Databricks Lakehouse ArchitectureDatabricks Platform AdministrationDelta LakeUnity CatalogDatabricks WorkflowsDatabricks Performance TuningPySparkPythonSQLETL/ELT PipelinesData ModelingData Quality FrameworksData LineageMetadata ManagementData GovernanceAzure DatabricksAzure Data Lake Storage (ADLS)
About the company
Tredence Inc.tredence.com
Tredence is a global data science solutions provider focused on solving the last mile problem in AI.
Job description
Summary
Tredence Inc. is seeking a Databricks Developer to build and manage modern data platforms using the Databricks Lakehouse architecture. The role focuses on developing scalable data pipelines, optimizing ETL/ELT workflows, implementing data quality and governance controls, and supporting analytics, reporting, and AI/ML initiatives.
Responsibilities
- Design and implement scalable data solutions using Databricks Lakehouse Architecture
- Develop and maintain data pipelines using PySpark, Python, and SQL
- Build and optimize ETL/ELT workflows for batch and near real-time data processing
- Implement Delta Lake features including ACID transactions, time travel, schema evolution, and data versioning
- Design and maintain enterprise data models to support reporting and analytics requirements
- Ensure data quality through validation, monitoring, reconciliation, and governance controls
- Develop and manage data catalogs, metadata management, and data lineage processes
- Collaborate with business stakeholders, architects, and analytics teams to gather and translate requirements into technical solutions
- Optimize Databricks workloads for performance, scalability, and cost efficiency
- Implement security, access controls, and governance best practices within the Databricks ecosystem
- Support troubleshooting, root cause analysis, and production issue resolution
- Contribute to data platform modernization and cloud migration initiatives
Skills
- • Strong experience with Databricks Architecture and platform administration
- • Hands-on expertise in Databricks Lakehouse Architecture
- • Deep understanding of Delta Lake concepts and implementation
- • Experience with Unity Catalog / Data Catalog and metadata management
- • Knowledge of Databricks Workflows, Jobs, Clusters, and Performance Tuning
- • Strong proficiency in PySpark for large-scale data processing
- • Advanced Python programming skills
- • Expert-level SQL development and query optimization
- • Experience in building robust ETL/ELT pipelines
- • Strong understanding of data modeling techniques including:
- • Experience implementing data quality frameworks and validation checks
- • Knowledge of data lineage, metadata management, and governance processes
- • Experience with data reconciliation, profiling, and monitoring tools
- • Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineering, or a related field
- • 4-8 years of experience in Data Engineering and Analytics
- • Minimum 3+ years of hands-on experience with Databricks and PySpark
- • Experience working in Agile development environments
- • Databricks Certified Data Engineer Associate/Professional
- • Microsoft Azure Data Engineer Associate (DP-203)
- • Databricks Lakehouse Fundamentals
- + Data Vault (preferred)
- • Azure Databricks
- • Azure Data Lake Storage (ADLS)
- • Azure Data Factory
- • Azure Synapse Analytics
- • CI/CD pipelines (Azure DevOps, GitHub Actions)
Qualifications
Must Haves
- • Strong experience with Databricks Architecture and platform administration
- • Hands-on expertise in Databricks Lakehouse Architecture
- • Deep understanding of Delta Lake concepts and implementation
- • Experience with Unity Catalog / Data Catalog and metadata management
- • Knowledge of Databricks Workflows, Jobs, Clusters, and Performance Tuning
- • Strong proficiency in PySpark for large-scale data processing
- • Advanced Python programming skills
- • Expert-level SQL development and query optimization
- • Experience in building robust ETL/ELT pipelines
- • Strong understanding of data modeling techniques including:
- • Experience implementing data quality frameworks and validation checks
- • Knowledge of data lineage, metadata management, and governance processes
- • Experience with data reconciliation, profiling, and monitoring tools
- • Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineering, or a related field
- • 4-8 years of experience in Data Engineering and Analytics
- • Minimum 3+ years of hands-on experience with Databricks and PySpark
- • Experience working in Agile development environments
- • Databricks Certified Data Engineer Associate/Professional
- • Microsoft Azure Data Engineer Associate (DP-203)
- • Databricks Lakehouse Fundamentals
Nice to Haves
- + Data Vault (preferred)
- • Azure Databricks
- • Azure Data Lake Storage (ADLS)
- • Azure Data Factory
- • Azure Synapse Analytics
- • CI/CD pipelines (Azure DevOps, GitHub Actions)
More jobs like this
