Data Engineer

Summary

LDSS Corp is a Woman-Owned Small Business based in Dayton, Ohio, committed to delivering high-impact solutions to federal agencies. They are seeking a skilled Data Engineer to build and optimize ETL data pipelines, develop scalable data models, and collaborate with cross-functional teams to support the Air Force’s Maintenance, Repair, and Overhaul program.

Responsibilities

• Build and optimize ETL data pipelines using Databricks and Apache Spark
• Develop scalable data models leveraging Spark Data Frames and Spark SQL for efficient querying, aggregation, and integration against numerous diverse data sources
• Utilize Python and Regular Expressions for advanced text parsing, pattern matching, and data cleansing to prepare structured datasets for downstream ingestion
• Use APIs for seamless data exchange and automation across systems
• Manage SharePoint data (lists, libraries, files, metadata) programmatically using Python and Microsoft Graph API
• Leverage Microsoft 365 tools for documentation and collaboration
• Develop and maintain Amazon Web Services (AWS) including S3
• Fully utilize the Microsoft 365 environment for cross-functional data sharing and productivity
• Use Jira to track and manage issues, tasks, and project workflows
• Collaborate with cross-functional teams and technical stakeholders to define data transformations of legacy systems to modern schemas
• Leverage SME collaboration to interpret business rules and build business logic into data extracts and transformation processes
• Build and maintain strong working relationships with SME’s, helping to identify requirements for data transformations based upon iterative product feedback

Skills

• Bachelor's degree in computer science, engineering, or a related field
• Experience in Databricks
• Experience in Apache Spark
• Experience in Python
• Experience in AWS
• Experience in Microsoft 365
• Experience in SharePoint
• Experience in developing and consuming APIs for data exchange between different systems
• Experience in Jira or other agile workflow management systems for issue and project tracking
• Proven ability to build trust and maintain strong partnerships with customers and stakeholders
• Takes initiative and drives projects forward with minimal supervision, anticipating challenges before they arise
• Meticulous in managing complex tasks and deadlines, ensuring accuracy and consistency across deliverables
• Thrives in cross-functional environments, fostering open communication and teamwork across diverse groups
• Rapidly assimilates new technologies and processes, adapting to evolving requirements in dynamic settings
• Ability to work collaboratively with a diverse team of contractors and government personnel
• Skilled in building strong customer relationships and engaging with Subject Matter Experts (SMEs) to derive business logic for data transformation methods and rules
• AWS Certified Solutions Architect
• Databricks Certified Data Engineer
• U.S federal government security clearance

Qualifications

Must Haves

• Bachelor's degree in computer science, engineering, or a related field
• Experience in Databricks
• Experience in Apache Spark
• Experience in Python
• Experience in AWS
• Experience in Microsoft 365
• Experience in SharePoint
• Experience in developing and consuming APIs for data exchange between different systems
• Experience in Jira or other agile workflow management systems for issue and project tracking
• Proven ability to build trust and maintain strong partnerships with customers and stakeholders
• Takes initiative and drives projects forward with minimal supervision, anticipating challenges before they arise
• Meticulous in managing complex tasks and deadlines, ensuring accuracy and consistency across deliverables
• Thrives in cross-functional environments, fostering open communication and teamwork across diverse groups
• Rapidly assimilates new technologies and processes, adapting to evolving requirements in dynamic settings
• Ability to work collaboratively with a diverse team of contractors and government personnel
• Skilled in building strong customer relationships and engaging with Subject Matter Experts (SMEs) to derive business logic for data transformation methods and rules

Nice to Haves

• AWS Certified Solutions Architect
• Databricks Certified Data Engineer
• U.S federal government security clearance

Benefits

• 401(k)
• 401(k) matching
• Dental insurance
• Flexible spending account
• Health insurance
• Health savings account
• Life insurance
• Paid time off
• Referral program
• Tuition reimbursement
• Vision insurance