Intellisoft Technologies
Posted 10 days agoVerified live 2d ago
data engineer
Brief overview
Remote
UndergradOr in progress
3+ yrsMinimum
ETL developmentELT developmentMySQLPL/SQLVerticaSnowflakeShell scriptingBash scriptingPythonData modelingSQL performance tuningDBTFivetranApache AirflowGitCI/CD pipelinesAWS cloud platforms
About the company
Intellisoft Technologiesintellisofttech.com
Intellisoft Technologies is an information technology company offering big data and clouding solutions.
Job description
Summary
Intellisoft Technologies is looking for a Data Engineer to design and implement robust ETL/ELT pipelines. The role involves working with various databases and collaborating with stakeholders to meet data needs while optimizing existing processes for performance.
Responsibilities
- Design and implement robust ETL/ELT pipelines to ingest, transform, and load data from various sources
- Work with relational and columnar databases including MySQL, PL/SQL, and Vertica to manage large datasets
- Develop and schedule data workflows using Shell/Bash scripting and Python
- Build and maintain data pipelines on Snowflake using ELT best practices
- Collaborate with data analysts, data scientists, and other stakeholders to understand data needs
- Optimize existing processes for scalability and performance
- Use tools like Airflow, Fivetran, and DBT for orchestration, transformation, and data movement
- Monitor data pipeline performance and resolve issues in a timely manner
- Document processes, data flows, and technical architecture
Skills
- Good experience in ETL/ELT development
- Strong experience with MySQL and PL/SQL
- Hands-on experience with Vertica and Snowflake
- Proficient in Shell/Bash scripting and Python
- Solid understanding of data modeling and SQL performance tuning
- Familiarity with DBT (Data Build Tool) for transformation workflows
- Basic knowledge of Fivetran and Apache Airflow
- Experience working in an Agile environment
- Excellent problem-solving and communication skills
- Experience with Git version control
- Exposure to CI/CD pipelines for data engineering
- Knowledge of AWS cloud platforms
- Familiarity with data governance and security best practices
Qualifications
Must Haves
- Good experience in ETL/ELT development
- Strong experience with MySQL and PL/SQL
- Hands-on experience with Vertica and Snowflake
- Proficient in Shell/Bash scripting and Python
- Solid understanding of data modeling and SQL performance tuning
- Familiarity with DBT (Data Build Tool) for transformation workflows
- Basic knowledge of Fivetran and Apache Airflow
- Experience working in an Agile environment
- Excellent problem-solving and communication skills
- Experience with Git version control
- Exposure to CI/CD pipelines for data engineering
- Knowledge of AWS cloud platforms
- Familiarity with data governance and security best practices
More jobs like this
