We have partnered with our client in their search for a Senior Data Engineer.
You will play a crucial role in designing, developing, and maintaining cloud-based applications, services and data pipelines using the AWS ecosystem. In addition, you will work on data engineering components to ensure efficient data processing and analysis. Your responsibilities include establishing and optimizing CI/CD pipelines for seamless software delivery. You will work collaboratively with cross-functional teams to meet business needs.
Responsibilities:
Full Stack Development:
- Design, develop, and maintain full-stack applications using Python as the primary programming language.
- Collaborate with front-end developers to create seamless user interfaces.
- Integrate user-facing elements with server-side logic.
- Utilize AWS services, including but not limited to EC2, Lambda, S3, RDS, and more, to build and deploy applications.
- Implement cloud-native solutions, including serverless architecture, containers, and microservices on AWS.
- Develop data pipelines to collect, process, and analyze data from various sources.
- Utilize AWS data services like Glue, Redshift, and Athena to build scalable data solutions.
- Ensure data quality, integrity, and availability for analytical purposes.
- Design and optimize database schemas on AWS RDS and other database solutions.
- Utilize AWS database services like RedShift and DynamoDB as needed.
- Create RESTful APIs and integrate them into web applications.
- Build integrations into 3rd party API services
- Ensure secure and efficient data communication between the front-end and back-end systems.
- Implement security best practices specific to AWS, including IAM, security groups, and encryption.
- Optimize application performance on AWS and troubleshoot AWS-specific issues.
- Establish and maintain CI/CD pipelines for automated testing, integration, and deployment.
- Implement infrastructure as code (IAC) practices to automate infrastructure provisioning and configuration.
- Write unit tests and conduct thorough testing to ensure software reliability and quality.
- Participate in code reviews to maintain codebase quality and consistency.
- Document code and technical designs for reference and knowledge sharing.
- Prepare system documentation for end-users and technical stakeholders.
- Work in an Agile development environment, participating in sprint planning, daily stand-ups, and retrospectives.
- Bachelor's degree in Computer Science, Software Engineering, or a related field (or equivalent experience).
- Proven experience as a Full Stack Developer with a strong focus on Python development.
- Proficiency in Python programming and popular Python web frameworks (e.g., Django, Flask).
- Hands-on experience with AWS cloud services and infrastructure.
- Solid understanding of front-end technologies (HTML, CSS, JavaScript) and modern JavaScript frameworks (e.g., React, Angular, Vue.js).
- Familiarity with database systems (SQL and NoSQL) and ORM frameworks.
- Strong knowledge of RESTful API design and integration.
- Experience with containerization (Docker) and container orchestration (Kubernetes) is a plus.
- Expertise in CI/CD tools and practices, including Jenkins, Travis CI, GitLab CI/CD, or similar.
- Knowledge of DevOps principles and continuous integration/continuous deployment (CI/CD) best practices.
- Data engineering skills, including data pipeline development, data modeling, and ETL processes and tools such as Informatica.
- Experience with AWS data services and analytics tools.
- Excellent problem-solving and debugging skills.
- Strong communication and collaboration skills.
- Ability to work effectively in a team and independently.
Title: Senior Data Engineer
Location: Remote
