For our client we are looking for a Data Engineer Lead.
Description:
Global development is changing the system landscape towards utilizing API consumption of data-stored in our client’s AWS hosted data lake. The aim is to migrate the client’s on-prem data to AWS and to build fault-tolerant API’s using AWS infrastructure components as building blocks. Our client strives to build API endpoints based on logical entities using state-of-the-art frameworks. Therefore, designing that makes sense from a domain perspective while combining the right technologies is key for this assignment.
Main work task:
Design and develop data pipelines in AWS using our client’s newly build data lake as primary target. Participate in discussions on technology choices and establishment of best practices in API development for our client.
Desired experience:
Experience in developing secure and scalable API’s using AWS etl tools as well as FastApi or similar python based API development framework is highly valued. It is also of interest with experience using the following:
- AWS ECS
- AWS Cloudformation
- AWS Glue
- AWS Lambda/SAM
- Python >= 3.7
- PostgreSQL/MySQL
- AWS APIGateWay
- CI/CD preferably using Azure DevOps
- Flask /WSGI
- Open API 2.0/3.0