The Data Engineer will be responsible for the following activities:
- Work closely with data architects and other stakeholders to design scalable and robust data architectures that meet the organization's requirements
- Develop and maintain data pipelines, which involve the extraction of data from various sources, data transformation to ensure quality and consistency, and loading the processed data into data warehouses or other storage systems
- Responsible for managing data warehouses and data lakes, ensuring their performance, scalability, and security
- Integrate data from different sources, such as databases, APIs, and external systems, to create unified and comprehensive datasets.
- Perform data transformations and implement Extract, Transform, Load (ETL) processes to convert raw data into formats suitable for analysis and reporting
- Collaborate with data scientists, analysts, and other stakeholders to establish data quality standards and implement data governance practices
- Optimise data processing and storage systems for performance and scalability
- Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data requirements and deliver solutions
