Talent.com
Emaratech
Senior Data EngineerEmaratech • Dubai, Dubai, AE
Senior Data Engineer

Senior Data Engineer

Emaratech • Dubai, Dubai, AE
منذ أكثر من 30 يومًا
نوع الوظيفة
  • Quick Apply
الوصف الوظيفي

About the Role

We are seeking a highly skilled and experienced Data Lake Cloud Engineer with a proven track record of designing, implementing, and maintaining large-scale cloud-based data lake platforms. This role requires a professional who can take ownership of our current data lake ecosystem, optimize its performance, and drive future enhancements with minimal oversight. The ideal candidate will have at least 5 years of hands-on experience in building enterprise-grade data lakes, strong cloud architecture expertise, and the ability to work with cutting-edge data ingestion, processing, and analytics tools.

Key Responsibilities

  • Take ownership of the existing enterprise data lake platform, ensuring scalability, reliability, and performance.
  • Lead the design, architecture, and implementation of cloud-native data lake solutions and integrations.
  • Manage and optimize data ingestion pipelines on Oracle OCI, using tools such as Apache NiFi, Kafka, Batch Processing of data, Data captures, and or CSV.
  • Design and implement pipelines for network data ingestion and file formats (e.g., Parquet, Avro, OCR, etc.), ensuring efficient storage, processing, and retrieval.
  • Build, configure, and tune query engines such as Trino (Presto), Spark, and Hive for efficient analytics and reporting.
  • Implement and maintain metadata management, data governance, and security frameworks.
  • Monitor and troubleshoot system performance, ensuring SLAs are met for ingestion, processing, and query workloads.
  • Automate platform deployment, monitoring, and maintenance with Infrastructure-as-Code (Terraform, CloudFormation, etc.).
  • Collaborate with data engineers, analysts, and business teams to understand data requirements and deliver solutions that maximize data accessibility and usability.
  • Keep the data platform up to date with the latest open-source and cloud-agnostic technologies, implementing upgrades and enhancements where needed.

Requirements

5+ years of proven, hands-on experience implementing and managing large-scale data lakes in the cloud (OCI). Strong expertise in:

  • Data ingestion & orchestration: Apache NiFi, Apache Kafka, CSV, and others
  • Data processing frameworks: Apache Spark, PySpark, Trino (Presto), Hive, Flink.
  • Storage & lakehouse architectures: Delta Lake, Apache Hudi, Iceberg, and cloud-native object storage (S3).
  • Query & analytics tools: Trino/Presto, SparkSQL, Metabase, or Apache Superset.
  • Experience with data lake file formats such as Apache Parquet, Avro, ORC, CSV, etc. including ingestion, parsing, and analytics within a data lake.
  • Solid understanding of data governance, lineage, cataloging, and security frameworks (Apache Atlas).
  • Experience with CI/CD and IaC (ArgoCD, Terraform, Ansible) for automated deployments.
  • Hands-on experience with cloud security best practices, including IAM, encryption, and network security.
  • Strong proficiency in Python or Java for data engineering and automation tasks.
  • Proven ability to work independently, quickly understand existing environments, and deliver results without extensive training.

Preferred Skills

  • Exposure to machine learning workflows integrated with data lakes.
  • Experience with real-time streaming data pipelines.
  • Familiarity with containerization and orchestration (Docker, Kubernetes).
  • Knowledge of cost optimization strategies in cloud-based data platforms.
إنشاء تنبيه وظيفي لهذا البحث

Senior Data Engineer • Dubai, Dubai, AE

الوظائف المماثلة

Senior Data Engineer

VirtusaDubai, Dubai, UAE

Job DescriptionSenior Data Engineer with strong expertise in Informatica DEI EDC CDC API development and cloud platforms to support scalable secure data integration in a telecom environmentKey Resp... أظهر المزيد

 • عَرْضٌ مُرَوَّجٌ له

Senior Data Engineer RibbitZ

TarakiDubai, Dubai, UAE

Senior Data Engineer (Cloud Analytics).We are seeking a highly skilled.The ideal candidate will have hands-on experience in building scalable data pipelines implementing lakehouse architectures and... أظهر المزيد

 • عَرْضٌ مُرَوَّجٌ له

Senior Data Engineer Pyspark with Data modelling

VirtusaDubai, Dubai, UAE

We are seeking a highly skilled Data Engineer with deep expertise in.Cloudera Data Platform (CDP) to join our data engineering team.As a Data Engineer you will be responsible for designing developi... أظهر المزيد

 • عَرْضٌ مُرَوَّجٌ له

Big Data Engineer

VirtusaDubai, Dubai, UAE

Role: Senior Lead Software Engineer.Strong functional knowledge/Experience of the domain Banking.Develop and optimize data pipelines using Spark Hive and Python on Cloudera.Develop real time data w... أظهر المزيد

 • عَرْضٌ مُرَوَّجٌ له

Sr Data Engineer, 3866

MindtelDubai, Dubai, UAE

Design develop and maintain robust ETL pipelines using Data Bricks and Azure Data Factory.Transform raw data into actionable insights to support business decision-making.Manage and optimize data st... أظهر المزيد

 • عَرْضٌ مُرَوَّجٌ له

Data Engineer

Entertainer FzDubai, Dubai, UAE

The ENTERTAINER is a leading digital company dedicated to adding value for consumers by bringing them the best incentive offers globally.We are a 100 digital datadriven tech company providing first... أظهر المزيد

 • عَرْضٌ مُرَوَّجٌ له

Data Engineer - Pyspark

VirtusaDubai, Dubai, UAE

About the RoleWe are seeking a highly skilled Data Engineer with deep expertise in PySpark and the Cloudera Data Platform (CDP) to join our data engineering team.As a Data Engineer you will be resp... أظهر المزيد

 • عَرْضٌ مُرَوَّجٌ له

Senior Market Data Engineer

BHFTDubai, Dubai, UAE

Historical Data Capture and Storage:.Design develop and maintain systems for the acquisition storage and retrieval of historical market data from multiple financial exchanges brokers and market dat... أظهر المزيد

 • عَرْضٌ مُرَوَّجٌ له

Data Engineer

Talent Bridge HR Consultancy DubaiDubai, Dubai, UAE

Build robust observable data pipelines that power research and production AI.Success means high pipeline reliability (on-time SLAs) strong data quality (validation & lineage) and enabling fast expe... أظهر المزيد

 • عَرْضٌ مُرَوَّجٌ له

Data Engineer

ArtefactDubai, Dubai, UAE

Artefact is a new generation of data service providers specialising in data consulting and datadriven digital marketing.It is dedicated to transforming data into business impact across the entire v... أظهر المزيد

 • عَرْضٌ مُرَوَّجٌ له

Senior Data Engineer (Python)

Parser LimitedDubai, Dubai, UAE

Technology alone does not create impact - the right teams do.Founded in 2018 Parser is a boutique technology services and consulting firm helping global organizations solve complex business challen... أظهر المزيد

 • عَرْضٌ مُرَوَّجٌ له

Senior Analytics Engineer

Delivery HeroDubai, Dubai, UAE

As the leading delivery company in the region we have a great responsibility and opportunity to impact the lives of millions of customers restaurant partners and riders.To realize our potential we ... أظهر المزيد

 • عَرْضٌ مُرَوَّجٌ له

Data Engineer PySpark

VirtusaDubai, Dubai, UAE

Pyspark JD:ResponsibilitiesData Pipeline Development: Design develop and maintain highly scalable and optimized ETL pipelines using PySpark on the Cloudera Data Platform ensuring data integrity and... أظهر المزيد

 • عَرْضٌ مُرَوَّجٌ له

Senior Data Engineer (MLAI)

Sana CommerceDubai, Dubai, UAE

Collaborating with Data Scientists and ML Engineers to understand.Leveraging distributed processing frameworks (e.Spark Databricks Azure Synapse) for.Data Engineer working with Azure and Databricks... أظهر المزيد

 • عَرْضٌ مُرَوَّجٌ له

Senior Data Engineer

BHFTDubai, Dubai, UAE

Architect batchstream pipelines (Airflow Kafka dbt) for diverse structured and unstructured marked data.Provide reusable SDKs in Python and Go for internal data producers.Implement and tune S3 colu... أظهر المزيد

 • عَرْضٌ مُرَوَّجٌ له

Senior Data Engineer

Sana CommerceDubai, Dubai, UAE

Data Pipeline Development & Infrastructure.Design build and maintain .Data Solutions & AI-Driven Applications.ML recommenders search functionality and AI-enhanced features.Commerce-rel... أظهر المزيد

 • عَرْضٌ مُرَوَّجٌ له

Senior AWS Data Engineer Data Platform Engineer

NorthBay SolutionsDubai, Dubai, UAE

Job Title: Senior AWS Data Engineer / Data Platform Engineer.We are seeking a highly experienced.The ideal candidate will have deep hands-on experience with AWS data services batch and streaming da... أظهر المزيد

 • عَرْضٌ مُرَوَّجٌ له

Senior Data Science Engineer

Antons Recruitment AgencyDubai, Dubai, UAE

The ideal candidate will have hands-on experience with.MLFlow Databricks and Azure ML.This role will be central to embedding ML models into enterprise data workflows and ensuring their ongoing perf... أظهر المزيد

 • عَرْضٌ مُرَوَّجٌ له

Data Engineer

SentraAIDubai, Dubai, UAE

At SentraAI Data Engineers work on.The pipelines you build will be used monitored and relied upon.When they break you are expected to help fix them.If you are not ready to take responsibility for p... أظهر المزيد

 • عَرْضٌ مُرَوَّجٌ له

Data Engineer (PySpark)

Black Pearl ConsultDubai, Dubai, UAE

We are seeking an experienced Data Engineer (PySpark) to design build optimize and maintain scalable data pipelines for production environments.The role requires strong hands-on experience in big d... أظهر المزيد