We are seeking an experienced Data Engineer (PySpark) to design, build, optimize, and maintain scalable data pipelines for production environments. The role requires strong hands-on experience in big data processing, pipeline optimization, and deployment using modern data engineering tools and frameworks.
Key Responsibilities
Design, develop, and maintain robust, scalable data pipelines using Python and PySpark
Perform data ingestion, transformation, cleansing, and validation across structured and unstructured datasets
Conduct Exploratory Data Analysis (EDA) to identify data patterns, anomalies, and quality issues
Apply data imputation techniques, data linking, and cleansing to ensure high data quality
Implement feature engineering pipelines to support analytics and downstream use cases
Optimize Spark jobs for performance, scalability, and cost efficiency
Deploy and tune production-grade data pipelines, ensuring reliability and performance
Automate workflows using Apache Airflow and/or Jenkins
Collaborate with cross-functional teams to integrate data solutions into production systems
Write and maintain unit tests to ensure code quality and reliability
Manage source code, CI/CD, and deployments using Git, GitHub, and GitHub Actions
Data Engineer (PySpark) • Abu Dhabi, United Arab Emirates
As a Business Data Engineer you will provide technical support for Power BI and SharePoint within Boskalis Nederland.You will work closely with our Netherlands-based team and support us in developi... أظهر المزيد
Location: Abu Dhabi United Arab Emirates.Thales people architect solutions that enable two-thirds of planes to take off and land safely.We create in-flight entertainment systems that engross 50 mil... أظهر المزيد
We are an emerging AI-native product-driven agile start-up under Abu Dhabi government AND we are seeking a motivated and technically versatile.You will play a key role in delivering data platforms ... أظهر المزيد
You will design and implement large-scale data solutions to support AI/ML initiatives and public sector digital transformation.Build robust data pipelines using.Prepare and process datasets for mac... أظهر المزيد
Our clients Research & Development team brings together specialists across disciplines to push the frontier of quantitative investing.They blend deep expertise with novel ideas to design and de... أظهر المزيد
Le Chene is supporting the delivery of applied.Collaborate with data science and platform teams.LangChain RAG PyTorch or TensorFlow.NET,Health Education,Fashion Designing,Fiber,Investigation. أظهر المزيد
The System Engineer will be responsible for managing integrating and optimizing enterprise applications including Hexagon EAM Microsoft Dynamics NAV & Business Central and data analytics platfo... أظهر المزيد
As a Data Analytics Engineer you will be part of the IT Data Services team working closely with business users across various departments to help them gain actionable insights from their ... أظهر المزيد
This role owns the foundations that everything else depends on.As a Data Platform Engineer at SentraAI you will build and operate the infrastructure tooling and services that underpin.AI systems sp... أظهر المزيد
You will work closely with product managers frontend engineers and other stakeholders to deliver reliable high-performance services that power our applications.Design develop and maintain backend s... أظهر المزيد
This is a production engineering role.As a Senior Data Engineer at SentraAI you will build migrate and operate data pipelines at enterprise scale involving.TB of data and 1000s of ETL workloads ins... أظهر المزيد
Design construct and maintain data pipelines for data ingestion transformation and storage.Ensure data integrity quality and security throughout the data lifecycle.Optimize data processing workflow... أظهر المزيد
We are looking for a highly skilled.The ideal candidate will have a solid background in developing and implementing AI solutions that integrate seamlessly with CRM platforms.Design develop and impl... أظهر المزيد
Imagine automating the real-time sensor data from our fleet and supporting on NOx reduction for sustainability this role you will have the chance to contribute to our operations by utilizing your ... أظهر المزيد
You will design and implement large-scale data solutions to support AI/ML initiatives and public sector digital transformation.Build robust data pipelines using.Prepare and process datasets for mac... أظهر المزيد
Must have Skills : Java (Expert) Python (Expert) SQL (Strong).Good To Have Skills : Kubernetes (Capable).Job Description : Responsibilities: Design build and maintain scalable server-side applicati... أظهر المزيد
Data Integration Data Warehouse & Data Management.Support setup monitoring and maintenance of data pipelines from SCADA CMMS CAFM and other systems into the centralized data warehouse.Maintain ... أظهر المزيد
This role supports the delivery of enterprise data platforms and analytics pipelines within a cloud-based environment for a government enterprise.Support batch and streaming data processing.Collabo... أظهر المزيد
Job Description: Data Warehouse and Machine Learning Engineer We are seeking a highly skilled and experienced Data Warehouse and Machine Learning Engineer to join our this role you will be respons... أظهر المزيد
Abu Dhabi (Offshore visits as per job requirements).Round trip - Economy class provided.Single-status accommodation will be provided during the period of single status.Family status benefits will b... أظهر المزيد
