Beschreibung

As a Data Platform Engineer, you will design and manage a cutting-edge data lake while developing scalable applications and ensuring data quality. This role involves collaborating with various teams to create a seamless data experience across MOIA.

Anforderungen

• •Platform mindset and enabling others
• •Strong programming skills in Python and familiarity with TypeScript
• •Experience building and operating data-intensive applications in cloud-native environments
• •Familiarity with Apache Spark, Iceberg, and data storage formats
• •Experience with Infrastructure as Code (CDK or Terraform)
• •Knowledge of monitoring, CI/CD, and DevOps practices
• •Familiarity with dbt or similar frameworks for data transformation
• •Experience with data cataloging, access control, or permission management systems
• •Experience with data privacy engineering, data retention automation, or data erasure workflows
• •Understanding of Iceberg table management and schema evolution best practices
• •Experience with SQL for analytical use cases

Aufgaben

• •Build and operate core data platform applications like Airflow and QuickSight
• •Design and manage a multi-tenant data lake on AWS
• •Develop data ingestion and transformation layers using Apache Spark and AWS services
• •Optimize storage solutions for high-volume unstructured data
• •Create internal services for data discoverability and governance
• •Implement data lifecycle and retention policies for compliance
• •Maintain infrastructure as code using AWS CDK and TypeScript
• •Ensure data quality and security through automation and monitoring
• •Collaborate with product development, compliance, and security teams

Tools & Technologien

Benefits