Beschreibung

As a Data Platform Engineer, you will design and build scalable data solutions on AWS, ensuring data quality and compliance while enabling teams to access and manage data efficiently. Your work will empower various teams to leverage data for innovative products.

Anforderungen

• •Platform mindset and enabling others
• •Strong programming skills in Python and familiarity with TypeScript
• •Experience building data-intensive applications in cloud-native environments
• •Familiarity with Apache Spark, Iceberg, and Parquet
• •Experience with Infrastructure as Code (CDK or Terraform)
• •Knowledge of monitoring, CI/CD, and DevOps practices
• •Familiarity with dbt or similar frameworks
• •Experience with data cataloging and permission management systems
• •Experience with data privacy engineering and data retention automation
• •Understanding of Iceberg table management and schema evolution
• •Experience with SQL for analytical use cases
• •Excitement about the role and MOIA’s mission

Aufgaben

• •Build and operate core data platform applications like Airflow, QuickSight, Data Export API, and CubeJS
• •Design and implement a multi-tenant data lake on AWS for scalability and performance
• •Develop data ingestion, transformation, and access layers using Apache Spark, Iceberg, and AWS services
• •Optimize storage and processing solutions for high-volume unstructured data
• •Create internal services to make data discoverable and easy to consume
• •Implement data lifecycle and retention policies for compliance and GDPR
• •Build and maintain infrastructure as code using AWS CDK and TypeScript
• •Ensure data quality, security, and observability through automation and monitoring
• •Collaborate with product development, compliance, and security teams for a seamless data experience

Tools & Technologien

Benefits