Description

You will enhance and maintain the data science infrastructure, monitor pipeline performance, and ensure system reliability while collaborating with various teams and mentoring colleagues.

Requirements

• •Master's degree in computer science
• •Willingness to work in a team
• •Ability to work in a structured manner
• •Strong software development background
• •Five or more years of experience
• •Extensive experience with Hadoop clusters
• •Extensive knowledge in information systems theory
• •Ability to write efficient Python code
• •Experience in software development practices
• •Good Linux knowledge
• •Fluent English, good German is a plus

Tasks

• •Collaborate with data engineering and platform teams to enhance infrastructure and pipelines.
• •Monitor and operate key data science pipelines, including on-call duties.
• •Analyze runtime behavior, including resource usage and error rates.
• •Maintain operating systems on the Cloudera cluster.
• •Monitor cluster health and respond to errors and outages.
• •Install and maintain software and development environments for data science.
• •Support network optimization between systems used by data science.
• •Coordinate with suppliers and datacenter services to resolve hardware issues.
• •Implement and enforce a logging and monitoring strategy for data science projects.
• •Define and enforce HDFS usage guidelines and monitor retention periods.
• •Manage Impala databases and tables.
• •Mentor team members and engage in code reviews for continuous improvement.

Tools & Technologies

Benefits