Postdoctoral Researcher / Postdoctoral Researcheress

Postdoctoral Researcher - OCCL

100%, Singapore, fixed-term

The Singapore-ETH Centre, established in 2010 by ETH Zurich - The Swiss Federal Institute of Technology and Singapore’s National Research Foundation (NRF), is a beacon of innovation. As ETH Zurich's sole research centre outside of Switzerland, it enhances the institution's capabilities to develop sustainable solutions addressing global challenges in Switzerland, Singapore, and surrounding regions.

Located in a rapidly urbanising region, the Singapore-ETH Centre is dedicated to providing practical solutions to critical urban sustainability, resilience, and health issues through its initiatives: Future Cities Lab Global (FCL Global) and Future Health Technologies (FHT).

This centre acts as an intellectual hub for research, fostering collaboration among principal investigators and researchers from diverse disciplines. To promote the exchange of ideas and expertise, our researchers actively collaborate with universities, research institutes, industry, and government agencies, translating knowledge into actionable solutions for real-world challenges.

Project Background

The current era of artificial intelligence is primarily propelled by advancements in computational power and infrastructure. As models grow to unprecedented sizes, their capabilities are enhanced through strategies such as reinforcement learning (RL) and innovative frameworks like the "graph of thoughts." We are firmly in the "Age of Computation," where breakthroughs in AI are directly correlated with the ability to harness and optimize substantial computational resources. However, the rapid increase in computational demands presents a significant challenge: energy limitations. Globally, new power plants are being built to accommodate AI workloads, highlighting the pressing need for efficiency to ensure these advancements benefit humanity sustainably. This is especially crucial in Singapore, where resources may be limited, potentially impeding the progress of researchers in this vital field.

Our collaborative efforts with the A*STAR Institute of High Performance Computing (IHPC) and other partners in Singapore, such as NTU, aim to be pivotal in this area, driving the development and implementation of innovative AI methods and saving millions of dollars in compute time while enhancing scientific productivity. To increase the efficacy of AI for science models, it is imperative to integrate simulations that encode prior scientific knowledge within these AI frameworks. This integration seeks to develop new models infused with robust prior knowledge, defined by fundamental equations and conservation laws. Our approach involves close collaboration with experts from A*STAR/IHPC, who possess extensive experience in high-performance computing and AI-driven scientific research. Together, we aim to push the boundaries of what AI can achieve in scientific domains, ensuring that simulations and AI methodologies are combined to produce highly accurate and reliable models.

Job Description

The Postdoctoral Researcher will co-lead the development of a novel open-source accelerator communication library (OCCL). This communication library will support both point-to-point and collective communication calls and will facilitate communication initiated by the host as well as within accelerator device kernels. It will accommodate a variety of devices and network substrates, including NVLINK and next-generation Ethernet networks.

Key components of this role include:

• Co-leading the design and development of an open-source accelerator communication library (OCCL) aimed at supporting point-to-point and collective communication for distributed AI workloads.
• Designing and implementing communication primitives and algorithms optimized for machine learning training and inference workloads, supporting both host applications and accelerator device kernels.
• Investigating and optimizing communication patterns for large-scale deep learning workloads, incorporating techniques for enhancing efficiency through communication scheduling and overlaps with computation.
• Ensuring the communication library is configurable and portable across various hardware platforms and network substrates, including modern high-performance interconnect technologies used in HPC and ML environments such as UCX, NVLINK, InfiniBand, UltraEthernet, and associated networking frameworks.
• Evaluating and optimizing the communication library and associated software components on GPU clusters and large-scale computing systems, ensuring performance scalability, robustness, and usability across heterogeneous computing architectures.
• Collaborating with researchers and engineers at partner institutions in Singapore and Zurich to integrate OCCL with the broader FastTrackAI software ecosystem.
• Maintaining effective communication and collaboration with supervisors, research staff, and external partners spread across Singapore and Zurich, ensuring alignment on research objectives and deliverables.
• Preparing and publishing key scientific outputs, including papers documenting research outcomes and system developments.
• Contributing to the co-supervision of MSc and BSc students engaged in related research topics.

Profile

• PhD in Computer Science or a related field.
• Strong background in computer science, parallel computing, compilers, performance optimization, and high-performance computer systems. Proficient in C and C++.
• Good working knowledge of existing high-level communication library design principles, such as MPI and NCCL.
• Experience and/or knowledge of modern communication and network technologies in HPC and ML environments, including UCX, NVLINK, InfiniBand, UltraEthernet, or related frameworks.
• Prior knowledge in communication optimization for large-scale ML training and inference workloads is essential. Candidates with publications in distributed ML systems, HPC systems, or related areas are highly desirable.
• Prior experience managing larger software engineering projects is required, as the aim of this project is to build a software product, not just a prototype.
• The distributed team across Singapore and Zurich necessitates the ability to work independently while communicating effectively with remote colleagues.
• Proficiency in written and spoken English is required.

Workplace

We Offer

• Accredited with 5 Tripartite Standards by the Tripartite Alliance for Fair & Progressive Employment Practices (TAFEP) Singapore.
• A diverse workplace with 32 nationalities, fostering ample opportunities for mutual learning.
• A positive and inclusive working environment.
• 25 days of annual leave for fixed-term contracts.
• 1 day of Birthday Leave.
• Annual dental benefits.
• A commitment to supporting employees in prioritizing their physical and mental wellness.
• Comprehensive healthcare insurance coverage.
• Flexible hybrid work arrangement (up to 2 days per week from home).
• Abundant networking opportunities across various disciplines.
• Accredited with NS mark certification.

The Singapore-ETH Centre is an equal opportunity and family-friendly employer. All candidates will be evaluated based on their merits and qualifications, with no regard to gender, race, age, or religion.

We Value Diversity and Sustainability

In line with our values, ETH Zurich encourages an inclusive culture, promoting equality of opportunity and valuing diversity. We foster a work and learning environment where the rights and dignity of all our staff and students are respected. Visit our Equal Opportunities and Diversity website to learn more about how we ensure a fair and open environment that allows everyone to thrive. Sustainability is at the core of our mission, and we are continuously working towards a climate-neutral future.

Curious? So are We.

We invite you to apply online using the form below. Only applications matching the job profile will be considered.