The Challenge – Scaling to meet demand

In 2025, the Renku team at the Swiss Data Science Center launched a new version of their platform focused on providing a streamlined hub bringing together the most important assets of data science projects, namely code, data and compute. With the connectivity-first approach of the platform, the team also decided to reimagine its infrastructure layer to simplify operations and allow for the scalability that new usage patterns required.

Managing Demand

[…] the usage on renkulab.io was often very “peaky”, meaning that a lot of resources were needed for a few hours a day […] this meant that either sometimes users could not get access to resources quickly enough, or we had to leave things over-provisioned. […] we decided to migrate our base platform to an infrastructure that affords us the flexibility our users require while also giving us the tools we need to optimize costs.

Rok Roškar, Renku Project Lead @ SDSC

Solution

As an open-source project, a key requirement for Renku is to minimize dependencies on proprietary and closed-source commercially licensed components. Kubernetes is a modern container platform and orchestrator with broad industry support, developed under the Cloud Native Computing Foundation (CNCF) as part of the Linux Foundation.
It allows the platform to use an established and mature technology to dynamically scale compute resources in Node pools “on demand”, while keeping cost under control.

Choosing OCRE and the Azure Kubernetes Service (AKS) for their core platform enables the team to keep the codebase largely independent from any specific cloud hyperscaler, while at the same time benefit from a managed, scalable PaaS cloud service, leveraging local Swiss data centers. This brings cost efficiency, ease of use, security, and stability.

The platform already provides data connectors to various scientific data sources, including cloud and on-premises data storage. In addition to providing compute resources from within the cluster, Renku offers compute integrations to alternative systems such as the CSCS Alps HPC cluster. It can easily be integrated with on-premises private clouds or Kubernetes resources from other commercial cloud providers.

By leveraging managed Kubernetes Cloud services for their core platform via the GÉANT OCRE framework, the Renku team was able to realize the following benefits:

Prioritize product development over IT infrastructure operations
Support short-term, variable workloads like research projects, hackathons, and computer science classes without requiring long commitments.
Use a reliable, open container platform compatible with widespread commercial and enterprise IT standards to benefit from broadly available skills in the IT market, while simplifying technology transfer and industry partnership initiatives.
Benefit from compliance and strong contractual commitments via existing enterprise agreements, support contracts and the OCRE 2024 pan-European framework.

Impact

By building their new cloud-based core platform, the Renku team could greatly reduce operational IT efforts like VM maintenance and cluster management, basically freeing one engineering expert to work on new features instead of managing the infrastructure.

Being able to dynamically scale “peaky” workloads, resource overprovisioning and infrastructure cost had been reduced.Renku is already in use at multiple research and educational departments and institutes, for example:

ETH Zürich, EPFL, University of Fribourg, HSLU, FHNW, BFH
PSI (Paul Scherrer Institute), Material Science Group(Story)

SwissCat+ Catalysis Hub, a national technology platform (Story)
Idiap Research Institute, AI Research (Announcement)
Used in Hackathons with Industry and Public Sector, e.g. Energy Data Hack Days 2025 with industry partners and Public Sector, e.g. SwissGrid, Kanton Aargau, AXPO, BKW and others

Currently, the platform has around 700 monthly users and launches ~1000-2000 compute sessions weekly.

Feel free to try out Renku here!

What is Renku ?

Renku is an Open-Source platform to securely connect data, code and compute to empower researchers to work in collaborative communities.

Developed at the Swiss Data Science Center (SDSC), a national research infrastructure in data science and artificial intelligence (AI) founded by EPFL and ETH Zurich, the platform is used by researchers, educators and students across all of Switzerland.

The Renku platform can connect various data sources and code repositories with on-demand compute resources for research, education and coding events (such as Hackathons) within minutes.

The platform is free for everyone to try at https://renkulab.io/. For higher tiers or support to deploy your own instance, contact the team at hello@renku.io.