🤔 Problem

In the SDSC Call for Collaborations, we received 4 proposals asking to integrate Renku with HPC:

<aside> <img src="/icons/info-alternate_gray.svg" alt="/icons/info-alternate_gray.svg" width="40px" />

Note: The documents linked below are internal only and are not available to those with whom this document has been shared externally.

</aside>

• ‣ from MeteoSwiss
• ‣ from Idiap and UniFr
• ‣ from PSI
• ‣ from LASTRO of EPFL and the FHNW

We heard this as a big signal worth paying attention to!

All these people want to “have Renku on HPC”. They want to be able to take advantage of the collaboration features that Renku offers, connected with the large compute resources of various HPC instances. 2 of the proposals are interested in CSCS/FirecREST, and there are also other individual on-premise clusters at Idiap, PSI, and UniFR.

More specifically, the pain points these users see boil down to 2 key themes:

• Lowering the barrier to entry to HPC systems for researchers
• Connect fragmented code, data, and compute resources to improve collaboration

Following from these 2 themes, the primary target audience we are looking to help here are novice to basic HPC users.

🍴 Appetite

6 weeks

More broadly, we are committed to spending 2 build cycles on CSCS. So, this first build is to get something up a running that the stakeholders can test (maybe in a ‘beta’ state), and then we can polish the functionality for production use in a second build.

🎯 Solution

This pitch proposes a first implementation of interactive Renku sessions running on CSCS. The solution outlined here addresses the two key pain points by:

• Lowering the barrier to entry of HPC by providing a Renku’s simple session interface as a method of accessing CSCS. It also lowers the barrier to entry for using CSCS specifically by simplifying authorization and helping users create custom runtime environments compatible with CSCS via Renku’s built-in environment creation.