This colloquium was CANCELED.

Topic: "Process Interrogation on Running Jobs"

Speaker: Doug Roberts, SHARCNET

Video link

 

--- 

For jobs to run efficiently on a cluster typically each worker process is assigned to run on a single core.  To ensure enough cores are available slurm scripts are used to communicate the unique resource requirements to the underlying scheduler when each and every job is submitted.  The Alliance wiki provides many sample slurm scripts that researchers can use "out of the box" with minimal changes. Some programs however may not behave as expected and start more running processes than cores were reserved for.  This can occur either immediately when a job starts or dynamically while it runs.  Under such circumstances compute nodes may gradually or quickly become overloaded (depending on the nature of the overloading and program characteristics) resulting in potential performance degradation.  Such scenario's can be caused by custom written codes, downloaded third party codes or commercial codes.  It will be the goal of this presentation to recognize such problematic situations and provide some strategies to resolve them as quickly as possible for subsequent jobs.  Doing so will ensure all programs run as efficiently on the limited available cluster resources.

The Compute Ontario Colloquia are weekly Zoom presentations on Advanced Research Computing, High Performance Computing, Research Data Management, and Research Software topics, delivered by staff from three Compute Ontario consortia (CAC, SciNet, SHARCNET) and guest speakers. The series began January 2023 and superseded similar series previously delivered by individual consortia (e.g. General Interest Seminars by SHARCNET or User Group Meeting TechTalks by SciNet). The colloquia are one hour long and include time for questions. No registration is required. Presentations are usually recorded and uploaded to the hosting consortium video channel (colloquia hosted by SHARCNET go to our youtube channel).