Topic: "File quotas and effective strategies to store large sets of small files"

Speaker: Rakesh Ravichandran, CAC

Video link

--- 

By default, every group can store up 500k files on Project space in 1TB of diskspace. If you have a large dataset of smaller file,i.e. ~2MB/file, you can quickly run out of inode quota. This webinar presents the usage file formats such as SQL Blob and Netcdf, to effectively store relatively a large set of smaller files.

The Compute Ontario Colloquia are weekly Zoom presentations on Advanced Research Computing, High Performance Computing, Research Data Management, and Research Software topics, delivered by staff from three Compute Ontario consortia (CAC, SciNet, SHARCNET) and guest speakers. The series began January 2023 and superseded similar series previously delivered by individual consortia (e.g. General Interest Seminars by SHARCNET or User Group Meeting TechTalks by SciNet). The colloquia are one hour long and include time for questions. No registration is required. Presentations are usually recorded and uploaded to the hosting consortium video channel.