Topic: "\;Serial a
nd parallel farming from A to Z"\;
Speaker: Sergey Mashchenko\, SHA
RCNET
Webinar link: SN-Seminars Vidyo room
If you need to run batches of related but independent (order of execution doesn'\;t matte r) jobs in SHARCNET or other consortia\, this seminar is for you. The batc hes can consist of serial jobs ("\;serial farming"\;) or parallel jobs ("\;MPI farming"\;\, "\;CUDA farming"\; etc.). Typica l situation when you might need this is when your calculation/simulation d epends on a few parameters which are poorly constrained. If the number of unconstrained parameters is small (say 1-4)\, you can sample the parameter space by first running a course grid of jobs (first batch)\, then analyze the results and run a finer grid of jobs\, zooming in onto the area of in terest (second\, third etc. batches of jobs). In the case of a larger numb er of parameters one can resort to the Monte-Carlo approach\, where one or more batches of jobs would attempt to explore the whole parameter space i n a pseudo-random fashion.
In this seminar I will briefly touch upon simplified ways to do job farming (from one-line shell commands to the ar ray jobs feature of the scheduler)\, but will spend most of my time descri bing our fairly sophisticated set of scripts developed to facilitate seria l etc. farming. With a bit of customization\, these scripts can be used to
I will also specifically cove r the scenario when one needs to run a large number (thousands) of very sh ort jobs. Our scripts can accomplish this task very efficiently\, by bundl ing up many small jobs into fewer larger jobs and utilizing dynamic worklo ad balancing.
Need help attending a webinar? See the SHARCNET He lp Wiki.
SUMMARY:Webinar "Serial and parallel farming from A to Z" URL;VALUE=URI:http://vidyo.computecanada.ca/flex.html?roomdirect.html&key=P r1GiEI51kFi END:VEVENT END:VCALENDAR