niconiconi on Nostr: You know parallelization in HPC is hard when you ask the question, "How do I send all ...

You know parallelization in HPC is hard when you ask the question, "How do I send all tasks from a controller to workers before starting a simulation, and collect all results after it finishes?" The answer is often, "Don't do this. The dataset is too large for any single node to handle, so nobody should ever have a global view of the full data at any point, including initialization and post-processing."