IEEE

RailS: Load Balancing for All-to-All Communication in Distributed Mixture-of-Experts Training

Abstract: Training Mixture-of-Experts (MoE) models introduces sparse and highly imbalanced all-to-all communication that dominates iteration time. Conventional load-balancing methods fail to exploit ...
IEEE

Task Scheduling with Optimized Load Balancing in Container-Based Cloud Computing Environments

Abstract: Challenge of Task Scheduling and Load Balancing Task scheduling and load balancing are the main challenges supported by the dynamic nature of cloud computing environments based on containers ...