-
@ renzume.
2025-02-27 19:22:16
Detailed profiling data from a training and inference framework is shared, highlighting communication-computation overlap strategies with PyTorch Profiler visualizations. The framework implements DualPipe with MoE layers across different configurations, including EP64/TP1 for training and EP32/TP1 for prefilling, demonstrating balanced routing and micro-batch optimization techniques.
https://github.com/deepseek-ai/profile-data
#performanceprofiling #deeplearning #moearchitecture #pytorch #parallelcomputing