Feb
27
DeepSeek DualPipe - a innovative bidirectional pipeline parallelism algorithm used in V3 and R1 training
DualPipe is an innovative bidirectional pipeline parallelism algorithm introduced in the DeepSeek-V3 Technical Report. It achieves full overlap of forward and backward computation-communication phases, also reducing pipeline bubbles. For detailed information on computation-communication overlap, please refer to the profile data.