From 488e7333fb8776e8ade7cee811ef4ed5a04679ae Mon Sep 17 00:00:00 2001 From: Patrick Liu <32552339+patrick-12sigma@users.noreply.github.com> Date: Wed, 19 Dec 2018 23:40:46 -0800 Subject: [PATCH] Update TECHNICAL_DETAILS.md --- TECHNICAL_DETAILS.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/TECHNICAL_DETAILS.md b/TECHNICAL_DETAILS.md index 43ea2e6..85f1854 100644 --- a/TECHNICAL_DETAILS.md +++ b/TECHNICAL_DETAILS.md @@ -96,4 +96,4 @@ Each process keeps an isolated model, data loader, and optimizer. Model parameters are only synchronized once at the begining. After a forward and backward pass, gradients will be allreduced among all GPUs, and the optimizer will update model parameters. -Since the gradients are all reduced, the model parameter stays the same for all processes after the iteration. +Since the gradients are allreduced, the model parameter stays the same for all processes after the iteration. -- GitLab