Moco Inplace Update Params (#1693)

Summary:
The out of place .data manipulation is slower/more memory, and doesnt work with cudagraphs.

Upstream pr link: facebookresearch/moco#141

Pull Request resolved: #1693

Reviewed By: xuzhao9

Differential Revision: D46453742

Pulled By: eellison

fbshipit-source-id: 2072103ba9edd29444ff8cc16b6dc393c4f7a95d

Author: eellison

torchbenchmark/models/moco/moco/builder.py

@@ -47,7 +47,7 @@ def _momentum_update_key_encoder(self):
         Momentum update of the key encoder
         """
         for param_q, param_k in zip(self.encoder_q.parameters(), self.encoder_k.parameters()):
-            param_k.data = param_k.data * self.m + param_q.data * (1. - self.m)
+            param_k.mul_(self.m).add_(param_q.mul(1. - self.m))
 
     @torch.no_grad()
     def _dequeue_and_enqueue(self, keys):