diffusers/docs/source/en/using-diffusers/reproducibility.mdx

<!--Copyright 2022 The HuggingFace Team. All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.
-->

# Reproducibility

Before reading about reproducibility for Diffusers, it is strongly recommended to take a look at 
[PyTorch's statement about reproducibility](https://pytorch.org/docs/stable/notes/randomness.html).

PyTorch states that 
> *completely reproducible results are not guaranteed across PyTorch releases, individual commits, or different platforms.*
While one can never expect the same results across platforms, one can expect results to be reproducible 
across releases, platforms, etc... within a certain tolerance. However, this tolerance strongly varies 
depending on the diffusion pipeline and checkpoint.

In the following, we show how to best control sources of randomness for diffusion models.

## Inference

During inference, diffusion pipelines heavily rely on random sampling operations, such as the creating the 
gaussian noise tensors to be denoised and adding noise to the scheduling step.

Let's have a look at an example. We run the [DDIM pipeline](./api/pipelines/ddim.mdx) 
for just two inference steps and return a numpy tensor to look into the numerical values of the output.

```python
from diffusers import DDIMPipeline
import numpy as np

model_id = "google/ddpm-cifar10-32"

# load model and scheduler
ddim = DDIMPipeline.from_pretrained(model_id)

# run pipeline for just two steps and return numpy tensor
image = ddim(num_inference_steps=2, output_type="np").images
print(np.abs(image).sum())
```

Running the above prints a value of 1464.2076, but running it again prints a different 
value of 1495.1768. What is going on here? Every time the pipeline is run, gaussian noise 
is created and step-wise denoised. To create the gaussian noise with [`torch.randn`](https://pytorch.org/docs/stable/generated/torch.randn.html), a different random seed is taken every time, thus leading to a different result.
This is a desired property of diffusion pipelines, as it means that the pipeline can create a different random image every time it is run. In many cases, one would like to generate the exact same image of a certain 
run, for which case an instance of a [PyTorch generator](https://pytorch.org/docs/stable/generated/torch.randn.html) has to be passed:

```python
import torch
from diffusers import DDIMPipeline
import numpy as np

model_id = "google/ddpm-cifar10-32"

# load model and scheduler
ddim = DDIMPipeline.from_pretrained(model_id)

# create a generator for reproducibility
generator = torch.Generator(device="cpu").manual_seed(0)

# run pipeline for just two steps and return numpy tensor
image = ddim(num_inference_steps=2, output_type="np", generator=generator).images
print(np.abs(image).sum())
```

Running the above always prints a value of 1491.1711 - also upon running it again because we 
define the generator object to be passed to all random functions of the pipeline.

If you run this code snippet on your specific hardware and version, you should get a similar, if not the same, result.

<Tip>

It might be a bit unintuitive at first to pass `generator` objects to the pipelines instead of 
just integer values representing the seed, but this is the recommended design when dealing with 
probabilistic models in PyTorch as generators are *random states* that are advanced and can thus be 
passed to multiple pipelines in a sequence.

</Tip>

Great! Now, we know how to write reproducible pipelines, but it gets a bit trickier since the above example only runs on the CPU. How do we also achieve reproducibility on GPU? 
In short, one should not expect full reproducibility across different hardware when running pipelines on GPU 
as matrix multiplications are less deterministic on GPU than on CPU and diffusion pipelines tend to require
a lot of matrix multiplications. Let's see what we can do to keep the randomness within limits across 
different GPU hardware.

To achieve maximum speed performance, it is recommended to create the generator directly on GPU when running 
the pipeline on GPU:

```python
import torch
from diffusers import DDIMPipeline
import numpy as np

model_id = "google/ddpm-cifar10-32"

# load model and scheduler
ddim = DDIMPipeline.from_pretrained(model_id)
ddim.to("cuda")

# create a generator for reproducibility
generator = torch.Generator(device="cuda").manual_seed(0)

# run pipeline for just two steps and return numpy tensor
image = ddim(num_inference_steps=2, output_type="np", generator=generator).images
print(np.abs(image).sum())
```

Running the above now prints a value of 1389.8634 - even though we're using the exact same seed! 
This is unfortunate as it means we cannot reproduce the results we achieved on GPU, also on CPU.
Nevertheless, it should be expected since the GPU uses a different random number generator than the CPU.

To circumvent this problem, we created a [`randn_tensor`](#diffusers.utils.randn_tensor) function, which can create random noise 
on the CPU and then move the tensor to GPU if necessary. The function is used everywhere inside the pipelines allowing the user to **always** pass a CPU generator even if the pipeline is run on GPU:

```python
import torch
from diffusers import DDIMPipeline
import numpy as np

model_id = "google/ddpm-cifar10-32"

# load model and scheduler
ddim = DDIMPipeline.from_pretrained(model_id)
ddim.to("cuda")

# create a generator for reproducibility
generator = torch.manual_seed(0)

# run pipeline for just two steps and return numpy tensor
image = ddim(num_inference_steps=2, output_type="np", generator=generator).images
print(np.abs(image).sum())
```

Running the above now prints a value of 1491.1713, much closer to the value of 1491.1711 when 
the pipeline is fully run on the CPU.

<Tip>

As a consequence, we recommend always passing a CPU generator if Reproducibility is important.
The loss of performance is often neglectable, but one can be sure to generate much more similar 
values than if the pipeline would have been run on CPU.

</Tip>

Finally, we noticed that more complex pipelines, such as [`UnCLIPPipeline`] are often extremely 
susceptible to precision error propagation and thus one cannot expect even similar results across 
different GPU hardware or PyTorch versions. In such cases, one has to make sure to run 
exactly the same hardware and PyTorch version for full Reproducibility.

## Randomness utilities

### randn_tensor
[[autodoc]] diffusers.utils.randn_tensor