All the possible errors that fastdebug can support and verbosify involving Pytorch
Errors
While some errrors are specifically designed for the fastai library, the general idea still holds true in raw Pytorch
as well.
The device error provides a much more readable error when a
and b
were on two different devices. An situation is below:
inp = torch.rand().cuda()
model = model.cpu()
try:
_ = model(inp)
except Exception as e:
device_error(e, 'Input type', 'Model weights')
And our new log:
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-28-981e0ace9c38> in <module>()
2 model(x)
3 except Exception as e:
----> 4 device_error(e, 'Input type', 'Model weights')
10 frames
/usr/local/lib/python3.7/dist-packages/torch/tensor.py in __torch_function__(cls, func, types, args, kwargs)
993
994 with _C.DisableTorchFunction():
--> 995 ret = func(*args, **kwargs)
996 return _convert(ret, cls)
997
RuntimeError: Mismatch between weight types
Input type has type: (torch.cuda.FloatTensor)
Model weights have type: (torch.FloatTensor)
Both should be the same.
By using forward hooks, we can locate our problem layers when they arrive rather than trying to figure out which one it is through a list of confusing errors.
For this tutorial and testing we'll purposefully write a broken model:
from torch import nn
m = nn.Sequential(
nn.Conv2d(3,3,1),
nn.ReLU(),
nn.Linear(3,2)
)
layer_error
can be used anywhere that you want to check that the inputs are right for some model.
Let's use our m
model from earlier to show an example:
inp = torch.rand(5,2, 3)
try:
m(inp)
except Exception as e:
layer_error(e, m, inp)
This will also work with multi-input and multi-output models:
class DoubleInputModel(nn.Sequential):
def __init__(self):
super().__init__()
self.layers = nn.Sequential(nn.Conv2d(3,3,1),
nn.ReLU(),
nn.Linear(3,2))
def forward(self, a, b):
return self.layers(a), self.layers(b)
model = DoubleInputModel()
inp = torch.rand(5,2, 3)
try:
model(inp, inp)
except Exception as e:
layer_error(e, model, inp, inp)
Much more readable!