by Guest » 03 Jan 2025, 16:50
Ich habe beim Training auf der GPU einen Kontrollpunkt gespeichert. Nachdem ich den Checkpoint neu geladen und das Training fortgesetzt habe, erhalte ich die folgende Fehlermeldung:
Code: Select all
Traceback (most recent call last):
File "main.py", line 140, in
train(model,optimizer,train_loader,val_loader,criteria=args.criterion,epoch=epoch,batch=batch)
File "main.py", line 71, in train
optimizer.step()
File "/opt/conda/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 26, in decorate_context
return func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/torch/optim/sgd.py", line 106, in step
buf.mul_(momentum).add_(d_p, alpha=1 - dampening)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
Mein Trainingscode lautet wie folgt:
Code: Select all
def train(model,optimizer,train_loader,val_loader,criteria,epoch=0,batch=0):
batch_count = batch
if criteria == 'l1':
criterion = L1_imp_Loss()
elif criteria == 'l2':
criterion = L2_imp_Loss()
if args.gpu and torch.cuda.is_available():
model.cuda()
criterion = criterion.cuda()
print(f'{datetime.datetime.now().time().replace(microsecond=0)} Starting to train..')
while epoch
Ich habe beim Training auf der GPU einen Kontrollpunkt gespeichert. Nachdem ich den Checkpoint neu geladen und das Training fortgesetzt habe, erhalte ich die folgende Fehlermeldung:
[code]Traceback (most recent call last):
File "main.py", line 140, in
train(model,optimizer,train_loader,val_loader,criteria=args.criterion,epoch=epoch,batch=batch)
File "main.py", line 71, in train
optimizer.step()
File "/opt/conda/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 26, in decorate_context
return func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/torch/optim/sgd.py", line 106, in step
buf.mul_(momentum).add_(d_p, alpha=1 - dampening)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
[/code]
Mein Trainingscode lautet wie folgt:
[code]def train(model,optimizer,train_loader,val_loader,criteria,epoch=0,batch=0):
batch_count = batch
if criteria == 'l1':
criterion = L1_imp_Loss()
elif criteria == 'l2':
criterion = L2_imp_Loss()
if args.gpu and torch.cuda.is_available():
model.cuda()
criterion = criterion.cuda()
print(f'{datetime.datetime.now().time().replace(microsecond=0)} Starting to train..')
while epoch