RuntimeError: CUDA error: device-side assert triggered

[shashini4296]

Traceback (most recent call last):
File "E:\Virtual try on\ACGPN-master\test.py", line 122, in
main()
File "E:\Virtual try on\ACGPN-master\test.py", line 97, in main
fake_image, warped_cloth, refined_cloth = model(Variable(data['label'].cuda()), Variable(data['edge'].cuda()), Variable(img_fore.cuda()), Variable(
File "C:\Users\SM\AppData\Roaming\Python\Python39\site-packages\torch\nn\modules\module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "E:\Virtual try on\ACGPN-master\models\pix2pixHD_model.py", line 492, in forward
return self.inference(label, pre_clothes_mask, img_fore, clothes_mask, clothes, all_clothes_label, real_image, pose, grid, mask_fore)
File "E:\Virtual try on\ACGPN-master\models\pix2pixHD_model.py", line 384, in inference
input_label, masked_label, all_clothes_label = self.encode_input(
File "E:\Virtual try on\ACGPN-master\models\pix2pixHD_model.py", line 233, in encode_input
1, (label_map * (1 - clothes_mask)).data.long().cuda(), 1.0)
File "C:\Users\SM\AppData\Roaming\Python\Python39\site-packages\torch\tensor.py", line 528, in rsub
return _C._VariableFunctions.rsub(self, other)
RuntimeError: CUDA error: device-side assert triggered

How can I fix this issue?

[minar09]

It might be related to your work environment, I think you may find solutions online. For instance: https://stackoverflow.com/questions/51691563/cuda-runtime-error-59-device-side-assert-triggered

[shashini4296]

@minar09 Thank you very much.
I found the issue is with the number of labels.
I'm using PGN (https://github.com/Engineering-Course/CIHP_PGN.git) to create the labels.
How can I fix this issue or is here any other better way to create labels.

[minar09]

Can you please clarify more details on your issue? Do you mean you have a mismatch with the number of labels or something else? CIHP_PGN generated segmentation labels number should be the same here in ACGPN, which means you need to update the code accordingly.

[shashini4296]

I need to get those 14 segmentation labels from the CHIP_PGN. Can you tell me how to update the code.

Background
Hair
Upclothes
Left-shoe
Right-shoe
Noise
Pants
Left_leg
Right_leg
Left_arm
Face
Right_arm

@minar09 Thank you very much.

[minar09]

Hi @shashini4296 , if your dataset images contain new segments that are not present in CIHP_PGN, and you need those to be labeled separately, then you need to train the network with your custom dataset and annotations. However, if your desired image labels are somewhat similar to CIHP_PGN format, you can generate the segmentation with the pretrained CIHP_PGN model first, and then assign new label values to the segmentation according to your needs. Hope that helps. Thanks!

[shashini4296]

@minar09 Thank you very much for the help