About the panoptic custom dataset training

[Synot1]

Hello, when I am training with panoptic segmentation, I followed the instructions on https://github.com/cocodataset/panopticapi/blob/master/panoptic_coco_categories.json to convert labelme to coco format, and I have also completed the dataset registration. However, during training, I keep encountering the following error:

[08/08 10:46:48 maskdino.data.dataset_mappers.coco_panoptic_new_baseline_dataset_mapper]: [COCOPanopticNewBaselineDatasetMapper] Full TransformGens used in training: [RandomFlip(), ResizeScale(min_scale=0.1, max_scale=2.0, target_height=64, target_width=64), FixedSizeCrop(crop_size=(64, 64))]
WARNING [08/08 10:46:49 d2.data.datasets.coco]:
Category ids in annotations are not in [1, #categories]! We'll apply a mapping for you.

[08/08 10:46:49 d2.data.datasets.coco]: Loaded 779 images in COCO format from D:\MaskDINO-main\MaskDINO-main\data\panotic_pathlearning\train\annotations\annotations.json
WARNING [08/08 10:46:49 d2.data.datasets.coco]: Filtered out 124 instances without valid segmentation. There might be issues in your dataset generation process. Please check https://detectron2.readthedocs.io/en/latest/tutorials/datasets.html carefully
[08/08 10:46:49 d2.data.build]: Removed 1 images with no usable annotations. 778 images left.
[08/08 10:46:49 d2.data.build]: Distribution of instances among all 7 categories:

category	#instances	category	#instances	category	#instances
ignore	0	pixel	11651	OK	0
g2p	0	mp1	0	dataline	8736
NG	0
total	20387
[08/08 10:46:49 d2.data.build]: Using training sampler TrainingSampler
[08/08 10:46:49 d2.data.common]: Serializing the dataset using: <class 'detectron2.data.common._TorchSerializedList'>
[08/08 10:46:49 d2.data.common]: Serializing 778 elements to byte tensors and concatenating them all ...
[08/08 10:46:49 d2.data.common]: Serialized dataset takes 3.66 MiB
[08/08 10:46:49 d2.data.build]: Making batched data loader with batch_size=1
[08/08 10:46:49 d2.checkpoint.detection_checkpoint]: [DetectionCheckpointer] Loading from D:\MaskDINO-main\MaskDINO-main\weight\maskdino_swinl_50ep_300q_hid2048_3sd1_panoptic_58.3pq.pth ...
[08/08 10:46:49 fvcore.common.checkpoint]: [Checkpointer] Loading from d:\MaskDINO-main\MaskDINO-main\weight\maskdino_swinl_50ep_300q_hid2048_3sd1_panoptic_58.3pq.pth ...
WARNING [08/08 10:46:50 fvcore.common.checkpoint]: Skip loading parameter 'sem_seg_head.predictor.class_embed.weight' to the model due to incompatible shapes: (133, 256) in the checkpoint but (6, 256) in the model! You might want to double check if this is expected.
WARNING [08/08 10:46:50 fvcore.common.checkpoint]: Skip loading parameter 'sem_seg_head.predictor.class_embed.bias' to the model due to incompatible shapes: (133,) in the checkpoint but (6,) in the model! You might want to double check if this is expected.
WARNING [08/08 10:46:50 fvcore.common.checkpoint]: Skip loading parameter 'sem_seg_head.predictor.label_enc.weight' to the model due to incompatible shapes: (133, 256) in the checkpoint but (6, 256) in the model! You might want to double check if this is expected.
WARNING [08/08 10:46:50 fvcore.common.checkpoint]: Skip loading parameter 'criterion.empty_weight' to the model due to incompatible shapes: (134,) in the checkpoint but (7,) in the model! You might want to double check if this is expected.
WARNING [08/08 10:46:50 fvcore.common.checkpoint]: Some model parameters or buffers are not found in the checkpoint:
criterion.empty_weight
sem_seg_head.predictor.class_embed.{bias, weight}
sem_seg_head.predictor.label_enc.weight
[08/08 10:46:50 d2.engine.train_loop]: Starting training from iteration 0
D:\MaskDINO-main\MaskDINO-main
D:\MaskDINO-main\MaskDINO-main
D:\MaskDINO-main\MaskDINO-main
D:\MaskDINO-main\MaskDINO-main
ERROR [08/08 10:46:55 d2.engine.train_loop]: Exception during training:
Traceback (most recent call last):
File "D:\MaskDINO-main\MaskDINO-main\detectron2\engine\train_loop.py", line 155, in train

self.run_step()

File "D:\MaskDINO-main\MaskDINO-main\detectron2\engine\defaults.py", line 498, in run_step
self._trainer.run_step()
File "D:\MaskDINO-main\MaskDINO-main\detectron2\engine\train_loop.py", line 494, in run_step
loss_dict = self.model(data)
File "D:\anaconda3\envs\maskdino\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "D:\MaskDINO-main\MaskDINO-main\maskdino\maskdino.py", line 268, in forward
outputs, mask_dict = self.sem_seg_head(features, targets=targets)
File "D:\anaconda3\envs\maskdino\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "D:\MaskDINO-main\MaskDINO-main\maskdino\modeling\meta_arch\maskdino_head.py", line 75, in forward
return self.layers(features, mask,targets=targets)
File "D:\MaskDINO-main\MaskDINO-main\maskdino\modeling\meta_arch\maskdino_head.py", line 80, in layers
predictions = self.predictor(multi_scale_features, mask_features, mask, targets=targets)
File "D:\anaconda3\envs\maskdino\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "D:\MaskDINO-main\MaskDINO-main\maskdino\modeling\transformer_decoder\maskdino_decoder.py", line 448, in forward
assert targets is not None
AssertionError
[08/08 10:46:55 d2.engine.hooks]: Total training time: 0:00:04 (0:00:00 on hooks)
[08/08 10:46:55 d2.utils.events]: iter: 0 lr: N/A max_mem: 1395M
Traceback (most recent call last):
File "D:\MaskDINO-main\MaskDINO-main\567.py", line 411, in
launch(
File "D:\MaskDINO-main\MaskDINO-main\detectron2\engine\launch.py", line 84, in launch
main_func(*args)
File "D:\MaskDINO-main\MaskDINO-main\567.py", line 398, in main
return trainer.train()
File "D:\MaskDINO-main\MaskDINO-main\detectron2\engine\defaults.py", line 488, in train
super().train(self.start_iter, self.max_iter)
File "D:\MaskDINO-main\MaskDINO-main\detectron2\engine\train_loop.py", line 155, in train
self.run_step()
File "D:\MaskDINO-main\MaskDINO-main\detectron2\engine\defaults.py", line 498, in run_step
self._trainer.run_step()
File "D:\MaskDINO-main\MaskDINO-main\detectron2\engine\train_loop.py", line 494, in run_step
loss_dict = self.model(data)
File "D:\anaconda3\envs\maskdino\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "D:\MaskDINO-main\MaskDINO-main\maskdino\maskdino.py", line 268, in forward
outputs, mask_dict = self.sem_seg_head(features, targets=targets)
File "D:\anaconda3\envs\maskdino\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "D:\MaskDINO-main\MaskDINO-main\maskdino\modeling\meta_arch\maskdino_head.py", line 75, in forward
return self.layers(features, mask,targets=targets)
File "D:\MaskDINO-main\MaskDINO-main\maskdino\modeling\meta_arch\maskdino_head.py", line 80, in layers
predictions = self.predictor(multi_scale_features, mask_features, mask, targets=targets)
File "D:\anaconda3\envs\maskdino\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "D:\MaskDINO-main\MaskDINO-main\maskdino\modeling\transformer_decoder\maskdino_decoder.py", line 448, in forward
assert targets is not None
AssertionError

In maskdino.py, my batched_inputs does not have instances, but I tried adding them without success. It still goes into the target = None else branch. How can I fix my code?

[ynwy]

您好，我在进行全景分割训练时，按照https://github.com/cocodataset/panopticapi/blob/master/panoptic_coco_categories.json上的说明将labelme转换为coco格式，并且完成了数据集的注册，但是在训练过程中，一直出现以下错误：

[08/08 10:46:48 maskdino.data.dataset_mappers.coco_panoptic_new_baseline_dataset_mapper]: [COCOPanopticNewBaselineDatasetMapper] 训练中使用的完整 TransformGens: [RandomFlip(), ResizeScale(min_scale=0.1, max_scale=2.0, target_height=64, target_width=64), FixedSizeCrop(crop_size=(64, 64))] 警告 [08/08 10:46:49 d2.data.datasets.coco]: 注释中的类别 ID 不在 [1, #categories] 中！我们将为您应用映射。

[08/08 10:46:49 d2.data.datasets.coco]: 从 D:\MaskDINO-main\MaskDINO-main\data\panotic_pathlearning\train\annotations\annotations.json 加载了 779 张 COCO 格式的图片 警告 [08/08 10:46:49 d2.data.datasets.coco]: 过滤掉 124 个没有有效分割的实例。您的数据集生成过程可能存在问题。请仔细检查https://detectron2.readthedocs.io/en/latest/tutorials/datasets.html [08/08 10:46:49 d2.data.build]: 删除了 1 张没有可用注释的图片。剩余 778 张图片。 [08/08 10:46:49 d2.data.build]: 在所有 7 个类别中的实例分布：

类别 #实例 类别 #实例 类别 #实例
忽略 0 像素 11651 好的 0
格2普 0 mp1 0 数据线 8736
天然气 0
全部的 20387
[08/08 10:46:49 d2.data.build]: 使用训练采样器 TrainingSampler
[08/08 10:46:49 d2.data.common]: 使用以下方法序列化数据集：<class 'detectron2.data.common._TorchSerializedList'>
[08/08 10:46:49 d2.data.common]: 将 778 个元素序列化为字节张量并将它们全部连接起来...
[08/08 10:46:49 d2.data.common]: 序列化数据集占用 3.66 MiB
[08/08 10:46:49 d2.data.build]: 使用 batch_size=1 制作批量数据加载器
[08/08 10:46:49 d2.checkpoint.detection_checkpoint]: [DetectionCheckpointer] 正在从 D:\MaskDINO-main\MaskDINO-main\weight\maskdino_swinl_50ep_300q_hid2048_3sd1_panoptic_58.3pq.pth 加载...
[08/08 10:46:49 fvcore.common.checkpoint]: [Checkpointer] 从 d:\MaskDINO-main\MaskDINO-main\weight\maskdino_swinl_50ep_300q_hid2048_3sd1_panoptic_58.3pq.pth 加载...
警告 [08/08 10:46:50 fvcore.common.checkpoint]: 由于形状不兼容，跳过将参数“sem_seg_head.predictor.class_embed.weight”加载到模型中：检查点中为 (133, 256)，但模型中为 (6, 256)！您可能需要仔细检查这是否是预期的。
警告 [08/08 10:46:50 fvcore.common.checkpoint]: 由于形状不兼容，跳过将参数“sem_seg_head.predictor.class_embed.bias”加载到模型中：检查点中为 (133,)，但模型中为 (6,)！您可能需要仔细检查这是否是预期的。
警告 [08/08 10:46:50 fvcore.common.checkpoint]: 由于形状不兼容，跳过将参数“sem_seg_head.predictor.label_enc.weight”加载到模型中：检查点中为 (133, 256)，但模型中为 (6, 256)！您可能需要仔细检查这是否符合预期。
警告 [08/08 10:46:50 fvcore.common.checkpoint]: 由于形状不兼容，跳过将参数“criterion.empty_weight”加载到模型中：检查点中为 (134,)，但模型中为 (7,)！您可能需要仔细检查这是否是预期的。
警告 [08/08 10:46:50 fvcore.common.checkpoint]: 检查点中未找到某些模型参数或缓冲区：
标准.空_权重
sem_seg_head.predictor.class_embed.{偏见，权重}
sem_seg_head.predictor.label_enc.权重
[08/08 10:46:50 d2.engine.train_loop]: 从迭代 0 开始训练
D:\MaskDINO-main\MaskDINO-main
D:\MaskDINO-main\MaskDINO-main
D:\MaskDINO-main\MaskDINO-main
D:\MaskDINO-main\MaskDINO-main
错误 [08/08 10:46:55 d2.engine.train_loop]: 训练期间出现异常：
回溯（最近一次调用最后一次）：
文件“D:\MaskDINO-main\MaskDINO-main\detectron2\engine\train_loop.py”，第 155 行，在训练中

self.run_step()

文件“D:\MaskDINO-main\MaskDINO-main\detectron2\engine\defaults.py”，第 498 行，在 run_step self._trainer.run_step() 文件“D:\MaskDINO-main\MaskDINO-main\detectron2\engine\train_loop.py”，第 494 行，在 run_step loss_dict = self.model(data) 文件“D:\anaconda3\envs\maskdino\lib\site-packages\torch\nn\modules\module.py”，第 1194 行，在 _call_impl return forward_call(*input, **kwargs) 文件“D:\MaskDINO-main\MaskDINO-main\maskdino\maskdino.py”，第 268 行，在 forward 输出中，mask_dict = self.sem_seg_head(features, target=targets) 文件“D:\anaconda3\envs\maskdino\lib\site-packages\torch\nn\modules\module.py”，第 1194 行，在 _call_impl 中 返回 forward_call(*input, **kwargs) 文件“D:\MaskDINO-main\MaskDINO-main\maskdino\modeling\meta_arch\maskdino_head.py”，第 75 行，在 forward return self.layers(features, mask,targets=targets) 文件“D:\MaskDINO-main\MaskDINO-main\maskdino\modeling\meta_arch\maskdino_head.py”，第 80 行，在 layeres predictions = self.predictor(multi_scale_features, mask_features, mask, target=targets) 文件“D:\anaconda3\envs\maskdino\lib\site-packages\torch\nn\modules\module.py”，行1194，在 _call_impl 中 返回 forward_call(*input, **kwargs) 文件“D:\MaskDINO-main\MaskDINO-main\maskdino\modeling\transformer_decoder\maskdino_decoder.py”，第 448 行，在前向 断言目标不是无 AssertionError [08/08 10:46:55 d2.engine.hooks]：总训练时间：0:00:04（0:00:00 上钩子） [08/08 10:46:55 d2.utils.events]：iter：0 lr：N/A max_mem：1395M Traceback（最近一次调用最后一次）： 文件“D:\MaskDINO-main\MaskDINO-main\567.py”，第 411 行，在 启动中（ 文件“D:\MaskDINO-main\MaskDINO-main\detectron2\engine\launch.py​​”，第 84 行，在启动 main_func(*args) 文件“D:\MaskDINO-main\MaskDINO-main\567.py”，第 398 行，在主 返回 trainer.train() 文件“D:\MaskDINO-main\MaskDINO-main\detectron2\engine\defaults.py”，第 488 行，在训练中 super().train(self.start_iter, self.max_iter) 文件“D:\MaskDINO-main\MaskDINO-main\detectron2\engine\train_loop.py”，第 155 行，在训练中 self.run_step() 文件“D:\MaskDINO-main\MaskDINO-main\detectron2\engine\defaults.py”，第 498 行，在 run_step self._trainer.run_step() 文件“D:\MaskDINO-main\MaskDINO-main\detectron2\engine\train_loop.py”，第 494 行，在 run_step loss_dict = self.model(data) 文件“D:\anaconda3\envs\maskdino\lib\site-packages\torch\nn\modules\module.py”，第 1194 行，在 _call_impl return forward_call(input,kwargs) 文件“D:\MaskDINO-main\MaskDINO-main\maskdino\maskdino.py”，第 268 行，正向 输出，mask_dict = self.sem_seg_head（features，targets = target） 文件“D：\ anaconda3 \ envs \ maskdino \ lib \ site-packages \ torch \ nn \ modules \ module.py”，第 1194 行，在 _call_impl 中 返回 forward_call（ input， kwargs） 文件“D：\ MaskDINO-main \ MaskDINO-main \ maskdino \ modeling \ meta_arch \ maskdino_head.py”，第 75 行，在 forward 中 返回 self.layers（features，mask，targets = target） 文件“D：\ MaskDINO-main \ MaskDINO-main \ maskdino \ modeling \ meta_arch \ maskdino_head.py”，第 80 行，在层中 预测 = self.predictor（multi_scale_features，mask_features，mask，targets = target） 文件“D:\anaconda3\envs\maskdino\lib\site-packages\torch\nn\modules\module.py”, 第 1194 行, 在 _call_impl 返回 forward_call(*input, **kwargs) 文件“D:\MaskDINO-main\MaskDINO-main\maskdino\modeling\transformer_decoder\maskdino_decoder.py”, 第 448 行, 在前向 断言目标不是 None AssertionError

在 maskdino.py 中，我的 batched_inputs 没有实例，但我尝试添加它们但没有成功。它仍然进入 target = None else 分支。我该如何修复我的代码？

同问，解决了吗？