GET_STARTED.md

Getting Started

Outline

• Natrual Corruption
• Unsupervised Domain Adaptation

Natural Corruption

All the models in the ./zoo are ready to run on nuScenes-C by running:

cd ./zoo/<MODEL>
bash tools/dist_robust_test.sh <CONFIG> <CHECKPOINT> <GPU_NUM>

The config files are included in ./config/robust_test folders. If you want to test your own models, please follow the instructions below:

Test Scripts

The test scripts on nuScenes-C are included in ./corruptions/tools/. First copy the test tools:

cp -r ./corruptions/tools ./zoo/<MODEL>

To test on nuScenes-c, simply run the following commands:

cd ./zoo/<MODEL>
bash tools/dist_robust_test.sh <CONFIG> <CHECKPOINT> <GPU_NUM>

However, there are a few things to do before you can run the above command successfully.

Custom Loading

If the original config uses LoadMultiViewImageFromFiles to load images. You can simply copy custom_loading.py to the corresponding folder

cp ./corruptions/project/mmdet3d_plugin/datasets/pipelines/custom_loading.py ./zoo/<MODEL>/projects/mmdet3d_plugin/datasets/pipelines/

Then add the Custom_LoadMultiViewImageFromFiles in the pipelines/__init__.py to register:

from .custom_loading import Custom_LoadMultiViewImageFromFiles
__all__ = ['Custom_LoadMultiViewImageFromFiles']

If you use your own way to load images, you can modify it to load nuScenes-C data by simply adding three attribute (i.e., corruption, severity, corruption_root) to the original class like this:

@PIPELINES.register_module()
class Custom_LoadMultiViewImageFromFiles(object):
    def __init__(self, to_float32=False, color_type='unchanged', corruption=None, severity=None, corruption_root=None):
        self.to_float32 = to_float32
        self.color_type = color_type

        ################################################################
        # the following attributes are used for loading nuScenes-c data
        ################################################################
        self.corruption = corruption
        self.severity = severity
        self.corruption_root = corruption_root
        if corruption is not None:
            assert severity in ['easy', 'mid', 'hard'], f"Specify a severity of corruption benchmark, now {severity}"
            assert corruption_root is not None, f"When benchmark corruption, specify nuScenes-C root"

Then modify the image path from the nuScenes to nuScenes-C one, here is a simple example:

def get_corruption_path(corruption_root, corruption, severity, filepath):
    folder, filename = os.path.split(filepath)
    _, subfolder = os.path.split(folder)
    return os.path.join(corruption_root, corruption, severity, subfolder, filename)

For more details, please refer to custom_loading.py to customize for loading nuScenes-C.

Custom Configuration

Then, modify the original loading module to the custom one defined above. Simply replace the loading module from LoadMultiViewImageFromFiles:

test_pipeline = [
    dict(type='LoadMultiViewImageFromFiles', to_float32=True),
    ...
]

to Custom_LoadMultiViewImageFromFiles:

corruption_root = path/to/nuScenes-c

test_pipeline = [
    dict(type='Custom_LoadMultiViewImageFromFiles', to_float32=True, corruption_root=corruption_root),
    ...
]

Lastly, specify the corruption types to be tested by adding:

corruptions = ['CameraCrash', 'FrameLost', 'MotionBlur', 'ColorQuant', 'Brightness', 
                'LowLight', 'Fog', 'Snow']

Unsupervised Domain Adaptation

First copy the custom_nuscenes and uda_nuscenes.py to the model folder:

cp -r uda/custom_nuscenes zoo/<MODEL>/
cp uda/projects/mmdet3d_plugin/datasets/uda_nuscenes.py zoo/<MODEL>/projects/mmdet3d_plugin/datasets

Add the UDANuScenesDataset module to datasets/__init__.py to register

Modify the ann_file config file to domain annotation generated in data prepration:

data = dict(
    train=dict(
        ...
        ann_file=anno_root + 'nuscenes_infos_boston_train.pkl',),
    val=dict(
        ## Modify to UDANuScenesDataset
        ## to test on the specific domain
        type='UDANuScenesDataset',
        ann_file=anno_root + 'nuscenes_infos_sing_val.pkl'),
    test=dict(    
        ## Modify to UDANuScenesDataset
        ## to test on the specific domain    
        type='UDANuScenesDataset',
        ann_file=anno_root + 'nuscenes_infos_sing_val.pkl'))