Migration¶

Migration

Migration from MMSelfSup 0.x¶

Warning

MMSelfSup 1.x depends on some new packages, you should create a new environment for MMSelfSup 1.x even if you have a well-rounded MMSelfSup 0.x environment before. Please refer to the install tutorial for required packages installation.

We introduce some modifications of MMSelfSup 1.x, to help users to migrate their projects based on MMSelfSup 0.x to 1.x smoothly.

Three important packages are listed below,

MMEngine: MMEngine is the base of all OpenMMLab 2.0 repos. Some modules, which are not specific to Computer Vision, are migrated from MMCV to this repo.
MMCV: The computer vision package of OpenMMLab. This is not a new dependency, but you need to upgrade it to above 2.0.0rc1 version.
MMClassification: The image classification package of OpenMMLab. This is not a new dependency, but you need to upgrade it to above 1.0.0rc0 version.

Config¶

This section illustrates the changes of our config files in _base_ folder, which includes three parts

Datasets: mmselfsup/configs/selfsup/_base_/datasets
Models: mmselfsup/configs/selfsup/_base_/models
Schedules: mmselfsup/configs/selfsup/_base_/schedules

Datasets¶

In MMSelfSup 0.x, we use key data to summarize all information, such as samples_per_gpu, train, val, etc.

In MMSelfSup 1.x, we separate train_dataloader, val_dataloader to summarize information correspodingly and the key data has been removed.

Original

data = dict(
    samples_per_gpu=32,  # total 32*8(gpu)=256
    workers_per_gpu=4,
    train=dict(
        type=dataset_type,
        data_source=dict(
            type=data_source,
            data_prefix='data/imagenet/train',
            ann_file='data/imagenet/meta/train.txt',
        ),
        num_views=[1, 1],
        pipelines=[train_pipeline1, train_pipeline2],
        prefetch=prefetch,
    ),
    val=...)

New

train_dataloader = dict(
    batch_size=32,
    num_workers=4,
    persistent_workers=True,
    sampler=dict(type='DefaultSampler', shuffle=True),
    collate_fn=dict(type='default_collate'),
    dataset=dict(
        type=dataset_type,
        data_root=data_root,
        ann_file='meta/train.txt',
        data_prefix=dict(img_path='train/'),
        pipeline=train_pipeline))
val_dataloader = ...

Besides, we remove the key of data_source to keep the pipeline format consistent with that in other OpenMMLab projects. Please refer to Config for more details.

Changes in pipeline:

Take MAE as an example of pipeline:

train_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(
        type='RandomResizedCrop',
        size=224,
        scale=(0.2, 1.0),
        backend='pillow',
        interpolation='bicubic'),
    dict(type='RandomFlip', prob=0.5),
    dict(type='PackSelfSupInputs', meta_keys=['img_path'])
]

Models¶

In the config of models, there are two main different parts from MMSeflSup 0.x.

There is a new key called data_preprocessor, which is responsible for preprocessing the data, like normalization, channel conversion, etc. For example:

model = dict(
    type='MAE',
    data_preprocessor=dict(
        mean=[123.675, 116.28, 103.53],
        std=[58.395, 57.12, 57.375],
        bgr_to_rgb=True),
    backbone=...,
    neck=...,
    head=...,
    init_cfg=...)

NOTE: data_preprocessor can be defined outside the model dict, which has higher priority than it in model dict.

For example bwlow, Runner would build data_preprocessor based on mean=[123.675, 116.28, 103.53] and std=[58.395, 57.12, 57.375], but omit the 127.5 of mean and std.

data_preprocessor=dict(
    mean=[123.675, 116.28, 103.53],
    std=[58.395, 57.12, 57.375],
    bgr_to_rgb=True)
model = dict(
    type='MAE',
    data_preprocessor=dict(
        mean=[127.5, 127.5, 127.5],
        std=[127.5, 127.5, 127.5],
        bgr_to_rgb=True)，
    backbone=...,
    neck=...,
    head=...,
    init_cfg=...)

Related codes in MMEngine: Runner could get key cfg.data_preprocessor in cfg directly and merge it to cfg.model.

There is a new key loss in head in MMSelfSup 1.x, to determine the loss function of the algorithm. For example:

model = dict(
    type='MAE',
    data_preprocessor=...,
    backbone=...,
    neck=...,
    head=dict(
        type='MAEPretrainHead',
        norm_pix=True,
        patch_size=16,
        loss=dict(type='MAEReconstructionLoss')),
    init_cfg=...)

Schedules¶

MMSelfSup 0.x	MMSelfSup 1.x	Remark
optimizer_config	/	It has been removed.
/	optim_wrapper	The `optim_wrapper` provides a common interface for updating parameters.
lr_config	param_scheduler	The `param_scheduler` is a list to set learning rate or other parameters, which is more flexible.
runner	train_cfg	The loop setting (`EpochBasedTrainLoop`, `IterBasedTrainLoop`) in `train_cfg` controls the work flow of the algorithm training.

Changes in optimizer and optimizer_config:

Now we use optim_wrapper field to specify all configuration about the optimization process. And the optimizer is a sub field of optim_wrapper now.
paramwise_cfg is also a sub field of optim_wrapper, instead of optimizer.
optimizer_config is removed now, and all configurations of it are moved to optim_wrapper.
grad_clip is renamed to clip_grad.

Original

optimizer = dict(
    type='AdamW',
    lr=0.0015,
    weight_decay=0.3,
    paramwise_options = dict(
        norm_decay_mult=0.0,
        bias_decay_mult=0.0,
    ))
optimizer_config = dict(grad_clip=dict(max_norm=1.0))

New

optim_wrapper = dict(
    optimizer=dict(type='AdamW', lr=0.0015, weight_decay=0.3),
    paramwise_cfg = dict(
        norm_decay_mult=0.0,
        bias_decay_mult=0.0,
    ),
    clip_gard=dict(max_norm=1.0),
)

Changes in lr_config:

The lr_config field is removed and we use new param_scheduler to replace it.
The warmup related arguments are removed, since we use a separate lr scheduler to implement this functionality. These introduced lr schedulers are very flexible, and you can use them to design many kinds of learning rate / momentum curves. See the tutorial for more details.

Original

lr_config = dict(
    policy='CosineAnnealing',
    min_lr=0,
    warmup='linear',
    warmup_iters=5,
    warmup_ratio=0.01,
    warmup_by_epoch=True)

New

param_scheduler = [
    # warmup
    dict(
        type='LinearLR',
        start_factor=0.01,
        by_epoch=True,
        end=5,
        # Update the learning rate after every iters.
        convert_to_iter_based=True),
    # main learning rate scheduler
    dict(type='CosineAnnealingLR', by_epoch=True, begin=5， end=200),
]

Changes in runner:

Most configuration in the original runner field is moved to train_cfg, val_cfg and test_cfg, which configure the loop in training, validation and test.

Original	runner = dict(type='EpochBasedRunner', max_epochs=200)
New	train_cfg = dict(by_epoch=True, max_epochs=200)

Runtime settings¶

Changes in checkpoint_config and log_config:

The checkpoint_config are moved to default_hooks.checkpoint and the log_config are moved to default_hooks.logger.

And we move many hooks settings from the script code to the default_hooks field in the runtime configuration.

default_hooks = dict(
    # record the time of every iterations.
    timer=dict(type='IterTimerHook'),
    # print log every 100 iterations.
    logger=dict(type='LoggerHook', interval=100),
    # enable the parameter scheduler.
    param_scheduler=dict(type='ParamSchedulerHook'),
    # save checkpoint per epoch, and automatically save the best checkpoint.
    checkpoint=dict(type='CheckpointHook', interval=1, save_best='auto'),
    # set sampler seed in distributed evrionment.
    sampler_seed=dict(type='DistSamplerSeedHook'),
    # validation results visualization, set True to enable it.
    visualization=dict(type='VisualizationHook', enable=False),
)

In addition, we splited the original logger to logger and visualizer. The logger is used to record information and the visualizer is used to show the logger in different backends, like terminal, TensorBoard and Wandb.

Original

log_config = dict(
    interval=100,
    hooks=[
        dict(type='TextLoggerHook'),
        dict(type='TensorboardLoggerHook'),
    ])

New

default_hooks = dict(
    ...
    logger=dict(type='LoggerHook', interval=100),
)
visualizer = dict(
    type='SelfSupVisualizer',
    vis_backends=[dict(type='LocalVisBackend'), dict(type='TensorboardVisBackend')],
)

Changes in load_from and resume_from:

The resume_from is removed. And we use resume and load_from to replace it.
- If resume=True and load_from is not None, resume training from the checkpoint in load_from.
- If resume=True and load_from is None, try to resume from the latest checkpoint in the work directory.
- If resume=False and load_from is not None, only load the checkpoint, not resume training.
- If resume=False and load_from is None, do not load nor resume.

Changes in dist_params:

The dist_params field is a sub field of env_cfg now. And there are some new configurations in the env_cfg.

env_cfg = dict(
    # whether to enable cudnn benchmark
    cudnn_benchmark=False,
    # set multi process parameters
    mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0),
    # set distributed parameters
    dist_cfg=dict(backend='nccl'),
)

Changes in workflow: workflow related functionalities are removed.
New field visualizer:

The visualizer is a new design in OpenMMLab 2.0 architecture. We use a visualizer instance in the runner to handle results & log visualization and save to different backends. See the MMEngine visualization tutorial for more details.

visualizer = dict(
    type='SelfSupVisualizer',
    vis_backends=[
        dict(type='LocalVisBackend'),
        # Uncomment the below line to save the log and visualization results to TensorBoard.
        # dict(type='TensorboardVisBackend')
    ]
)

New field default_scope: The start point to search module for all registries. The default_scope in MMSelfSup is mmselfsup. See the registry tutorial for more details.

Package¶

The table below records the general modification of the folders and files.

MMSelfSup 0.x	MMSelfSup 1.x	Remark
apis	/	Currently, the `apis` folder has been removed, it might be added in the future.
core	engine	The `core` folder has been renamed to `engine`, which includes `hooks`, `opimizers`.
datasets	datasets	The datasets is implemented according to different datasets, such as ImageNet, Places205.
datasets/data_sources	/	The `data_sources` has been removed and the directory of `datasets` now is consistent with other OpenMMLab projects.
datasets/pipelines	datasets/transforms	The `pipelines` folder has been renamed to `transforms`.
/	evaluation	The `evaluation` is created for some evaluation functions or classes, such as KNN function or layer for detection.
/	models/losses	The `losses` folder is created to provide different loss implementations, which is from `heads`
/	structures	The `structures` folder is for the implementation of data structures. In MMSelfSup, we implement a new data structure, `selfsup_data_sample`, to pass and receive data throughout the training/val process.
/	visualization	The `visualization` folder contains the visualizer, which is responsible for some visualization tasks like visualizing data augmentation.