README.md

Algorithms

Introduction

We provide the config files for training and pretrained models for inference on the optimal configurations.

Notes

Download the pretrained backbones from here:

1. resnet50_coco_pose.pth
2. hrnet_coco_pose.pth
3. hrnetw48_coco_pose.pth

Download the above resources and arrange them in the following file structure:

mmhuman3d
├── mmhuman3d
├── docs
├── tests
├── tools
├── configs
└── data
    └── checkpoints
        ├── resnet50_coco_pose.pth
        ├── hrnet_coco_pose.pth
        └── hrnetw48_coco_pose.pth

Results and Models

We evaluate on 3DPW. Values are PA-MPJPE.

Our baseline models for HMR, SPIN and GraphCMR can reach the reported results in the respective works. For PARE, the original work trains the model on MPII for pose estimation task and later on EFT-COCO for mesh recovery before training on he full set of datasets. To keep consistent with the practice adopted throughout our work, we benchmark PARE by training it fro scratch with only ImageNet initialisation. For Graphormer, the original work evaluates on H36M every epoch before fine-tuning the best H36M model on 3DPW-train (Protocol 1) for 5 epochs. To keep consistent, we adopt Protocol 2 instead.

This is a summary of our results:

Algorithms	Datasets	Backbone	Initialisation	Normal	L1	L1+COCO	L1+COCO+Aug
HMR	H36M, MI, COCO, LSP, LSPET, MPII	ResNet-50	ImageNet	64.55	58.20	51.80	51.66
SPIN	H36M, MI, COCO, LSP, LSPET, MPII	ResNet-50	HMR(ImageNet)	59.00	57.08	51.54	50.69
GraphCMR	H36M, COCO, LSP, LSPET, MPII, UP3D	ResNet-50	ImageNet	70.51	67.20	61.74	60.26
PARE	H36M, MI, EFT-[COCO, LSPET, MPII]	HRNet-W32	ImageNet	61.99	61.13	59.98	58.32
Graphormer	H36M, MuCo, COCO, UP3D, MPII	HRNet-W48	ImageNet	63.18	63.47	59.66	58.82

These are the provided logs:

Config	Datasets	Backbone	Variant	3DPW	Log
HMR	H36M, MI, COCO, LSP, LSPET, MPII	ResNet-50	Normal	64.55	log
HMR	H36M, MI, COCO, LSP, LSPET, MPII	ResNet-50	L1	58.20	log
HMR	H36M, MI, COCO, LSP, LSPET, MPII	ResNet-50	L1 + COCO	51.8	log
HMR	H36M, MI, COCO, LSP, LSPET, MPII	ResNet-50	L1 + COCO + Aug	51.66	log
SPIN	H36M, MI, COCO, LSP, LSPET, MPII	ResNet-50	Normal	59.00	log
SPIN	H36M, MI, COCO, LSP, LSPET, MPII	ResNet-50	L1	57.08	log
SPIN	H36M, MI, COCO, LSP, LSPET, MPII	ResNet-50	L1 + COCO	51.54	log
SPIN	H36M, MI, COCO, LSP, LSPET, MPII	ResNet-50	L1 + COCO + Aug	50.69	log
GraphCMR	H36M, COCO, LSP, LSPET, MPII, UP3D	ResNet-50	Normal	70.51	log
GraphCMR	H36M, COCO, LSP, LSPET, MPII, UP3D	ResNet-50	L1	67.2	log
GraphCMR	H36M, COCO, LSP, LSPET, MPII, UP3D	ResNet-50	L1 + COCO	61.74	log
GraphCMR	H36M, COCO, LSP, LSPET, MPII, UP3D	ResNet-50	L1 + COCO + Aug	60.26	log
PARE	H36M, MI, EFT-[COCO, LSPET, MPII]	HRNet-W32	Normal	61.99	log
PARE	H36M, MI, EFT-[COCO, LSPET, MPII]	HRNet-W32	L1	61.13	log
PARE	H36M, MI, EFT-[COCO, LSPET, MPII]	HRNet-W32	L1 + COCO	59.98	log
PARE	H36M, MI, EFT-[COCO, LSPET, MPII]	HRNet-W32	L1 + COCO + Aug	58.32	log
Graphormer	H36M, MuCo, COCO, UP3D, MPII	HRNet-W48	Normal	63.18	log
Graphormer	H36M, MuCo, COCO, UP3D, MPII	HRNet-W48	L1	63.47	log
Graphormer	H36M, MuCo, COCO, UP3D, MPII	HRNet-W48	L1 + COCO	59.66	log
Graphormer	H36M, MuCo, COCO, UP3D, MPII	HRNet-W48	L1 + COCO + Aug	58.82	log