Accuracy higher when training myself

[xsacha]

I followed the instructions in the README and trained faceboxes against WIDER-FACES and after 300 epochs (14 hrs, Titan V) I end up with 98.55% vs. the pretrained model with 98.47%.

For Pascal, I get 97.05%, which is higher than both your pretrained model and the original paper.

I also noticed the yoffset you use (20% of the box height) affects the accuracy a fair bit. Changing it to 30% of box height increased Pascal to 97.10%. I notice other variants of faceboxes use + 4, which doesn't work as well. Why is this offset in there?

Is this explained solely by a newer version of Pytorch (v1.0.1) or you did not get to 300 epochs?

[dectld]

Can you share your trained model?

[xsacha]

Do you not get the same result? Just training the model as-is in README gets this result.

When compared against supplied pre-trained (which tested with scores as reported) model, there is large differences in results.

I think it may be a result of the change to data_augment.py

[dectld]

I didn't get the same result, my result was even worse than the pretrained model, so i wonder if you can share your model?By the way, did you test your model on FDDB, my trained model didn't get good result on it.

[xsacha]

The outputs for fddb it creates need to be converted to work in the face-eval and I haven't bothered to do it.

What version of pytorch did you use?

[dectld]

1.0.0

[xsacha]

I think it is because the data augmentation is randomised in the batches. Each training will be against different data.
You can greatly improve results by improving the augmentation at the expense of training time.
This is why it has been so fast and not deterministic.

@dectId: https://drive.google.com/file/d/1aRdcf692S2-oxNdoc6oOjIVvoyvSHl52/view?usp=sharing
What is FDDB performance on this?

[dectld]

I test your model on FDDB, the performance is 0.960

[zisianw]

Hi @xsacha, glad to know you are obtaining better results! The offset you mentioned follows Caffe implementation which can be found here. I believe it was an empirical findings to improve performance slightly. The results in README were obtained when we were using PyTorch v0.4 and based on the final model trained up to 300 epochs. If I remember correctly, the last few models before 300 epochs also produce similar result and some would be marginally better than the final model. Due to the randomness in training data, results could be slightly different.
cc: @sfzhang15

[xsacha]

@zsianw I believe the y offset is actually an error in the bounding box with even numbers on the last anchor box, which also affects the loss.

As you know the width and height of each anchor box is math.ceil(width/step), math.ceil(height/step). The result from the model appears to differ when these are odd or even as confirmed by video playback.

[zisianw]

@xsacha I didn't get the odd or even part, could you elaborate more on "bounding box with even numbers on the last anchor box" and "width and height of each anchor box is math.ceil(width/step), math.ceil(height/step)"?
The anchor box has equal width and height, Inception3 [32x32, 64x64, 128x128], Conv3_2 [256x256], Conv4_2 [512x512].

[xsacha]

It appears it is an artifact of using torch jit. When I trace the model to use in C++, some of the constants no longer change with the input size (they get fixed to 1024x1024 or whatever I trace with).

Also I have trained 5 models to 300 epochs and all get similar accuracy and all higher than your pretrained model. So I think something in Pytorch must have been improved to get these results.

[zisianw]

@xsacha Cool! Thanks for the update. Were the test time hyperparameters same as this repo too? I haven't been working on face detection for some time but would definitely look into it. Did you train with 1 or 2 GPU?

[xsacha]

I trained with a single Titan V in 14 hours. I didn't change any code in this repository whatsoever to get this model/result.

Inference time is unchanged I think? 60 fps on a GTX1060 for a single 1080p stream depending on minimum face size. 6 fps on the CPU. Could probably be optimised further using Intel mkldnn, etc.

By the way, the jitted version in pytorch loses about 2% accuracy (99 vs 97) due to some constants. I haven't looked in to why that is yet.

[xsacha]

By the way, I can now outperform the pretrained model on this github in 30 epochs (<1.5 hrs on Titan V) just by removing some bad images from WIDER_FACE (not adding any). Faces that are blurred out or have black boxes over the top of them will not train well!!!

Even faster with pooling modifications.

This is a very nice and fast base model, especially fast to train.
Thank you very much.

[MCDM2018]

Hi @xsacha i m interested in your modification about pooling. What kind of modification did you do?
Can you explain about that?