Does initializing the position embedding to pure zeros for ViT make it worse? #1668
Description
Has anyone tried initializing the position embedding to pure zeros for ViT?
Does initializing the position embedding to pure zeros make it better or worse than using trunc_normal_()?