FlashAttentionKernel2 have implement for cuda ，do we have example use it

[mullerhai]

Hello, I see that FlashAttentionKernel2 has been implemented. Do we have test cases and benchmarks for it? How much faster is it compared to regular attention on CUDA? thanks

[iangellove]

FlashAttentionKernel2 is i try to implement attention in cuda. Unfortunately FlashAttentionKernel2 is slower than GemmStridedBatched + softmax_cudnn. if you has better implement for flashattention cuda version.plase tell me.