Question about SVE implementation in ggml_vec_dot_q8_0_q8_0()

[huanglsh]

While reading through the function ggml_vec_dot_q8_0_q8_0() in
ggml.c/src/ggml-cpu/arch/arm/quants.c, I noticed something in the SVE implementation that I’d like to better understand.

When vector_length equals 512, the following code appears:

const svint8_t qx_32 = svld1_s8(ph32, x0->qs);
svint8_t qx_64 = svld1_s8(pl32, x0->qs + 2);

From my understanding, the second load might have been intended to use x1->qsinstead of x0->qs + 2, like this:

const svint8_t qx_32 = svld1_s8(ph32, x0->qs);
svint8_t qx_64 = svld1_s8(pl32, x1->qs);

Could someone help clarify whether I’m misunderstanding the logic,
or if both loads are indeed meant to come from the same pointer?

Thanks in advance! I’m trying to understand the data layout and SVE usage here more deeply.

[truecoder34]

Hi @huanglsh , have you solved this issue? This behavior also looks strange for me. Actually it will be great to check this implementation on ARM Machine supporting SVE 512 bits.
If i understand correct offset +2
svint8_t qy_64 = svld1_s8(pl32, y0->qs + 2); is not suitable here . Entire SVE512 implementation can be redesigned.

Hi @xctan , seems you have developed this implementation, could you please clarify why do we need +2 offset ?

Thank you in advance.