Search results

The evaluation relies entirely on LLM-as-judge scoring without any human annotation or preference validation, making it unclear how well the measured metrics align with real user perception.

Status: Open.

#10 In Generative-Engine-Marketing/GEM-Bench;

· Qingbolan opened on Nov 14, 2025

Results are only presented with niche LLM of doubao-1.5

llm-eval-more-modelsResults are only presented with niche LLM doubao-1.5Results are only presented with niche LLM doubao-1.5

rebuttlerebuttle of paper submitrebuttle of paper submit

Status: Open.

#9 In Generative-Engine-Marketing/GEM-Bench;

· Qingbolan opened on Nov 13, 2025

For Figure 2, are the shown weaknesses of Ad-Chat due to inherent limitations of prompt-based insertion, or mainly due to the specific prompt used? Were alternative prompt designs evaluated?

adchat-prompt-design-analysisAre weaknesses of Ad-Chat due to prompt insertion limitations? alternatives?Are weaknesses of Ad-Chat due to prompt insertion limitations? alternatives?

rebuttlerebuttle of paper submitrebuttle of paper submit

Status: Open.

#8 In Generative-Engine-Marketing/GEM-Bench;

· Qingbolan opened on Nov 13, 2025

How sensitive are the results to the number of ads (k > 1) inserted into each response?

ads-k-sensitivity-studySensitivity to number of ads (k>1)Sensitivity to number of ads (k>1)

rebuttlerebuttle of paper submitrebuttle of paper submit

Status: Open.

#7 In Generative-Engine-Marketing/GEM-Bench;

· Qingbolan opened on Nov 13, 2025

The chatbot datasets contain only 10 and 100 queries, respectively, which may limit statistical robustness and generalizability to diverse domains.

expand-chatbot-dataset-queriesChatbot datasets contain only 10 and 100 queries; generalizability concernChatbot datasets contain only 10 and 100 queries; generalizability concern

rebuttlerebuttle of paper submitrebuttle of paper submit

Status: Open.

#6 In Generative-Engine-Marketing/GEM-Bench;

· Qingbolan opened on Nov 13, 2025