SIGIR-AP 2025 in Xi’an, China
Date:
From December 7–10, 2025, SIGIR-AP 2025 was held in Xi’an, China. As one of the organizers, I co-organized the BREV-RAG workshop — Beyond Relevance-based EValuation of RAG systems — at SIGIR-AP 2025.
Workshop website: http://sakailab.com/brev-rag/
Why BREV-RAG?
Evaluation of Retrieval-Augmented Generation (RAG) systems is often dominated by relevance-focused criteria (e.g., relevance, correctness, groundedness). However, real-world RAG deployments demand a broader view of quality: reliability, responsibility, usefulness, and other dimensions that are not fully captured by traditional relevance-centric evaluation.
BREV-RAG aims to advance this discussion and encourage more comprehensive, actionable evaluation protocols for RAG systems.
What we did at the workshop
- We accepted and presented 5 papers in the first half of the session.
- The second half focused on group discussions, using the accepted papers as “seed materials” to explore new evaluation axes and practical methodologies beyond relevance-based metrics.
- We plan to consolidate the workshop outcomes and report them.