BenchmarkQED: Automated benchmarking of RAG systems

BenchmarkQED takes RAG benchmarking to another level. Imagine LazyGraphRAG smashing through competition—even when wielding a hefty 1M-token context. The only hitch? It occasionally stumbles on direct relevance for local queries. But fear not, AutoQ is in its corner, crafting a smorgasbord of synthetic queries that hammer out consistent, fair RAG assessments, shrugging off dataset quirks like a seasoned pro.