Root Signals Introduces Root Judge: The State-of-the-Art Judge Model for Measuring the Reliability of LLM Applications

‍

PALO ALTO, California, United States — Root Signals, a leader in large language model (LLM) evaluation and AI application quality control, proudly announces the release of Root Judge, a groundbreaking LLM that sets a new standard for reliable, customizable and locally-deployable evaluation models. Root Judge is built as a fine-tuned version of Meta’s Llama-3.3-70B-Instruct, one of the most powerful mid-sized open-weights models.

Root Judge is primarily designed to serve as an LLM-as-a-Judge, enabling organizations to:

• Detect Context-Grounded Hallucinations: Automatically detect, describe and block hallucinations in Retrieval-Augmented-Generation (RAG) pipelines.
• Facilitate Pairwise Preference Judgments: Use customizable rubrics for tasks like inference-time compute optimization or synthetic data generation requiring Best-of-N decisions.
• Support Privacy-Focused Deployments: Avoid sending sensitive data over the public internet while leveraging cutting-edge LLM capabilities.

Solid Training Foundation

Root Judge was meticulously post-trained on a high-quality, human-annotated dataset mix, designed for pairwise preference judgments and multi-turn instruction-following tasks with source citation. Leveraging advanced optimization techniques, such as Direct Preference Optimization (DPO) with Identity Preference Optimization (IPO) loss, the model underwent training on 384 AMD Radeon Instinct™ MI250X GPUs using the LUMI Supercomputer.

”With solutions for reliable and explainable AI, Root Signals is contributing to a critical topic to enterprises. The successful training of Root Judge on the LUMI supercomputer demonstrates both the power of AMD compute platforms and the vibrancy of Finland's AI ecosystem. This is exactly the kind of innovation we need to see more of in Finland and Europe,” says Peter Sarlin, Co-Founder and CVP, AMD Silo AI.

‍

Why Root Judge Stands Out:

1. Fine-Tuned Excellence: State-of-the-Art hallucination detection, outperforming both closed source frontier models such as OpenAI's GPT-4o, o1-mini, o1-preview and Anthropic's Sonnet-3.5 as well as other open source Judge LLMs of similar size.
2. Explainable Outputs: Designed to provide transparent justifications for scoring, enhancing trust in AI-driven assessments.
3. Open Access for Innovation: With open weights and a focus on privacy-centric deployments, Root Judge fosters innovation while addressing data security concerns.

‍

A Model Built for the Future 

“Root Judge represents a major leap in how organizations can evaluate and optimize their LLM systems,” says Ari Heljakka, CEO of Root Signals. “Its ability to transparently deliver context-grounded judgments ensures that businesses can deploy AI responsibly and effectively, while optimizing inference costs and ensuring privacy.”

‍

Root Judge’s applications extend across industries, making it a versatile tool for enterprises, developers, and researchers seeking reliable AI solutions tailored to their needs.

‍

Availability

Root Judge is now available under an open weights license, allowing developers and enterprises to integrate and customize it for their specific evaluation workflows. The model can immediately also be used and compared to other mainstream closed and open LLMs on Root Signals EvalOps platform that allows building, optimizing and managing customized measurement layers powered by LLM judges, to precisely monitor AI application and agent behaviors in production.

To learn more about Root Judge and to explore its transformative capabilities, visit https://www.rootsignals.ai/root-judge-llm.

‍

About Root Signals

Founded in 2023 by a team of AGI researchers and engineers, Root Signals solves the GenAI reliability problem for teams and enterprises. This helps businesses adopt GenAI faster and more effectively by making AI measurable and controllable. Root Signals has offices in Palo Alto, United States, and Helsinki, Finland. For more information, visit https://www.rootsignals.ai/ or reach out to hello@rootsignals.ai.

‍

Media Contact: 

Ari Heljakka, PhD
CEO, Root Signals
ari.heljakka@rootsignals.ai
+358 50 428 0606

Media kit: Root Judge Media Kit

AMD, AMD Instinct, and combinations thereof are trademarks of Advanced Micro Devices, Inc. Other names are for informational purposes only and may be trademarks of their respective owners.

‍