Root Signals Bulletin - January 2025 📅‍

‍

Autumn has been a busy and exciting time for us at Root Signals. R&D has been in full swing, and for the Holiday season, we are excited to announce some significant developments:

‍

Fact-checking capabilities reach a new level!

‍

We have been diving in deep to offer new Faithfulness and Truthfulness RAG evaluators based on the principle of decomposition. They provide fine-grained and explainable fact checking. Some of the most notable features include:

• One-by-one detailed analysis of each verifiable claim extracted from the evaluated text.
• Parallel execution, along with context filtering and retrieval, helps maintain low latencies..
• Confidence analysis enables fine distinctions and interpretable results
• A systematic multi-step procedure ensures comparable and consistent results.

We are currently optimizing our servers to ensure they operate at minimum latency.

‍

‍

‍

Python SDK and Async Support

‍

We have released new versions of python SDKs (latest version being v1.3.2). Now, your Custom Evaluators can have an arbitrary number of variables in their prompt. We introduced asynchronous support to be able to run Evaluators in parallel. Bug fixes in Root JSON Schema evaluators are deployed.

‍

‍

Documentation

‍

We have improved our Python SDK documentation in various ways. The overall flow of our docs is revamped and examples regarding Evaluators are re-visited. We have a complete new section regarding asynchronous support, showcasing how to run judges in parallel. We also introduced Release Notes (changelog) as well.

Application

‍

We introduce the ability to invite new members to your organization (Settings -> Organization). From now on

‍

‍

We have a new look & feel all across the application, especially  in Custom Evaluator builder. Several small bug fixes are also released.

‍

‍

Stay tuned for early 2025 updates! 🚀