We’ve introduced a Wizard to streamline evaluation creation and accelerate your workflow.The API-based approach revolutionizes the evaluation process by enabling rapid evaluation creation through just a few clicks, eliminating the need for manual audio file generation and upload procedures. This streamlined method significantly reduces setup time while maintaining the same comprehensive evaluation capabilities.Currently, the API-based evaluation method supports single audio evaluation for naturalness and quality evaluation and double audio evaluation for voice similarity and preferences evaluation. We are actively working to extend API-based support to additional evaluation types and will be rolling out these capabilities in upcoming releases.
All audio inputs are automatically normalized to -21 LUFS before presentation to human evaluators, ensuring consistent and accurate evaluation results regardless of original audio levels.
You can now customize your report titles and descriptions at any time. When publishing reports publicly, you have full control over audio playability settings, model names, and tags.
Additionally, you can generate custom short URLs instead of using lengthy default URLs.
Multi-language voice data collection from diverse global contributors to improve model performance. This service is currently available to selected early customers who are seeing significant improvements in model accuracy and speed.
Add comprehensive descriptions for your models and comparisons using markdown formatting, similar to README.md files on GitHub or Hugging Face. This enables rich documentation with formatted text, links, and structured content.
Comprehensive voice similarity measurement for voice cloning applications. Our system supports three-way similarity analysis comparing a reference voice with two target outputs, enabling precise evaluation of voice cloning quality.
In three-way similarity analysis, you can now ask specific questions related to each individual model. Through model-related questions, you can conduct separate evaluations for each model within the triple audio evaluation framework, providing the same level of detailed evaluation as single audio evaluations.This feature enables comprehensive analysis by allowing evaluators to focus on specific aspects of each model’s performance while maintaining the comparative context of the three-way evaluation structure.
So far, you can use the provided evaluation templates. Now, you can add your own messages and followup questions up to 8. Instead of asking a single question, you can ask multiple questions on different types.
We revamped the whole report presentation. With the new concept of model_tag, you can group the evaulations into models and multiple tags and analyze more details.
One of the core questions behind the report is that “why did the evaluator think so?”.So we added this new feature to ask the evaluators to annotate the words/phrases and put reasons behind the evaluation ratings.
We previously made the file upload 2x faster. Again, we have made 6x faster upload, total 12x faster than last month.It can be faster depending on your internet speed.
In addition, we have tons of ideas around evaluation management, reliability, modality, speed improvement, and so on.
If you have cool ideas, please feel free to let us know at hello@podonos.com.