Resaro Europe GmbH

Quality index for AI solutions

AI test platform for performing automated, context-based quality and performance tests of AI solutions

• Contains ready-made test methods, including tests of CV models that analyze e.g. X-ray images of luggage, aerial/drone images, static and robotic surveillance sensors from cameras and other sensors
• Extension, modification and optimization of data sets by diffusion-based, synthetic object generation in the image for selected use cases (e.g. object generation in X-ray images)
• Test data for outlier analysis, property/label correlation analysis, similarity analysis and distribution drift analysis for the comparison of post-production data between test and production data sets
• Model scoring for precision/recall per class across target/object types, object localization, vulnerability analysis, visibility of blind spots in the model and investigative analysis tools, including confusion matrix and spatial heatmap generation
• Supports a range of CV model architectures, from classic CNN-based (Convolutional Neural Networks) to ViT-based (Vision Transformer) architectures
• Pre-built test library with open-source benchmarks and proprietary, application-specific tests.
• Creation of synthetic, application-specific data sets, including multi-turn data sets and simulations of different user profiles.
• Tests various aspects of system performance, including input/output guard accuracy and retrievability, output accuracy and relevance, toxicity, bias, robustness to noise and different types of input, and AI safety.
• Security testing for application-specific vulnerabilities using current jailbreak techniques.
• Automated adaptive red-teaming for security vulnerabilities.
• Uses the AI Solution Quality Index (ASQI) framework as a comprehensive guide to identify which tests to run and report on various aspects of performance as well as target performance thresholds.
• Automatic reporting for each test. Comparison of results with previous versions and other systems.
• A web-based interface to manage benchmarks, view/compare score reports, approve/reject results and generate/manage synthetic data sets.
• APIs that enable the integration of tests into existing user workflows such as deployment pipelines.
• Ability to add your own tests or modify existing ones

Approved Intelligence Platform

An end-to-end evaluation system for automating and scaling tests. The platform works in both cloud and on-premises environments and provides robust, reusable modules to run customized, comprehensive AI tests securely and efficiently. The platform includes a full suite of tests to assess the quality, interoperability, performance, compliance and security of AI systems.

AI Solution Quality Index

The AI Solution Quality Index (ASQI) supports the definition of target quality levels based on configurable quality indicators and mappings from common governance frameworks. In conjunction with modular test packages for the solutions to be tested, automatic selection of technical tests and automated reporting for business, governance and technical stakeholders.