Skip to Content
← Back to services

AI performance evaluation based on testing datasets

Virtual
Pricing/Discount Options: Call #2
Unique Identifier: db0c01ad-4f15-475f-a863-f18489eef454

Service Description

To ensure that their solution works accordingly regarding a task, AI provider have to evaluate their system using a specific dataset. The performance obtained on this dataset helps to prove the adequate performance of their AI solution. However, the evaluation process can be often quite hard for an AI provider to do properly: the evaluation dataset needs to be correctly qualified, and the creation of the evaluation protocol as well as the analysis of the results are not easy tasks.

This service allows AI provider to benefit of the LNE expertise with a full evaluation of their AI system: using an evaluation dataset created for the test or provided by a partner of the TEF project, an assessment of the performance of the AI system is done, and an analysis of its behavior provided. This work also includes a quality assessment of the evaluation dataset. The scope of the analysis of the evaluation results can include robustness and resilience evaluation, depending of the needs of the SMEs.

With this service, the customer will have a full assessment of its solution, allowing them to answer the accuracy requirements of the AI regulation, while also having a full report following all transparency and reproductibility requirements. This service generally takes around 2 months, depending of the needs of the customers.

Offerings: Model & Algorithm (Development, Optimization & Evaluation, etc.)
Provider Logo

Provider & Contact

Provider Organisation Laboratoire National De Metrologie Et D'Essais (LNE)
Provider Country France
Published Email guillaume.bernard@lne.fr
Billing: per AI functionnality
Full Price €20.000-40.000 EUR
Reduced Price €6000-12000 EUR
Pricing Detail

These prices are only tentative and will be further discussed with the SME or startup depending on their needs and the scope of the evaluation.

Operational Details

Service Standards to be defined (see "Method reference")
  • to be defined (see "Method reference")

Related services

  • AI cybersecurity evaluation

    Laboratoire National De Metrologie Et D'Essais (LNE)

    Evaluation of the AI system regarding its robustness against cybersecurity issues ( risk assessment, secure Data, access Control and Authentication, etc …) This will include the design of test protocols, the realization of tests, the analysis of results and production of a test reports.

    View service →
  • AI model evaluation/assessment: Clinical model validation

    Unidade Local De Saúde De Coimbra EPE (ULS Coimbra EPE)

    The service offers SMEs expert evaluation and validation of their AI models intended for clinical use. Leveraging the hospital's domain expertise in healthcare and data analytics, this service assesses the accuracy and clinical suitability of AI models in real-world clinical scenarios by assessing models against clinically relevant metrics, benchmarks, and regulatory standards, it ensures their safety, reliability, and effectiveness in real-world healthcare settings.

    View service →
  • AI model evaluation/assessment: Clinical model validation

    Centro Hospitalar De Sao Joao Epe (CHSJ)

    The service offers SMEs expert evaluation and validation of their AI models intended for clinical use. Leveraging the hospital's domain expertise in healthcare and data analytics, this service assesses the accuracy and clinical suitability of AI models in real-world clinical scenarios by assessing models against clinically relevant metrics, benchmarks, and regulatory standards, it ensures their safety, reliability, and effectiveness in real-world healthcare settings.

    View service →
  • AI Model performance evaluation

    Multitel (MULTITEL)

    This service provides an independent and reproducible evaluation of AI model performance using well-established quantitative metrics. The evaluation is conducted in a controlled and documented execution environment to ensure traceability and repeatability of results. Performance is assessed on client-provided datasets and models, and associated measurement uncertainties are systematically analyzed and reported. The service delivers a detailed and interpretable evaluation report. All activities are performed under ISO 9001 certified processes.

    View service →
  • AI performance evaluation based on mixed testing environments

    Laboratoire National De Metrologie Et D'Essais (LNE)

    Based on the domain of the medical device or robot provided by the customer, the simulation will create a test environment relevant to the system to be evaluated using a video-projection system. The service will be provided using the LE.IA Immersion platform, a platform dedicated to the evaluation of AI. It consists of a video-projection system that makes it possible to reconstitute an "artificial nature" and to immerse the systems to be characterised (personal assistance robots, intelligent cameras, civil and military intervention robots, etc.) in a simulated dynamic reality. It allows the system (robot, camera, etc.) to be subjected to a multitude of test scenarios and to evaluate its reactions in a controlled environment.

    View service →
  • AI performance evaluation based on virtual testing environments

    Laboratoire National De Metrologie Et D'Essais (LNE)

    The system provided by the customer, for instance a medical device using an AI module, will be tested using a simulated environment. The service will be provided using the LE.IA Simulation, a fully simulated test environment (the robot or the medical device and its operating environment are simulated). It allows to generate a very large number of test scenarios that are not feasible in a physical environment.

    View service →