Skip to Content
← Back to services

Software framework for the comparison and benchmarking of AI and traditional algorithm

Virtual
Unique Identifier: 7a96d86d-07a2-48f9-bfa1-35affbe3b6f0

Service Description

Providing an Open Source software framework to simplify the comparison of ML and traditional algorithms

Method Description: A generic way to build object-oriented datasets and algorithm pipelines and tools to evaluate them.

Method reference: https://github.com/mad-lab-fau/tpcp

Offerings: Platform (trusted research environment, authentication federation, etc.)
Provider Logo

Provider & Contact

Provider Organisation Friedrich-Alexander-Universitaet Erlangen-Nuernberg (FAU)
Provider Country Germany
Published Email bjoern.eskofier@fau.de
Billing: per hour
Full Price €120 EUR
Reduced Price €No discount can be provided
Pricing Detail

The software is free, the price indicated is only for the consulting/support. This price by hour is an estimate.

Operational Details

Service Inputs Complex/Multimodal Dataset(s)
Service Outputs Framework for standardized data handling, ML Pipelines, and algorithm evaluation
Certification Support None
Service Standards None

Related services

  • AI cybersecurity evaluation

    Laboratoire National De Metrologie Et D'Essais (LNE)

    Evaluation of the AI system regarding its robustness against cybersecurity issues ( risk assessment, secure Data, access Control and Authentication, etc …) This will include the design of test protocols, the realization of tests, the analysis of results and production of a test reports.

    View service →
  • AI model evaluation/assessment: Clinical model validation

    Centro Hospitalar De Sao Joao Epe (CHSJ)

    The service offers SMEs expert evaluation and validation of their AI models intended for clinical use. Leveraging the hospital's domain expertise in healthcare and data analytics, this service assesses the accuracy and clinical suitability of AI models in real-world clinical scenarios by assessing models against clinically relevant metrics, benchmarks, and regulatory standards, it ensures their safety, reliability, and effectiveness in real-world healthcare settings.

    View service →
  • AI model evaluation/assessment: Clinical model validation

    Unidade Local De Saúde De Coimbra EPE (ULS Coimbra EPE)

    The service offers SMEs expert evaluation and validation of their AI models intended for clinical use. Leveraging the hospital's domain expertise in healthcare and data analytics, this service assesses the accuracy and clinical suitability of AI models in real-world clinical scenarios by assessing models against clinically relevant metrics, benchmarks, and regulatory standards, it ensures their safety, reliability, and effectiveness in real-world healthcare settings.

    View service →
  • AI Model performance evaluation

    Multitel (MULTITEL)

    This service provides an independent and reproducible evaluation of AI model performance using well-established quantitative metrics. The evaluation is conducted in a controlled and documented execution environment to ensure traceability and repeatability of results. Performance is assessed on client-provided datasets and models, and associated measurement uncertainties are systematically analyzed and reported. The service delivers a detailed and interpretable evaluation report. All activities are performed under ISO 9001 certified processes.

    View service →
  • AI performance evaluation based on mixed testing environments

    Laboratoire National De Metrologie Et D'Essais (LNE)

    Based on the domain of the medical device or robot provided by the customer, the simulation will create a test environment relevant to the system to be evaluated using a video-projection system. The service will be provided using the LE.IA Immersion platform, a platform dedicated to the evaluation of AI. It consists of a video-projection system that makes it possible to reconstitute an "artificial nature" and to immerse the systems to be characterised (personal assistance robots, intelligent cameras, civil and military intervention robots, etc.) in a simulated dynamic reality. It allows the system (robot, camera, etc.) to be subjected to a multitude of test scenarios and to evaluate its reactions in a controlled environment.

    View service →
  • AI performance evaluation based on testing datasets

    Laboratoire National De Metrologie Et D'Essais (LNE)

    To ensure that their solution works accordingly regarding a task, AI provider have to evaluate their system using a specific dataset. The performance obtained on this dataset helps to prove the adequate performance of their AI solution. However, the evaluation process can be often quite hard for an AI provider to do properly: the evaluation dataset needs to be correctly qualified, and the creation of the evaluation protocol as well as the analysis of the results are not easy tasks. This service allows AI provider to benefit of the LNE expertise with a full evaluation of their AI system: using an evaluation dataset created for the test or provided by a partner of the TEF project, an assessment of the performance of the AI system is done, and an analysis of its behavior provided. This work also includes a quality assessment of the evaluation dataset. The scope of the analysis of the evaluation results can include robustness and resilience evaluation, depending of the needs of the SMEs. With this service, the customer will have a full assessment of its solution, allowing them to answer the accuracy requirements of the AI regulation, while also having a full report following all transparency and reproductibility requirements. This service generally takes around 2 months, depending of the needs of the customers.

    View service →