Jan 30, 2025 07:00:00

Mozilla.ai releases Lumigator, a tool for finding AI models that fit your needs

The rapid development of AI has given rise to a number of language models that demonstrate high performance in a variety of fields. However, because language models have strengths and weaknesses, selecting the right model for a particular use case has become a difficult challenge for developers. Mozilla.ai,

Mozilla 's AI division, has recently announced Lumigator , a tool that allows developers to make decisions about selecting AI models regardless of their knowledge of machine learning.

Lumigator is here!
https://blog.mozilla.ai/lumigator-is-here-2/

Mozilla.ai Lumigator — Lumigator 0.0.1 documentation
https://mozilla-ai.github.io/lumigator/

Lumigator is an open source platform that guides users through the process of selecting the right language model for their needs. According to the official blog, the main features of Lumigator are as follows:

Quantitative evaluation: Metric-based model comparison and insights
We compare closed and open source models side-by-side using metrics such as BERTScore and ROUGE , which are key metrics in natural language processing.

- Supports both Hugging Face and OpenAPI compatible models
Evaluate models available on the Hugging Face Model Hub or accessible via APIs such as OpenAI or Mistral

UI and API components
Either the user interface or API integration, we'll recommend whichever suits your workflow.

- Software Development Kit (SDK) and documentation for developers
Lumigator is publicly available on GitHub, with detailed SDK and documentation provided.

The Lumigator UI looks like this: First, upload the dataset you want to apply AI to from 'Provide Dataset.'

Once you have uploaded the dataset, start the experiment section by clicking “Create experiment.”

After selecting the model to use in the experiment, click 'Run experiment' to run the experiment.

Once the experiment is complete, it will show either 'SUCCEEDED' or 'FAILED'.

For each model used in the experiment, a table comparing the evaluation metrics is displayed, along with a comparison of model predictions and actual predictions for each instance in the dataset, allowing you to choose from a range of models to suit your dataset.

The installation procedure for Lumigator is detailed on the following page.

Installation — Lumigator 0.0.1 documentation

'AI is transforming how we build applications, but choosing the right model shouldn't be a black box,' said Juliana Rodriguez Araújo, principal product manager at Mozilla. 'Lumigator wants to empower all engineers who want to integrate AI into their products, from beginners to AI experts, to make informed decisions.'

The source code for Lumigator is available on GitHub.

GitHub - mozilla-ai/lumigator: Source code for Mozilla.ai's Lumigator platform