VERA-AI: Responsible AI Choices for Flemish SMEs

A customer calls with a question about a product, and the answer can be found in a technical datasheet stored somewhere on the company server. By the time an employee has located the document, the conversation is already over. In more than one hundred interviews with Flemish SMEs, situations like this emerged repeatedly.

AI can help address this challenge, but it is not always the most appropriate solution. For the same information retrieval task, three approaches may appear similar from the outside while operating in fundamentally different ways.

With traditional automation, a search function uses predefined rules to retrieve the correct document without relying on a language model. This approach is predictable and cost-effective, but employees still need to find the answer themselves within the retrieved information.

In an LLM workflow, the process follows predefined steps. The system identifies the most relevant passages and a large language model, such as the one behind ChatGPT, generates an answer while referencing its sources. This offers greater ease of use while maintaining a degree of control and transparency.

An AI agent goes a step further by independently deciding which sources to consult and which actions to take. While this approach offers the highest level of flexibility and autonomy, it is also the most expensive and the most difficult to manage and govern.

In addition to choosing a technical approach, companies must decide whether models and data should remain on-premises or be hosted through a cloud service, while ensuring compliance with the GDPR and the EU AI Act. Most SMEs do not have dedicated AI teams to support these decisions, and the suitability of a chosen solution often becomes apparent only after significant development work has already been completed.

Existing decision-making frameworks are typically designed for large organisations and rarely link technology selection to a systematic evaluation of the solution’s actual impact and value after implementation. As a result, SMEs often lack practical guidance for making informed, evidence-based AI adoption decisions.

We will develop and refine the methodology through iterative cycles of design, testing, and evaluation, following a design science research approach. During the first phase, interviews with companies and experts will be conducted to understand how SMEs currently make decisions about AI adoption.

Based on these insights, we will co-create four practical components of the methodology together with participating companies:

1. Knowledge Task Typology
   A framework for identifying and categorising the type of knowledge task that needs to be supported.
2. Decision Matrix
   A tool for determining which approach is most appropriate for a given task, ranging from classical automation to LLM-based workflows or AI agents, and for defining the level of autonomy the system should have.
3. Privacy-by-Design Canvas
   A structured framework to determine, for different categories of data, whether information should remain under local control or may be processed through cloud-based services.
4. Evaluation Framework
   A framework for measuring the impact of the selected solution in terms of efficiency, quality, user adoption, and employee experience.

The methodology will subsequently be applied and validated in three Flemish SMEs from different sectors, preferably including at least one company in the (bio)technology domain. Each participating company will contribute a different type of knowledge-intensive task, such as retrieving information from technical datasheets, capturing the expertise of a specialist, or supporting quotation processes that span multiple information systems.

For each company, we will develop working prototypes representing the approaches identified as viable candidates. To ensure a fair comparison, the evaluation will follow a process similar to the use of an answer key in an examination. Together with experienced employees, we will compile realistic questions based on everyday work situations, including the correct answers and the source documents in which those answers can be found. Each prototype will be tested against the same set of questions, and its performance will be assessed based on the accuracy of both the generated answers and the cited sources.

Before testing begins, the methodology will generate a prediction regarding which approach is expected to perform best. If the observed results do not match this prediction, the evaluation criteria will be refined and incorporated into the next iteration of the methodology.

After a trial period with the selected approach, outcomes will be compared with a baseline measurement of how the task was previously performed, allowing us to quantify improvements in efficiency, quality, and user experience.

The prototypes developed within the project are intended for research and validation purposes. The implementation and further development of production-ready solutions remain the responsibility of the participating company and its IT partner. Prior to the start of each pilot, a data processing agreement will be established to define which data may be used and where it will be processed, ensuring compliance with applicable data protection regulations.