Rethinking market research with RAG and fine-tuning

A question that companies that want to understand the market better or advise their customers better are asking us today:

"We have large amounts of structured and unstructured data. Is there a way to make this information usable for market research with the help of generative AI? Ideally in such a way that specific questions can be answered - such as the perception of our customers' brands, the positioning of individual product lines or the resonance of our communication in the market?"

This - or something similar - is how decision-makers from marketing, strategy or management formulate their question when they deal with the topic of "AI-supported market research". And the question is justified. Because it reflects a fundamental change:

Access to data used to be expensive. Today, the bottleneck is proper understanding.

Just a few years ago, data-driven approaches in market research often failed due to the sheer availability of relevant information. Studies were expensive, panel data was difficult to access and proprietary data was stored in unstructured silos.

Those days are over. Today, most companies - whether large corporations or SMEs - have a variety of data sources at their disposal:

• CRM systems,
• E-mail correspondence,
• Website tracking,
• Support tickets,
• Surveys,
• Social media feedback,
• and much more.

What's more, with open source models, cost-efficient APIs and locally installable AI infrastructure, the use of generative AI is economically feasible - even without an in-house data science department.

The new challenge is therefore no longer technical, but strategic and conceptual in nature:

• How do I choose the right methods?
• How do I structure my questions so that they lead to insights?
• And how do I make sure that the AI understands my content, my language and my objectives?

Want to get started? These tools offer an inexpensive start

Not every company needs to set up its own AI system immediately. There are some sensible and cost-effective ways to gain initial insights:

• SEO tools provide information on how competitors are found online - if they are visible there at all. A common surprise is that many companies are effectively invisible online, which means that any traditional competitive analysis comes to nothing.
• Grok allows initial sentiment analyses - for example of customer ratings or comments in the form of tweets - without having to train your own model.
• ChatGPT with file upload is a quick way to systematically evaluate support tickets, internal documentation or customer feedback, for example - ideal for initial, exploratory analyses.

These tools offer initial access - no more, but also no less. They can give you a feel for where data is located, what insights can be derived and how information flows can be improved within the company. However, if you want to take the next step, you need something else: an architecture that neatly combines data, models and business logic.

This is precisely why we have started to think differently about our projects.

Instead of selling ready-made tools or relying on generic solutions, we develop architectures that fit the reality of our customers - with all their vagueness, data gaps and operational peculiarities. This often starts with seemingly simple questions ("What do our customers really say?") and ends with the structured implementation of AI-supported market research that can be integrated into ongoing operations - rather than standing next to them as a foreign body.

We start where others stop: When standard solutions don't work, when tools alone don't provide answers and when market research should be more than just a PDF with colorful bars.

Retrieval Augmented Generation (RAG) as a foundation

One promising approach is the use of RAG (Retrieval-Augmented Generation). This involves equipping a generative AI - such as a language model - with intelligent access to internal company content.

Instead of generating answers purely from pre-trained knowledge, the system specifically searches through relevant content - such as customer feedback, market analyses, product descriptions or support dialogs - and actively incorporates this into the answer.

An example: Imagine you ask: "How do customers in Spanish-speaking countries react to our new price structure?"

A well-constructed RAG system analyzes feedback emails, helpdesk tickets and sales calls from El Salvador and neighboring markets - and provides a well-founded, data-based answer rather than a general assessment.

This transforms your database from an archive into an active knowledge system.

Why fine-tuning makes all the difference

However, a RAG application remains superficial if the underlying language model is not adapted to your industry, your terminology and your communication style.

The so-called Fine-tuning with your own data makes it possible to further develop an AI model using your own examples. In other words, the AI learns how to You which meanings Your terms, and which connections in yours market.

In the field of market research, for example, this means:

• The AI recognizes when feedback is meant ironically,
• understands industry-specific signal words (e.g. in B2B purchasing processes),
• and distinguishes between genuine objections and superficial dissatisfaction.

This results not only in analytical but also contextualized insights. And these are what make the difference - especially where decisions should be based on sound understanding rather than gut feeling.

There are many tools - impact is created through interaction

The tools are there. The data is available. And the technical hurdles have been lowered.

What remains is the question: How do I implement this in a sensible and targeted way?

The answer to this question depends not only on the tool, but above all on the combination of professional experience, conceptual clarity and technological sensitivity.

Many of our customers do not use our support because they have to - but because they want to find their bearings more quickly, argue more precisely and make more sustainable decisions.