Analysing numerical data is a cornerstone of decision-making for any organization and this is no less true for marketing agencies or consulting companies. Spotting trends, understanding evolutions, and identifying risks in these numbers requires precision, speed, and clarity. Traditionally, professionals have relied on tools like Excel or Power BI to crunch numbers and create dashboards, or more recently, they have experimented with Large Language Models (LLMs) to extract insights. Yet, these approaches come with their own limitations.
At MetaMarketing, we saw an opportunity to bridge the gap between these tools by creating a smarter, AI app for generating insights from numbers. Our app analyses clients' marketing or financial data with remarkable accuracy and generates comprehensive, client-ready reports that highlight trends, evolutions, and risks in the data. Here is how we approached this challenge and why the solution stands apart.
One study found that GPT-4 achieved an accuracy of only 58% on a quantitative analysis benchmark
Identifying the Problem
Excel and Power BI are powerful tools, but they require users to invest significant time and expertise to build dashboards, interpret results, and present findings. While these platforms excel at handling raw numbers, they are not designed to provide context or narrative around the data. Insights derived from them often depend heavily on the skill of the user.
On the other hand, LLMs, such as GPT-based models, offer impressive capabilities in generating human-like text and interpreting context. However, they struggle with numerical precision, particularly when it comes to performing calculations or data-heavy tasks. For instance, research has shown that LLMs often misinterpret statistical information, face difficulties in reconciling data from multiple sources, and lack the causal reasoning necessary for deep marketing insights.
Studies have found that LLMs encounter significant difficulties when performing complex data analyses. For example, when tasked with interpreting statistical trends across multiple marketing campaigns, an LLM may misattribute changes in engagement rates to incorrect factors due to a lack of numerical depth. Consider a scenario where click-through rates increase but conversions decrease. An LLM might fail to correlate this with a poorly optimised landing page, instead attributing it to an irrelevant factor like ad design.
Causal Reasoning Limitations
Another key limitation is causal reasoning. LLMs often struggle to determine why a particular trend is occurring. For example, if a campaign's ROI declines despite an increase in impressions, an LLM might not integrate causal knowledge such as seasonal fluctuations or budget reallocations across channels. This can lead to incomplete or misleading conclusions.
Studies confirm this limitation. One study found that GPT-4 achieved an accuracy of only 58% on the QRData benchmark, highlighting significant room for improvement. Among open-source models, Deepseek-coder-instruct attained the highest accuracy at just 37%, further underlining the challenges of using LLMs alone for data-heavy tasks.
This duality presented a clear opportunity: to build a solution that combines the best of both worlds. We needed a solution that could ensure numerical accuracy while leveraging LLMs to make the results accessible and insightful.
By combining LLMs with a data engine an industry-specific algorithms, businesses can produce reports with the speed of computers and the language fluency of LLMs.
Building the Data Engine
The backbone of the MetaMarketing solution is a custom-built data engine. This engine processes numerical data with precision, using advanced statistical and analytical techniques to identify trends, pinpoint risks, and uncover underlying patterns. Unlike an LLM working alone, this engine guarantees the mathematical reliability that business analysis demands.
Our design approach considered findings from the latest research on LLMs. We recognised that LLMs often underperform in handling complex datasets without guidance or pre-processed inputs. To address this, we implemented a preprocessing layer that ensures the data fed into the LLM is clean, normalised, and annotated with key statistical insights. This layer allows the LLM to focus on generating narratives while leaving the heavy numerical lifting to the data engine.
Layered on top of this robust data engine is an integration with LLM technology. Here, the LLM plays to its strengths: generating natural-language narratives that convert the insights into text. For example, rather than just stating, "Campaign ROI declined by 3%," the MetaMarketing solution uses the outcome of the data engine to provide further understanding of the data. It contextualises the result, writing: "Campaign ROI declined by 3%, driven by a 12% increase in ad spend and a 5% drop in click-through rates in the West European region." This seamless blend of number crunching, algorithms and storytelling ensures that users not only see the final results but also understand the reasons behind them.
Domain expertise
Our approach also incorporated domain expertise. By embedding marketing knowledge into the solution, we ensured that the generated insights were relevant and actionable. For instance, the MetaMarketing app can automatically flag underperforming campaigns or highlight growth opportunities based on established benchmarks. Additionally, the user interface was designed to prioritise simplicity. Users can upload data and receive detailed reports without needing to navigate a steep learning curve.
Real-World Impact
The impact of this approach is already evident. Tasks that previously took analysts days to complete can now be done in minutes. The MetaMarketing solution not only identifies trends but also explains their implications, reducing the need for manual interpretation and ensuring that reports are client-ready from the start. Consulting agencies using the MetaMarketing app can provide their clients with more insightful advice in minutes, enhancing their value proposition and standing out in a competitive market.
Looking Ahead
This app represents a significant step forward in numerical analysis, but it is only the beginning. Future iterations could integrate predictive analytics, enabling users to anticipate risks and opportunities before they materialise. Benchmarking capabilities will also be added, allowing firms to compare campaign performance against industry standards. The possibilities are vast, and the role of AI in transforming consulting services is only set to grow.
Conclusion
By combining a robust numerical data engine with large language models, it is possible to create a smarter, more efficient way to generate business insights from quantitative data. This integration combines the high-speed performance of cloud computing with the language fluency of LLMs. It goes beyond the limitations of Excel, Power BI, or standalone LLMs, providing a solution that is as accurate as it is insightful. For consulting agencies and organisations aiming to improve their quantitative analysis capabilities, this approach is transformative. Those who fail to adopt it risk being outpaced by those who embrace it.
Comentários