Monday, March 9, 2026

Artificial intelligence that scored 95% – until consultants found out it was artificial intelligence

Share

Presented by SAP


When SAP conducted a silent, internal experiment to gauge consultants’ attitudes toward AI, the results were striking. Five teams were asked to review responses to over 1,000 business requirements from SAP’s second AI pilot, Joule for Consultants, which would normally take several weeks.

Four teams were told that the analysis was conducted by junior trainees who had recently graduated. They reviewed the material, found it impressive, and rated it at about 95% right.

The fifth team was told that the same answers came from the artificial intelligence.

They rejected almost everything.

Only when asked to check each answer individually did they discover that the AI ​​was actually highly right – revealing detailed insights that the consultants had initially dismissed. Overall accuracy? Again about 95%.

“The lesson here is that we need to be especially careful when introducing AI – especially in how we communicate with senior consultants about its capabilities and how they incorporate it into their workflows,” says Guillermo B. Vazquez Mendez, principal architect, RI Transformation and Business Architecture, SAP America Inc.

The experiment has since become an eye-opening starting point for SAP’s efforts toward the 2030 consultant: a practitioner who is deeply human, powered by artificial intelligence, and no longer burdened by the technical work of the past.

Overcoming skepticism towards artificial intelligence

The resistance is not surprising, notes Vazquez. Consultants with twenty or thirty years of experience have extensive institutional knowledge and an understandable degree of caution.

However, AI second pilots like Joule for Consultants are not a replacement for expertise. They reinforce it.

“What Joule really does is make their very expensive time much more efficient,” Vazquez says. “This eliminates office work so employees can focus on providing high-quality responses in a fraction of the time.”

He constantly emphasizes this message: “Artificial intelligence will not replace you. It is a tool for you. Human supervision is always needed. But now, instead of wasting time looking for documentation, you gain a lot of time and increase the effectiveness and detail of your answers.”

The consultant has moved on from technology implementation to business insight

Historically, consultants have spent approximately 80% of their time understanding technical systems – how processes work, data flows and functions are performed. Customers, on the other hand, spend 80% of their time focused on their business.

This is where Joule comes in.

“There is a gap there, and the bridge is artificial intelligence,” Vazquez says. “It reverses the time equation, allowing consultants to invest more energy in understanding the client’s industry and business goals. Artificial intelligence takes the brunt of the technical effort so consultants can focus on achieving the right business results.”

Accelerating up-to-date consultants

Artificial intelligence is also changing the way up-to-date employees learn.

“We are pleased that Joule acts as a bridge between older consultants who are slower to adapt and interns and new consultants who already have technical knowledge,” says Vazquez.

Younger consultants develop faster because Joule helps them operate independently. Meanwhile, seniors get involved where their knowledge is most essential.

This is also where many consultants learn the basics of newfangled AI co-pilots. Much of the work depends on rapid design – for example, instructing Joule to serve as a senior principal technology architect specializing in finance and SAP S/4HANA 2023, and then asking her to analyze business requirements and deliver the results as tables or PowerPoint slides.

Once they understand how to formulate prompts, consultants consistently produce higher-quality, more structured responses.

Recent architects can also communicate better with their more experienced counterparts. They know what they don’t know and can ask targeted questions, which makes mentoring much smoother. Vazquez adds that there is a real synergy – senior consultants see how quickly up-to-date hires adapt and learn thanks to AI, and this momentum encourages them to keep up and implement the technology themselves.

We look to the future of AI co-pilots

“We are still in the early stages of developing artificial intelligence – we are toddlers,” Vazquez says. “Right now, co-pilots are relying on fast engineering to get good answers. The better you are at prompting, the better answer you will get.”

But this represents only the earliest phase of what these systems will ultimately do. As the co-pilot matures, it will no longer be narrow to responding to prompts and will begin to interpret entire business processes – understanding the sequence of steps, identifying where human intervention is needed, and detecting where an AI agent can take control. This change leads directly to agent-based AI.

Deep knowledge of SAP processes enables this evolution. The company has mapped more than 3,500 business processes across industries — a repository Vazquez calls “some of the most valuable, rigorously tested processes developed in the last 50 years.” Every day, SAP systems support approximately $7.3 trillion in global trade, providing emerging AI agents with a opulent foundation for navigation and thought.

“With this level of process and data visibility, we can take a real step forward,” he says, “by equipping our consultants with agent-based AI that can solve complex challenges and push us toward increasingly autonomous systems.”


Sponsored articles are content created by a company that pays to publish or has a business relationship with VentureBeat and is always clearly marked. For more information, please contact us sales@venturebeat.com.

Latest Posts

More News