How does a knowledge graph help AI agents discover and use APIs?

Knowledge graphs provide semantic context and relationships between APIs and business processes. This helps agents overcome challenges like incomplete metadata and domain-specific terminology, allowing them to find the correct APIs and understand the sequence in which to call them.

What are the main challenges faced by AI agents when interacting with business systems?

Two main challenges are API discovery, due to fragmented metadata and ambiguous terminology in large systems, and understanding business process context, as APIs must be called in a specific sequence defined by processes that vary between customers.

How does SAP use knowledge graphs for its AI agents like Juul?

SAP positions its knowledge graph at the center between data and AI. It translates information into a usable format for AI capabilities, providing semantic context about APIs and business processes to enhance the automation capabilities of agents like Juul.

Can knowledge graphs help manage large numbers of APIs for AI agents?

Yes, knowledge graphs act as a central repository for API metadata and relationships. This structured approach allows for scalable discovery and orchestration using 'broad tools', rather than overloading the agent with individual tool definitions for every API.

What is the role of ontologies in a knowledge graph for AI?

Ontologies provide the semantics and structure to the knowledge graph. They define classes, hierarchies, and properties of entities and relationships, enabling inference and a deeper understanding of the data, beyond just raw facts.

What are the limitations of using a broad tool approach with knowledge graphs for AI agents?

Limitations include metadata not always being AI-ready, requiring enrichment. There can also be extensibility issues if specialized, bespoke tools overlap in scope with the broad tools, and managing the complexity of a growing knowledge graph requires strategies like using subgraphs.

Key Moments

AI Dev 25 x NYC | Christoph Meyer, Lars Heling: Improving AI Agent Discovery with a Knowledge Graph

DeepLearning.AI

Education6 min read37 min video

Dec 3, 2025|713 views|11

Save to Pod

Key Moments

TL;DR

Knowledge graphs enhance AI agents by providing semantic and process context for discovering and executing API tools effectively.

Key Insights

Knowledge graphs are crucial for enabling AI agents to effectively discover and execute tasks within complex enterprise systems.

LLMs make AI agents fluent, while knowledge graphs make them effective by providing necessary semantic and process context.

Challenges for AI agents include API discovery due to fragmented metadata and ambiguous terminology, and understanding business process context for API execution order.

SAP's approach uses a knowledge graph to aggregate metadata, enabling semantic retrieval for API discovery and providing business process context.

Knowledge graph embeddings can be generated using textual descriptions of nodes and their connected neighbors to improve relevance for AI agent tasks.

A demonstrated workflow involves using knowledge graphs for enhanced API discovery, process completion, and fetching API specifications for agent use.

Knowledge graphs facilitate the addition of new context, such as mappings between natural language descriptions and technical identifiers, to improve agent performance.

The 'broad tools' approach, powered by knowledge graphs, allows agents to interact with thousands of APIs without needing a tool for each one, simplifying tool orchestration.

Managing and extending knowledge graphs requires careful consideration, with techniques like subgraphs used to serve relevant information to specific applications or agents.

THE ROLE OF KNOWLEDGE GRAPHS IN AI AGENTS

The session highlights the critical role of knowledge graphs in augmenting AI agents, particularly in enterprise environments. While large language models (LLMs) provide fluency, knowledge graphs offer the effectiveness needed for complex tasks. They achieve this by furnishing AI agents with vital semantic and process-related context. This context is essential for agents to reliably discover and safely invoke the correct tools and APIs across intricate SAP systems, transforming raw data into actionable insights for automation.

CHALLENGES IN AI AGENT TOOL DISCOVERY AND EXECUTION

AI agents face significant hurdles in discovering and executing appropriate APIs. One primary challenge is API discovery itself, especially within vast and diverse system landscapes like SAP's, where API metadata is often fragmented across multiple sources and jargon can lead to ambiguity (e.g., 'PO' meaning product owner or purchase order). Furthermore, APIs cannot be invoked randomly; their execution must adhere to specific business process sequences, which are often complex and customer-specific, adding another layer of difficulty for agents to accurately discern and follow.

SAP'S KNOWLEDGE GRAPH-DRIVEN APPROACH

SAP addresses these challenges by integrating a knowledge graph at the core of its AI architecture, positioned centrally between data and AI capabilities. This knowledge graph acts as a translator, making information more usable and actionable for AI. The approach involves a set of 'broad tools' for API discovery, utilizing vector databases and user utterances to identify candidate APIs. These candidates are then further refined using a retrieve API tool that accesses API specifications and business process context directly from the knowledge graph, which also generates the embeddings used for initial discovery.

KNOWLEDGE GRAPH STRUCTURE AND SEMANTICS

A knowledge graph represents structured information as a graph, using triples (subject, predicate, object) to define relationships between entities like people, places, APIs, or systems. Its power is significantly amplified by an ontology, which provides semantic meaning and hierarchies to these entities and relationships. For example, an ontology can define that an API is a type of 'tool' and 'located in' is a transitive property. This semantic richness allows for inferring facts not explicitly stated and provides a structured foundation for the AI agent's understanding of the environment.

COMPONENTS AND EMBEDDING STRATEGIES

SAP's metadata knowledge graph comprises several layers, including a resource discovery layer for metadata and an API services/tools layer that integrates information from specifications like OpenAPI. Crucially, business process information, often modeled using BPMN or Arato, is also incorporated to define the sequence and connections between APIs. A key advantage is using the knowledge graph's structure to generate embeddings for retrieval. A textual description-based model represents nodes by their own text and that of their connected neighbors, enhancing the relevance of semantic search for AI tasks.

DEMONSTRATED WORKFLOW AND ENHANCED DISCOVERY

A demonstration illustrates building an agent that interacts with business APIs using a knowledge graph. Initially, a vector retrieval step identifies candidate APIs based on user queries. However, this basic retrieval might miss crucial steps in a business process. The system then leverages the knowledge graph to retrieve process information, specifically incoming and outgoing edges representing process flows. This enhances the discovery by explicitly including related APIs and their process context, providing the agent with a more complete set of tools and understanding for correct process execution.

AGENT CONSTRUCTION AND PROMPT ENGINEERING

The process of constructing an agent involves fetching API specifications from the knowledge graph after enhanced discovery. These specifications, along with process information, are then wrapped as tools, often using frameworks like LangChain. The agent's prompt is carefully engineered to instruct it to perform discovery, pay attention to process information, and handle specific API relationships like header-item levels. Mock functions simulate API interactions in the demo, showcasing how agents can correctly execute complex processes, such as creating a purchase order by first generating a purchase requisition, as dictated by the underlying business process.

DYNAMIC CONTEXT AND ADAPTABILITY

The knowledge graph's adaptability is demonstrated by adding new context, such as mapping natural language status descriptions ('active') to technical IDs (e.g., '02') for purchase orders. When the agent's initial query returns incomplete results, updating the knowledge graph with this mapping allows the agent to incorporate the correct filter in subsequent API calls, yielding the desired, accurate outcome. This showcases the graph's utility for enriching metadata and enhancing query precision for AI agents, making them more robust and responsive to diverse user needs.

THE 'BROAD TOOLS' APPROACH AND ITS ADVANTAGES

The presented approach utilizes 'broad tools,' enabling an agent to interact with thousands of APIs without needing a unique tool for each. This strategy scales effectively and sidesteps the issue of overwhelming the agent with an excessive number of tools, thereby solving the problem of tool orchestration. By relying on knowledge graph metadata, this method provides agents with the necessary context to navigate and utilize complex API landscapes efficiently, abstracting away much of the complexity of direct API interaction.

LIMITATIONS AND MANAGEMENT OF KNOWLEDGE GRAPHS

Despite its benefits, challenges remain. Not all metadata is 'AI-ready,' sometimes lacking descriptions or containing technical IDs that require additional context. The extensibility of knowledge graphs also poses a management challenge; as they grow, discovery can become more complex. Techniques like using subgraphs help manage this by serving only relevant information to specific agents or applications. Furthermore, evolving business processes or adding new features requires continuous maintenance and enrichment of the knowledge graph, demanding ongoing effort to keep it accurate and useful.

COMPARING KNOWLEDGE GRAPHS TO TRADITIONAL DATABASES

When questioned about using SQL instead of knowledge graphs, the presenters emphasized the distinct advantages of the latter for AI agents. While SQL databases store data, knowledge graphs explicitly model relationships and business processes. In large enterprises, SQL alone struggles to encapsulate the intricate roles, permissions, and process logic baked into higher layers. APIs, as curated interfaces, already narrow the scope and intent of data interaction, and knowledge graphs make this structured, process-aware information discoverable and usable for agents, which raw SQL queries cannot easily achieve.

Mentioned in This Episode

●Software & Apps

●Companies

●Organizations

●Concepts

AI Agent Development with Knowledge Graph: Dos and Don'ts

Practical takeaways from this episode

Do This

Aggregate diverse metadata into a harmonized knowledge graph.

Use the knowledge graph to provide semantic context for AI agents.

Leverage knowledge graph structure to generate embeddings for retrieval.

Incorporate business process context into the knowledge graph.

Update the knowledge graph with new context (e.g., mappings) to improve agent accuracy.

Use broad tools to enable agents to interact with thousands of APIs scalably.

Trace metadata provenance back to the knowledge graph and LLM.

Avoid This

Do not rely solely on raw API access without semantic context.

Do not assume API specifications are always AI-ready; enrich them.

Do not overload agents with thousands of individual tools; use broad tools.

Be mindful of overlapping scope when combining broad tools with specialized bespoke tools.

Avoid using raw SQL queries directly for complex agent tasks without additional context layers.

Common Questions

A knowledge graph represents structured information as a graph of entities and their relationships. For AI agents, it provides rich semantic context about APIs, services, and business processes, enabling better discovery and execution of tasks.

Topics

Knowledge Graph API Discovery Business Process Automation SAP Vector Database Tool Orchestration Semantic Context

Mentioned in this video

Organizations

SAP Business AI

The division within SAP focused on embedding AI across all SAP solutions, including ERP and supply chain management.

Software & Apps

SAP Knowledge Graph

A knowledge graph developed by SAP positioned at the center between data and AI, designed to translate information for AI to make it more usable and actionable.

Arato

A workflow system mentioned in the context of representing business processes in the knowledge graph.

FIS

An in-memory vector database used for retrieval in the demo.

Concepts

BPMN

Business Process Model and Notation, used for representing business processes within the knowledge graph.