Multi-Agent Report Generation using Agents as Tools¶
In this notebook, we will explore how to create a multi-agent system that uses a top-level agent to orchestrate a group of agents as tools. Specifically, we will create a system that can research, write, and review a report on a given topic.
This notebook will assume that you have already either read the basic agent workflow notebook or the general agent documentation.
Setup¶
In this example, we will use OpenAI
as our LLM. For all LLMs, check out the examples documentation or LlamaHub for a list of all supported LLMs and how to install/use them.
If we wanted, each agent could have a different LLM, but for this example, we will use the same LLM for all agents.
%pip install llama-index
from llama_index.llms.openai import OpenAI
sub_agent_llm = OpenAI(model="gpt-4.1-mini", api_key="sk-...")
orchestrator_llm = OpenAI(model="o3-mini", api_key="sk-...")
System Design¶
Our system will have three agents:
- A
ResearchAgent
that will search the web for information on the given topic. - A
WriteAgent
that will write the report using the information found by theResearchAgent
. - A
ReviewAgent
that will review the report and provide feedback.
We will then use a top-level agent to orchestrate the other agents to write our report.
While there are many ways to implement this system, in this case, we will use a single web_search
tool to search the web for information on the given topic.
%pip install tavily-python
from tavily import AsyncTavilyClient
async def search_web(query: str) -> str:
"""Useful for using the web to answer questions."""
client = AsyncTavilyClient(api_key="tvly-...")
return str(await client.search(query))
With our tool defined, we can now create our sub-agents.
If the LLM you are using supports tool calling, you can use the FunctionAgent
class. Otherwise, you can use the ReActAgent
class.
from llama_index.core.agent.workflow import FunctionAgent, ReActAgent
research_agent = FunctionAgent(
system_prompt=(
"You are the ResearchAgent that can search the web for information on a given topic and record notes on the topic. "
"You should output notes on the topic in a structured format."
),
llm=sub_agent_llm,
tools=[search_web],
)
write_agent = FunctionAgent(
system_prompt=(
"You are the WriteAgent that can write a report on a given topic. "
"Your report should be in a markdown format. The content should be grounded in the research notes. "
"Return your markdown report surrounded by <report>...</report> tags."
),
llm=sub_agent_llm,
tools=[],
)
review_agent = FunctionAgent(
system_prompt=(
"You are the ReviewAgent that can review the write report and provide feedback. "
"Your review should either approve the current report or request changes to be implemented."
),
llm=sub_agent_llm,
tools=[],
)
With our sub-agents defined, we can then convert them into tools that can be used by the top-level agent.
import re
from llama_index.core.workflow import Context
async def call_research_agent(ctx: Context, prompt: str) -> str:
"""Useful for recording research notes based on a specific prompt."""
result = await research_agent.run(
user_msg=f"Write some notes about the following: {prompt}"
)
state = await ctx.store.get("state")
state["research_notes"].append(str(result))
await ctx.store.set("state", state)
return str(result)
async def call_write_agent(ctx: Context) -> str:
"""Useful for writing a report based on the research notes or revising the report based on feedback."""
state = await ctx.store.get("state")
notes = state.get("research_notes", None)
if not notes:
return "No research notes to write from."
user_msg = f"Write a markdown report from the following notes. Be sure to output the report in the following format: <report>...</report>:\n\n"
# Add the feedback to the user message if it exists
feedback = state.get("review", None)
if feedback:
user_msg += f"<feedback>{feedback}</feedback>\n\n"
# Add the research notes to the user message
notes = "\n\n".join(notes)
user_msg += f"<research_notes>{notes}</research_notes>\n\n"
# Run the write agent
result = await write_agent.run(user_msg=user_msg)
report = re.search(r"<report>(.*)</report>", str(result), re.DOTALL).group(
1
)
state["report_content"] = str(report)
await ctx.store.set("state", state)
return str(report)
async def call_review_agent(ctx: Context) -> str:
"""Useful for reviewing the report and providing feedback."""
state = await ctx.store.get("state")
report = state.get("report_content", None)
if not report:
return "No report content to review."
result = await review_agent.run(
user_msg=f"Review the following report: {report}"
)
state["review"] = result
await ctx.store.set("state", state)
return result
Creating the Top-Level Orchestrator Agent¶
With our sub-agents defined as tools, we can now create our top-level orchestrator agent.
orchestrator = FunctionAgent(
system_prompt=(
"You are an expert in the field of report writing. "
"You are given a user request and a list of tools that can help with the request. "
"You are to orchestrate the tools to research, write, and review a report on the given topic. "
"Once the review is positive, you should notify the user that the report is ready to be accessed."
),
llm=orchestrator_llm,
tools=[
call_research_agent,
call_write_agent,
call_review_agent,
],
initial_state={
"research_notes": [],
"report_content": None,
"review": None,
},
)
Running the Agent¶
Let's run our agents! We can iterate over events as the workflow runs.
from llama_index.core.agent.workflow import (
AgentInput,
AgentOutput,
ToolCall,
ToolCallResult,
AgentStream,
)
from llama_index.core.workflow import Context
# Create a context for the orchestrator to hold history/state
ctx = Context(orchestrator)
async def run_orchestrator(ctx: Context, user_msg: str):
handler = orchestrator.run(
user_msg=user_msg,
ctx=ctx,
)
async for event in handler.stream_events():
if isinstance(event, AgentStream):
if event.delta:
print(event.delta, end="", flush=True)
# elif isinstance(event, AgentInput):
# print("📥 Input:", event.input)
elif isinstance(event, AgentOutput):
# Skip printing the output since we are streaming above
# if event.response.content:
# print("📤 Output:", event.response.content)
if event.tool_calls:
print(
"🛠️ Planning to use tools:",
[call.tool_name for call in event.tool_calls],
)
elif isinstance(event, ToolCallResult):
print(f"🔧 Tool Result ({event.tool_name}):")
print(f" Arguments: {event.tool_kwargs}")
print(f" Output: {event.tool_output}")
elif isinstance(event, ToolCall):
print(f"🔨 Calling Tool: {event.tool_name}")
print(f" With arguments: {event.tool_kwargs}")
await run_orchestrator(
ctx=ctx,
user_msg=(
"Write me a report on the history of the internet. "
"Briefly describe the history of the internet, including the development of the internet, the development of the web, "
"and the development of the internet in the 21st century."
),
)
🛠️ Planning to use tools: ['call_research_agent'] 🔨 Calling Tool: call_research_agent With arguments: {'prompt': 'Write a detailed research note on the history of the internet, covering the development of the internet, the development of the web, and the development of the internet in the 21st century.'} 🔧 Tool Result (call_research_agent): Arguments: {'prompt': 'Write a detailed research note on the history of the internet, covering the development of the internet, the development of the web, and the development of the internet in the 21st century.'} Output: Research Notes on the History of the Internet 1. Development of the Internet: - The internet's origins trace back to the late 1960s with the U.S. Defense Department's Advanced Research Projects Agency Network (ARPANET), designed as a military defense system during the Cold War. - ARPANET was the first network to implement the protocol suite TCP/IP, which became the technical foundation of the modern Internet. - The Network Working Group evolved into the Internet Working Group to coordinate the growing research community. - In the 1970s, commercial packet networks emerged, primarily to provide remote computer access. - The National Science Foundation (NSF) expanded access to the scientific and academic community and helped make TCP/IP the standard for federally supported research networks. - The internet grew through interconnected commercial backbones linked by network access points (NAPs). 2. Development of the World Wide Web: - Invented by Tim Berners-Lee in 1989 while working at CERN, the World Wide Web introduced a "web" of linked information accessible to anyone on the Internet. - By December 1990, Berners-Lee developed the essential tools: HTTP (HyperText Transfer Protocol), HTML (HyperText Markup Language), the first web browser/editor, the first web server, and the first website. - The Web allowed easy access to existing information and linked resources, initially serving CERN scientists. - In 1994, Berners-Lee founded the World Wide Web Consortium (W3C) at MIT to create open standards for the Web. - The Web evolved from Web 1.0 (basic, static pages) to Web 2.0 (interactive, user-generated content) starting around 2003, and further towards Web 3.0 (semantic web, intelligent data) from 2014 onwards. 3. Development of the Internet in the 21st Century: - The 21st century saw transformative developments such as broadband, fiber-optic technology, and mobile internet. - The rise of smartphones revolutionized mobile browsing and internet access. - Cloud computing emerged, allowing data storage and processing on remote servers, changing how businesses and individuals manage information. - The Internet of Things (IoT) connected everyday devices to the internet, expanding the internet's reach into daily life. - Social media platforms became dominant, reshaping communication and information sharing. - The internet's infrastructure and services have continuously evolved to support increasing data demands and new technologies. These notes summarize the key milestones and technological advancements that shaped the internet from its inception to its current state in the 21st century. 🛠️ Planning to use tools: ['call_write_agent'] 🔨 Calling Tool: call_write_agent With arguments: {} 🔧 Tool Result (call_write_agent): Arguments: {} Output: # History of the Internet ## 1. Development of the Internet The origins of the internet date back to the late 1960s with the creation of the Advanced Research Projects Agency Network (ARPANET) by the U.S. Defense Department. Initially designed as a military defense system during the Cold War, ARPANET was the first network to implement the TCP/IP protocol suite, which later became the technical foundation of the modern Internet. The Network Working Group, which coordinated early research efforts, evolved into the Internet Working Group as the research community expanded. During the 1970s, commercial packet networks began to emerge, primarily to provide remote computer access. The National Science Foundation (NSF) played a crucial role by expanding internet access to the scientific and academic communities and promoting TCP/IP as the standard for federally supported research networks. The internet grew further through interconnected commercial backbones linked by network access points (NAPs), facilitating broader connectivity. ## 2. Development of the World Wide Web The World Wide Web was invented in 1989 by Tim Berners-Lee while working at CERN. It introduced a "web" of linked information accessible to anyone on the Internet. By December 1990, Berners-Lee had developed the essential tools that formed the Web's foundation: HTTP (HyperText Transfer Protocol), HTML (HyperText Markup Language), the first web browser/editor, the first web server, and the first website. Initially serving CERN scientists, the Web allowed easy access to existing information and linked resources. In 1994, Berners-Lee founded the World Wide Web Consortium (W3C) at MIT to create open standards for the Web, ensuring its continued growth and interoperability. The Web evolved through several stages: - **Web 1.0:** Basic, static pages. - **Web 2.0:** Starting around 2003, characterized by interactive, user-generated content. - **Web 3.0:** From 2014 onwards, focusing on the semantic web and intelligent data. ## 3. Development of the Internet in the 21st Century The 21st century brought transformative advancements to the internet, including broadband and fiber-optic technologies that significantly increased data transmission speeds. The rise of smartphones revolutionized mobile browsing and internet access, making the internet ubiquitous. Cloud computing emerged as a major innovation, enabling data storage and processing on remote servers, which transformed how businesses and individuals manage information. The Internet of Things (IoT) connected everyday devices to the internet, expanding its reach into daily life. Social media platforms became dominant forces, reshaping communication and information sharing globally. Throughout these developments, the internet's infrastructure and services have continuously evolved to support increasing data demands and new technologies. --- This report summarizes the key milestones and technological advancements that have shaped the internet from its inception in the late 1960s to its current state in the 21st century. 🛠️ Planning to use tools: ['call_review_agent'] 🔨 Calling Tool: call_review_agent With arguments: {} 🔧 Tool Result (call_review_agent): Arguments: {} Output: The report titled "History of the Internet" is well-structured, clear, and provides a concise overview of the major developments in the evolution of the internet. It effectively covers the origins, the invention and growth of the World Wide Web, and significant 21st-century advancements. Strengths: - The chronological organization helps readers follow the progression of internet technology. - Key figures and organizations (e.g., ARPANET, Tim Berners-Lee, NSF, W3C) are appropriately highlighted. - The explanation of Web 1.0, 2.0, and 3.0 stages adds valuable context. - The inclusion of recent technologies such as cloud computing, IoT, and social media reflects current trends. Suggestions for improvement: 1. **Add citations or references:** The report would benefit from citing sources or references to support the historical facts and technological descriptions. 2. **Clarify technical terms:** While the report is generally accessible, briefly defining terms like TCP/IP, NAPs, and semantic web could help readers unfamiliar with networking jargon. 3. **Expand on social impact:** Consider including a brief discussion on how the internet has impacted society, economy, and culture to provide a more holistic view. 4. **Minor formatting:** The section numbering is inconsistent (e.g., "1.", "2.", "3." but no numbering for the introduction or conclusion). Adding a brief introduction and conclusion section with numbering or consistent formatting would improve flow. Overall, the report is informative and well-written. With the suggested enhancements, it would be even more comprehensive and reader-friendly. Recommendation: **Approve with minor revisions** to incorporate citations, clarify terms, and consider adding social impact context. 🛠️ Planning to use tools: ['call_write_agent'] 🔨 Calling Tool: call_write_agent With arguments: {} 🔧 Tool Result (call_write_agent): Arguments: {} Output: # History of the Internet ## 1. Introduction The internet is a transformative technology that has reshaped communication, information sharing, and society at large. This report provides a concise overview of the major developments in the evolution of the internet, from its origins in the late 1960s to the advanced technologies and societal impacts of the 21st century. ## 2. Development of the Internet The origins of the internet date back to the late 1960s with the creation of the Advanced Research Projects Agency Network (ARPANET) by the U.S. Department of Defense. ARPANET was initially designed as a military defense communication system during the Cold War. It was the first network to implement the Transmission Control Protocol/Internet Protocol (TCP/IP), a suite of communication protocols that became the technical foundation of the modern internet. TCP/IP enables different networks to interconnect and communicate seamlessly. During the 1970s, commercial packet-switched networks emerged, primarily to provide remote computer access. The National Science Foundation (NSF) played a crucial role in expanding internet access to the scientific and academic communities and helped establish TCP/IP as the standard protocol for federally supported research networks. The internet's growth was further supported by interconnected commercial backbones linked through Network Access Points (NAPs), which facilitated data exchange between different service providers. ## 3. Development of the World Wide Web In 1989, Tim Berners-Lee, working at CERN, invented the World Wide Web (WWW), which introduced a system of linked information accessible to anyone connected to the internet. By December 1990, Berners-Lee had developed the essential components of the Web: HyperText Transfer Protocol (HTTP), HyperText Markup Language (HTML), the first web browser/editor, the first web server, and the first website. These innovations allowed users to easily access and navigate information through hyperlinks. Initially serving CERN scientists, the Web rapidly expanded to the public. In 1994, Berners-Lee founded the World Wide Web Consortium (W3C) at MIT to develop open standards ensuring the Web's interoperability and growth. The Web has evolved through several stages: - **Web 1.0**: Characterized by static, read-only web pages. - **Web 2.0**: Beginning around 2003, marked by interactive, user-generated content and social media platforms. - **Web 3.0**: Emerging from 2014 onwards, focusing on the semantic web and intelligent data processing to create more personalized and meaningful online experiences. ## 4. Development of the Internet in the 21st Century The 21st century has witnessed transformative advancements in internet technology and infrastructure. Broadband and fiber-optic technologies have significantly increased data transmission speeds. The proliferation of smartphones revolutionized mobile internet access, enabling users to connect anytime and anywhere. Cloud computing emerged as a paradigm shift, allowing data storage and processing on remote servers rather than local devices. This innovation has changed how businesses and individuals manage information and applications. The Internet of Things (IoT) has expanded the internet's reach by connecting everyday devices—such as home appliances, vehicles, and wearable technology—to the network, enabling new functionalities and data-driven services. Social media platforms have become dominant forces in communication and information sharing, reshaping social interactions, marketing, and news dissemination. The internet's infrastructure and services continue to evolve to meet increasing data demands and support emerging technologies. ## 5. Social Impact of the Internet Beyond technological advancements, the internet has profoundly impacted society, the economy, and culture. It has democratized access to information, facilitated global communication, and enabled new forms of social interaction. Economically, it has created new industries, transformed traditional business models, and fostered innovation. Culturally, the internet has influenced media consumption, education, and the way communities form and interact. However, these changes also bring challenges such as privacy concerns, digital divides, misinformation, and cybersecurity threats, which require ongoing attention and management. ## 6. Conclusion The history of the internet is marked by continuous innovation and expansion, from its military origins to a global network integral to modern life. Key figures like Tim Berners-Lee and organizations such as ARPANET, NSF, and W3C have played pivotal roles in its development. Understanding the technical foundations, evolutionary stages of the Web, and recent technological trends provides valuable context for appreciating the internet's role today. Incorporating social impact considerations offers a more holistic view of this transformative technology. --- *Note: This report would benefit from citations to authoritative sources for historical facts and technical explanations to enhance credibility and provide readers with avenues for further research.* The revised report on the history of the internet is now complete and ready for your review. Would you like to access the final report?
With our report written and revised/reviewed, we can inspect the final report in the state.
state = await ctx.store.get("state")
print(state["report_content"])
# History of the Internet ## 1. Introduction The internet is a transformative technology that has reshaped communication, information sharing, and society at large. This report provides a concise overview of the major developments in the evolution of the internet, from its origins in the late 1960s to the advanced technologies and societal impacts of the 21st century. ## 2. Development of the Internet The origins of the internet date back to the late 1960s with the creation of the Advanced Research Projects Agency Network (ARPANET) by the U.S. Department of Defense. ARPANET was initially designed as a military defense communication system during the Cold War. It was the first network to implement the Transmission Control Protocol/Internet Protocol (TCP/IP), a suite of communication protocols that became the technical foundation of the modern internet. TCP/IP enables different networks to interconnect and communicate seamlessly. During the 1970s, commercial packet-switched networks emerged, primarily to provide remote computer access. The National Science Foundation (NSF) played a crucial role in expanding internet access to the scientific and academic communities and helped establish TCP/IP as the standard protocol for federally supported research networks. The internet's growth was further supported by interconnected commercial backbones linked through Network Access Points (NAPs), which facilitated data exchange between different service providers. ## 3. Development of the World Wide Web In 1989, Tim Berners-Lee, working at CERN, invented the World Wide Web (WWW), which introduced a system of linked information accessible to anyone connected to the internet. By December 1990, Berners-Lee had developed the essential components of the Web: HyperText Transfer Protocol (HTTP), HyperText Markup Language (HTML), the first web browser/editor, the first web server, and the first website. These innovations allowed users to easily access and navigate information through hyperlinks. Initially serving CERN scientists, the Web rapidly expanded to the public. In 1994, Berners-Lee founded the World Wide Web Consortium (W3C) at MIT to develop open standards ensuring the Web's interoperability and growth. The Web has evolved through several stages: - **Web 1.0**: Characterized by static, read-only web pages. - **Web 2.0**: Beginning around 2003, marked by interactive, user-generated content and social media platforms. - **Web 3.0**: Emerging from 2014 onwards, focusing on the semantic web and intelligent data processing to create more personalized and meaningful online experiences. ## 4. Development of the Internet in the 21st Century The 21st century has witnessed transformative advancements in internet technology and infrastructure. Broadband and fiber-optic technologies have significantly increased data transmission speeds. The proliferation of smartphones revolutionized mobile internet access, enabling users to connect anytime and anywhere. Cloud computing emerged as a paradigm shift, allowing data storage and processing on remote servers rather than local devices. This innovation has changed how businesses and individuals manage information and applications. The Internet of Things (IoT) has expanded the internet's reach by connecting everyday devices—such as home appliances, vehicles, and wearable technology—to the network, enabling new functionalities and data-driven services. Social media platforms have become dominant forces in communication and information sharing, reshaping social interactions, marketing, and news dissemination. The internet's infrastructure and services continue to evolve to meet increasing data demands and support emerging technologies. ## 5. Social Impact of the Internet Beyond technological advancements, the internet has profoundly impacted society, the economy, and culture. It has democratized access to information, facilitated global communication, and enabled new forms of social interaction. Economically, it has created new industries, transformed traditional business models, and fostered innovation. Culturally, the internet has influenced media consumption, education, and the way communities form and interact. However, these changes also bring challenges such as privacy concerns, digital divides, misinformation, and cybersecurity threats, which require ongoing attention and management. ## 6. Conclusion The history of the internet is marked by continuous innovation and expansion, from its military origins to a global network integral to modern life. Key figures like Tim Berners-Lee and organizations such as ARPANET, NSF, and W3C have played pivotal roles in its development. Understanding the technical foundations, evolutionary stages of the Web, and recent technological trends provides valuable context for appreciating the internet's role today. Incorporating social impact considerations offers a more holistic view of this transformative technology. --- *Note: This report would benefit from citations to authoritative sources for historical facts and technical explanations to enhance credibility and provide readers with avenues for further research.*