In modern AI applications, communication between clients and tools isn’t always as simple as calling an API. The Model Context Protocol (MCP) provides a standardized way for clients to exchange information, invoke tools, and maintain shared context over persistent connections — typically via Server-Sent Events (SSE).

In this post, I’ll walk you through:

• Building an MCP SSE server using FastAPI

• Securing it with JWT authentication

• Implementing a custom Python client to connect, authenticate, and use MCP tools
  We’ll also build a simple BMI Calculator tool to demo tool calling through MCP.

🎛️ Project Overview

We’ll build:

• A FastAPI server that exposes an MCP-compliant SSE endpoint

• A token-based auth system

• Simple tools to get weather and time for a given location

• A Python client that authenticates, connects via SSE, and invokes the tool dynamically

📦 Tech Stack

• Python 3.11+

• Python MCP SDK

• FastAPI

• aiohttp

• PyJWT

• Pydantic

• loguru (for clean logs)

Setting up the MCP SSE Server (server.py)

Let’s first import the relevant libraries and get credentials and keys from env variables

import datetime
import os
from zoneinfo import ZoneInfo
from fastapi import  FastAPI, HTTPException, Request
from pydantic import BaseModel
import requests
from starlette.applications import Starlette
from starlette.routing import Route, Mount
import jwt
from mcp.server.fastmcp import FastMCP
from mcp.server.sse import SseServerTransport
from loguru import logger

from dotenv import load_dotenv

load_dotenv()

Let’s initialize the MCP server and setup the tools

# Initialize the MCP server with your tools
mcp = FastMCP(
    name="Weather and Time SSE Server"
)


@mcp.tool()
def TimeTool(input_timezone):
    "Provides the current time for a given city's timezone like Asia/Kolkata, America/New_York etc. If no timezone is provided, it returns the local time."
    format = "%Y-%m-%d %H:%M:%S %Z%z"
    current_time = datetime.datetime.now()    
    if input_timezone:
        print("TimeZone", input_timezone)
        current_time =  current_time.astimezone(ZoneInfo(input_timezone))
    return f"The current time is {current_time}."

transport = SseServerTransport("/messages/")


@mcp.tool()
def weather_tool(location: str):
    """Provides weather information for a given location"""        
    api_key = os.getenv("OPENWEATHERMAP_API_KEY")
    url = f"http://api.openweathermap.org/data/2.5/weather?q={location}&appid={api_key}&units=metric"
    response = requests.get(url)
    data = response.json()
    if data["cod"] == 200:
        temp = data["main"]["temp"]
        description = data["weather"][0]["description"]
        return f"The weather in {location} is currently {description} with a temperature of {temp}°C."
    else:
        return f"Sorry, I couldn't find weather information for {location}."

We will now setup the JWT auth system and use the starlette routes to essentially expose the app to the clients.

SECRET_KEY = "my_super_secret_key"
ALGORITHM = "HS256"     
def check_auth(request: Request):
    auth = request.headers.get("authorization", "")        
    if auth.startswith("Bearer "):
        token = auth.split(" ", 1)[1]
        try:
            payload = jwt.decode(token, SECRET_KEY, algorithms=[ALGORITHM])
            return True
        except jwt.ExpiredSignatureError:
            raise HTTPException(status_code=401, detail="Token expired")
        except jwt.InvalidTokenError:
            raise HTTPException(status_code=401, detail="Invalid token")

    raise HTTPException(status_code=401, detail="Unauthorized")

async def handle_sse(request):
    check_auth(request=request)
    # Prepare bidirectional streams over SSE
    async with transport.connect_sse(
        request.scope,
        request.receive,
        request._send
    ) as (in_stream, out_stream):
        # Run the MCP server: read JSON-RPC from in_stream, write replies to out_stream
        await mcp._mcp_server.run(
            in_stream,
            out_stream,
            mcp._mcp_server.create_initialization_options()
        )


#Build a small Starlette app for the two MCP endpoints
sse_app = Starlette(
    routes=[
        Route("/sse", handle_sse, methods=["GET"]),
        # Note the trailing slash to avoid 307 redirects
        Mount("/messages/", app=transport.handle_post_message)
    ]
)

Now let’s setup a FastAPI app and add a route for the clients to create a token. We will create a mock client store to simulate credential manager.

app = FastAPI()

# Mock client store
CLIENTS = {
    "test_client": "secret_1234"
}


class TokenRequest(BaseModel):
    client_id: str
    client_secret: str


@app.post("/token")
def generate_token(request: TokenRequest):
    if request.client_id in CLIENTS and CLIENTS[request.client_id] == request.client_secret:
        payload = {
            "sub": request.client_id,
            "exp": datetime.datetime.now() + datetime.timedelta(minutes=60)
        }
        token = jwt.encode(payload, SECRET_KEY, algorithm=ALGORITHM)
        return {"access_token": token}
    else:
        raise HTTPException(status_code=401, detail="Invalid credentials")

Mount the starlette app to FastAPI and setup gunicorn server to run the app.

app.mount("/", sse_app)

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8100)

🔒 Why JWT-Based Auth?

Using client_id / client_secret via POST is scalable:

• You can rotate credentials

• Enforce expiration

• Integrate external OAuth2/OIDC providers

• Track session-level auth

Setting up the MCP SSE Client (client.py)

Setting up client is similar to what we have seen previously in this series. The only change being we generate the token and pass that as headers in the SSE Client.

import asyncio
import json
from typing import Optional
from mcp import ClientSession
from mcp.client.sse import sse_client
from openai import OpenAI
import mcp.client.sse as _sse_mod
from httpx import AsyncClient as _BaseAsyncClient
from loguru import logger
import aiohttp

from dotenv import load_dotenv

load_dotenv()  # load environment variables from .env

import httpx
_orig_request = httpx.AsyncClient.request

async def _patched_request(self, method, url, *args, **kwargs):
    # ensure follow_redirects is set so 307 → /messages/ works
    kwargs.setdefault("follow_redirects", True)
    return await _orig_request(self, method, url, *args, **kwargs)

httpx.AsyncClient.request = _patched_request
def llm_client(message: str):
    client = OpenAI()

    completion = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "You are an intelligent Assistant. You will execute tasks as instructed"},
            {
                "role": "user",
                "content": message,
            },
        ],
    )

    result = completion.choices[0].message.content
    return result



def get_prompt_to_identify_tool_and_arguements(query, tools):
    tools_description = "\n".join([f"{tool.name}: {tool.description}, {tool.inputSchema}" for tool in tools.tools])
    return  ("You are a helpful assistant with access to these tools:\n\n"
                f"{tools_description}\n"
                "Choose the appropriate tool based on the user's question. \n"
                f"User's Question: {query}\n"                
                "If no tool is needed, reply directly.\n\n"
                "IMPORTANT: When you need to use a tool, you must ONLY respond with "                
                "the exact JSON object format below, nothing else:\n"
                "Keep the values in str "
                "{\n"
                '    "tool": "tool-name",\n'
                '    "arguments": {\n'
                '        "argument-name": "value"\n'
                "    }\n"
                "}\n\n")




TOKEN_URL = "http://localhost:8100/token"
SSE_URL = "http://localhost:8100/sse"

async def get_token():
    payload = {"client_id": "test_client", "client_secret": "secret_1234"}
    async with aiohttp.ClientSession() as session:
        async with session.post(TOKEN_URL, json=payload) as resp:
            if resp.status != 200:
                logger.error(f"Failed to get token: {resp.status}")
                raise Exception("Unable to authenticate. Ensure you are using valid credentials")
            data = await resp.json()
            logger.info("Successfully generated token")
            return data["access_token"]

async def main(query:str):        
    try:
        auth_token = await get_token()    
        headers = {"Authorization": f"Bearer {auth_token}"}
        async with sse_client(url=SSE_URL,headers=headers) as (in_stream, out_stream):
            # 2) Create an MCP session over those streams
            async with ClientSession(in_stream, out_stream) as session:
                # 3) Initialize
                info = await session.initialize()
                logger.info(f"Connected to {info.serverInfo.name} v{info.serverInfo.version}")

                # 4) List tools
                tools = (await session.list_tools())
                logger.info(tools)            

                prompt = get_prompt_to_identify_tool_and_arguements(query,tools)
                logger.info(f"Printing Prompt \n {prompt}")

                response = llm_client(prompt)
                print(response)

                tool_call = json.loads(response)

                result = await session.call_tool(tool_call["tool"], arguments=tool_call["arguments"])
                logger.success(f"User query: {query}, Tool Response: {result.content[0].text}")
    except Exception as e:
        print(f"Encountered error: {e}")



if __name__ == "__main__":

    queries = ["What is the time in Bengaluru?", "What is the weather like right now in Dubai?"]
    for query in queries:
        asyncio.run(main(query))

Response when the client is run.

2025-05-18 18:00:48.230 | SUCCESS  | __main__:main:103 - User query: What is the weather like right now in Dubai?, Tool Response: The weather in Dubai is currently clear sky with a temperature of 30.37°C.

In this post, I’ll walk you through how to:

• Create an MCP Server using FastMCP

• Expose a tool that calculates BMI

• Build a Client that communicates with this server via stdio

• Use OpenAI’s GPT model to decide which tool to call and how to call it

Let’s break this down line by line — code and concept.

What is MCP?

Before diving into code, let’s understand what MCP is.

MCP (Model Context Protocol) is an open protocol developed by Anthropic to standardize how LLMs interact with tools. Think of it as the USB-C of AI apps — a universal way to connect and interact with tools, APIs, and services without writing tons of custom glue code.

Here’s a simple analogy:

• USB-C Port: One port to rule them all — display, power, storage.

• MCP: One protocol to access tools — calculators, search, databases, or any custom service.

By adopting MCP, we avoid the pain of writing one-off integration code for every LLM interaction. Instead, we define tools once and let the LLM figure out how to use them.

Courtesy: https://norahsakal.com

🛠️ Project Overview

We’ll build two things:

1. An MCP Server that exposes a simple tool to calculate BMI.

2. An MCP Client that communicates with the server via an LLM (like OpenAI GPT) and invokes the tool.

Let’s get started.

MCP Server

There are just 2 dependencies we need to install

pip install "mcp[cli]"
pip install openai

Let’s create a file called bmi_server.py.

from mcp.server.fastmcp import FastMCP

mcp = FastMCP("BMI Server")

print(f"Starting server {mcp.name}")

@mcp.tool()
def calculate_bmi(weight_kg:float, height_m:float) -> float:
    """
    Calculate BMI given weight in kg and height in meters.
    """
    if height_m <= 0:
        raise ValueError("Height must be greater than zero.")
    return weight_kg / (height_m ** 2)


if __name__ == "__main__":
    mcp.run(transport="stdio")

What’s happening here?

• We use the FastMCP class to create the MCP server

• We create a BMI tool using the @mcp.tool()decorator.

• It takes weight and height and returns BMI.

• The server exposes this tool using standard input/output transport.

• We want this file to be run independently and not as a module

Obviously, this is a fundamental MCP server, but this should get us started on building our own MCP client.

MCP Client

Now, let’s create the client in a file named bmi_client.py.

Import all the dependencies. The key ones here being ClientSession, StdioServerParameters and stdio_client from the mcp package and OpenAI class to communicate with the LLM.

import asyncio
from openai import OpenAI
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
import os
import json

We need to now establish a way to communicate with the server we just wrote. Let’s use the StdioServerParameters class from the mcp package to do that.

server_params = StdioServerParameters(command="python", args=["bmi-server.py"])

Essentially, we are telling the client what command to run and how to use the server file.

Let’s write a generic method that to communicate with an LLM API. This should simply take a prompt and return a response.

def llm_client(message:str):
    """
    Send a message to the LLM and return the response.
    """
    # Initialize the OpenAI client
    openai_client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

    # Send the message to the LLM
    response = openai_client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role":"system",
                    "content":"You are an intelligent assistant. You will execute tasks as prompted",
                    "role": "user", "content": message}],
        max_tokens=250,
        temperature=0.2
    )

    # Extract and return the response content
    return response.choices[0].message.content.strip()

We need to write a prompt that does 2 things

1. Share the context with the LLM about the tools it has at its disposal
2. Instruct a structured output so that we can execute the tool the LLM selected

def get_prompt_to_identify_tool_and_arguments(query,tools):
    tools_description = "\n".join([f"- {tool.name}, {tool.description}, {tool.inputSchema} " for tool in tools])
    return  ("You are a helpful assistant with access to these tools:\n\n"
                f"{tools_description}\n"
                "Choose the appropriate tool based on the user's question. \n"
                f"User's Question: {query}\n"                
                "If no tool is needed, reply directly.\n\n"
                "IMPORTANT: When you need to use a tool, you must ONLY respond with "                
                "the exact JSON object format below, nothing else:\n"
                "Keep the values in str "
                "{\n"
                '    "tool": "tool-name",\n'
                '    "arguments": {\n'
                '        "argument-name": "value"\n'
                "    }\n"
                "}\n\n")

We are passing two arguments - the original query from the user and the list of tools from the server(s). Let’s now see how get the list of tools from the server.

async def run(query: str):
    async with stdio_client(server_params) as (read, write):
        async with ClientSession(read,write) as session:

            await session.initialize()

            # Get the list of available tools
            tools = await session.list_tools()

            print(f"Available tools: {tools}")

            prompt = get_prompt_to_identify_tool_and_arguments(query,tools.tools)

            llm_response = llm_client(prompt)
            print(f"LLM Response: {llm_response}")

            tool_call = json.loads(llm_response)

            result = await session.call_tool(tool_call["tool"], arguments=tool_call["arguments"])

            print(f"BMI for weight {tool_call["arguments"]["weight_kg"]}kg and height {tool_call["arguments"]["height_m"]}m is {result.content[0].text}")

Key things happening here:

• We start the BMI Server process using stdio_client, and get access to its read and write streams

• Next, we create a session and initialize it. This sets up the communication between our client and the server.

• We then ask the server for a list of all available tools. In our case, it will return the calculate_bmi function we exposed earlier.

• “We prepare a detailed prompt using the earlier function, passing in the user's query and available tools. This prompt helps the language model figure out what tool to call and what arguments to pass.

• We parse the JSON response, extract the tool name and arguments, and then call the tool using session.call_tool.

• Finally, we return the output from the tool call — in this case, the calculated BMI.

To make all of the above work, we will add an entry point. When this script is run, it sets the user query — here we ask to calculate BMI — and runs the run() function using asyncio. The final result is printed on screen!

if __name__ == "__main__":
    import asyncio
    query = "Calculate BMI for height 5ft 10inches and weight 80kg"
    print(f"Sending query: {query}")
    result = asyncio.run(run(query))
    print(f"Result: {result}")

And that’s how you build an intelligent MCP client that can call tools dynamically using OpenAI and a BMI server. The magic lies in combining tool discovery, LLM prompting, and tool invocation — all within a simple and elegant flow.

What Are Guardrails in AI Agents?

Guardrails act as a safety net for AI agents, ensuring they operate within predefined boundaries and preventing misuse.

They work alongside agents, validating user inputs and outputs to safeguard against errors and inappropriate responses.

There are two types of guardrails:

• Input Guardrails: Validate user inputs before processing.

• Output Guardrails: Ensure the final response is appropriate before delivering it to the user.

Let’s see them in action!

Input Guardrails

• These validate initial user inputs before passing them to expensive models.

• They operate in three steps: receiving input, running validation functions, and triggering errors if misuse is detected.

Input guardrails are mechanisms put in place to validate, sanitize, and preprocess user inputs before they reach an AI model. These safeguards help in preventing:

• Malicious injections (e.g., prompt injection attacks)

• Profanity, hate speech, and harmful language

• Unstructured or irrelevant input that reduces model efficiency

• Bias amplification

By implementing input guardrails, developers can ensure that AI models receive well-structured and appropriate input, leading to better and safer outputs.

Why Are Input Guardrails Important?

1. Security: Prevents prompt injections, SQL injections, and adversarial attacks.

2. Quality Assurance: Filters out irrelevant or poorly structured queries.

3. Bias Mitigation: Helps remove explicit bias in prompts.

4. User Experience: Ensures clear and understandable input for meaningful responses.

5. Compliance: Adheres to ethical AI principles and regulatory requirements.

How to Implement Input Guardrails

Create the guardrail agent as below. Use the @input_guardrail decorator for the guardrail method.

More here in the documentation.

Input guardrails run in 3 steps:

1. First, the guardrail receives the same input passed to the agent.

2. Next, the guardrail function runs to produce a GuardrailFunctionOutput, which is then wrapped in an InputGuardrailResult

3. Finally, we check if .tripwire_triggered is true. If true, an InputGuardrailTripwireTriggered exception is raised, so you can appropriately respond to the user or handle the exception.

Pass the guardrail as an argument to the triage agent.

Handle the InputGuardrailTripwireTriggered exception.

Exception raised when the input is “Why is my order delayed? You guys are pathetic“

Output Guardrails

• These validate the final outputs generated by agents before they are delivered to users.

• They operate similarly to input guardrails but focus on the output stage to ensure accuracy and safety.

Output guardrails run in 3 steps:

1. First, the guardrail receives the same input passed to the agent.

2. Next, the guardrail function runs to produce a GuardrailFunctionOutput, which is then wrapped in an OutputGuardrailResult

3. Finally, we check if .tripwire_triggered is true. If true, an OutputGuardrailTripwireTriggered exception is raised, so you can appropriately respond to the user or handle the exception.

In the example below - we want to check for the “card” in the response of the order_agent and raise an exception accordingly.

Add the output guardrail as the argument to the final agent in the workflow.

We simulated the response of the order_agent for order 12346 to contain the word “card” and this is how the exception is caught.

Code

https://github.com/zahere-dev/openai-agents-sdk-tutorial

Conclusion:

• Guardrails are vital components of AI agent systems, ensuring they operate safely and efficiently.

• By implementing guardrails, developers can enhance user trust and prevent misuse scenarios effectively.

The Problem at Hand

Uber's data platform processes approximately 1.2 million interactive queries monthly, with 36% of these coming from the operations organization. This group—comprising engineers, data scientists, and operations professionals—analyzes data from hundreds of thousands of tables across various domains to derive actionable insights.

However, the process of composing and executing queries was a bottleneck:

• 10 minutes per query: Each actor spent an average of 10 minutes composing a query.

• Inefficiency Loop: Users would sift through datasets, run queries, and validate results in a repetitive cycle.

• Wasted Time: The cumulative effect of this inefficiency led to significant lost productivity.

This challenge is not unique to Uber. It resonates across industries, from e-commerce to customer support, where operations teams grapple with similar inefficiencies.

Enter QueryGPT: The Hackathon Solution

In 2023, a team at Uber's hackathon introduced QueryGPT, a prototype designed to streamline the query-generation process. Here's how it worked:

1. Metadata-Driven Query Generation:

   • Stored 20 SQL queries with metadata (table and schema information).

   • Mapped natural language prompts to these queries.

2. Few-Shot Prompting:

   • Used a Retrieval-Augmented Generation (RAG) technique to fetch relevant queries.

   • Generated SQL queries in response to user prompts.

3. Initial Results:

   • Reduced query composition time from 10 minutes to 3 minutes.

   • Achieved an 18% productivity gain.

While this was a promising start, the prototype faced scalability and technical challenges, necessitating further iterations.

Challenges and Iterative Solutions

Key Challenges

1. Prompt-to-Schema Mismatch:

   • The system struggled to align user prompts with relevant schemas.

2. Token Limitations:

   • Some schemas had over 200 columns, leading to token counts exceeding GPT-4's 32k limit.

The Final Architecture

The refined system, powered by Azure OpenAI and GPT-4, demonstrated remarkable efficiency:

• Context Optimization: Leveraged a context window of 128k tokens to handle large schemas.

• Human Validation: Ensured precision through user acknowledgment of suggested tables.

• Scalable Design: Addressed the challenge of querying across hundreds of thousands of datasets.

Uber's engineering team implemented a robust architecture combining SQL, RAG, agents, and custom configurations. Here's a breakdown:

1. Domain-Specific Curation:

   • Decomposed datasets into business domains/workflows (e.g., mobility, trips, support).

   • Allowed the system to focus on smaller, relevant subsets of data.

2. Intent Agent:

   • Classified user prompts to map them to the appropriate domain or workspace.

   • Likely employed a vector-store-based intent classifier for high accuracy.

3. Table Agent:

   • Retrieved domain-specific tables and displayed them in a user-friendly interface.

   • Enabled human-in-the-loop validation to ensure table relevance.

4. Enhanced RAG Pipeline:

   • Generated few-shot prompts tailored to the specific domain.

   • Sent refined prompts to GPT-4 for SQL query generation.

Real-World Impact

By the 20th iteration, Uber's Query GPT achieved a staggering 140,000 hours saved monthly across its operations organization. This success underscores the value of combining AI, domain-specific curation, and user-centric design.

Key Takeaways for Your Business

Uber's solution offers valuable insights for tackling similar challenges in other industries:

1. Break Down Data Silos:

   • Organize datasets by business domains to streamline data retrieval.

2. Implement Intent Detection:

   • Use AI-driven agents to map user queries to relevant datasets or workspaces.

3. Leverage Human-in-the-Loop Systems:

   • Involve users in validating AI-generated outputs for enhanced accuracy.

4. Iterate for Scalability:

   • Start small, learn from challenges, and scale iteratively.

The Future of AI in Operations

Uber's journey with QueryGPT exemplifies the transformative potential of generative AI in operational analytics. By reducing manual effort and empowering teams with intelligent tools, businesses can unlock unprecedented productivity gains.

Whether you're in e-commerce, customer support, or any data-intensive field, the principles behind Uber's success can guide your own AI-driven innovations.

Want to delve deeper into the technical details? Check out Uber's engineering blog here for the full story.

Google's recent white paper on "Agents" has created quite a buzz.

The paper explores the concept of AI agents and delves into their architecture and potential. Let's break down what this white paper offers, its key takeaways, and some areas where it could improve.

The Marketing Angle: A Platform-Centric View?

At first glance, the white paper feels like a marketing tool for Google's Vertex AI. And that's perfectly fine—after all, companies often use such publications to showcase their platforms.

However, adopting a more platform-agnostic approach could have made the paper more universally applicable.

For instance, many examples in the white paper are tied to Vertex AI-specific features, which might be unfamiliar to those using other agentic frameworks.

Additionally, certain concepts, like extensions, are introduced but not elaborated on in sufficient detail, leaving room for better documentation and clarity.

Despite these limitations, the paper provides a solid starting point for understanding agents. Let’s dive into the key concepts.

What is an Agent?

Google defines an agent as:

"An application that attempts to achieve a goal by observing the world and acting upon it using the tools at its disposal."

This definition is both simple and powerful. It captures the essence of what an agent is without overcomplicating things. While many experts on platforms like LinkedIn and YouTube often layer terms like reasoning, context awareness, and more onto the definition, the core idea remains straightforward.

Interestingly, my personal favorite definition comes from Hugging Face, which describes AI agents as:

"Programs where LLM outputs control the workflows."

This succinctly highlights the operational dynamics of agents, especially when integrated with language models.

The Agentic Architecture: Core Components

The white paper also details the architecture of agents, a topic I’ve previously discussed on my channel. Here's a simplified breakdown of the three primary components that define an agentic system:

1. The Model

At the heart of any agent lies a language model. This serves as the foundation for the agent's intelligence and capabilities. Trained on extensive datasets, the model enables the agent to comprehend language, process instructions, and provide knowledge.

In an agentic framework, the model is not just a passive responder. Its capabilities drive the decision-making processes within the orchestration layer.

2. The Tools

Tools are what set agents apart from simple LLM calls. Since LLMs are inherently limited—they can’t interact with external systems or access real-time information—tools extend their capabilities.

Agents use tools to interact with the external world, making them more dynamic and useful. Frameworks like LangChain and LlamaIndex exemplify how tools can augment the performance of agentic systems, enabling them to achieve their goals effectively.

3. The Orchestration Layer

Often referred to as the reasoning loop, this layer governs the agent's ability to:

• Plan: Decide the next steps in a workflow.

• Reason: Analyze the gathered information.

• Execute: Take action based on the plan.

This iterative process is the backbone of an agent’s functionality, ensuring it can adapt and respond intelligently to various scenarios.

Tools (Key Takeaways)

1. Extensions

   • Definition: Extensions are interfaces that bridge the gap between APIs and agents. They allow for seamless API execution by teaching agents how to use APIs via examples.

   • Use Case: Extensions are ideal for scenarios where the agent needs to dynamically interact with APIs like booking flights or fetching weather data. They reduce ambiguity in API calls by guiding the agent with context and examples.

   • Your Example: Your custom tool implementation fits here because it defines tools with a name and description, guiding the LLM to invoke the correct tool and arguments based on context.

2. Functions

   • Definition: Functions are reusable logic modules that allow developers to define behavior and handle specific tasks.

   • Difference: Unlike extensions, functions offload API execution to client-side logic or middleware, especially in cases where security or authentication constraints prevent direct calls from the LLM.

   • Your Observations: Google's distinction clarifies that functions give developers fine-grained control and decouple execution from the agent, making iteration easier without redeploying infrastructure.

As we approach 2025, there are rapid advancements in technology that we, as engineers, cannot afford to ignore. Whether you’re looking to add value to your organization, grow your skillset, or simply future-proof your career, there are certain core skills that will set you apart.

Let’s dive right in.

The New Baseline: AI and Business Context Understanding

Gone are the days when being a full-stack engineer meant just knowing how to develop and deploy applications. Today, understanding the "why" behind what you’re building is just as important as the "how."

If you find yourself working on user stories without knowing the business context or understanding the decisions made by your engineering or product managers, it’s time to rethink your approach.

Business understanding is what differentiates an average engineer from a great one.

And as AI continues to integrate into every facet of technology, this understanding will become even more critical.

Here’s my take on the essential skills every full-stack engineer should master to thrive in this new era.

1. Core AI/ML Engineering Skills

Large Language Models (LLMs)

By now, you’ve probably experimented with tools like ChatGPT or integrated APIs into your projects. But to truly excel, you need to:

• Understand how LLMs work at a deeper level.

• Learn about fine-tuning models for specific use cases.

• Master prompt engineering – arranging instructions in ways that yield the best results.

Retrieval-Augmented Generation (RAG)

RAG will remain indispensable for enterprise applications, even as LLMs handle increasingly large context windows. Why? Because RAG ensures that only the most relevant data is fed into the model, optimizing performance and cost. Gaining expertise in designing RAG pipelines and integrating them with business workflows is a must.

AI Agents

The buzzword for 2025 is "agents." Multi-agent frameworks and systems are set to revolutionize how we build solutions. Understanding how to design and orchestrate these systems will keep you ahead of the curve.

2. Technical Stack Mastery

Python: The Go-To Language

Python’s ecosystem for AI and ML is unparalleled. From frameworks like PyTorch and scikit-learn to tools like LangChain and LlamaIndex, Python should be in every engineer’s toolkit. If you’re new to the space, start experimenting with these frameworks.

Cloud Platforms and Vector Databases

• Familiarize yourself with cloud platforms like AWS, Azure, or GCP, and tools like VMware Tanzu.

• Learn about vector databases, such as PGVector, which allow efficient storage and retrieval of embeddings. Even spinning up a simple Docker instance can give you hands-on experience.

API Development

Frameworks like FastAPI and Flask make it easier to create and deploy AI-powered web applications. Combine this with an understanding of socket programming for real-time communication tools like chatbots, and you’ll be unstoppable.

3. Project and Professional Skills

Business Context Awareness

This is the game-changer. Knowing how your application aligns with business goals will not only make you a better engineer but also ensure your contributions are recognized. Always ask:

• Why is this feature important?

• How will this decision impact the user or business?

Experimentation Over Perfection

Start small. Use APIs to create prototypes and simulate workflows. For instance:

• Identify areas in your organization where generative AI can add value.

• Build mock solutions with sample data to prove concepts.

MLOps and Deployment

Understanding MLOps tools and practices is becoming essential for deploying AI solutions. Even if you’re not directly managing infrastructure, knowing how to streamline deployment pipelines will make you invaluable to your team.

My Journey: From Experimentation to Mastery

When I transitioned into AI engineering in early 2023, I didn’t start by mastering machine learning fundamentals. Instead, I began tinkering with LLM APIs and building prototypes. This hands-on experimentation allowed me to solve real-world problems while gradually deepening my knowledge of ML fundamentals over the next six months.

This approach worked wonders for me, and I recommend it to anyone looking to enter the field. Don’t get bogged down by theory. Instead, balance learning with application. Keep a 60/40 or even 50/50 ratio between theory and practice to maximize your growth.

Final Thoughts

The expectations for full-stack engineers are evolving, and AI skills are no longer optional.

By mastering core AI/ML engineering concepts, staying updated with the latest frameworks, and understanding business context, you’ll position yourself as a leader in your field.

So, what’s your game plan for 2025? Let me know in the comments below. And if you’re ready to start tinkering, I’ve got tons of tutorials on my channel to help you get started. Until next time, happy coding!

Dynamic prompt generation has become a cornerstone of modern AI workflows.

Whether you're building personalized email campaigns, travel itineraries, or AI-driven recommendations, the ability to generate structured content dynamically is invaluable.

In this blog, we'll explore how Jinja2, a powerful templating engine, stands out in this domain and compare it with tools like LangChain for crafting dynamic prompts.

Why Dynamic Prompting Matters

If you manage an AI assistant tasked with creating personalized travel itineraries or summarizing user activities, static prompts won't cut it here – you need templates that adapt to the data at hand. This is where tools like Jinja2 and LangChain's prompt templates shine.

Jinja2: The All-Rounder for Dynamic Templates

Jinja2 is a versatile templating engine widely known for its use in web development but equally adept at generating dynamic text for emails, reports, and prompts. Here's why Jinja2 should be in your toolkit:

1. Seamless Integration of Logic

Jinja2 allows you to embed loops, conditionals, and filters directly in your templates. For example, creating tailored recommendations becomes straightforward:

Dear {{ user_name }},

Here’s a summary of your recent activities:
{% for activity in activities %}
- On {{ activity.date }}: {{ activity.description }}
{% endfor %}

{% if status == "pass" %}
Congratulations on passing the test. Keep up the great work!
{% else %}
Keep trying, and you'll get there!
{% endif %}

2. Readable and Reusable

With its clean syntax, Jinja2 makes templates easy to maintain and reuse across projects. It's perfect for use cases like:

• Personalized emails

• Travel itineraries

• AI-driven content generation

3. Performance Efficiency

Jinja2 minimizes overhead, making it an excellent choice for applications requiring rapid dynamic rendering.

Comparing Jinja2 and LangChain for Prompt Templates

While Jinja2 excels in general-purpose dynamic content generation, LangChain's PromptTemplate is specifically designed for AI workflows, making it the go-to for LLM integrations.

LangChain Example:

from langchain.prompts import PromptTemplate

template = """
Dear {user_name},

Here’s a summary of your recent activities:
{activities}

Here are some tailored recommendations for you:
{recommendations}

{closing_note}
"""

prompt = PromptTemplate(
    input_variables=["user_name", "activities", "recommendations", "closing_note"],
    template=template,
)

email = prompt.format(
    user_name="Alice",
    activities="- Completed the Python course.\n- Joined the AI workshop.",
    recommendations="- Read 'Deep Learning for Beginners'.\n- Join the Advanced AI Projects Club.",
    closing_note="Congratulations on passing the test!",
)

print(email)

Key Differences:

• Flexibility: Jinja2 supports complex logic directly in the template, while LangChain separates logic and content.

• AI Integration: LangChain is optimized for workflows where prompts are fed into LLMs.

• Learning Curve: Jinja2 has a gentler curve for general developers, whereas LangChain is ideal for those already in the AI ecosystem.

Real-World Use Case: Personalized Travel Itineraries

Using Jinja2, you can craft luxurious travel experiences tailored to user preferences. Here's an example:

Input Data

codeinput_data = {
    "name": "John Doe",
    "destination": "Paris, France",
    "interests": ["art", "history", "fine dining"],
    "travel_type": "luxury",
    "suggested_activities": [
        {"name": "Private Louvre Tour", "description": "Explore iconic art pieces with a guide."},
        {"name": "Seine River Dinner Cruise", "description": "Enjoy a gourmet dinner on a Seine cruise."},
    ],
    "recommended_accommodations": [
        {"name": "Le Meurice", "description": "5-star luxury hotel with Michelin dining."},
    ],
}

Jinja2 Template

 """
Traveler Profile:
- Name: {{ name }}
- Age: {{ age }}
- Travel Dates: {{ travel_dates }}
- Travel Destination: {{ destination }}
- Interests: {{ interests|join(", ") }}

{% if travel_type == 'luxury' %}
The traveler prefers a luxury experience. Suggest the following premium activities and accommodations:
{% elif travel_type == 'adventure' %}
The traveler seeks adventure. Recommend these thrilling activities and adventurous destinations:
{% else %}
The traveler is interested in a balanced experience. Consider these activities and attractions:
{% endif %}

{% for activity in suggested_activities %}
- {{ activity.name }}: {{ activity.description }}
  {% if activity.requirements %}
  Requirements: {{ activity.requirements|join(", ") }}
  {% endif %}
{% endfor %}

{% if recommended_accommodations %}
Recommended Accommodations:
{% for accommodation in recommended_accommodations %}
- {{ accommodation.name }}: {{ accommodation.description }}
  Location: {{ accommodation.location }}
  Amenities: {{ accommodation.amenities|join(", ") }}
{% endfor %}
{% endif %}

Traveler's Notes:
{% if traveler_notes %}
{% for note in traveler_notes %}
- {{ note }}
{% endfor %}
{% endif %}

Based on the above information, create a 3-day itinerary tailored to the traveler’s preferences and needs. Ensure activities, meals, and downtime are appropriately balanced.
""

Output

Traveler Profile:
- Name: John Doe
- Age: 35
- Travel Dates: 2024-01-15 to 2024-01-18
- Travel Destination: Paris, France
- Interests: art, history, fine dining, luxury shopping


The traveler prefers a luxury experience. Suggest the following premium activities and accommodations:



- Private Louvre Tour: Enjoy a private, guided tour of the Louvre Museum, exploring its iconic art pieces.

  Requirements: Comfortable walking shoes, Museum pass


- Seine River Dinner Cruise: Experience a luxurious evening with a gourmet dinner on a Seine River cruise.

  Requirements: Formal attire


- Champs-Élysées Shopping Tour: Indulge in a day of shopping at high-end boutiques along the Champs-Élysées.




Recommended Accommodations:

- Le Meurice: A 5-star luxury hotel offering Michelin-star dining and exceptional service.
  Location: Rue de Rivoli, Paris
  Amenities: Spa, Fine dining, Concierge service

- Hôtel Plaza Athénée: Iconic Parisian hotel with stunning views of the Eiffel Tower.
  Location: Avenue Montaigne, Paris
  Amenities: Luxury suites, Haute couture stores, Gourmet restaurants



Traveler's Notes:


- Has dietary restrictions: no shellfish.

- Prefers private tours over group activities.



Based on the above information, create a 3-day itinerary tailored to the traveler’s preferences and needs. Ensure activities, meals, and downtime are appropriately balanced.

Conclusion

If you're building general-purpose templates or working with structured data, Jinja2 is a clear winner. For AI-centric workflows, LangChain simplifies the integration with LLMs.

Ready to take your prompts to the next level? Dive into Jinja2 or LangChain and unlock the full potential of dynamic content creation!

Whether you're tracking a personal wishlist or monitoring competitors' pricing for your business, automating this process can save time and effort.

In this guide, we’ll explore how to build a price monitoring agent using the Pydantic AI framework—a robust agentic framework from the creators of Pantic, a popular data validation library.

This tutorial is part one of a series. Today, we’ll focus on building a scraper agent to extract key product details like title, description, price, and more.

In the next part, we’ll expand this agent to store data in a database and send notifications for price changes.

What is Pydantic AI?

Pydantic AI is revolutionizing the way developers build applications that leverage Generative AI. As a Python Agent Framework, it simplifies the creation of production-grade applications by integrating robust data validation with the power of LLMs. Here’s why Pydantic AI stands out:

• Built on Proven Foundations: Developed by the creators of Pydantic, which is widely used in various AI frameworks like OpenAI and LangChain, Pydantic AI inherits a strong legacy of type safety and structured data management.

• Model-Agnostic Flexibility: Currently supporting models like OpenAI, Gemini, and Groq, Pydantic AI allows developers to easily implement support for additional models through a simple interface. This flexibility ensures that your application can adapt to various AI technologies without significant overhead.

• Enhanced Developer Experience: With features like vanilla Python control flow and a novel dependency injection system, Pydantic AI empowers developers to apply familiar coding practices. This leads to more maintainable code and a smoother development process.

• Streamlined Response Validation: The framework not only validates incoming data but also ensures that responses from LLMs are structured and validated, enhancing reliability in application behavior.

Overview of the Price Monitoring Agent

Our agent will:

1. Scrape product details (title, description, price, currency, and image URL) from a given URL.

2. Parse the information into a structured format.

3. Prepare for database storage and notification handling (to be implemented in part two).

Here’s how the process works (diagram above)

1. Input: Product page URL

2. Scraper Tool: Extracts structured data using Beautiful Soup and Markdownify.

3. Agent: Processes the scraped data using Pydantic AI for type-safe responses.

Watch the video for full tutorial

This post is an extension of my earlier guide, "Building an AI Agent from Scratch."

If you’re new here, I recommend revisiting that post to get up to speed, as we’ll build upon its concepts and code.

In this project, we’ll tackle orchestrating actions between multiple agents, enabling seamless execution of tasks such as fetching weather information and the current time. Let’s jump in!

What Is a Multi-Agent Orchestrator?

A Multi-Agent Orchestrator is a system that:

1. Identifies the intent of a user’s input.

2. Selects the appropriate agent to handle the request.

3. Executes tasks using tools associated with the agent.

Think of it as a manager assigning tasks to specialized team members. This orchestration ensures complex queries involving multiple tasks are handled efficiently.

What We’ll Build

We’ll create:

• Agents: Specialized entities to handle tasks like fetching weather or time.

• Tools: Functional utilities for the agents, such as APIs or database queries.

• Orchestrator: The central system managing task delegation and execution.

Key Components of an Agent

An agent has three main components:

1. Reasoning Loop: Decides the next action based on context.

2. Model: Uses a language model (LLM) for decision-making.

3. Tools: A list of utilities to perform specific tasks.

Our agents will dynamically decide which tool to use, making them highly adaptable.

The Agent Class

Here’s a high-level breakdown of the Agent class:

• Constructor: Initializes the agent with a name, description, tools, and an LLM model.

• Process Input: Takes user input, decides on a tool, and executes the task.

• Prompting: Constructs a prompt for the LLM to guide decision-making.

Agents also handle parsing JSON responses from the LLM to ensure smooth execution.

from abc import ABC, abstractmethod
import ast
import os
import requests
from llm.llm_ops import query_llm
from tools.base_tool import Tool
import json


class Agent:
    def __init__(self, Name: str, Description: str, Tools: list, Model: str):        
        self.memory = []
        self.name = Name
        self.description = Description
        self.tools = Tools
        self.model = Model
        self.max_memory = 10

    def json_parser(self, input_string):

      print(type(input_string))

      python_dict = ast.literal_eval(input_string)
      json_string = json.dumps(python_dict)
      json_dict = json.loads(json_string)

      if isinstance(json_dict, dict) or isinstance(json_dict,list):
        return json_dict

      raise "Invalid JSON response"

    def process_input(self, user_input):
        self.memory.append(f"User: {user_input}")
        12

        context = "\n".join(self.memory)
        tool_descriptions = "\n".join([f"- {tool.name()}: {tool.description()}" for tool in self.tools])
        response_format = {"action":"", "args":""}

        prompt = f"""Context:
        {context}

        Available tools:
        {tool_descriptions}

        Based on the user's input and context, decide if you should use a tool or respond directly.        
        If you identify a action, respond with the tool name and the arguments for the tool.        
        If you decide to respond directly to the user then make the action "respond_to_user" with args as your response in the following format.

        Response Format:
        {response_format}

        """

        response = query_llm(prompt)
        self.memory.append(f"Agent: {response}")

        response_dict = self.json_parser(response)

        # Check if any tool can handle the input
        for tool in self.tools:
            if tool.name().lower() == response_dict["action"].lower():
                return tool.use(response_dict["args"])

        return response_dict

The Orchestrator

The orchestrator coordinates multiple agents:

1. Accepts user input.

2. Selects the right agent based on the intent.

3. Manages task execution, including cases where multiple tasks are requested.

Core Features of the Orchestrator:

• Maintains context by storing user queries, agent responses, and intermediate results.

• Uses a reasoning loop to determine the next steps.

• Constructs prompts to guide the LLM in selecting the right agent and tools.

import ast
import json
from llm.llm_ops import query_llm
from agents.base_agent import Agent
from logger import log_message

class AgentOrchestrator:
    def __init__(self, agents: list[Agent]):
        self.agents = agents
        self.memory = []  # Stores the reasoning and action steps taken
        self.max_memory = 10

    def json_parser(self, input_string):

      print(type(input_string))

      python_dict = ast.literal_eval(input_string)
      json_string = json.dumps(python_dict)
      json_dict = json.loads(json_string)

      if isinstance(json_dict, dict) or isinstance(json_dict,list):
        return json_dict

      raise "Invalid JSON response"

    def orchestrate_task(self, user_input: str):        
        self.memory = self.memory[-self.max_memory:]

        context = "\n".join(self.memory)

        print(f"Context: {context}")

        response_format = {"action":"", "input":"", "next_action":""}

        def get_prompt(user_input):
            return f"""

                Use the context from memory to plan next steps.                
                Context:
                {context}

                You are an expert intent classifier.
                You need will use the context provided and the user's input to classify the intent select the appropriate agent.                
                You will rewrite the input for the agent so that the agent can efficiently execute the task.                                                

                Here are the available agents and their descriptions:
                {", ".join([f"- {agent.name}: {agent.description}" for agent in self.agents])}

                User Input:
                {user_input}              

                ###Guidelines###
                - Sometimes you might have to use multiple agent's to solve user's input. You have to do that in a loop.
                - The original userinput could have multiple tasks, you will use the context to understand the previous actions taken and the next steps you should take.
                - Read the context, take your time to understand, see if there were many tasks and if you executed them all
                - If there are no actions to be taken, then make the action "respond_to_user" with your final thoughts combining all previous responses as input.
                - Respond with "respond_to_user" only when there are no agents to select from or there is no next_action
                - You will return the agent name in the form of {response_format}
                - Always return valid JSON like {response_format} and nothing else.                

                """


        response = ""
        loop_count = 0
        self.memory = self.memory[-10:]        
        prompt = get_prompt(user_input)
        llm_response = query_llm(prompt)

        llm_response = self.json_parser(llm_response)
        print(f"LLM Response: {llm_response}")

        self.memory.append(f"Orchestrator: {llm_response}")


        action=  llm_response["action"]
        user_input = llm_response["input"]

        print(f"Action identified by LLM: {action}")


        if action == "respond_to_user":
            return llm_response
        for agent in self.agents:
            if agent.name == action:
                print("*******************Found Agent Name*******************************")
                agent_response = agent.process_input(user_input)
                print(f"{action} response: {agent_response}")
                self.memory.append(f"Agent Response for Task: {agent_response}")
                print(self.memory)
                return agent_response                


    def run(self):
        print("LLM Agent: Hello! How can I assist you today?")
        user_input = input("You: ")
        self.memory.append(f"User: {user_input}")

        while True:            
            if user_input.lower() in ["exit", "bye", "close"]:
                print("See you later!")
                break

            response = self.orchestrate_task(user_input)
            print(f"Final response of orchestrator {response}")
            if isinstance(response, dict) and response["action"] == "respond_to_user":                
                log_message(f"Reponse from Agent: {response["input"]}", "RESPONSE")
                user_input = input("You: ")
                self.memory.append(f"User: {user_input}")                
            elif response == "No action or agent needed":
                print("Reponse from Agent: ", response)
                user_input = input("You: ")
            else:
                user_input = response

Tools in Action

Agents use tools to perform tasks. For example:

• Weather Tool: Fetches real-time weather data from OpenWeatherMap.

• Time Tool: Determines the local time for a given city, even without a timezone.

Each tool includes:

• A name and description to guide the LLM.

• A use method to perform the task.

import os
import requests
from tools.base_tool import Tool

class WeatherTool(Tool):
    def name(self):
        return "Weather Tool"

    def description(self):
        return "Provides weather information for a given location. The payload is just the location. Example: New York"

    def use(self, location:str):        
        api_key = os.getenv("OPENWEATHERMAP_API_KEY")
        url = f"http://api.openweathermap.org/data/2.5/weather?q={location}&appid={api_key}&units=metric"
        response = requests.get(url)
        data = response.json()
        if data["cod"] == 200:
            temp = data["main"]["temp"]
            description = data["weather"][0]["description"]
            response = f"The weather in {location} is currently {description} with a temperature of {temp}°C."
            print(response)
            return response
        else:
            return f"Sorry, I couldn't find weather information for {location}."

Demo: Running the Orchestrator

Here’s a quick demonstration:

1. Query: “What’s the weather in Bangalore, and what’s the current time?”

2. Execution:

   • The orchestrator identifies the intent (weather and time).

   • Delegates tasks to the respective agents.

   • Combines responses to provide the final answer.

Example Output:

• "The weather in Bangalore is misty with a temperature of 22°C. The current time in Bangalore is 12:27 AM."

from agents.base_agent import Agent
from tools.weather_tool import WeatherTool
from tools.time_tool import TimeTool
from orchestrator import AgentOrchestrator

from dotenv import load_dotenv
import os

# Load environment variables from .env file
load_dotenv()

# Create Weather Agent
weather_agent = Agent(
    Name="Weather Agent",
    Description="Provides weather information for a given location",
    Tools=[WeatherTool()],
    Model="gpt-4o-mini"
)

# Create Time Agent
time_agent = Agent(
    Name="Time Agent",
    Description="Provides the current time for a given city",
    Tools=[TimeTool()],
    Model="gpt-4o-mini"
)

# Create AgentOrchestrator
agent_orchestrator = AgentOrchestrator([weather_agent, time_agent])

# Run the orchestrator
agent_orchestrator.run()

What’s Next?

This orchestrator is just the beginning. You can:

• Add more agents for tasks like translation, currency conversion, or database queries.

• Optimize prompts for better LLM responses.

• Extend the system for real-world applications like customer support or smart assistants.

Full Code

Final Thoughts

Building a Multi-Agent Orchestrator showcases the power of combining LLMs with task-specific agents. By modularizing tasks and leveraging context effectively, you can create systems that are both scalable and intelligent.

Stay tuned for more updates, and feel free to share your thoughts or ask questions in the comments below. Don’t forget to check out the accompanying video for a detailed walkthrough of the code.

Happy coding! 🚀

The Goal

Welcome to today's post, where we'll dive into building a web scraping agent using Firecrawl, an open-source library with a paid service option.

This guide will show you how to use FireCrawl to create a scraping agent that takes a URL and custom instructions, then extracts specific content and outputs it in a structured format.

Why Firecrawl?

Traditional web scraping methods rely on manually defining XPath selectors or CSS selectors, which often requires detailed knowledge of a webpage’s structure.

Firecrawl, however, allows us to leverage natural language instructions to scrape data, making it easier to gather specific elements from a page without extensive coding.

This post will demonstrate Firecrawl’s powerful abstraction over conventional scraping techniques by integrating it with a simple agent interface.

Disclaimer

This example uses the Books to Scrape website —a practice site designed for web scraping.

Please always ensure that scraping is allowed by a site’s terms of service, especially with commercial websites.

Code Walkthrough

https://colab.research.google.com/drive/1Fd39Q0oukKrIJNyzN1ICX9L6QzIEOni-

High-Level Agent Workflow

Our scraping agent comprises two main components:

1. User Interface Agent – This component processes user queries, confirms instructions, and determines if additional information is needed.

2. Scraper Agent – This agent handles the scraping process by fetching content from the given URL, parsing it based on the user’s instructions, and outputting the results in a structured format.

import os
from google.colab import userdata
import json
import pandas as pd
from io import StringIO
from openai import OpenAI
from swarm import Agent
from swarm.repl import run_demo_loop
from firecrawl import FirecrawlApp
import nest_asyncio # required for notebooks
nest_asyncio.apply()

os.environ['OPENAI_API_KEY'] = userdata.get("OPENAI_API_KEY")
os.environ['FIRECRAWL_API_KEY'] = userdata.get("FIRECRAWL_API_KEY")

client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

# Initialize FirecrawlApp and OpenAI
app = FirecrawlApp(api_key=os.getenv("FIRECRAWL_API_KEY"))
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

def scrape_website(url):
    """Scrape a website using Firecrawl."""
    scrape_status = app.scrape_url(
        url,
        params={'formats': ['markdown']}
    )
    return scrape_status

def generate_completion(role, task, content):
    """Generate a completion using OpenAI."""
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": f"You are a {role}. {task}"},
            {"role": "user", "content": content}
        ]
    )
    return response.choices[0].message.content

def json_to_csv_downloadable(json_data, filename="output.csv"):
    """Takes JSON data and converts to csv"""
    # Parse the JSON data if it's in string format
    if isinstance(json_data, str):
        json_data = json.loads(json_data)
    # Convert JSON data to a DataFrame
    df = pd.DataFrame(json_data)
    # Save the DataFrame to CSV
    # Display the DataFrame as CSV
    csv_data = df.to_csv(index=False)
    display(csv_data)  # This will show CSV data as plain text 

def handoff_to_parser():
    """Hand off the website content to the parser agent."""
    return parser_agent

def handoff_to_csv_writer():
    """Hand off the parsed content to csv writer"""
    return json_to_csv_downloadable

def handoff_to_website_scraper():
    """Hand off the url to the website scraper agent."""
    return website_scraper_agent

user_interface_agent = Agent(
    name="User Interface Agent",
    model="gpt-4o-mini",
    instructions="You are a user interface agent that handles all interactions with the user. You need to always start with a URL that the user wants to extract content from. The user will expect specific content to be extracted. Ask clarification questions if needed. Be concise.",
    functions=[handoff_to_website_scraper],
)

website_scraper_agent = Agent(
    name="Website Scraper Agent",
    instructions="You are a website scraper agent specialized in scraping website content. If the scraped content is valid, handoff to csv writer",
    functions=[scrape_website,json_to_csv_downloadable],
)

run_demo_loop method in Swarm allows use to run an agent in a loop. In this case, it is the interface agent.

if __name__ == "__main__":
    # Run the demo loop with the user interface agent
    run_demo_loop(user_interface_agent, stream=True)

Talk to the Agent

Let’s run the code and give it the following instruction

get title and price from the top 2 products in https://books.toscrape.com/

The agent return the following response

User: get title and price from the top 2 products in https://books.toscrape.com/ 
User Interface Agent: handoff_to_website_scraper()
Website Scraper Agent: scrape_website()
Website Scraper Agent: json_to_csv_downloadable()
title,price\nA Light in the Attic,£51.77\nTipping the Velvet,£53.74\n
Website Scraper Agent: The title and price of the top 2 books from the website have been extracted and formatted into a CSV file named `top_2_books.csv`. You can download it from the file management system.

Advanced Instructions with FireCrawl

Firecrawl shines when handling more complex queries.

Here’s how you can fetch additional details, such as the product description, from individual book pages.

By providing natural language instructions, you can automate navigation through categories and fetch granular data without manually defining selectors.

User: Get the top 2 products from the Philosophy category

Response.

Website Scraper Agent: Here are the product descriptions for the top 2 Philosophy books:

1. **Sophie's World**
   - **Description**: A page-turning novel that is also an exploration of the great philosophical concepts of Western thought, Sophie’s World has fired the imagination of readers all over the world, with more than twenty million copies in print. One day fourteen-year-old Sophie Amundsen comes home from school to find in her mailbox two notes, with one question on each: “Who are you?” and “Where does the world come from?” From that irresistible beginning, Sophie becomes obsessed with questions that take her far beyond what she knows of her Norwegian village. Through those letters, she enrolls in a kind of correspondence course, covering Socrates to Sartre, with a mysterious philosopher, while receiving letters addressed to another girl. Who is Hilde? And why does her mail keep turning up? To unravel this riddle, Sophie must use the philosophy she is learning—but the truth turns out to be far more complicated than she could have imagined.
   - **URL**: [Sophie's World](https://books.toscrape.com/catalogue/sophies-world_966/index.html)

2. **The Death of Humanity: and the Case for Life**
   - **Description**: Do you believe human life is inherently valuable? Unfortunately, in the secularized age of state-sanctioned euthanasia and abortion-on-demand, many are losing faith in the simple value of human life. To the disillusioned, human beings are a cosmic accident whose intrinsic value is worth no more than other animals. The Death of Humanity explores our culture's declining respect for the sanctity of human life, drawing on philosophy and history to reveal the dark road ahead for society if we lose our faith in human life.
   - **URL**: [The Death of Humanity](https://books.toscrape.com/catalogue/the-death-of-humanity-and-the-case-for-life_932/index.html)

The Benefits of LLM-Based Scraping

Prior to Large Language Models (LLMs), scraping required precise coding to navigate page structures. LLM integration within tools like Firecrawl reduces this complexity, letting us use natural language commands to scrape data.

This feature enables flexible, conversational interactions with a webpage, significantly reducing the manual effort.

Thank you for following along with this tutorial! If you enjoyed this guide, please share it with others interested in web scraping and stay tuned for a comparison between FireCrawl and CrowdAI in the next post!

What is Claude’s ‘Computer Use’?

Claude’s Sonnet 3.5 can now interact with the OS. It can perform task like you would - Browsing websites using a Web Browser, using excel sheet to fill data, etc.

The goal of this Article

Installation

A detailed explanation of running the code is available in this repo.

We need a Linux distribution to run ‘Computer Use’ in an isolated environment (Docker).

To install in Linux subsystem on Windows, run the below command in CMD.

Follow this link for more details.

wsl --install

You will need a anthropic API key. You can get that from here.

Start docker and run the the below snippet in CMD/bash/PowerShell.

SET ANTHROPIC_API_KEY=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
docker run ^
    -e ANTHROPIC_API_KEY=%ANTHROPIC_API_KEY% ^
    -v %USERPROFILE%\.anthropic:/home/computeruse/.anthropic ^
    -p 5900:5900 ^
    -p 8501:8501 ^
    -p 6080:6080 ^
    -p 8080:8080 ^
    -it ghcr.io/anthropics/anthropic-quickstarts:computer-use-demo-latest

After all the installations, you should see the agent running in localhost:8080

Challenge 1: Extracting Article Titles from My Newsletter

The first task aimed to test the agent's ability to interact with a webpage and extract data. I provided the agent with a URL to my newsletter, asking it to capture all article titles and store them in an Excel sheet. This sounds simple, but it required the agent to navigate a few hurdles:

1. Navigating the Website: The agent successfully opened the URL in Firefox and accessed the page.

2. Handling a Subscription Page: It encountered a subscription page and asked for input on how to proceed. After selecting the public archive, it moved forward.

3. Using RSS Feeds: The agent smartly identified an RSS feed on the page and used it to extract article titles.

The task was completed, and the agent saved around 20 articles in an Excel file. While it wasn’t a straightforward front-end automation use case, the agent demonstrated adaptability in handling unexpected scenarios.

Challenge 2: Finding the Latest Video on My YouTube Channel

Next, I asked the agent to find the latest video from my YouTube channel. Here’s how it went:

1. Channel Search: The agent navigated to YouTube and searched for my channel, "Zahin Tab."

2. Fetching the Latest Video: It successfully identified my latest video titled "I Created a Blogging Agent in 5 Minutes Using OpenAI's Form".

Challenge 3: Find the number of likes in the video (from Task 2)

Extracting Likes: I then asked the agent to find the number of likes on that video, and it efficiently navigated the page to retrieve this data.

The agent performed well here, understanding my commands even when they weren't perfectly specific. This task demonstrated its ability to interpret and execute instructions with some degree of context awareness.

Challenge 4: Finding Similar Videos under 4 Minutes

This task was more complex and required the agent to not only search for videos but also filter results by duration. Here’s the breakdown:

1. Understanding the Prompt: I asked the agent to find videos similar to my latest one, but with a duration of under 4 minutes.

2. Using Filters: The agent used the search bar to enter the query, applied a duration filter, and selected the "Under 4 minutes" option.

3. Evaluating Results: It was able to find a few similar videos, although some of the results didn’t exactly match the duration criteria.

This task highlighted the potential for LLMs to automate more complex workflows by combining search, filters, and contextual understanding, making them highly useful in automation scenarios.

Challenge 5: Automating Form Filling

One of the most common use cases in the RPA (Robotic Process Automation) industry is automating form filling. Here’s how the agent performed:

1. Accessing the Form: I provided a URL to a dummy form from RoboForm, a password manager.

2. Filling in Data: The agent quickly identified the form fields and fill, first name, and last name.

3. Handling Errors: Midway through, I encountered some technical issues and rate limits, which required restarting the application.

This task demonstrated the potential of using LLMs for repetitive tasks like filling out forms, which could significantly save time in enterprise environments.

The Bigger Picture: Andrej Karpathy's Vision of LLMs and Automation

Reflecting on these experiments, I couldn’t help but think about Andrej Karpathy's vision for the future of LLMs—essentially envisioning an "LLMOS" (Large Language Model Operating System).

Imagine an LLM that can access OS-level functions, with unlimited memory and a singular focus on completing specific tasks. Here’s what that could look like in practice:

• Intelligent Decision-Making: LLMs trained on specific datasets could make small decisions autonomously, handling routine tasks without human intervention.

• Enterprise Integration: In a business setting, this could mean automating tasks like customer support, form filling, or data entry, making processes more efficient.

This vision excites me as someone who has been in the automation space for a long time, and it shows how the lines between front-end automation and intelligent decision-making are beginning to blur.

Conclusion: The Future of Front-End Automation with LLMs

These experiments were a glimpse into the future of AI-driven automation. From extracting data to interacting with websites and filling out forms, LLMs are proving to be versatile tools. While there are challenges like rate limits and technical hiccups, the potential for transforming enterprise automation is immense.

If you found this exploration interesting and want to learn more about the intersection of AI and automation, don’t forget to subscribe to my YouTube channel. I regularly share insights and tutorials on how to leverage AI tools effectively.

See you in the next post!

What we are building

The goal is to explore how to use Swarm by building a simple agent that takes a user query, performs detailed research, and generates a structured blog post based on the findings.

This will help you understand the capabilities of Swarm, especially in orchestrating tasks among multiple agents.

What is Swarm?

Swarm is a multi-agent orchestration framework introduced by OpenAI. Although currently labeled as experimental and educational, it has quickly become a favorite among developers, earning praise as one of the best frameworks for building and coordinating multi-agent systems.

Key Concepts of the Swarm Framework

Swarm's design is centered around two fundamental components:

Agents: These are autonomous units that operate with a set of instructions and make decisions independently.

Each agent can specialize in a particular task, such as gathering data or writing content.

agent = Agent(
   instructions="You are a helpful agent."
)

Handoffs: This mechanism allows one agent to transfer control of a task or conversation to another agent.

It enables seamless collaboration between agents, ensuring smooth execution of complex workflows.

sales_agent = Agent(name="Sales Agent")

def handoff_to_sales():
   return sales_agent

agent = Agent(functions=[transfer_to_sales])

response = client.run(agent, [{"role":"user", "content":"Transfer me to sales."}])
print(response.agent.name)

By leveraging these components, Swarm helps developers create systems where tasks can be broken down into smaller, specialized sub-tasks, handled by different agents.

Use Case: Creating a Technical Content Writing Agent

Code: https://colab.research.google.com/drive/1phDFUasrZxjChabWo_oWwWuNKwfjuJ_B?authuser=1#scrollTo=zyxmU_W9ZCxp

To illustrate the potential of Swarm, let’s build a multi-agent system that generates a blog post based on user input. The system comprises three agents:

1. Interface Agent: Interacts with the user, refines the query if needed, and passes the query to the Researcher Agent.

2. Researcher Agent: Conducts detailed research on the query and prepares a research report.

3. Blogger Agent: Uses the research report to create a structured blog post.

The workflow is simple: the user provides a theme or query, such as "Top 5 Technical Skills for 2025." The Interface Agent refines the query if necessary, then hands it over to the Researcher Agent, which gathers and analyzes relevant information. Finally, the Blogger Agent compiles the research into a well-written blog post.

How the Swarm Framework Works

Illustration Overview

Below is a high-level overview of the agent workflow:

• User Input: The user provides a theme, e.g., "Why is R fast?".

• Interface Agent: Takes the user’s input, asks clarifying questions if needed, and then passes the refined query to the Researcher Agent.

• Researcher Agent: Gathers data from search engines, scrapes content from top results, and analyzes the information to produce a research report.

• Blogger Agent: Converts the research report into a blog post and returns it to the Interface Agent.

• Interface Agent: Delivers the completed blog post to the user.

This setup demonstrates how Swarm’s agent-based architecture simplifies the process of delegating tasks and orchestrating their execution.

Building the Swarm Agents

Let’s dive into the code, which consists of the three agents and their respective functions.

Agent Class Structure

Each agent is defined as a class with parameters like name, model, instructions, and functions. The functions include methods for interacting with APIs, databases, and other agents. A key method is the handoff method, which allows one agent to pass control to another:

Researcher Agent

The Researcher Agent uses a package called GPT Researcher to gather and analyze content from various search engines.

This agent performs a deep dive into the topic provided by the user

import nest_asyncio # required for notebooks
nest_asyncio.apply()

from gpt_researcher import GPTResearcher
import asyncio

client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

async def get_report(query: str) -> str:
    report_type = "research_report"
    researcher = GPTResearcher(query, report_type)
    research_result =  await researcher.conduct_research()
    report =  await researcher.write_report()

    # Get additional information
    research_context = researcher.get_research_context()
    research_costs = researcher.get_costs()
    research_images = researcher.get_research_images()
    research_sources = researcher.get_research_sources()

    return {'report':report}

def research_topic(query: str) -> str:
  """Generate research report"""
  return asyncio.run(get_report(query))

def handoff_to_researcher():
    """Hand off the user query to the researcher agent."""
    print("Handing off to Researcher Agent")
    return researcher_agent

researcher_agent = Agent(
    name="Researcher Agent",
    model="gpt-4o-mini",
    instructions="You are a researcher agent specialized in researching. If you are satisfied with the research, handoff the report to blogger",
    functions=[research_topic, handoff_to_blogger],
)

The handoff method transfers control to the Blogger Agent once the research is complete.

3. Blogger Agent

The Blogger Agent creates a blog post based on the research report. It uses the language model to generate a well-structured article:

def generate_completion(role, task, content):
    """Generate a completion using OpenAI."""
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": f"You are a {role}. {task}"},
            {"role": "user", "content": content}
        ]
    )
    return response.choices[0].message.content

def handoff_to_blogger():
    """Hand off the research report to the blogger agent."""
    print("Handing off to Blogger Agent")
    return blogger_agent

def generate_blog_content(research_data):
    """Generate technical blog content on research report using OpenAI."""
    content = generate_completion(
        "Technical Content Creator",
        "Create compelling technical content for a blog based on the following research report.",
        research_data
    )
    return {"content": content}


blogger_agent = Agent(
    name="Blogger Agent",
    model="gpt-4o-mini",
    instructions="You are a top technical blogger agent specialized in creating compelling technical content for blogs based on research report. Be concise.",
    functions=[generate_blog_content],
)

4. Interface Agent

The Interface Agent communicates with the user, refines the query, and then initiates the workflow by handing control over to the Researcher Agent:

user_interface_agent = Agent(
    name="User Interface Agent",
    model="gpt-4o-mini",
    instructions="You are a user interface agent that handles all interactions with the user. You need to always start with a theme or topic that the user wants to research. Ask clarification questions if needed. Be concise.",
    functions=[handoff_to_researcher],
)

Running the System

With the agents defined, you can now run the system:

1. Install the necessary packages:

    ! pip install git+https://github.com/openai/swarm.git openai gpt-researcher nest_asyncio
   

2. Initialize the agents and run the user input loop:

    import os
    from google.colab import userdata
    from openai import OpenAI
    from swarm import Agent
    from swarm.repl import run_demo_loop
   
    os.environ['OPENAI_API_KEY'] = userdata.get("OPENAI_API_KEY")
    os.environ['TAVILY_API_KEY'] = userdata.get("TAVILY_KEY")
    FAST_LLM="openai:gpt-4o-mini"
    os.environ['FAST_LLM'] = FAST_LLM
    os.environ['SMART_LLM'] = FAST_LLM
   

    if __name__ == "__main__":
        # Run the demo loop with the user interface agent
        run_demo_loop(user_interface_agent, stream=True)
   

For the theme "Top 5 Technical Skills for 2025," the system will generate a structured blog post by conducting research and formatting the findings into readable content.

Example Output: Blog Post Generated by the Agent

Here's an example of what the generated blog post might look like:

Essential Skills for Software Developers: Staying Competitive in 2025

As we approach 2025, the pace of technological advancement presents both opportunities and challenges for software developers. The landscape is evolving with new innovations in artificial intelligence, cloud computing, and data analytics, among others. To remain competitive and relevant in the job market, developers must cultivate certain technical skills that are poised to dominate the industry. Here’s a detailed look at the top five technical skills every software developer should focus on in the coming years.

1. Mastering Artificial Intelligence and Machine Learning

Artificial Intelligence (AI) and Machine Learning (ML) are no longer just buzzwords; they are core components of modern software applications. Industries from healthcare to finance are leveraging AI to automate processes, analyze data, and enhance customer experiences.

What to Learn:

• Programming languages: Python is the go-to language for AI and ML, alongside familiarity with frameworks like TensorFlow and PyTorch.

• Key areas: Focus on natural language processing (NLP) and computer vision, as they will continue to grow in demand.

• Real-world application: Engage in projects that allow you to implement algorithms and create systems that learn and adapt based on data.

2. Embracing Cloud Computing and Serverless Architecture

The migration to cloud computing is reshaping how businesses operate. Proficiency in cloud platforms such as Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform is becoming a mandatory skill for developers. Furthermore, an understanding of serverless architecture, which eliminates the need to manage servers, is a differentiator that can elevate your career.

What to Learn:

• Cloud platforms: Gain hands-on experience with major cloud services and their offerings.

• Serverless frameworks: Explore platforms like AWS Lambda to understand how to deploy applications more swiftly.

• Collaboration: Familiarize yourself with DevOps practices to enhance collaboration in software development and operations.

3. Building Cybersecurity Fundamentals

With the surge in cyber threats, understanding cybersecurity is critical for developers. Knowledge of secure coding practices, data encryption, and vulnerability assessment is essential for protecting applications and sensitive user information.

What to Learn:

• Security principles: Study secure coding standards and tools for threat modeling and incident response.

• Sandbox testing: Experiment with penetration testing frameworks to gain practical knowledge.

• Developing secure code: Integrate security considerations into your day-to-day coding practices and workflow.

4. Capitalizing on Low-Code and No-Code Development

Low-code and no-code platforms are transforming application development by making it accessible to non-developers. These platforms enable rapid application development, which can markedly accelerate a company’s digital transformation efforts.

What to Learn:

• Tools and platforms: Explore tools like OutSystems, Mendix, or Bubble to understand how they work and their capabilities.

• Integration skills: Learn how to integrate low-code solutions with traditional back-end systems.

• Process optimization: Understand how these platforms can streamline workflows and provide rapid iterations.

5. Diving Into Data Science and Analytics

In our data-centric world, the ability to analyze and interpret data is invaluable. Understanding data science principles can empower developers to turn insights into actionable strategies.

What to Learn:

• Data tools and languages: Gain proficiency in SQL, R, and Python for data manipulation and visualization.

• Statistical analysis: Understand key concepts of statistics and machine learning to derive meaningful insights from data.

• Predictive analysis: Use machine learning algorithms to forecast trends and contribute to data-driven decision-making processes.

Conclusion

As technology continues to change at a rapid pace, developers must engage in continuous learning to stay up to date. The top five skills outlined above—AI and ML, cloud computing and serverless architecture, cybersecurity fundamentals, low-code/no-code development, and data science—are critical for any developer aiming to thrive in the industry by 2025.

Investing in these skills will not only enhance your employability but also equip you to contribute effectively to innovative projects that shape the technology landscape. Embrace this opportunity to upgrade your expertise and solidify your place in the future of software development.

References

• Hadalgi, N. (2024). The Most In-Demand Programming Skills for 2025: Staying Ahead in a Rapidly Evolving Tech Landscape. LinkedIn.

• Teal HQ. (2024). Top Skills for Software Developers in 2024 (+Most Underrated Skills). Teal HQ.

Educative. (2023). Top Software Developer Skills To Learn in 2024. Educative.

Conclusion

The Swarm framework simplifies the creation of complex multi-agent systems, allowing developers to break down intricate tasks into manageable components.

By combining specialized agents for tasks like research and content creation, developers can automate time-consuming processes, such as generating technical blog posts from user queries.

With this guide, you now have a foundational understanding of Swarm’s capabilities. The next step is to explore more advanced use cases, such as orchestrating agents for more complex workflows.

Stay tuned for future posts where we dive deeper into Swarm and its applications in AI development!

Traditional RAG methods can struggle with intricate questions due to their reliance on smaller text segments that may not encapsulate the broader themes or details of the original documents.

When building a RAG-based solution, answer the following questions before creating the indexing pipeline.

What type of queries will the system handle?

• Are the queries typically seeking specific details, or do they require a broader contextual understanding?

How important is precision versus context in the system’s responses?

• Should the system prioritize precise answers to detailed questions (favoring smaller chunks), or should it provide more comprehensive responses even if they are less precise (favoring larger chunks)?

How much detail or noise is acceptable in the retrieved results?

• Will smaller chunks provide too little context, or will larger chunks introduce too much irrelevant information?

Can the user’s query context vary significantly?

• If the user queries tend to be more context-dependent, would using larger chunks improve understanding, or could the system risk missing key details?

What is the nature of the content being used for retrieval?

• Does the content lend itself better to smaller chunks (e.g., factual, concise information), or does it require larger chunks to capture essential relationships and context (e.g., narrative or complex data)?

What is Parent Document Retrieval (PDR)?

Parent Document Retrieval (PDR) is a sophisticated technique utilized in Retrieval-Augmented Generation (RAG) models to enhance the retrieval process by obtaining full parent documents to augment the LLM generation.

This method addresses the limitations of standard RAG approaches that often rely on smaller text chunks, which may lack the necessary context for complex queries.

By retrieving complete parent documents, PDR allows for a more comprehensive understanding of the material, leading to richer and more informative responses, particularly for nuanced inquiries.

Where is Parent Document Retrieval Applied?

PDR is applicable in various domains where context-rich responses are essential. Some common applications include:

• Customer Support: Enhancing automated systems to provide detailed responses based on comprehensive product documentation.

• Legal and Compliance: Assisting in retrieving relevant legal documents or regulations that require in-depth understanding.

• Research and Academia: Facilitating access to full research papers or articles when specific sections are referenced.

• Content Generation: Improving the quality of content produced by language models by providing them with extensive background information.

When Should Parent Document Retrieval be Used?

PDR should be employed particularly in scenarios where:

• The queries are complex or multifaceted, requiring detailed context.

• The available data consists of lengthy documents that need to be segmented for better comprehension.

• There is a need to ensure high accuracy and relevance in responses generated by language models.

• Users seek comprehensive answers rather than brief snippets of information.

Using PDR can significantly enhance the performance of RAG systems in these situations

Who Benefits from Parent Document Retrieval?

Various stakeholders can benefit from PDR, including:

• Developers and Data Scientists: Those working on RAG systems can leverage PDR to improve model performance and user satisfaction.

• Businesses: Organizations seeking efficient customer support solutions can enhance their automated systems with PDR.

• Researchers and Academics: Individuals needing thorough literature reviews or data analysis can utilize PDR for more effective information retrieval.

• End Users: Anyone seeking detailed and contextually rich information will benefit from systems employing PDR.

How to implement PDR using LangChain?

You can go through the entire code in this notebook. Sharing some important snippets below.

from langchain.schema import Document
from langchain.vectorstores import Chroma
from langchain.retrievers import ParentDocumentRetriever
from langchain.chains import RetrievalQA
from langchain_openai import OpenAI
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.storage import InMemoryStore
from langchain.document_loaders import TextLoader,WebBaseLoader
from langchain_openai import OpenAIEmbeddings
from langchain_openai import ChatOpenAI
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
import os
import tiktoken
from google.colab import userdata

  # Loading a single website
loaders = [
    WebBaseLoader("https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3864624/"),
    WebBaseLoader("https://www.mayoclinic.org/diseases-conditions/lung-cancer/symptoms-causes/syc-20374620")
]

docs = []
for loader in loaders:
    token_count = num_tokens_from_string(str(loader.load()),"cl100k_base")
    print(f"Tokens for {loader.web_path}: {token_count}")
    docs.extend(loader.load())

Output below

Tokens for https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3864624/: 43869
Tokens for https://www.mayoclinic.org/diseases-conditions/lung-cancer/symptoms-causes/syc-20374620: 4997

Create parentsplitter and childsplitter using the RecursiveCharacterTextSplitter module in LangChain.

Use Chrom DB as a vector store and LangChain’s InMemoryStore to store large docs or chunks.

# This text splitter is used to create the parent documents
parent_splitter = RecursiveCharacterTextSplitter(chunk_size=2000)
# This text splitter is used to create the child documents
# It should create documents smaller than the parent
child_splitter = RecursiveCharacterTextSplitter(chunk_size=400)
# The vectorstore to use to index the child chunks
vectorstore = Chroma(
    collection_name="split_parents", embedding_function=OpenAIEmbeddings(model="text-embedding-3-small")
)
# The storage layer for the parent documents
store = InMemoryStore()

Instantiate the ParentDocumentRetriever class using the above as parameters.

retriever = ParentDocumentRetriever(
    vectorstore=vectorstore,
    docstore=store,
    child_splitter=child_splitter,
    parent_splitter=parent_splitter,
)

Add the docs we created using the WebPageLoader to the retriever object.

retriever.add_documents(docs)

Test the vector store with a query.

sub_docs = vectorstore.similarity_search("What are the symptoms of Lung Cancer?")
print(sub_docs[0].page_content)

The response.

SymptomsLung cancer typically doesn't cause symptoms early on. Symptoms of lung cancer usually happen when the disease is advanced.
Signs and symptoms of lung cancer that happen in and around the lungs may include:

A new cough that doesn't go away.
Chest pain.
Coughing up blood, even a small amount.
Hoarseness.
Shortness of breath.
Wheezing.

Let’s see the parent doc that is returned for the same query using the retriever object.

retrieved_docs = retriever.invoke("What are the symptoms of Lung Cancer?")
print(retrieved_docs[0].page_content)

Response.

Patient Care & Health Information
Diseases & Conditions


Lung cancer

Request an Appointment
Symptoms &causesDiagnosis &treatmentDoctors &departmentsCare atMayo Clinic






Print




Overview




        Lung cancer
        Enlarge image









Close



Lung cancer


Lung cancer
Lung cancer begins in the cells of the lungs.





Lung cancer is a kind of cancer that starts as a growth of cells in the lungs. The lungs are two spongy organs in the chest that control breathing.
Lung cancer is the leading cause of cancer deaths worldwide.
People who smoke have the greatest risk of lung cancer. The risk of lung cancer increases with the length of time and number of cigarettes smoked. Quitting smoking, even after smoking for many years, significantly lowers the chances of developing lung cancer. Lung cancer also can happen in people who have never smoked.Products & ServicesA Book: Mayo Clinic Family Health BookNewsletter: Mayo Clinic Health Letter — Digital EditionShow more products from Mayo Clinic





SymptomsLung cancer typically doesn't cause symptoms early on. Symptoms of lung cancer usually happen when the disease is advanced.
Signs and symptoms of lung cancer that happen in and around the lungs may include:

A new cough that doesn't go away.
Chest pain.
Coughing up blood, even a small amount.
Hoarseness.
Shortness of breath.
Wheezing.

Signs and symptoms that happen when lung cancer spreads to other parts of the body may include:

Bone pain.
Headache.
Losing weight without trying.
Loss of appetite.
Swelling in the face or neck.

When to see a doctorMake an appointment with your doctor or other healthcare professional if you have any symptoms that worry you.
If you smoke and haven't been able to quit, make an appointment. Your healthcare professional can recommend strategies for quitting smoking. These may include counseling, medicines and nicotine replacement products.
Request an appointment
1925

Let’s create a chain using an LLM to generate responses using the query and the context from the retriever.

from langchain_core.runnables import RunnablePassthrough

# Prompt template
template = """Answer the question based only on the following context:
{context}
Question: {question}
"""
prompt = ChatPromptTemplate.from_template(template)

# LLM
model = ChatOpenAI(temperature=0, model="gpt-4o-mini")

# RAG pipeline
chain = (
    {"context": retriever, "question": RunnablePassthrough()}
    | prompt
    | model
    | StrOutputParser()
)

chain.invoke("How are smoking and Lung Cancer related?")

Response

Smoking is the primary cause of most lung cancers.
It introduces cancer-causing substances, known as carcinogens, into the lungs, which damage the cells that line the lung tissue.
This damage can lead to changes in the cells' DNA, causing them to grow and multiply uncontrollably, ultimately resulting in cancer.
Additionally, smoking can also affect non-smokers through secondhand smoke exposure. While lung cancer can occur in individuals who have never smoked, the exact causes in these cases may not be clear. Overall, smoking significantly increases the risk of developing lung cancer.

As you can see the response seems to be pretty neat and grounded.

Let me know in the comments how your experiments went.

This recent social media post by Rahul Pandey, co-founder of Taro Community and ex-Meta Staff Engineer, really caught my attention.

In it, Rahul shares a story about an engineer from Khan Academy who, back in 2013, got hired by Facebook because of her open-source contributions to React. Naturally, I was curious and dove into research—and what I found was mind-blowing.

This engineer didn’t just contribute to open source; she actually saved ReactJS. Let me tell you the story of Sophie Alpert.

Sophie Alpert’s journey in tech is the stuff of movies—an inspiring narrative filled with passion, innovation, and leadership.

When we think about engineers reaching the heights of people like Zuckerberg or Musk, the odds are staggering—maybe one in a billion.

But Sophie's path shows us what every engineer truly aspires to: not just a successful career at top companies, but real, lasting contributions to the community.

If you prefer watching a video

Early Life and Education

Sophie’s story begins in Colorado. Her parents, both familiar with coding, nurtured her love for technology from an early age. While other kids were reading comic books or novels, Sophie was reading computer manuals for fun. That’s how deep her passion ran.

By middle and high school, she was creating websites using Dreamweaver, coding personal projects, and even doing freelance work. Her skills were evolving fast.

She later went on to study computer science at Carnegie Mellon University, but the traditional classroom wasn’t cutting it for her. Sophie craved hands-on experience. So, after a summer internship at Khan Academy, where she developed interactive math tools, she made a bold decision—she dropped out of college to work there full-time.

Career Milestones

At Khan Academy, Sophie’s contributions were pivotal. She helped develop educational tools that transformed user experiences, reinforcing her belief that practical applications beat out theoretical knowledge any day.

But it was her transition to the React core team at Facebook that marked a defining moment in her career. Initially, Sophie was just an enthusiastic open-source contributor, but she didn’t stop there. Over time, her work was so impactful that she was invited to lead the React team at Facebook.

Her focus? Making React more efficient and accessible for developers.

There’s a fantastic documentary by the YouTube channel HoneyPot that goes into this transition in detail, and it highlights just how instrumental Sophie’s contributions were.

Her work with React was so good, in fact, that Facebook couldn’t resist hiring her.

And as they say—the rest is history.

Today, React is the most popular web framework in the world, and its influence only continues to grow. Sure, open-source is a community effort, but if it weren’t for Sophie’s contributions and her fresh perspective back in 2013, React could’ve easily been just another failed tech experiment.

Insights and Philosophy

Throughout her career, Sophie has been vocal about the importance of side projects. To her, these unstructured projects are the ultimate test of creativity and problem-solving. They’re where real passion shines, and they often lead to breakthroughs that traditional work can’t always offer.

Sophie Alpert’s story is a testament to following your passion, breaking norms, and carving your own path in tech. She shows us that real-world experience can often lead to industry-changing contributions—no degree required.

Picture this: AI, not just searching for keywords, but truly grasping the intent behind your query. An AI that doesn’t just retrieve what you asked for, but what you need to know.

This is the third article in the Advanced RAG series, where we are following the journey of TechnoHealth Solution and how they implement RAG for their several use cases.

Today, we’re unlocking one of the most efficient indexing techniques — multi-representation indexing.

What is Multi-Representation Indexing?

Based on the paper Dense X Retrieval or Proposition-based Retrieval, Multi-representation indexing involves creating and storing representations of each document within the retrieval system.

Representations here could mean traditional keyword analysis, deep semantic understanding or summary, and even visual elements like images or diagrams of the documents.

Why is Multi-Representation Indexing Used?

Multi-representation indexing improves accuracy, adapts to different types of documents, and is flexible enough to handle complex information, like research papers, code, or even e-commerce product listings

The primary motivations for employing multi-representation indexing include:

• Improved Retrieval Accuracy

• Contextual Understanding

• Flexibility for Document Types

• Handling Complex Information

When is Multi-Representation Indexing Used?

Multi-representation indexing is particularly beneficial in scenarios where:

• Complex Queries Are Common: Users often ask nuanced questions requiring understanding beyond simple keyword matching.

• Diverse Document Formats Are Involved: Environments where documents come in various formats and types, such as academic research or product catalogs.

• Applications Require Enhanced Semantic Understanding: For applications benefiting from deeper semantic insights, such as conversational agents or advanced search engines.

Where is Multi-Representation Indexing Used?

This advanced indexing technique is applied across various domains, improving the relevance and depth of information retrieval mainly

1. Search Engines and Information Retrieval Systems

2. Conversational AI and ChatbotsWho benefits from multi-representation indexing?

• End-users interacting with AI systems, such as search engines, chatbots, or recommendation engines, benefit from more accurate and contextually relevant information. This includes professionals in various fields:

How Multi-Representation Indexing Works

Implementation Steps

1. Document Loading and Chunking

   • Documents are loaded from sources such as web pages and split into manageable chunks using techniques like recursive character splitting.

2. Summarization

   • A Language Model (LLM) chain is used to generate concise summaries of each chunk, preserving essential details crucial for later retrieval.

3. Setting Up the Multi-Vector Retriever

   • A MultiVectorRetriever instance is initialized, linking the vector store (optimized summaries) and document store (original chunks) using unique keys.

4. Adding Documents

   • Optimized summaries and original chunks are added to their respective stores, ensuring they are linked correctly for retrieval.

5. Query and Retrieval

   • Queries are processed using the MultiVectorRetriever, leveraging the optimized summaries for efficient document retrieval based on relevance.

I’ll add two key snippets below.

1. We are extracting representations like table data and text from this pdf using the Unstructured library.

from typing import Any

from pydantic import BaseModel
from unstructured.partition.pdf import partition_pdf

# Get elements
raw_pdf_elements = partition_pdf(
    filename=path,
    # Unstructured first finds embedded image blocks
    extract_images_in_pdf=False,
    # Use layout model (YOLOX) to get bounding boxes (for tables) and find titles
    # Titles are any sub-section of the document
    infer_table_structure=True,
    # Post processing to aggregate text once we have the title
    chunking_strategy="by_title",
    # Chunking params to aggregate text blocks
    # Attempt to create a new chunk 3800 chars
    # Attempt to keep chunks > 2000 chars
    max_characters=4000,
    new_after_n_chars=3800,
    combine_text_under_n_chars=2000,
    image_output_dir_path=path,
)

2. Using the MultiVectorRetriever module to leverage two data stores - one for summaries and other for full docs.

import uuid

from langchain.retrievers.multi_vector import MultiVectorRetriever
from langchain.storage import InMemoryStore
from langchain_chroma import Chroma
from langchain_core.documents import Document
from langchain_openai import OpenAIEmbeddings

# The vectorstore to use to index the child chunks
vectorstore = Chroma(collection_name="summaries", embedding_function=OpenAIEmbeddings(model="text-embedding-3-small"))

# The storage layer for the parent documents
store = InMemoryStore()
id_key = "doc_id"

# The retriever (empty to start)
retriever = MultiVectorRetriever(
    vectorstore=vectorstore,
    docstore=store,
    id_key=id_key,
)

# Add texts
doc_ids = [str(uuid.uuid4()) for _ in texts]
summary_texts = [
    Document(page_content=s, metadata={id_key: doc_ids[i]})
    for i, s in enumerate(text_summaries)
]
retriever.vectorstore.add_documents(summary_texts)
retriever.docstore.mset(list(zip(doc_ids, texts)))

# Add tables
table_ids = [str(uuid.uuid4()) for _ in tables]
summary_tables = [
    Document(page_content=s, metadata={id_key: table_ids[i]})
    for i, s in enumerate(table_summaries)
]
retriever.vectorstore.add_documents(summary_tables)
retriever.docstore.mset(list(zip(table_ids, tables)))

We then leverage the retriever in the RAG chaing and this is the final output after invoking the object with the user query.

chain.invoke("What is Dual Immunotherapy without Chemotherapy?")

Dual Immunotherapy without Chemotherapy refers to a treatment approach for metastatic non-small cell lung cancer (NSCLC) that involves the use of two immune checkpoint inhibitors (ICIs) without the addition of chemotherapy. An example of this is the combination of nivolumab and ipilimumab, which has been FDA-approved for first-line treatment in patients with PD-L1 expression ≥ 1% and without EGFR/ALK alterations. 

In the phase III trial Checkmate 227, this combination demonstrated a positive overall survival (OS) benefit across all PD-L1 expression subgroups, with a 4-year OS rate of 29% for patients receiving nivolumab plus ipilimumab compared to 18% for those receiving chemotherapy. This indicates that dual immunotherapy can provide significant long-term survival benefits for patients with advanced NSCLC.

For full code and check out the github repo below.

This is part of the Advanced RAG Series: Part 1

When working with Retrieval Augmented Generation (RAG) models, selecting the right chunking method can make a huge difference in performance.

In my latest YouTube video, I dive deep into the three main chunking approaches—Semantic, Recursive, and Fixed Size—and evaluate their performance based on four critical metrics: context precision, faithfulness, answer relevancy, and context recall.

The chunking method you choose can impact how accurate and relevant the AI-generated answers are. So, which method strikes the perfect balance between retaining enough context and providing highly relevant, faithful responses?

In the video, I break down:

• How Semantic Chunking performed in capturing context but struggled with relevancy.

• Why Recursive Chunking emerged as a strong contender with high accuracy and relevancy.

• The surprising strengths of Fixed Size Chunking, especially in context retention.

If you're interested in fine-tuning your RAG models or curious about which chunking method works best, this video is packed with insights that will help you make the right choice. Check out the full breakdown in the embedded video below!

Watch the full analysis and find out which chunking method is best for your use case:

The Context

TechnoHealth Solutions, a fictitious, mid-sized tech company that builds software for hospitals, has identified a critical issue facing healthcare professionals - information overload and outdated knowledge.

The Problem:

Picture this:

• Doctors spend hours buried in research instead of treating patients

• Nurses struggling to keep up with thousands of new medical studies published every week

• Patients receiving inconsistent care because their providers can't access the latest information

• The constant fear of medical errors due to outdated knowledge

It's a healthcare nightmare, and it's happening right now in hospitals around the world.

The Failed Solutions:

TechnoHealth Solutions first thought, "Let's just build a better search engine!" But they quickly realized that would only add to the problem. Doctors don't have time to sift through endless documents.

Then they considered an AI chatbot. But two major roadblocks appeared:

1. The risk of AI "hallucinations" – making up false medical information

2. The sheer volume of medical data exceeded what current AI systems could handle

The Breakthrough:

Finally, TechnoHealth Solutions developed MedAssist AI, a system that leverages Retrieval Augmented Generation (RAG) technology to address these challenges. Here's how RAG solved their problem:

But what exactly is RAG, and how does it solve this medical knowledge crisis?

What is RAG?

Here’s how it works.

Retrieval: When a user asks a question or provides a prompt, RAG first retrieves relevant passages from a vast knowledge base. This knowledge base could be the internet, a company’s internal documents, or any other source of text data.

Augmentation: The retrieved passages are then used to “augment” the LLM’s knowledge. This can involve various techniques, such as summarizing or encoding the key information.

Generation: Finally, the LLM leverages its understanding of language along with the augmented information to generate a response. This response can be an answer to a question, a creative text format based on a prompt, or any other form of text generation.

Why RAG?

Boosted Factual Accuracy: RAG strengthens LLMs by connecting them to external sources of information, like databases or live feeds. This ensures that their responses are based on real-world facts rather than relying solely on what they were trained on.

Expert Knowledge in Specific Domains: General-purpose LLMs are like a jack of all trades—they know a bit about everything but aren’t experts in any one field. With RAG, you can integrate specific knowledge bases, allowing the AI to answer highly specialized questions.

Fewer Mistakes (Reduced Hallucination): LLMs sometimes make up information that sounds convincing but isn’t true. RAG reduces this risk by providing the AI with reliable sources to back up its claims, leading to more trustworthy responses.

Adaptability to New Information: The world is constantly evolving, and LLMs trained on older data can quickly become outdated. RAG solves this by giving AI access to up-to-date sources, so it can always provide current information.

Customizable and Scalable: RAG isn’t a one-size-fits-all solution. It can be adjusted to fit your needs, whether you're working with limited resources or require more power for complex tasks.

TechnoHealth Solutions’ RAG Implementation

Here’s how TechnoHealth Solutions built their RAG solution to make their AI smarter, faster, and more reliable.

Step 1: Building an Indexing Pipeline

The first step was to organize and process their data. TechnoHealth consolidated all their data sources, from medical reports to research papers. They then:

• Processed the documents by breaking them down into smaller, manageable chunks.

• Passed these chunks through an embedding model, which turned them into vector representations, or "embeddings." These are mathematical versions of the text that make it easy for the system to compare and find similarities.

• Finally, they stored all these embeddings in a vector database. Think of this as a specialized storage space for these mathematical text chunks.

Step 2: Building the Retrieval System

When a user asks a question, TechnoHealth’s retrieval system goes to work:

• It first converts the user’s question into an embedding (just like they did with the documents).

• Then, it compares this question embedding to all the text chunks stored in the vector database to find the most similar chunks.

Step 3: Augmenting the LLM

Now comes the magic of augmenting the LLM’s knowledge:

• The system crafts a prompt that includes:

  • Instructions for the LLM.

  • The user’s question.

  • The most relevant documents retrieved from the database (the context).

• This carefully crafted prompt tells the LLM how to use the data to generate a better response

Step 4: Generating the Response

Finally, the crafted prompt is sent to the LLM:

• The LLM uses the instructions and the context to generate a complete, informed response to the user’s query.

• The response is sent back to the user, more accurate and grounded in real-world data than if the LLM had answered from its general knowledge alone.

Conclusion:

By combining indexing, retrieval, augmentation, and generation, TechnoHealth Solutions built a RAG system that ensures every answer their AI provides is based on real, up-to-date knowledge.

In the next article, we will talk about pre-retrieval optimizations.

These two posts—just a few lines of text by two influential voices in the tech space—have ignited a storm of conversations.

My social media feeds are buzzing—filled with commentary, debates, and threads that seem to extend endlessly. And the chatter isn’t dying down; in fact, it’s only growing louder.

What’s clear is this: They’re an indication, a signal—an alarm —that something has fundamentally shifted in the software industry.

In a recent LinkedIn post, Amazon CEO Andy Jassy dropped a bombshell: Amazon Q, their generative AI assistant, has dramatically redefined what’s possible in software development.

Take this: what used to take 50 developer days to upgrade applications to Java 17 now takes mere hours. The equivalent of 4,500 developer-years of work saved—just like that.

Amazon upgraded more than half of its production Java systems in less than six months. A process that usually demands extensive time and resources was completed in a fraction of the time, at a fraction of the cost.

And the most striking part? 79% of the auto-generated code reviews were shipped by developers without any additional tweaks.

The AI didn’t just assist—it delivered.

And it’s not just the corporate giants making waves. Influential voices in AI research are also weighing in on the transformative power of AI in coding.

In a recent tweet, AI researcher Andrej Karpathy shared his experience with AI-assisted programming. He found these tools so effective that they’ve completely reshaped his coding workflow.

Karpathy describes a new way of coding—what he calls "half-coding." He writes prompts in plain English, reviews the AI-generated code diffs, and lets the AI handle the heavy lifting, completing substantial portions of code in record time.

Karpathy says - he can’t imagine going back to the way things were.

And neither can I. I’ve been using these code generators for over a year now and my productivity has positively increased.

Coding is Undergoing a Seismic Shift

So let’s start with a fact: coding is about to or is already undergoing a seismic shift.

While I was researching about this topic, I stumbled across Russell Kaplan's thread on x.com which resonated well with me.

It’s a fascinating thread - So I will break down some of the tweets that were thought-provoking.

Here’s why this matters. Research labs around the world are pouring resources into making AI models better at coding and reasoning. This isn’t just incremental progress—it’s a massive leap forward. These models are being trained to write code, reason through problems, and improve themselves in ways we haven’t seen before.

Why coding? What makes it so special?

The answer lies in the unique advantage coding offers: it’s a domain where AI can learn through “self-play.”

Unlike other fields where data is limited by human expertise, code can be tested, tweaked, and optimized automatically. It’s a playground for AI, where models can write code, run it, and check for consistency—all without human intervention. This kind of automatic supervision is not just beneficial; it’s revolutionary.

But what is self-play?

To understand, self-play - lets learn about the model that pioneered the concept itself.

AlphaGo, developed by DeepMind, is a groundbreaking AI that made history by defeating human world champions in the ancient game of Go.

One of its key innovations was the use of self-play, where AlphaGo played countless games against versions of itself.

This technique allowed the AI to continuously improve, discovering new strategies and refining its decision-making process without human input.

Fascinating isn’t it.

Using this as an inspiration, researcher’s from Microsoft and MIT, in the paper “Language Models Can Teach Themselves to Program Better” demonstrated an answer to the question

“Can an LM design its own programming problems to improve its problemsolving ability?

Rather than using English problem descriptions which are ambiguous and hard to verify, they generated puzzles.

Self-play using programming puzzles

Here’s how they built the pipeline.

Puzzle Generation: The language model generates new puzzles by sampling from the training set, combining them, and creating additional puzzles within its context window. These puzzles are then filtered for syntactic validity and to exclude trivial solutions.

Solution Generation: The model attempts to solve the valid puzzles using a few-shot learning strategy, with a predetermined number of attempts per puzzle.

Solution Verification: Generated solutions are verified using a Python interpreter. From these, up to m correct and concise solutions are selected for each puzzle.

Fine-Tuning: The model is fine-tuned on the selected puzzle-solution pairs.

The result?

The diagram illustrates how the iterative process of generating, verifying, and fine-tuning on synthetic data significantly improves the performance of the language model in solving puzzles.

Initial Evaluation: Without any fine-tuning, GPT-Neo solves 7.5% of the held-out test puzzles.

Evaluation After Fine-Tuning on Unverified Data: After fine-tuning on the unverified synthetic data, the model's performance improves, solving 21.5% of the held-out test puzzles.

Evaluation After Fine-Tuning on Verified Data: Fine-tuning on the verified synthetic data further enhances the model's performance, allowing it to solve 38.2% of the held-out test puzzles.

Now, let’s look ahead.

In just a few years, software engineering will be almost unrecognizable. Imagine having an army of coding agents at your disposal—each one capable of handling tasks from start to finish. This isn’t science fiction; it’s where we’re headed.

Engineers will transition from writing lines of code to managing these agents, overseeing the architecture of systems, and making high-level decisions.

This shift will redefine the role of the software engineer. In this new world, coding becomes less about the syntax and more about the strategy—understanding what needs to be built and why.

It’s like moving from being a craftsman to a project manager, where your focus is on the bigger picture.

To extend this idea, I return to Andrej Karpathy’s vision of AI resembling an operating system—essentially, a powerful agent. Andrej describes this as an entity with more knowledge than any single human on all subjects.

Now, imagine this as a specialized agent focused on a specific domain or business use case.

It’s possible, and we may not be very far from it.

Russel calls this “Software Abundance”

As coding becomes 10 times more accessible, we’ll see a proliferation of what can be called “single-use software”—apps and websites designed for specific, often one-off purposes.

This abundance won’t just democratize software creation; it will fundamentally change how we think about software itself. Imagine tools being created for specific events, or small businesses commissioning custom apps for limited-time campaigns.

What was once impractical will become routine.

As this new reality unfolds, the role of the software engineer will continue to evolve. Just as engineers transitioned from assembly language to high-level languages like Python, they will adapt to a world where English, rather than code, becomes the primary tool of communication.

This change will require a shift in mindset—from focusing on how to write code, to understanding what needs to be accomplished.

The job of a software engineer will become more about defining problems and architecting solutions, with coding agents handling the execution.

Conclusion

We’re on the brink of a transformation that will not only change how we code but also how we think about building software altogether.

And what can help navigate the ambiguities during this phase is Adaptability.

If you’re as excited about this future as I am, hit the subscribe button to stay adaptable to the new possibilities.

Leave a comment below—how do you think AI will change the way you work or build?

Picture this: You're at work, head down, churning out projects like a machine. You're efficient, you're productive—you're doing exactly what's expected of you, right?

Well, not quite.

There's an invisible currency in the workplace that many of us overlook. It's not measured in ROI or any metrics.

It's measured in something immeasurable - your connections.

You know, those seemingly pointless conversations about the weather, your weekend, or what you had for lunch.

For many of us in the engineering world, small talk feels, well… like a waste of time.

But here’s the thing—small talk is actually a big deal. And not just in some fluffy, feel-good way. It can genuinely boost your career. Seriously.

For over a year, I've been diving deep into the concept of adaptability. And I’ve realized something fascinating—being adaptive isn’t just one skill. It’s built on four core pillars

These four pillars are the foundation of thriving in our rapidly changing world.

But today, we’re zooming in on one misunderstood pillar: Communication.

And when we talk about communication in the context of adaptability, we’re talking about something much more fundamental—and ironically, much more casual.

Small Talk.

Historical Context of Small Talk

Let’s take a quick trip back to 1923. An anthropologist named Bronisław Malinowski coined the term ‘phatic communion’. Just a fancy way of saying small talk.

Malinowski described it as a type of speech where connections are made simply through the exchange of words—nothing deep, nothing groundbreaking, just everyday chatter.

But here’s the interesting part: he recognized that this kind of talk, these seemingly meaningless words, actually create bonds between people. It’s a fundamental part of how human beings connect.

In a follow-up study, Coupland et al delved into the subtleties of phatic communion. They revealed that the simple phrase 'How are you?' is more than just a casual question—it's a social tool. It helps people navigate their roles in a conversation, express politeness, and even subtly influence the power dynamics at play.

I also came across an insightful HBR article titled 'The Surprising Power of Simply Asking Coworkers How They’re Doing.' According to a survey featured in the article, 39% of respondents said they feel the greatest sense of belonging when their colleagues check in with them, both personally and professionally."

Now that we have established the perceived benefits of small talk through research and content from respected sources, let me share my perspective on the subject of small talk

My Perspective

I have seen great leaders do small talk

I've had the good fortune of working in three different industries, and in every single company, one thing I've consistently noticed is the most successful leaders don’t dive straight into business.

They use small talk to start meetings, to ease the tension in the room, and to create a comfortable atmosphere.

It’s a simple gesture, but it's incredibly powerful and empathetic. They could just as easily dive into the agenda, but they choose to connect first.

My relationship with my managers

When it comes to my relationship with managers, the connection didn’t stop at work. I’ve bonded with great managers over WWE, comic-book movies, and football.

These conversations showed my personality and gave me a glimpse into theirs. This connection wasn’t just for fun—it helped me manage up more effectively.

And I believe it helped my managers understand how to coach me and deliver tough feedback in a way that I could truly absorb.

Working with colleagues

Working with colleagues across the globe, especially in different cultural contexts, small talk has been a game-changer.

Whenever I met a new colleague from outside India, I made sure to spend the first 10 minutes or so just chatting about anything other than work.

This laid a solid foundation for the meeting and, as a bonus, often led to lasting friendships.

Now that we’ve established why small talk matters, let me share three key benefits that can genuinely impact your career.

3 Key Benefits of Small Talk

Breaking the Ice

Small talk is a great way to ease into more serious discussions. When you engage in light conversation, you create a comfortable environment for deeper conversations later on.

Building Connections

Engaging in small talk allows you to connect with colleagues on a personal level. This connection can lead to trust, making it easier for you to collaborate and work effectively together.

Visibility

Regularly engaging in small talk increases your visibility within the organization. When people see you as approachable and friendly, they are more likely to think of you when opportunities arise.

So, how does all this translate into promotions?

The Impact on Promotions

Networking

The more connections you have, the more advocates you create within your organization. These advocates can vouch for your skills and work ethic when promotion opportunities arise.

Being Top of Mind

When you engage in small talk regularly, you keep yourself on the radar of decision-makers. They are more likely to think of you when considering candidates for promotions.

Creating Allies

Building relationships through small talk can lead to mentorship opportunities. Mentors can provide guidance, support, and even recommend you for promotions.

Conclusion

I’m not suggesting that small talk is the only key to promotions or the most important soft skill.

It’s important to recognize that this advice is based on the assumption that you’re already excellent at your job and fully deserving of that role.

But when the decision comes down to equally talented individuals, small talk can be the factor that sets you apart.

It’s the person who’s approachable, who connects well with others, and who’s seen as easy to work with that often comes to mind first.

My popular article

##