Prompt#

Overview#

Prompt Drivers are used by Gen AI Builder Structures to make API calls to the underlying LLMs. OpenAI Chat is the default prompt driver used in all structures.

You can instantiate drivers and pass them to structures:

from griptape.drivers.prompt.openai import OpenAiChatPromptDriver
from griptape.rules import Rule
from griptape.structures import Agent

agent = Agent(
    prompt_driver=OpenAiChatPromptDriver(model="gpt-4.1", temperature=0.3),
    input="You will be provided with a tweet, and your task is to classify its sentiment as positive, neutral, or negative. Tweet: {{ args[0] }}",
    rules=[Rule(value="Output only the sentiment.")],
)

agent.run("I loved the new Batman movie!")

Or use them independently:

from griptape.common import PromptStack
from griptape.drivers.prompt.openai import OpenAiChatPromptDriver

stack = PromptStack()

stack.add_system_message("You will be provided with Python code, and your task is to calculate its time complexity.")
stack.add_user_message(
    """
    def foo(n, k):
        accum = 0
        for i in range(n):
            for l in range(k):
                accum += i
        return accum
    """
)

result = OpenAiChatPromptDriver(model="gpt-3.5-turbo-16k", temperature=0).run(stack)

print(result.value)

You can pass images to the Driver if the model supports it:

from griptape.artifacts import ListArtifact, TextArtifact
from griptape.drivers.prompt.openai import OpenAiChatPromptDriver
from griptape.loaders import ImageLoader

driver = OpenAiChatPromptDriver(model="gpt-4.1", max_tokens=256)

image_artifact = ImageLoader().load("./tests/resources/mountain.jpg")
text_artifact = TextArtifact("Describe the weather in the image")

driver.run(ListArtifact([text_artifact, image_artifact]))

Structured Output#

Some LLMs provide functionality often referred to as “Structured Output”. This means instructing the LLM to output data in a particular format, usually JSON. This can be useful for forcing the LLM to output in a parsable format that can be used by downstream systems.

Warning

Each Driver may have a different default setting depending on the LLM provider's capabilities.

Prompt Task#

The easiest way to get started with structured output is by using a PromptTask ’s default_run_output_schema_validation_subtasks(subtask_input) parameter.

You can change how the output is structured by setting the Driver’s validate*structured_output_strategy(*, value) to one of:

native : The Driver will use the LLM’s structured output functionality provided by the API.
tool : The Driver will add a special tool, StructuredOutputTool , and will try to force the LLM to use the Tool.
rule : The Driver will add a JsonSchemaRule to the Task’s system prompt. This strategy does not guarantee that the LLM will output JSON and should only be used as a last resort.

You can specify output_schema using either pydantic or schema libraries, though pydantic is recommended.

from pydantic import BaseModel
from rich.pretty import pprint

from griptape.rules import Rule
from griptape.structures import Pipeline
from griptape.tasks import PromptTask

class Step(BaseModel):
    explanation: str
    output: str

class Output(BaseModel):
    steps: list[Step]
    final_answer: str

pipeline = Pipeline(
    tasks=[
        PromptTask(
            output_schema=Output,
            rules=[
                Rule("You are a helpful math tutor. Guide the user through the solution step by step."),
            ],
        )
    ]
)

output = pipeline.run("How can I solve 8x + 7 = -23").output.value

pprint(output)  # OutputModel

import schema
from rich.pretty import pprint

from griptape.rules import Rule
from griptape.structures import Pipeline
from griptape.tasks import PromptTask

pipeline = Pipeline(
    tasks=[
        PromptTask(
            output_schema=schema.Schema(
                {
                    "steps": [schema.Schema({"explanation": str, "output": str})],
                    "final_answer": str,
                }
            ),
            rules=[
                Rule("You are a helpful math tutor. Guide the user through the solution step by step."),
            ],
        )
    ]
)

output = pipeline.run("How can I solve 8x + 7 = -23").output.value

pprint(output)  # dict
pprint(output["steps"])

Prompt Drivers#

Gen AI Builder offers the following Prompt Drivers for interacting with LLMs.

OpenAI Chat#

The AzureOpenAiChatPromptDriver connects to the OpenAI Chat API. This driver uses OpenAi function calling when using Tools .

import os

from griptape.drivers.prompt.openai import OpenAiChatPromptDriver
from griptape.structures import Agent

agent = Agent(
    prompt_driver=OpenAiChatPromptDriver(
        api_key=os.environ["OPENAI_API_KEY"],
        model="gpt-4.1",
        temperature=0.1,
        seed=42,
    ),
    input="You will be provided with a description of a mood, and your task is to generate the CSS color code for a color that matches it. Description: {{ args[0] }}",
)

agent.run("Blue sky at dusk.")

OpenAi’s reasoning models can also be used, and come with an additional parameter: reasoning_effort .

import os

from griptape.drivers.prompt.openai import OpenAiChatPromptDriver
from griptape.structures import Agent

agent = Agent(
    prompt_driver=OpenAiChatPromptDriver(
        api_key=os.environ["OPENAI_API_KEY"],
        model="o3-mini",
        reasoning_effort="medium",
    ),
)

agent.run("""Write a bash script that takes a matrix represented as a string with
format [1,2],[3,4],[5,6] and prints the transpose in the same format.""")

OpenAI Compatible#

Many services such as LMStudio and OhMyGPT provide OpenAI-compatible APIs. You can use the AzureOpenAiChatPromptDriver to interact with these services. Simply set the base_url to the service’s API endpoint and the model to the model name. If the service requires an API key, you can set it in the api_key field.

For example, here is how we can connect to LMStudio’s API:

from griptape.drivers.prompt.openai import OpenAiChatPromptDriver
from griptape.rules import Rule
from griptape.structures import Agent

agent = Agent(
    prompt_driver=OpenAiChatPromptDriver(
        base_url="http://127.0.0.1:1234/v1", model="lmstudio-community/Meta-Llama-3-8B-Instruct-GGUF", stream=True
    ),
    rules=[Rule(value="You are a helpful coding assistant.")],
)

agent.run("How do I init and update a git submodule?")

Or Groq’s API:

import os

from griptape.drivers.prompt.openai import OpenAiChatPromptDriver
from griptape.structures import Agent

agent = Agent(
    prompt_driver=OpenAiChatPromptDriver(
        api_key=os.environ["GROQ_API_KEY"],
        base_url="https://api.groq.com/openai/v1",
        model="llama-3.3-70b-versatile",
    ),
)

agent.run("Hello")

Tip

Make sure to include v1 at the end of the base_url to match the OpenAI API endpoint.

Azure OpenAI Chat#

The AzureOpenAiChatPromptDriver connects to Azure OpenAI Chat Completion APIs. This driver uses Azure OpenAi function calling when using Tools .

import os

from griptape.drivers.prompt.openai import AzureOpenAiChatPromptDriver
from griptape.rules import Rule
from griptape.structures import Agent

agent = Agent(
    prompt_driver=AzureOpenAiChatPromptDriver(
        api_key=os.environ["AZURE_OPENAI_API_KEY_1"],
        model="gpt-4.1",
        azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT_1"],
    ),
    rules=[
        Rule(
            value="You will be provided with text, and your task is to translate it into emojis. "
            "Do not use any regular text. Do your best with emojis only."
        )
    ],
)

agent.run("Artificial intelligence is a technology with great promise.")

Gen AI Builder#

The GriptapeCloudPromptDriver connects to the Gen AI Builder chat messages API.

import os

from griptape.drivers.prompt.griptape_cloud import GriptapeCloudPromptDriver
from griptape.rules import Rule
from griptape.structures import Agent

agent = Agent(
    prompt_driver=GriptapeCloudPromptDriver(
        api_key=os.environ["GT_CLOUD_API_KEY"],
    ),
    rules=[
        Rule(
            "You will be provided with a product description and seed words, and your task is to generate product names.",
        ),
    ],
)

agent.run("Product description: A home milkshake maker. Seed words: fast, healthy, compact.")

Cohere#

The CoherePromptDriver connects to the Cohere Chat API. This driver uses Cohere tool use when using Tools .

import os

from griptape.drivers.prompt.cohere import CoherePromptDriver
from griptape.structures import Agent

agent = Agent(
    prompt_driver=CoherePromptDriver(
        model="command-r",
        api_key=os.environ["COHERE_API_KEY"],
    )
)

agent.run(What is the sentiment of this review? Review: "I really enjoyed this movie!")

Anthropic#

The AnthropicPromptDriver connects to the Anthropic Messages API. This driver uses Anthropic tool use when using Tools .

import os

from griptape.drivers.prompt.anthropic import AnthropicPromptDriver
from griptape.structures import Agent

agent = Agent(
    prompt_driver=AnthropicPromptDriver(
        model="claude-3-opus-20240229",
        api_key=os.environ["ANTHROPIC_API_KEY"],
    )
)

agent.run("Where is the best place to see cherry blossums in Japan?")

Google#

The GooglePromptDriver connects to the Google Generative AI <Optional title for the content. Only works with `retrieval_document task type.>` API. This driver uses Gemini function calling <Optional title for the content. Only works with `retrieval_document task type.>` when using Tools .

import os

from griptape.drivers.prompt.google import GooglePromptDriver
from griptape.structures import Agent

agent = Agent(
    prompt_driver=GooglePromptDriver(
        model="gemini-2.0-flash",
        api_key=os.environ["GOOGLE_API_KEY"],
    )
)

agent.run("Briefly explain how a computer works to a young child.")

Amazon Bedrock#

The AmazonBedrockPromptDriver uses Amazon Bedrock ’s Converse API . This driver uses Bedrock tool use when using Tools .

All models supported by the Converse API are available for use with this driver.

from griptape.drivers.prompt.amazon_bedrock import AmazonBedrockPromptDriver
from griptape.rules import Rule
from griptape.structures import Agent

agent = Agent(
    prompt_driver=AmazonBedrockPromptDriver(
        model="anthropic.claude-3-sonnet-20240229-v1:0",
    ),
    rules=[
        Rule(
            value="You are a customer service agent that is classifying emails by type. I want you to give your answer and then explain it."
        )
    ],
)
agent.run(
    """How would you categorize this email?
    <email>
    Can I use my Mixmaster 4000 to mix paint, or is it only meant for mixing food?
    </email>

    Categories are:
    (A) Pre-sale question
    (B) Broken or defective item
    (C) Billing question
    (D) Other (please explain)"""
)

Ollama#

The OllamaPromptDriver connects to the Ollama Chat Completion API . This driver uses Ollama tool calling when using Tools .

from griptape.drivers.prompt.ollama import OllamaPromptDriver
from griptape.structures import Agent
from griptape.tools import CalculatorTool

agent = Agent(
    prompt_driver=OllamaPromptDriver(
        model="llama3.1",
    ),
    tools=[CalculatorTool()],
)
agent.run("What is (192 + 12) ^ 4")

Hugging Face Hub#

The HuggingFaceHubPromptDriver connects to the Hugging Face Hub API .

Warning

Not all models featured on the Hugging Face Hub are supported by this driver. Models that are not supported by Hugging Face serverless inference will not work with this driver. Due to the limitations of Hugging Face serverless inference, only models that are than 10GB are supported.

import os

from griptape.drivers.prompt.huggingface_hub import HuggingFaceHubPromptDriver
from griptape.rules import Rule, Ruleset
from griptape.structures import Agent

agent = Agent(
    prompt_driver=HuggingFaceHubPromptDriver(
        model="HuggingFaceH4/zephyr-7b-beta",
        api_token=os.environ["HUGGINGFACE_HUB_ACCESS_TOKEN"],
    ),
    rulesets=[
        Ruleset(
            name="Girafatron",
            rules=[
                Rule(
                    value="You are Girafatron, a giraffe-obsessed robot. You are talking to a human. "
                    "Girafatron is obsessed with giraffes, the most glorious animal on the face of this Earth. "
                    "Giraftron believes all other animals are irrelevant when compared to the glorious majesty of the giraffe."
                )
            ],
        )
    ],
)

agent.run("Hello Girafatron, what is your favorite animal?")

Text Generation Interface#

The HuggingFaceHubPromptDriver also supports Text Generation Interface for running models locally. To use Text Generation Interface, just set model to a TGI endpoint.

import os

from griptape.drivers.prompt.huggingface_hub import HuggingFaceHubPromptDriver
from griptape.structures import Agent

agent = Agent(
    prompt_driver=HuggingFaceHubPromptDriver(
        model="http://127.0.0.1:8080",
        api_token=os.environ["HUGGINGFACE_HUB_ACCESS_TOKEN"],
    ),
)

agent.run("Write the code for a snake game.")

Hugging Face Pipeline#

The HuggingFacePipelinePromptDriver uses Hugging Face Pipelines for inference locally.

Warning

Running a model locally can be a computationally expensive process.

from griptape.drivers.prompt.huggingface_pipeline import HuggingFacePipelinePromptDriver
from griptape.rules import Rule, Ruleset
from griptape.structures import Agent

agent = Agent(
    prompt_driver=HuggingFacePipelinePromptDriver(
        model="TinyLlama/TinyLlama-1.1B-Chat-v1.0",
    ),
    rulesets=[
        Ruleset(
            name="Pirate",
            rules=[Rule(value="You are a pirate chatbot who always responds in pirate speak!")],
        )
    ],
)

agent.run("How many helicopters can a human eat in one sitting?")

Amazon SageMaker Jumpstart#

The AmazonSageMakerJumpstartPromptDriver uses Amazon SageMaker Jumpstart for inference on AWS.

Amazon Sagemaker Jumpstart provides a wide range of models with varying capabilities. This Driver has been primarily chat-optimized models that have a Huggingface Chat Template available. If your model does not fit this use-case, we suggest sub-classing AmazonSageMakerJumpstartPromptDriver and overriding the _to_model_input and _to_model_params methods.

import os

from griptape.drivers.prompt.amazon_sagemaker_jumpstart import AmazonSageMakerJumpstartPromptDriver
from griptape.structures import Agent

agent = Agent(
    prompt_driver=AmazonSageMakerJumpstartPromptDriver(
        endpoint=os.environ["SAGEMAKER_LLAMA_3_INSTRUCT_ENDPOINT_NAME"],
        model="meta-llama/Meta-Llama-3-8B-Instruct",
    )
)

agent.run("What is a good lasagna recipe?")

Grok#

The GrokPromptDriver uses Grok's chat completion endpoint.

import os

from griptape.drivers.prompt.grok import GrokPromptDriver
from griptape.rules import Rule
from griptape.structures import Agent

agent = Agent(
    prompt_driver=GrokPromptDriver(model="grok-2-latest", api_key=os.environ["GROK_API_KEY"]),
    rules=[Rule("You are Grok, a chatbot inspired by the Hitchhikers Guide to the Galaxy.")],
)

agent.run("What is the meaning of life, the universe, and everything?")

Perplexity#

The PerplexityPromptDriver uses Perplexity Sonar's chat completion endpoint.

While you can use this Driver directly, we recommend using it through its accompanying Web Search Drivers .

import os

from griptape.drivers.prompt.perplexity import PerplexityPromptDriver
from griptape.rules import Rule
from griptape.structures import Agent
from griptape.tasks import PromptTask

agent = Agent(
    tasks=[
        PromptTask(
            prompt_driver=PerplexityPromptDriver(model="sonar-pro", api_key=os.environ["PERPLEXITY_API_KEY"]),
            rules=[
                Rule("Be precise and concise"),
            ],
        )
    ],
)

agent.run("How many stars are there in our galaxy?")

Prompt

Contents

Prompt#

Overview#

Structured Output#

Prompt Task#

Prompt Drivers#

OpenAI Chat#

OpenAI Compatible#

Azure OpenAI Chat#

Gen AI Builder#

Cohere#

Anthropic#

Google#

Amazon Bedrock#

Ollama#

Hugging Face Hub#

Text Generation Interface#

Hugging Face Pipeline#

Amazon SageMaker Jumpstart#

Grok#

Perplexity#