> ## Documentation Index
> Fetch the complete documentation index at: https://braintrust.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Braintrust Python SDK

> Python API reference for Braintrust SDK v0.5.6

This page provides a complete, auto-generated reference for the Braintrust Python SDK. For usage guidance on important APIs, see the [API Reference](/sdks/python/api-reference). Also see the [Braintrust Python SDK](https://github.com/braintrustdata/braintrust-sdk-python) on GitHub.

## Requirements

* Python 3.10 or higher

## Installation

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
pip install braintrust
```

## Functions

### Eval

A function you can use to define an evaluator. This is a convenience wrapper around the `Evaluator` class.

<ParamField path="name" type="str">
  The name of the evaluator. This corresponds to a project name in Braintrust.
</ParamField>

<ParamField path="data" type="EvalData[Input, Output]">
  Returns an iterator over the evaluation dataset. Each element of the iterator should be a `EvalCase`.
</ParamField>

<ParamField path="task" type="EvalTask[Input, Output]">
  Runs the evaluation task on a single input. The `hooks` object can be used to add metadata to the evaluation.
</ParamField>

<ParamField path="scores" type="Sequence[EvalScorer[Input, Output]]">
  A list of scorers to evaluate the results of the task. Each scorer can be a Scorer object or a function that takes an `EvalScorerArgs` object and returns a `Score` object.
</ParamField>

<ParamField path="experiment_name" type="str | None">
  (Optional) Experiment name. If not specified, a name will be generated automatically.
</ParamField>

<ParamField path="trial_count" type="int">
  The number of times to run the evaluator per input. This is useful for evaluating applications that have non-deterministic behavior and gives you both a stronger aggregate measure and a sense of the variance in the results.
</ParamField>

<ParamField path="metadata" type="Metadata | None">
  (Optional) A dictionary with additional data about the test example, model outputs, or just about anything else that's relevant, that you can use to help find and analyze examples later. For example, you could log the `prompt`, example's `id`, or anything else that would be useful to slice/dice later. The values in `metadata` can be any JSON-serializable type, but its keys must be strings.
</ParamField>

<ParamField path="is_public" type="bool">
  (Optional) Whether the experiment should be public. Defaults to false.
</ParamField>

<ParamField path="update" type="bool" />

<ParamField path="reporter" type="ReporterDef[Input, Output, EvalReport] | None">
  (Optional) A reporter that takes an evaluator and its result and returns a report.
</ParamField>

<ParamField path="reporter.name" type="str" required />

<ParamField path="reporter.report_eval" type="Callable[[Evaluator[Input, Output], EvalResultWithSummary[Input, Output], bool, bool], EvalReport | Awaitable[EvalReport]]" required />

<ParamField path="reporter.report_run" type="Callable[[list[EvalReport], bool, bool], bool | Awaitable[bool]]" required />

<ParamField path="timeout" type="float | None">
  (Optional) The duration, in seconds, after which to time out the evaluation. Defaults to None, in which case there is no timeout.
</ParamField>

<ParamField path="max_concurrency" type="int | None" />

<ParamField path="project_id" type="str | None">
  (Optional) If specified, uses the given project ID instead of the evaluator's name to identify the project.
</ParamField>

<ParamField path="base_experiment_name" type="str | None">
  An optional experiment name to use as a base. If specified, the new experiment will be summarized and compared to this experiment.
</ParamField>

<ParamField path="base_experiment_id" type="str | None">
  An optional experiment id to use as a base. If specified, the new experiment will be summarized and compared to this experiment. This takes precedence over `base_experiment_name` if specified.
</ParamField>

<ParamField path="git_metadata_settings" type="GitMetadataSettings | None">
  Optional settings for collecting git metadata. By default, will collect all git metadata fields allowed in org-level settings.
</ParamField>

<ParamField path="git_metadata_settings.collect" type="Literal['all', 'some', 'none']" required />

<ParamField path="git_metadata_settings.fields" type="list[str] | None" required />

<ParamField path="repo_info" type="RepoInfo | None">
  Optionally explicitly specify the git metadata for this experiment. This takes precedence over `git_metadata_settings` if specified.
</ParamField>

<ParamField path="repo_info.commit" type="str | None" required />

<ParamField path="repo_info.branch" type="str | None" required />

<ParamField path="repo_info.tag" type="str | None" required />

<ParamField path="repo_info.dirty" type="bool | None" required />

<ParamField path="repo_info.author_name" type="str | None" required />

<ParamField path="repo_info.author_email" type="str | None" required />

<ParamField path="repo_info.commit_message" type="str | None" required />

<ParamField path="repo_info.commit_time" type="str | None" required />

<ParamField path="repo_info.git_diff" type="str | None" required />

<ParamField path="error_score_handler" type="ErrorScoreHandler | None">
  Optionally supply a custom function to specifically handle score values when tasks or scoring functions have errored.
</ParamField>

<ParamField path="description" type="str | None">
  An optional description for the experiment.
</ParamField>

<ParamField path="summarize_scores" type="bool">
  Whether to summarize the scores of the experiment after it has run.
</ParamField>

<ParamField path="no_send_logs" type="bool">
  Do not send logs to Braintrust. When True, the evaluation runs locally and builds a local summary instead of creating an experiment. Defaults to False.
</ParamField>

<ParamField path="parameters" type="EvalParameters | None">
  A set of parameters that will be passed to the evaluator.
</ParamField>

<ParamField path="on_start" type="Callable[[ExperimentSummary], None] | None">
  An optional callback that will be called when the evaluation starts. It receives the `ExperimentSummary` object, which can be used to display metadata about the experiment.
</ParamField>

<ParamField path="stream" type="Callable[[SSEProgressEvent], None] | None">
  A function that will be called with progress events, which can be used to display intermediate progress.
</ParamField>

<ParamField path="parent" type="str | None">
  If specified, instead of creating a new experiment object, the Eval() will populate the object or span specified by this parent.
</ParamField>

<ParamField path="state" type="BraintrustState | None">
  Optional BraintrustState to use for the evaluation. If not specified, the global login state will be used.
</ParamField>

<ParamField path="enable_cache" type="bool">
  Whether to enable the span cache for this evaluation. Defaults to True. The span cache stores span data on disk to minimize memory usage and allow scorers to read spans without server round-trips.
</ParamField>

### EvalAsync

A function you can use to define an evaluator. This is a convenience wrapper around the `Evaluator` class.

<ParamField path="name" type="str">
  The name of the evaluator. This corresponds to a project name in Braintrust.
</ParamField>

<ParamField path="data" type="EvalData[Input, Output]">
  Returns an iterator over the evaluation dataset. Each element of the iterator should be a `EvalCase`.
</ParamField>

<ParamField path="task" type="EvalTask[Input, Output]">
  Runs the evaluation task on a single input. The `hooks` object can be used to add metadata to the evaluation.
</ParamField>

<ParamField path="scores" type="Sequence[EvalScorer[Input, Output]]">
  A list of scorers to evaluate the results of the task. Each scorer can be a Scorer object or a function that takes an `EvalScorerArgs` object and returns a `Score` object.
</ParamField>

<ParamField path="experiment_name" type="str | None">
  (Optional) Experiment name. If not specified, a name will be generated automatically.
</ParamField>

<ParamField path="trial_count" type="int">
  The number of times to run the evaluator per input. This is useful for evaluating applications that have non-deterministic behavior and gives you both a stronger aggregate measure and a sense of the variance in the results.
</ParamField>

<ParamField path="metadata" type="Metadata | None">
  (Optional) A dictionary with additional data about the test example, model outputs, or just about anything else that's relevant, that you can use to help find and analyze examples later. For example, you could log the `prompt`, example's `id`, or anything else that would be useful to slice/dice later. The values in `metadata` can be any JSON-serializable type, but its keys must be strings.
</ParamField>

<ParamField path="is_public" type="bool">
  (Optional) Whether the experiment should be public. Defaults to false.
</ParamField>

<ParamField path="update" type="bool" />

<ParamField path="reporter" type="ReporterDef[Input, Output, EvalReport] | None">
  (Optional) A reporter that takes an evaluator and its result and returns a report.
</ParamField>

<ParamField path="reporter.name" type="str" required />

<ParamField path="reporter.report_eval" type="Callable[[Evaluator[Input, Output], EvalResultWithSummary[Input, Output], bool, bool], EvalReport | Awaitable[EvalReport]]" required />

<ParamField path="reporter.report_run" type="Callable[[list[EvalReport], bool, bool], bool | Awaitable[bool]]" required />

<ParamField path="timeout" type="float | None">
  (Optional) The duration, in seconds, after which to time out the evaluation. Defaults to None, in which case there is no timeout.
</ParamField>

<ParamField path="max_concurrency" type="int | None" />

<ParamField path="project_id" type="str | None">
  (Optional) If specified, uses the given project ID instead of the evaluator's name to identify the project.
</ParamField>

<ParamField path="base_experiment_name" type="str | None">
  An optional experiment name to use as a base. If specified, the new experiment will be summarized and compared to this experiment.
</ParamField>

<ParamField path="base_experiment_id" type="str | None">
  An optional experiment id to use as a base. If specified, the new experiment will be summarized and compared to this experiment. This takes precedence over `base_experiment_name` if specified.
</ParamField>

<ParamField path="git_metadata_settings" type="GitMetadataSettings | None">
  Optional settings for collecting git metadata. By default, will collect all git metadata fields allowed in org-level settings.
</ParamField>

<ParamField path="git_metadata_settings.collect" type="Literal['all', 'some', 'none']" required />

<ParamField path="git_metadata_settings.fields" type="list[str] | None" required />

<ParamField path="repo_info" type="RepoInfo | None">
  Optionally explicitly specify the git metadata for this experiment. This takes precedence over `git_metadata_settings` if specified.
</ParamField>

<ParamField path="repo_info.commit" type="str | None" required />

<ParamField path="repo_info.branch" type="str | None" required />

<ParamField path="repo_info.tag" type="str | None" required />

<ParamField path="repo_info.dirty" type="bool | None" required />

<ParamField path="repo_info.author_name" type="str | None" required />

<ParamField path="repo_info.author_email" type="str | None" required />

<ParamField path="repo_info.commit_message" type="str | None" required />

<ParamField path="repo_info.commit_time" type="str | None" required />

<ParamField path="repo_info.git_diff" type="str | None" required />

<ParamField path="error_score_handler" type="ErrorScoreHandler | None">
  Optionally supply a custom function to specifically handle score values when tasks or scoring functions have errored.
</ParamField>

<ParamField path="description" type="str | None">
  An optional description for the experiment.
</ParamField>

<ParamField path="summarize_scores" type="bool">
  Whether to summarize the scores of the experiment after it has run.
</ParamField>

<ParamField path="no_send_logs" type="bool">
  Do not send logs to Braintrust. When True, the evaluation runs locally and builds a local summary instead of creating an experiment. Defaults to False.
</ParamField>

<ParamField path="parameters" type="EvalParameters | None">
  A set of parameters that will be passed to the evaluator.
</ParamField>

<ParamField path="on_start" type="Callable[[ExperimentSummary], None] | None">
  An optional callback that will be called when the evaluation starts. It receives the `ExperimentSummary` object, which can be used to display metadata about the experiment.
</ParamField>

<ParamField path="stream" type="Callable[[SSEProgressEvent], None] | None">
  A function that will be called with progress events, which can be used to display intermediate progress.
</ParamField>

<ParamField path="parent" type="str | None">
  If specified, instead of creating a new experiment object, the Eval() will populate the object or span specified by this parent.
</ParamField>

<ParamField path="state" type="BraintrustState | None">
  Optional BraintrustState to use for the evaluation. If not specified, the global login state will be used.
</ParamField>

<ParamField path="enable_cache" type="bool">
  Whether to enable the span cache for this evaluation. Defaults to True. The span cache stores span data on disk to minimize memory usage and allow scorers to read spans without server round-trips.
</ParamField>

### Reporter

A function you can use to define a reporter. This is a convenience wrapper around the `ReporterDef` class.

<ParamField path="name" type="str">
  The name of the reporter.
</ParamField>

<ParamField path="report_eval" type="Callable[[Evaluator[Input, Output], EvalResultWithSummary[Input, Output], bool, bool], EvalReport | Awaitable[EvalReport]]">
  A function that takes an evaluator and its result and returns a report.
</ParamField>

<ParamField path="report_run" type="Callable[[list[EvalReport], bool, bool], bool | Awaitable[bool]]">
  A function that takes all evaluator results and returns a boolean indicating whether the run was successful.
</ParamField>

### current\_experiment

Returns the currently-active experiment (set by `braintrust.init(...)`). Returns None if no current experiment has been set.

### current\_logger

Returns the currently-active logger (set by `braintrust.init_logger(...)`). Returns None if no current logger has been set.

### current\_span

Return the currently-active span for logging (set by running a span under a context manager). If there is no active span, returns a no-op span object, which supports the same interface as spans but does no logging.

### flush

Flush any pending rows to the server.

### get\_prompt\_versions

Get the versions for a specific prompt.

<ParamField path="project_id" type="str">
  The ID of the project to query
</ParamField>

<ParamField path="prompt_id" type="str">
  The ID of the prompt to get versions for
</ParamField>

### get\_span\_parent\_object

Mainly for internal use. Return the parent object for starting a span in a global context. Applies precedence: current span > propagated parent string > experiment > logger.

<ParamField path="parent" type="str | None" />

<ParamField path="state" type="BraintrustState | None" />

### init

Log in, and then initialize a new experiment in a specified project. If the project does not exist, it will be created.

<ParamField path="project" type="str | None">
  The name of the project to create the experiment in. Must specify at least one of `project` or `project_id`.
</ParamField>

<ParamField path="experiment" type="str | None">
  The name of the experiment to create. If not specified, a name will be generated automatically.
</ParamField>

<ParamField path="description" type="str | None">
  (Optional) An optional description of the experiment.
</ParamField>

<ParamField path="dataset" type="Optional[Dataset] | DatasetRef">
  (Optional) A dataset to associate with the experiment. The dataset must be initialized with `braintrust.init_dataset` before passing it into the experiment.
</ParamField>

<ParamField path="open" type="bool">
  If the experiment already exists, open it in read-only mode. Throws an error if the experiment does not already exist.
</ParamField>

<ParamField path="base_experiment" type="str | None">
  An optional experiment name to use as a base. If specified, the new experiment will be summarized and compared to this experiment. Otherwise, it will pick an experiment by finding the closest ancestor on the default (e.g. main) branch.
</ParamField>

<ParamField path="is_public" type="bool">
  An optional parameter to control whether the experiment is publicly visible to anybody with the link or privately visible to only members of the organization. Defaults to private.
</ParamField>

<ParamField path="app_url" type="str | None">
  The URL of the Braintrust App. Defaults to [https://www.braintrust.dev](https://www.braintrust.dev).
</ParamField>

<ParamField path="api_key" type="str | None">
  The API key to use. If the parameter is not specified, will try to use the `BRAINTRUST_API_KEY` environment variable. If no API key is specified, will prompt the user to login.
</ParamField>

<ParamField path="org_name" type="str | None">
  (Optional) The name of a specific organization to connect to. This is useful if you belong to multiple.
</ParamField>

<ParamField path="metadata" type="Metadata | None">
  (Optional) a dictionary with additional data about the test example, model outputs, or just about anything else that's relevant, that you can use to help find and analyze examples later. For example, you could log the `prompt`, example's `id`, or anything else that would be useful to slice/dice later. The values in `metadata` can be any JSON-serializable type, but its keys must be strings.
</ParamField>

<ParamField path="git_metadata_settings" type="GitMetadataSettings | None">
  (Optional) Settings for collecting git metadata. By default, will collect all git metadata fields allowed in org-level settings.
</ParamField>

<ParamField path="git_metadata_settings.collect" type="Literal['all', 'some', 'none']" required />

<ParamField path="git_metadata_settings.fields" type="list[str] | None" required />

<ParamField path="set_current" type="bool">
  If true (the default), set the global current-experiment to the newly-created one.
</ParamField>

<ParamField path="update" type="bool | None">
  If the experiment already exists, continue logging to it. If it does not exist, creates the experiment with the specified arguments.
</ParamField>

<ParamField path="project_id" type="str | None">
  The id of the project to create the experiment in. This takes precedence over `project` if specified.
</ParamField>

<ParamField path="base_experiment_id" type="str | None">
  An optional experiment id to use as a base. If specified, the new experiment will be summarized and compared to this. This takes precedence over `base_experiment` if specified.
</ParamField>

<ParamField path="repo_info" type="RepoInfo | None">
  (Optional) Explicitly specify the git metadata for this experiment. This takes precedence over `git_metadata_settings` if specified.
</ParamField>

<ParamField path="repo_info.commit" type="str | None" required />

<ParamField path="repo_info.branch" type="str | None" required />

<ParamField path="repo_info.tag" type="str | None" required />

<ParamField path="repo_info.dirty" type="bool | None" required />

<ParamField path="repo_info.author_name" type="str | None" required />

<ParamField path="repo_info.author_email" type="str | None" required />

<ParamField path="repo_info.commit_message" type="str | None" required />

<ParamField path="repo_info.commit_time" type="str | None" required />

<ParamField path="repo_info.git_diff" type="str | None" required />

<ParamField path="state" type="BraintrustState | None">
  (Optional) A BraintrustState object to use. If not specified, will use the global state. This is for advanced use only.
</ParamField>

### init\_dataset

Create a new dataset in a specified project. If the project does not exist, it will be created.

<ParamField path="project" type="str | None" />

<ParamField path="name" type="str | None">
  The name of the dataset to create. If not specified, a name will be generated automatically.
</ParamField>

<ParamField path="description" type="str | None">
  An optional description of the dataset.
</ParamField>

<ParamField path="version" type="str | int | None">
  An optional version of the dataset (to read). If not specified, the latest version will be used.
</ParamField>

<ParamField path="app_url" type="str | None">
  The URL of the Braintrust App. Defaults to [https://www.braintrust.dev](https://www.braintrust.dev).
</ParamField>

<ParamField path="api_key" type="str | None">
  The API key to use. If the parameter is not specified, will try to use the `BRAINTRUST_API_KEY` environment variable. If no API key is specified, will prompt the user to login.
</ParamField>

<ParamField path="org_name" type="str | None">
  (Optional) The name of a specific organization to connect to. This is useful if you belong to multiple.
</ParamField>

<ParamField path="project_id" type="str | None">
  The id of the project to create the dataset in. This takes precedence over `project` if specified.
</ParamField>

<ParamField path="metadata" type="Metadata | None">
  (Optional) a dictionary with additional data about the dataset. The values in `metadata` can be any JSON-serializable type, but its keys must be strings.
</ParamField>

<ParamField path="use_output" type="bool">
  (Deprecated) If True, records will be fetched from this dataset in the legacy format, with the "expected" field renamed to "output". This option will be removed in a future version of Braintrust.
</ParamField>

<ParamField path="_internal_btql" type="dict[str, Any] | None">
  (Internal) If specified, the dataset will be created with the given BTQL filters.
</ParamField>

<ParamField path="state" type="BraintrustState | None">
  (Internal) The Braintrust state to use. If not specified, will use the global state. For advanced use only.
</ParamField>

### init\_experiment

Alias for `init`

<ParamField path="args" type="Any" />

<ParamField path="kwargs" type="Any" />

### init\_function

Creates a function that can be used as either a task or scorer in the Eval framework. When used as a task, it will invoke the specified Braintrust function with the input. When used as a scorer, it will invoke the function with the scorer arguments.

<ParamField path="project_name" type="str">
  The name of the project containing the function.
</ParamField>

<ParamField path="slug" type="str">
  The slug of the function to invoke.
</ParamField>

<ParamField path="version" type="str | None">
  Optional version of the function to use. Defaults to latest.
</ParamField>

### init\_logger

Create a new logger in a specified project. If the project does not exist, it will be created.

<ParamField path="project" type="str | None">
  The name of the project to log into. If unspecified, will default to the Global project.
</ParamField>

<ParamField path="project_id" type="str | None">
  The id of the project to log into. This takes precedence over project if specified.
</ParamField>

<ParamField path="async_flush" type="bool">
  If true (the default), log events will be batched and sent asynchronously in a background thread. If false, log events will be sent synchronously. Set to false in serverless environments.
</ParamField>

<ParamField path="app_url" type="str | None">
  The URL of the Braintrust API. Defaults to [https://www.braintrust.dev](https://www.braintrust.dev).
</ParamField>

<ParamField path="api_key" type="str | None">
  The API key to use. If the parameter is not specified, will try to use the `BRAINTRUST_API_KEY` environment variable. If no API key is specified, will prompt the user to login.
</ParamField>

<ParamField path="org_name" type="str | None">
  (Optional) The name of a specific organization to connect to. This is useful if you belong to multiple.
</ParamField>

<ParamField path="force_login" type="bool">
  Login again, even if you have already logged in (by default, the logger will not login if you are already logged in)
</ParamField>

<ParamField path="set_current" type="bool">
  If true (the default), set the global current-experiment to the newly-created one.
</ParamField>

<ParamField path="state" type="BraintrustState | None" />

### invoke

Invoke a Braintrust function, returning a `BraintrustStream` or the value as a plain Python object.

<ParamField path="function_id" type="str | None">
  The ID of the function to invoke.
</ParamField>

<ParamField path="version" type="str | None">
  The version of the function to invoke.
</ParamField>

<ParamField path="prompt_session_id" type="str | None">
  The ID of the prompt session to invoke the function from.
</ParamField>

<ParamField path="prompt_session_function_id" type="str | None">
  The ID of the function in the prompt session to invoke.
</ParamField>

<ParamField path="project_name" type="str | None">
  The name of the project containing the function to invoke.
</ParamField>

<ParamField path="project_id" type="str | None">
  The ID of the project to use for execution context (API keys, project defaults, etc.).
  This is not the project the function belongs to, but the project context for the invocation.
</ParamField>

<ParamField path="slug" type="str | None">
  The slug of the function to invoke.
</ParamField>

<ParamField path="global_function" type="str | None">
  The name of the global function to invoke.
</ParamField>

<ParamField path="function_type" type="FunctionTypeEnum | None">
  The type of the global function to invoke. If unspecified, defaults to 'scorer'
  for backward compatibility.
</ParamField>

<ParamField path="input" type="Any">
  The input to the function. This will be logged as the `input` field in the span.
</ParamField>

<ParamField path="messages" type="list[Any] | None">
  Additional OpenAI-style messages to add to the prompt (only works for llm functions).
</ParamField>

<ParamField path="metadata" type="dict[str, Any] | None">
  Additional metadata to add to the span. This will be logged as the `metadata` field in the span.
  It will also be available as the \{\{metadata}} field in the prompt and as the `metadata` argument
  to the function.
</ParamField>

<ParamField path="tags" type="list[str] | None">
  Tags to add to the span. This will be logged as the `tags` field in the span.
</ParamField>

<ParamField path="parent" type="Exportable | str | None">
  The parent of the function. This can be an existing span, logger, or experiment, or
  the output of `.export()` if you are distributed tracing. If unspecified, will use
  the same semantics as `traced()` to determine the parent and no-op if not in a tracing
  context.
</ParamField>

<ParamField path="stream" type="bool">
  Whether to stream the function's output. If True, the function will return a
  `BraintrustStream`, otherwise it will return the output of the function as a JSON
  object.
</ParamField>

<ParamField path="mode" type="ModeType | None">
  The response shape of the function if returning tool calls. If "auto", will return
  a string if the function returns a string, and a JSON object otherwise. If "parallel",
  will return an array of JSON objects with one object per tool call.
</ParamField>

<ParamField path="strict" type="bool | None">
  Whether to use strict mode for the function. If true, the function will throw an
  error if the variable names in the prompt do not match the input keys.
</ParamField>

<ParamField path="org_name" type="str | None">
  The name of the Braintrust organization to use.
</ParamField>

<ParamField path="api_key" type="str | None">
  The API key to use for authentication.
</ParamField>

<ParamField path="app_url" type="str | None">
  The URL of the Braintrust application.
</ParamField>

<ParamField path="force_login" type="bool">
  Whether to force a new login even if already logged in.
</ParamField>

### load\_prompt

Loads a prompt from the specified project.

<ParamField path="project" type="str | None">
  The name of the project to load the prompt from. Must specify at least one of `project` or `project_id`.
</ParamField>

<ParamField path="slug" type="str | None">
  The slug of the prompt to load.
</ParamField>

<ParamField path="version" type="str | int | None">
  An optional version of the prompt (to read). If not specified, the latest version will be used.
</ParamField>

<ParamField path="project_id" type="str | None">
  The id of the project to load the prompt from. This takes precedence over `project` if specified.
</ParamField>

<ParamField path="id" type="str | None">
  The id of a specific prompt to load. If specified, this takes precedence over all other parameters (project, slug, version).
</ParamField>

<ParamField path="defaults" type="Mapping[str, Any] | None">
  (Optional) A dictionary of default values to use when rendering the prompt. Prompt values will override these defaults.
</ParamField>

<ParamField path="no_trace" type="bool">
  If true, do not include logging metadata for this prompt when build() is called.
</ParamField>

<ParamField path="environment" type="str | None">
  The environment to load the prompt from. Cannot be used together with version.
</ParamField>

<ParamField path="app_url" type="str | None">
  The URL of the Braintrust App. Defaults to [https://www.braintrust.dev](https://www.braintrust.dev).
</ParamField>

<ParamField path="api_key" type="str | None">
  The API key to use. If the parameter is not specified, will try to use the `BRAINTRUST_API_KEY` environment variable. If no API key is specified, will prompt the user to login.
</ParamField>

<ParamField path="org_name" type="str | None">
  (Optional) The name of a specific organization to connect to. This is useful if you belong to multiple.
</ParamField>

### log

Log a single event to the current experiment. The event will be batched and uploaded behind the scenes.

<ParamField path="event" type="Any" />

### login

Log into Braintrust. This will prompt you for your API token, which you can find at [https://www.braintrust.dev/app/token](https://www.braintrust.dev/app/token). This method is called automatically by `init()`.

<ParamField path="app_url" type="str | None">
  The URL of the Braintrust App. Defaults to [https://www.braintrust.dev](https://www.braintrust.dev).
</ParamField>

<ParamField path="api_key" type="str | None">
  The API key to use. If the parameter is not specified, will try to use the `BRAINTRUST_API_KEY` environment variable. If no API key is specified, will prompt the user to login.
</ParamField>

<ParamField path="org_name" type="str | None">
  (Optional) The name of a specific organization to connect to. This is useful if you belong to multiple.
</ParamField>

<ParamField path="force_login" type="bool">
  Login again, even if you have already logged in (by default, this function will exit quickly if you have already logged in)
</ParamField>

### parent\_context

Context manager to temporarily set the parent context for spans.

<ParamField path="parent" type="str | None">
  The parent string to set during the context
</ParamField>

<ParamField path="state" type="BraintrustState | None">
  Optional BraintrustState to use. If not provided, uses the global state.
</ParamField>

### parse\_stream

Parse a BraintrustStream into its final value.

<ParamField path="stream" type="BraintrustStream">
  The BraintrustStream to parse.
</ParamField>

### patch\_anthropic

Patch Anthropic to add Braintrust tracing globally.

### patch\_litellm

Patch LiteLLM to add Braintrust tracing.

### patch\_openai

Patch OpenAI to add Braintrust tracing globally.

### permalink

Format a permalink to the Braintrust application for viewing the span represented by the provided `slug`.

<ParamField path="slug" type="str">
  The identifier generated from `Span.export`.
</ParamField>

<ParamField path="org_name" type="str | None">
  The org name to use. If not provided, the org name will be inferred from the global login state.
</ParamField>

<ParamField path="app_url" type="str | None">
  The app URL to use. If not provided, the app URL will be inferred from the global login state.
</ParamField>

### prettify\_params

Clean up parameters by filtering out NOT\_GIVEN values and serializing response\_format.

<ParamField path="params" type="dict[str, Any]" />

### register\_otel\_flush

Register a callback to flush OTEL spans. This is called by the OTEL integration when it initializes a span processor/exporter.

<ParamField path="callback" type="Any">
  The async callback function to flush OTEL spans.
</ParamField>

### run\_evaluator

Wrapper on \_run\_evaluator\_internal that times out execution after evaluator.timeout.

<ParamField path="experiment" type="Experiment | None" />

<ParamField path="evaluator" type="Evaluator[Input, Output]" />

<ParamField path="evaluator.project_name" type="str" required />

<ParamField path="evaluator.eval_name" type="str" required />

<ParamField path="evaluator.data" type="EvalData[Input, Output]" required />

<ParamField path="evaluator.task" type="EvalTask[Input, Output]" required />

<ParamField path="evaluator.scores" type="list[EvalScorer[Input, Output]]" required />

<ParamField path="evaluator.experiment_name" type="str | None" required />

<ParamField path="evaluator.metadata" type="Metadata | None" required />

<ParamField path="evaluator.trial_count" type="int" required />

<ParamField path="evaluator.is_public" type="bool" required />

<ParamField path="evaluator.update" type="bool" required />

<ParamField path="evaluator.timeout" type="float | None" required />

<ParamField path="evaluator.max_concurrency" type="int | None" required />

<ParamField path="evaluator.project_id" type="str | None" required />

<ParamField path="evaluator.base_experiment_name" type="str | None" required />

<ParamField path="evaluator.base_experiment_id" type="str | None" required />

<ParamField path="evaluator.git_metadata_settings" type="GitMetadataSettings | None" required />

<ParamField path="evaluator.repo_info" type="RepoInfo | None" required />

<ParamField path="evaluator.error_score_handler" type="ErrorScoreHandler | None" required />

<ParamField path="evaluator.description" type="str | None" required />

<ParamField path="evaluator.summarize_scores" type="bool" required />

<ParamField path="evaluator.parameters" type="EvalParameters | None" required />

<ParamField path="position" type="int | None" />

<ParamField path="filters" type="list[Filter]" />

<ParamField path="stream" type="Callable[[SSEProgressEvent], None] | None" />

<ParamField path="state" type="BraintrustState | None" />

<ParamField path="enable_cache" type="bool" />

### serialize\_response\_format

Serialize response format for logging.

<ParamField path="response_format" type="Any" />

### set\_http\_adapter

Specify a custom HTTP adapter to use for all network requests. This is useful for setting custom retry policies, timeouts, etc. Braintrust uses the `requests` library, so the adapter should be an instance of `requests.adapters.HTTPAdapter`. Alternatively, consider sub-classing our `RetryRequestExceptionsAdapter` to get automatic retries on network-related exceptions.

<ParamField path="adapter" type="HTTPAdapter">
  The adapter to use.
</ParamField>

### set\_masking\_function

Set a global masking function that will be applied to all logged data before sending to Braintrust. The masking function will be applied after records are merged but before they are sent to the backend.

<ParamField path="masking_function" type="Callable[[Any], Any] | None">
  A function that takes a JSON-serializable object and returns a masked version. Set to None to disable masking.
</ParamField>

### set\_thread\_pool\_max\_workers

Set the maximum number of threads to use for running evaluators. By default, this is the number of CPUs on the machine.

<ParamField path="max_workers" type="Any" />

### span\_components\_to\_object\_id

Utility function to resolve the object ID of a SpanComponentsV4 object. This function may trigger a login to braintrust if the object ID is encoded lazily.

<ParamField path="components" type="SpanComponentsV4" />

<ParamField path="components.object_type" type="SpanObjectTypeV3" required />

<ParamField path="components.object_id" type="str | None" required />

<ParamField path="components.compute_object_metadata_args" type="dict | None" required />

<ParamField path="components.row_id" type="str | None" required />

<ParamField path="components.span_id" type="str | None" required />

<ParamField path="components.root_span_id" type="str | None" required />

<ParamField path="components.propagated_event" type="dict | None" required />

### start\_span

Lower-level alternative to `@traced` for starting a span at the toplevel. It creates a span under the first active object (using the same precedence order as `@traced`), or if `parent` is specified, under the specified parent row, or returns a no-op span object.

<ParamField path="name" type="str | None" />

<ParamField path="type" type="SpanTypeAttribute | None" />

<ParamField path="span_attributes" type="SpanAttributes | Mapping[str, Any] | None" />

<ParamField path="start_time" type="float | None" />

<ParamField path="set_current" type="bool | None" />

<ParamField path="parent" type="str | None" />

<ParamField path="propagated_event" type="dict[str, Any] | None" />

<ParamField path="state" type="BraintrustState | None" />

<ParamField path="event" type="Any" />

### summarize

Summarize the current experiment, including the scores (compared to the closest reference experiment) and metadata.

<ParamField path="summarize_scores" type="bool">
  Whether to summarize the scores. If False, only the metadata will be returned.
</ParamField>

<ParamField path="comparison_experiment_id" type="str | None">
  The experiment to compare against. If None, the most recent experiment on the comparison\_commit will be used.
</ParamField>

### traced

Decorator to trace the wrapped function. Can either be applied bare (`@traced`) or by providing arguments (`@traced(*span_args, **span_kwargs)`), which will be forwarded to the created span. See `Span.start_span` for full details on the span arguments.

<ParamField path="span_args" type="Any" />

<ParamField path="span_kwargs" type="Any" />

### update\_span

Update a span using the output of `span.export()`. It is important that you only resume updating to a span once the original span has been fully written and flushed, since otherwise updates to the span may conflict with the original span.

<ParamField path="exported" type="str">
  The output of `span.export()`.
</ParamField>

<ParamField path="event" type="Any" />

### wrap\_anthropic

Wrap an `Anthropic` object (or AsyncAnthropic) to add tracing. If Braintrust is not configured, this is a no-op. If this is not an `Anthropic` object, this function is a no-op.

<ParamField path="client" type="Any" />

### wrap\_litellm

Wrap the litellm module to add tracing. If Braintrust is not configured, nothing will be traced.

<ParamField path="litellm_module" type="Any">
  The litellm module
</ParamField>

### wrap\_openai

Wrap the openai module (pre v1) or OpenAI instance (post v1) to add tracing. If Braintrust is not configured, nothing will be traced. If this is not an `OpenAI` object, this function is a no-op.

<ParamField path="openai" type="Any">
  The openai module or OpenAI object
</ParamField>

## Classes

### AsyncResponseWrapper

Wrapper that properly preserves async context manager behavior for OpenAI responses.

<span class="text-sm">Methods</span>

`__init__()`

### AsyncScorerLike

Protocol for asynchronous scorers that implement the eval\_async interface. The framework will prefer this interface if available.

<span class="text-sm">Methods</span>

`eval_async()`

### Attachment

Represents an attachment to be uploaded and the associated metadata.

<span class="text-sm">Properties</span>

<ParamField path="reference" type="AttachmentReference">
  The object that replaces this `Attachment` at upload time.
</ParamField>

<ParamField path="data" type="bytes">
  The attachment contents. This is a lazy value that will read the attachment contents from disk or memory on first access.
</ParamField>

<span class="text-sm">Methods</span>

`__init__()`, `upload()`, `debug_info()`

### BaseExperiment

Use this to specify that the dataset should actually be the data from a previous (base) experiment. If you do not specify a name, Braintrust will automatically figure out the best base experiment to use based on your git history (or fall back to timestamps).

<span class="text-sm">Properties</span>

<ParamField path="name" type="str | None">
  The name of the base experiment to use. If unspecified, Braintrust will automatically figure out the best base
  using your git history (or fall back to timestamps).
</ParamField>

<span class="text-sm">Methods</span>

`__init__()`

### BraintrustConsoleChunk

A console chunk from a Braintrust stream.

<span class="text-sm">Properties</span>

<ParamField path="message" type="str" />

<ParamField path="stream" type="Literal['stderr', 'stdout']" />

<ParamField path="type" type="Literal['console']" />

<span class="text-sm">Methods</span>

`__init__()`

### BraintrustErrorChunk

An error chunk from a Braintrust stream.

<span class="text-sm">Properties</span>

<ParamField path="data" type="str" />

<ParamField path="type" type="Literal['error']" />

<span class="text-sm">Methods</span>

`__init__()`

### BraintrustInvokeError

An error that occurs during a Braintrust stream.

### BraintrustJsonChunk

A chunk of JSON data from a Braintrust stream.

<span class="text-sm">Properties</span>

<ParamField path="data" type="str" />

<ParamField path="type" type="Literal['json_delta']" />

<span class="text-sm">Methods</span>

`__init__()`

### BraintrustProgressChunk

A progress chunk from a Braintrust stream.

<span class="text-sm">Properties</span>

<ParamField path="data" type="str" />

<ParamField path="id" type="str" />

<ParamField path="object_type" type="str" />

<ParamField path="format" type="str" />

<ParamField path="output_type" type="str" />

<ParamField path="name" type="str" />

<ParamField path="event" type="Literal['json_delta', 'text_delta', 'reasoning_delta']" />

<ParamField path="type" type="Literal['progress']" />

<span class="text-sm">Methods</span>

`__init__()`

### BraintrustStream

A Braintrust stream. This is a wrapper around a generator of `BraintrustStreamChunk`, with utility methods to make them easy to log and convert into various formats.

<span class="text-sm">Properties</span>

<ParamField path="stream" type="Any" />

<span class="text-sm">Methods</span>

`__init__()`, `copy()`, `final_value()`

### BraintrustTextChunk

A chunk of text data from a Braintrust stream.

<span class="text-sm">Properties</span>

<ParamField path="data" type="str" />

<ParamField path="type" type="Literal['text_delta']" />

<span class="text-sm">Methods</span>

`__init__()`

### CodeFunction

A generic callable, with metadata.

<span class="text-sm">Properties</span>

<ParamField path="project" type="Project" />

<ParamField path="handler" type="Callable[..., Any]" />

<ParamField path="name" type="str" />

<ParamField path="slug" type="str" />

<ParamField path="type_" type="str" />

<ParamField path="description" type="str | None" />

<ParamField path="parameters" type="Any" />

<ParamField path="returns" type="Any" />

<ParamField path="if_exists" type="IfExists | None" />

<ParamField path="metadata" type="dict[str, Any] | None" />

<span class="text-sm">Methods</span>

`__init__()`

### CodePrompt

A prompt defined in code, with metadata.

<span class="text-sm">Properties</span>

<ParamField path="project" type="Project" />

<ParamField path="name" type="str" />

<ParamField path="slug" type="str" />

<ParamField path="prompt" type="PromptData" />

<ParamField path="tool_functions" type="list[CodeFunction | SavedFunctionId]" />

<ParamField path="description" type="str | None" />

<ParamField path="function_type" type="str | None" />

<ParamField path="id" type="str | None" />

<ParamField path="if_exists" type="IfExists | None" />

<ParamField path="metadata" type="dict[str, Any] | None" />

<span class="text-sm">Methods</span>

`to_function_definition()`, `__init__()`

### CompletionWrapper

Wrapper for LiteLLM completion functions with tracing support.

<span class="text-sm">Properties</span>

<ParamField path="completion_fn" type="Any" />

<ParamField path="acompletion_fn" type="Any" />

<span class="text-sm">Methods</span>

`__init__()`, `completion()`, `acompletion()`

### DataSummary

Summary of a dataset's data.

<span class="text-sm">Properties</span>

<ParamField path="new_records" type="int">
  New or updated records added in this session.
</ParamField>

<ParamField path="total_records" type="int">
  Total records in the dataset.
</ParamField>

<span class="text-sm">Methods</span>

`__init__()`

### Dataset

A dataset is a collection of records, such as model inputs and outputs, which represent data you can use to evaluate and fine-tune models. You can log production data to datasets, curate them with interesting examples, edit/delete records, and run evaluations against them.

<span class="text-sm">Properties</span>

<ParamField path="new_records" type="Any" />

<ParamField path="state" type="Any" />

<ParamField path="id" type="str" />

<ParamField path="name" type="str" />

<ParamField path="data" type="Any" />

<ParamField path="project" type="Any" />

<ParamField path="logging_state" type="BraintrustState" />

<span class="text-sm">Methods</span>

`__init__()`, `insert()`, `update()`, `delete()`, `summarize()`, `close()`, `flush()`

### DatasetRef

Reference to a dataset by ID and optional version.

<span class="text-sm">Properties</span>

<ParamField path="id" type="str" />

<ParamField path="version" type="str" />

### DatasetSummary

Summary of a dataset's scores and metadata.

<span class="text-sm">Properties</span>

<ParamField path="project_name" type="str">
  Name of the project that the dataset belongs to.
</ParamField>

<ParamField path="dataset_name" type="str">
  Name of the dataset.
</ParamField>

<ParamField path="project_url" type="str">
  URL to the project's page in the Braintrust app.
</ParamField>

<ParamField path="dataset_url" type="str">
  URL to the experiment's page in the Braintrust app.
</ParamField>

<ParamField path="data_summary" type="DataSummary | None">
  Summary of the dataset's data.
</ParamField>

<span class="text-sm">Methods</span>

`__init__()`

### EmbeddingWrapper

Wrapper for LiteLLM embedding functions.

<span class="text-sm">Properties</span>

<ParamField path="embedding_fn" type="Any" />

<span class="text-sm">Methods</span>

`__init__()`, `embedding()`

### EvalCase

An evaluation case. This is a single input to the evaluation task, along with an optional expected output, metadata, and tags.

<span class="text-sm">Properties</span>

<ParamField path="input" type="Input" />

<ParamField path="expected" type="Output | None" />

<ParamField path="metadata" type="Metadata | None" />

<ParamField path="tags" type="Sequence[str] | None" />

<ParamField path="id" type="str | None" />

<ParamField path="created" type="str | None" />

<span class="text-sm">Methods</span>

`__init__()`

### EvalHooks

An object that can be used to add metadata to an evaluation. This is passed to the `task` function.

<span class="text-sm">Properties</span>

<ParamField path="metadata" type="Metadata">
  The metadata object for the current evaluation. You can mutate this object to add or remove metadata.
</ParamField>

<ParamField path="expected" type="Output | None">
  The expected output for the current evaluation.
</ParamField>

<ParamField path="span" type="Span">
  Access the span under which the task is run. Also accessible via braintrust.current\_span()
</ParamField>

<ParamField path="trial_index" type="int">
  The index of the current trial (0-based). This is useful when trial\_count > 1.
</ParamField>

<ParamField path="tags" type="Sequence[str]">
  The tags for the current evaluation. You can mutate this object to add or remove tags.
</ParamField>

<ParamField path="parameters" type="dict[str, Any] | None">
  The parameters for the current evaluation. These are the validated parameter values
  that were passed to the evaluator.
</ParamField>

<span class="text-sm">Methods</span>

`report_progress()`, `meta()`

### EvalResult

The result of an evaluation. This includes the input, expected output, actual output, and metadata.

<span class="text-sm">Properties</span>

<ParamField path="input" type="Input" />

<ParamField path="output" type="Output" />

<ParamField path="scores" type="dict[str, float | None]" />

<ParamField path="expected" type="Output | None" />

<ParamField path="metadata" type="Metadata | None" />

<ParamField path="tags" type="list[str] | None" />

<ParamField path="error" type="Exception | None" />

<ParamField path="exc_info" type="str | None" />

<span class="text-sm">Methods</span>

`__init__()`

### EvalScorerArgs

Arguments passed to an evaluator scorer. This includes the input, expected output, actual output, and metadata.

<span class="text-sm">Properties</span>

<ParamField path="input" type="Input" />

<ParamField path="output" type="Output" />

<ParamField path="expected" type="Output | None" />

<ParamField path="metadata" type="Metadata | None" />

### Evaluator

An evaluator is an abstraction that defines an evaluation dataset, a task to run on the dataset, and a set of scorers to evaluate the results of the task. Each method attribute can be synchronous or asynchronous (for optimal performance, it is recommended to provide asynchronous implementations).

<span class="text-sm">Properties</span>

<ParamField path="project_name" type="str">
  The name of the project the eval falls under.
</ParamField>

<ParamField path="eval_name" type="str">
  A name that describes the experiment. You do not need to change it each time the experiment runs.
</ParamField>

<ParamField path="data" type="EvalData[Input, Output]">
  Returns an iterator over the evaluation dataset. Each element of the iterator should be an `EvalCase` or a dict
  with the same fields as an `EvalCase` (`input`, `expected`, `metadata`).
</ParamField>

<ParamField path="task" type="EvalTask[Input, Output]">
  Runs the evaluation task on a single input. The `hooks` object can be used to add metadata to the evaluation.
</ParamField>

<ParamField path="scores" type="list[EvalScorer[Input, Output]]">
  A list of scorers to evaluate the results of the task. Each scorer can be a Scorer object or a function
  that takes `input`, `output`, and `expected` arguments and returns a `Score` object. The function can be async.
</ParamField>

<ParamField path="experiment_name" type="str | None">
  Optional experiment name. If not specified, a name will be generated automatically.
</ParamField>

<ParamField path="metadata" type="Metadata | None">
  A dictionary with additional data about the test example, model outputs, or just about anything else that's
  relevant, that you can use to help find and analyze examples later. For example, you could log the `prompt`,
  example's `id`, or anything else that would be useful to slice/dice later. The values in `metadata` can be any
  JSON-serializable type, but its keys must be strings.
</ParamField>

<ParamField path="trial_count" type="int">
  The number of times to run the evaluator per input. This is useful for evaluating applications that
  have non-deterministic behavior and gives you both a stronger aggregate measure and a sense of the
  variance in the results.
</ParamField>

<ParamField path="is_public" type="bool">
  Whether the experiment should be public. Defaults to false.
</ParamField>

<ParamField path="update" type="bool">
  Whether to update an existing experiment with `experiment_name` if one exists. Defaults to false.
</ParamField>

<ParamField path="timeout" type="float | None">
  The duration, in seconds, after which to time out the evaluation.
  Defaults to None, in which case there is no timeout.
</ParamField>

<ParamField path="max_concurrency" type="int | None">
  The maximum number of tasks/scorers that will be run concurrently.
  Defaults to None, in which case there is no max concurrency.
</ParamField>

<ParamField path="project_id" type="str | None">
  If specified, uses the given project ID instead of the evaluator's name to identify the project.
</ParamField>

<ParamField path="base_experiment_name" type="str | None">
  An optional experiment name to use as a base. If specified, the new experiment will be summarized and
  compared to this experiment.
</ParamField>

<ParamField path="base_experiment_id" type="str | None">
  An optional experiment id to use as a base. If specified, the new experiment will be summarized and
  compared to this experiment. This takes precedence over `base_experiment_name` if specified.
</ParamField>

<ParamField path="git_metadata_settings" type="GitMetadataSettings | None">
  Optional settings for collecting git metadata. By default, will collect all
  git metadata fields allowed in org-level settings.
</ParamField>

<ParamField path="repo_info" type="RepoInfo | None">
  Optionally explicitly specify the git metadata for this experiment. This
  takes precedence over `git_metadata_settings` if specified.
</ParamField>

<ParamField path="error_score_handler" type="ErrorScoreHandler | None">
  Optionally supply a custom function to specifically handle score values when tasks or scoring functions have errored.
  A default implementation is exported as `default_error_score_handler` which will log a 0 score to the root span for any scorer that was not run.
</ParamField>

<ParamField path="description" type="str | None">
  An optional description for the experiment.
</ParamField>

<ParamField path="summarize_scores" type="bool">
  Whether to summarize the scores of the experiment after it has run.
</ParamField>

<ParamField path="parameters" type="EvalParameters | None">
  A set of parameters that will be passed to the evaluator.
  Can be used to define prompts or other configurable values.
</ParamField>

<span class="text-sm">Methods</span>

`__init__()`

### Experiment

An experiment is a collection of logged events, such as model inputs and outputs, which represent a snapshot of your application at a particular point in time. An experiment is meant to capture more than just the model you use, and includes the data you use to test, pre- and post- processing code, comparison metrics (scores), and any other metadata you want to include.

<span class="text-sm">Properties</span>

<ParamField path="dataset" type="Any" />

<ParamField path="last_start_time" type="Any" />

<ParamField path="state" type="Any" />

<ParamField path="id" type="str" />

<ParamField path="name" type="str" />

<ParamField path="data" type="Mapping[str, Any]" />

<ParamField path="project" type="ObjectMetadata" />

<ParamField path="logging_state" type="BraintrustState" />

<span class="text-sm">Methods</span>

`__init__()`, `log()`, `log_feedback()`, `start_span()`, `update_span()`, `fetch_base_experiment()`, `summarize()`, `export()`, `close()`, `flush()`

### ExperimentSummary

Summary of an experiment's scores and metadata.

<span class="text-sm">Properties</span>

<ParamField path="project_name" type="str">
  Name of the project that the experiment belongs to.
</ParamField>

<ParamField path="project_id" type="str | None">
  ID of the project. May be `None` if the eval was run locally.
</ParamField>

<ParamField path="experiment_id" type="str | None">
  ID of the experiment. May be `None` if the eval was run locally.
</ParamField>

<ParamField path="experiment_name" type="str">
  Name of the experiment.
</ParamField>

<ParamField path="project_url" type="str | None">
  URL to the project's page in the Braintrust app.
</ParamField>

<ParamField path="experiment_url" type="str | None">
  URL to the experiment's page in the Braintrust app.
</ParamField>

<ParamField path="comparison_experiment_name" type="str | None">
  The experiment scores are baselined against.
</ParamField>

<ParamField path="scores" type="dict[str, ScoreSummary]">
  Summary of the experiment's scores.
</ParamField>

<ParamField path="metrics" type="dict[str, MetricSummary]">
  Summary of the experiment's metrics.
</ParamField>

<span class="text-sm">Methods</span>

`__init__()`

### ExternalAttachment

Represents an attachment that resides in an external object store and the associated metadata.

<span class="text-sm">Properties</span>

<ParamField path="reference" type="AttachmentReference">
  The object that replaces this `Attachment` at upload time.
</ParamField>

<ParamField path="data" type="bytes">
  The attachment contents. This is a lazy value that will read the attachment contents from the external object store on first access.
</ParamField>

<span class="text-sm">Methods</span>

`__init__()`, `upload()`, `debug_info()`

### JSONAttachment

A convenience class for creating attachments from JSON-serializable objects.

<span class="text-sm">Methods</span>

`__init__()`

### LiteLLMWrapper

Main wrapper for the LiteLLM module.

<span class="text-sm">Methods</span>

`__init__()`, `completion()`, `acompletion()`, `responses()`, `aresponses()`, `embedding()`, `moderation()`

### MetricSummary

Summary of a metric's performance.

<span class="text-sm">Properties</span>

<ParamField path="name" type="str">
  Name of the metric.
</ParamField>

<ParamField path="metric" type="float | int">
  Average metric across all examples.
</ParamField>

<ParamField path="unit" type="str">
  Unit label for the metric.
</ParamField>

<ParamField path="improvements" type="int | None">
  Number of improvements in the metric.
</ParamField>

<ParamField path="regressions" type="int | None">
  Number of regressions in the metric.
</ParamField>

<ParamField path="diff" type="float | None">
  Difference in metric between the current and reference experiment.
</ParamField>

<span class="text-sm">Methods</span>

`__init__()`

### ModerationWrapper

Wrapper for LiteLLM moderation functions.

<span class="text-sm">Properties</span>

<ParamField path="moderation_fn" type="Any" />

<span class="text-sm">Methods</span>

`__init__()`, `moderation()`

### NamedWrapper

Wrapper that preserves access to the original wrapped object's attributes.

<span class="text-sm">Methods</span>

`__init__()`

### Project

A handle to a Braintrust project.

<span class="text-sm">Properties</span>

<ParamField path="name" type="Any" />

<ParamField path="tools" type="Any" />

<ParamField path="prompts" type="Any" />

<ParamField path="scorers" type="Any" />

<span class="text-sm">Methods</span>

`__init__()`, `add_code_function()`, `add_prompt()`, `publish()`

### ProjectBuilder

Creates handles to Braintrust projects.

<span class="text-sm">Methods</span>

`create()`

### Prompt

A prompt object consists of prompt text, a model, and model parameters (such as temperature), which can be used to generate completions or chat messages. The prompt object supports calling `.build()` which uses mustache templating to build the prompt with the given formatting options and returns a plain dictionary that includes the built prompt and arguments. The dictionary can be passed as kwargs to the OpenAI client or modified as you see fit.

<span class="text-sm">Properties</span>

<ParamField path="defaults" type="Any" />

<ParamField path="no_trace" type="Any" />

<ParamField path="id" type="str" />

<ParamField path="name" type="str" />

<ParamField path="slug" type="str" />

<ParamField path="prompt" type="PromptBlockData | None" />

<ParamField path="version" type="str" />

<ParamField path="options" type="PromptOptions" />

<span class="text-sm">Methods</span>

`__init__()`, `from_prompt_data()`, `build()`

### PromptBuilder

Builder to create a prompt in Braintrust.

<span class="text-sm">Properties</span>

<ParamField path="project" type="Any" />

<span class="text-sm">Methods</span>

`__init__()`, `create()`

### ReadonlyAttachment

A readonly alternative to `Attachment`, which can be used for fetching already-uploaded Attachments.

<span class="text-sm">Properties</span>

<ParamField path="reference" type="Any" />

<ParamField path="data" type="bytes">
  The attachment contents. This is a lazy value that will read the attachment contents from the object store on first access.
</ParamField>

<span class="text-sm">Methods</span>

`__init__()`, `metadata()`, `status()`

### ReadonlyExperiment

A read-only view of an experiment, initialized by passing `open=True` to `init()`.

<span class="text-sm">Properties</span>

<ParamField path="state" type="Any" />

<ParamField path="id" type="str" />

<ParamField path="logging_state" type="BraintrustState" />

<span class="text-sm">Methods</span>

`__init__()`, `as_dataset()`

### RepoInfo

Information about the current HEAD of the repo.

<span class="text-sm">Properties</span>

<ParamField path="commit" type="str | None" />

<ParamField path="branch" type="str | None" />

<ParamField path="tag" type="str | None" />

<ParamField path="dirty" type="bool | None" />

<ParamField path="author_name" type="str | None" />

<ParamField path="author_email" type="str | None" />

<ParamField path="commit_message" type="str | None" />

<ParamField path="commit_time" type="str | None" />

<ParamField path="git_diff" type="str | None" />

<span class="text-sm">Methods</span>

`__init__()`

### ReporterDef

A reporter takes an evaluator and its result and returns a report.

<span class="text-sm">Properties</span>

<ParamField path="name" type="str">
  The name of the reporter.
</ParamField>

<ParamField path="report_eval" type="Callable[[Evaluator[Input, Output], EvalResultWithSummary[Input, Output], bool, bool], EvalReport | Awaitable[EvalReport]]">
  A function that takes an evaluator and its result and returns a report.
</ParamField>

<ParamField path="report_run" type="Callable[[list[EvalReport], bool, bool], bool | Awaitable[bool]]">
  A function that takes all evaluator results and returns a boolean indicating whether the run was successful.
  If you return false, the `braintrust eval` command will exit with a non-zero status code.
</ParamField>

<span class="text-sm">Methods</span>

`__init__()`

### ResponsesWrapper

Wrapper for LiteLLM responses functions with tracing support.

<span class="text-sm">Properties</span>

<ParamField path="responses_fn" type="Any" />

<ParamField path="aresponses_fn" type="Any" />

<span class="text-sm">Methods</span>

`__init__()`, `responses()`, `aresponses()`

### RetryRequestExceptionsAdapter

An HTTP adapter that automatically retries requests on connection exceptions.

<span class="text-sm">Properties</span>

<ParamField path="base_num_retries" type="Any" />

<ParamField path="backoff_factor" type="Any" />

<ParamField path="default_timeout_secs" type="Any" />

<span class="text-sm">Methods</span>

`__init__()`, `send()`

### SSEProgressEvent

A progress event that can be reported during task execution, specifically for SSE (Server-Sent Events) streams. This is a subclass of TaskProgressEvent with additional fields for SSE-specific metadata.

<span class="text-sm">Properties</span>

<ParamField path="id" type="str" />

<ParamField path="object_type" type="str" />

<ParamField path="origin" type="ObjectReference" />

<ParamField path="name" type="str" />

### ScoreSummary

Summary of a score's performance.

<span class="text-sm">Properties</span>

<ParamField path="name" type="str">
  Name of the score.
</ParamField>

<ParamField path="score" type="float">
  Average score across all examples.
</ParamField>

<ParamField path="improvements" type="int | None">
  Number of improvements in the score.
</ParamField>

<ParamField path="regressions" type="int | None">
  Number of regressions in the score.
</ParamField>

<ParamField path="diff" type="float | None">
  Difference in score between the current and reference experiment.
</ParamField>

<span class="text-sm">Methods</span>

`__init__()`

### ScorerBuilder

Builder to create a scorer in Braintrust.

<span class="text-sm">Properties</span>

<ParamField path="project" type="Any" />

<span class="text-sm">Methods</span>

`__init__()`, `create()`

### Span

A Span encapsulates logged data and metrics for a unit of work. This interface is shared by all span implementations.

<span class="text-sm">Properties</span>

<ParamField path="id" type="str">
  Row ID of the span.
</ParamField>

<ParamField path="name" type="str">
  Name of the span, for display purposes only.
</ParamField>

<span class="text-sm">Methods</span>

`log()`, `log_feedback()`, `start_span()`, `export()`, `link()`, `permalink()`, `end()`, `flush()`, `close()`, `set_attributes()`, `set_current()`, `unset_current()`

### SpanIds

The three IDs that define a span's position in the trace tree.

<span class="text-sm">Properties</span>

<ParamField path="span_id" type="str" />

<ParamField path="root_span_id" type="str" />

<ParamField path="span_parents" type="list[str] | None" />

<span class="text-sm">Methods</span>

`__init__()`

### SpanImpl

Primary implementation of the `Span` interface. See the `Span` interface for full details on each method.

<span class="text-sm">Properties</span>

<ParamField path="can_set_current" type="bool" />

<ParamField path="state" type="Any" />

<ParamField path="parent_object_type" type="Any" />

<ParamField path="parent_object_id" type="Any" />

<ParamField path="parent_compute_object_metadata_args" type="Any" />

<ParamField path="propagated_event" type="Any" />

<ParamField path="span_id" type="Any" />

<ParamField path="root_span_id" type="Any" />

<ParamField path="span_parents" type="Any" />

<ParamField path="id" type="str" />

<ParamField path="name" type="str" />

<span class="text-sm">Methods</span>

`__init__()`, `set_attributes()`, `log()`, `log_internal()`, `log_feedback()`, `start_span()`, `end()`, `export()`, `link()`, `permalink()`, `close()`, `flush()`, `set_current()`, `unset_current()`

### SpanScope

Scope for operating on a single span.

<span class="text-sm">Properties</span>

<ParamField path="type" type="Literal['span']" />

<ParamField path="id" type="str" />

<ParamField path="root_span_id" type="str" />

### SyncScorerLike

Protocol for synchronous scorers that implement the callable interface. This is the most common interface and is used when no async version is available.

<span class="text-sm">Methods</span>

`__call__()`

### TaskProgressEvent

Progress event that can be reported during task execution.

<span class="text-sm">Properties</span>

<ParamField path="format" type="FunctionFormat" />

<ParamField path="output_type" type="FunctionOutputType" />

<ParamField path="event" type="Literal['reasoning_delta', 'text_delta', 'json_delta', 'error', 'console', 'start', 'done', 'progress']" />

<ParamField path="data" type="str" />

<span class="text-sm">Methods</span>

`__init__()`

### ToolBuilder

Builder to create a tool in Braintrust.

<span class="text-sm">Properties</span>

<ParamField path="project" type="Any" />

<span class="text-sm">Methods</span>

`__init__()`, `create()`

### TraceScope

Scope for operating on an entire trace.

<span class="text-sm">Properties</span>

<ParamField path="type" type="Literal['trace']" />

<ParamField path="root_span_id" type="str" />

### TracedMessageStream

TracedMessageStream wraps both sync and async message streams. Obviously only one makes sense at a time

<span class="text-sm">Methods</span>

`__init__()`