Arize
Arize ML observability and LLM evaluation platform connector for managing projects, datasets, experiments, prompts, evaluators, annotation workflows, models, monitors, and dashboards.
Categories: AI, Observability, Machine Learning
What you can ask
- Surface the production model variants whose accuracy dropped more than 5% week-over-week.
- Pull the input drift report for our recommender model over the last 7 days.
- List all open issues tagged
data-qualityacross our model registry.
Set up
Arize connects with credentials from your Arize account. Have these values ready:
- API key
- Go to Organization > Connectors and find Arize.
- Click Enable and choose which tools to allow.
- Each member enters their own values the first time they use it.
To use one account for the whole organization, turn on Share one connection with everyone. Every member's agents then act as that account, with exactly the access it has.
Tools
Arize offers 84 tools. Admins choose which ones agents can use. See Tool permission groups.
Read (39)
View and search information without changing it.
| Tool | What it does |
|---|---|
get_ai_integration | Get an Arize AI integration by ID. |
get_annotation_config | Get an Arize annotation config by ID. |
get_annotation_queue | Get an Arize annotation queue by ID. |
get_dataset | Get an Arize dataset by ID. |
get_evaluator | Get an Arize evaluator by ID. |
get_evaluator_version | Get a specific Arize evaluator version by version ID. |
get_experiment | Get an Arize experiment by ID. |
get_model | Get an Arize model by ID including its versions via GraphQL. |
get_model_schema | Get an Arize model's features and tags schema via GraphQL. |
get_model_tracing_schema | Get an Arize model's tracing schema including span properties and LLM evals via GraphQL. |
get_monitor_audit_log | Get the audit log for an Arize monitor showing configuration changes via GraphQL. |
get_monitor_history | Get historical calculations for an Arize monitor within a time range via GraphQL. |
get_project | Get an Arize project by ID. |
get_prompt | Get an Arize prompt by ID with its resolved version. |
get_prompt_version | Get a specific Arize prompt version by version ID. |
get_space | Get an Arize space by ID. |
list_ai_integrations | List all Arize AI provider integrations with pagination. |
list_annotation_configs | List all Arize annotation configs with pagination. |
list_annotation_queue_records | List records in an Arize annotation queue with pagination. |
list_annotation_queues | List all Arize annotation queues with pagination. |
list_api_keys | List all Arize API keys with pagination. |
list_custom_metrics | List custom metrics for an Arize model via GraphQL. |
list_dataset_examples | List examples in an Arize dataset with pagination. |
list_datasets | List Arize datasets with optional space filter and pagination. |
list_evaluator_versions | List versions of an Arize evaluator with pagination. |
list_evaluators | List all Arize evaluators with pagination. |
list_experiment_runs | List runs for an Arize experiment with pagination. |
list_experiments | List Arize experiments with optional dataset filter and pagination. |
list_integrations | List integrations in the Arize organization via GraphQL. |
list_models | List models in an Arize space via GraphQL. |
list_monitors | List Arize monitors for a space via GraphQL. |
list_projects | List all Arize projects with pagination. |
list_prompt_versions | List all versions of an Arize prompt with pagination. |
list_prompts | List all Arize prompts with pagination. |
list_spaces | List all Arize spaces with pagination. |
list_spans | Search and list Arize spans for a project with optional time range and filters. |
list_users | List users in the Arize organization via GraphQL. |
resolve_prompt_label | Resolve an Arize prompt label to its current version. |
validate_credential | Validate Arize API key by listing projects. |
Write (22)
Create or send data.
| Tool | What it does |
|---|---|
add_annotations | Add annotations to an Arize model record via GraphQL. |
add_dataset_examples | Append new examples to an Arize dataset. |
assign_space_member | Assign a user to an Arize space with a role via GraphQL. |
create_ai_integration | Create an Arize AI provider integration. |
create_annotation_config | Create an Arize annotation config. |
create_annotation_queue | Create an Arize annotation queue with optional project and annotation config associations. |
create_api_key | Create a new Arize API key. |
create_custom_metric | Create a custom metric for an Arize model via GraphQL. |
create_dashboard | Create a new Arize dashboard in a space via GraphQL. |
create_dashboard_from_template | Create an Arize dashboard from a template via GraphQL. |
create_data_quality_monitor | Create an Arize data quality monitor for a model feature via GraphQL. |
create_dataset | Create a new Arize dataset in a space with initial examples. |
create_drift_monitor | Create an Arize drift monitor for a model feature via GraphQL. |
create_evaluator | Create a new Arize LLM evaluator (type=template). |
create_evaluator_version | Create a new version of an Arize evaluator. |
create_experiment | Create a new Arize experiment on a dataset. |
create_integration_key | Create an integration key for an Arize integration via GraphQL. |
create_performance_monitor | Create an Arize performance monitor for a model metric via GraphQL. |
create_project | Create a new Arize project in a space. |
create_prompt | Create a new Arize prompt with an initial version. |
create_prompt_version | Create a new version of an Arize prompt. |
create_space | Create a new Arize space. organization_id is auto-fetched from existing spaces; provide it explicitly for accounts with no spaces yet. |
Delete or overwrite (23)
Remove or replace existing data.
| Tool | What it does |
|---|---|
delete_ai_integration | Delete an Arize AI integration. |
delete_annotation_config | Delete an Arize annotation config. |
delete_annotation_queue | Delete an Arize annotation queue. |
delete_api_key | Delete an Arize API key by ID. |
delete_dataset | Delete an Arize dataset by ID. |
delete_evaluator | Delete an Arize evaluator by ID. |
delete_experiment | Delete an Arize experiment by ID. |
delete_integration_key | Delete an Arize integration key via GraphQL. |
delete_monitor | Delete an Arize monitor via GraphQL. |
delete_project | Delete an Arize project by ID. |
delete_prompt | Delete an Arize prompt by ID. |
regenerate_api_key | Regenerate an Arize API key by ID. |
remove_prompt_version_label | Remove a specific label from an Arize prompt version. |
remove_space_member | Remove a user from an Arize space via GraphQL. |
set_prompt_version_labels | Set labels on an Arize prompt version (e.g., 'production', 'staging'). |
update_ai_integration | Update an Arize AI integration name or API key. |
update_annotation_queue | Update an Arize annotation queue name, description, or configs. |
update_custom_metric | Update an Arize custom metric name, expression, or description via GraphQL. |
update_dataset_examples | Update existing examples in an Arize dataset. |
update_evaluator | Update an Arize evaluator name or description. |
update_monitor | Update an Arize monitor's mute status, threshold, or contacts via GraphQL. |
update_prompt | Update an Arize prompt's description. |
update_space | Update an Arize space name or description. |