Skip to main content

Arize

Arize ML observability and LLM evaluation platform connector for managing projects, datasets, experiments, prompts, evaluators, annotation workflows, models, monitors, and dashboards.

Categories: AI, Observability, Machine Learning

What you can ask​

  • Surface the production model variants whose accuracy dropped more than 5% week-over-week.
  • Pull the input drift report for our recommender model over the last 7 days.
  • List all open issues tagged data-quality across our model registry.

Set up​

Arize connects with credentials from your Arize account. Have these values ready:

  • API key
  1. Go to Organization > Connectors and find Arize.
  2. Click Enable and choose which tools to allow.
  3. Each member enters their own values the first time they use it.

To use one account for the whole organization, turn on Share one connection with everyone. Every member's agents then act as that account, with exactly the access it has.

Tools​

Arize offers 84 tools. Admins choose which ones agents can use. See Tool permission groups.

Read (39)​

View and search information without changing it.

ToolWhat it does
get_ai_integrationGet an Arize AI integration by ID.
get_annotation_configGet an Arize annotation config by ID.
get_annotation_queueGet an Arize annotation queue by ID.
get_datasetGet an Arize dataset by ID.
get_evaluatorGet an Arize evaluator by ID.
get_evaluator_versionGet a specific Arize evaluator version by version ID.
get_experimentGet an Arize experiment by ID.
get_modelGet an Arize model by ID including its versions via GraphQL.
get_model_schemaGet an Arize model's features and tags schema via GraphQL.
get_model_tracing_schemaGet an Arize model's tracing schema including span properties and LLM evals via GraphQL.
get_monitor_audit_logGet the audit log for an Arize monitor showing configuration changes via GraphQL.
get_monitor_historyGet historical calculations for an Arize monitor within a time range via GraphQL.
get_projectGet an Arize project by ID.
get_promptGet an Arize prompt by ID with its resolved version.
get_prompt_versionGet a specific Arize prompt version by version ID.
get_spaceGet an Arize space by ID.
list_ai_integrationsList all Arize AI provider integrations with pagination.
list_annotation_configsList all Arize annotation configs with pagination.
list_annotation_queue_recordsList records in an Arize annotation queue with pagination.
list_annotation_queuesList all Arize annotation queues with pagination.
list_api_keysList all Arize API keys with pagination.
list_custom_metricsList custom metrics for an Arize model via GraphQL.
list_dataset_examplesList examples in an Arize dataset with pagination.
list_datasetsList Arize datasets with optional space filter and pagination.
list_evaluator_versionsList versions of an Arize evaluator with pagination.
list_evaluatorsList all Arize evaluators with pagination.
list_experiment_runsList runs for an Arize experiment with pagination.
list_experimentsList Arize experiments with optional dataset filter and pagination.
list_integrationsList integrations in the Arize organization via GraphQL.
list_modelsList models in an Arize space via GraphQL.
list_monitorsList Arize monitors for a space via GraphQL.
list_projectsList all Arize projects with pagination.
list_prompt_versionsList all versions of an Arize prompt with pagination.
list_promptsList all Arize prompts with pagination.
list_spacesList all Arize spaces with pagination.
list_spansSearch and list Arize spans for a project with optional time range and filters.
list_usersList users in the Arize organization via GraphQL.
resolve_prompt_labelResolve an Arize prompt label to its current version.
validate_credentialValidate Arize API key by listing projects.

Write (22)​

Create or send data.

ToolWhat it does
add_annotationsAdd annotations to an Arize model record via GraphQL.
add_dataset_examplesAppend new examples to an Arize dataset.
assign_space_memberAssign a user to an Arize space with a role via GraphQL.
create_ai_integrationCreate an Arize AI provider integration.
create_annotation_configCreate an Arize annotation config.
create_annotation_queueCreate an Arize annotation queue with optional project and annotation config associations.
create_api_keyCreate a new Arize API key.
create_custom_metricCreate a custom metric for an Arize model via GraphQL.
create_dashboardCreate a new Arize dashboard in a space via GraphQL.
create_dashboard_from_templateCreate an Arize dashboard from a template via GraphQL.
create_data_quality_monitorCreate an Arize data quality monitor for a model feature via GraphQL.
create_datasetCreate a new Arize dataset in a space with initial examples.
create_drift_monitorCreate an Arize drift monitor for a model feature via GraphQL.
create_evaluatorCreate a new Arize LLM evaluator (type=template).
create_evaluator_versionCreate a new version of an Arize evaluator.
create_experimentCreate a new Arize experiment on a dataset.
create_integration_keyCreate an integration key for an Arize integration via GraphQL.
create_performance_monitorCreate an Arize performance monitor for a model metric via GraphQL.
create_projectCreate a new Arize project in a space.
create_promptCreate a new Arize prompt with an initial version.
create_prompt_versionCreate a new version of an Arize prompt.
create_spaceCreate a new Arize space. organization_id is auto-fetched from existing spaces; provide it explicitly for accounts with no spaces yet.

Delete or overwrite (23)​

Remove or replace existing data.

ToolWhat it does
delete_ai_integrationDelete an Arize AI integration.
delete_annotation_configDelete an Arize annotation config.
delete_annotation_queueDelete an Arize annotation queue.
delete_api_keyDelete an Arize API key by ID.
delete_datasetDelete an Arize dataset by ID.
delete_evaluatorDelete an Arize evaluator by ID.
delete_experimentDelete an Arize experiment by ID.
delete_integration_keyDelete an Arize integration key via GraphQL.
delete_monitorDelete an Arize monitor via GraphQL.
delete_projectDelete an Arize project by ID.
delete_promptDelete an Arize prompt by ID.
regenerate_api_keyRegenerate an Arize API key by ID.
remove_prompt_version_labelRemove a specific label from an Arize prompt version.
remove_space_memberRemove a user from an Arize space via GraphQL.
set_prompt_version_labelsSet labels on an Arize prompt version (e.g., 'production', 'staging').
update_ai_integrationUpdate an Arize AI integration name or API key.
update_annotation_queueUpdate an Arize annotation queue name, description, or configs.
update_custom_metricUpdate an Arize custom metric name, expression, or description via GraphQL.
update_dataset_examplesUpdate existing examples in an Arize dataset.
update_evaluatorUpdate an Arize evaluator name or description.
update_monitorUpdate an Arize monitor's mute status, threshold, or contacts via GraphQL.
update_promptUpdate an Arize prompt's description.
update_spaceUpdate an Arize space name or description.
⌘J