Skip to content
Book a demoStart free
Connectors

CircleCI connector

Set up the CircleCI connector in Kaivo: authentication, configuration, the 12 BigQuery tables it syncs, and answers to common questions.

CircleCI connector pricing, tier and credits on kaivo.io

Kaivo is a fully managed data platform that syncs your CircleCI data into a Google BigQuery warehouse and keeps it up to date automatically. There is no pipeline to build and no infrastructure to run, so you can spend your time analysing your data from CircleCI instead of moving it.

Sync your CircleCI data into BigQuery with Kaivo to analyse build and pipeline performance across your projects.

CategoryTech
AuthenticationAPI key
SetupSelf-service
  1. Sign up for Kaivo and create a workspace.
  2. Connect your CircleCI account.
  3. Choose which tables to sync.
  4. Wait for the initial sync to finish.
  5. Query your data in BigQuery or your favourite AI or BI tool.

Authenticate with your API Key.

Field Description
API Key

When you set up the connector, you provide:

Field Description
Organization ID
Your organization ID. Find it at https://app.circleci.com/settings/organization/circleci/<org-id>/overview.

| | Start Date |

Any data before this date will not be fetched.

| | Project ID |

Your project ID. Find it in the project settings at https://app.circleci.com/settings/project/circleci/<org-slug>/<project-id>.

| | Workflow ID |

Workflow IDs to sync. Each workflow ID can be found in the pipeline build URL, e.g. https://app.circleci.com/pipelines/circleci/.../workflows/<workflow-id>.

|

Kaivo syncs 12 tables from CircleCI into a dedicated dataset in your BigQuery warehouse. Click any table to see its columns and types.

context (5 columns)
Column Type Description
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
created_at STRING
id STRING
name STRING
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
insights_branches (4 columns)
Column Type Description
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
org_id STRING
project_id STRING
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
Column Type Description
_kaivo_parent_id STRING Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo.
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
value STRING
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
insights_metrics (14 columns)
Column Type Description
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
org_id STRING
project_data__metrics__success_rate FLOAT64
project_data__metrics__throughput FLOAT64
project_data__metrics__total_credits_used FLOAT64
project_data__metrics__total_duration_secs FLOAT64
project_data__metrics__total_runs FLOAT64
project_data__trends__success_rate FLOAT64
project_data__trends__throughput FLOAT64
project_data__trends__total_credits_used FLOAT64
project_data__trends__total_duration_secs FLOAT64
project_data__trends__total_runs FLOAT64
project_id STRING
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
Column Type Description
_kaivo_parent_id STRING Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo.
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
value STRING
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
Column Type Description
_kaivo_parent_id STRING Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo.
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
value STRING
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
Column Type Description
_kaivo_parent_id STRING Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo.
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
branch STRING
metrics__p95_duration_secs FLOAT64
metrics__success_rate FLOAT64
metrics__total_credits_used FLOAT64
metrics__total_runs FLOAT64
trends__p95_duration_secs FLOAT64
trends__success_rate FLOAT64
trends__total_credits_used FLOAT64
trends__total_runs FLOAT64
workflow_name STRING
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
Column Type Description
_kaivo_parent_id STRING Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo.
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
metrics__p95_duration_secs FLOAT64
metrics__success_rate FLOAT64
metrics__total_credits_used FLOAT64
metrics__total_runs FLOAT64
trends__p95_duration_secs FLOAT64
trends__success_rate FLOAT64
trends__total_credits_used FLOAT64
trends__total_runs FLOAT64
workflow_name STRING
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
jobs (22 columns)
Column Type Description
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
created_at STRING
duration FLOAT64
executor__type STRING
executor__resource_class STRING
latest_workflow__id STRING
latest_workflow__name STRING
name STRING
number FLOAT64
organization__name STRING
parallelism FLOAT64
pipeline__id STRING
project__external_url STRING
project__id STRING
project__name STRING
project__slug STRING
queued_at STRING
started_at STRING
status STRING
stopped_at STRING
web_url STRING
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
Column Type Description
_kaivo_parent_id STRING Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo.
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
value JSON
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
Column Type Description
_kaivo_parent_id STRING Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo.
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
value JSON
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
Column Type Description
_kaivo_parent_id STRING Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo.
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
index FLOAT64
status STRING
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
me (25 columns)
Column Type Description
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
admin BOOL
analytics_id STRING
basic_email_prefs STRING
bitbucket_authorized BOOL
containers FLOAT64
created_at STRING
days_left_in_trial FLOAT64
dev_admin BOOL
email_authentication__email_verified BOOL
email_authentication__login STRING
in_beta_program BOOL
login STRING
name STRING
num_projects_followed FLOAT64
parallelism FLOAT64
privacy_optout BOOL
pusher_id STRING
selected_email STRING
sign_in_count FLOAT64
student BOOL
trial_end STRING
web_ui_pipelines_first_opt_in BOOL
web_ui_pipelines_optout STRING
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
Column Type Description
_kaivo_parent_id STRING Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo.
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
value STRING
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
Column Type Description
_kaivo_parent_id STRING Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo.
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
value JSON
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
Column Type Description
_kaivo_parent_id STRING Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo.
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
value JSON
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
pipelines (49 columns)
Column Type Description
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
created_at STRING
id STRING
number FLOAT64
project_slug STRING
state STRING
trigger__type STRING
trigger__actor__login STRING
trigger__received_at STRING
trigger_parameters__circleci__actor_id STRING
trigger_parameters__circleci__event_time STRING
trigger_parameters__circleci__event_type STRING
trigger_parameters__circleci__provider_actor_id STRING
trigger_parameters__circleci__provider_login STRING
trigger_parameters__circleci__provider_name STRING
trigger_parameters__circleci__trigger_type STRING
trigger_parameters__event__description STRING
trigger_parameters__event__title STRING
trigger_parameters__git__branch STRING
trigger_parameters__git__checkout_sha STRING
trigger_parameters__git__checkout_url STRING
trigger_parameters__git__ref STRING
trigger_parameters__git__repo_url STRING
trigger_parameters__github_app__branch STRING
trigger_parameters__github_app__checkout_sha STRING
trigger_parameters__github_app__commit_author_email STRING
trigger_parameters__github_app__commit_author_name STRING
trigger_parameters__github_app__commit_message STRING
trigger_parameters__github_app__commit_sha STRING
trigger_parameters__github_app__commit_timestamp STRING
trigger_parameters__github_app__commit_title STRING
trigger_parameters__github_app__content_ref STRING
trigger_parameters__github_app__default_branch STRING
trigger_parameters__github_app__forced STRING
trigger_parameters__github_app__owner STRING
trigger_parameters__github_app__ref STRING
trigger_parameters__github_app__repo_full_name STRING
trigger_parameters__github_app__repo_id STRING
trigger_parameters__github_app__repo_name STRING
trigger_parameters__github_app__repo_url STRING
trigger_parameters__github_app__total_commits_count STRING
trigger_parameters__github_app__user_avatar STRING
trigger_parameters__github_app__user_id STRING
trigger_parameters__github_app__user_name STRING
trigger_parameters__github_app__user_username STRING
trigger_parameters__github_app__web_url STRING
trigger_parameters__webhook__body STRING
updated_at STRING
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
Column Type Description
_kaivo_parent_id STRING Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo.
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
value JSON
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
projects (40 columns)
Column Type Description
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
branches__circleci_project_setup__is_using_workflows BOOL
branches__circleci_project_setup__last_success__added_at STRING
branches__circleci_project_setup__last_success__build_num FLOAT64
branches__circleci_project_setup__last_success__is_2_0_job BOOL
branches__circleci_project_setup__last_success__is_workflow_job BOOL
branches__circleci_project_setup__last_success__outcome STRING
branches__circleci_project_setup__last_success__pushed_at STRING
branches__circleci_project_setup__last_success__status STRING
branches__circleci_project_setup__last_success__vcs_revision STRING
branches__circleci_project_setup__latest_completed_workflows__build__created_at STRING
branches__circleci_project_setup__latest_completed_workflows__build__id STRING
branches__circleci_project_setup__latest_completed_workflows__build__status STRING
branches__circleci_project_setup__latest_workflows__build__created_at STRING
branches__circleci_project_setup__latest_workflows__build__id STRING
branches__circleci_project_setup__latest_workflows__build__status STRING
branches__main__is_using_workflows BOOL
branches__main__last_success__added_at STRING
branches__main__last_success__build_num FLOAT64
branches__main__last_success__is_2_0_job BOOL
branches__main__last_success__is_workflow_job BOOL
branches__main__last_success__outcome STRING
branches__main__last_success__pushed_at STRING
branches__main__last_success__status STRING
branches__main__last_success__vcs_revision STRING
branches__main__latest_completed_workflows__generate_config__created_at STRING
branches__main__latest_completed_workflows__generate_config__id STRING
branches__main__latest_completed_workflows__generate_config__status STRING
branches__main__latest_workflows__generate_config__created_at STRING
branches__main__latest_workflows__generate_config__id STRING
branches__main__latest_workflows__generate_config__status STRING
default_branch STRING
following BOOL
has_usable_key BOOL
oss BOOL
reponame STRING
username STRING
vcs_type STRING
vcs_url STRING
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.

Subtable: projects__branches__circleci_project_setup__pusher_logins

Section titled “Subtable: projects__branches__circleci_project_setup__pusher_logins”
Column Type Description
_kaivo_parent_id STRING Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo.
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
value STRING
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.

Subtable: projects__branches__circleci_project_setup__recent_builds

Section titled “Subtable: projects__branches__circleci_project_setup__recent_builds”
Column Type Description
_kaivo_parent_id STRING Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo.
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
added_at STRING
build_num FLOAT64
is_2_0_job BOOL
is_workflow_job BOOL
outcome STRING
pushed_at STRING
status STRING
vcs_revision STRING
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.

Subtable: projects__branches__circleci_project_setup__running_builds

Section titled “Subtable: projects__branches__circleci_project_setup__running_builds”
Column Type Description
_kaivo_parent_id STRING Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo.
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
value JSON
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
Column Type Description
_kaivo_parent_id STRING Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo.
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
value STRING
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
Column Type Description
_kaivo_parent_id STRING Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo.
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
added_at STRING
build_num FLOAT64
is_2_0_job BOOL
is_workflow_job BOOL
outcome STRING
pushed_at STRING
status STRING
vcs_revision STRING
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
Column Type Description
_kaivo_parent_id STRING Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo.
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
value JSON
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
self_collaborations (7 columns)
Column Type Description
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
avatar_url STRING
id STRING
name STRING
slug STRING
vcs_type STRING
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
self_ids (5 columns)
Column Type Description
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
id STRING
login STRING
name STRING
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
specific_project (11 columns)
Column Type Description
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
id STRING
name STRING
organization_id STRING
organization_name STRING
organization_slug STRING
slug STRING
vcs_info__default_branch STRING
vcs_info__provider STRING
vcs_info__vcs_url STRING
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
workflow (11 columns)
Column Type Description
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
created_at STRING
id STRING
name STRING
pipeline_id STRING
pipeline_number FLOAT64
project_slug STRING
started_by STRING
status STRING
stopped_at STRING
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
workflow_jobs (10 columns)
Column Type Description
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
type STRING
id STRING
job_number FLOAT64
name STRING
project_slug STRING
started_at STRING
status STRING
stopped_at STRING
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.
Column Type Description
_kaivo_parent_id STRING Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo.
_kaivo_id STRING Primary key that uniquely identifies the row. Auto-generated by Kaivo.
value JSON
_kaivo_extracted_at TIMESTAMP Timestamp that shows when the row was extracted. Auto-generated by Kaivo.

After the first load, Kaivo keeps your BigQuery warehouse up to date for you. Where CircleCI supports it, each sync pulls only new and changed records so it stays fast; otherwise it refreshes the whole table. Every record keeps its original ID, so you won’t get duplicate rows.

It depends on how much history is in your CircleCI account. Most initial syncs finish within minutes, while large accounts can take a few hours. After that, syncs only fetch new and changed records, so they’re much faster.

Yes. You pick which tables to sync when you set up the connection and can change the selection later. Tables you don’t select are never copied to your warehouse.

New fields are never added automatically. You choose which fields to sync, so data you haven’t selected (sensitive personal data, for example) never lands in your warehouse. When a new field appears, it becomes available for you to add. What happens to removed or renamed fields depends on a table’s sync mode: full-refresh tables always match what’s currently in CircleCI, so dropped fields disappear, while incremental tables keep their existing columns and history, so an old field stays and newly added fields fill in over time.

Your data lives in your own Kaivo-managed BigQuery warehouse, so the most direct option is to delete or anonymise specific records right in BigQuery. If you delete data in CircleCI instead, full-refresh tables drop it on the next sync, while incremental tables keep it, so you would remove the row in BigQuery or ask us to run a full refresh. To remove everything, delete the CircleCI connector in Kaivo and all of its synced data is deleted with it.

Use pipelines and workflows to track run frequency, duration, and failures over time.

Use insights_metrics to find the slowest and most failure-prone workflows.

Join projects with jobs to compare CI health across repositories.

Once CircleCI data lands in your Kaivo-managed BigQuery warehouse, you can explore it with AI tools or any BI tool that connects to BigQuery. Here’s how the most common destinations work with CircleCI data.

Use Kaivo’s MCP server to give Claude secure, workspace-scoped access to your data. Setup guide →

Microsoft’s BI tool with a native BigQuery connector. Supports direct query and scheduled refresh. Setup guide →

Free Google BI tool with native BigQuery support. One-click connection to your Kaivo warehouse; great for SMB teams on Google Workspace. Setup guide →

The premium analytics standard, with native BigQuery integration. Setup guide →

Use Connected Sheets to query BigQuery directly from a spreadsheet, with no SQL. Setup guide →

Connect via Power Query’s BigQuery connector. Setup guide →

Open-source BI tool with strong BigQuery support. Setup guide →

See our pricing page for CircleCI connector pricing and plan details.

  • Adform: Sync Adform to BigQuery.
  • Amplitude: Sync Amplitude to BigQuery.
  • Auth0: Sync Auth0 to BigQuery.
  • Convex: Sync Convex to BigQuery.
  • GitHub: Sync GitHub to BigQuery.
  • GitLab: Sync GitLab to BigQuery.