X connector
Set up the X connector in Kaivo: authentication, configuration, the 2 BigQuery tables it syncs, and answers to common questions.
X connector pricing, tier and credits on kaivo.ioKaivo is a fully managed data platform that syncs your X data into a Google BigQuery warehouse and keeps it up to date automatically. There is no pipeline to build and no infrastructure to run, so you can spend your time analysing your marketing and sales data instead of moving it.
What is the X connector
Section titled “What is the X connector”Sync your X posts and authors into BigQuery with Kaivo to track social performance alongside the rest of your data.
| Category | Marketing & Sales |
|---|---|
| Authentication | API key |
| Setup | Self-service |
Getting started with the X connector
Section titled “Getting started with the X connector”- Sign up for Kaivo and create a workspace.
- Connect your X account.
- Choose which tables to sync.
- Wait for the initial sync to finish.
- Query your data in BigQuery or your favourite AI or BI tool.
Authenticating X
Section titled “Authenticating X”Authenticate with your Access Token.
| Field | Description |
|---|---|
| Access Token | |
| App only Bearer Token. See the docs for more information on how to obtain this token. |
|
Configuring the X connector
Section titled “Configuring the X connector”When you set up the connector, you provide:
| Field | Description |
|---|---|
| Search Query | |
| Query for matching Tweets. See the query guide for help building your search query. |
| | Start Date |
Any data before this date will not be fetched.
|
Tables and columns synced from X
Section titled “Tables and columns synced from X”Kaivo syncs 2 tables from X into a dedicated dataset in your BigQuery warehouse. Click any table to see its columns and types.
authors (20 columns)
| Column | Type | Description |
|---|---|---|
_kaivo_id |
STRING |
Primary key that uniquely identifies the row. Auto-generated by Kaivo. |
description |
STRING |
|
created_at |
STRING |
|
id |
STRING |
|
location |
STRING |
|
name |
STRING |
|
pinned_tweet_id |
STRING |
|
profile_image_url |
STRING |
|
protected |
BOOL |
|
public_metrics__followers_count |
FLOAT64 |
|
public_metrics__following_count |
FLOAT64 |
|
public_metrics__like_count |
FLOAT64 |
|
public_metrics__listed_count |
FLOAT64 |
|
public_metrics__media_count |
FLOAT64 |
|
public_metrics__tweet_count |
FLOAT64 |
|
url |
STRING |
|
username |
STRING |
|
verified |
BOOL |
|
verified_type |
STRING |
|
_kaivo_extracted_at |
TIMESTAMP |
Timestamp that shows when the row was extracted. Auto-generated by Kaivo. |
Subtable: authors__entities__description__hashtags
Section titled “Subtable: authors__entities__description__hashtags”| Column | Type | Description |
|---|---|---|
_kaivo_parent_id |
STRING |
Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo. |
_kaivo_id |
STRING |
Primary key that uniquely identifies the row. Auto-generated by Kaivo. |
end |
FLOAT64 |
|
start |
FLOAT64 |
|
tag |
STRING |
|
_kaivo_extracted_at |
TIMESTAMP |
Timestamp that shows when the row was extracted. Auto-generated by Kaivo. |
Subtable: authors__entities__description__mentions
Section titled “Subtable: authors__entities__description__mentions”| Column | Type | Description |
|---|---|---|
_kaivo_parent_id |
STRING |
Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo. |
_kaivo_id |
STRING |
Primary key that uniquely identifies the row. Auto-generated by Kaivo. |
end |
FLOAT64 |
|
start |
FLOAT64 |
|
username |
STRING |
|
_kaivo_extracted_at |
TIMESTAMP |
Timestamp that shows when the row was extracted. Auto-generated by Kaivo. |
Subtable: authors__entities__description__urls
Section titled “Subtable: authors__entities__description__urls”| Column | Type | Description |
|---|---|---|
_kaivo_parent_id |
STRING |
Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo. |
_kaivo_id |
STRING |
Primary key that uniquely identifies the row. Auto-generated by Kaivo. |
display_url |
STRING |
|
end |
FLOAT64 |
|
expanded_url |
STRING |
|
start |
FLOAT64 |
|
url |
STRING |
|
_kaivo_extracted_at |
TIMESTAMP |
Timestamp that shows when the row was extracted. Auto-generated by Kaivo. |
Subtable: authors__entities__url__urls
Section titled “Subtable: authors__entities__url__urls”| Column | Type | Description |
|---|---|---|
_kaivo_parent_id |
STRING |
Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo. |
_kaivo_id |
STRING |
Primary key that uniquely identifies the row. Auto-generated by Kaivo. |
display_url |
STRING |
|
end |
FLOAT64 |
|
expanded_url |
STRING |
|
start |
FLOAT64 |
|
url |
STRING |
|
_kaivo_extracted_at |
TIMESTAMP |
Timestamp that shows when the row was extracted. Auto-generated by Kaivo. |
tweets (15 columns)
| Column | Type | Description |
|---|---|---|
_kaivo_id |
STRING |
Primary key that uniquely identifies the row. Auto-generated by Kaivo. |
author_id |
STRING |
|
conversation_id |
STRING |
|
created_at |
STRING |
|
edit_controls__editable_until |
STRING |
|
edit_controls__edits_remaining |
FLOAT64 |
|
edit_controls__is_edit_eligible |
BOOL |
|
geo__place_id |
STRING |
|
id |
STRING |
|
in_reply_to_user_id |
STRING |
|
lang |
STRING |
|
possibly_sensitive |
BOOL |
|
reply_settings |
STRING |
|
text |
STRING |
|
_kaivo_extracted_at |
TIMESTAMP |
Timestamp that shows when the row was extracted. Auto-generated by Kaivo. |
Subtable: tweets__attachments__media_keys
Section titled “Subtable: tweets__attachments__media_keys”| Column | Type | Description |
|---|---|---|
_kaivo_parent_id |
STRING |
Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo. |
_kaivo_id |
STRING |
Primary key that uniquely identifies the row. Auto-generated by Kaivo. |
value |
STRING |
|
_kaivo_extracted_at |
TIMESTAMP |
Timestamp that shows when the row was extracted. Auto-generated by Kaivo. |
Subtable: tweets__attachments__poll_ids
Section titled “Subtable: tweets__attachments__poll_ids”| Column | Type | Description |
|---|---|---|
_kaivo_parent_id |
STRING |
Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo. |
_kaivo_id |
STRING |
Primary key that uniquely identifies the row. Auto-generated by Kaivo. |
value |
STRING |
|
_kaivo_extracted_at |
TIMESTAMP |
Timestamp that shows when the row was extracted. Auto-generated by Kaivo. |
Subtable: tweets__context_annotations
Section titled “Subtable: tweets__context_annotations”| Column | Type | Description |
|---|---|---|
_kaivo_parent_id |
STRING |
Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo. |
_kaivo_id |
STRING |
Primary key that uniquely identifies the row. Auto-generated by Kaivo. |
domain__description |
STRING |
|
domain__id |
STRING |
|
domain__name |
STRING |
|
entity__description |
STRING |
|
entity__id |
STRING |
|
entity__name |
STRING |
|
_kaivo_extracted_at |
TIMESTAMP |
Timestamp that shows when the row was extracted. Auto-generated by Kaivo. |
Subtable: tweets__edit_history_tweet_ids
Section titled “Subtable: tweets__edit_history_tweet_ids”| Column | Type | Description |
|---|---|---|
_kaivo_parent_id |
STRING |
Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo. |
_kaivo_id |
STRING |
Primary key that uniquely identifies the row. Auto-generated by Kaivo. |
value |
STRING |
|
_kaivo_extracted_at |
TIMESTAMP |
Timestamp that shows when the row was extracted. Auto-generated by Kaivo. |
Subtable: tweets__entities__annotations
Section titled “Subtable: tweets__entities__annotations”| Column | Type | Description |
|---|---|---|
_kaivo_parent_id |
STRING |
Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo. |
_kaivo_id |
STRING |
Primary key that uniquely identifies the row. Auto-generated by Kaivo. |
type |
STRING |
|
end |
FLOAT64 |
|
normalized_text |
STRING |
|
probability |
FLOAT64 |
|
start |
FLOAT64 |
|
_kaivo_extracted_at |
TIMESTAMP |
Timestamp that shows when the row was extracted. Auto-generated by Kaivo. |
Subtable: tweets__entities__cashtags
Section titled “Subtable: tweets__entities__cashtags”| Column | Type | Description |
|---|---|---|
_kaivo_parent_id |
STRING |
Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo. |
_kaivo_id |
STRING |
Primary key that uniquely identifies the row. Auto-generated by Kaivo. |
end |
FLOAT64 |
|
start |
FLOAT64 |
|
tag |
STRING |
|
_kaivo_extracted_at |
TIMESTAMP |
Timestamp that shows when the row was extracted. Auto-generated by Kaivo. |
Subtable: tweets__entities__hashtags
Section titled “Subtable: tweets__entities__hashtags”| Column | Type | Description |
|---|---|---|
_kaivo_parent_id |
STRING |
Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo. |
_kaivo_id |
STRING |
Primary key that uniquely identifies the row. Auto-generated by Kaivo. |
end |
FLOAT64 |
|
start |
FLOAT64 |
|
tag |
STRING |
|
_kaivo_extracted_at |
TIMESTAMP |
Timestamp that shows when the row was extracted. Auto-generated by Kaivo. |
Subtable: tweets__entities__mentions
Section titled “Subtable: tweets__entities__mentions”| Column | Type | Description |
|---|---|---|
_kaivo_parent_id |
STRING |
Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo. |
_kaivo_id |
STRING |
Primary key that uniquely identifies the row. Auto-generated by Kaivo. |
end |
FLOAT64 |
|
id |
STRING |
|
start |
FLOAT64 |
|
username |
STRING |
|
_kaivo_extracted_at |
TIMESTAMP |
Timestamp that shows when the row was extracted. Auto-generated by Kaivo. |
Subtable: tweets__entities__urls
Section titled “Subtable: tweets__entities__urls”| Column | Type | Description |
|---|---|---|
_kaivo_parent_id |
STRING |
Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo. |
_kaivo_id |
STRING |
Primary key that uniquely identifies the row. Auto-generated by Kaivo. |
description |
STRING |
|
display_url |
STRING |
|
end |
FLOAT64 |
|
expanded_url |
STRING |
|
media_key |
STRING |
|
start |
FLOAT64 |
|
status |
FLOAT64 |
|
title |
STRING |
|
unwound_url |
STRING |
|
url |
STRING |
|
_kaivo_extracted_at |
TIMESTAMP |
Timestamp that shows when the row was extracted. Auto-generated by Kaivo. |
Subtable: tweets__entities__urls__images
Section titled “Subtable: tweets__entities__urls__images”| Column | Type | Description |
|---|---|---|
_kaivo_parent_id |
STRING |
Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo. |
_kaivo_id |
STRING |
Primary key that uniquely identifies the row. Auto-generated by Kaivo. |
height |
FLOAT64 |
|
url |
STRING |
|
width |
FLOAT64 |
|
_kaivo_extracted_at |
TIMESTAMP |
Timestamp that shows when the row was extracted. Auto-generated by Kaivo. |
Subtable: tweets__referenced_tweets
Section titled “Subtable: tweets__referenced_tweets”| Column | Type | Description |
|---|---|---|
_kaivo_parent_id |
STRING |
Foreign key referencing _kaivo_id in the parent table. Auto-generated by Kaivo. |
_kaivo_id |
STRING |
Primary key that uniquely identifies the row. Auto-generated by Kaivo. |
type |
STRING |
|
id |
STRING |
|
_kaivo_extracted_at |
TIMESTAMP |
Timestamp that shows when the row was extracted. Auto-generated by Kaivo. |
How the X sync works
Section titled “How the X sync works”After the first load, Kaivo keeps your BigQuery warehouse up to date for you. Where X supports it, each sync pulls only new and changed records so it stays fast; otherwise it refreshes the whole table. Every record keeps its original ID, so you won’t get duplicate rows.
Frequently asked questions
Section titled “Frequently asked questions”How long does the initial sync take for X?
Section titled “How long does the initial sync take for X?”It depends on how much history is in your X account. Most initial syncs finish within minutes, while large accounts can take a few hours. After that, syncs only fetch new and changed records, so they’re much faster.
Can I sync only some tables or columns?
Section titled “Can I sync only some tables or columns?”Yes. You pick which tables to sync when you set up the connection and can change the selection later. Tables you don’t select are never copied to your warehouse.
What happens when X’s schema changes?
Section titled “What happens when X’s schema changes?”New fields are never added automatically. You choose which fields to sync, so data you haven’t selected (sensitive personal data, for example) never lands in your warehouse. When a new field appears, it becomes available for you to add. What happens to removed or renamed fields depends on a table’s sync mode: full-refresh tables always match what’s currently in X, so dropped fields disappear, while incremental tables keep their existing columns and history, so an old field stays and newly added fields fill in over time.
How do I handle GDPR or data deletion requests?
Section titled “How do I handle GDPR or data deletion requests?”Your data lives in your own Kaivo-managed BigQuery warehouse, so the most direct option is to delete or anonymise specific records right in BigQuery. If you delete data in X instead, full-refresh tables drop it on the next sync, while incremental tables keep it, so you would remove the row in BigQuery or ask us to run a full refresh. To remove everything, delete the X connector in Kaivo and all of its synced data is deleted with it.
Common use cases for X data
Section titled “Common use cases for X data”Post performance
Section titled “Post performance”Analyse tweets to track engagement and reach over time.
Content trends
Section titled “Content trends”Report on which posts and topics resonate most with your audience.
Author insights
Section titled “Author insights”Use authors to understand the accounts behind your tracked posts.
Social and marketing together
Section titled “Social and marketing together”Combine X data with your other marketing channels in BigQuery for one view.
Use X data in your AI and BI tools
Section titled “Use X data in your AI and BI tools”Once X data lands in your Kaivo-managed BigQuery warehouse, you can explore it with AI tools or any BI tool that connects to BigQuery. Here’s how the most common destinations work with X data.
Claude
Section titled “Claude”Use Kaivo’s MCP server to give Claude secure, workspace-scoped access to your data. Setup guide →
Power BI
Section titled “Power BI”Microsoft’s BI tool with a native BigQuery connector. Supports direct query and scheduled refresh. Setup guide →
Data Studio
Section titled “Data Studio”Free Google BI tool with native BigQuery support. One-click connection to your Kaivo warehouse; great for SMB teams on Google Workspace. Setup guide →
Tableau
Section titled “Tableau”The premium analytics standard, with native BigQuery integration. Setup guide →
Google Sheets
Section titled “Google Sheets”Use Connected Sheets to query BigQuery directly from a spreadsheet, with no SQL. Setup guide →
Connect via Power Query’s BigQuery connector. Setup guide →
Metabase
Section titled “Metabase”Open-source BI tool with strong BigQuery support. Setup guide →
See our pricing page for X connector pricing and plan details.
Related connectors
Section titled “Related connectors”- Google Ads: Sync Google Ads to BigQuery.
- Custobar: Sync Custobar to BigQuery.
- Google Search Console: Sync Google Search Console to BigQuery.
- HubSpot: Sync HubSpot to BigQuery.
- Adform: Sync Adform to BigQuery.
- Amplitude: Sync Amplitude to BigQuery.