Extension DataStore BigQuery


Extension Basics

Title
DataStore BigQuery
Name
ckanext-datastore-bigquery
Type
Public extension
Description
Google BigQuery backend for CKAN DataStore enabling scalable cloud-based data storage and querying.
CKAN versions

>=2.8.0 <2.9.0

Show details

These CKAN Versions are exactely matched:

Download-Url (zip)
Download-Url commit date
2024-05-29
Url to repo
Category
Cloud Infrastructure & Storage


Background Infos

Description (long)
Show details

The DataStore BigQuery extension revolutionizes CKAN’s data storage architecture by replacing the traditional PostgreSQL DataStore with Google BigQuery’s enterprise-grade cloud data warehouse. This powerful extension enables organizations to handle massive datasets with petabyte-scale storage capacity and lightning-fast analytical queries. The extension seamlessly translates CKAN’s datastore_search and datastore_search_sql API calls to BigQuery SQL, maintaining complete API compatibility while delivering dramatically improved performance for large datasets. Users benefit from BigQuery’s columnar storage, automatic indexing, and distributed computing capabilities that can process billions of records in seconds. The extension supports advanced SQL features, geographic data types, machine learning functions, and real-time streaming data ingestion. Authentication is handled through Google Cloud service accounts with fine-grained access controls and audit logging. Essential for organizations with big data requirements, real-time analytics needs, and those seeking to leverage Google Cloud’s advanced data processing capabilities while maintaining CKAN’s familiar interface and API structure.

Version
Latest
Version release date
2024-05-29
Contact name
Datopian Team
Contakt email
Contact Url
(not set)


Installation Guide

Configuration hints

Requires Google Cloud credentials and BigQuery project setup.

Plugins to configure (ckan.ini)
bigquery
CKAN Settings (ckan.ini)
# ckan.datastore.write_url = 'bigquery://your_project_id' # ckan.datastore.read_url = 'bigquery://your_project_id' # ckan.datastore.sqlsearch.enabled = true # ckanext.bigquery.google_cloud_credentials = '/path/to/credentials.json' # ckanext.bigquery.project = 'your_project_id' # ckanext.bigquery.dataset = 'your_dataset_id'
DB migration to be executed
(not set)
<< back to Extensions