Extension XLoader


Extension Basics

Title
XLoader
Name
ckanext-xloader
Type
Public extension
Description
Loads CSV (and similar) data into CKAN's DataStore with improved performance compared to DataPusher.
CKAN versions

~2.10.0, ~2.11.0

Show details

These CKAN Versions are exactely matched:

Download-Url (zip)
Download-Url commit date
2024-11-20
Url to repo
Category
Cloud Infrastructure & Storage


Background Infos

Description (long)
Show details

XLoader is a CKAN extension that provides a replacement for DataPusher with better performance and more robust data loading. It automatically detects data types and creates appropriate DataStore table structures. The extension provides progress monitoring, improved error handling, and is significantly faster than the original DataPusher for large datasets. XLoader supports various formats including CSV, XLS, XLSX, TSV, and ODS files with compression support and configurable processing options.

Version
1.0.1
Version release date
2024-11-20
Contact name
CKAN Development Team
Contakt email
Contact Url
(not set)


Installation Guide

Configuration hints

Configure formats to process, maximum file size limits, and database connection for job processing. Ensure proper PostgreSQL permissions for DataStore operations.

Plugins to configure (ckan.ini)
xloader
CKAN Settings (ckan.ini)
# ckanext.xloader.formats = csv xls xlsx tsv ods # ckanext.xloader.max_content_length = 1000000000 # ckanext.xloader.just_load_with_messytables = false # ckanext.xloader.compression_support = true # ckanext.xloader.jobs_db.uri = postgresql://ckan:pass@localhost/ckan_jobs
DB migration to be executed
(not set)
<< back to Extensions