Extension Data.Vic Harvester


Extension Basics

Title
Data.Vic Harvester
Name
ckanext-datavic-harvester
Type
Public extension
Description
Custom harvester for Data.Vic schema and sources
CKAN versions
Download-Url (zip)
Download-Url commit date
2022-07-04
Url to repo
Category
Data Management & Quality


Background Infos

Description (long)
Show details

A specialized harvester extension for CKAN designed to work with Data.Vic’s custom metadata schema. This extension extends the standard ckanext-harvest functionality with multiple harvester types specifically tailored for Victorian government data sources. Includes four harvester implementations: DataVicCKANHarvester for standard CKAN instances with Data.Vic schema, DataVicDCATJSONHarvester for DCAT JSON sources, MetaShareHarvester for MetaShare repositories, and DelwpHarvester for Department of Environment, Land, Water and Planning data. The extension supports an ignore_private_datasets parameter to exclude private records when harvesting from public Data.Vic sources to the Data Directory. Requires ckanext-harvest to be installed and enabled. When creating a new harvest source via the harvest admin UI, select ‘CKAN Harvester for Data.Vic’ as the harvest type.

Version
0.0.1
Version release date
2022-07-04
Contact name
Salsa Digital
Contakt email
Contact Url
(not set)


Installation Guide

Configuration hints

Installation: Activate CKAN virtualenv (. /app/src/ckan/default/bin/activate), cd /app/src/ckan/default/src/ckanext-datavic-harvester, python setup.py develop. Add ‘datavic_ckan_harvester’ to ckan.plugins in your CKAN config file (by default /app/ckan/default/ckan.ini). Restart CKAN: docker-compose restart ckan (Docker deployment) or sudo service apache2 reload (traditional). Prerequisites: ckanext-harvest extension must be installed and enabled in CKAN .ini file. Harvester Types: When creating a new harvest source via the ckanext-harvest admin UI, select ‘CKAN Harvester for Data.Vic’ as the harvest type. Available harvesters: datavic_ckan_harvester (standard CKAN with Data.Vic schema), datavic_dcat_json_harvester (DCAT JSON sources), metashare_harvester (MetaShare repositories), delwp_harvester (Department of Environment, Land, Water and Planning). Additional Parameters: ignore_private_datasets (default: False) - set to True on Public data.vic harvest from Data Directory to exclude private records. Plugin Names: datavic_ckan_harvester, datavic_dcat_json_harvester, metashare_harvester, delwp_harvester. Python 3.7+ supported.

Plugins to configure (ckan.ini)
datavic_ckan_harvester datavic_dcat_json_harvester metashare_harvester delwp_harvester
CKAN Settings (ckan.ini)
# Keine spezifischen Umgebungsvariablen # Harvest source parameters konfigurierbar via Admin UI
DB migration to be executed
(not set)
<< back to Extensions