~2.9, ~2.10, ~2.11

Requirements: - CKAN 2.2+ - Celery (for background task processing) - Python 2.6 or 2.7 - GDAL/OGR libraries (for shapefile conversion) - Python packages: fiona, shapely

Installation:

1. Install GDAL/OGR system dependencies: sudo apt-get update sudo apt-get install gdal-bin libgdal-dev python-gdal

2. Activate CKAN virtualenv: . /usr/lib/ckan/default/bin/activate

3. Install Python geospatial libraries: pip install fiona shapely

4. Install extension: pip install ckanext-geopusher

   Or from source: git clone https://github.com/datacats/ckanext-geopusher.git cd ckanext-geopusher python setup.py develop

5. Add plugin to ckan.plugins in production.ini: ckan.plugins = … geopusher …

6. Ensure Celery is configured: ckan.celery.queues = celery

7. Restart CKAN: sudo service apache2 reload

8. Start Celery worker: paster celeryd –config=/etc/ckan/default/production.ini

Configuration:

Optional configuration settings in production.ini:

Minimum hours to wait before re-checking a resource (default: 24)

ckanext.geopusher.some_setting = some_default_value

Note: Specific configuration options are documented in the extension code. Refer to plugin source for detailed settings.

Celery Setup:

Geopusher requires Celery for background processing:

1. Install Celery if not already installed: pip install celery

2. Configure broker in production.ini:

   Redis (recommended)

   ckan.celery.broker_backend = redis ckan.celery.broker_host = localhost:6379

   Or RabbitMQ

   ckan.celery.broker_backend = rabbitmq ckan.celery.broker_host = localhost

3. Start Celery worker: paster celeryd –config=/etc/ckan/default/production.ini

   For production (using supervisor): Create /etc/supervisor/conf.d/ckan-celery.conf:

   [program:ckan-celery] command=/usr/lib/ckan/default/bin/paster celeryd –config=/etc/ckan/default/production.ini directory=/usr/lib/ckan/default/src/ckan user=www-data autostart=true autorestart=true stderr_logfile=/var/log/ckan/ckan-celery.err.log stdout_logfile=/var/log/ckan/ckan-celery.out.log

4. Reload supervisor: sudo supervisorctl reread sudo supervisorctl update

Paster Commands:

geopusher command provides administrative functions:

paster geopusher [subcommand] –config=production.ini

Available subcommands documented in source code.

Usage:

Automatic Conversion:

1. Upload shapefile as resource:

   • Upload ZIP file containing shapefile components
   • Must include: .shp, .shx, .dbf files
   • Optional: .prj (projection), .cpg (codepage)

2. Geopusher automatically:

   • Detects shapefile resource
   • Queues conversion task in Celery
   • Converts shapefile to GeoJSON
   • Creates new GeoJSON resource
   • Links GeoJSON to original shapefile

3. View results:

   • GeoJSON resource appears in dataset
   • Compatible with geospatial viewer extensions
   • Can be displayed on map viewers

Supported Formats:

Input: - Shapefile (as ZIP archive) - Must contain: .shp, .shx, .dbf - Recommended: .prj for projection info

Output: - GeoJSON (RFC 7946 compliant) - WGS84 coordinate system (EPSG:4326) - Suitable for web mapping libraries

Integration with Viewers:

Geopusher works well with geospatial viewer extensions:

• ckanext-geoview: OpenLayers/Leaflet viewers
• ckanext-spatial: Spatial search and preview
• ckanext-mapviews: Various map visualization options

GeoJSON resources can be immediately displayed on maps.

Development:

1. Clone repository: git clone https://github.com/datacats/ckanext-geopusher.git cd ckanext-geopusher

2. Install for development: python setup.py develop pip install -r dev-requirements.txt

3. Create test.ini from test-core.ini template

4. Run tests: nosetests –nologcapture –with-pylons=test.ini

5. Run with coverage: pip install coverage nosetests –nologcapture –with-pylons=test.ini –with-coverage –cover-package=ckanext.geopusher –cover-inclusive –cover-erase –cover-tests

Troubleshooting:

1. Conversion not happening:

   • Check Celery worker is running: ps aux | grep celery
   • Verify shapefile ZIP contains required files
   • Check CKAN logs for errors: tail -f /var/log/ckan/ckan.log
   • Ensure GDAL is properly installed: gdalinfo –version

2. GDAL import errors:

   • Install system GDAL: sudo apt-get install gdal-bin
   • Install Python GDAL: pip install GDAL==$(gdal-config –version)
   • Set library path: export LD_LIBRARY_PATH=/usr/lib

3. Fiona/Shapely errors:

   • Install build dependencies: sudo apt-get install build-essential
   • Install GEOS: sudo apt-get install libgeos-dev
   • Reinstall packages: pip install –force-reinstall fiona shapely

4. Celery tasks not executing:

   • Check broker connection (Redis/RabbitMQ)
   • Verify celery configuration in production.ini
   • Check task queue: celery -A ckan.lib.celery_app inspect active
   • Review Celery logs

5. Projection errors:

   • Ensure .prj file included in shapefile ZIP
   • GeoJSON defaults to WGS84 (EPSG:4326)
   • Verify projection definition is recognized

Celery Task Import:

The extension registers Celery tasks via entry point:

[ckan.celery_task] tasks=ckanext.geopusher.celery_import:task_imports

Tasks are automatically registered when Celery worker starts.

Development Status: Beta (4)

License: AGPL v3.0 or later

Keywords: CKAN, Geopusher, Shape, GeoJSON, shapefile, geospatial, conversion

Developer: BoxKite / Datacats

Related Extensions: - ckanext-geoview: For displaying converted GeoJSON - ckanext-spatial: Spatial search capabilities - ckanext-scheming: Schema for geographic metadata

PyPI: https://pypi.python.org/pypi/ckanext-geopusher

CI/CD: - Travis CI: https://travis-ci.org/datacats/ckanext-geopusher - Coveralls: https://coveralls.io/r/datacats/ckanext-geopusher