Extension ZIP File Preview

~2.9, ~2.10, ~2.11

Requirements: - CKAN 2.3+ (Resource view support required) - Python 2.6 or 2.7 - zipfile Python module (standard library) - Network access for externally hosted ZIP files

Installation:

1. Activate CKAN virtualenv: . /usr/lib/ckan/default/bin/activate

2. Install extension: pip install ckanext-zippreview

   Or from source: git clone https://github.com/datagovau/ckanext-zippreview.git cd ckanext-zippreview python setup.py develop

3. Install dependencies: pip install -r dev-requirements.txt

4. Add plugin to ckan.plugins in production.ini: ckan.plugins = … zip_view …

5. Add to default views (optional but recommended): ckan.views.default_views = … zip_view …

   This automatically creates ZIP previews for new ZIP resources.

6. Restart CKAN: sudo service apache2 reload

Configuration:

Default Views (Automatic Preview Creation):

Add zip_view to default views for automatic preview generation:

ckan.views.default_views = image_view text_view recline_view zip_view

When enabled, ZIP preview automatically created for: - Resources with .zip extension - Resources with format = “ZIP” - Resources with MIME type application/zip

Manual View Creation:

If not using default_views, create views manually:

1. Go to resource page
2. Click “Manage” → “Views”
3. Click “New view”
4. Select “ZIP Preview” from view types
5. Configure and save

Usage:

Automatic Preview:

1. Upload or link to ZIP file as resource
2. Set format to “ZIP” or use .zip extension
3. ZIP preview automatically created (if in default_views)
4. Preview shows file listing with:
   • File/folder names
   • File sizes
   • Directory structure
   • Nested folders (expandable)

Manual Preview Creation:

1. Navigate to resource
2. Click “Manage” button
3. Select “Views” tab
4. Click “New view”
5. Choose “ZIP Preview”
6. Save view

Preview Display Features:

File Listing: - Hierarchical directory structure - File names and paths - File sizes (human-readable: KB, MB, GB) - Folder expansion/collapse - Total archive size - Number of files

Supported ZIP Features: - Standard ZIP archives - Nested directories - Large files (size displayed) - Compressed and uncompressed

Limitations: - Only shows file listing (no content preview) - Cannot extract individual files via interface - Large archives may take time to process - Password-protected ZIPs not supported

External ZIP Files:

Works with externally hosted ZIP files:

1. Add resource with external ZIP URL
2. Set format to “ZIP”
3. Extension downloads and previews remote ZIP
4. Requires network access from CKAN server

Security Considerations:

External Files: - CKAN server downloads external ZIPs - Ensure trusted sources only - Monitor disk usage - Consider timeout settings

ZIP Bombs: - Extension reads ZIP header only - No full extraction occurs - Safe against most ZIP bomb attacks - Large listings may impact performance

Development:

1. Clone repository: git clone https://github.com/datagovau/ckanext-zippreview.git cd ckanext-zippreview

2. Install for development: python setup.py develop pip install -r dev-requirements.txt

3. Create test.ini from template

4. Run tests: pytest –ckan-ini test.ini

5. Test with sample ZIP files:

   • Create test resources with various ZIP structures
   • Test nested directories
   • Test large file listings
   • Test external URLs

Customization:

Customize Preview Template:

Override templates/zippreview/view.html in your theme:

{% extends “base.html” %}

{% block primary_content %}

{% endblock %}

Customize Styling:

Add CSS for ZIP preview:

.zip-preview { border: 1px solid #ddd; padding: 20px; border-radius: 4px; }

.zip-file-list { list-style: none; font-family: monospace; }

.zip-file-list li { padding: 5px 0; border-bottom: 1px solid #eee; }

.file-name { flex: 1; }

.file-size { color: #666; font-size: 0.9em; }

Troubleshooting:

1. Preview not appearing:

   • Verify zip_view plugin is enabled
   • Check resource format is set to “ZIP”
   • Ensure .zip file extension
   • Verify resource view created
   • Check CKAN version >= 2.3

2. External ZIP not loading:

   • Verify URL is accessible from CKAN server
   • Check firewall/network rules
   • Test URL manually: wget
   • Review CKAN logs for errors
   • Check disk space for download

3. Large archives slow:

   • Listing very large archives takes time
   • Consider timeout settings
   • Monitor server resources
   • Optimize archive structure if possible

4. Empty preview:

   • Verify ZIP file is valid
   • Test ZIP locally: unzip -l file.zip
   • Check for corrupted archives
   • Review error logs

5. Permission errors:

   • Verify CKAN has read access to files
   • Check temporary directory permissions
   • Ensure web server user can access storage

Performance Optimization:

1. Caching:

   • Cache file listings for large archives
   • Store preview data in resource extras
   • Refresh on resource update

2. Timeouts:

   • Set reasonable timeouts for external downloads
   • Limit archive processing time
   • Provide fallback for large files

3. Resource Limits:

   • Consider max archive size
   • Limit number of files displayed
   • Paginate very large listings

Best Practices:

1. Resource Metadata:

   • Always set format to “ZIP”
   • Use descriptive resource names
   • Add description of archive contents
   • Document archive structure

2. File Organization:

   • Use clear directory structure in ZIPs
   • Include README files in archives
   • Use descriptive file names
   • Avoid deeply nested structures

3. External Resources:

   • Verify external URLs are stable
   • Monitor for broken links
   • Consider local copies for critical data
   • Document external dependencies

4. User Guidance:

   • Explain what preview shows (listing only)
   • Provide download link for full archive
   • Document how to extract files
   • Include usage instructions

Use Cases:

1. Data Archives:

   • Multiple CSV files in one package
   • Let users see what’s inside before downloading
   • Preview structure of data collections

2. Geographic Data:

   • Shapefiles (multiple components in ZIP)
   • Show file structure
   • Help users identify correct files

3. Document Collections:

   • Multiple PDFs or reports
   • Preview document list
   • Help users find specific documents

4. Code Repositories:

   • Source code archives
   • Show directory structure
   • Help developers navigate

Integration:

With Other View Types:

Combine with other previews:

ckan.views.default_views = image_view text_view recline_view pdf_view zip_view

• Images: Preview individual image files
• Text: Preview README or text files
• PDF: Preview documentation
• ZIP: Preview archive structure

With Data Extraction:

Extend to extract specific files:

class EnhancedZipView(ZipPreviewPlugin): def can_extract_file(self, filename): # Allow extracting specific file types return filename.endswith((‘.csv’, ‘.json’, ‘.txt’))

def extract_file(self, archive, filename):
    # Extract and preview individual file
    return archive.read(filename)

Development Status: Beta (4)

License: AGPL v3.0 or later

Keywords: CKAN, zip, compressed, archive, preview, resource view

Developer: Alex Sadleir / data.gov.au

Related Extensions: - ckanext-pdfview: PDF file preview - ckanext-geoview: Geospatial data preview - ckanext-textview: Text file preview - ckanext-archiver: Archive downloading and checking