### Install ckanapi Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/README.md Install the ckanapi library using pip. ```bash pip install ckanapi ``` -------------------------------- ### CKAN CLI: Get Help Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/quick-reference.md Access help documentation for the CKAN CLI and its subcommands. ```bash ckanapi -h ckanapi action -h ``` -------------------------------- ### Example Source Location Reference Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/INDEX.md This example shows the format for source location information, indicating the file path and line number ranges for code definitions. It aids in cross-referencing documentation with the actual source code. ```text Source location: ckanapi/remoteckan.py, lines 23-122 ``` -------------------------------- ### CLI Configuration: Explicit Config File Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/configuration.md Example of using the `ckanapi` CLI with an explicit configuration file path. ```bash ckanapi action package_list -c /etc/ckan/production.ini ``` -------------------------------- ### Install ckanapi Source: https://github.com/ckan/ckanapi/blob/master/README.md Installation commands for ckanapi using pip or conda. ```bash pip install ckanapi ``` ```bash conda install -c conda-forge ckanapi ``` -------------------------------- ### Create Package with Correct Arguments Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/quick-reference.md This example demonstrates the correct way to create a package by providing all required arguments, including 'name' and 'title'. It handles potential ValidationErrors. ```python from ckanapi import ValidationError try: dataset = ckan.action.package_create( name='my-dataset', title='My Dataset' ) except ValidationError as e: print(f"Errors: {e.error_dict}") ``` -------------------------------- ### Initialize RemoteCKAN Client (GET-only) Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/quick-reference.md Create an instance of RemoteCKAN configured to only allow GET requests. ```python # GET-only requests ckan = RemoteCKAN('https://demo.ckan.org', get_only=True) ``` -------------------------------- ### CKAN Action Result Examples Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/types.md Demonstrates how to call CKAN actions and the typical return types for different actions like group_list, package_show, and package_search. ```python from ckanapi import RemoteCKAN ckan = RemoteCKAN('https://demo.ckan.org') # Returns list of strings groups = ckan.action.group_list() # Type: list[str] # Returns dict with dataset information dataset = ckan.action.package_show(id='my-dataset') # Type: dict with keys: id, name, title, resources, etc. # Returns dict with search results search_result = ckan.action.package_search(q='test', rows=10) # Type: dict with keys: count, results, facets, etc. ``` -------------------------------- ### CLI Configuration: development.ini in Current Directory Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/configuration.md Example of the `ckanapi` CLI implicitly using `development.ini` located in the current directory. ```bash cd /path/to/ckan/src ckanapi action package_list ``` -------------------------------- ### CLI Configuration: Remote Server Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/configuration.md Example of using the `ckanapi` CLI to connect to a remote CKAN instance without needing a local configuration file. ```bash ckanapi action package_list -r https://demo.ckan.org ``` -------------------------------- ### Get datastore resource info Source: https://github.com/ckan/ckanapi/blob/master/README.md Retrieve detailed information about a resource in the datastore. ```bash $ ckanapi action datastore_info id=my-resource-id-or-alias { "meta": { "aliases": [ "test_alias" ], "count": 1000, ... } ``` -------------------------------- ### Perform Valid CKAN Package Searches Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/errors.md Examples of constructing valid search queries using different operators and conditions for package searches. ```python from ckanapi import RemoteCKAN ckan = RemoteCKAN('https://demo.ckan.org') # Valid queries result1 = ckan.action.package_search(q='government') result2 = ckan.action.package_search(q='+organization:example') result3 = ckan.action.package_search(q='+tags:open +res_format:CSV') ``` -------------------------------- ### CLI Configuration: CKAN_INI Environment Variable Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/configuration.md Example of using the `ckanapi` CLI with the configuration file path specified via the `CKAN_INI` environment variable. ```bash export CKAN_INI=/etc/ckan/production.ini ckanapi action package_list ``` -------------------------------- ### ActionShortcut Mechanism Example Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/action-shortcut.md Illustrates how ActionShortcut uses Python's descriptor protocol to create action methods dynamically and calls `call_action`. ```python from ckanapi import RemoteCKAN ckan = RemoteCKAN('https://demo.ckan.org') # When you write: result = ckan.action.package_show(id='my-dataset') # ActionShortcut does: # 1. __getattr__('package_show') returns a function # 2. That function separates {'id': 'my-dataset'} into data and files # 3. Calls ckan.call_action('package_show', data_dict={'id': 'my-dataset'}, files={}) # 4. Returns the result ``` -------------------------------- ### Handle Authorization Issues for Public and Private Actions Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/errors.md This example demonstrates how to handle NotAuthorized exceptions for both anonymous and authenticated users, showing how to gracefully manage access to public and private resources. ```python from ckanapi import RemoteCKAN, NotAuthorized ckan_anon = RemoteCKAN('https://demo.ckan.org') ckan_auth = RemoteCKAN('https://demo.ckan.org', apikey='valid-key') try: # Public read succeeds datasets = ckan_anon.action.package_list() except NotAuthorized: print('Cannot list packages') try: # Private write fails dataset = ckan_anon.action.package_create(name='test', title='Test') except NotAuthorized: print('Must authenticate to create packages') dataset = ckan_auth.action.package_create(name='test', title='Test') ``` -------------------------------- ### Create Package with Missing Arguments (Wrong) Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/quick-reference.md This example shows an incorrect attempt to create a package without the required 'name' argument, which will raise a ValidationError. ```python try: ckan.action.package_create(title='Missing name') except ValidationError as e: # e.error_dict has the field errors print(e.error_dict) ``` -------------------------------- ### Complex Package Search Query Source: https://github.com/ckan/ckanapi/blob/master/README.md Perform a package search with multiple query parameters combined using boolean operators. This example searches for packages with specific organization, resource format, and tags. ```python from ckanapi import RemoteCKAN ua = 'ckanapiexample/1.0 (+http://example.com/my/website)' demo = RemoteCKAN('https://demo.ckan.org', user_agent=ua) packages = demo.action.package_search(q='+organization:sample-organization +res_format:GeoJSON +tags:geojson') print(packages) ``` -------------------------------- ### Fetching Package List and User Show with Shortcuts Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/localckan.md Shows how to use action shortcuts to retrieve a list of datasets and specific user information. ```python # Works with any CKAN action datasets = ckan.action.package_list(limit=10) user = ckan.action.user_show(id='alice') ``` -------------------------------- ### Using Action Shortcuts for Package Show Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/localckan.md Illustrates the use of action shortcuts for fetching package details, comparing it to the direct call_action method. ```python ckan = LocalCKAN() # These are equivalent: result1 = ckan.action.package_show(id='my-dataset') result2 = ckan.call_action('package_show', {'id': 'my-dataset'}) ``` -------------------------------- ### Package Creation and List Actions Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/localckan.md Demonstrates creating a package and listing packages, showing that the data dictionary is not affected by internal modifications after a call. ```python data = {'name': 'my-dataset', 'title': 'My Dataset'} result1 = ckan.call_action('package_create', data) result2 = ckan.action.package_list() # Clean, isolated call ``` -------------------------------- ### Resource Creation with File Upload Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/localckan.md Shows how to create a resource by uploading a file from the local filesystem. ```python ckan = LocalCKAN() # Standard file upload with open('/path/to/data.csv', 'rb') as f: resource = ckan.action.resource_create( package_id='my-dataset', url='dummy', upload=f ) ``` -------------------------------- ### Resource Creation with File-like Object Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/localckan.md Demonstrates creating a resource using a file-like object without a name, passed as a tuple with a filename. ```python from io import BytesIO data = BytesIO(b'CSV data here') # Pass as tuple with filename resource = ckan.call_action( 'resource_create', {'package_id': 'my-dataset', 'url': 'dummy'}, files={'upload': ('data.csv', data)} ) ``` -------------------------------- ### Initialize LocalCKAN Client (Site Admin) Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/quick-reference.md Create an instance of LocalCKAN for interacting with a local CKAN site as the site administrator. ```python # Site admin (default) ckan = LocalCKAN() ``` -------------------------------- ### Search datasets with facets Source: https://github.com/ckan/ckanapi/blob/master/README.md Get dataset counts per organization using KEY:JSON parameters. ```bash $ ckanapi action package_search facet.field:'["organization"]' rows:0 { "facets": { "organization": { "org1": 42, "org2": 21, ... } }, ... } ``` -------------------------------- ### Initialize RemoteCKAN Client (with API Key) Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/quick-reference.md Create an instance of RemoteCKAN with an API key for authenticated access. ```python # With API key ckan = RemoteCKAN('https://demo.ckan.org', apikey='my-key') ``` -------------------------------- ### Search Datasets with Facets Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/quick-reference.md Retrieve facet counts for fields like organization and tags. Set rows to 0 to only get facets. ```python result = ckan.action.package_search( q='', facet_field=['organization', 'tags'], rows=0 ) # Access facets for org, count in result['facets']['organization'].items(): print(f"{org}: {count} datasets") ``` -------------------------------- ### Create a Dataset Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/quick-reference.md Create a new dataset with basic metadata, tags, and resources. ```python dataset = ckan.action.package_create( name='my-dataset', title='My Dataset', author='John Doe', author_email='john@example.com', tags=[{'name': 'tag1'}, {'name': 'tag2'}], resources=[{ 'url': 'https://example.com/data.csv', 'format': 'CSV', 'description': 'Data file' }] ) ``` -------------------------------- ### Get Site Admin Username Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/localckan.md Retrieve the username of the site administrator by calling the 'get_site_user' action. This is useful for identifying the default user context. ```python ckan = LocalCKAN() admin = ckan.get_site_username() print(f"Site admin is: {admin}") ``` -------------------------------- ### Initialize LocalCKAN Client (Custom Context) Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/quick-reference.md Create an instance of LocalCKAN with a custom context dictionary to control behavior, such as ignoring authentication. ```python # With custom context ckan = LocalCKAN(context={'ignore_auth': False}) ``` -------------------------------- ### CKAN CLI: Create Dataset Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/quick-reference.md Create a new dataset via the command line, specifying metadata and optionally a configuration file. ```bash ckanapi action package_create name=my-dataset title='My Dataset' -c /etc/ckan/production.ini ``` -------------------------------- ### Importing and Instantiating ActionShortcut Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/action-shortcut.md Demonstrates how to import ActionShortcut and access it via the .action attribute of a CKAN client instance, or by direct instantiation (though less common). ```python from ckanapi import RemoteCKAN from ckanapi.common import ActionShortcut ckan = RemoteCKAN('https://demo.ckan.org') # The .action attribute is already an ActionShortcut instance action = ckan.action # Or create one manually (not typical): action = ActionShortcut(ckan) result = action.package_list() ``` -------------------------------- ### Create a resource with file upload Source: https://github.com/ckan/ckanapi/blob/master/README.md Upload a file to a resource using the KEY@FILE syntax. ```bash $ ckanapi action resource_create package_id=my-dataset-with-files \ upload@/path/to/file/to/upload.csv ``` -------------------------------- ### Initialize RemoteCKAN Client (with Custom User Agent) Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/quick-reference.md Create an instance of RemoteCKAN, specifying a custom user agent string for identification. ```python # With custom user agent ckan = RemoteCKAN('https://demo.ckan.org', user_agent='myapp/1.0') ``` -------------------------------- ### Handling NotAuthorized Exception Source: https://github.com/ckan/ckanapi/blob/master/README.md Catch the NotAuthorized exception when attempting an action that requires authentication or access to a non-existent resource. This example demonstrates a failed package creation attempt. ```python from ckanapi import RemoteCKAN, NotAuthorized ua = 'ckanapiexample/1.0 (+http://example.com/my/website)' demo = RemoteCKAN('https://demo.ckan.org', apikey='phony-key', user_agent=ua) try: pkg = demo.action.package_create(name='my-dataset', title='not going to work') except NotAuthorized: print('denied') ``` -------------------------------- ### Initialize LocalCKAN Client (Anonymous) Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/quick-reference.md Create an instance of LocalCKAN to simulate anonymous user access. ```python # Anonymous ckan = LocalCKAN(username='') ``` -------------------------------- ### Initialize LocalCKAN Client (Specific User) Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/quick-reference.md Create an instance of LocalCKAN, specifying a particular user to act as. ```python # Specific user ckan = LocalCKAN(username='alice') ``` -------------------------------- ### Safely Get Dataset, Returning None if Not Found Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/errors.md Implement a safe function to retrieve a dataset, returning None if a NotFound exception is raised. This pattern is useful for checking the existence of a dataset before proceeding. ```python from ckanapi import RemoteCKAN, NotFound ckan = RemoteCKAN('https://demo.ckan.org') def get_dataset_safe(dataset_id): try: return ckan.action.package_show(id=dataset_id) except NotFound: return None dataset = get_dataset_safe('my-dataset') if dataset is None: print('Dataset does not exist') ``` -------------------------------- ### LocalCKAN Data Isolation Example Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/localckan.md Demonstrates LocalCKAN's data isolation features, ensuring that mutations to data_dict or context within one action call do not affect subsequent calls. This is crucial for reliable internal CKAN operations. ```python ckan = LocalCKAN() ``` -------------------------------- ### Initialize TestAppCKAN Source: https://github.com/ckan/ckanapi/blob/master/README.md Make action requests to a webtest.TestApp instance for CKAN testing. ```python from ckanapi import TestAppCKAN from webtest import TestApp test_app = TestApp(...) demo = TestAppCKAN(test_app, apikey='my-test-key') groups = demo.action.group_list(id='data-explorer') ``` -------------------------------- ### Import Datasets (Create Only) Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/cli-commands.md Import datasets, but only create new records and skip any existing ones. This is useful for initial data loading. ```bash ckanapi load datasets -I new-datasets.jsonl -n -c /etc/ckan/production.ini ``` -------------------------------- ### Run Tests Source: https://github.com/ckan/ckanapi/blob/master/README.md Execute the test suite using the unittest module. ```bash python -m unittest discover ``` -------------------------------- ### Create a Resource Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/quick-reference.md Add a new resource to an existing dataset. Uploads a file from the local system. ```python with open('data.csv', 'rb') as f: resource = ckan.action.resource_create( package_id='my-dataset', url='dummy', name='Data File', format='CSV', upload=f ) ``` -------------------------------- ### Initialize TestAppCKAN Client Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/quick-reference.md Create an instance of TestAppCKAN to test CKAN API actions against a webtest.TestApp instance. ```python from webtest import TestApp from ckanapi import TestAppCKAN test_app = TestApp(...) ckan = TestAppCKAN(test_app) ckan = TestAppCKAN(test_app, apikey='test-key') ``` -------------------------------- ### Initialize RemoteCKAN Client (with Custom Session) Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/quick-reference.md Create an instance of RemoteCKAN using a pre-configured requests.Session object. ```python # Custom session import requests session = requests.Session() ckan = RemoteCKAN('https://demo.ckan.org', session=session) ``` -------------------------------- ### Instantiate TestAppCKAN Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/testappckan.md Create a TestAppCKAN client for testing against a WebTest TestApp instance. Authentication can be provided via an API key. ```python from ckanapi import TestAppCKAN from webtest import TestApp from ckan.config.middleware import make_app # Create a test app from the CKAN application test_app = TestApp(make_app({}, **test_config)) # Create TestAppCKAN with or without authentication ckan_test = TestAppCKAN(test_app) ckan_auth = TestAppCKAN(test_app, apikey='test-api-key') # Use like a normal CKAN API client result = ckan_test.action.package_list() dataset = ckan_auth.action.package_create(name='test-dataset', title='Test') ``` -------------------------------- ### Import Datasets with Parallel Processes Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/cli-commands.md Import datasets using multiple parallel processes for faster data ingestion. Specify the number of processes and configuration. ```bash ckanapi load datasets -I datasets.jsonl -p 4 -c /etc/ckan/production.ini ``` -------------------------------- ### Mixed Arguments with File Upload Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/action-shortcut.md Illustrates how to use ActionShortcut with a mix of regular arguments and file uploads, such as updating an organization's details and image. ```python from ckanapi import RemoteCKAN ckan = RemoteCKAN('https://demo.ckan.org', apikey='my-key') # Some arguments as files, some as data with open('image.png', 'rb') as image: org = ckan.action.organization_patch( id='my-org', title='Updated Title', image_upload=image # Detected as file ) ``` -------------------------------- ### Export Datasets as Datapackages Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/cli-commands.md Export datasets and download their resources, outputting them as datapackages. Specify a directory to store the downloaded resources and datapackage metadata. ```bash ckanapi dump datasets --all --datapackages=/tmp/packages/ -c /etc/ckan/production.ini ``` -------------------------------- ### Import Compressed File Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/cli-commands.md Import datasets from a gzip-compressed JSON lines file. The '-z' option indicates that the input file is compressed. ```bash ckanapi load datasets -I datasets.jsonl.gz -z -c /etc/ckan/production.ini ``` -------------------------------- ### Upload File via ActionShortcut Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/quick-reference.md Upload a file as part of a resource creation using the ckan.action shortcut. Supports direct file objects or a (filename, file) tuple. ```python # Single file with open('data.csv', 'rb') as f: resource = ckan.action.resource_create( package_id='dataset-id', url='dummy', upload=f ) ``` ```python # Tuple format (filename, file) import io data = io.BytesIO(b'CSV data') resource = ckan.action.resource_create( package_id='dataset-id', url='dummy', upload=('data.csv', data) ) ``` -------------------------------- ### Load Datasets with Checkpoint Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/cli-commands.md Loads datasets from a gzipped JSONL file, resuming from the last checkpoint. Use the -s flag to specify the checkpoint interval and -c for the configuration file. ```bash ckanapi load datasets -I datasets.jsonl.gz -z -s 1000 -c /etc/ckan/production.ini ``` -------------------------------- ### CKAN Package Search with Facets Usage Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/types.md Demonstrates how to perform a package search with specific query parameters and facet fields, then process the results and facet data. ```python from ckanapi import RemoteCKAN ckan = RemoteCKAN('https://demo.ckan.org') # Search with facets result = ckan.action.package_search( q='government', facet_field=['organization', 'tags'], rows=20 ) print(f"Found {result['count']} datasets") for dataset in result['results']: print(f" - {dataset['name']}: {dataset['title']}") ``` -------------------------------- ### LocalCKAN Constructor Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/localckan.md Initializes a LocalCKAN client. This client can be used to execute CKAN actions directly from within a CKAN extension or plugin. You can specify a username to execute actions as, or leave it as None to use the site admin user. An optional context dictionary can also be provided. ```APIDOC ## LocalCKAN Constructor ### Description Creates a LocalCKAN client for calling CKAN actions directly from a CKAN extension or plugin. ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters Table | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | username | str | No | None | Username to execute actions as. If None, uses the site admin user (obtained via 'get_site_user' action). Can be an empty string for anonymous access | | context | dict | No | None | Base context dictionary for actions. 'user' key is automatically added/overridden with the username | ### Example ```python from ckanapi import LocalCKAN # Use site admin user (default) registry = LocalCKAN() new_dataset = registry.action.package_create(name='my-dataset', title='My Dataset') # Anonymous access (limited permissions) anon = LocalCKAN(username='') status = anon.action.status_show() # Specific user user_registry = LocalCKAN(username='alice') packages = user_registry.action.package_list() # Custom context context = { 'ignore_auth': False, 'model': None, # Will be set by LocalCKAN } ckan = LocalCKAN(context=context) ``` ``` -------------------------------- ### Instantiate LocalCKAN Client Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/localckan.md Create instances of LocalCKAN to interact with CKAN's action system. Supports default site admin, anonymous access, specific users, and custom contexts. ```python from ckanapi import LocalCKAN # Use site admin user (default) registry = LocalCKAN() new_dataset = registry.action.package_create(name='my-dataset', title='My Dataset') # Anonymous access (limited permissions) anon = LocalCKAN(username='') status = anon.action.status_show() # Specific user user_registry = LocalCKAN(username='alice') packages = user_registry.action.package_list() # Custom context context = { 'ignore_auth': False, 'model': None, # Will be set by LocalCKAN } ckan = LocalCKAN(context=context) ``` -------------------------------- ### Load datasets from file Source: https://github.com/ckan/ckanapi/blob/master/README.md Updates or creates datasets from a JSON lines file using 3 parallel worker processes. ```bash $ ckanapi load datasets -I datasets.jsonl.gz -z -p 3 -c /etc/ckan/production.ini ``` -------------------------------- ### CKAN Package Create, Patch, and Show Usage Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/types.md Shows how to create, update (patch), and retrieve a CKAN dataset using the ckanapi library, demonstrating the use of package dictionaries. ```python from ckanapi import RemoteCKAN ckan = RemoteCKAN('https://demo.ckan.org') # Create dataset dataset = ckan.action.package_create( name='my-dataset', title='My Dataset', author='John Doe', tags=[{'name': 'open-data'}, {'name': 'government'}], resources=[{ 'url': 'https://example.com/data.csv', 'format': 'CSV', 'description': 'Main data file' }] ) # Update dataset updated = ckan.action.package_patch( id='my-dataset', title='Updated Title' ) # Retrieve dataset dataset = ckan.action.package_show(id='my-dataset') print(dataset['title']) print(dataset['tags']) for resource in dataset['resources']: print(resource['url']) ``` -------------------------------- ### Perform Basic Package Search Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/quick-reference.md Execute a basic package search query using the package_search action and iterate through the results. The result includes a count of found datasets and a list of dataset dictionaries. ```python result = ckan.action.package_search(q='government') print(f"Found {result['count']} datasets") for dataset in result['results']: print(f" - {dataset['name']}") ``` -------------------------------- ### CKAN CLI: Load Datasets Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/README.md Load datasets from a JSON lines file into a CKAN instance using the CLI, specifying the configuration file. ```bash ckanapi load datasets -I datasets.jsonl -c /etc/ckan/production.ini ``` -------------------------------- ### Batch Upload Resources Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/cli-commands.md Use the 'ckanapi batch' command to perform bulk operations on resources, specifying an input file and local files for upload. Configuration is done via a specified INI file. ```bash cat << 'EOF' > batch-upload.jsonl {"action":"resource_patch","data":{"id":"res-1"},"files":{"upload":"data1.csv"}} {"action":"resource_patch","data":{"id":"res-2"},"files":{"upload":"data2.csv"}} EOF ckanapi batch -I batch-upload.jsonl --local-files -c /etc/ckan/production.ini ``` -------------------------------- ### Tuple File Format for Uploads Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/action-shortcut.md Demonstrates how to pass files using the requests library-compatible tuple format (filename, file_object) with ActionShortcut. ```python # File passed as (filename, file_object) tuple import io csv_data = io.BytesIO(b'id,name\n1,test') resource = ckan.action.resource_create( package_id='my-dataset', url='dummy', upload=('data.csv', csv_data) # Tuple with filename, file object ) ``` -------------------------------- ### Use is_file_like in ActionShortcut Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/common-functions.md Demonstrates how file-like objects are automatically detected and handled when passed as keyword arguments to ckanapi ActionShortcut methods. ```python from ckanapi import RemoteCKAN ckan = RemoteCKAN('https://demo.ckan.org', apikey='key') # This automatically detects 'upload' is file-like with open('data.csv', 'rb') as f: resource = ckan.action.resource_create( package_id='dataset-id', url='dummy', upload=f # Detected by is_file_like ) ``` -------------------------------- ### Import TestAppCKAN Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/testappckan.md Import the TestAppCKAN class from the ckanapi library. ```python from ckanapi import TestAppCKAN ``` -------------------------------- ### Equivalent CKAN Action Calls Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/action-shortcut.md Compares three equivalent ways to call a CKAN action: using the convenient action shortcut, call_action with keyword arguments, and call_action with a data_dict. ```python from ckanapi import RemoteCKAN ckan = RemoteCKAN('https://demo.ckan.org') # All of these are equivalent: # 1. Using action shortcut (most convenient) result1 = ckan.action.package_list(limit=10) # 2. Using call_action with keyword arguments result2 = ckan.call_action('package_list', {'limit': 10}) # 3. Using call_action with data_dict result3 = ckan.call_action('package_list', data_dict={'limit': 10}) ``` -------------------------------- ### Usage Patterns Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/common-functions.md Demonstrates common usage patterns for the CKAN API client, including preparing actions for requests and detecting file uploads. ```APIDOC ## Usage Patterns ### Converting between representations ```python from ckanapi.common import prepare_action, reverse_apicontroller_action import requests # Prepare action url_path, data, headers = prepare_action( 'package_show', {'id': 'my-dataset'}, apikey='key' ) # Make request base_url = 'https://demo.ckan.org/' response = requests.post( base_url + url_path, data=data, headers=headers ) # Parse response result = reverse_apicontroller_action(url_path, response.status_code, response.text) ``` ### Detecting file uploads in custom code ```python from ckanapi.common import is_file_like def process_kwargs(**kwargs): files = {} data = {} for key, value in kwargs.items(): if is_file_like(value): files[key] = value else: data[key] = value return data, files # Usage data, files = process_kwargs( name='dataset', upload=open('data.csv', 'rb') ) # data: {'name': 'dataset'} # files: {'upload': } ``` ``` -------------------------------- ### Initialize RemoteCKAN Client (Anonymous) Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/quick-reference.md Create an instance of RemoteCKAN for anonymous access to a CKAN instance. ```python # Anonymous ckan = RemoteCKAN('https://demo.ckan.org') ``` -------------------------------- ### Comparing LocalCKAN with CKAN's get_action Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/localckan.md Highlights the cleaner and safer API of LocalCKAN compared to CKAN's direct get_action function for package creation. ```python # Instead of CKAN's get_action (which can leak context between calls): from ckan.logic import get_action action = get_action('package_create') result = action({}, {'name': 'my-dataset'}) # Confusing context/data split # Use LocalCKAN (cleaner, safer): from ckanapi import LocalCKAN ckan = LocalCKAN() result = ckan.action.package_create(name='my-dataset') ``` -------------------------------- ### Common Mistake: API Key with LocalCKAN Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/quick-reference.md Do not use the 'apikey' parameter with `LocalCKAN`. Authentication is handled via username or other local mechanisms. ```python # Wrong: # ckan = LocalCKAN() # ckan.call_action('package_list', apikey='key') # Error! # Correct: ckan = LocalCKAN(username='alice') ckan.call_action('package_list') ``` -------------------------------- ### Use Action Shortcuts Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/testappckan.md Utilize action shortcuts provided by the `.action` attribute for convenient calling of CKAN actions, similar to other CKAN API clients. ```python from webtest import TestApp from ckanapi import TestAppCKAN from ckan.config.middleware import make_app test_app = TestApp(make_app({}, **test_config)) ckan = TestAppCKAN(test_app) # These are equivalent: result1 = ckan.action.package_show(id='my-dataset') result2 = ckan.call_action('package_show', {'id': 'my-dataset'}) # Works with any CKAN action groups = ckan.action.group_list() org = ckan.action.organization_create(name='test-org', title='Test Organization') ``` -------------------------------- ### TestAppCKAN Constructor Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/testappckan.md Initializes a TestAppCKAN client for testing a CKAN application. It requires a WebTest TestApp instance and optionally accepts an API key for authentication. ```APIDOC ## TestAppCKAN Constructor ### Description Creates a TestAppCKAN client for testing against a WebTest TestApp instance. ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters Table | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | test_app | webtest.TestApp | Yes | — | A WebTest TestApp instance wrapping the CKAN application under test | | apikey | str | No | None | API key for authentication. If provided, sent as 'X-CKAN-API-Key' header | ### Request Example ```python from ckanapi import TestAppCKAN from webtest import TestApp from ckan.config.middleware import make_app # Create a test app from the CKAN application test_app = TestApp(make_app({}, **test_config)) # Create TestAppCKAN with or without authentication ckan_test = TestAppCKAN(test_app) ckan_auth = TestAppCKAN(test_app, apikey='test-api-key') # Use like a normal CKAN API client result = ckan_test.action.package_list() dataset = ckan_auth.action.package_create(name='test-dataset', title='Test') ``` ``` -------------------------------- ### Run batch actions Source: https://github.com/ckan/ckanapi/blob/master/README.md Executes multiple API actions from a JSON lines file, optionally including local file uploads. ```bash $ cat update-emails.jsonl {"action":"package_patch","data":{"id":"dataset-1","maintainer_email":"new@example.com"}} {"action":"package_patch","data":{"id":"dataset-2","maintainer_email":"new@example.com"}} {"action":"package_patch","data":{"id":"dataset-3","maintainer_email":"new@example.com"}} $ ckanapi batch -I update-emails.jsonl ``` ```bash $ cat upload-files.jsonl {"action":"resource_patch","data":{"id":"408e1b1d-d0ca-50ca-9ae6-aedcee37aaa9"},"files":{"upload":"data1.csv"}} {"action":"resource_patch","data":{"id":"c1eab17f-c2d0-536d-a3f6-41a3dfe6a2c3"},"files":{"upload":"data2.csv"}} {"action":"resource_patch","data":{"id":"8ed068c2-4d4c-5f20-90db-39d2d596ce1a"},"files":{"upload":"data3.csv"}} $ ckanapi batch -I upload-files.jsonl --local-files ``` -------------------------------- ### Dump datasets with parallel processes and gzip compression Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/cli-commands.md Performs a dump of all datasets using multiple parallel processes and outputs the result compressed with gzip. Requires a CKAN configuration file. ```bash ckanapi dump datasets --all -O datasets.jsonl.gz -z -p 4 -c /etc/ckan/production.ini ``` -------------------------------- ### Initialize RemoteCKAN Client (Context Manager) Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/quick-reference.md Use RemoteCKAN as a context manager for automatic session handling. The client is automatically closed upon exiting the 'with' block. ```python # Context manager with RemoteCKAN('https://demo.ckan.org') as ckan: result = ckan.action.package_list() ``` -------------------------------- ### Import Datasets (Update Only) Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/cli-commands.md Import datasets, but only update existing records and skip any new ones. This is useful for updating existing data. ```bash ckanapi load datasets -I updates.jsonl -o -c /etc/ckan/production.ini ``` -------------------------------- ### CKAN CLI: Search and Export Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/README.md Search for datasets on a CKAN instance using the CLI and export the results to a JSON lines file. ```bash ckanapi search datasets q=government -O results.jsonl ``` -------------------------------- ### ActionShortcut Usage Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/action-shortcut.md Demonstrates how to use the ActionShortcut to call CKAN actions, including direct method calls and file uploads. ```APIDOC ## ActionShortcut Class Provides a flexible interface to call CKAN actions as methods on a CKAN client instance. **Module**: `ckanapi.common` **Import**: ```python from ckanapi.common import ActionShortcut ``` ### Constructor: ActionShortcut ```python ActionShortcut(ckan: RemoteCKAN | LocalCKAN | TestAppCKAN) ``` Creates an ActionShortcut interface for a CKAN client instance. | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | ckan | CKAN client | Yes | A RemoteCKAN, LocalCKAN, or TestAppCKAN instance | **Attributes**: - `_ckan`: CKAN client — The underlying client instance **Example**: ```python from ckanapi import RemoteCKAN from ckanapi.common import ActionShortcut ckan = RemoteCKAN('https://demo.ckan.org') action = ckan.action # Or create one manually (not typical): action = ActionShortcut(ckan) result = action.package_list() ``` ## Dynamic Method Calling ActionShortcut uses Python's `__getattr__` to dynamically create action methods. Any attribute access becomes an action call. **Syntax**: ```python ckan.action.ACTION_NAME(keyword=value, ...) ``` This is equivalent to: ```python ckan.call_action('ACTION_NAME', {'keyword': value, ...}) ``` **Example**: ```python from ckanapi import RemoteCKAN ckan = RemoteCKAN('https://demo.ckan.org') # Using action shortcut (most convenient) result1 = ckan.action.package_list(limit=10) # Using call_action with keyword arguments result2 = ckan.call_action('package_list', {'limit': 10}) # Using call_action with data_dict result3 = ckan.call_action('package_list', data_dict={'limit': 10}) ``` ## File Upload Support ActionShortcut automatically detects file-like objects and passes them separately to `call_action` as the `files` parameter. **Detection rules**: - Objects with a `read` attribute are treated as files - Tuples of length ≥2 where the second element has a `read` attribute are treated as files (requests library format) **Example**: ```python from ckanapi import RemoteCKAN ckan = RemoteCKAN('https://demo.ckan.org', apikey='my-key') # File detected automatically by the action shortcut with open('data.csv', 'rb') as f: resource = ckan.action.resource_create( package_id='my-dataset', url='dummy', # ignored but required by CKAN<2.6 upload=f # automatically detected as file, sent as multipart ) ``` This is equivalent to: ```python with open('data.csv', 'rb') as f: resource = ckan.call_action( 'resource_create', { 'package_id': 'my-dataset', 'url': 'dummy' }, files={'upload': f} ) ``` **Mixed arguments**: ```python from ckanapi import RemoteCKAN ckan = RemoteCKAN('https://demo.ckan.org', apikey='my-key') # Some arguments as files, some as data with open('image.png', 'rb') as image: org = ckan.action.organization_patch( id='my-org', title='Updated Title', image_upload=image # Detected as file ) ``` **Tuple file format (requests library compatible)**: ```python # File passed as (filename, file_object) tuple import io csv_data = io.BytesIO(b'id,name\n1,test') resource = ckan.action.resource_create( package_id='my-dataset', url='dummy', upload=('data.csv', csv_data) # Tuple with filename, file object ) ``` ## Complex Arguments ActionShortcut passes all keyword arguments as a dictionary to `call_action`. Complex values like lists and nested dicts are supported. **Example**: ```python from ckanapi import RemoteCKAN ckan = RemoteCKAN('https://demo.ckan.org') # Complex nested structure result = ckan.action.package_search( q='government', facet_field=['organization', 'groups'], # List facet_mincount=1, # Integer rows=20 ) # Facet queries with complex syntax packages = ckan.action.package_search( q='+organization:example +res_format:CSV', facet_pivot=['organization,tags'] ) ``` ## Action Names with Underscores and Plugins ActionShortcut works with all CKAN core actions and plugin-provided actions. Plugin actions may use underscores or other conventions. **Example**: ```python from ckanapi import RemoteCKAN ckan = RemoteCKAN('https://demo.ckan.org') # Core CKAN actions groups = ckan.action.group_list() org = ckan.action.organization_show(id='example') # Plugin actions (e.g., Showcase extension) if hasattr(ckan.action, 'ckanext_showcase_list'): showcases = ckan.action.ckanext_showcase_list() ``` ``` -------------------------------- ### Load organizations with logos Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/cli-commands.md Loads organizations from a JSONL file, including uploading associated logos. Requires a CKAN configuration file. ```bash ckanapi load organizations -I orgs.jsonl --upload-logo -c /etc/ckan/production.ini ``` -------------------------------- ### View a dataset Source: https://github.com/ckan/ckanapi/blob/master/README.md Retrieve dataset details using a KEY=STRING parameter. ```bash $ ckanapi action package_show id=my-dataset-name { "name": "my-dataset-name", ... } ``` -------------------------------- ### Calling Core and Plugin Actions Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/action-shortcut.md Illustrates how ActionShortcut can be used to call both standard CKAN core actions and actions provided by CKAN plugins. ```python from ckanapi import RemoteCKAN ckan = RemoteCKAN('https://demo.ckan.org') # Core CKAN actions groups = ckan.action.group_list() org = ckan.action.organization_show(id='example') # Plugin actions (e.g., Showcase extension) if hasattr(ckan.action, 'ckanext_showcase_list'): showcases = ckan.action.ckanext_showcase_list() ``` -------------------------------- ### Import LocalCKAN Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/localckan.md Import the LocalCKAN class from the ckanapi library. This is a prerequisite for using LocalCKAN. ```python from ckanapi import LocalCKAN ``` -------------------------------- ### Export All Datasets to File Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/cli-commands.md Use the 'ckanapi dump' command to export all datasets to a JSON lines file. Specify the output file and configuration. ```bash ckanapi dump datasets --all -O datasets.jsonl -c /etc/ckan/production.ini ``` -------------------------------- ### Set Both Connection and Read Timeouts via Environment Variables Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/configuration.md Configure both connection and read timeouts for HTTP requests using CKANAPI_REQUEST_TIMEOUT and CKANAPI_REQUEST_READ_TIMEOUT environment variables. The connection timeout is applied first, followed by the read timeout. ```bash export CKANAPI_REQUEST_TIMEOUT=5 # Connection timeout export CKANAPI_REQUEST_READ_TIMEOUT=30 # Read timeout python my_script.py ``` -------------------------------- ### Initialize RemoteCKAN Client Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/remoteckan.md Instantiate the RemoteCKAN client for anonymous or authenticated access to a remote CKAN instance. Supports custom user agents and session management. ```python from ckanapi import RemoteCKAN # Anonymous access demo = RemoteCKAN('https://demo.ckan.org') groups = demo.action.group_list() # Authenticated access my_ckan = RemoteCKAN( 'https://my-ckan.example.com', apikey='my-secret-api-key', user_agent='myapp/1.0 (+https://example.com)' ) dataset = my_ckan.action.package_show(id='my-dataset') # Using context manager for proper session cleanup with RemoteCKAN('https://demo.ckan.org') as ckan: result = ckan.action.package_list() ``` -------------------------------- ### Prepare Simple Action for HTTP Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/common-functions.md Prepares a simple CKAN action call without file uploads for HTTP transmission, including URL, JSON data, and headers. ```python from ckanapi.common import prepare_action # Simple action without files url, data, headers = prepare_action( 'package_show', {'id': 'my-dataset'}, apikey='my-key' ) # Returns: # url: 'api/action/package_show' # data: b'{"id": "my-dataset"}' # headers: { # 'Content-Type': 'application/json', # 'X-CKAN-API-Key': 'my-key', # 'Authorization': 'my-key' # } ``` -------------------------------- ### Common Mistake: File Upload Format Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/quick-reference.md Ensure that the 'upload' parameter in `resource_create` receives a file-like object (opened file handle), not just a file path string. ```python # Wrong: # resource = ckan.action.resource_create( # package_id='dataset-id', # upload='data.csv' # String, not file # ) # Correct: with open('data.csv', 'rb') as f: resource = ckan.action.resource_create( package_id='dataset-id', upload=f ) ``` -------------------------------- ### LocalCKAN Logging Configuration Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/localckan.md Explains how to enable logging for LocalCKAN calls by setting 'ckanapi.log_local' to True in the CKAN INI file. ```text ckanapi.log_local: True ``` -------------------------------- ### Prepare Action with File Upload for HTTP Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/common-functions.md Prepares a CKAN action call that includes a file upload for HTTP transmission, handling multipart form data and necessary headers. ```python from ckanapi.common import prepare_action # Action with file upload url, data, headers = prepare_action( 'resource_create', {'package_id': 'my-dataset', 'url': 'dummy'}, apikey='my-key', files={'upload': open('data.csv', 'rb')} ) # Returns: # url: 'api/action/resource_create' # data: { # 'package_id': b'my-dataset', # 'url': b'dummy' # } # headers: { # 'X-CKAN-API-Key': 'my-key', # 'Authorization': 'my-key' # } ``` -------------------------------- ### Import RemoteCKAN Class Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/remoteckan.md Import the RemoteCKAN class from the ckanapi library to begin using the client. ```python from ckanapi import RemoteCKAN ``` -------------------------------- ### Call CKAN Actions as Methods Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/README.md Demonstrates how to use the ActionShortcut to call CKAN API actions directly as methods on the client object. This simplifies common API interactions. ```python ckan.action.package_list() # Calls the package_list action ckan.action.organization_show(id='example') # Calls organization_show ``` -------------------------------- ### Configure RemoteCKAN with a Custom Requests Session Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/configuration.md Shows how to create and configure a custom `requests.Session` object with custom headers and retry strategies, then pass it to `RemoteCKAN`. ```python from ckanapi import RemoteCKAN import requests # Create custom session session = requests.Session() session.headers.update({'X-Custom-Header': 'value'}) # Mount custom adapters for proxies, retries, etc. from requests.adapters import HTTPAdapter from urllib3.util.retry import Retry retry_strategy = Retry( total=3, status_forcelist=[429, 500, 502, 503, 504], allowed_methods=['GET', 'POST'] ) adapter = HTTPAdapter(max_retries=retry_strategy) session.mount('https://', adapter) # Use custom session ckan = RemoteCKAN('https://demo.ckan.org', session=session) result = ckan.action.package_list() ``` -------------------------------- ### Search and export datasets to a file Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/cli-commands.md Searches for datasets matching a query and exports the results to a specified file. Uses a remote CKAN site URL. ```bash ckanapi search datasets q=government -O results.jsonl -r https://demo.ckan.org ``` -------------------------------- ### Export Datasets with Gzip Compression Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/cli-commands.md Export datasets and compress the output file using gzip. This is useful for reducing file size for large exports. ```bash ckanapi dump datasets --all -O datasets.jsonl.gz -z -r https://demo.ckan.org ``` -------------------------------- ### ActionShortcut Interface Source: https://github.com/ckan/ckanapi/blob/master/_autodocs/api-reference/testappckan.md Provides convenient shortcuts for calling CKAN actions directly as methods on the `.action` attribute of a TestAppCKAN instance. ```APIDOC ## ActionShortcut Interface ### Description The `.action` attribute on TestAppCKAN provides action shortcuts for convenient calling, just like RemoteCKAN and LocalCKAN. ### Example ```python from webtest import TestApp from ckanapi import TestAppCKAN from ckan.config.middleware import make_app test_app = TestApp(make_app({}, **test_config)) ckan = TestAppCKAN(test_app) # These are equivalent: result1 = ckan.action.package_show(id='my-dataset') result2 = ckan.call_action('package_show', {'id': 'my-dataset'}) # Works with any CKAN action groups = ckan.action.group_list() org = ckan.action.organization_create(name='test-org', title='Test Organization') ``` ```