### Install obi-auth Source: https://github.com/openbraininstitute/entitysdk/blob/main/README.md Install the obi-auth helper library for obtaining access tokens. This is a prerequisite for authenticating with the entitycore service. ```bash pip install obi-auth ``` -------------------------------- ### Install entitysdk Source: https://github.com/openbraininstitute/entitysdk/blob/main/README.md Install the entitysdk Python library using pip. Ensure you have Python 3.11 or higher. ```bash pip install entitysdk ``` -------------------------------- ### Initialize Entity SDK Client Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/05_emcell_mesh.ipynb Imports necessary libraries and initializes the Entity SDK client with authentication and project context. Ensure you have obi_auth and entitysdk installed. ```python import obi_auth from entitysdk import Client from obi_auth import get_token from obi_notebook import get_projects token = get_token(environment="staging", auth_mode="daf") project_context = get_projects.get_projects(token, env="staging") client = Client(environment="staging", project_context=project_context, token_manager=token) ``` -------------------------------- ### Import necessary libraries Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/06_analysis_notebook.ipynb Imports common libraries and specific models from the entitysdk for analysis notebook operations. Ensure these are installed in your environment. ```python import io import os from entitysdk import models from entitysdk.client import Client from entitysdk.common import ProjectContext from entitysdk.models.analysis_notebook_environment import ( DockerRuntimeInfo, OsRuntimeInfo, PythonRuntimeInfo, RuntimeInfo, ) from entitysdk.models.analysis_notebook_template import ( AnalysisNotebookTemplateInputType, AnalysisNotebookTemplateSpecifications, DockerDependency, PythonDependency, ) from entitysdk import models from entitysdk.types import AnalysisScale, AssetLabel, ContentType, EntityType from rich import print as rprint ``` -------------------------------- ### Register Initial Simulation Generation Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/04_simulation_campaign.ipynb Registers a SimulationGeneration entity with only a start time and the used campaign. It will be updated later with generated simulations and an end time. ```python sim_generation = client.register_entity( models.SimulationGeneration( start_time=datetime.now(UTC), used=[campaign], ) ) ``` -------------------------------- ### Initialize Entity SDK Client Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/03_circuit.ipynb Set up the Entity SDK client with API URL, project context, and authentication token. Ensure ACCESS_TOKEN is set in your environment. ```python import os import tempfile from pathlib import Path from rich import print as rprint from utils import create_mock_circuit_dir from entitysdk import Client, ProjectContext, models, types entitycore_api_url = "http://127.0.0.1:8000" project_context = ProjectContext( virtual_lab_id="a98b7abc-fc46-4700-9e3d-37137812c730", project_id="0dbced5f-cc3d-488a-8c7f-cfb8ea039dc6", ) token = os.getenv("ACCESS_TOKEN", "XXX") client = Client(api_url=entitycore_api_url, project_context=project_context, token_manager=token) ``` -------------------------------- ### Initialize Entity SDK Client Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/04_simulation_campaign.ipynb Sets up the Entity SDK client with API URL, project context, and authentication token. Ensure the ACCESS_TOKEN environment variable is set. ```python import os from datetime import UTC, datetime from rich import print as rprint from entitysdk import Client, ProjectContext, models entitycore_api_url = "http://127.0.0.1:8000" project_context = ProjectContext( virtual_lab_id="a98b7abc-fc46-4700-9e3d-37137812c730", project_id="0dbced5f-cc3d-488a-8c7f-cfb8ea039dc6", ) token = os.getenv("ACCESS_TOKEN", "XXX") client = Client(api_url=entitycore_api_url, project_context=project_context, token_manager=token) ``` -------------------------------- ### Initialize EntitySDK Client Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/01_searching.ipynb Sets up the EntitySDK client with API URL, project context, and authentication token. Ensure the ACCESS_TOKEN environment variable is set or provide a default. ```python import os from rich import print as rprint from entitysdk.client import Client from entitysdk.common import ProjectContext from entitysdk.models import ( MTypeClass, Organization, Person, CellMorphology, Role, Species, Strain, ) entitycore_api_url = "http://127.0.0.1:8000" project_context = ProjectContext( virtual_lab_id="a98b7abc-fc46-4700-9e3d-37137812c730", project_id="0dbced5f-cc3d-488a-8c7f-cfb8ea039dc6", ) token = os.getenv("ACCESS_TOKEN", "XXX") client = Client(api_url=entitycore_api_url, project_context=project_context, token_manager=token) # uncomment for staging # from obi_auth import get_token # token = get_token(environment="staging") # client = Client(environment="staging") ``` -------------------------------- ### Initialize EntitySDK Client Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/02_morphology.ipynb Sets up the EntitySDK client for local environment usage. Ensure the ACCESS_TOKEN environment variable is set or replace 'XXX' with a valid token. ```python import io import os import tempfile from pathlib import Path from rich import print as rprint from entitysdk import ( Client, ProjectContext, ) from entitysdk.models import ( BrainLocation, BrainRegion, CellMorphology, CellMorphologyProtocol, Contribution, MTypeClass, MTypeClassification, Organization, Role, Species, Strain, Subject, ) from entitysdk.types import ( CellMorphologyGenerationType, CellMorphologyProtocolDesign, SlicingDirectionType, ) environment = "local" project_context = ProjectContext( virtual_lab_id="a98b7abc-fc46-4700-9e3d-37137812c730", project_id="0dbced5f-cc3d-488a-8c7f-cfb8ea039dc6", ) token = os.getenv("ACCESS_TOKEN", "XXX") client = Client(environment=environment, project_context=project_context, token_manager=token) # uncomment for staging # from obi_auth import get_token # token = get_token(environment="staging") # Replace this with your vlab project url in staging # url = "https://staging.openbraininstitute.org/app/virtual-lab/594fd60d-7a38-436f-939d-500feaa13bba/ff89ca07-6613-4922-9ab0-2637221db8b5" # client = Client.from_vlab(url, token_manager=token) ``` -------------------------------- ### Initialize project context and token Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/06_analysis_notebook.ipynb Sets up the API URL, project context (virtual lab and project IDs), and retrieves an access token. The ACCESS_TOKEN environment variable is used if available, otherwise 'XXX' is used as a fallback. ```python entitycore_api_url = "http://127.0.0.1:8000" project_context = ProjectContext( virtual_lab_id="a98b7abc-fc46-4700-9e3d-37137812c730", project_id="0dbced5f-cc3d-488a-8c7f-cfb8ea039dc6", ) token = os.getenv("ACCESS_TOKEN", "XXX") ``` -------------------------------- ### Initialize EntitySDK Client Source: https://github.com/openbraininstitute/entitysdk/blob/main/README.md Initialize the entitysdk client with project context, environment, and an access token. The project context requires project and virtual lab IDs. ```python from uuid import UUID from entitysdk import Client, ProjectContext, models # Initialize client client = Client( project_context=ProjectContext( project_id=UUID("your-project-id"), virtual_lab_id=UUID("your-lab-id") ), environment="staging", token_manager=token ) ``` -------------------------------- ### Initialize the Client object Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/06_analysis_notebook.ipynb Creates an instance of the Client class using the API URL, project context, and token manager. This client is used to interact with the entitysdk. ```python client = Client(api_url=entitycore_api_url, project_context=project_context, token_manager=token) ``` -------------------------------- ### Clone Repository and Run Tests Source: https://github.com/openbraininstitute/entitysdk/blob/main/README.md Clone the entitysdk repository and run tests using tox. Tox manages linting, testing, and packaging checks. ```bash # Clone the repository git clone https://github.com/your-org/entitysdk.git # Run linting, tests, and check-packaging tox ``` -------------------------------- ### Configure Project Context Source: https://github.com/openbraininstitute/entitysdk/blob/main/README.md Create a ProjectContext object with valid project and virtual lab UUIDs. This is required for initializing the entitysdk client. ```python from uuid import UUID from entitysdk import ProjectContext project_context = ProjectContext( project_id=UUID("12345678-1234-1234-1234-123456789012"), virtual_lab_id=UUID("87654321-4321-4321-4321-210987654321") ) ``` -------------------------------- ### Upload Circuit Directory Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/03_circuit.ipynb Upload a directory of files associated with a registered circuit. The files are organized relative to a temporary directory. Ensure `create_mock_circuit_dir` is available. ```python with tempfile.TemporaryDirectory() as tdir: # create a hierarchy of files to upload create_mock_circuit_dir(tdir) files = {str(path.relative_to(tdir)): path for path in Path(tdir).rglob("*") if path.is_file()} directory_asset = client.upload_directory( entity_id=registered_circuit.id, entity_type=models.Circuit, name="circuit", label=types.AssetLabel.sonata_circuit, paths=files, ) ``` -------------------------------- ### Register Simulation Campaign Gradually Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/04_simulation_campaign.ipynb Registers a SimulationCampaign entity as the first step in gradual provenance registration. This campaign will be updated later. ```python campaign = client.register_entity( models.SimulationCampaign( name="my-campaign", description="my-campaign-description", entity_id=circuit.id, scan_parameters={"foo": "bar"}, ) ) ``` -------------------------------- ### Register Analysis Notebook Environment Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/06_analysis_notebook.ipynb Registers a new analysis notebook environment with detailed runtime information. Uploads a requirements.txt file as an asset to the registered environment. ```python environment = models.AnalysisNotebookEnvironment( authorized_public=False, runtime_info=RuntimeInfo( schema_version=1, python=PythonRuntimeInfo( version="3.9.21", # platform.python_version() implementation="CPython", # platform.python_implementation() executable="/usr/bin/python", # sys.executable ), docker=DockerRuntimeInfo( image_repository="openbraininstitute/obi-notebook-image", image_tag="2025.09.24-2", image_digest="3406990b6e4c7192317b6fdc5680498744f6142f01f0287f4ee0420d8c74063c", docker_version="28.4.0", ), os=OsRuntimeInfo( system="Linux", # platform.system() release="5.14.0-427.28.1.el9_4.x86_64", # platform.release() version="#1 SMP PREEMPT_DYNAMIC Fri Aug 2 03:44:10 EDT 2024", # platform.version() machine="x86_64", # platform.machine() processor="x86_64", # platform.processor() ), ), ) environment = client.register_entity(environment) # use an in-memory buffer to upload the asset # or use client.upload_file() alternatively buffer = io.BytesIO(b"rich==14.1.0") asset_requirements = client.upload_content( entity_id=environment.id, entity_type=models.AnalysisNotebookEnvironment, file_content=buffer, file_name="requirements.txt", file_content_type=ContentType.text_plain, asset_label=AssetLabel.requirements, ) rprint(asset_requirements) ``` -------------------------------- ### Create and register an AnalysisNotebookTemplate Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/06_analysis_notebook.ipynb Defines and registers a new analysis notebook template with specified Python and Docker dependencies, and input specifications. This is required if the template does not already exist. ```python template = models.AnalysisNotebookTemplate( authorized_public=False, name="Skeletonize cell mesh and extract spines", description="In this notebook...", scale=AnalysisScale.cellular, specifications=AnalysisNotebookTemplateSpecifications( schema_version=1, python=PythonDependency( version=">=3.10,<3.13", ), docker=DockerDependency( image_repository="openbraininstitute/obi-notebook-image", image_tag=">=2025.09.24-2", image_digest="3406990b6e4c7192317b6fdc5680498744f6142f01f0287f4ee0420d8c74063c", docker_version=">=20.10", ), inputs=[ AnalysisNotebookTemplateInputType( name="mesh_ids", entity_type=EntityType.em_cell_mesh, is_list=True, count_min=1, count_max=3, ) ], ), ) template = client.register_entity(template) rprint(template) ``` -------------------------------- ### List Circuit Directory Contents Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/03_circuit.ipynb List the files within an uploaded directory asset for a circuit. Requires the entity ID, type, and the asset ID of the uploaded directory. ```python files = client.list_directory( entity_id=registered_circuit.id, entity_type=models.Circuit, asset_id=directory_asset.id ) ``` -------------------------------- ### Download Circuit Directory Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/03_circuit.ipynb Download the contents of an uploaded directory asset for a circuit to a specified output path. The files are then listed from the download location. ```python with tempfile.TemporaryDirectory() as tdir: client.download_directory( entity_id=registered_circuit.id, entity_type=models.Circuit, asset_id=directory_asset.id, output_path=Path(tdir), ) files = {str(path.relative_to(tdir)): path for path in Path(tdir).rglob("*") if path.is_file()} rprint(files) ``` -------------------------------- ### Print Listed Directory Files Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/03_circuit.ipynb Display the list of files obtained from the circuit directory using rich print. ```python rprint(files) ``` -------------------------------- ### Upload notebook asset Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/06_analysis_notebook.ipynb Uploads the notebook file (.ipynb) as an asset to the registered template. An in-memory buffer is used here, but client.upload_file() can also be used. ```python # use an in-memory buffer to upload the asset, # or use client.upload_file() alternatively buffer = io.BytesIO(b"...") asset_ipynb = client.upload_content( entity_id=template.id, entity_type=models.AnalysisNotebookTemplate, file_content=buffer, file_name="skeletonize_cell_mesh_and_extract_spines.ipynb", file_content_type=ContentType.application_x_ipynb_json, asset_label=AssetLabel.jupyter_notebook, ) rprint(asset_ipynb) ``` -------------------------------- ### Download and Read Cell Morphology Asset Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/02_morphology.ipynb Fetches a CellMorphology entity, downloads its SWC asset, and then reads the content of an H5 file. Ensure the SWC asset is downloaded to './my-file.swc' and the H5 file is accessible. ```python downloaded = client.download_assets( fetched, selection={"content_type": "application/swc"}, output_path="./my-file.swc", ).one() content = client.download_content( entity_id=fetched.id, entity_type=type(fetched), asset_id=downloaded.asset.id ) print(content) print(Path("my-file.h5").read_text()) ``` -------------------------------- ### Obtain Access Token Source: https://github.com/openbraininstitute/entitysdk/blob/main/README.md Retrieve a valid access token using the obi-auth library. Specify the environment, e.g., 'staging'. ```python from obi_auth import get_token token = get_token(environment="staging") ``` -------------------------------- ### Register Simulations Gradually Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/04_simulation_campaign.ipynb Registers multiple Simulation entities, linking them to the previously registered campaign. These simulations are part of a gradual provenance registration process. ```python simulations = [ client.register_entity( models.Simulation( name=f"sim-{i}", description=f"sim-{i}", scan_parameters={"foo": "bar"}, number_neurons=5, entity_id=circuit.id, simulation_campaign_id=campaign.id, ) ) for i in range(5) ] ``` -------------------------------- ### Print Role Count and First Two Entries Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/01_searching.ipynb Prints the total number of roles found and the first two role entries. Useful for inspecting a subset of the data. ```python rprint(len(all_roles)) rprint(all_roles[:2]) ``` -------------------------------- ### Checkout main branch Source: https://github.com/openbraininstitute/entitysdk/blob/main/CONTRIBUTING.md Switch to the main branch locally after your changes have been merged. ```shell git checkout main -f ``` -------------------------------- ### Create and Register Subject Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/02_morphology.ipynb Creates a new Subject entity. It's recommended to reuse existing subjects if possible. The registered subject object is returned. ```python subject = Subject( name="my-subject-for-morphology", description="my-subject-description", sex="male", species=species, strain=strain, ) subject = client.register_entity(subject) rprint(subject) ``` -------------------------------- ### Print Updated Simulation Generation Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/04_simulation_campaign.ipynb Prints the updated SimulationGeneration entity to the console after adding generated simulations and end time. ```python rprint(updated_sim_generation) ``` -------------------------------- ### Register a Subject Entity Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/03_circuit.ipynb Search for a species and then create and register a new subject entity. The species must exist in the system. ```python species = client.search_entity( entity_type=models.Species, query={"name__ilike": "Mus musculus"} ).first() subject = models.Subject( name="my-subject", description="my-subject-description", sex="male", species=species ) subject = client.register_entity(subject) ``` -------------------------------- ### Create and Register Morphology Protocol Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/02_morphology.ipynb Creates a new CellMorphologyProtocol. Reusing existing protocols is recommended. The registered protocol object is returned. ```python morphology_protocol = CellMorphologyProtocol( generation_type=CellMorphologyGenerationType.digital_reconstruction, protocol_document="https://example.com/", protocol_design=CellMorphologyProtocolDesign.cell_patch, slicing_thickness=20.0, slicing_direction=SlicingDirectionType.horizontal, name="my_name", description="my_description", ) rprint(morphology_protocol.__class__.__name__) morphology_protocol = client.register_entity(morphology_protocol) rprint(morphology_protocol) ``` -------------------------------- ### Display First Search Result Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/02_morphology.ipynb Prints the first hit from a previous search query. This is a simple way to inspect the details of a single search result. ```python rprint(hits[0]) ``` -------------------------------- ### Upload requirements asset Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/06_analysis_notebook.ipynb Uploads the requirements.txt file as an asset to the registered template. This specifies the Python dependencies for the notebook. An in-memory buffer is used, but client.upload_file() is an alternative. ```python # use an in-memory buffer to upload the asset # or use client.upload_file() alternatively buffer = io.BytesIO(b"rich==14.1.0") asset_requirements = client.upload_content( entity_id=template.id, entity_type=models.AnalysisNotebookTemplate, file_content=buffer, file_name="requirements.txt", file_content_type=ContentType.text_plain, asset_label=AssetLabel.requirements, ) rprint(asset_requirements) ``` -------------------------------- ### Register Simulation Generation and Executions Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/04_simulation_campaign.ipynb Registers a SimulationGeneration entity linking to the campaign, and multiple SimulationExecution entities linking simulations to results. This is part of registering provenance at once. ```python simulation_generation = client.register_entity( models.SimulationGeneration( start_time=datetime.now(UTC), used=[campaign], generated=simulations, ) ) simulation_executions = [ client.register_entity( models.SimulationExecution( used=[simulations[i]], generated=[simulation_results[i]], start_time=datetime.now(UTC), status="done", ) ) for i in range(5) ] ``` -------------------------------- ### Search Analysis Notebook Templates Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/06_analysis_notebook.ipynb Searches for and retrieves all registered analysis notebook templates. Iterates through the results and prints each template. ```python templates = client.search_entity(entity_type=models.AnalysisNotebookTemplate) for t in templates: rprint(t) ``` -------------------------------- ### Print Registered Provenance Data Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/04_simulation_campaign.ipynb Prints the registered SimulationGeneration and SimulationExecution entities to the console using rich print for better formatting. ```python rprint(simulation_generation) rprint(simulation_executions) ``` -------------------------------- ### Print Organization Count and First Two Entries Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/01_searching.ipynb Prints the total number of organizations found and the first two organization entries. Useful for inspecting a subset of the data. ```python rprint(len(all_orgs)) rprint(all_orgs[:2]) ``` -------------------------------- ### Register Analysis Notebook Execution Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/06_analysis_notebook.ipynb Registers an analysis notebook execution, linking it to a template, environment, and generated results. The 'used' field can list input entities. ```python execution = models.AnalysisNotebookExecution( authorized_public=False, start_time="2025-11-03T08:40:59.794317Z", end_time="2025-11-03T08:45:00.000000Z", used= [], # add the list of input entities, if any generated= [result], analysis_notebook_template=template, analysis_notebook_environment=environment, ) execution = client.register_entity(execution) rprint(execution) ``` -------------------------------- ### Generate Server Schemas Source: https://github.com/openbraininstitute/entitysdk/blob/main/README.md Update server schemas in src/entitysdk/_server_schemas.py using the tox command. This is used for importing enum types. ```bash tox -e generate-server-schemas ``` -------------------------------- ### Create and Register EMDenseReconstructionDataset Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/05_emcell_mesh.ipynb Creates an EMDenseReconstructionDataset entity using provided brain region and subject information, then registers it with the client. This requires pre-existing brain_region and subject entities. ```python from entitysdk.models import EMDenseReconstructionDataset,BrainRegion,Subject brain_region = client.search_entity(entity_type=BrainRegion, query={"annotation_value": 68}).one() subject = client.search_entity(entity_type=Subject).first() em_dense_dataset = EMDenseReconstructionDataset( brain_region= brain_region, subject=subject, **{ 'name': 'this is an example', 'description': 'this is a test dataset', 'fixation': None, 'staining_type': None, 'slicing_thickness': 40.0, 'microscope_type': None, 'detector': None, 'slicing_direction': None, 'landmarks': None, 'voltage': None, 'current': None, 'dose': None, 'temperature': None, 'volume_resolution_x_nm': 4.0, 'volume_resolution_y_nm': 4.0, 'volume_resolution_z_nm': 40.0, 'release_url': 'http://microns-explorer.org', 'cave_client_url': 'https://global.daf-apis.com', 'cave_datastack': 'minnie65_public', 'precomputed_mesh_url': 'precomputed://gs://iarpa_microns/minnie/minnie65/seg_m1300/', 'cell_identifying_property': 'pt_root_id'}) em_dense_reconstruction = client.register_entity(entity=em_dense_dataset, project_context=project_context) ``` -------------------------------- ### Upload Morphology Assets Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/02_morphology.ipynb Demonstrates uploading morphology data files associated with a registered CellMorphology entity. Supports both file paths and in-memory buffers. ```python with tempfile.TemporaryDirectory() as tdir: file1 = Path(tdir, "morph.h5") file1.write_text("h5") file2 = Path(tdir, "morph.swc") file2.write_text("swc") # use a filepath to register first asset asset1 = client.upload_file( entity_id=registered.id, entity_type=CellMorphology, file_path=file1, file_content_type="application/x-hdf5", asset_label="morphology", ) rprint(asset1) # use an in-memory buffer to upload second asset buffer = io.BytesIO(b"morph bytes buffer") asset2 = client.upload_content( entity_id=registered.id, entity_type=CellMorphology, file_content=buffer, file_name="buffer.swc", file_content_type="application/swc", asset_label="morphology", ) rprint(asset2) ``` -------------------------------- ### Update Simulation Generation with Generated Simulations Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/04_simulation_campaign.ipynb Updates an existing SimulationGeneration entity to include the IDs of the generated simulations and set an end time. This demonstrates gradual provenance updates. ```python data_to_update = { "generated_ids": [s.id for s in simulations], "end_time": datetime.now(UTC), } updated_sim_generation = client.update_entity( entity_id=sim_generation.id, entity_type=type(sim_generation), attrs_or_entity=data_to_update ) ``` -------------------------------- ### Update local main with upstream changes Source: https://github.com/openbraininstitute/entitysdk/blob/main/CONTRIBUTING.md Synchronize your local main branch with the latest changes from the upstream repository. ```shell git pull --ff upstream main ``` -------------------------------- ### Print Updated Simulation Executions Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/04_simulation_campaign.ipynb Prints the updated SimulationExecution entities to the console after associating them with results and setting their final status. ```python rprint(updated_executions) ``` -------------------------------- ### Push branch to GitHub Source: https://github.com/openbraininstitute/entitysdk/blob/main/CONTRIBUTING.md Push your local branch to your GitHub fork to make it available for a pull request. ```shell git push origin my-fix-branch ``` -------------------------------- ### Register External Directory as Asset Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/02_morphology.ipynb Registers an existing directory from AWS S3 as an asset for a Circuit entity. This is useful for linking entire directories of data, such as simulation outputs or datasets, to an entity. ```python asset = client.register_asset( entity_id=registered.id, entity_type=Circuit, name="sonata_circuit", storage_path="path/to/circuit/directory", storage_type="aws_s3_open", is_directory=True, content_type="application/vnd.directory", asset_label="sonata_circuit", ) ``` -------------------------------- ### Create a new git branch Source: https://github.com/openbraininstitute/entitysdk/blob/main/CONTRIBUTING.md Before making changes, create a new git branch from the main branch to isolate your work. ```shell git checkout -b my-fix-branch main ``` -------------------------------- ### Print Fetched Circuit Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/03_circuit.ipynb Display the details of the fetched circuit entity using rich print for enhanced readability. ```python rprint(fetched) ``` -------------------------------- ### Print MTypeClass Count and First Two Entries Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/01_searching.ipynb Prints the total number of MTypeClasses found and the first two entries. Useful for inspecting a subset of the data. ```python rprint(len(all_mtypes)) rprint(all_mtypes[:2]) ``` -------------------------------- ### Upload Asset File Source: https://github.com/openbraininstitute/entitysdk/blob/main/README.md Upload a file as an asset associated with an entity. Specify the entity ID, type, file path, and content type. ```python # Upload an asset client.upload_file( entity_id=morphology.id, entity_type=models.CellMorphology, file_path="path/to/file.swc", file_content_type="application/swc", ) ``` -------------------------------- ### Create and Register EMCellMesh Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/05_emcell_mesh.ipynb Creates an EMCellMesh entity using specified generation method, mesh type, and associated dataset, brain region, and subject. It then registers the mesh with the client. ```python from entitysdk.models import EMCellMesh, EMCellMeshGenerationMethod, EMCellMeshType em_mesh = EMCellMesh( generation_method=EMCellMeshGenerationMethod.marching_cubes, mesh_type=EMCellMeshType.dynamic, brain_region=brain_region, subject=subject, em_dense_reconstruction_dataset=em_dense_reconstruction, **{"release_version": 1512, "dense_reconstruction_cell_id": 12, "level_of_detail": 1, }) em_mesh_registered = client.register_entity(entity=em_mesh, project_context=project_context) ``` -------------------------------- ### Print Person Count and First Two Entries Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/01_searching.ipynb Prints the total number of persons found and the first two person entries. Useful for inspecting a subset of the data. ```python rprint(len(all_users)) rprint(all_users[:2]) ``` -------------------------------- ### Commit changes Source: https://github.com/openbraininstitute/entitysdk/blob/main/CONTRIBUTING.md Stage and commit your changes using a descriptive message. The '-a' flag stages all tracked, modified files. ```shell git commit -a ``` -------------------------------- ### Register the Circuit Entity Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/03_circuit.ipynb Register the defined circuit entity with the Entity SDK. This makes the circuit available for further operations. ```python registered_circuit = client.register_entity(circuit) ``` -------------------------------- ### Search All Roles Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/01_searching.ipynb Retrieves all entities of type Role. This is a basic search operation. ```python all_roles = client.search_entity(entity_type=Role).all() ``` -------------------------------- ### List Morphology Protocols Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/02_morphology.ipynb Retrieves all registered CellMorphologyProtocol entities. Useful for checking existing protocols. ```python protocols = client.search_entity(entity_type=CellMorphologyProtocol).all() rprint(protocols) ``` -------------------------------- ### Create a Circuit Entity Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/03_circuit.ipynb Define a circuit entity with details such as name, description, subject, brain region, and synapse counts. Requires a pre-registered subject and a valid brain region. ```python brain_region = client.search_entity(entity_type=models.BrainRegion, query={"acronym": "CB"}).first() circuit = models.Circuit( name="my-circuit", description="my-circuit", subject=subject, brain_region=brain_region, number_synapses=2, number_neurons=5, number_connections=10, scale="microcircuit", build_category="em_reconstruction", ) ``` -------------------------------- ### Register Simulation Executions Gradually Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/04_simulation_campaign.ipynb Registers multiple SimulationExecution entities, linking them to specific simulations. These executions are created before their results are finalized. ```python executions = [ client.register_entity( models.SimulationExecution( used=[simulations[i]], start_time=datetime.now(UTC), status="created", ) ) for i in range(5) ] ``` -------------------------------- ### Register Simulation Campaign and Related Entities Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/04_simulation_campaign.ipynb Registers a SimulationCampaign, multiple Simulations, and their corresponding SimulationResults. This is done when all entities are available upfront. ```python campaign = client.register_entity( models.SimulationCampaign( name="my-campaign", description="my-campaign-description", entity_id=circuit.id, scan_parameters={"foo": "bar"}, ) ) simulations = [ client.register_entity( models.Simulation( name=f"sim-{i}", description=f"sim-{i}", scan_parameters={"foo": "bar"}, number_neurons=5, entity_id=circuit.id, simulation_campaign_id=campaign.id, ) ) for i in range(5) ] simulation_results = [ client.register_entity( models.SimulationResult( name=f"result-{i}", description=f"result-{i}", simulation_id=simulations[i].id, ) ) for i in range(5) ] ``` -------------------------------- ### Print Strain Count and First Two Entries Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/01_searching.ipynb Prints the total number of strains found and the first two strain entries. Useful for inspecting a subset of the data. ```python rprint(len(all_strains)) rprint(all_strains[:2]) ``` -------------------------------- ### Register Analysis Notebook Result Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/06_analysis_notebook.ipynb Creates and registers a new analysis notebook result entity. Uploads a Jupyter notebook file as an asset associated with the result. ```python result = models.AnalysisNotebookResult( authorized_public=False, name="Test notebook name", description="Test notebook result", ) result = client.register_entity(result) rprint(result) # use an in-memory buffer to upload the asset, # or use client.upload_file() alternatively buffer = io.BytesIO(b"...") asset_ipynb = client.upload_content( entity_id=result.id, entity_type=models.AnalysisNotebookResult, file_content=buffer, file_name="skeletonize_cell_mesh_and_extract_spines_result.ipynb", file_content_type=ContentType.application_x_ipynb_json, asset_label=AssetLabel.jupyter_notebook, ) rprint(asset_ipynb) ``` -------------------------------- ### Register Simulation Results Gradually Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/04_simulation_campaign.ipynb Registers multiple SimulationResult entities, linking them to their respective simulations. These results are created to be associated with the simulation executions. ```python results = [ client.register_entity( models.SimulationResult( name=f"result-{i}", description=f"result-{i}", simulation_id=simulations[i].id, ) ) for i in range(5) ] ``` -------------------------------- ### Rebase and force push updates Source: https://github.com/openbraininstitute/entitysdk/blob/main/CONTRIBUTING.md If changes are requested, rebase your branch onto the main branch and force push to update your pull request. ```shell git rebase main -i git push -f ``` -------------------------------- ### Create Cell Morphology Object Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/02_morphology.ipynb Instantiates a CellMorphology object with associated subject, brain region, and location. This object is then registered. ```python brain_location = BrainLocation( x=4101.52490234375, y=1173.8499755859375, z=4744.60009765625, ) morphology = CellMorphology( cell_morphology_protocol=morphology_protocol, name="my-morph", description="A morphology", subject=subject, brain_region=brain_region, location=brain_location, legacy_id=None, authorized_public=False, ) rprint(morphology) registered = client.register_entity(entity=morphology) rprint(registered) ``` -------------------------------- ### Fetch Circuit Entity Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/03_circuit.ipynb Retrieve a specific circuit entity from the system using its ID and type. This fetches the latest version of the entity. ```python fetched = client.get_entity(entity_id=registered_circuit.id, entity_type=models.Circuit) ``` -------------------------------- ### Print Morphology Count and First Two Entries Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/01_searching.ipynb Prints the total number of morphologies found and the first two entries. Useful for inspecting a subset of the data. ```python rprint(len(morphs)) rprint(morphs[:2]) ``` -------------------------------- ### Update Simulation Executions with Results Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/04_simulation_campaign.ipynb Updates existing SimulationExecution entities to include the IDs of their generated results, set an end time, and update their status to 'done'. ```python updated_executions = [ client.update_entity( entity_id=executions[i].id, entity_type=models.SimulationExecution, attrs_or_entity={ "generated_ids": [results[i].id], "end_time": datetime.now(UTC), "status": "done", }, ) for i in range(5) ] ``` -------------------------------- ### Search All Organizations Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/01_searching.ipynb Retrieves all entities of type Organization. This is a basic search operation. ```python all_orgs = client.search_entity(entity_type=Organization).all() ``` -------------------------------- ### Print Species Count and Data Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/01_searching.ipynb Prints the total number of species found and the species data itself. Useful for verifying search results. ```python rprint(len(all_species)) rprint(all_species) ``` -------------------------------- ### Find Morphologies by MType Label Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/01_searching.ipynb Searches for CellMorphology entities where the 'mtype__pref_label' field matches 'SR_PC'. This demonstrates searching with a specific query. ```python morphs = client.search_entity( entity_type=CellMorphology, query={"mtype__pref_label": "SR_PC"} ).all() ``` -------------------------------- ### Search All MTypeClasses Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/01_searching.ipynb Retrieves all entities of type MTypeClass. This is a basic search operation. ```python all_mtypes = client.search_entity(entity_type=MTypeClass).all() ``` -------------------------------- ### Search All Species Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/01_searching.ipynb Retrieves all entities of type Species. This is a basic search operation. ```python all_species = client.search_entity(entity_type=Species).all() ``` -------------------------------- ### Upload Cell Surface Mesh File Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/05_emcell_mesh.ipynb Uploads an .obj file as an asset associated with a registered EMCellMesh. This requires the ID of the registered mesh and the local path to the .obj file. ```python asset_obj = client.upload_file( entity_id=em_mesh_registered.id, entity_type=EMCellMesh, file_path="90.obj", file_content_type="application/obj", asset_label="cell_surface_mesh", project_context=project_context, ) ``` -------------------------------- ### Search All Persons Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/01_searching.ipynb Retrieves all entities of type Person. This is a basic search operation. ```python all_users = client.search_entity(entity_type=Person).all() ``` -------------------------------- ### Search for EMDenseReconstructionDataset Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/05_emcell_mesh.ipynb Searches for an EMDenseReconstructionDataset by its name. This is useful for retrieving an existing dataset to use in further operations. ```python em_dense_reconstruction = client.search_entity(entity_type=EMDenseReconstructionDataset, query={'name':'this is an example'} ).one() ``` -------------------------------- ### Print Registered Circuit Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/03_circuit.ipynb Display the details of the registered circuit entity using rich print for better formatting. ```python rprint(registered_circuit) ``` -------------------------------- ### Register Subject and Circuit Entities Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/04_simulation_campaign.ipynb Registers a new Subject and a Circuit entity. This involves searching for existing Species and BrainRegion entities first. The Circuit entity requires details like number of synapses and neurons. ```python species = client.search_entity( entity_type=models.Species, query={"name__ilike": "Homo sapiens"} ).one() subject = models.Subject( name="my-subject", description="my-subject-description", sex="male", species=species ) subject = client.register_entity(subject) brain_region = client.search_entity(entity_type=models.BrainRegion, query={"acronym": "CB"}).first() circuit = models.Circuit( name="my-circuit", description="my-circuit", subject=subject, brain_region=brain_region, number_synapses=2, number_neurons=5, number_connections=10, scale="microcircuit", build_category="em_reconstruction", ) circuit = client.register_entity(circuit) ``` -------------------------------- ### Search All Strains Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/01_searching.ipynb Retrieves all entities of type Strain. This is a basic search operation. ```python all_strains = client.search_entity(entity_type=Strain).all() ``` -------------------------------- ### Search for Cell Morphology Entities Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/02_morphology.ipynb Searches for CellMorphology entities with a name similar to 'my-morph', paginating results. This is useful for finding specific entities based on partial name matches. ```python hits = client.search_entity( entity_type=CellMorphology, query={"name__ilike": "my-morph", "page": 1, "page_size": 2}, limit=None, ).all() print("Number of results: ", len(hits)) ``` -------------------------------- ### Search Cell Morphologies with Limit Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/01_searching.ipynb Retrieves up to 10 entities of type CellMorphology. The limit parameter controls the maximum number of results. ```python morphs = client.search_entity(entity_type=CellMorphology, limit=10).all() ``` -------------------------------- ### Search for Strain Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/02_morphology.ipynb Searches for a 'Cux2-CreERT2' strain entity. Ensure the strain exists in the database. ```python strain = client.search_entity(entity_type=Strain, query={"name": "Cux2-CreERT2"}).one() ``` ```python rprint(strain) ``` -------------------------------- ### Update JSON Payloads Source: https://github.com/openbraininstitute/entitysdk/blob/main/README.md Automatically update JSON payloads in tests/unit/models/data/extracted from entitycore. This command checks out entitycore, runs tests, and extracts traces. ```bash tox -e update-traces ``` -------------------------------- ### Search for EMCellMesh Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/05_emcell_mesh.ipynb Searches for an EMCellMesh entity based on its release version. This allows retrieval of a previously registered cell mesh. ```python em_mesh_registered = client.search_entity(entity_type=EMCellMesh, query={'release_version':1512} ).one() ``` -------------------------------- ### Add Contribution to Morphology Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/02_morphology.ipynb Registers a Contribution linking an Organization and Role to the CellMorphology entity. Requires searching for an Organization and Role first. ```python agent = client.search_entity(entity_type=Organization, limit=1).one() role = client.search_entity(entity_type=Role, limit=1).one() contribution = Contribution( agent=agent, role=role, entity=registered, ) contribution = client.register_entity(contribution) rprint(contribution) ``` -------------------------------- ### Register External HDF5 File as Asset Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/02_morphology.ipynb Registers an existing HDF5 file from AWS S3 as an asset for a CellMorphology entity. This is used when you have data stored externally and want to link it to an entity in the SDK. ```python asset = client.register_asset( entity_id=registered.id, entity_type=CellMorphology, name="my-morphology.h5", storage_path="path/to/morph.h5", storage_type="aws_s3_open", is_directory=False, content_type="application/x-hdf5", asset_label="morphology", ) ``` -------------------------------- ### Search for Species Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/02_morphology.ipynb Searches for a 'Mus musculus' species entity. Ensure the species exists in the database. ```python species = client.search_entity(entity_type=Species, query={"name": "Mus musculus"}, limit=10).one() ``` ```python rprint(species) ``` -------------------------------- ### Search for Brain Region Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/02_morphology.ipynb Searches for a brain region by its annotation value. Ensure the brain region exists. ```python brain_region = client.search_entity(entity_type=BrainRegion, query={"annotation_value": 68}).one() ``` ```python rprint(brain_region) ``` -------------------------------- ### Search for Cell Morphologies Source: https://github.com/openbraininstitute/entitysdk/blob/main/README.md Search for entities of type CellMorphology using a query. The iterator can be used to retrieve results, with a limit on the number of results. ```python # Search for morphologies iterator = client.search_entity( entity_type=models.CellMorphology, query={"mtype__pref_label": "L5_TPC:A"}, limit=1, ) morphology = next(iterator) ``` -------------------------------- ### Add MType Classification to Morphology Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/02_morphology.ipynb Associates an MTypeClass with a registered CellMorphology entity by creating an MTypeClassification. Requires searching for the MTypeClass first. ```python mtype = client.search_entity(entity_type=MTypeClass, query={"pref_label": "L5_PC"}).one() mtype_classification = MTypeClassification( mtype_class_id=mtype.id, entity_id=registered.id, authorized_public=True ) mtype_classification = client.register_entity(mtype_classification) ``` -------------------------------- ### Delete SWC Asset from Cell Morphology Entity Source: https://github.com/openbraininstitute/entitysdk/blob/main/examples/02_morphology.ipynb Iterates through the assets of a fetched CellMorphology entity, finds the SWC asset, deletes it, and then re-fetches the entity to show the updated assets. This is useful for cleaning up or managing entity assets. ```python for asset in fetched.assets: if asset.content_type == "application/swc": print("Deleting asset ", asset.id) deleted_asset = client.delete_asset( entity_id=fetched.id, entity_type=type(registered), asset_id=asset.id, ) break rprint(deleted_asset) fetched = client.get_entity(entity_id=registered.id, entity_type=CellMorphology) rprint(fetched.assets) ``` -------------------------------- ### Delete local branch Source: https://github.com/openbraininstitute/entitysdk/blob/main/CONTRIBUTING.md Remove the local feature branch after it has been merged and deleted from the remote. ```shell git branch -D my-fix-branch ``` -------------------------------- ### Delete remote branch on GitHub Source: https://github.com/openbraininstitute/entitysdk/blob/main/CONTRIBUTING.md After your pull request is merged, you can delete the remote branch from GitHub. ```shell git push origin --delete my-fix-branch ```