### Setup Synapse Project Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/tutorials/python/storage_location.md Initializes the Synapse client and retrieves a project. Ensure you have completed installation and authentication setup. ```python import synapseclient from synapseclient.core.storage_locations import StorageLocationType syn = synapseclient.Synapse() syn.login() # Replace with your project ID project_id = "syn12345" project = syn.get(project_id) ``` -------------------------------- ### Setup Project and Folder for Migration Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/tutorials/python/migration.md Initializes the Synapse client and retrieves a project and folder. Ensure you have completed installation and authentication. ```python import synapseclient from synapseclient.core.config import configs from synapseclient.models import Project, Folder syn = synapseclient.Synapse() syn.login(configs.get("username"), configs.get("password")) # Get a project and folder project = syn.get("syn12345") # Replace with your project ID folder = syn.get("syn67890") # Replace with your folder ID assert isinstance(project, Project) assert isinstance(folder, Folder) ``` -------------------------------- ### Build and Serve Documentation Locally Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/CLAUDE.md Installs documentation dependencies and starts a local web server to preview the MkDocs documentation. Changes to source files will trigger live reloads. ```bash pip install -e ".[docs]" && mkdocs serve ``` -------------------------------- ### Setup Project for Proxy Storage Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/tutorials/python/proxy_storage_location.md Sets up the project and retrieves necessary information for subsequent operations. Ensure you have completed installation and authentication prerequisites. ```python import synapseclient from synapseclient.models import StorageLocation, StorageLocationType syn = synapseclient.Synapse() syn.login() # Get a project to use for this tutorial project = syn.get_project("syn123456") ``` -------------------------------- ### Local Synapse MCP Server Installation Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/guides/synapse_mcp.md Steps to install and run the Synapse MCP server locally for air-gapped environments or development. Requires cloning the repository, installing dependencies, setting a Synapse Personal Access Token (PAT), and starting the server. ```bash git clone https://github.com/Sage-Bionetworks/synapse-mcp.git cd synapse-mcp pip install -e . export SYNAPSE_PAT="your_personal_access_token" synapse-mcp ``` -------------------------------- ### Install Synapse Client from Source Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/README.md Install the Synapse Python Client by cloning the source code repository and using pip. ```bash git clone git://github.com/Sage-Bionetworks/synapsePythonClient.git cd synapsePythonClient pip install . ``` -------------------------------- ### Install Synapse Client from Source Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/tutorials/installation.md Install the synapseclient package from the local source code using pip. Use '-e' for an editable installation. ```bash pip install . ``` ```bash pip install -e . ``` -------------------------------- ### Complete Example Script for Metadata Curation Workflows Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/guides/extensions/curator/metadata_curation.md This script demonstrates the setup and creation of both record-based and file-based metadata curation workflows. It includes authentication, schema discovery, and task creation for both methods. ```python from pprint import pprint from synapseclient.extensions.curator import ( create_record_based_metadata_task, create_file_based_metadata_task, query_schema_registry ) from synapseclient import Synapse # Step 1: Authenticate syn = Synapse() syn.login() # Step 2: Find schema schema_uri = query_schema_registry( synapse_client=syn, dcc="ad", datatype="IndividualAnimalMetadataTemplate" ) print("Using schema:", schema_uri) # Step 3A: Create record-based workflow record_set, curation_task, data_grid = create_record_based_metadata_task( synapse_client=syn, project_id="syn123456789", folder_id="syn987654321", record_set_name="AnimalMetadata_Records", record_set_description="Centralized animal study metadata", curation_task_name="AnimalMetadata_Curation", upsert_keys=["StudyKey"], instructions="Complete metadata for all study animals using StudyKey to link records to data files.", schema_uri=schema_uri, bind_schema_to_record_set=True, assignee_principal_id="123456" # Optional: Assign to a user or team ) print(f"Record-based workflow created:") print(f" RecordSet: {record_set.id}") print(f" CurationTask: {curation_task.task_id}") # Step 3B: Create file-based workflow entity_view_id, task_id = create_file_based_metadata_task( synapse_client=syn, folder_id="syn987654321", curation_task_name="FileMetadata_Curation", instructions="Annotate each file with complete metadata according to schema.", attach_wiki=True, entity_view_name="Animal Study Files View", schema_uri=schema_uri, assignee_principal_id="123456" # Optional: Assign to a user or team ) print(f"File-based workflow created:") print(f" EntityView: {entity_view_id}") print(f" CurationTask: {task_id}") ``` -------------------------------- ### Working with Synapse Agents Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/reference/experimental/sync/agent.md This script demonstrates how to work with Synapse agents, including registering, getting, starting sessions, and interacting with them. ```python from synapseclient.models import Agent # Register a new agent agent_id = Agent.register(name="MyAgent", version="1.0", description="A simple agent.") print(f"Registered agent with ID: {agent_id}") # Get an existing agent agent = Agent.get(agent_id) print(f"Retrieved agent: {agent.name} (v{agent.version})") # Start a new session with the agent session = agent.start_session() print(f"Started session with ID: {session.id}") # Get the current session current_session = agent.get_session(session.id) print(f"Retrieved session: {current_session.id}") # Prompt the agent response = session.prompt("Hello, agent!") print(f"Agent response: {response}") # Get chat history chat_history = session.get_chat_history() print("Chat history:") for message in chat_history: print(f"- {message['sender']}: {message['message']}") ``` -------------------------------- ### Verify Synapse Client Installation Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/tutorials/installation.md After installation, use this command to show details about the installed synapseclient package. ```bash pip show synapseclient ``` -------------------------------- ### Full Source Code for Virtual Table Tutorial Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/tutorials/python/virtualtable.md This is the complete Python script used throughout the Virtual Tables tutorial, including setup, table creation, and various Virtual Table examples. ```python from synapse.ml.core.platform import SynapsePlatform from synapse.ml.core.dataset import Dataset from synapse.ml.core.schema import Schema from synapse.ml.core.exceptions import SynapseMLException import pandas as pd # Log in to Synapse # Replace 'My uniquely named project about Alzheimer\'s Disease' with your project name platform = SynapsePlatform(project_name='My uniquely named project about Alzheimer\'s Disease') # Get the project project = platform.get_project() # Define schema for the source table schema = Schema([ Schema.Column('sample_id', 'string'), Schema.Column('patient_id', 'string'), Schema.Column('age', 'integer'), Schema.Column('diagnosis', 'string') ]) # Create sample data data = [ {'sample_id': 'S1', 'patient_id': 'P1', 'age': 70, 'diagnosis': 'Alzheimer\'s'}, {'sample_id': 'S2', 'patient_id': 'P2', 'age': 65, 'diagnosis': 'Healthy'}, {'sample_id': 'S3', 'patient_id': 'P3', 'age': 72, 'diagnosis': 'Alzheimer\'s'}, {'sample_id': 'S4', 'patient_id': 'P4', 'age': 68, 'diagnosis': 'Healthy'}, {'sample_id': 'S5', 'patient_id': 'P5', 'age': 75, 'diagnosis': 'Alzheimer\'s'}, {'sample_id': 'S6', 'patient_id': 'P6', 'age': 80, 'diagnosis': 'Healthy'} ] df = pd.DataFrame(data) dataset = Dataset.from_pandas(df, schema) # Create the source table in Synapse source_table_name = 'source_table_for_virtual_tables' try: project.create_table(source_table_name, dataset, overwrite=True) print(f'Table {source_table_name} created successfully.') except SynapseMLException as e: print(f'Error creating table {source_table_name}: {e}') # Create a basic virtual table vt_name_basic = 'vt_basic' vt_sql_basic = f'SELECT * FROM {source_table_name}' project.create_virtual_table(vt_name_basic, vt_sql_basic, overwrite=True) print(f'Virtual table {vt_name_basic} created successfully.') # Query the basic virtual table results_basic = project.query_table(vt_name_basic) print(f'Results from the basic virtual table:\n{results_basic}') # Create a virtual table with column selection vt_name_col_select = 'vt_col_select' vt_sql_col_select = 'SELECT patient_id, age FROM source_table_for_virtual_tables' project.create_virtual_table(vt_name_col_select, vt_sql_col_select, overwrite=True) print(f'Virtual table {vt_name_col_select} created successfully.') # Query the virtual table with column selection results_col_select = project.query_table(vt_name_col_select) print(f'Results from the virtual table with column selection:\n{results_col_select}') # Create a virtual table with filtering vt_name_filter = 'vt_filter' vt_sql_filter = 'SELECT * FROM source_table_for_virtual_tables WHERE age > 70' project.create_virtual_table(vt_name_filter, vt_sql_filter, overwrite=True) print(f'Virtual table {vt_name_filter} created successfully.') # Query the virtual table with filtering results_filter = project.query_table(vt_name_filter) print(f'Results from the virtual table with filtering:\n{results_filter}') # Create a virtual table with ordering vt_name_order = 'vt_order' vt_sql_order = 'SELECT * FROM source_table_for_virtual_tables ORDER BY age DESC' project.create_virtual_table(vt_name_order, vt_sql_order, overwrite=True) print(f'Virtual table {vt_name_order} created successfully.') # Query the virtual table with ordering results_order = project.query_table(vt_name_order) print(f'Results from the virtual table with ordering:\n{results_order}') # Create a virtual table with aggregation vt_name_agg = 'vt_agg' vt_sql_agg = 'SELECT diagnosis, COUNT(*) AS patient_count FROM source_table_for_virtual_tables GROUP BY diagnosis' project.create_virtual_table(vt_name_agg, vt_sql_agg, overwrite=True) print(f'Virtual table {vt_name_agg} created successfully.') # Query the virtual table with aggregation results_agg = project.query_table(vt_name_agg) print(f'Results from the virtual table with aggregation:\n{results_agg}') ``` -------------------------------- ### Serve Local Documentation Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/CONTRIBUTING.md Use this command to start a local HTTP server for viewing the documentation pages. Ensure you are in the root directory of the project. ```bash mkdocs serve ``` -------------------------------- ### Initialize Synapse Client and Project Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/tutorials/python/wiki.md Sets up the Synapse client and retrieves a project entity. Ensure you have completed authentication setup. ```python from synapseclient import Synapse syn = Synapse() syn.login() # Replace with your project ID project_id = "syn12345678" project_entity = syn.get(project_id) ``` -------------------------------- ### Install Synapse Client from Git Reference Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/README.md Install the Synapse Python Client directly from a specific branch or commit hash using pip. ```bash pip install git+https://github.com/Sage-Bionetworks/synapsePythonClient@master pip install git+https://github.com/Sage-Bionetworks/synapsePythonClient@my-commit-hash ``` -------------------------------- ### Install SFTP dependencies on Unix Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/guides/data_storage.md Install necessary libraries for SFTP communication on Unix-like systems. Requires Python development headers. ```bash sudo apt-get install python-dev ``` -------------------------------- ### Install Synapse Client with Python3 and Pip3 Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/tutorials/installation.md Install the synapseclient package using python3 and pip3, with options for optional dependencies. ```bash python3 -m pip3 install --upgrade synapseclient ``` ```bash python3 -m pip3 install --upgrade "synapseclient[pandas]" ``` ```bash python3 -m pip3 install --upgrade "synapseclient[pandas, pysftp, boto3]" ``` -------------------------------- ### Install Synapse Client with Pipenv Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/CONTRIBUTING.md Install the Synapse client and its development dependencies locally using pipenv. Ensure you are in the root directory of the cloned repository. ```bash # Verify you are at the root directory for the cloned repository (ie: `cd synapsePythonClient`) # To develop locally you want to add --dev pipenv install --dev # Set your active session to the virtual environment you created pipenv shell # Note: The 'Python Environment Manager' extension in vscode is recommended here ``` -------------------------------- ### Setup Organizer Script Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/tutorials/python/submission.md Imports necessary libraries for Synapse challenge organization and scoring. ```python import synapseclient from synapseclient.core.exceptions import SynapseHTTPError syn = synapseclient.Synapse() syn.login(rememberMe=True) # Get the evaluation queue ID from the Evaluation tutorial # For this example, we'll use a placeholder ID. EVALUATION_ID = "syn1234567" # Get the submission ID you want to score. # For this example, we'll use a placeholder ID. SUBMISSION_ID = "12345" ``` -------------------------------- ### Dataclass and Method Example Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/CONTRIBUTING.md Example demonstrating how to document class attributes in both a docstring and under the class attribute itself, along with function examples. This format is required for auto-doc generation. ```python @dataclass class MyDataClass: """ The description of this class Attributes: my_attribute: My attribute description. Example: Example Doing something cool my_instance = MyDataClass() Doing something else cool my_instance = MyDataClass() """ my_attribute: bool = None """My attribute description.""" ``` -------------------------------- ### Set Up Synapse Python Client Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/tutorials/python/json_schema.md Initializes the Synapse client for interacting with the Synapse service. Ensure you have completed the installation and project tutorials. ```python from synapseclient import Synapse syn = Synapse() syn.login() print('Successfully logged in to Synapse.') ``` -------------------------------- ### Install Python and Manage Environments with Pyenv Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/tutorials/installation.md Install a specific Python version using pyenv, set it as global, create a virtual environment, and activate it. ```bash pyenv install -v 3.14.0 pyenv global 3.14.0 python -m venv env source env/bin/activate ``` -------------------------------- ### Install Synapse Client with Pip Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/README.md Install the Synapse Python Client using pip. Optional dependencies like pandas, pysftp, and boto3 can be included for enhanced functionality. ```bash pip install --upgrade synapseclient pip install --upgrade "synapseclient[pandas]" pip install --upgrade "synapseclient[pandas, pysftp, boto3]" ``` -------------------------------- ### Example Script: Working with Teams Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/reference/experimental/sync/team.md This script demonstrates various operations for managing teams, including creation, retrieval, and member management. It serves as a practical example for using the experimental team interfaces. ```python from synapseclient.core.exceptions import SynapseHTTPError def create_team(syn, name, description=""): try: team = syn.create_team(name, description) print(f"Created team: {team.name} ({team.id})") return team except SynapseHTTPError as e: print(f"Error creating team: {e}") return None def get_team(syn, team_id): try: team = syn.teams.get(team_id) print(f"Retrieved team: {team.name} ({team.id})") return team except SynapseHTTPError as e: print(f"Error retrieving team: {e}") return None def get_team_by_name(syn, name): try: team = syn.teams.get_by_name(name) print(f"Retrieved team by name: {team.name} ({team.id})") return team except SynapseHTTPError as e: print(f"Error retrieving team by name: {e}") return None def delete_team(syn, team_id): try: syn.delete_team(team_id) print(f"Deleted team: {team_id}") except SynapseHTTPError as e: print(f"Error deleting team: {e}") def list_team_members(syn, team_id): try: members = syn.get_team_members(team_id) print(f"Members of team {team_id}:") for member in members: print(f"- {member.memberId} ({member.roles})") except SynapseHTTPError as e: print(f"Error listing team members: {e}") def add_team_member(syn, team_id, user_id, roles=["MEMBER"]): try: syn.add_team_member(team_id, user_id, roles=roles) print(f"Added user {user_id} to team {team_id} with roles {roles}") except SynapseHTTPError as e: print(f"Error adding team member: {e}") def remove_team_member(syn, team_id, user_id): try: syn.remove_team_member(team_id, user_id) print(f"Removed user {user_id} from team {team_id}") except SynapseHTTPError as e: print(f"Error removing team member: {e}") def invite_user_to_team(syn, team_id, user_id): try: syn.invite_user_to_team(team_id, user_id) print(f"Invited user {user_id} to team {team_id}") except SynapseHTTPError as e: print(f"Error inviting user to team: {e}") def get_open_team_invitations(syn, team_id): try: invitations = syn.get_open_team_invitations(team_id) print(f"Open invitations for team {team_id}:") for invitation in invitations: print(f"- User ID: {invitation.inviteeId}, Status: {invitation.status}") except SynapseHTTPError as e: print(f"Error getting open team invitations: {e}") def get_user_membership_status(syn, team_id, user_id): try: status = syn.get_user_membership_status(team_id, user_id) print(f"Membership status of user {user_id} in team {team_id}: {status}") return status except SynapseHTTPError as e: print(f"Error getting user membership status: {e}") return None if __name__ == '__main__': # This is a placeholder for demonstration purposes. # Replace with actual Synapse authentication and object instantiation. # syn = synapseclient.login(email='your_email', password='your_password') print("This script requires a Synapse client object to run.") print("Please instantiate and authenticate a synapseclient.Synapse object.") # Example usage (requires a syn object): # team_name = "My Test Team" # team_description = "A team for testing purposes." # created_team = create_team(syn, team_name, team_description) # if created_team: # team_id = created_team.id # print(f"Team created with ID: {team_id}") # # Get team by ID # retrieved_team_id = get_team(syn, team_id) # # Get team by name # retrieved_team_name = get_team_by_name(syn, team_name) # # Add a member (replace 'user_id_to_add' with a valid Synapse user ID) # # add_team_member(syn, team_id, 'user_id_to_add') # # List members # list_team_members(syn, team_id) # # Invite a user (replace 'user_id_to_invite' with a valid Synapse user ID) # # invite_user_to_team(syn, team_id, 'user_id_to_invite') # # Get open invitations # get_open_team_invitations(syn, team_id) # # Get user membership status (replace 'user_id_to_check' with a valid Synapse user ID) # # get_user_membership_status(syn, team_id, 'user_id_to_check') # # Remove a member (replace 'user_id_to_remove' with a valid Synapse user ID) # # remove_team_member(syn, team_id, 'user_id_to_remove') # # Delete the team # # delete_team(syn, team_id) # else: # print("Team creation failed, skipping further operations.") ``` -------------------------------- ### Complete Validation Workflow Example Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/guides/extensions/curator/metadata_curation.md Demonstrates the end-to-end process of creating a curation task, preparing test data, uploading it, and initiating the validation process via Grid export. This example requires Synapse login and specific project/folder IDs. ```python from synapseclient import Synapse from synapseclient.extensions.curator import create_record_based_metadata_task, query_schema_registry from synapseclient.models import RecordSet from synapseclient.models.curation import Grid import pandas as pd import tempfile import os import time syn = Synapse() syn.login() # Step 1: Find the schema schema_uri = query_schema_registry( synapse_client=syn, dcc="ad", datatype="IndividualAnimalMetadataTemplate" ) # Step 1.5: Create initial test data with validation examples # Row 1: VALID - all required fields present and valid # Row 2: INVALID - missing required field 'genotype' # Row 3: INVALID - invalid enum value for 'sex' ("other" not in enum) test_data = pd.DataFrame({ "individualID": ["ANIMAL001", "ANIMAL002", "ANIMAL003"], "species": ["Mouse", "Mouse", "Mouse"], "sex": ["female", "male", "other"], # Row 3: invalid enum "genotype": ["5XFAD", None, "APOE4KI"], # Row 2: missing required field "genotypeBackground": ["C57BL/6J", "C57BL/6J", "C57BL/6J"], "modelSystemName": ["5XFAD", "5XFAD", "APOE4KI"], "dateBirth": ["2024-01-15", "2024-02-20", "2024-03-10"], "individualIdSource": ["JAX", "JAX", "JAX"], }) # Create a temporary CSV file with the test data temp_fd, temp_csv = tempfile.mkstemp(suffix=".csv") os.close(temp_fd) test_data.to_csv(temp_csv, index=False) # Step 2: Create the curation task (this creates an empty template RecordSet) record_set, curation_task, data_grid = create_record_based_metadata_task( synapse_client=syn, project_id="syn123456789", folder_id="syn987654321", record_set_name="AnimalMetadata_Records", record_set_description="Animal study metadata with validation", curation_task_name="AnimalMetadata_Validation_Example", upsert_keys=["individualID"], instructions="Enter metadata for each animal. All required fields must be completed.", schema_uri=schema_uri, bind_schema_to_record_set=True, ) time.sleep(10) print(f"Curation task created with ID: {curation_task.task_id}") print(f"RecordSet created with ID: {record_set.id}") # Step 2.5: Upload the test data to the RecordSet record_set = RecordSet(id=record_set.id).get(synapse_client=syn) print("\nUploading test data to RecordSet...") record_set.path = temp_csv record_set = record_set.store(synapse_client=syn) print(f"Test data uploaded to RecordSet {record_set.id}") # Step 3: Collaborators enter data via the web UI, OR you can create/export a Grid programmatically # For demonstration, here's the programmatic approach: print("\nCreating Grid session for data entry...") grid = Grid(record_set_id=record_set.id).create() print("Grid session created. Users can now enter data.") # After data entry is complete (either via web UI or programmatically), # export the Grid to generate validation results print("\nExporting Grid to RecordSet to generate validation results...") grid.export_to_record_set() ``` -------------------------------- ### Setup Participant Script Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/tutorials/python/submission.md Imports necessary libraries for Synapse challenge participation. ```python import synapseclient from synapseclient.core.exceptions import SynapseHTTPError syn = synapseclient.Synapse() syn.login(rememberMe=True) # Get the evaluation queue ID from the Evaluation tutorial # For this example, we'll use a placeholder ID. EVALUATION_ID = "syn1234567" # Get the entity ID of the file you want to submit. # For this example, we'll use a placeholder ID. ENTITY_ID = "syn7654321" ``` -------------------------------- ### Explore project contents Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/guides/synapse_mcp.md Example prompt to list all folders and files within a Synapse project and provide a summary. ```natural_language List all folders and files in syn5678901 and summarize what the project contains. ``` -------------------------------- ### Full Tutorial Script Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/tutorials/python/submissionview.md The complete source code for the SubmissionView tutorial, including all steps from setup to querying snapshots. ```python from synapseclient.core.exceptions import SynapseHTTPError from synapseclient import Synapse, SubmissionView, Column import tempfile import time syn = Synapse() syn.login() # 1. Set up and create an evaluation queue project_name = "My uniquely named project about Alzheimer\'s Disease" project = syn.get(f"synProject:{project_name}") evaluation_name = "Test Evaluation Queue for Alzheimer conference" try: evaluation = syn.create_evaluation( name=evaluation_name, description="A test evaluation queue for submission view tutorial.", project_id=project.id, status="OPEN", acl={} ) except SynapseHTTPError as e: if e.response.status_code == 409: evaluation = syn.getEvaluationByName(evaluation_name) else: raise e # 2. Create a SubmissionView for the evaluation queue metric_a_col = Column(name="metric_A", columnType="DOUBLE") metric_b_col = Column(name="metric_B", columnType="DOUBLE") submission_view = SubmissionView( name="My Submission View", description="A view of submissions for the Alzheimer conference.", parentId=project.id, evaluation_ids=[evaluation.id], columns=[metric_a_col, metric_b_col] ) syn.store(submission_view) # 3. Create and submit a file to the evaluation queue with tempfile.NamedTemporaryFile(suffix=".txt", delete=False) as f: f.write(b"This is a test file.") file_path = f.name submission = syn.submit(evaluation.id, file_path, name="Test Submission") print(f"Submitted file {file_path} to evaluation {evaluation.id} as submission {submission.id}") # 4. Query and update the submission status time.sleep(5) query_results = syn.chunkedQuery(f"SELECT * FROM {submission_view.id}") for row in query_results: if row['ROW_ID'] == submission.id: submission_status = syn.getSubmissionStatus(submission.id) submission_status['status'] = 'SCORED' submission_status['metric_A'] = 1.0 submission_status['metric_B'] = 2.0 syn.store(submission_status) print("Submission status: SCORED") break # 5. Modify the SubmissionView scope evaluation_name_2 = "Second Test Evaluation Queue for Alzheimer conference" try: evaluation_2 = syn.create_evaluation( name=evaluation_name_2, description="A second test evaluation queue.", project_id=project.id, status="OPEN", acl={} ) except SynapseHTTPError as e: if e.response.status_code == 409: evaluation_2 = syn.getEvaluationByName(evaluation_name_2) else: raise e submission_view.evaluation_ids.append(evaluation_2.id) syn.store(submission_view) # 6. Create a snapshot of the view snapshot = syn.create_submission_view_snapshot(submission_view.id) # 7. Query the snapshot snapshot_query_results = syn.chunkedQuery(f"SELECT * FROM {snapshot.id}") for row in snapshot_query_results: print(row) ``` -------------------------------- ### Install Synapse Python Client for Development Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/CLAUDE.md Installs the Synapse Python Client in editable mode with development dependencies. Includes optional dependencies for boto3, pandas, pysftp, tests, curator, and dev tools. ```bash pip install -e ".[boto3,pandas,pysftp,tests,curator,dev]" ``` -------------------------------- ### Setup Constants for Bulk Download Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/tutorials/python/download_data_in_bulk.md Define constants for the download directory and project ID. Ensure the target directory exists before proceeding. ```python import synapseclient import os from synapseclient.core.download.download_functions import download_file from synapseclient.core.exceptions import SynapseHTTPError # Set up constants DOWNLOAD_DIR = os.path.expanduser("~/my_ad_project") PROJECT_ID = "syn27360740" # Ensure download directory exists os.makedirs(DOWNLOAD_DIR, exist_ok=True) ``` -------------------------------- ### List files in a Synapse project Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/guides/synapse_mcp.md Example prompt to list all files within a specified Synapse project. ```natural_language What files are in the project syn12345678? ``` -------------------------------- ### Authenticate and Import Functions Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/guides/extensions/curator/metadata_curation.md Initializes the Synapse client and logs in. Ensure you have the synapseclient and curator extension installed. ```python from synapseclient.extensions.curator import ( create_record_based_metadata_task, create_file_based_metadata_task, query_schema_registry ) from synapseclient import Synapse syn = Synapse() syn.login() ``` -------------------------------- ### Example Script for Folder Operations Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/reference/experimental/sync/folder.md This script demonstrates object-oriented programming concepts for working with Synapse folders. It requires the 'oop_poc_folder.py' script to be available. ```python from synapseclient.models import Folder def folder_example(): # Create a folder object my_folder = Folder(name='my_new_folder', parent='syn12345') # Store the folder in Synapse my_folder = Folder.store(my_folder) print(f"Created folder: {my_folder.id}") # Sync folder contents from Synapse # Assuming 'syn12345' is a folder ID # Folder.sync_from_synapse('syn12345', local_path='./local_folder') # Sync folder contents to Synapse # Assuming './local_folder' is a local directory # Folder.sync_to_synapse('./local_folder', parent_id='syn12345') # Delete the folder # Folder.delete(my_folder.id) pass if __name__ == '__main__': folder_example() ``` -------------------------------- ### Get Synapse Client Version Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/ISSUE_TEMPLATE.md Run this code to retrieve the installed version of the Synapse Python client. This is often required for bug reports. ```python import synapseclient synapseclient.__version__ ``` -------------------------------- ### Full Synapse Migration Setup Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/guides/data_storage.md This script demonstrates setting up the Synapse client and defining parameters for a full data storage migration. It includes logging in and preparing paths for the migration index database. ```python import os import synapseutils import synapseclient my_synapse_project_or_folder_to_migrate = "syn123" external_bucket_name = "my-external-synapse-bucket" external_bucket_base_key = "path/within/bucket/" my_user_id = "1234" # a path on disk where this utility can create a sqlite database to store its index. # nothing needs to exist at this path, but it must be a valid path on a volume with sufficient # disk space to store a meta data listing of all the contents in the indexed entity. # a rough rule of thumb is 100kB per 1000 entities indexed. db_path = os.path.expanduser( f"~/synapseMigration/{my_synapse_project_or_folder_to_migrate}_my.db" ) syn = synapseclient.Synapse() # Log-in with ~.synapseConfig `authToken` syn.login() ``` -------------------------------- ### Initialize Synapse and Project Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/tutorials/python/table.md Sets up the Synapse client and retrieves a Synapse project. Ensure you are logged in to Synapse. ```python import synapseclient from synapseclient.models import Column, Project, query, SchemaStorageStrategy, Table syn = synapseclient.Synapse() syn.login() project = Project(name="My uniquely named project about Alzheimer's Disease").get() ``` -------------------------------- ### Set Up and Get Project Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/tutorials/python/sharing_settings.md Before running the tutorial, you must edit the script to set a valid PRINCIPAL_ID. This ID should be a Synapse user ID or team ID to grant permissions to. ```python import synapseclient from synapseclient.core.log import setup_logging setup_logging(level='INFO') # --- IMPORTANT: Set your PRINCIPAL_ID here! --- # This should be a Synapse user ID or team ID that you want to grant permissions to. # For example, to grant permissions to yourself, use your Synapse user ID. # To grant permissions to all authenticated users, use '273948'. # To grant permissions to the public, use '273949'. PRINCIPAL_ID = 123456789 # Replace with a valid Synapse ID syn = synapseclient.Synapse() syn.login() # Get or create a project to use for this tutorial # Replace 'MyTutorialProject' with the name of an existing project or a new name project_name = "MyTutorialProject" project = syn.get_or_create_project(project_name) print(f"Using project: {project.name} ({project.id})") ``` -------------------------------- ### Full Source Code for Tutorial Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/tutorials/python/proxy_storage_location.md This snippet contains the complete source code used throughout the tutorial, combining all steps for setting up a proxy storage location and associating files. ```python import synapseclient from synapseclient.models import StorageLocation, StorageLocationType syn = synapseclient.Synapse() syn.login() # Get a project to use for this tutorial project = syn.get_project("syn123456") # Create a proxy storage location proxy_url = "https://my-proxy-server.example.com" shared_secret = "my-secret-key" storage_location = StorageLocation( storageLocationType=StorageLocationType.PROXY, proxyUrl=proxy_url, proxySharedSecretKey=shared_secret, benefactorId=project.id ) created_storage_location = syn.store(storage_location) print(f"Created proxy storage location: {created_storage_location.id}") print(f" Proxy URL: {created_storage_location.proxyUrl}") print(f" Benefactor ID: {created_storage_location.benefactorId}") # Register a file via ProxyFileHandle file_md5 = "d41d8cd98f00b204e9800998ecf8427e" file_size = 1024 relative_file_path = "path/to/my/file.txt" proxy_file_handle = syn.create_proxy_file_handle( md5=file_md5, file_size=file_size, relative_path=relative_file_path, storage_location_id=created_storage_location.id ) print(proxy_file_handle) # Associate the ProxyFileHandle with a File entity file_entity = syn.create_file_entity( name="my_proxy_file.txt", parent=project.id, data_file_handle_id=proxy_file_handle['id'] ) print(f"Created File entity: {file_entity.id}") ``` -------------------------------- ### Start RStudio with Python using Docker Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/tutorials/reticulate.md Use this Docker command to launch an RStudio instance pre-configured with Python, Python virtual environment tools, and pip. This is useful for setting up a development environment for this guide. ```bash docker run --rm -it -p 8787:8787 \ -e PASSWORD=rstudio \ rocker/rstudio:latest \ bash -lc " apt-get update && apt-get install -y python3 python3-venv python3-pip python3-dev build-essential libcurl4-openssl-dev libssl-dev libxml2-dev && /init " ``` -------------------------------- ### Log in, get project, and create tables with data Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/tutorials/python/virtualtable.md This code sets up the Synapse environment by logging in, retrieving the project, and creating source tables with sample data. Ensure you replace the placeholder project name with your actual project name. ```python from synapse.ml.core.platform import SynapsePlatform from synapse.ml.core.dataset import Dataset from synapse.ml.core.schema import Schema from synapse.ml.core.exceptions import SynapseMLException import pandas as pd # Log in to Synapse # Replace 'My uniquely named project about Alzheimer\'s Disease' with your project name platform = SynapsePlatform(project_name='My uniquely named project about Alzheimer\'s Disease') # Get the project project = platform.get_project() # Define schema for the source table schema = Schema([ Schema.Column('sample_id', 'string'), Schema.Column('patient_id', 'string'), Schema.Column('age', 'integer'), Schema.Column('diagnosis', 'string') ]) # Create sample data data = [ {'sample_id': 'S1', 'patient_id': 'P1', 'age': 70, 'diagnosis': 'Alzheimer\'s'}, {'sample_id': 'S2', 'patient_id': 'P2', 'age': 65, 'diagnosis': 'Healthy'}, {'sample_id': 'S3', 'patient_id': 'P3', 'age': 72, 'diagnosis': 'Alzheimer\'s'}, {'sample_id': 'S4', 'patient_id': 'P4', 'age': 68, 'diagnosis': 'Healthy'}, {'sample_id': 'S5', 'patient_id': 'P5', 'age': 75, 'diagnosis': 'Alzheimer\'s'}, {'sample_id': 'S6', 'patient_id': 'P6', 'age': 80, 'diagnosis': 'Healthy'} ] df = pd.DataFrame(data) dataset = Dataset.from_pandas(df, schema) # Create the source table in Synapse source_table_name = 'source_table_for_virtual_tables' try: project.create_table(source_table_name, dataset, overwrite=True) print(f'Table {source_table_name} created successfully.') except SynapseMLException as e: print(f'Error creating table {source_table_name}: {e}') ``` -------------------------------- ### Full Tutorial Script for File Operations Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/tutorials/python/file.md This is the complete source code for the tutorial, demonstrating various file operations including creating, storing, and walking through project structures. It requires Synapse login and project setup. ```python import synapseclient import synapseutils # Log in to Synapse syn = synapseclient.Synapse() syn.login() # Define project details project_name = "My uniquely named project about Alzheimer's Disease" project_id = syn.findEntityId(project_name) # If project doesn't exist, create it if project_id is None: project = synapseclient.Project(name=project_name) project = syn.store(project) project_id = project.id print(f"Created project: {project.name} ({project.id})") else: print(f"Found project: {project_name} ({project_id})") # Create some folders and files # Folder 1 biospecimen_experiment_1 = synapseclient.Folder(name="biospecimen_experiment_1", parent=project_id) biospecimen_experiment_1 = syn.store(biospecimen_experiment_1) # Folder 2 biospecimen_experiment_2 = synapseclient.Folder(name="biospecimen_experiment_2", parent=project_id) biospecimen_experiment_2 = syn.store(biospecimen_experiment_2) # Folder 3 single_cell_RNAseq_batch_1 = synapseclient.Folder(name="single_cell_RNAseq_batch_1", parent=project_id) single_cell_RNAseq_batch_1 = syn.store(single_cell_RNAseq_batch_1) # Folder 4 single_cell_RNAseq_batch_2 = synapseclient.Folder(name="single_cell_RNAseq_batch_2", parent=project_id) single_cell_RNAseq_batch_2 = syn.store(single_cell_RNAseq_batch_2) # File A fileA = synapseclient.File(name="fileA.txt", parent=biospecimen_experiment_1.id, path="./fileA.txt", synid=None) fileA = syn.store(fileA) # File B fileB = synapseclient.File(name="fileB.txt", parent=biospecimen_experiment_1.id, path="./fileB.txt", synid=None) fileB = syn.store(fileB) # File C fileC = synapseclient.File(name="fileC.txt", parent=biospecimen_experiment_2.id, path="./fileC.txt", synid=None) fileC = syn.store(fileC) # File D fileD = synapseclient.File(name="fileD.txt", parent=biospecimen_experiment_2.id, path="./fileD.txt", synid=None) fileD = syn.store(fileD) # File E fileE = synapseclient.File(name="SRR12345678_R1.fastq.gz", parent=single_cell_RNAseq_batch_1.id, path="./SRR12345678_R1.fastq.gz", synid=None) fileE = syn.store(fileE) # File F fileF = synapseclient.File(name="SRR12345678_R2.fastq.gz", parent=single_cell_RNAseq_batch_1.id, path="./SRR12345678_R2.fastq.gz", synid=None) fileF = syn.store(fileF) # File G fileG = synapseclient.File(name="SRR12345678_R1.fastq.gz", parent=single_cell_RNAseq_batch_2.id, path="./SRR12345678_R1.fastq.gz", synid=None) fileG = syn.store(fileG) # File H fileH = synapseclient.File(name="SRR12345678_R2.fastq.gz", parent=single_cell_RNAseq_batch_2.id, path="./SRR12345678_R2.fastq.gz", synid=None) fileH = syn.store(fileH) # List all Folders and Files within my project print("\nListing all Folders and Files within my project:\n") for (dirpath, dirnames, filenames) in synapseutils.walk(syn, project_id): print(f"Directory ({dirpath.id}): {dirpath.name}") for dirname in dirnames: print(f"Directory ({dirname.id}): {dirname.name}") for filename in filenames: print(f"File ({filename.id}): {filename.name}") ``` -------------------------------- ### Full Tutorial Script Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/tutorials/python/dataset_collection.md The complete Python script for the Dataset Collection tutorial, demonstrating all steps from authentication to snapshotting. ```python from synapseclient import Synapse from synapseclient.core.dataset_collection import DatasetCollection from synapseclient.core.column import Column syn = Synapse() syn.login() # 1. Get the ID of your Synapse project project_id = syn.my_projects()[0]['id'] print(f'Using project ID: {project_id}') # 2. Create your Dataset Collection dataset_collection = DatasetCollection(name='My Dataset Collection', parent=project_id) dataset_collection.store() print(f'Dataset Collection created: {dataset_collection.id}') # 3. Add Datasets to the Dataset Collection # Replace with actual dataset IDs you want to add dataset_ids = ['syn123', 'syn456'] # Example IDs for dataset_id in dataset_ids: try: dataset_collection.add_item(dataset_id) print(f'Added {dataset_id} to collection') except Exception as e: print(f'Could not add {dataset_id}: {e}') dataset_collection.store() print('Stored changes to collection.') # 4. Retrieve the Dataset Collection retrieved_collection = DatasetCollection(id=dataset_collection.id) retrieved_collection.get() print(f'Retrieved collection: {retrieved_collection.id}') # 5. Add a custom column to the Dataset Collection custom_column = Column(name='My Custom Column', columnType='STRING', description='A custom annotation') dataset_collection.add_column(custom_column) dataset_collection.store() print('Added custom column and stored collection.') # Update the Dataset Collection with some values try: dataset_collection.update_item_annotation(item_id='syn123', column_name='My Custom Column', value='Value for syn123') print('Updated annotation for syn123') except Exception as e: print(f'Could not update annotation for syn123: {e}') try: dataset_collection.update_item_annotation(item_id='syn456', column_name='My Custom Column', value='Value for syn456') print('Updated annotation for syn456') except Exception as e: print(f'Could not update annotation for syn456: {e}') dataset_collection.store() print('Stored updated annotations.') # 6. Query the Dataset Collection # Note: Replace 'syn12345' with the actual ID of your collection if querying directly # For this example, we query the collection object itself query_string = f'SELECT * FROM {dataset_collection.id} WHERE "My Custom Column" = "Value for syn123"' print(f'Querying with: {query_string}') query_results = dataset_collection.query(query_string) print('Query Results:') print(query_results) # 7. Save a snapshot of the Dataset Collection snapshot = dataset_collection.store_snapshot() print(f'Snapshot created: {snapshot.id}') ``` -------------------------------- ### Full Tutorial Script Source: https://github.com/sage-bionetworks/synapsepythonclient/blob/develop/docs/tutorials/python/download_data_by_synid.md This snippet contains the complete source code for the tutorial, including imports, login, file mapping, and concurrent download logic. ```python import synapseclient from synapseclient.models import File import asyncio syn = synapseclient.Synapse() syn.login() # Map Synapse IDs to local download directories # Files can be directed to different directories as needed. file_map = { "syn12345": "~/temp/subdir1", "syn67890": "~/temp/subdir2", "syn11223": "~/temp/subdir1", } async def download_files(file_map): download_tasks = [] for syn_id, local_path in file_map.items(): download_tasks.append(File.get_async(syn_id, path=local_path)) await asyncio.gather(*download_tasks) asyncio.run(download_files(file_map)) print(f"Retrieved {len(file_map)} files") ```