### Install Neo4j Database with push.sh Source: https://github.com/opentreeoflife/germinator/blob/master/deploy/README.md Installs a Neo4j database on the server by unpacking a provided `.db.tgz` archive and restarting the Neo4j service. This command requires sufficient disk space. The previous database version is moved to `graph.db.previous` for recovery. ```bash ./push.sh -c {configfile} install-db downloads/{app}-{20151104}.db.tgz {app} ``` -------------------------------- ### Install Neo4j Database from Another AWS Instance Source: https://github.com/opentreeoflife/germinator/blob/master/deploy/README.md Installs a Neo4j database on an AWS instance by copying a `.db.tgz` file from another server. This optimizes bandwidth usage. SSH key setup is required for secure file transfer. ```bash ssh -i ~/.ssh/opentree/opentree.pem opentree@ot83.opentreeoflife.org scp ot74.opentreeoflife.org:downloads/treemachine-20151104.db.tgz downloads/ exit ./push.sh -c {ot83-configfile} install-db downloads/treemachine-20151104.db.tgz treemachine ``` -------------------------------- ### Taxon Information Retrieval Example Source: https://github.com/opentreeoflife/germinator/wiki/Taxonomic-service-for-adding-new-taxa An example of the JSON output received when querying for taxon information using an OTT ID after it has been indexed. It includes details like suppression status, sources, names, and flags. ```json { "is_suppressed" : false, "tax_sources" : [ "additions-99000030-99000031:99000030" ], "unique_name" : "Test taxon 1", "synonyms" : [ ], "name" : "Test taxon 1", "flags" : [ ], "ott_id" : 99000030, "rank" : "species" } ``` -------------------------------- ### Setup Phylesystem API with push.sh Source: https://github.com/opentreeoflife/germinator/blob/master/deploy/README.md Sets up the phylesystem API using the `push.sh` script. This operation requires the `OPENTREE_GH_IDENTITY` environment variable to be set in the configuration file, pointing to the SSH private key for GitHub access, to enable writing to studies. ```bash ./push.sh -c {configfile} phylesystem-api ``` -------------------------------- ### Smoke Test Deployment Script Source: https://github.com/opentreeoflife/germinator/blob/master/deploy/README.md Executes a basic smoke test for infrastructure setup by writing 'x' to the console. This command verifies the fundamental mechanics of the deployment system. ```bash ./push.sh -c {configfile} echo x ``` -------------------------------- ### GET /tree_of_life/about Source: https://github.com/opentreeoflife/germinator/wiki/Open-Tree-of-Life-Web-APIs Returns metadata and information about the current synthetic tree version. ```APIDOC ## GET /tree_of_life/about ### Description Returns information about the current synthetic tree, including versioning and source data details. ### Method GET ### Endpoint /tree_of_life/about ### Response #### Success Response (200) - **tree_version** (string) - The current version identifier of the synthetic tree. - **author** (string) - The creator or source of the tree. #### Response Example { "tree_version": "v3.0", "author": "OpenTreeOfLife" } ``` -------------------------------- ### Create and Copy Neo4j Database Archive Source: https://github.com/opentreeoflife/germinator/blob/master/deploy/README.md Creates a compressed tar archive of a Neo4j database directory and copies it to a remote server using `scp`. This is a preparatory step before installing the database on the server. Ensure adequate disk space before proceeding. ```bash tar -C {txxxmachine}/data/newlocaldb.db -czf newlocaldb.db.tgz . scp -p newlocaldb.db.tgz {host}:downloads/{app}-{20151104}.db.tgz ``` -------------------------------- ### ArguSON JSON Structure Example Source: https://github.com/opentreeoflife/germinator/wiki/"Arguson"-format This JSON object demonstrates the structure of an ArguSON payload, representing a synthetic tree. It includes lineage information (truncated) and the hierarchical 'children' structure, detailing nodes, taxa, and their relationships. Note that 'lineage' is truncated for brevity in this example. ```JSON { "arguson" : { "lineage" : [ { "supported_by" : { "taxonomy" : "ott2.9draft12" }, "taxon" : { "tax_sources" : [ "ncbi:30412", "worms:136996", "gbif:9312", "irmng:100635" ], "unique_name" : "Gaviidae", "name" : "Gaviidae", "rank" : "family", "ott_id" : 928350 }, "num_tips" : 5, "node_id" : "ott928350" }, { "supported_by" : { "taxonomy" : "ott2.9draft12" }, "taxon" : { "tax_sources" : [ "ncbi:30411", "worms:148793", "gbif:7192754", "irmng:10349" ], "unique_name" : "Gaviiformes", "name" : "Gaviiformes", "rank" : "order", "ott_id" : 70684 }, "num_tips" : 5, "node_id" : "ott70684" }, ... { "supported_by" : { "taxonomy" : "ott2.9draft12" }, "taxon" : { "tax_sources" : [ "silva:D11377/#1", "ncbi:2759" ], "unique_name" : "Eukaryota", "name" : "Eukaryota", "rank" : "domain", "ott_id" : 304358 }, "num_tips" : 2123117, "node_id" : "ott304358" }, { "taxon" : { "tax_sources" : [ "ncbi:131567" ], "unique_name" : "cellular organisms", "name" : "cellular organisms", "rank" : "no rank", "ott_id" : 93302 }, "num_tips" : 2424255, "node_id" : "ott93302" } ], "children" : [ { "supported_by" : { "taxonomy" : "ott2.9draft12" }, "taxon" : { "tax_sources" : [ "ncbi:37040", "worms:137188", "gbif:2481958", "irmng:10855603" ], "unique_name" : "Gavia stellata", "name" : "Gavia stellata", "rank" : "species", "ott_id" : 1057044 }, "num_tips" : 0, "terminal" : { "pg_420@tree522" : "ott1057044", "ot_104@tree1" : "node2" }, "node_id" : "ott1057044" }, { "descendant_name_list" : [ "Gavia arctica", "Gavia adamsii" ], "children" : [ { "descendant_name_list" : [ "Gavia arctica", "Gavia pacifica" ], "children" : [ { "supported_by" : { "taxonomy" : "ott2.9draft12" }, "taxon" : { "tax_sources" : [ "ncbi:57069", "worms:137186", "gbif:2481959", "irmng:10934623" ], "unique_name" : "Gavia arctica", "name" : "Gavia arctica", "rank" : "species", "ott_id" : 1085739 }, "num_tips" : 0, "terminal" : { "pg_420@tree522" : "ott1085739", "ot_104@tree1" : "node5" }, "node_id" : "ott1085739" }, { "supported_by" : { "taxonomy" : "ott2.9draft12" }, "taxon" : { "tax_sources" : [ "ncbi:85097", "gbif:2481955", "irmng:10589926" ], ``` -------------------------------- ### Example JSON Response for node_info Source: https://github.com/opentreeoflife/germinator/wiki/Synthetic-tree-API-v3 This is an example of the JSON response received from the node_info endpoint when querying for a specific node. It includes details such as partial path, supported by information, source ID map, taxon details, number of tips, terminal node information, and the node ID. ```json { "partial_path_of" : { "pg_2573@tree5959" : "node1011373" }, "supported_by" : { "taxonomy" : "ott2.9draft12" }, "source_id_map" : { "pg_2573@tree5959" : { "git_sha" : "e163a9245c15aefbba79defb685df98d5a73cab3", "tree_id" : "tree5959", "study_id" : "pg_2573" }, "taxonomy" : { "name" : "ott", "version" : "ott2.9draft12" }, "pg_2822@tree6569" : { "git_sha" : "e163a9245c15aefbba79defb685df98d5a73cab3", "tree_id" : "tree6569", "study_id" : "pg_2822" } }, "taxon" : { "tax_sources" : [ "ncbi:8782", "worms:1836", "gbif:212", "irmng:1142" ], "unique_name" : "Aves", "name" : "Aves", "rank" : "class", "ott_id" : 81461 }, "num_tips" : 14302, "terminal" : { "pg_2822@tree6569" : "ott240568" }, "node_id" : "ott81461" } ``` -------------------------------- ### Treemachine - Synthesize Subtree Source: https://github.com/opentreeoflife/germinator/blob/master/neo4j_services_docs.md Initiates the synthesis process for a subtree starting from a given root node. ```APIDOC ## POST /treemachine/ext/GoLS/graphdb/synthesizeSubtree ### Description Initiates the default synthesis process and stores the synthesized branches for the subgraph starting from a given root node. ### Method POST ### Endpoint http://devapi.opentreeoflife.org/treemachine/ext/GoLS/graphdb/synthesizeSubtree ### Parameters #### Query Parameters - **rootottId** (string) - Optional - The OTToL ID of the node to use as the root for synthesis. If omitted, the root of all life is used. ### Request Example ```json { "rootottId": "ottol.12345" } ``` ### Response #### Success Response (200) - **status** (string) - Indicates the success of the synthesis process. #### Response Example ```json { "status": "Synthesis initiated successfully." } ``` ``` -------------------------------- ### Support Descriptor Example Source: https://github.com/opentreeoflife/germinator/wiki/Open-Tree-API-datatypes The support-descriptor details source tree nodes corresponding to synthetic tree nodes. It's a dictionary where keys are source tree IDs and values are node IDs or lists of node IDs, indicating relationships like 'supported_by', 'conflicts_with', etc. ```json { "partial_path_of": { "pg_2573@tree5959": "node1011373" }, "supported_by": { "taxonomy": "ott2.9draft12" }, "terminal": { "pg_2822@tree6569": "ott240568" } } ``` -------------------------------- ### Example Subtree Response (JSON) Source: https://github.com/opentreeoflife/germinator/wiki/Synthetic-tree-API-v3 This is an example of a successful response from the /tree_of_life/subtree endpoint when requesting the Newick format. It contains the Newick string representation of the subtree. ```json { "newick" : "(Gavia_stellata_ott1057044,((Gavia_arctica_ott1085739,Gavia_pacifica_ott651474)mrcaott651474ott1085739,(Gavia_immer_ott1057518,Gavia_adamsii_ott90560)mrcaott90560ott1057518)mrcaott90560ott651474)Gavia_ott803675;" } ``` -------------------------------- ### Deploy Individual Component with push.sh Source: https://github.com/opentreeoflife/germinator/blob/master/deploy/README.md Deploys a specific component by passing its name as an argument to the `push.sh` script. This can stop a service, update its plugin, and restart it. For a full list of components, refer to `sample.config`. ```bash ./push.sh -c {configfile} treemachine ``` -------------------------------- ### Unpacking and Serving Synth Tree on 'files' Server Source: https://github.com/opentreeoflife/germinator/wiki/Late-2019-tree-deployment-steps This process involves logging into the 'files' server, creating the necessary directory structure for the new synthesis tree version, moving the transferred tarballs into this directory, and then unarchiving them. This makes the tree data accessible for browsing. ```bash ssh opentree@files.opentreeoflife.org # Create the appropriate destination directory: cd /var/www/html/synthesis mkdir "opentree${tree_version}" cd "opentree${tree_version}" # now we can move the tar archives mv ~/"opentree${tree_version}.tgz" . mv ~/"opentree${tree_version}_tree.tgz" . # unarchive the files, to allow browsing the data artifacts on `files` tar xfvz "opentree${tree_version}.tgz" tar xfvz "opentree${tree_version}_tree.tgz" ``` -------------------------------- ### Deploy Neo4j Database using push.sh (Bash) Source: https://github.com/opentreeoflife/germinator/blob/master/new_synthetic_tree.md This script deploys a compressed neo4j database to a server. It unpacks the database, makes it available to neo4j, and restarts the neo4j service. Requires a configuration file, the database tarball path, and a service name. Ensure sufficient disk space for unpacking. ```bash # Example usage: # ./push.sh -c {configfile} install-db downloads/treemachine-{YYYYMMDD}.db.tgz treemachine # Replace {configfile} with the path to your configuration file # Replace {YYYYMMDD} with the date stamp of the database tarball CONFIG_FILE="deploy/config.conf" DB_TARBALL="downloads/treemachine-20231027.db.tgz" SERVICE_NAME="treemachine" ./push.sh -c "${CONFIG_FILE}" install-db "${DB_TARBALL}" "${SERVICE_NAME}" ``` -------------------------------- ### Treemachine - Get Synthetic Tree Source: https://github.com/opentreeoflife/germinator/blob/master/neo4j_services_docs.md Retrieves the synthetic tree of life in various formats. ```APIDOC ## GET /treemachine/ext/GoLS/graphdb/getSyntheticTree ### Description Returns the synthetic tree of life. The format can be 'newick' or 'arguson'. ### Method GET ### Endpoint http://devapi.opentreeoflife.org/treemachine/ext/GoLS/graphdb/getSyntheticTree ### Parameters #### Query Parameters - **treeID** (string) - Required - The identifier for the synthesis (e.g., 'otol.draft.22'). - **format** (string) - Optional - The name of the return format (default is 'newick'). Allowed: 'newick', 'arguson'. - **maxDepth** (integer) - Optional - Controls the maximum number of edges between leaves and the node. Default is 5. Negative value means no pruning. - **subtreeNodeID** (string) - Optional - The node ID to serve as the root of the returned tree. ### Request Example ``` http://devapi.opentreeoflife.org/treemachine/ext/GoLS/graphdb/getSyntheticTree?treeID=otol.draft.22&format=newick ``` ### Response #### Success Response (200) - **newick** (string) - The tree in Newick format (if format is 'newick'). - **treeID** (string) - The identifier of the tree (if format is 'newick'). - **(object)** - The tree in Arguson JSON format (if format is 'arguson'). #### Response Example (newick) ```json { "newick": "(A:1,B:2,(C:3,D:4)E:5)F;", "treeID": "otol.draft.22" } ``` ``` -------------------------------- ### GET /taxonomy/taxon_info Source: https://github.com/opentreeoflife/germinator/wiki/Open-Tree-of-Life-Web-APIs Retrieves detailed information about a specific taxon using its unique identifier. ```APIDOC ## GET /taxonomy/taxon_info ### Description Returns metadata and taxonomic hierarchy information for a specific taxon ID. ### Method GET ### Endpoint /taxonomy/taxon_info ### Query Parameters - **ott_id** (integer) - Required - The unique Open Tree Taxonomy identifier. ### Response #### Success Response (200) - **name** (string) - The scientific name of the taxon. - **rank** (string) - The taxonomic rank. #### Response Example { "name": "Homo sapiens", "rank": "species" } ``` -------------------------------- ### Creating and Transferring Synth Tree Tarballs Source: https://github.com/opentreeoflife/germinator/wiki/Late-2019-tree-deployment-steps This section details the process of creating a compressed tarball of the synthesis tree on the 'nexttree' server and then securely copying it to the local machine and subsequently to the 'files' server. It involves SSHing into 'nexttree', running a script to create the tarball, and using SCP for file transfers. ```bash ssh opentree@nexttree.opentreeoflife.org cd synth/tree_builds/ bash ~/synth/propinquity/bin/make_tree_tarball.sh ./opentree"${tree_version}" opentree"${tree_version}" "${tree_version}" tar cfvz opentree"${tree_version}".tgz opentree"${tree_version}" exit # from your local machine: scp opentree@nexttree.opentreeoflife.org:~/synth/tree_builds/opentree"${tree_version}".tgz . scp opentree"${tree_version}".tgz opentree@files.opentreeoflife.org:~ scp opentree@nexttree.opentreeoflife.org:~/synth/tree_builds/opentree"${tree_version}_tree".tgz . scp opentree"${tree_version}_tree".tgz opentree@files.opentreeoflife.org:~ ``` -------------------------------- ### Taxomachine - Get Node ID From Name Source: https://github.com/opentreeoflife/germinator/blob/master/neo4j_services_docs.md Retrieves the node ID for a given taxonomic name. ```APIDOC ## GET /taxomachine/ext/GetJsons/graphdb/getNodeIDJSONFromName ### Description Returns a JSON object containing node IDs for nodes matching a given name. ### Method GET ### Endpoint http://devapi.opentreeoflife.org/taxomachine/ext/GetJsons/graphdb/getNodeIDJSONFromName ### Parameters #### Query Parameters - **nodename** (string) - Optional - The name of the node to find. ### Request Example ``` http://devapi.opentreeoflife.org/taxomachine/ext/GetJsons/graphdb/getNodeIDJSONFromName?nodename=Homo%20sapiens ``` ### Response #### Success Response (200) - **nodeIDs** (array) - An array of node IDs matching the provided name. #### Response Example ```json { "nodeIDs": [ 12345, 67890 ] } ``` ``` -------------------------------- ### POST /check_for_newly_deployed_tree Source: https://github.com/opentreeoflife/germinator/wiki/Deploying-new-synth-with-otcetera-services Triggers the API server to scan the filesystem for a newly uploaded synthetic tree archive and initiates the deployment process. ```APIDOC ## POST /check_for_newly_deployed_tree ### Description Triggers the server to check for a newly uploaded tree archive. If found, it validates the archive, launches the web service instance, and updates the internal version-to-port mapping. ### Method POST ### Endpoint http://api.opentreeoflife.org/check_for_newly_deployed_tree ### Parameters None ### Request Example POST /check_for_newly_deployed_tree ### Response #### Success Response (200) - **status** (string) - Deployment status message. - **version** (string) - The version of the tree successfully deployed. #### Response Example { "status": "success", "version": "v2.1.0", "port": 8081 } ``` -------------------------------- ### Taxomachine - Get Contexts JSON Source: https://github.com/opentreeoflife/germinator/blob/master/neo4j_services_docs.md Retrieves a list of available taxonomic contexts in JSON format. ```APIDOC ## GET /taxomachine/ext/TNRS/graphdb/getContextsJSON ### Description Returns information on available taxonomic contexts in JSON format. ### Method GET ### Endpoint http://devapi.opentreeoflife.org/taxomachine/ext/TNRS/graphdb/getContextsJSON ### Parameters None. ### Response #### Success Response (200) - **contexts** (array) - An array of available taxonomic context names. #### Response Example ```json { "contexts": [ "Primates", "Mammalia", "Animalia" ] } ``` ``` -------------------------------- ### Compress Neo4j Database for Deployment (Bash) Source: https://github.com/opentreeoflife/germinator/blob/master/new_synthetic_tree.md This command compresses the neo4j database directory into a tarball for transfer and deployment. It requires the path to the treemachine data directory and a date for identification. Ensure adequate disk space for compression. ```bash # Example usage: # tar -C {treemachine}/data/graph.db -czf treemachine-{YYYYMMDD}.db.tgz . # Replace {treemachine} with the actual path to your treemachine data directory # Replace {YYYYMMDD} with the ISO 8601 date (e.g., 20231027) TREEMACHINE_DATA_PATH="/path/to/treemachine/data/graph.db" DATE_STAMP="20231027" tar -C "${TREEMACHINE_DATA_PATH}" -czf "treemachine-${DATE_STAMP}.db.tgz" . ``` -------------------------------- ### S3 Backup of Synthesis Tree Data Source: https://github.com/opentreeoflife/germinator/wiki/Late-2019-tree-deployment-steps This command synchronizes the newly deployed synthesis tree directory from the 'files' server to an S3 bucket on opentreeoflifebackup.org. It requires the AWS environment to be activated and assumes the user has the necessary permissions to perform the sync operation. ```bash source awsenv/bin/activate # puts the aws command on PATH cd /var/www/html aws s3 sync --exact-timestamps "synthesis/${tree_version}" "s3://opentreeoflifebackup.org/synthesis/${tree_version}" ``` -------------------------------- ### Deploy Synthesis Tree Version to Remote Server Source: https://github.com/opentreeoflife/germinator/wiki/Testing-out-a-new-tree-created-with-propinquity A sequence of shell commands to package a local build, transfer it to a remote server via SCP, extract it, and update the server configuration. This process assumes the existence of a local build directory and requires SSH access to the target host. ```bash VERSION=10.5 OPENTREE_LOCAL=$HOME/Devel/OpenTree ARTIFACT_DIR=${OPENTREE_LOCAL}/synth-runs/10.5 # Build a tree # ./propinquity/build_at_dir.sh ${ARTIFACT_DIR} # Copy files to correct place on ot39 HOST=ot39.opentreeoflife.org OPENTREE_REMOTE=Applications/OpenTree SYNTH_PAR_REMOTE=${OPENTREE_REMOTE}/synth-par cp -a ${ARTIFACT_DIR} opentree${VERSION} tar -zcf opentree${VERSION}_output.tgz opentree${VERSION} scp opentreeVERSION_output.tgz ${HOST}:${SYNTH_PAR_REMOTE}/ ssh ${HOST} "cd ${SYNTH_PAR_REMOTE}; tar -zxf opentree10.5_output.tgz" # Restart server cd ${OPENTREE_LOCAL}/germinator/deploy ./push.sh -c ../../deployed-systems/development/ot39.config otcetera # Update version in synthesis.json # ssh ${HOST} # cd repo/opentree/webapp/static/statistics # nano synthesis.json ``` -------------------------------- ### Taxomachine - Get Context For Names Source: https://github.com/opentreeoflife/germinator/blob/master/neo4j_services_docs.md Finds the least inclusive taxonomic context for a given set of taxon names. ```APIDOC ## POST /taxomachine/ext/TNRS/graphdb/getContextForNames ### Description Finds the least inclusive taxonomic context defined for the provided set of taxon names. ### Method POST ### Endpoint http://devapi.opentreeoflife.org/taxomachine/ext/TNRS/graphdb/getContextForNames ### Parameters #### Query Parameters - **names** (array) - Required - An array of taxon names to query. ### Request Example ```json { "names": ["Homo sapiens", "Pan troglodytes"] } ``` ### Response #### Success Response (200) - **contexts** (array) - An array of contexts, each associated with the input names. #### Response Example ```json { "contexts": [ { "names": ["Homo sapiens", "Pan troglodytes"], "context": "Primates" } ] } ``` ``` -------------------------------- ### Studies API v3 - Get Study Source: https://github.com/opentreeoflife/germinator/wiki/Open-Tree-of-Life-Web-APIs Retrieves a specific study by its ID. This endpoint is part of the Studies API v3. ```APIDOC ## GET /study/{STUDY_ID} ### Description Return a study. ### Method GET ### Endpoint /study/{STUDY_ID} ### Parameters #### Path Parameters - **STUDY_ID** (string) - Required - The unique identifier of the study. ### Request Example ``` GET /study/12345 ``` ### Response #### Success Response (200) - **study_data** (object) - The data for the requested study. #### Response Example ```json { "study_data": { "id": "12345", "name": "Example Study", "author": "John Doe", "year": 2023 } } ``` ``` -------------------------------- ### GET /tree_of_life/subtree Source: https://github.com/opentreeoflife/germinator/wiki/"Arguson"-format Retrieves a subtree from the synthetic tree in ArguSON format by specifying the format parameter. ```APIDOC ## GET /tree_of_life/subtree ### Description Returns a nested JSON object representing a subtree of the synthetic tree. When the format parameter is set to 'arguson', the response follows the ArguSON schema. ### Method GET ### Endpoint /tree_of_life/subtree ### Parameters #### Query Parameters - **node_id** (string) - Required - The ID of the root node for the subtree. - **format** (string) - Required - Must be set to "arguson". - **max_height** (integer) - Optional - Limits the depth of the returned tree. ### Request Example GET /tree_of_life/subtree?node_id=ott928350&format=arguson ### Response #### Success Response (200) - **lineage** (list) - List of node descriptors representing the path to the root. - **children** (list) - Recursive list of child nodes. - **node_id** (string) - Unique identifier for the node. - **taxon** (object) - Taxon descriptor if the node represents a taxon. #### Response Example { "arguson": { "lineage": [ { "node_id": "ott928350", "taxon": { "name": "Gaviidae", "rank": "family", "ott_id": 928350 }, "num_tips": 5 } ], "children": [ { "node_id": "ott1057044", "taxon": { "name": "Gavia stellata", "rank": "species", "ott_id": 1057044 }, "num_tips": 0 } ] } } ``` -------------------------------- ### Ansible Playbook Execution for Synth Tree Deployment Source: https://github.com/opentreeoflife/germinator/wiki/Late-2019-tree-deployment-steps Executes Ansible playbooks on the 'nexttree' server to build, rebuild, and serve the synthesis tree. This process involves pulling the latest Ansible configurations, committing changes, and then running specific playbooks for the build pipeline, synth tree rebuild, and serving the next tree. ```bash cd $OPENTREE_ROOT/ot-ansible/ git pull origin # Hand edit next_synth_version in group_vars/all.yml and (if needed propinquity_branch in roles/fetch-propinquity/vars/main.yml) git commit -m "updated next_synth_version" -a git push origin master ansible-playbook --limit=nexttree playbk-build-synth-pipeline.yml ansible-playbook --limit=nexttree playbk-rebuild-synth-tree.yml ansible-playbook --limit=nexttree playbk-serve-next-tree.yml ``` -------------------------------- ### GET /get_next_synth_patch_version Source: https://github.com/opentreeoflife/germinator/wiki/Deploying-new-synth-with-otcetera-services Retrieves the next available version number for a synthetic tree patch, with options to trigger a minor or major version bump. ```APIDOC ## GET /get_next_synth_patch_version ### Description Retrieves the next version identifier for a synthetic tree patch. Allows for optional version bumping to handle minor or major releases. ### Method GET ### Endpoint http://api.opentreeoflife.org/get_next_synth_patch_version ### Parameters #### Query Parameters - **bump_minor** (boolean) - Optional - If true, increments the minor version number. - **bump_major** (boolean) - Optional - If true, increments the major version number. ### Request Example GET /get_next_synth_patch_version?bump_minor=false&bump_major=false ### Response #### Success Response (200) - **version** (string) - The next version string (e.g., "v1.2.3"). #### Response Example { "version": "v2.1.0" } ``` -------------------------------- ### Allocate and Assign OTT IDs Source: https://github.com/opentreeoflife/germinator/wiki/Deploying-a-new-taxonomy-version Manual steps to reserve a range of taxonomy IDs by updating the phylesystem-api configuration and committing the changes to GitHub. ```bash sudo apache2ctl stop # Edit next_ott_id.json manually git commit -m "Update ID range" git push ./push.sh -c ... phylesystem-api ``` -------------------------------- ### GET /study/{STUDY_ID}/tree/{TREE_ID} Source: https://github.com/opentreeoflife/germinator/wiki/Studies-API-v3 Retrieves a specific tree structure from within a defined phylogenetic study. ```APIDOC ## GET /study/{STUDY_ID}/tree/{TREE_ID} ### Description Returns a JSON object representing a specific tree within a study. Supports fine-grained access to tree data. ### Method GET ### Endpoint /study/{STUDY_ID}/tree/{TREE_ID} ### Parameters #### Path Parameters - **STUDY_ID** (string) - Required - The unique identifier of the study. - **TREE_ID** (string) - Required - The unique identifier of the tree within the study. ### Request Example curl https://api.opentreeoflife.org/v3/study/pg_1144/tree/tree2324 ### Response #### Success Response (200) - **tree** (object) - The tree data structure. #### Response Example { "tree": { "@id": "tree2324", ... } } ``` -------------------------------- ### Production Deployment with Germinator Scripts Source: https://github.com/opentreeoflife/germinator/wiki/Late-2019-tree-deployment-steps This script sequence deploys the new synthesis tree to production. It involves pulling the latest deployment configurations, updating the SYNTH_URL in the production API configuration file, committing these changes, and then executing the germinator's push script. ```bash cd "$OPENTREE_ROOT/deployed-systems" git pull origin hand-edit `SYNTH_URL` in `production/api.config` to be `SYNTH_URL=http://files.opentreeoflife.org/synthesis/opentree${tree_version}/opentree${tree_version}.tgz` git commit -m "updated SYNTH_URL" production/api.config git push origin master cd ../germinator/deploy bash ./push.sh -c "$OPENTREE_ROOT/deployed-systems/production/api.config" otcetera ``` -------------------------------- ### Index OTI Database with push.sh Source: https://github.com/opentreeoflife/germinator/blob/master/deploy/README.md Indexes all studies in the study store for the OTI application. This operation is performed using the `push.sh` script with the `index-db` command. ```bash ./push.sh -c {configfile} index-db ``` -------------------------------- ### GET /study/{study_id} Source: https://context7.com/opentreeoflife/germinator/llms.txt Retrieves a complete study in various formats including NexSON, NEXUS, Newick, or NeXML. ```APIDOC ## GET /study/{study_id} ### Description Retrieves the full content of a study. File format can be specified via file extension. ### Method GET ### Endpoint https://api.opentreeoflife.org/v3/study/{study_id} ### Parameters #### Path Parameters - **study_id** (string) - Required - The unique identifier of the study #### Query Parameters - **tip_label** (string) - Optional - Format for tip labels (e.g., "ot:ottid", "ot:otttaxonname") ### Response #### Success Response (200) - **body** (file) - The study data in the requested format ``` -------------------------------- ### Create Tarballs for Synthetic Tree (Bash) Source: https://github.com/opentreeoflife/germinator/blob/master/new_synthetic_tree.md This script creates two compressed tar archives for the synthetic tree: a small summary archive and a large archive containing all synthesis outputs. It requires the propinquity tool and specific output files from the synthesis process. ```bash #!/bin/bash # Example usage: # bin/make_tarballs.sh SYNTH_VERSION=$1 # Create summary archive tar -czf opentree${SYNTH_VERSION}_tree.tgz \ labelled_supertree/labelled_supertree.tre \ labelled_supertree/labelled_supertree_ottnames.tre \ grafted_solution/grafted_solution.tre \ grafted_solution/grafted_solution_ottnames.tre \ annotated_supertree/annotations.json \ README.md # Create large output archive tar -czf opentree${SYNTH_VERSION}_output.tgz \ labelled_supertree \ grafted_solution \ annotated_supertree \ *.html ``` -------------------------------- ### Studies API v3 - Get Tree from Study Source: https://github.com/opentreeoflife/germinator/wiki/Open-Tree-of-Life-Web-APIs Retrieves a specific tree from within a study by its study ID and tree ID. This endpoint is part of the Studies API v3. ```APIDOC ## GET /study/{STUDY_ID}/tree/{TREE_ID} ### Description Return a tree from within a study. ### Method GET ### Endpoint /study/{STUDY_ID}/tree/{TREE_ID} ### Parameters #### Path Parameters - **STUDY_ID** (string) - Required - The unique identifier of the study. - **TREE_ID** (string) - Required - The unique identifier of the tree within the study. ### Request Example ``` GET /study/12345/tree/treeABC ``` ### Response #### Success Response (200) - **tree_data** (object) - The data for the requested tree. #### Response Example ```json { "tree_data": { "id": "treeABC", "name": "Example Tree", "newick": "(A:1,B:2,(C:3,D:4)E:5)F;" } } ``` ``` -------------------------------- ### Run OpenTreeOfLife API Integration Tests (bash) Source: https://context7.com/opentreeoflife/germinator/llms.txt This snippet provides bash commands for running integration tests against an OpenTreeOfLife API deployment. It covers running all tests, tests for a specific component, and executing an individual test file. Environment variables may need to be set for specific test configurations. ```bash cd germinator ws-tests/run_tests.sh https://devapi.opentreeoflife.org ``` ```bash ws-tests/run_tests.sh -t ../oti/ws-tests https://devapi.opentreeoflife.org ``` ```bash cd oti/ws-tests export PYTHONPATH=$PWD/../../germinator/ws-tests:$PYTHONPATH python test-basic.py host:apihost=https://devapi.opentreeoflife.org ``` -------------------------------- ### GET /study/{STUDY_ID} Source: https://github.com/opentreeoflife/germinator/wiki/Studies-API-v3 Retrieves a phylogenetic study in various formats including NexSON, NEXUS, Newick, or NeXML based on the file extension. ```APIDOC ## GET /study/{STUDY_ID} ### Description Returns a study object. The 'data' property contains the NexSON representation. File format conversion is supported via extensions (.nex, .tre, .nexml, .nexson). ### Method GET ### Endpoint /study/{STUDY_ID} ### Parameters #### Path Parameters - **STUDY_ID** (string) - Required - The unique identifier of the study. #### Query Parameters - **tip_label** (string) - Optional - Specifies the label format for NEXUS/Newick (ot:originallabel, ot:ottid, or ot:otttaxonname). ### Request Example curl https://api.opentreeoflife.org/v3/study/pg_1144 ### Response #### Success Response (200) - **data** (object) - The study content in the requested format. #### Response Example { "data": { "nexml": { ... } } } ``` -------------------------------- ### Update files.opentreeoflife.org Website Source: https://github.com/opentreeoflife/germinator/blob/master/deploy/README.md Updates the `files.opentreeoflife.org` website by copying the contents of the local `files.opentreeoflife.org` directory to the web root. Files present remotely but not locally are not affected. The destination is controlled by the `FILES_HOST` setting. ```bash ./push.sh -c {configfile} files ``` -------------------------------- ### Generate Final Taxonomy and Deploy Source: https://github.com/opentreeoflife/germinator/wiki/Deploying-a-new-taxonomy-version Finalizing the taxonomy build and transferring the resulting tarball to the distribution server. ```bash make works time scp -p tarballs/ott2.10draft6.tgz files:files.opentreeoflife.org/ott/ott2.10/ ``` -------------------------------- ### GET /db/data/ext/taxonomy_v3/graphdb/taxon_info Source: https://github.com/opentreeoflife/germinator/wiki/Taxonomic-service-for-adding-new-taxa Retrieves information about a specific taxon using its OTT ID. This endpoint is useful for verifying that newly added taxa are correctly indexed and visible. ```APIDOC ## GET /db/data/ext/taxonomy_v3/graphdb/taxon_info ### Description Retrieves detailed information for a given taxon identified by its OTT ID. This is useful for verifying taxon details after processing additions. ### Method GET ### Endpoint /db/data/ext/taxonomy_v3/graphdb/taxon_info ### Parameters #### Query Parameters - **ott_id** (integer) - Required - The OTT ID of the taxon to retrieve information for. ### Response #### Success Response (200) - **is_suppressed** (boolean) - Indicates if the taxon is suppressed. - **tax_sources** (array of strings) - List of sources for the taxon information. - **unique_name** (string) - The unique name of the taxon. - **synonyms** (array of objects) - A list of synonyms for the taxon. - **name** (string) - The common name of the taxon. - **flags** (array of strings) - Flags associated with the taxon. - **ott_id** (integer) - The OTT ID of the taxon. - **rank** (string) - The taxonomic rank of the taxon. #### Response Example ```json { "is_suppressed" : false, "tax_sources" : [ "additions-99000030-99000031:99000030" ], "unique_name" : "Test taxon 1", "synonyms" : [ ], "name" : "Test taxon 1", "flags" : [ ], "ott_id" : 99000030, "rank" : "species" } ``` ``` -------------------------------- ### Induced Subtree API Response Structure Source: https://github.com/opentreeoflife/germinator/wiki/Synthetic-tree-API-v3 Example JSON response from the induced_subtree API endpoint. It includes the 'newick' string representing the subtree and a list of 'supporting_studies' that contribute to the tree. ```json { "broken": {}, "newick": "(((((((((((((((((((((((((((((Cinclus_ott267845)Cinclidae_ott496027)mrcaott1566ott496009)mrcaott1566ott3598440)mrcaott246ott1566)mrcaott246ott5934)mrcaott246ott3364)mrcaott246ott1488)mrcaott246ott10351)mrcaott246ott176461)mrcaott246ott22325)mrcaott246ott4820)mrcaott246ott32658)mrcaott246ott5929)mrcaott246ott44866)mrcaott246ott428578)mrcaott246ott3212)Passeriformes_ott1041547)mrcaott246ott7113)mrcaott246ott47588)mrcaott246ott3600042)mrcaott246ott2907)mrcaott246ott1858)mrcaott246ott928360)mrcaott246ott5272)mrcaott246ott7145)mrcaott246ott5021)mrcaott246ott5481,((((((((((((((((Perdix_ott102710)mrcaott53700ott466627)mrcaott53700ott572162)mrcaott4765ott53700)mrcaott4765ott51354)mrcaott4765ott415487)mrcaott4765ott49310)mrcaott4765ott49319)mrcaott4765ott54193)mrcaott4765ott151684)mrcaott4765ott104461)mrcaott4765ott75785)mrcaott4765ott109888)mrcaott4765ott6520194)Galliformes_ott837585)mrcaott4765ott6520193,(((((((Clangula_ott316878)mrcaott145504ott316879)mrcaott30843ott145504)mrcaott30843ott962771)mrcaott30843ott75874)Anatidae_ott765193)mrcaott30843ott714464)Anseriformes_ott241841)mrcaott4765ott30843)mrcaott246ott4765,((Struthio_ott292466)Struthionidae_ott857849)mrcaott84218ott1007318)Aves_ott81461", "supporting_studies": [ "ot_425@tree1", "ot_521@tree1", "ot_809@tree2", "pg_2804@tree6511", "ot_290@tree1", "ot_783@tree1", "pg_2015@tree4152", "ot_531@tree1", "ot_782@Tr48736", "ot_841@tree1", "pg_2913@tree6742", "ot_534@tree1", "ot_757@Tr70438", "pg_420@tree522", "ot_412@Tr77157", "ot_500@tree1", "ot_532@tree1", "pg_2404@tree5068", "pg_2404@tree6388", "pg_2913@tree6741", "ot_111@tree1", "ot_753@tree1", "pg_2577@tree5980", "ot_1002@tree1", "ot_1345@Tr109614", "ot_855@tree1", "ot_1174@tree3", "pg_2866@tree6656", "ot_1022@tree1", "ot_314@tree1", "ot_1516@Tr55113" ] } ``` -------------------------------- ### Restart Apache with push.sh Source: https://github.com/opentreeoflife/germinator/blob/master/deploy/README.md Restarts the Apache web server. This command is typically used after updating Apache configuration files. ```bash ./push.sh -c {configfile} apache ``` -------------------------------- ### Indexing New Taxa with TNRS Source: https://github.com/opentreeoflife/germinator/wiki/Taxonomic-service-for-adding-new-taxa Shows how to use `curl` to POST an additions document to the `/v3/taxonomy/process_additions` endpoint for indexing new taxa. This makes the taxa visible in subsequent taxonomy and TNRS service calls. ```bash curl -X POST http://localhost:7476/db/data/ext/taxonomy_v3/graphdb/taxon_info \ -H "content-type:application/json" -d '{"ott_id": 99000030}' ```