### R: Set up googleCloudStorageR Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_setup.html This function guides the user through the setup process for googleCloudStorageR. It assumes a Google Cloud Platform project is already set up and uses a wizard to generate authentication keys and configure authentication. Owner access to the project is required. After setup, R should be restarted and the library reloaded. ```R library(googleCloudStorageR) gcs_setup() ``` -------------------------------- ### Example: Saving a Directory to GCS Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_save_all.html A practical example demonstrating how to save a specific directory to a designated GCS bucket using bucket-level access control. ```R gcs_save_all( directory = "path-to-all-images", bucket = "my-bucket", predefinedAcl = "bucketLevel") ``` -------------------------------- ### Install googleCloudStorageR package Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/index.html Provides instructions for installing the package from CRAN or the development version from GitHub using the remotes package. ```R # Install latest stable version from CRAN install.packages("googleCloudStorageR") # Install development version from GitHub if(!require("remotes")){ install.packages("remotes") } remotes::install_github("cloudyr/googleCloudStorageR") ``` -------------------------------- ### Install googleCloudStorageR from GitHub Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/Readme.html Installs the development version of the package from the cloudyr GitHub repository. It first checks for the remotes package dependency. ```R if(!require("remotes")){ install.packages("remotes") } remotes::install_github("cloudyr/googleCloudStorageR") ``` -------------------------------- ### Example: Uploading with Bucket-Level ACL (R) Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_upload.html Shows how to upload a file to a bucket that has bucket-level access control enabled, by setting the predefinedAcl argument to 'bucketLevel'. ```R gcs_upload(mtcars, predefinedAcl = "bucketLevel") ``` -------------------------------- ### Example: Setting Global Bucket (R) Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_upload.html Shows how to set a default Google Cloud Storage bucket for subsequent operations, avoiding the need to specify the bucket name in every function call. ```R gcs_global_bucket("my-bucket") ``` -------------------------------- ### Example: Uploading Dataframe (R) Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_upload.html Demonstrates the default behavior of uploading an R data frame to Google Cloud Storage, which is automatically converted to a CSV file. ```R gcs_upload(mtcars) ``` -------------------------------- ### Example: Uploading Multiple Files (R) Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_upload.html Illustrates how to upload multiple files from a directory to Google Cloud Storage, ensuring each file is uploaded with its original name. This is typically done within a loop or using lapply. ```R my_files <- list.files("my_uploads") lapply(my_files, function(x) gcs_upload(x, name = x)) ``` -------------------------------- ### Retrieve Bucket Metadata using gcs_get_bucket Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_get_bucket.html This example demonstrates how to list available buckets for a project and then fetch the metadata for the first bucket in the list. It requires an authenticated session with Google Cloud Storage. ```R buckets <- gcs_list_buckets("your-project") # Use the name of the bucket to get more meta data bucket_meta <- gcs_get_bucket(buckets$name[[1]]) ``` -------------------------------- ### Compose Multiple Objects into One Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_compose_objects.html This example demonstrates how to set the global bucket, select a subset of objects from the bucket, and merge them into a single destination file using gcs_compose_objects. ```R gcs_global_bucket("your-bucket") objs <- gcs_list_objects() # Select up to 32 objects to compose compose_me <- objs$name[1:30] # Compose the objects into a new file gcs_compose_objects(compose_me, "composed/test.json") ``` -------------------------------- ### Example: Specifying Uploaded File Name (R) Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_upload.html Illustrates how to explicitly define the name of the file once it is uploaded to Google Cloud Storage, overriding the default naming convention. ```R gcs_upload(mtcars, name = "my_mtcars.csv") ``` -------------------------------- ### Example: R code for Google Cloud Storage Bucket Versioning Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_version_bucket.html This R code demonstrates how to use the gcs_version_bucket function to interact with Google Cloud Storage bucket versioning. It shows examples of disabling versioning, checking its status, enabling it, and listing versioned objects. Ensure you have authenticated access to your Google Cloud project. ```R buck <- gcs_get_global_bucket() gcs_version_bucket(buck, action = "disable") gcs_version_bucket(buck, action = "status") # Versioning is NOT ENABLED for "your-bucket" gcs_version_bucket(buck, action = "enable") # TRUE gcs_version_bucket(buck, action = "status") # Versioning is ENABLED for "your-bucket" gcs_version_bucket(buck, action = "list") ``` -------------------------------- ### Install googleCloudStorageR from CRAN Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/Readme.html Installs the latest stable version of the googleCloudStorageR package directly from the Comprehensive R Archive Network (CRAN). ```R install.packages("googleCloudStorageR") ``` -------------------------------- ### Set and Get Default Google Cloud Storage Bucket (R) Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/articles/googleCloudStorageR.html Demonstrates how to set a default bucket for the current R session using gcs_global_bucket() and then retrieve it using gcs_get_global_bucket(). This is useful for simplifying subsequent operations that target a specific bucket. ```R gcs_global_bucket("your-default-bucket-2") gcs_get_global_bucket() # Expected output: [1] "your-default-bucket-2" ``` -------------------------------- ### Create and apply a GCS lifecycle rule Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_create_lifecycle.html This example demonstrates how to create a lifecycle rule to delete objects after 30 days and apply it to a new Google Cloud Storage bucket using the gcs_create_bucket function. ```R lifecycle <- gcs_create_lifecycle(age = 30) gcs_create_bucket("your-bucket-lifecycle", projectId = "your-project", location = "EUROPE-NORTH1", storageClass = "REGIONAL", lifecycle = list(lifecycle)) ``` -------------------------------- ### GET /buckets/{bucket}/notificationConfigs Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_list_pubsub.html Lists all notification configurations associated with a specified Google Cloud Storage bucket. ```APIDOC ## GET /buckets/{bucket}/notificationConfigs ### Description Retrieves a list of all notification configurations currently set up for the specified Google Cloud Storage bucket to monitor object changes. ### Method GET ### Endpoint /buckets/{bucket}/notificationConfigs ### Parameters #### Path Parameters - **bucket** (string) - Required - The name of the Google Cloud Storage bucket to query. ### Request Example N/A (Uses function call: gcs_list_pubsub(bucket = "my-bucket")) ### Response #### Success Response (200) - **items** (array) - A list of notification configuration objects. #### Response Example { "kind": "storage#notifications", "items": [ { "id": "1", "topic": "projects/my-project/topics/my-topic", "event_types": ["OBJECT_FINALIZE"], "payload_format": "JSON_API_V1" } ] } ``` -------------------------------- ### GET /objects/{objectName} Source: https://github.com/cloudyr/googlecloudstorager/blob/master/tests/testthat/_snaps/objects.md Retrieves the metadata for a specific object within a bucket. This endpoint returns a comprehensive list of properties including storage class, size, and timing information. ```APIDOC ## GET /objects/{objectName} ### Description Retrieves metadata for a specific object stored in a Google Cloud Storage bucket. ### Method GET ### Endpoint /objects/{objectName} ### Parameters #### Path Parameters - **objectName** (string) - Required - The unique name of the object within the bucket. ### Response #### Success Response (200) - **id** (string) - The ID of the object. - **name** (string) - The name of the object. - **bucket** (string) - The name of the bucket containing the object. - **size** (integer) - The size of the object in bytes. - **contentType** (string) - The content type of the object. - **storageClass** (string) - The storage class of the object. - **timeCreated** (timestamp) - The creation time of the object. - **updated** (timestamp) - The last update time of the object. #### Response Example { "id": "bucket/object/123", "name": "example.txt", "bucket": "my-bucket", "size": 1024, "contentType": "text/plain", "storageClass": "STANDARD", "timeCreated": "2023-01-01T00:00:00Z", "updated": "2023-01-01T00:00:00Z" } ``` -------------------------------- ### Get Google Cloud Storage Bucket Information (R) Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/articles/googleCloudStorageR.html Retrieves detailed information about a specific Google Cloud Storage bucket. This function takes the bucket name as an argument and returns an object containing metadata such as project number, location, class, and creation/update timestamps. ```R library(googleCloudStorageR) bucket <- "your-bucket" bucket_info <- gcs_get_bucket(bucket) ``` -------------------------------- ### Create Google Cloud Storage Buckets Source: https://context7.com/cloudyr/googlecloudstorager/llms.txt Demonstrates how to create storage buckets with specific configurations such as location, storage class, versioning, and lifecycle rules. ```r regional_bucket <- gcs_create_bucket(name = "my-regional-bucket", projectId = "my-project-id", location = "US-EAST1", storageClass = "REGIONAL") versioned_bucket <- gcs_create_bucket(name = "my-versioned-bucket", projectId = "my-project-id", versioning = TRUE) lifecycle_rule <- gcs_create_lifecycle(age = 30) lifecycle_bucket <- gcs_create_bucket(name = "my-lifecycle-bucket", projectId = "my-project-id", location = "EUROPE-NORTH1", storageClass = "REGIONAL", lifecycle = list(lifecycle_rule)) ``` -------------------------------- ### GET error.message Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/error.message.html Retrieves the error message from an object that has failed. ```APIDOC ## GET error.message ### Description Extracts the error message from an object that has failed during a Google Cloud Storage operation. ### Method GET ### Endpoint error.message(test_me) ### Parameters #### Path Parameters - **test_me** (object) - Required - An object that has failed (must satisfy is.error). ### Request Example error.message(my_failed_object) ### Response #### Success Response (200) - **message** (string) - The descriptive error message associated with the failed object. #### Response Example "Error: 404 Not Found" ``` -------------------------------- ### GET /gcs_list_objects Source: https://github.com/cloudyr/googlecloudstorager/blob/master/NEWS.md Lists all objects contained within a specified Google Cloud Storage bucket. ```APIDOC ## GET /gcs_list_objects ### Description Retrieves a list of objects stored in the specified bucket. ### Method GET ### Endpoint /gcs_list_objects ### Parameters #### Query Parameters - **bucket** (string) - Required - The name of the bucket to list. ### Request Example GET /gcs_list_objects?bucket=my-bucket ### Response #### Success Response (200) - **objects** (array) - A list of object metadata objects. #### Response Example { "objects": [ {"name": "file1.csv"}, {"name": "file2.csv"} ] } ``` -------------------------------- ### List Objects in Bucket Source: https://context7.com/cloudyr/googlecloudstorager/llms.txt Demonstrates listing objects with filtering, directory-like navigation using delimiters, and retrieving detailed metadata. ```r gcs_global_bucket("my-bucket") objects <- gcs_list_objects() data_files <- gcs_list_objects(prefix = "data/") root_objects <- gcs_list_objects(delimiter = "/") detailed <- gcs_list_objects(detail = "full") all_versions <- gcs_list_objects(versions = TRUE) ``` -------------------------------- ### GET /gcs_list_objects Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_list_objects.html Lists objects in a specified Google Cloud Storage bucket with options for filtering and detail levels. ```APIDOC ## GET gcs_list_objects ### Description Lists objects in a bucket. Optionally filters by prefix and supports directory-like listing using a delimiter. ### Method GET ### Endpoint gcs_list_objects(bucket, detail, prefix, delimiter) ### Parameters #### Query Parameters - **bucket** (string) - Optional - The name of the bucket containing the objects. Defaults to global bucket. - **detail** (string) - Optional - Set level of detail: 'summary', 'more', or 'full'. - **prefix** (string) - Optional - Filter results to objects whose names begin with this prefix. - **delimiter** (string) - Optional - Use to list objects like a directory listing. ### Request Example ```r gcs_list_objects(bucket = "my-bucket", detail = "summary", prefix = "data/") ``` ### Response #### Success Response (200) - **data.frame** (object) - A data frame containing the list of objects and their metadata based on the requested detail level. #### Response Example ```json { "name": "data/file1.csv", "size": "1024", "updated": "2023-10-01T12:00:00Z" } ``` ``` -------------------------------- ### Authenticate with Google Cloud Storage Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_auth.html Demonstrates various methods for authenticating with Google Cloud Storage, including using a JSON key file, environment variables for custom project credentials, and switching between multiple authenticated email accounts. ```R library(googleCloudStorageR) # Authenticate using a specific JSON file gcs_auth("location_of_json_file.json") # Configure custom project credentials via environment variables Sys.setenv("GAR_CLIENT_JSON" = "location/of/file.json") library(googleCloudStorageR) gcs_auth() # Reauthenticate or switch between email accounts gcs_auth(email = "my@email.com") gcs_list_buckets("my-project") gcs_auth(email = "work@mybusiness.com") gcs_list_buckets("my-project") ``` -------------------------------- ### List Google Cloud Storage Buckets (R) Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/articles/googleCloudStorageR.html Demonstrates how to list all available Google Cloud Storage buckets associated with a given project. This function requires the project ID as an argument and returns a data frame containing bucket information. ```R library(googleCloudStorageR) proj <- "your-project" buckets <- gcs_list_buckets(proj) ``` -------------------------------- ### Automate Session Persistence in R Source: https://context7.com/cloudyr/googlecloudstorager/llms.txt Configure automatic workspace loading and saving on R startup and exit using .Rprofile and _gcssave.yaml. ```r .First <- function() { googleCloudStorageR::gcs_first() } .Last <- function() { googleCloudStorageR::gcs_last() } ``` -------------------------------- ### Download Objects from Cloud Storage Source: https://context7.com/cloudyr/googlecloudstorager/llms.txt Shows how to download objects into R or save them to the local disk, including using custom parsers and retrieving metadata. ```r gcs_global_bucket("my-bucket") data <- gcs_get_object("data.csv") gcs_get_object("report.pdf", saveToDisk = "local_report.pdf") data <- gcs_get_object("gs://my-bucket/data.csv") meta <- gcs_get_object("data.csv", meta = TRUE) model <- gcs_get_object("model.rds", parseFunction = gcs_parse_rds) ``` -------------------------------- ### Get Object Access Control List with gcs_get_object_acl Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/articles/googleCloudStorageR.html Retrieves the current access control list (ACL) for a specified object in Google Cloud Storage. This function is useful for auditing or verifying permissions for specific entities. ```R acl <- gcs_get_object_acl("your-object.csv", entity = "joe@blogs.com") acl$role acl <- gcs_get_object_acl("your-object.csv", entity_type = "allUsers") acl$role ``` -------------------------------- ### POST /buckets Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_create_bucket.html Creates a new Google Cloud Storage bucket within a specified project. ```APIDOC ## POST /buckets ### Description Creates a new bucket in your Google Cloud project with configurable storage classes, locations, and access control lists. ### Method POST ### Endpoint gcs_create_bucket(name, projectId, ...) ### Parameters #### Request Body - **name** (string) - Required - Globally unique name of the bucket to create. - **projectId** (string) - Required - A valid Google project ID. - **location** (string) - Optional - Location of the bucket (default: "US"). - **storageClass** (string) - Optional - Type of bucket (e.g., "STANDARD", "NEARLINE", "COLDLINE"). - **predefinedAcl** (string) - Optional - Predefined access controls for the bucket. - **versioning** (boolean) - Optional - Set if the bucket supports object versioning (default: FALSE). ### Request Example { "name": "my-new-bucket", "projectId": "my-gcp-project", "location": "US", "storageClass": "STANDARD" } ### Response #### Success Response (200) - **bucket** (object) - Returns the metadata of the newly created bucket. #### Response Example { "kind": "storage#bucket", "name": "my-new-bucket", "location": "US", "storageClass": "STANDARD" } ``` -------------------------------- ### Run Roxygen Documentation Generation Source: https://github.com/cloudyr/googlecloudstorager/blob/master/CONTRIBUTING.md This command generates documentation markup using roxygen2. It's essential to run this after making changes to roxygen comments in R files to update the package's documentation. ```bash Rscript -e "devtools::document()" ``` -------------------------------- ### Get Bucket ACL - googleCloudStorageR Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_get_bucket_acl.html Retrieves the Access Control List (ACL) entry for a specified entity on a Google Cloud Storage bucket. This function is useful for inspecting permissions on buckets. It requires the bucket name and optionally accepts an entity and its type. ```R gcs_get_bucket_acl( bucket = gcs_get_global_bucket(), entity = "", entity_type = c("user", "group", "domain", "project", "allUsers", "allAuthenticatedUsers") ) ``` ```R buck_meta <- gcs_get_bucket(projection = "full") acl <- gcs_get_bucket_acl(entity_type = "project", entity = gsub("project-", "", buck_meta$acl$entity[[1]])) ``` -------------------------------- ### Handle NULL arguments in cat0 - R Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/cat0.html Demonstrates the behavior of the cat0 function in the googleCloudStorageR package when an argument is NULL. If an argument is NULL, the function produces no line output, ensuring clean handling of missing or undefined inputs. ```R cat0(prefix = "", x) ``` -------------------------------- ### Manage Bucket Versioning Source: https://context7.com/cloudyr/googlecloudstorager/llms.txt Explains how to check, enable, disable, and list archived object versions within a storage bucket. ```r gcs_global_bucket("my-bucket") gcs_version_bucket("my-bucket", action = "status") gcs_version_bucket("my-bucket", action = "enable") archived <- gcs_version_bucket("my-bucket", action = "list") gcs_version_bucket("my-bucket", action = "disable") ``` -------------------------------- ### Get Error Message from googleCloudStorageR Object Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/error.message.html This function retrieves the error message from an object that has failed during a googleCloudStorageR operation. It takes a single argument, which is the object that encountered the error, and returns the associated error message string. This is useful for debugging failed API calls or data processing steps. ```R error.message(test_me) ``` -------------------------------- ### Download Object from Google Cloud Storage as Raw Data (R) Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/articles/googleCloudStorageR.html Illustrates how to download an object from Google Cloud Storage without automatic parsing, setting parseObject to FALSE. This allows for manual parsing using functions like httr::content() afterwards, providing more control over the data handling process. ```R library(googleCloudStorageR) proj <- "your-project" bucket <- "your-bucket" objects <- gcs_list_objects() # Assumes default bucket is set or specified raw_download <- gcs_get_object(objects$name[[1]], parseObject = FALSE) ``` -------------------------------- ### Get Object Metadata using R Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/articles/googleCloudStorageR.html This R code snippet shows how to retrieve metadata for a specific object stored in Google Cloud Storage. It requires the object name and bucket name as input. By setting the 'meta' argument to TRUE, the function returns detailed information about the object, such as its creation time, size, and content type. ```R gcs_get_object("your-object", "your-bucket", meta = TRUE) ``` -------------------------------- ### Create Bucket Lifecycle Rules in R Source: https://context7.com/cloudyr/googlecloudstorager/llms.txt Demonstrates how to define various lifecycle rules such as age-based deletion, date-based deletion, versioning limits, and non-current version management. These rules are then applied to a bucket during creation using the gcs_create_bucket function. ```R age_rule <- gcs_create_lifecycle(age = 30) date_rule <- gcs_create_lifecycle(createdBefore = "2023-01-01") version_rule <- gcs_create_lifecycle(numNewerVersions = 3) archive_rule <- gcs_create_lifecycle(isLive = FALSE) gcs_create_bucket( name = "managed-bucket", projectId = "my-project", lifecycle = list(age_rule, archive_rule) ) ``` -------------------------------- ### Initialize Google Cloud Storage Object Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/Object.html Constructs an Object representation for Google Cloud Storage. This function accepts various metadata parameters including bucket name, content type, and access control lists. ```R Object(acl = NULL, bucket = NULL, cacheControl = NULL, componentCount = NULL, contentDisposition = NULL, contentEncoding = NULL, contentLanguage = NULL, contentType = NULL, crc32c = NULL, customerEncryption = NULL, etag = NULL, generation = NULL, id = NULL, md5Hash = NULL, mediaLink = NULL, metadata = NULL, metageneration = NULL, name = NULL, owner = NULL, selfLink = NULL, size = NULL, storageClass = NULL, timeCreated = NULL, timeDeleted = NULL, updated = NULL) ``` -------------------------------- ### Google Cloud Storage Lifecycle Management Source: https://context7.com/cloudyr/googlecloudstorager/llms.txt Demonstrates how to create lifecycle rules for Google Cloud Storage buckets, such as deleting objects older than a specified number of days, deleting objects created before a specific date, keeping a limited number of recent versions, or deleting non-current versions. It also shows how to apply these rules when creating a new bucket. ```APIDOC ## Lifecycle Management Examples ### Description Examples of creating and applying lifecycle rules to Google Cloud Storage buckets. ### Method N/A (Illustrative R code) ### Endpoint N/A ### Parameters N/A ### Request Example ```r # Delete objects older than 30 days age_rule <- gcs_create_lifecycle(age = 30) # Delete objects created before a specific date date_rule <- gcs_create_lifecycle(createdBefore = "2023-01-01") # Keep only 3 most recent versions (for versioned buckets) version_rule <- gcs_create_lifecycle(numNewerVersions = 3) # Delete non-current (archived) versions archive_rule <- gcs_create_lifecycle(isLive = FALSE) # Apply multiple rules when creating a bucket gcs_create_bucket( name = "managed-bucket", projectId = "my-project", lifecycle = list(age_rule, archive_rule) ) ``` ### Response N/A (Illustrative R code execution) ``` -------------------------------- ### Retrieve and Parse GCS Objects Source: https://context7.com/cloudyr/googlecloudstorager/llms.txt Demonstrates how to fetch objects from GCS, including using custom parsing functions, downloading specific versions by generation ID, and batch downloading files from a directory. ```R custom_parser <- function(object) { suppressWarnings(httr::content(object, encoding = "UTF-8")) } data <- gcs_get_object("data.csv", parseFunction = custom_parser) old_version <- gcs_get_object("data.csv", generation = "1560468815691234") my_folder <- "reports/" objs <- gcs_list_objects(prefix = my_folder) dir.create(my_folder, showWarnings = FALSE) lapply(objs$name, function(x) gcs_get_object(x, saveToDisk = x)) ``` -------------------------------- ### Upload File via Shiny App using R Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/articles/googleCloudStorageR.html This R code demonstrates how to create a Shiny application that allows users to log in using googleAuthR, select a file, and upload it to Google Cloud Storage. It utilizes the googleCloudStorageR and googleAuthR libraries for authentication and file uploading. The app provides a user interface for file input, naming the object, and submitting the upload, with feedback on the upload status. ```R library("shiny") library("googleAuthR") library("googleCloudStorageR") # options(shiny.port = 1221) # print(source('shiny_test.R')$value) or push the "Run App" button in RStudio shinyApp( ui = shinyUI( fluidPage( googleAuthR::googleAuthUI("login"), fileInput("picture", "picture"), textInput("filename", label = "Name on Google Cloud Storage",value = "myObject"), actionButton("submit", "submit"), textOutput("meta_file") ) ), server = shinyServer(function(input, output, session){ access_token <- shiny::callModule(googleAuth, "login") meta <- eventReactive(input$submit, { message("Uploading to Google Cloud Storage") # from googleCloudStorageR with_shiny(gcs_upload, file = input$picture$datapath, # enter your bucket name here bucket = "gogauth-test", type = input$picture$type, name = input$filename, shiny_access_token = access_token()) }) output$meta_file <- renderText({ req(meta()) str(meta()) paste("Uploaded: ", meta()$name) }) }) ) ``` -------------------------------- ### Set Customer Message Log Level with myMessage Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/myMessage.html Demonstrates how to set the log level for customer messages using the `myMessage` function. The `level` argument controls the severity of messages displayed, with 0 being 'everything' and 3 being 'important'. ```R myMessage(..., level = 1) ``` -------------------------------- ### Object Metadata Management Source: https://context7.com/cloudyr/googlecloudstorager/llms.txt Illustrates how to create metadata objects for Google Cloud Storage uploads, including setting cache control, content disposition, content language, custom metadata key-value pairs, and content encoding. ```APIDOC ## Object Metadata ### gcs_metadata_object Creates metadata objects to attach to uploads, including cache control, content disposition, and custom metadata. ### Method N/A (Illustrative R code) ### Endpoint N/A ### Parameters N/A ### Request Example ```r # Create metadata with cache control metadata <- gcs_metadata_object( cacheControl = "public, max-age=86400", # Cache for 1 day contentDisposition = "attachment; filename=report.pdf", contentLanguage = "en" ) # Upload with metadata gcs_upload("report.pdf", object_metadata = metadata) # Custom metadata key-value pairs custom_meta <- gcs_metadata_object( metadata = list( author = "Data Team", version = "1.0", department = "Analytics" ) ) gcs_upload("analysis.csv", object_metadata = custom_meta) # Specify content encoding for compressed files compressed_meta <- gcs_metadata_object( contentEncoding = "gzip" ) gcs_upload("data.csv.gz", object_metadata = compressed_meta) ``` ### Response N/A (Illustrative R code execution) ``` -------------------------------- ### Configure Environment Variables for GCS Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/articles/googleCloudStorageR.html Sets the default bucket and authentication file path using environment variables. This can be done via R code or in an .Renviron file. ```R Sys.setenv("GCS_DEFAULT_BUCKET" = "my-default-bucket", "GCS_AUTH_FILE" = "/fullpath/to/service-auth.json") ``` -------------------------------- ### Manage GCS Objects: Delete and Copy Source: https://context7.com/cloudyr/googlecloudstorager/llms.txt Shows how to delete objects or entire buckets, and how to copy objects between locations or buckets with optional ACL settings. ```R gcs_global_bucket("my-bucket") gcs_delete_object("old_file.csv") gcs_delete_object("gs://my-bucket/another_file.csv") gcs_delete_object("data.csv", generation = "1560468815691234") gcs_delete_bucket_objects("my-bucket") gcs_copy_object(source_object = "original.csv", destination_object = "backup/original_backup.csv") gcs_copy_object(source_object = "data.csv", destination_object = "data.csv", source_bucket = "source-bucket", destination_bucket = "destination-bucket") ``` -------------------------------- ### Set and Retrieve Global Default Bucket Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/articles/googleCloudStorageR.html Configures a global default bucket to avoid passing the bucket name to every function call, and verifies the current setting. ```R Sys.setenv("GCS_DEFAULT_BUCKET" = "my-default-bucket") library(googleCloudStorageR) gcs_get_global_bucket() ``` -------------------------------- ### Custom Object Function for Upload (R) Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_upload.html Demonstrates how to provide a custom function to transform R objects into a specific file format before uploading to Google Cloud Storage. The custom function receives the R object and the output filename as arguments. ```R f <- function(input, output) { write.csv2(input, file = output) } gcs_upload(mtcars, name = "mtcars_csv2.csv", object_function = f) ``` -------------------------------- ### Generate Download and Signed URLs Source: https://context7.com/cloudyr/googlecloudstorager/llms.txt Creates authenticated or public download URLs for objects, and generates time-limited signed URLs for secure, temporary access. ```R gcs_global_bucket("my-bucket") auth_url <- gcs_download_url("data.csv") public_url <- gcs_download_url("public_file.csv", public = TRUE) obj_meta <- gcs_get_object("confidential_report.pdf", meta = TRUE) signed_url_24h <- gcs_signed_url(obj_meta, expiration_ts = Sys.time() + (24 * 3600)) temp_file <- tempfile(fileext = ".pdf") download.file(signed_url_24h, destfile = temp_file) ``` -------------------------------- ### List Objects in a Google Cloud Storage Bucket (R) Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/articles/googleCloudStorageR.html Lists all objects within a specified Google Cloud Storage bucket. If no bucket is specified, it defaults to the globally set default bucket. The output is a data frame containing object names and other metadata. ```R library(googleCloudStorageR) # Assumes default bucket is set or specified objects <- gcs_list_objects() ``` -------------------------------- ### Loading R objects from Google Cloud Storage Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_save.html Demonstrates how to restore R objects previously saved to a Google Cloud Storage bucket using gcs_load. ```R gcs_load(file = "mydata.RData", bucket = "your_bucket") ``` -------------------------------- ### Load All Files from Google Cloud Storage Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_save_all.html Loads all files from a specified directory in Google Cloud Storage. The files are downloaded and unzipped to the specified or default directory. ```APIDOC ## GET /cloudyr/googlecloudstorager/gcs_load_all ### Description Loads all files from a specified directory in Google Cloud Storage. The files are downloaded and unzipped to the specified or default directory. ### Method GET ### Endpoint /cloudyr/googlecloudstorager/gcs_load_all ### Parameters #### Path Parameters None #### Query Parameters - **directory** (string) - Optional - The Google Cloud Storage directory to download from. Defaults to the current working directory. - **bucket** (string) - Optional - The Google Cloud Storage bucket to download from. Defaults to the global bucket. - **exdir** (string) - Optional - The directory to unzip the files into. Defaults to the `directory` parameter. - **list** (boolean) - Optional - If TRUE, only lists where the files would unzip to without downloading. Defaults to FALSE. ### Request Example ```json { "directory": "path/to/remote/files", "bucket": "your-gcs-bucket", "exdir": "local/destination/path", "list": false } ``` ### Response #### Success Response (200) - **success** (boolean) - TRUE if the download and unzip operation was successful. #### Response Example ```json { "success": true } ``` ``` -------------------------------- ### Set Upload Limit (R) Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_upload.html Configures the size limit in bytes that determines whether a 'simple' or 'resumable' upload type is used. This is useful for optimizing uploads on different network conditions. ```R gcs_upload_set_limit(upload_limit = 5000000L) ``` -------------------------------- ### POST /gcs_create_bucket Source: https://github.com/cloudyr/googlecloudstorager/blob/master/NEWS.md Creates a new bucket in Google Cloud Storage. ```APIDOC ## POST /gcs_create_bucket ### Description Creates a new bucket within the authenticated Google Cloud project. ### Method POST ### Endpoint /gcs_create_bucket ### Parameters #### Request Body - **bucket_name** (string) - Required - The globally unique name for the new bucket. ### Request Example { "bucket_name": "new-unique-bucket-name" } ### Response #### Success Response (200) - **status** (string) - Confirmation of bucket creation. ``` -------------------------------- ### List Pub/Sub notifications for a bucket in R Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_list_pubsub.html Retrieves a list of notification configurations associated with a specified Google Cloud Storage bucket. It defaults to the global bucket if no argument is provided. ```R gcs_list_pubsub(bucket = gcs_get_global_bucket()) ``` -------------------------------- ### List and Retrieve Bucket Metadata Source: https://context7.com/cloudyr/googlecloudstorager/llms.txt Lists available buckets or retrieves detailed metadata for a specific bucket, including storage class and location. ```R buckets <- gcs_list_buckets("my-project-id") bucket_info <- gcs_get_bucket("my-bucket-name") bucket_full <- gcs_get_bucket("my-bucket-name", projection = "full") ``` -------------------------------- ### Download Object to Disk Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/articles/googleCloudStorageR.html Downloads an object from a GCS bucket and saves it directly to a local file path. ```R gcs_get_object(objects$name[[1]], saveToDisk = "csv_downloaded.csv") ``` -------------------------------- ### Create Signed URL for Google Cloud Storage Object (R) Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_signed_url.html Generates a signed URL for a Google Cloud Storage object, providing time-limited access. This URL can be shared with users who may not have a Google account. The function requires a meta object obtained from `gcs_get_object` and allows specifying an expiration timestamp, HTTP verb, and optional MD5 hash. ```R obj <- gcs_get_object("your_file", meta = TRUE) signed <- gcs_signed_url(obj) temp <- tempfile() on.exit(unlink(temp)) download.file(signed, destfile = temp) file.exists(temp) ``` -------------------------------- ### Manage Global Bucket Settings Source: https://context7.com/cloudyr/googlecloudstorager/llms.txt Sets or retrieves the default bucket for the current R session to simplify function calls. ```R gcs_global_bucket("my-bucket-name") current_bucket <- gcs_get_global_bucket() print(current_bucket) gcs_list_objects() gcs_upload(mtcars) ``` -------------------------------- ### Delete Google Cloud Storage Buckets Source: https://context7.com/cloudyr/googlecloudstorager/llms.txt Shows how to delete buckets. Includes the use of the force_delete parameter to remove buckets that are not empty. ```r gcs_delete_bucket("my-empty-bucket") gcs_delete_bucket("my-bucket-with-objects", force_delete = TRUE) ``` -------------------------------- ### Build and Check R Package Source: https://github.com/cloudyr/googlecloudstorager/blob/master/CONTRIBUTING.md Commands to build the R package tarball and check for errors. These steps should be performed before submitting a pull request to ensure the package is in a valid state. ```bash R CMD build googleCloudStorageR ``` ```bash R CMD check googleCloudStorageR_VERSION.tar.gz ``` -------------------------------- ### Create Pub/Sub Notification for GCS Bucket - R Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_create_pubsub.html This function configures a Google Cloud Storage bucket to send notifications to a Pub/Sub topic for supported event types. It requires the Pub/Sub API to be enabled, appropriate permissions, and an existing Pub/Sub topic. The function takes the topic name, project ID, bucket name, and optionally specific event types as input. ```R gcs_create_pubsub( topic, project, bucket = gcs_get_global_bucket(), event_types = NULL ) ``` ```R project <- "myproject" bucket <- "mybucket" # get the email to give access gcs_get_service_email(project) # once email has access, create a new pub/sub topic for your bucket gcs_create_pubsub("gcs_r", project, bucket) ``` -------------------------------- ### Create Pub/Sub Notification for a Bucket Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_create_pubsub.html Configures a Google Cloud Storage bucket to send notifications to a Pub/Sub topic for specified events. ```APIDOC ## POST /storage/v1/b/{bucket}/notification ### Description Creates a Pub/Sub notification configuration for a Google Cloud Storage bucket. This allows for real-time tracking of object changes by publishing messages to a specified Pub/Sub topic. ### Method POST ### Endpoint `/storage/v1/b/{bucket}/notification` ### Parameters #### Path Parameters - **bucket** (string) - Required - The name of the bucket for which to create the notification configuration. #### Query Parameters - **project** (string) - Required - The project ID associated with the Pub/Sub topic. #### Request Body - **topic** (string) - Required - The name of the Pub/Sub topic to which notifications will be sent. - **event_types** (array of strings) - Optional - A list of event types to trigger notifications. If not specified, all supported event types are used. Example: `["OBJECT_FINALIZE", "OBJECT_DELETE"]` ### Request Example ```json { "topic": "projects/myproject/topics/gcs_notifications", "event_types": ["OBJECT_FINALIZE"] } ``` ### Response #### Success Response (200) - **kind** (string) - The type of the resource. - **id** (string) - The ID of the notification configuration. - **selfLink** (string) - The URI of the notification configuration. - **topic** (string) - The Pub/Sub topic name. - **event_types** (array of strings) - The event types for which notifications are sent. #### Response Example ```json { "kind": "storage#notification", "id": "1", "selfLink": "https://storage.googleapis.com/storage/v1/b/mybucket/notificationconfigs/1", "topic": "projects/myproject/topics/gcs_notifications", "event_types": ["OBJECT_FINALIZE"] } ``` ### Details To use this functionality, ensure the Pub/Sub API is enabled for your project, you have sufficient permissions on the bucket and project, an existing Pub/Sub topic, and your service account has at least `pubsub.publisher` permission. ### See Also - [https://cloud.google.com/storage/docs/reporting-changes](https://cloud.google.com/storage/docs/reporting-changes) - `gcs_delete_pubsub()` - `gcs_get_service_email()` - `gcs_list_pubsub()` ``` -------------------------------- ### Upload Objects to GCS Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/articles/googleCloudStorageR.html Demonstrates various ways to upload data to GCS, including files, data frames, and lists. It also shows how to use custom functions for data transformation during the upload process. ```R # Upload a file write.csv(mtcars, file = filename) gcs_upload(filename) # Upload R data.frame directly gcs_upload(mtcars) # Upload R list as JSON gcs_upload(list(a = 1, b = 3, c = list(d = 2, e = 5))) # Upload with custom function f <- function(input, output) write.csv(input, row.names = FALSE, file = output) gcs_upload(mtcars, object_function = f, type = "text/csv") # Upload with bucket-level ACL gcs_upload(mtcars, bucket = "mark-bucketlevel-acl", predefinedAcl = "bucketLevel") ``` -------------------------------- ### Download Object from Google Cloud Storage to R Object (R) Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/articles/googleCloudStorageR.html Shows how to download an object from a Google Cloud Storage bucket directly into an R object. The function gcs_get_object() attempts to guess the appropriate R object type for parsing. Be cautious with large objects to avoid running out of memory. ```R library(googleCloudStorageR) proj <- "your-project" bucket <- "your-bucket" objects <- gcs_list_objects() # Assumes default bucket is set or specified parsed_download <- gcs_get_object(objects$name[[1]]) ``` -------------------------------- ### POST /substitute.list Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/substitute.list.html Performs a substitution operation on a nested list structure using a template and a replacement list. ```APIDOC ## POST /substitute.list ### Description This function takes a template named list and a replacement named list to substitute values within the nested structure of the template. ### Method POST ### Endpoint /substitute.list ### Parameters #### Request Body - **template** (list) - Required - A template named list containing the structure to be populated. - **replace_me** (list) - Required - A named list containing the values to substitute into the template. ### Request Example { "template": {"bucket": "{{bucket_name}}", "config": {"timeout": "{{timeout_val}}"}}, "replace_me": {"bucket_name": "my-storage-bucket", "timeout_val": "30"} } ### Response #### Success Response (200) - **result** (list) - The template list with values substituted from replace_me. #### Response Example { "bucket": "my-storage-bucket", "config": { "timeout": "30" } } ``` -------------------------------- ### Source R Script from Google Cloud Storage Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_source.html This function downloads an R script from Google Cloud Storage and executes it immediately using the `source()` function. ```APIDOC ## gcs_source ### Description Download an R script and run it immediately via [source](https://rdrr.io/r/base/source.html) ### Method Not applicable (R function) ### Endpoint Not applicable (R function) ### Parameters #### Arguments - **script** (character) - The name of the script on GCS - **bucket** (character) - Bucket the stored objects are in. Defaults to the global bucket obtained via `gcs_get_global_bucket()`. - **...** - Additional arguments passed to the `source()` function. ### Request Example ```R gcs_source("my_script.R", bucket = "my-gcs-bucket") ``` ### Response #### Success Response - **TRUE** (logical) - Indicates that the script was successfully sourced. #### Response Example ```R TRUE ``` ### See Also - `gcs_load()` - `gcs_save_all()` - `gcs_save_image()` - `gcs_save()` ``` -------------------------------- ### Saving R objects to Google Cloud Storage Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_save.html Demonstrates how to save specific R objects into a .RData file and upload them to a Google Cloud Storage bucket using gcs_save. ```R gcs_save(ob1, ob2, ob3, file = "mydata.RData") ``` -------------------------------- ### Manage R Workspace Backups Source: https://context7.com/cloudyr/googlecloudstorager/llms.txt Save and restore entire R workspaces or directories to Cloud Storage for backup and recovery purposes. ```r gcs_global_bucket("my-bucket") # Save entire R workspace gcs_save_image() # Save all files in a directory (zipped) gcs_save_all( directory = "my-project-folder", pattern = "\\.(R|csv|rds)$" ) # Load/restore all files from cloud gcs_load_all(directory = "my-project-folder", exdir = "restored-folder") # Delete cloud backup gcs_delete_all(directory = "old-backup") ``` -------------------------------- ### Generate Google Cloud Storage Download URL (R) Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_download_url.html Generates a download URL for a specified object in a Google Cloud Storage bucket. The function can create public URLs or authenticated URLs, depending on the 'public' argument. Bucket names should match the length of object names or be a single value. ```R gcs_download_url(object_name, bucket = gcs_get_global_bucket(), public = FALSE) ``` -------------------------------- ### List Google Cloud Storage Buckets (R) Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_list_buckets.html Lists buckets accessible by a specified project ID. Supports filtering by prefix and controlling the level of detail returned. The function returns a data.frame containing bucket information. ```R buckets <- gcs_list_buckets("your-project") ## use the name of the bucket to get more meta data bucket_meta <- gcs_get_bucket(buckets$name[[1]]) ``` -------------------------------- ### Cloud Script Execution Source: https://context7.com/cloudyr/googlecloudstorager/llms.txt Enables sourcing R scripts directly from Cloud Storage buckets. ```APIDOC ## GET /gcs_source ### Description Sources an R script file stored in Google Cloud Storage into the current R session. ### Method GET ### Parameters #### Query Parameters - **file** (string) - Required - Path to the R script in the bucket. - **bucket** (string) - Optional - The bucket name if not using the global default. ### Response #### Success Response (200) - **status** (string) - Execution status of the sourced script. ``` -------------------------------- ### Create a New GCS Bucket Source: https://context7.com/cloudyr/googlecloudstorager/llms.txt Creates a new storage bucket within a specified Google Cloud project. ```R new_bucket <- gcs_create_bucket( name = "my-new-bucket-unique-name", projectId = "my-project-id" ) ``` -------------------------------- ### Automatically Save/Load R Session with gcs_first/gcs_last Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_first.html These functions are designed to be placed in your .Rprofile file to automatically save your R session to Google Cloud Storage on exit and load it on startup. They utilize environment variables or a _gcssave.yaml file for configuration, specifying the GCS bucket and optional loading directories or file patterns. The functions call gcs_save_all and gcs_load_all internally. ```R .First <- function(){ googleCloudStorageR::gcs_first() } .Last <- function(){ googleCloudStorageR::gcs_last() } ``` -------------------------------- ### Create Google Cloud Storage Bucket - R Source: https://github.com/cloudyr/googlecloudstorager/blob/master/docs/reference/gcs_create_bucket.html The gcs_create_bucket function creates a new bucket in Google Cloud Storage. It requires a unique bucket name and a project ID. Optional arguments control the bucket's location, storage class, access controls, versioning, and lifecycle rules. The function returns information about the created bucket. ```R gcs_create_bucket( name, projectId, location = "US", storageClass = c("MULTI_REGIONAL", "REGIONAL", "STANDARD", "NEARLINE", "COLDLINE", "DURABLE_REDUCED_AVAILABILITY"), predefinedAcl = c("projectPrivate", "authenticatedRead", "private", "publicRead", "publicReadWrite"), predefinedDefaultObjectAcl = c("bucketOwnerFullControl", "bucketOwnerRead", "authenticatedRead", "private", "projectPrivate", "publicRead"), projection = c("noAcl", "full"), versioning = FALSE, lifecycle = NULL ) ```