===============
LIBRARY RULES
===============
From library maintainers:
- Anyscale runs on top of Ray. For Ray framework APIs (Ray Core, Ray Data, Ray Train, Ray Serve, Ray Tune), prefer the ray-project/ray library.

### Install and Run Vector Locally

Source: https://docs.anyscale.com/monitoring/exporting-logs.md

Installs Vector, sets up necessary directories, and starts Vector with a specified configuration file. Also includes a command to generate fake log content for testing.

```bash
# Install Vector.
sudo apt-get install curl -y
curl --proto '=https' --tlsv1.2 -sSfL https://sh.vector.dev | bash
source /home/ray/.profile

# Create a state directory for Vector & make it accessible.
sudo mkdir -p /var/lib/vector/
sudo chmod 777 /var/lib/vector/

# Run Vector
vector --config vector.yaml

# In a new tab, generate fake log content.
mkdir -p /tmp/ray/session_fake/logs/
for i in {1..5000}; do echo "Log Line $i" >> /tmp/ray/session_fake/logs/job-driver-fake.log && echo "Wrote line $i" && sleep 1; done

# Look for warnings / errors in Vector - if you don't see any, check upstreams to see if logs & metrics are being received.
```

--------------------------------

### Anyscale Cloud Deletion Example (Auto Setup)

Source: https://docs.anyscale.com/clouds/aws/manage.md

This example demonstrates the command and output for deleting an Anyscale cloud set up using anyscale cloud setup. It shows the confirmation prompt and the process of deleting cloud resources via CloudFormation. Note that the S3 bucket associated with the cloud is not automatically deleted.

```bash
anyscale cloud delete --name example_cloud_name
```

```text
Authenticating
Loaded Anyscale authentication token from ANYSCALE_CLI_TOKEN.

Output
If the cloud cld_prcjv8jc9tmbv3q54mc2h7dnl6 is deleted, you won't be able to access existing clusters of this cloud.
For more information, please refer to the documentation https://docs.anyscale.com/user-guide/onboard/clouds#cloud-deletion
Continue? [y/N]: y
(anyscale +6.8s)
Track progress of cloudformation at https://ap-southeast-1.console.aws.amazon.com/cloudformation/home?region=ap-southeast-1#/stacks/stackinfo?stackId=arn:aws:cloudformation:ap-southeast-1:123456:stack/cld-prcjv8jc32r89fniuf23/123456789
⠸ Deleting cloud resources through cloudformation...(anyscale +49.1s) Cloudformation stack arn:aws:cloudformation:ap-southeast-1:815664363732:stack/cld-prcjv8jc32r89fniuf23/123456789 is deleted.
(anyscale +49.1s)
The S3 bucket (anyscale-production-data-cld-prcjv8jc32r89fniuf23) associated with this cloud still exists.
If you no longer need the data associated with this bucket, please delete it.
(anyscale +49.4s) Deleted cloud with name example_cloud_name.
```

--------------------------------

### Get Project Example

Source: https://docs.anyscale.com/reference/sdk/project.md

Example of how to retrieve a project using its ID.

```APIDOC
Examples:
```python
import anyscale
from anyscale.project.models import Project

project: Project = anyscale.project.get(project_id="my-project-id")
```
```

--------------------------------

### Install Editable Python Package from Workspace

Source: https://docs.anyscale.com/dependency-management/containerfiles.md

This example shows how to install an editable Python package named 'my_module' located within the workspace directory. It first copies the working directory and then installs the package.

```dockerfile
FROM anyscale/ray:2.55.1-slim-py312-cu128

COPY --chown=ray:ray working_dir /home/ray/default
RUN cd /home/ray/default/my_module && pip install -e .
```

--------------------------------

### Clone Example Repository

Source: https://docs.anyscale.com/tutorials/deploy-llama-3-1-70b.md

Clone the example repository containing the deployment configuration and application code. Navigate into the cloned directory.

```bash
git clone https://github.com/anyscale/examples.git
cd examples/deploy_llama_3_1_70b
```

--------------------------------

### Anyscale CLI, SDK, and YAML Configuration Example

Source: https://docs.anyscale.com/reference/index

Demonstrates how to specify the working directory across CLI, SDK, and YAML configurations.

```plaintext
CLI: `--working-dir ./path`
SDK model: `working_dir="./path"`
YAML config for CLI: `working_dir: "./path"`
```

--------------------------------

### Clone and Install Editable Package from GitHub

Source: https://docs.anyscale.com/dependency-management/containerfiles.md

This example clones a GitHub repository into the workspace and then installs it as an editable package. It includes a check to avoid re-cloning if the directory already exists.

```dockerfile
# Mount the working directory of the workspace into the image.
COPY --chown=ray:ray working_dir /home/ray/default/

# Check if Git repo is cloned and clone if not. 
RUN cd /home/ray/default && \
    [ -d sampleproject ] || git clone https://github.com/pypa/sampleproject

# Install Python module from cloned repo.
RUN cd /home/ray/default/sampleproject && pip install -e .
```

--------------------------------

### Clone the Example Repository

Source: https://docs.anyscale.com/tutorials/video-curation.md

Clones the Anyscale examples repository and navigates into the video curation directory.

```bash
git clone https://github.com/anyscale/examples.git
cd examples/video_curation
```

--------------------------------

### Test Anyscale CLI Installation

Source: https://docs.anyscale.com/reference/quickstart-cli.md

Verify that the Anyscale CLI is installed and configured correctly by listing your owned jobs. A successful execution without errors indicates proper setup.

```bash
anyscale job list
```

--------------------------------

### Example Project Creation Output

Source: https://docs.anyscale.com/reference/quickstart-sdk.md

This is an example of the success output when a project is created.

```bash
Project created successfully: https://console.anyscale.com/projects/prj_1234
```

--------------------------------

### Compute Config CLI Get

Source: https://docs.anyscale.com/reference/compute-config-api

Example of retrieving a compute configuration using the Anyscale CLI.

```APIDOC
## Compute Config CLI Get

### Description
Retrieves a compute configuration using the Anyscale CLI.

### Command
`anyscale compute-config get -n <config-name>`

### Example
```bash
$ anyscale compute-config get -n my-compute-config
name: my-compute-config:1
id: cpt_r4b4b3621rl3uggg7llj3mvme6
config:
  cloud: my-cloud
  head_node:
    instance_type: m5.8xlarge
  worker_nodes:
  - instance_type: m5.8xlarge
    min_nodes: 5
    max_nodes: 5
  - instance_type: m5.4xlarge
    min_nodes: 1
    max_nodes: 10
    market_type: SPOT
```
```

--------------------------------

### Set up a Cloud Provider using anyscale cloud setup

Source: https://docs.anyscale.com/reference/cloud

Use this command to configure a new cloud provider. Specify the provider type, region, and optionally a name, compute stack, and other provider-specific details. The `--functional-verify` flag can be used to immediately check if the cloud is operational.

```bash
anyscale cloud setup --provider aws --region us-east-1 --name my-aws-cloud --stack vm --anyscale-managed --functional-verify
```

```bash
anyscale cloud setup --provider azure --region eastus --name my-azure-cloud --stack k8s --cluster-name my-k8s-cluster --namespace anyscale --values-file ./helm-values.yaml
```

```bash
anyscale cloud setup --provider gcp --region us-west1 --name my-gcp-cloud --project-id "my-gcp-project-123" --stack vm --shared-storage object-storage --enable-head-node-fault-tolerance
```

```bash
anyscale cloud setup --provider aws --region eu-west-2 --name another-aws-cloud --stack k8s --cluster-name "prod-cluster" --namespace "anyscale-prod" --skip-resources
```

```bash
anyscale cloud setup --provider azure --region westus2 --name "azure-dev" --stack vm --resource-group "dev-rg" --disable-auto-add-user --yes
```

--------------------------------

### Set up User and Virtual Environment

Source: https://docs.anyscale.com/container-image/image-requirement.md

Initializes the 'ray' user, creates a system-wide Python virtual environment, and updates the PATH.

```bash
su --login ray
python3 -m venv --system-site-packages /home/ray/virtualenv
export PATH=/home/ray/virtualenv/bin:$PATH
```

--------------------------------

### Configure Workload Timeouts in Cluster Settings

Source: https://docs.anyscale.com/configuration/compute/advanced.md

Adjust workload starting and recovering timeouts using JSON configuration. This example sets the starting timeout to 1 hour and the recovering timeout to 10 minutes.

```json
{
  "workload_starting_timeout": "1h",
  "workload_recovering_timeout": "10m"
}
```

--------------------------------

### Configure Workload Timeouts via CLI Flags

Source: https://docs.anyscale.com/configuration/compute/advanced.md

Set workload starting and recovering timeouts using the `flags` field in the CLI configuration. This example increases the starting timeout to 1 hour and decreases the recovering timeout to 10 minutes.

```yaml
cloud: <cloud-name>
head_node:
  instance_type: <head-instance-type>
worker_nodes:
  - instance_type: <worker-instance-type>
flags:
  workload_starting_timeout: 1h
  workload_recovering_timeout: 10m
```

--------------------------------

### Clone Example Repository

Source: https://docs.anyscale.com/tutorials/fineweb-dedup.md

Clone the Anyscale examples repository and navigate to the fineweb_dedup directory.

```bash
git clone https://github.com/anyscale/examples.git
cd examples/fineweb_dedup
```

--------------------------------

### Amazon S3 Dataset Configuration

Source: https://docs.anyscale.com/llm/fine-tuning/data-preparation.md

Example configuration for using a dataset stored in Amazon S3. Requires the 's3fs' library to be installed.

```json
{
  "my-custom-sft-s3": {
    "cloud_file_name": "s3://my-bucket/alpaca_en_demo.json",
    "columns": {
      "prompt": "instruction",
      "query": "input",
      "response": "output"
    }
  }
}
```