Onyxia

https://docs.onyxia.sh/
Admin
Onyxia is an open-source cloud-native data science platform for Kubernetes that provides users with...
Tokens:60,133
Snippets:366
Trust Score:7.1
Update:1 week ago
Show doc for...
Tokens:
Raw
### Clone and run Onyxia web app locally

Source: https://docs.onyxia.sh/contributors-doc/onyxia

Commands to clone the Onyxia repository, navigate to the web directory, install dependencies, and start the development server. Optionally, edit the .env.local.yaml file (created automatically on first run) to test against your own Onyxia instance.

```bash
git clone https://github.com/InseeFrLab/onyxia
cd onyxia/web

yarn install

# To start the app locally
yarn dev

# If you want to test against your own Onyxia instance edit the .env.local.yaml
# file (created automatically the first time you run `yarn dev`)
```

--------------------------------

### Install Onyxia with custom values and remove manual version tags

Source: https://docs.onyxia.sh/release-notes/migrating-to-the-new-helm-repo

Shows the new installation workflow: add the new repo, create a values file without the web and api image tags, and install the chart. The version is now managed solely by the chart version.

```bash
helm repo add onyxia https://inseefrlab.github.io/onyxia

DOMAIN=my-domain.net

cat << EOF > ./onyxia-values.yaml
# ...
web:
  image:
-   tag: 2.29.4
api:
  image:
-   tag: v0.32   
# ...
EOF

helm install onyxia onyxia/onyxia -f onyxia-values.yaml
```

--------------------------------

### Initialize Vault in dev mode with Helm

Source: https://docs.onyxia.sh/admin-doc/readme/vault

Commands to add the HashiCorp Helm repo, set the domain, create a vault-values.yaml file for dev mode with ingress, and install Vault. This is for initial setup only; do not use dev mode in production.

```bash
helm repo add hashicorp https://helm.releases.hashicorp.com
 
DOMAIN=my-domain.net

cat << EOF > ./vault-values.yaml
server:
  dev:
    enabled: true
    # Set VAULT_DEV_ROOT_TOKEN_ID value
    devRootToken: "root"
  ingress:
    enabled: true
    annotations:
      kubernetes.io/ingress.class: nginx
    hosts:
      - host: "vault.lab.$DOMAIN"
    tls:
      - hosts:
          - vault.lab.$DOMAIN
EOF

helm install vault hashicorp/vault -f vault-values.yaml
```

--------------------------------

### Example Markdown document for onboarding

Source: https://docs.onyxia.sh/admin-doc/custom-pages

Example content of onboarding_en.md, a Markdown document that can be linked from the UI. It demonstrates standard Markdown, HTML image embedding, code snippets, and links to other pages.

```markdown
# This is a test document in english

This could be for example a guide specific to your Onyxia instance.  

## It's standard markdown

You can embed images, including with HTML syntax:  

<img src="%PUBLIC_URL%/custom-resources/preview.png" width="100%">  

You can render code snippets:  

```bash
echo "Hello world"
```

You can also link to pages of your instance: [Catalog](/catalog).

You can link to [another document](%PUBLIC_URL%/custom-resources/onboarding_sub_en.md).

<a href="%PUBLIC_URL%/launcher/ide/rstudio?name=rstudio&version=2.3.2&s3=region-ec97c721&resources.limits.cpu=«22700m»&autoLaunch=true">
    <img height=20 src="https://user-images.githubusercontent.com/6702424/173724486-30b6232a-c5d2-40da-a0cc-4d4a11824135.png">
</a>
```

--------------------------------

### Install Onyxia with Helm

Source: https://docs.onyxia.sh/admin-doc/readme/s3-explorer-standalone-deployment

Adds the Onyxia Helm repository, updates it, and installs or upgrades the Onyxia release in the 'onyxia' namespace using the provided values file.

```bash
helm repo add onyxia https://inseefrlab.github.io/onyxia
helm repo update

helm upgrade --install onyxia onyxia/onyxia \
  --namespace onyxia \
  --create-namespace \
  --values onyxia-values.yaml

```

--------------------------------

### Obtain TLS certificate with certbot

Source: https://docs.onyxia.sh/admin-doc/readme/kubernetes

Installs certbot and runs a manual DNS challenge to obtain a wildcard certificate for datalab.my-domain.net and *.lab.my-domain.net. Requires sudo and manual DNS TXT record setup.

```bash
brew install certbot #On Mac, lookup how to install certbot for your OS

#Because we need a wildcard certificate we have to complete the DNS callange.  
sudo certbot certonly --manual --preferred-challenges dns

# When asked for the domains you wish to optains a certificate for enter:
#   datalab.my-domain.net *.lab.my-domain.net
```

--------------------------------

### Update Helm repository and install command

Source: https://docs.onyxia.sh/release-notes/migrating-to-the-new-helm-repo

Shows the diff for switching the Helm repository from inseefrlab/helm-charts to onyxia and updating the install command to use onyxia/onyxia.

```bash
-helm repo add inseefrlab https://inseefrlab.github.io/helm-charts
+helm repo add onyxia https://inseefrlab.github.io/onyxia

-helm install onyxia inseefrlab/helm-charts
+helm install onyxia onyxia/onyxia
```

--------------------------------

### Helm oneliner to install Onyxia with minimal configuration

Source: https://docs.onyxia.sh/admin-doc

Adds the Onyxia Helm repository, creates a minimal values file with an ingress host, and installs the chart. This results in a degraded mode instance lacking authentication, S3 explorer, and secret management, but still allows launching services from the catalog.

```bash
helm repo add onyxia https://inseefrlab.github.io/onyxia

cat << EOF > ./onyxia-values.yaml
ingress:
  enabled: true
  hosts:
    - host: onyxia.my-domain.net
EOF

helm install onyxia onyxia/onyxia -f onyxia-values.yaml

# Navigate to https://onyxia.my-domain.net
```

--------------------------------

### Install ingress-nginx with custom values

Source: https://docs.onyxia.sh/admin-doc/readme/kubernetes

Install ingress-nginx using Helm with a custom values file that sets the default SSL certificate and watches ingress objects without a class. This enables SSL termination for both wildcard and datalab domains.

```bash
cat << EOF > ./ingress-nginx-values.yaml
controller:
  extraArgs:
    default-ssl-certificate: "ingress-nginx/onyxia-tls"
  watchIngressWithoutClass: true
EOF

helm install ingress-nginx ingress-nginx \
    --repo https://kubernetes.github.io/ingress-nginx \
    --version 4.9.1 \
    --namespace ingress-nginx \
    -f ./ingress-nginx-values.yaml
```

--------------------------------

### Install ArgoCD with Helm and configure ingress

Source: https://docs.onyxia.sh/admin-doc/readme/gitops

Sets up ArgoCD using Helm with a custom values file that configures the ingress for argocd.lab.my-domain.net. Requires a domain and DNS pointing to the cluster.

```bash
DOMAIN=my-domain.net

cat << EOF > ./argocd-values.yaml
server:
  extraArgs:
    - --insecure
  ingress:
    #ingressClassName: nginx
    enabled: true
    hostname: argocd.lab.$DOMAIN
    extraTls:
      - hosts:
          - argocd.lab.$DOMAIN
EOF

helm install argocd argo-cd \
  --repo https://argoproj.github.io/argo-helm \
  --version 6.0.9 \
  -f ./argocd-values.yaml
```

--------------------------------

### Link onyxia-ui locally in onyxia-web

Source: https://docs.onyxia.sh/contributors-doc/onyxia/dependencies

Steps to link onyxia-ui locally in onyxia-web. Clone onyxia, install its dependencies, then clone onyxia-ui, build it, link it with yarn link-in-web, and run TypeScript watch. After linking, start the web app to see live updates. If installing/updating dependencies, remove node_modules and link again.

```shell
cd ~/github

```

```shell
git clone https//github.com/InseeFrLab/onyxia

```

```shell
cd onyxia/web
yarn install

```

```shell
cd ~/github/onyxia #This is just a suggestion, clone wherever you see fit.
git clone https://github.com/InseeFrLab/onyxia-ui ui
cd ui
yarn install
yarn build
yarn link-in-web
npx tsc -w

```

```shell
# Open a new terminal
cd ~/github/onyxia/web
yarn start

```

--------------------------------

### Install ingress-nginx on AWS

Source: https://docs.onyxia.sh/admin-doc/readme/kubernetes

Applies the ingress-nginx controller manifest for AWS. Use the GCP or Azure commands for those providers.

```bash
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.2.0/deploy/static/provider/aws/deploy.yaml
```

--------------------------------

### Example user profile context values

Source: https://docs.onyxia.sh/admin-doc/catalog-of-services/custom-catalogs/declarative-user-profile

Shows the structure of xOnyxiaContext.user.profile after the user fills the form. The values are accessible programmatically and can be used in charts and deployments.

```json
{
  "generalInfo": {
    "firstName": "Joseph",
    "lastName": "Garrone",
    "email": "joseph.garrone@code.gouv.fr"
  },
  "git": {
    "username": "garronej",
    "email": "joseph.garrone.gj@gmail.com"
  }
}
```

--------------------------------

### Create DNS CNAME records for Onyxia

Source: https://docs.onyxia.sh/admin-doc/readme/kubernetes

Alternative DNS records using CNAME for DDNS domains. Replace the example domain with your own.

```bash
datalab.my-domain.net CNAME jhon-doe-home.ddns.net.
*.lab.my-domain.net   CNAME jhon-doe-home.ddnc.net.
```

--------------------------------

### Update Onyxia configuration to enable MinIO

Source: https://docs.onyxia.sh/admin-doc/readme/data-s3

Clone the onyxia-ops repository, navigate to the Onyxia app directory, and replace the configuration file with `values-minio-enabled.yaml.yaml` (or `values-s3-explorer-only.yaml` for standalone S3 Explorer mode). Commit and push to enable MinIO.

```bash
git clone https://github.com/<your-github-org>/onyxia-ops
cd onyxia-ops
cd apps/onyxia
mv values-minio-enabled.yaml.yaml values.yaml
# OR: For deploying Onyxia in s3 Explorer standalone mode:
# mv values-s3-explorer-only.yaml values.yaml
git commit -am "Enable MinIO"
git push
```

--------------------------------

### overwriteDefaultWith example in values.schema.json

Source: https://docs.onyxia.sh/admin-doc/catalog-of-services/custom-catalogs/onyxia-extension

Example of using the x-onyxia overwriteDefaultWith option in a values.schema.json file. The default value is dynamically generated by concatenating project.id, k8s.randomSubdomain, and k8s.domain using the {{}} syntax.

```json
"hostname": {
  "type": "string",
  "form": true,
  "title": "Hostname",
  "x-onyxia": {
    "overwriteDefaultWith": "{{project.id}}-{{k8s.randomSubdomain}}.{{k8s.domain}}"
  }
}
```

--------------------------------

### Using useThunks to get isUserLoggedIn (recommended approach)

Source: https://docs.onyxia.sh/contributors-doc/onyxia/architecture

Shows the recommended way to get `isUserLoggedIn` via thunks, because the value is constant from the app's perspective (login redirects and reloads the app).

```typescript
const { userAuthenticationThunks } = useThunks();
const isUserLoggedIn = userAuthenticationThunks.getIsUserLoggedIn();
```

--------------------------------

### S3 OIDC ID token claim example

Source: https://docs.onyxia.sh/admin-doc/s3-configuration

Example JSON object in an S3 OIDC ID token containing the preferred_username claim value 'alice'. This claim is used by the bookmark to resolve to the bucket name.

```json
{
  "preferred_username": "alice"
}
```

--------------------------------

### Launch Storybook locally

Source: https://docs.onyxia.sh/contributors-doc/onyxia/dependencies

Launches Storybook locally to test graphical components in isolation. Run from the web directory.

```shell
yarn storybook

```

--------------------------------

### Create DNS A records for Onyxia

Source: https://docs.onyxia.sh/admin-doc/readme/kubernetes

Create DNS A records for the Onyxia datalab and wildcard subdomain. Replace <YOUR_IP> with your internet box routable IP.

```bash
datalab.my-domain.net A <YOUR_IP>
*.lab.my-domain.net   A <YOUR_IP>
```

--------------------------------

### overwriteListEnumWith example in values.schema.json

Source: https://docs.onyxia.sh/admin-doc/catalog-of-services/custom-catalogs/onyxia-extension

Example of using the x-onyxia overwriteListEnumWith option to dynamically generate the options for a select field. The default is set to the first group from the user's decoded ID token, and the list is populated from the groups array.

```json
"group": {
  "type": "string",
  "default": "",
  "listEnum": [""],
  "x-onyxia": {
    "overwriteDefaultWith": "{{user.decodedIdToken.groups[0]}}",
    "overwriteListEnumWith": "{{user.decodedIdToken.groups}}"
  }
}
```

--------------------------------

### Onyxia Configuration for S3 Explorer (onyxia-values.yaml)

Source: https://docs.onyxia.sh/admin-doc/readme/s3-explorer-standalone-deployment

Helm values for Onyxia to enable the S3 Explorer with MinIO, including the web environment variables and a bookmark that uses the preferred_username claim.

```yaml
ingress:
  enabled: true
  hosts:
    - host: onyxia.example.com

web:
  env:
    HEADER_TEXT_FOCUS: "S3 Explorer"
    S3: |
      {
        URL: "https://minio.example.com",
        region: "us-east-1",
        pathStyleAccess: true,
        sts: {
          role: {
            profileName: "default",
            roleARN: "",
            roleSessionName: ""
          },
          oidcConfiguration: {
            issuerURI: "https://auth.example.com/realms/my-realm",
            clientID: "onyxia-minio"
          }
        },
        bookmarks: [
          {
            s3Uri: "s3://user-$1/",
            title: { en: "Personal bucket", fr: "Bucket personnel" },
            claimName: "preferred_username",
            forProfileName: "default"
          }
        ]
      }

api:
  enabled: false
```

--------------------------------

### Replace workingDirectory with bookmarks (bucketMode: multi)

Source: https://docs.onyxia.sh/release-notes/v10-greater-than-v11

Shows the v10 to v11 migration for S3 configuration when bucketMode is 'multi'. The old workingDirectory block is replaced with a bookmarks array, each entry using s3Uri with $1 substitution for claimName values.

```yaml
 onyxia:
   api:
     regions:
       [
         {
           "data": {
             "S3": {
-              "workingDirectory": {
-                "bucketMode": "multi",
-                "bucketNamePrefix": "",
-                "bucketNamePrefixGroup": "projet-"
-              },
+              "bookmarks": [
+                {
+                  "s3Uri": "s3://$1/",
+                  "title": "Personal",
+                  "claimName": "preferred_username"
+                },
+                {
+                  "s3Uri": "s3://projet-$1/",
+                  "title": "Group $1",
+                  "claimName": "groups"
+                }
+              ]
             }
           }
         }
       ]
```

--------------------------------

### Get ArgoCD initial admin password

Source: https://docs.onyxia.sh/admin-doc/readme/gitops

Retrieves the auto-generated admin password for ArgoCD. Wait for all pods to be ready before running.

```bash
kubectl get secret argocd-initial-admin-secret \
  -o jsonpath="{.data.password}" | base64 -d
```

--------------------------------

### Deploy MinIO: clone onyxia-ops and set OIDC client secret

Source: https://docs.onyxia.sh/admin-doc/readme/data-s3

Clone the onyxia-ops repository, navigate to the MinIO app directory, and replace the `$KEYCLOAK_MINIO_CLIENT_SECRET` placeholder in values.yaml with the copied client secret. Commit and push the change to deploy MinIO.

```bash
git clone https://github.com/<your-github-org>/onyxia-ops
cd onyxia-ops
cd apps/minio
# In the values.yaml file replace `$KEYCLOAK_MINIO_CLIENT_SECRET` by the value
# you have copied in the previous step.
git commit -am "Set minio OIDC client secret"
git push
```

--------------------------------

### Rename bookmarkedDirectories to bookmarks

Source: https://docs.onyxia.sh/release-notes/v10-greater-than-v11

Shows the migration from the old 'bookmarkedDirectories' key to the new 'bookmarks' key, and the change from 'fullPath' to 's3Uri' with the new path format.

```yaml
 onyxia:
   api:
     regions:
       [
         {
           "data": {
             "S3": {
-              "bookmarkedDirectories": [
+              "bookmarks": [
                 {
-                  "fullPath": "donnees-insee/diffusion/",
+                  "s3Uri": "s3://donnees-insee/diffusion/",
                   "title": {
                     "fr": "Données de diffusion",
                     "en": "Dissemination Data"
                   }
                 }
               ]
             }
           }
         }
       ]
```
Tokens:
Raw