Fly.io Postgres Elixir

https://github.com/superfly/fly_postgres_elixir
Admin
Fly Postgres facilitates geographically distributed Elixir applications by managing primary/replica...
Tokens:2,363
Snippets:22
Trust Score:9.4
License:Apache-2.0
Update:1 year ago
Show doc for...
Tokens:
Raw
### Install Elixir dependencies

Source: https://github.com/superfly/fly_postgres_elixir/blob/main/README.md

Run `mix deps.get` to fetch and install the `fly_postgres` package and its dependencies.

```Shell
mix deps.get
```

--------------------------------

### Elixir Application Supervision Tree Update for Single Ecto Repo

Source: https://github.com/superfly/fly_postgres_elixir/blob/main/README.md

This Elixir code demonstrates how to modify the `start` function within `MyApp.Application` to integrate `Fly.RPC` and `Fly.Postgres.LSN.Supervisor` into the application's supervision tree. It also ensures the primary Ecto repository (`MyApp.Repo.Local`) is started, which is essential for `fly_postgres_elixir` functionality.

```Elixir
defmodule MyApp.Application do
  use Application

  def start(_type, _args) do
    # ...

    children = [
      # Start the RPC server
      {Fly.RPC, []},
      # Start the Ecto repository
      MyApp.Repo.Local,
      # Start the supervisor for LSN tracking
      {Fly.Postgres.LSN.Supervisor, repo: MyApp.Repo.Local},
      #...
    ]

    # ...
  end
end
```

--------------------------------

### Original Ecto Repo definition

Source: https://github.com/superfly/fly_postgres_elixir/blob/main/README.md

Example of a standard `Ecto.Repo` definition before integrating `Fly.Postgres`.

```Elixir
defmodule MyApp.Repo do
  use Ecto.Repo,
    otp_app: :my_app,
    adapter: Ecto.Adapters.Postgres
end
```

--------------------------------

### Elixir: Performing Ecto Operations in Migrations Using Local Repo

Source: https://github.com/superfly/fly_postgres_elixir/blob/main/README.md

Explains why standard `MyApp.Repo` calls fail in Ecto migrations due to the application's Tracker not being started. It provides the solution of using `MyApp.Repo.Local` for database modifications within migrations, as migrations run in the primary region with direct database access, ensuring safe execution.

```Elixir
MyApp.Repo.Local.insert(...) or MyApp.Repo.Local.update(...)
```

--------------------------------

### Elixir Interacting with PostgreSQL LSN Functions

Source: https://github.com/superfly/fly_postgres_elixir/blob/main/DEV_NOTES.md

Elixir code examples demonstrating how to query PostgreSQL for current and last replayed LSN values, cast them to text, and use them in calls to a stored procedure like `watch_for_lsn_change`. It also illustrates LSN type casting and subtraction.

```Elixir
Core.Repo.Local.query!("select CAST(pg_current_wal_insert_lsn() AS TEXT)")
```

```Elixir
Core.Repo.Local.query!("select CAST(pg_last_wal_replay_lsn() AS TEXT)")
```

```Elixir
Core.Repo.Local.query!("SELECT watch_for_lsn_change('7/A25801C8', 10);")
```

```Elixir
%Postgrex.Result{rows: [[lsn]]} = Core.Repo.Local.query!("select CAST(pg_current_wal_insert_lsn() AS TEXT)")

Core.Repo.Local.query!("SELECT watch_for_lsn_change('#{lsn}', 10);")
```

```Elixir
Core.Repo.Local.query!("SELECT pg_last_wal_replay_lsn() >= CAST('0/28B1A830' AS pg_lsn)")
```

```Elixir
Core.Repo.Local.query!("select pg_current_wal_insert_lsn() - '0/28B66750'::pg_lsn")
```

```Elixir
lsn = "0/28B66730"
Core.Repo.Local.query!("SELECT watch_for_lsn_change('#{lsn}'::pg_lsn, 2);")
Core.Repo.Local.query!("SELECT watch_for_lsn_change(NULL, 2);")
Core.Repo.Local.query!("SELECT watch_for_lsn_change($1::pg_lsn, 2);", [nil])
```

--------------------------------

### Elixir Project Release and Development Workflow

Source: https://github.com/superfly/fly_postgres_elixir/blob/main/DEV_NOTES.md

Essential `mix` commands for managing an Elixir project's release cycle, including testing, formatting, building, publishing to Hex.pm, and configuring development dependencies from Git.

```Elixir
mix test
```

```Elixir
mix format
```

```Elixir
mix hex.build
```

```Elixir
mix hex.publish
```

```Elixir
{:fly_rpc, git: "https://github.com/superfly/fly_rpc_elixir.git", branch: "dev-branch-name"},
```

```Elixir
mix deps.unlock fly_rpc
```

```Elixir
mix deps.update fly_rpc
```

--------------------------------

### Configure Ecto Repo with Fly.Postgres

Source: https://github.com/superfly/fly_postgres_elixir/blob/main/README.md

Modify your `Ecto.Repo` to use `Fly.Repo` and dynamically configure the database URL based on the runtime environment. This enables primary/replica connection handling.

```Elixir
defmodule MyApp.Repo.Local do
  use Ecto.Repo,
    otp_app: :my_app,
    adapter: Ecto.Adapters.Postgres

  @env Mix.env()

  # Dynamically configure the database url based on runtime and build
  # environments.
  def init(_type, config) do
    Fly.Postgres.config_repo_url(config, @env)
  end
end

defmodule MyApp.Repo do
  use Fly.Repo, local_repo: MyApp.Repo.Local
end
```

--------------------------------

### PostgreSQL Write-Ahead Log (WAL) Functions and Datatype Reference

Source: https://github.com/superfly/fly_postgres_elixir/blob/main/DEV_NOTES.md

Detailed API documentation for PostgreSQL functions (`pg_current_wal_insert_lsn`, `pg_last_wal_replay_lsn`) and the `pg_lsn` datatype, outlining their purpose, return values, and behavioral characteristics.

```APIDOC
pg_current_wal_insert_lsn():
  - Purpose: Call for the new LSN after a modification (Create, Update, Delete).
  - Returns: Always returns an LSN value.

pg_last_wal_replay_lsn():
  - Purpose: Call on a replica to get the most recent LSN that was replicated locally.
  - Returns: An LSN value on replica DBs; nil on Primary DB (where it does not receive replication).

pg_lsn (Postgres datatype):
  - Characteristics: Not directly comparable (cannot use < or >).
  - Operations: Can subtract two LSN values to get the difference as a DOUBLE/float.
  - Comparison: The difference can be compared (0 = equal).
```

--------------------------------

### Run Ecto migrations

Source: https://github.com/superfly/fly_postgres_elixir/blob/main/README.md

Execute the Ecto migrations to apply the database changes, including the `Fly.Postgres` stored procedure.

```Shell
mix ecto.migrate
```

--------------------------------

### Add Fly.Postgres dependency to mix.exs

Source: https://github.com/superfly/fly_postgres_elixir/blob/main/README.md

Add the `fly_postgres` dependency to your Elixir project's `mix.exs` file to include it in your application. This also pulls in `fly_rpc` as a dependency.

```Elixir
def deps do
  [
    {:fly_postgres, "~> 0.3.0"}
  ]
end
```

--------------------------------

### List Fly.io Backup Regions

Source: https://github.com/superfly/fly_postgres_elixir/blob/main/README.md

This command displays the current list of regions configured as backup regions for your Fly.io application. It helps in understanding where your application instances might be deployed by default, which can impact fly_postgres assumptions.

```shell
fly regions backup list
```

--------------------------------

### Elixir Application Supervision Tree Update for Multiple Ecto Repos

Source: https://github.com/superfly/fly_postgres_elixir/blob/main/README.md

This Elixir code illustrates the necessary adjustments to the application's supervision tree when an application utilizes multiple `Ecto.Repo` instances with `Fly.Postgres`. Each repository requires its own `Fly.Postgres.LSN.Supervisor` instance, which can be uniquely identified using the `name` option for better management and tracking.

```Elixir
defmodule MyApp.Application do
  use Application

  def start(_type, _args) do
    # ...

    children = [
      # Start the RPC server
      {Fly.RPC, []},
      # Start Ecto repositories
      MyApp.Repo.Local_1,
      MyApp.Repo.Local_2,
      # Start the supervisor for LSN tracking and name them.
      {Fly.Postgres.LSN.Supervisor, repo: MyApp.Repo.Local_1, name: :repo_tracker_1},
      {Fly.Postgres.LSN.Supervisor, repo: MyApp.Repo.Local_2, name: :repo_tracker_2},
      #...
    ]

    # ...
  end
end
```

--------------------------------

### Implement Fly.Postgres stored procedure migration

Source: https://github.com/superfly/fly_postgres_elixir/blob/main/README.md

Define the `up` and `down` functions in the generated Ecto migration to call the `Fly.Postgres.Migrations.V01` functions, which create and drop the necessary stored procedure.

```Elixir
defmodule MyApp.Repo.Local.Migrations.AddFlyPostgresProc do
  use Ecto.Migration

  def up do
    Fly.Postgres.Migrations.V01.up()
  end

  def down do
    Fly.Postgres.Migrations.V01.down()
  end
end
```

--------------------------------

### SQL Casting Text to PostgreSQL LSN Type

Source: https://github.com/superfly/fly_postgres_elixir/blob/main/DEV_NOTES.md

Demonstrates the SQL syntax for converting a text string representation of an LSN into the native `pg_lsn` datatype.

```SQL
CAST('0/28B1A830' AS pg_lsn)
```

--------------------------------

### Distribute Fly.io Instances Evenly Across Regions

Source: https://github.com/superfly/fly_postgres_elixir/blob/main/README.md

This command scales your Fly.io application to a specified instance count while ensuring an even distribution across regions. The `--max-per-region 1` flag prevents multiple instances from deploying to the same region, which is crucial for applications like those using fly_postgres that rely on instances being in specific primary or replica regions.

```shell
fly scale count 2 --max-per-region 1
```

--------------------------------

### Set Specific Fly.io Backup Regions

Source: https://github.com/superfly/fly_postgres_elixir/blob/main/README.md

This command explicitly sets the allowed backup regions for your Fly.io application. By specifying regions like `lax` and `syd`, you ensure that deployments only occur within these defined regions, preventing instances from appearing in unintended backup regions and maintaining fly_postgres assumptions.

```shell
fly regions backup lax syd
```

--------------------------------

### Override Ecto migration path for Fly.Postgres

Source: https://github.com/superfly/fly_postgres_elixir/blob/main/README.md

Configure your application to ensure Ecto migrations are generated in the correct `priv/repo` directory after renaming your main `Ecto.Repo`.

```Elixir
config :my_app, MyApp.Repo.Local,
  priv: "priv/repo"
```

--------------------------------

### Generate Ecto migration for Fly.Postgres stored procedure

Source: https://github.com/superfly/fly_postgres_elixir/blob/main/README.md

Generate a new Ecto migration file to add the `watch_for_lsn_change` stored procedure required by `Fly.Postgres`.

```Shell
mix ecto.gen.migration add_fly_postgres_proc
```

--------------------------------

### Configure Primary Region in fly.toml for Multi-Region Deployment

Source: https://github.com/superfly/fly_postgres_elixir/blob/main/README.md

This YAML configuration snippet for `fly.toml` sets the `PRIMARY_REGION` environment variable. This variable is crucial for applications deployed across multiple Fly.io regions, as it designates the region where the primary PostgreSQL database resides, ensuring optimal write performance.

```YAML
[env]
  PRIMARY_REGION = "syd"
```

--------------------------------

### Elixir: Asynchronous Database Modifications with await: false

Source: https://github.com/superfly/fly_postgres_elixir/blob/main/README.md

Explains how to perform database modifications (insert, update, delete) without waiting for local replication. This is useful when immediate consistency is not required or for fire-and-forget operations, allowing the function to continue execution without blocking for replication.

```Elixir
MyApp.Repo.insert(changeset, await: false)

MyApp.Repo.update(changeset, await: false)

MyApp.Repo.delete(item, await: false)
```

--------------------------------

### Elixir: Execute Complex Database Operations on Primary Region with Replication Wait

Source: https://github.com/superfly/fly_postgres_elixir/blob/main/README.md

Describes how to explicitly execute a function containing multiple database operations on the primary region to improve performance for complex business logic. The call blocks until any relevant database changes are replicated locally, ensuring data consistency.

```Elixir
Fly.Postgres.rpc_and_wait(MyModule, :do_complex_work, [arg1, arg2])
```

--------------------------------

### Elixir: Execute Remote Procedure Call on Primary Region Without Replication Wait

Source: https://github.com/superfly/fly_postgres_elixir/blob/main/README.md

Shows how to use the `Fly.RPC` library directly to perform a remote procedure call on the primary region without waiting for database replication. This is suitable for fire-and-forget tasks or when the function result is sufficient without needing local data consistency immediately.

```Elixir
Fly.RPC.rpc_primary({MyModule, :do_work, [arg1, arg2]})

Fly.RPC.rpc_region(:primary, {MyModule, :do_work, [arg1, arg2]})
```
Tokens:
Raw