### Start DataFetcher with custom options Source: https://hexdocs.pm/data_fetcher/DataFetcher-function-child_spec.html Example of starting a DataFetcher as a child in a supervisor. This snippet shows how to set a unique name, a custom fetcher function, and a specific interval. ```elixir defp my_fetcher_spec, do: { DataFetcher, name: :my_fetcher, fetcher: &MyWorker.fetch/1, interval: :timer.minutes(20) } ``` -------------------------------- ### DataFetcher Example Usage in Application Source: https://hexdocs.pm/data_fetcher/DataFetcher-function-child_spec.html An example of how to integrate the DataFetcher into your application's supervision tree. ```APIDOC ## Example Usage ### Description This example shows how to include a DataFetcher child spec in your `MyApp.Application`'s children list. ### Code ```elixir # lib/my_app/application.ex defmodule MyApp.Application do use Application @impl true def start(_type, _args) do children = [ my_fetcher_spec(), ... ] opts = [strategy: :one_for_one, name: MySupervisor] Supervisor.start_link(children, opts) end defp my_fetcher_spec, do: { DataFetcher, name: :my_fetcher, fetcher: &MyWorker.fetch/1, interval: :timer.minutes(20) } end ``` ``` -------------------------------- ### DataFetcher.child_spec/1 Source: https://hexdocs.pm/data_fetcher/DataFetcher.html Builds the supervisor child specification to start a data fetcher process. ```APIDOC ## child_spec(opts) ### Description Builds the supervisor child spec to start the data_fetcher. ### Parameters #### Request Body - **name** (atom) - Required - The identity of the job, should be unique. - **fetcher** (fetcher_def) - Required - Definition of how data is fetched from the source. Must return {:ok, data}. - **interval** (non_neg_integer) - Optional - Wait time in ms between fetches. Default is 10 minutes. - **cache_storage** (atom) - Optional - Cache storage adapter (e.g., ETS or persistent_term). Default is ETS. ``` -------------------------------- ### Implement DataFetcher in Application Source: https://hexdocs.pm/data_fetcher/DataFetcher.html Example of integrating DataFetcher into an Elixir application supervisor tree. ```elixir # lib/my_app/application.ex defmodule MyApp.Application do use Application @impl true def start(_type, _args) do children = [ my_fetcher_spec(), ... ] opts = [strategy: :one_for_one, name: MySupervisor] Supervisor.start_link(children, opts) end defp my_fetcher_spec, do: { DataFetcher, name: :my_fetcher, fetcher: &MyWorker.fetch/1, interval: :timer.minutes(20) } end ``` -------------------------------- ### Retrieve fetcher result Source: https://hexdocs.pm/data_fetcher/DataFetcher-function-result.html Demonstrates starting a DataFetcher supervisor and retrieving the result by name. The function waits for completion if the fetch is in progress. ```elixir iex> opts = [ ...> name: :function_fetcher, ...> fetcher: fn -> {:ok, %{foo: 1}} end ...> ] ...> ...> {:ok, _} = Supervisor.start_link([{DataFetcher, opts}], strategy: :one_for_one) ...> ...> DataFetcher.result(:function_fetcher) %{foo: 1} ``` -------------------------------- ### DataFetcher - child_spec/1 Source: https://hexdocs.pm/data_fetcher/index.html Builds the supervisor child specification for starting a DataFetcher process. It supports various options for configuring the fetcher's behavior, including name, fetcher definition, interval, and cache storage. ```APIDOC ## POST /api/users ### Description Build the supervisor child spec to start the data_fetcher. ### Method POST ### Endpoint /api/users ### Parameters #### Path Parameters - **userId** (integer) - Required - The ID of the user to retrieve. #### Query Parameters - **include_details** (boolean) - Optional - Whether to include detailed user information. #### Request Body - **username** (string) - Required - The username for the new user. - **email** (string) - Required - The email address for the new user. ### Request Example ```json { "username": "johndoe", "email": "john.doe@example.com" } ``` ### Response #### Success Response (200) - **id** (integer) - The unique identifier for the created user. - **username** (string) - The username of the created user. - **email** (string) - The email address of the created user. #### Response Example ```json { "id": 123, "username": "johndoe", "email": "john.doe@example.com" } ``` ``` -------------------------------- ### Get data from cache Source: https://hexdocs.pm/data_fetcher/DataFetcher.CacheStorage-callback-get.html Retrieves data from the cache by its name. Returns the cached result or nil if not found. ```Elixir get(name :: atom()) :: {:ok, result :: any()} | nil ``` -------------------------------- ### DataFetcher Result Example Source: https://hexdocs.pm/data_fetcher/index.html Retrieves the result of a data fetch job. If the data is still being fetched on the first round, it waits for successful completion. Otherwise, it returns cached data immediately. ```elixir iex> opts = [ ...> name: :function_fetcher, ...> fetcher: fn -> {:ok, %{foo: 1}} end ...> ] ...> ...> {:ok, _} = Supervisor.start_link([{DataFetcher, opts}], strategy: :one_for_one) ... ...> DataFetcher.result(:function_fetcher) %{foo: 1} ``` -------------------------------- ### DataFetcher child_spec Function Source: https://hexdocs.pm/data_fetcher/DataFetcher-function-child_spec.html This section describes the `child_spec` function of the DataFetcher module, which is used to build a supervisor child specification for starting the data fetcher. ```APIDOC ## DataFetcher.child_spec/1 ### Description Build the supervisor child spec to start the data_fetcher. ### Method N/A (Function definition) ### Endpoint N/A ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Request Body N/A ### Supported Options - **`:name`** (atom) - Required - The identity of the job, should be unique. - **`:fetcher`** (module function) - Required - Definition of how data is fetched from the source. A fetcher must return `{:ok, data}` if successful. - **`:interval`** (integer) - Optional - A value in `ms` indicating how long it waits until next fetching. The execution time will not affect the interval, neither does the failure of a fetching. 10 minutes by default. - **`:cache_storage`** (module) - Optional - What cache storage adapter to store the data. Ets by default. See Cache Storage section below. ### Request Example N/A ### Response #### Success Response (200) N/A #### Response Example N/A ``` -------------------------------- ### DataFetcher.CacheStorage.init Source: https://hexdocs.pm/data_fetcher/DataFetcher.CacheStorage-callback-init.html Initializes the cache storage with a given keyword. ```APIDOC ## init(keyword) ### Description Initializes the cache storage. ### Method N/A (Callback function) ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Request Body N/A ### Request Example N/A ### Response #### Success Response (200) `:ok` - Indicates successful initialization. #### Error Response `{:error, any()}` - Indicates an error during initialization, with details provided in the tuple. #### Response Example ``` :ok ``` ``` {:error, "Initialization failed"} ``` ``` -------------------------------- ### Initialize cache storage Source: https://hexdocs.pm/data_fetcher/DataFetcher.CacheStorage.html Initializes the cache storage with a given keyword. Returns :ok on success or an error tuple. ```elixir init(keyword()) :: :ok | {:error, any()} ``` -------------------------------- ### DataFetcher.CacheStorage.PersistentTerm Source: https://hexdocs.pm/data_fetcher/DataFetcher.CacheStorage.PersistentTerm.html Overview of the PersistentTerm storage adapter implementation. ```APIDOC ## PersistentTerm Storage Adapter ### Description The `DataFetcher.CacheStorage.PersistentTerm` module serves as a storage adapter for caching mechanisms within the `data_fetcher` library (v0.1.0). It utilizes Erlang's persistent_term for efficient, read-heavy data storage. ``` -------------------------------- ### DataFetcher.CacheStorage Module Source: https://hexdocs.pm/data_fetcher/api-reference.html Defines the cache storage behavior. ```APIDOC ## DataFetcher.CacheStorage Module ### Description Cache storage behaviour. ### Sub-modules - DataFetcher.CacheStorage.Ets - DataFetcher.CacheStorage.PersistentTerm ``` -------------------------------- ### DataFetcher.CacheStorage Callbacks Source: https://hexdocs.pm/data_fetcher/DataFetcher.CacheStorage.html This section outlines the available callbacks for interacting with the CacheStorage. ```APIDOC ## GET /cache_storage/get ### Description Retrieves a cached result by its name. ### Method GET ### Endpoint /cache_storage/get ### Parameters #### Query Parameters - **name** (atom) - Required - The name of the cached item to retrieve. ### Response #### Success Response (200) - **result** (any) - The cached result if found. #### Response Example ```json { "result": "some_value" } ``` ## INIT /cache_storage/init ### Description Initializes the cache storage with a given keyword. ### Method POST ### Endpoint /cache_storage/init ### Parameters #### Request Body - **keyword** (keyword()) - Required - The keyword to use for initialization. ### Response #### Success Response (200) - **status** (atom) - Indicates success (:ok) or failure (:error). #### Response Example ```json { "status": ":ok" } ``` ## PUT /cache_storage/put ### Description Stores a result in the cache storage with a given name. ### Method PUT ### Endpoint /cache_storage/put ### Parameters #### Request Body - **name** (atom) - Required - The name to associate with the cached result. - **result** (any) - Required - The result to cache. ### Response #### Success Response (200) - **status** (atom) - Indicates success (:ok) or failure (:error). #### Response Example ```json { "status": ":ok" } ``` ``` -------------------------------- ### Define option type Source: https://hexdocs.pm/data_fetcher/DataFetcher.html Defines the configuration options available for the fetcher. ```elixir option() :: {:name, atom()} | {:cache_storage, atom()} | {:fetcher, fetcher_def()} | {:interval, ms :: non_neg_integer()} ``` -------------------------------- ### DataFetcher Module Source: https://hexdocs.pm/data_fetcher/api-reference.html Provides an abstraction on periodic data fetching jobs. ```APIDOC ## DataFetcher Module ### Description An abstraction on periodic data fetching jobs. ### Sub-modules - DataFetcher.CacheStorage ``` -------------------------------- ### DataFetcher.CacheStorage.Ets Adapter Source: https://hexdocs.pm/data_fetcher/api-reference.html Implements ETS storage adapter for cache. ```APIDOC ## DataFetcher.CacheStorage.Ets Adapter ### Description Ets storage adapter ``` -------------------------------- ### DataFetcher Cache Storage Configuration Source: https://hexdocs.pm/data_fetcher/DataFetcher-function-child_spec.html Details on configuring the cache storage for the DataFetcher, including default ETS and alternative PersistentTerm adapter. ```APIDOC ## DataFetcher Cache Storage ### Description When data is fetched from the source, it will be saved in the cache storage. By default it uses an ETS table so that later reads can directly read from the ETS table. This is adequate for most cases unless the data size is huge. In such large data cases, copying and messaging between processes can be expensive and have performance penalties. If the data rarely or never changes, you can improve the performance in such cases by employing the `persistent_term` adapter, which is faster at reading no matter how large the data is. Be aware that on the other hand, writing into the cache storage is slower for `persistent_term` because global GC will happen for every write. ### Configuration #### Global Configuration ```elixir config :data_fetcher, :cache_storage, DataFetcher.CacheStorage.PersistentTerm ``` #### Per-Fetcher Configuration Use the `:cache_storage` option when defining the fetcher: ```elixir {DataFetcher, [..., cache_storage: DataFetcher.CacheStorage.PersistentTerm, ...]}` ``` ``` -------------------------------- ### DataFetcher.CacheStorage.PersistentTerm Adapter Source: https://hexdocs.pm/data_fetcher/api-reference.html Implements persistent term storage adapter for cache. ```APIDOC ## DataFetcher.CacheStorage.PersistentTerm Adapter ### Description persistent_term storage adapter ``` -------------------------------- ### Configure DataFetcher cache storage globally Source: https://hexdocs.pm/data_fetcher/DataFetcher-function-child_spec.html Change the cache storage adapter globally for DataFetcher. This is useful for large datasets where persistent_term offers better read performance. ```elixir config :data_fetcher, :cache_storage, DataFetcher.CacheStorage.PersistentTerm ``` -------------------------------- ### Put data into cache Source: https://hexdocs.pm/data_fetcher/DataFetcher.CacheStorage.html Stores a result in the cache associated with a given name. Returns :ok on success or an error tuple. ```elixir put(name :: atom(), result :: any()) :: :ok | {:error, any()} ``` -------------------------------- ### DataFetcher.CacheStorage.get Source: https://hexdocs.pm/data_fetcher/DataFetcher.CacheStorage-callback-get.html Retrieves cached data by its name. Returns the cached result if found, otherwise returns nil. ```APIDOC ## DataFetcher.CacheStorage.get ### Description Retrieves cached data by its name. Returns the cached result if found, otherwise returns nil. ### Method GET (implied, as it's a retrieval operation) ### Endpoint /websites/hexdocs_pm_data_fetcher/cache_storage/get ### Parameters #### Path Parameters - **name** (atom) - Required - The name of the cache entry to retrieve. ### Response #### Success Response (200) - **result** (any) - The cached data if found. #### Response Example ```json {:ok, "cached_value"} ``` #### Nil Response - **nil** - Returned when the cache entry is not found. #### Nil Response Example ``` nil ``` ``` -------------------------------- ### put(name, result) Source: https://hexdocs.pm/data_fetcher/DataFetcher.CacheStorage-callback-put.html Stores a result in the cache associated with the given name. ```APIDOC ## put(name, result) ### Description Stores a result in the cache associated with the given name. ### Parameters - **name** (atom) - Required - The identifier for the cache entry. - **result** (any) - Required - The data to be stored. ### Response - **Success** (:ok) - Returned when the operation is successful. - **Error** ({:error, any()}) - Returned if the operation fails. ``` -------------------------------- ### Define result function Source: https://hexdocs.pm/data_fetcher/DataFetcher.html Defines the specification for retrieving the result of a fetch job. ```elixir result(fetcher_name :: atom()) :: any() ``` -------------------------------- ### Define child_spec function Source: https://hexdocs.pm/data_fetcher/DataFetcher.html Defines the supervisor child specification for the fetcher. ```elixir child_spec([option()]) :: Supervisor.child_spec() ``` -------------------------------- ### DataFetcher.result/1 Source: https://hexdocs.pm/data_fetcher/DataFetcher.html Retrieves the latest result of a specific fetcher job. ```APIDOC ## result(name) ### Description Get result of the fetch. If the data is still being fetched at the first round, it will wait until the data is successfully fetched before returning the result. ### Parameters #### Path Parameters - **name** (atom) - Required - The identifier of the fetch job. ### Response #### Success Response (200) - **result** (any) - The fetched data stored in the cache. ``` -------------------------------- ### Define fetcher_def type Source: https://hexdocs.pm/data_fetcher/DataFetcher.html Defines the specification for a fetcher function or module. ```elixir fetcher_def() :: (() -> {:ok, any()}) | mfa() | module() ``` -------------------------------- ### DataFetcher.result API Source: https://hexdocs.pm/data_fetcher/DataFetcher-function-result.html Retrieves the result of a data fetch operation. If the data is still being fetched, it waits for completion. Otherwise, it returns cached data. ```APIDOC ## DataFetcher.result ### Description Retrieves the result of a data fetch operation. If the data is still being fetched, it waits for completion. Otherwise, it returns cached data. ### Method GET (conceptual, as it's a function call) ### Endpoint N/A (Function call within a module) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters - **name** (atom) - Required - The identifier of the fetch job. ### Returns - **any** - The result of the data fetch operation. ### Response Example ```json { "example": "%{foo: 1}" } ``` ### Example ```elixir opts = [ name: :function_fetcher, fetcher: fn -> {:ok, %{foo: 1}} end ] {:ok, _} = Supervisor.start_link([{DataFetcher, opts}], strategy: :one_for_one) DataFetcher.result(:function_fetcher) # => %{foo: 1} ``` ``` -------------------------------- ### DataFetcher - result/1 Source: https://hexdocs.pm/data_fetcher/index.html Retrieves the result of a data fetch job. If the data is currently being fetched, this function will wait for the fetch to complete. Otherwise, it returns the cached data immediately. ```APIDOC ## GET /api/users/{userId} ### Description Get result of the fetch. ### Method GET ### Endpoint /api/users/{userId} ### Parameters #### Path Parameters - **name** (atom) - Required - The identifier of the fetch job. ### Response #### Success Response (200) - **any** - The result of the fetch operation. ### Response Example ```json { "data": "some_fetched_data" } ``` ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.