### Content Migration with Html2Markdown Source: https://hexdocs.pm/html2markdown/index.html Convert existing HTML content to Markdown, with an example of normalizing whitespace during the conversion process. ```elixir # Convert blog posts from HTML to Markdown html_content |> Html2Markdown.convert(%{normalize_whitespace: true}) |> save_as_markdown() ``` -------------------------------- ### Html2Markdown Default Options Function Source: https://hexdocs.pm/html2markdown/Html2Markdown.Options.html Provides the default configuration map for Html2Markdown. Call this function to get a baseline set of options before customization. ```elixir @spec defaults() :: t() ``` -------------------------------- ### Email Processing with Html2Markdown Source: https://hexdocs.pm/html2markdown/Html2Markdown.html Clean up HTML emails by converting them to Markdown. This example shows how to specify tags and classes to exclude during conversion, such as styles and unsubscribe links. ```elixir # Clean up HTML emails email_body |> Html2Markdown.convert(%{ non_content_tags: ["style", "meta", "link"], navigation_classes: ["unsubscribe", "footer"] }) ``` -------------------------------- ### block_element?(tag) Source: https://hexdocs.pm/html2markdown/Html2Markdown.ElementTypes.html Checks if an HTML element is a block-level element. Block elements typically start on a new line and should be separated by blank lines in Markdown. ```APIDOC ## block_element?(tag) ### Description Determines if an HTML element is a block-level element. Block elements typically start on a new line and take up the full width available. They should be separated by blank lines in markdown. ### Method `Html2Markdown.ElementTypes.block_element?(tag)` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example None ### Response #### Success Response (boolean) * `true`: if the element is a block-level element. * `false`: if the element is not a block-level element. #### Response Example ```elixir "p" ``` ```elixir Html2Markdown.ElementTypes.block_element?("p") #=> true Hmtl2Markdown.ElementTypes.block_element?("span") #=> false ``` ``` -------------------------------- ### Html2Markdown Default and Custom Options Source: https://hexdocs.pm/html2markdown/Html2Markdown.Options.html Demonstrates how to initialize default options and merge custom configurations with them. Use this to override specific settings like navigation classes or whitespace normalization. ```elixir options = Options.defaults() # Merge custom options with defaults custom = Options.merge(%{ navigation_classes: ["custom-nav", "advertisement"], normalize_whitespace: false }) ``` -------------------------------- ### Run project tests Source: https://hexdocs.pm/html2markdown/readme.html Execute the test suite and generate coverage reports. ```bash # Run all tests mix test # Run tests with coverage mix coveralls.html ``` -------------------------------- ### Run tests with coverage Source: https://hexdocs.pm/html2markdown/index.html Execute all tests and generate a code coverage report in HTML format. ```bash mix coveralls.html ``` -------------------------------- ### Run all tests Source: https://hexdocs.pm/html2markdown/index.html Execute all tests in the project to ensure code quality and functionality. ```bash mix test ``` -------------------------------- ### Html2Markdown Configuration Options Source: https://hexdocs.pm/html2markdown/index.html Demonstrates various configuration options for Html2Markdown, including navigation classes, non-content tags, markdown flavor, and whitespace normalization. ```elixir Html2Markdown.convert(html, %{ # CSS classes that identify navigation elements to remove navigation_classes: ["footer", "menu", "nav", "sidebar", "aside"], # HTML tags to filter out during conversion non_content_tags: ["script", "style", "form", "nav", ...], # Markdown flavor (currently :basic, future: :gfm, :commonmark) markdown_flavor: :basic, # Normalize whitespace (collapses multiple spaces, trims) normalize_whitespace: true }) ``` -------------------------------- ### Run all quality checks Source: https://hexdocs.pm/html2markdown/index.html Perform a comprehensive set of quality checks, including formatting, security analysis, and linting. ```bash mix quality ``` -------------------------------- ### Check code formatting Source: https://hexdocs.pm/html2markdown/index.html Verify that the code adheres to the project's formatting standards without making changes. ```bash mix format --check-formatted ``` -------------------------------- ### Convert HTML to Markdown with custom navigation classes Source: https://hexdocs.pm/html2markdown/Html2Markdown.html Demonstrates converting an HTML string to Markdown while specifying custom navigation classes to be removed. ```elixir iex> Html2Markdown.convert("
Hello
", %{navigation_classes: ["custom-nav"]}) "Hello" ``` -------------------------------- ### Run code quality checks Source: https://hexdocs.pm/html2markdown/readme.html Execute formatting, linting, and security analysis tools. ```bash # Run all quality checks (formatting, security, linting) mix quality # Individual checks mix format --check-formatted # Code formatting mix credo --only warning # Code linting mix sobelow --config # Security analysis ``` -------------------------------- ### Code linting with Credo Source: https://hexdocs.pm/html2markdown/index.html Run Credo for code linting, focusing on warnings to identify potential issues. ```bash mix credo --only warning ``` -------------------------------- ### Security analysis with Sobelow Source: https://hexdocs.pm/html2markdown/index.html Perform a security analysis of the codebase using Sobelow with the default configuration. ```bash mix sobelow --config ``` -------------------------------- ### Html2Markdown.Options Source: https://hexdocs.pm/html2markdown/api-reference.html Manages and applies configuration options for the HTML to Markdown conversion process. ```APIDOC ## Html2Markdown.Options ### Description Handles configuration options for HTML to Markdown conversion. ``` -------------------------------- ### Content Migration with Html2Markdown Source: https://hexdocs.pm/html2markdown/Html2Markdown.html Convert HTML content, such as WordPress posts, to Markdown files. This snippet demonstrates piping the HTML through Html2Markdown and writing the output to a file. ```elixir # Convert WordPress posts to Markdown post_html |> Html2Markdown.convert() |> File.write!("post.md") ``` -------------------------------- ### Html2Markdown Conversion Options Source: https://hexdocs.pm/html2markdown/Html2Markdown.html This section details the options available for the `Html2Markdown.convert` function. ```APIDOC ## __Options * `:navigation_classes` - List of CSS classes to identify navigation elements to remove. Defaults to `["footer", "menu", "nav", "sidebar", "aside"]` * `:non_content_tags` - List of HTML tags to filter out during conversion. Defaults to common non-content tags like script, style, form, etc. * `:markdown_flavor` - Markdown flavor to use. Currently only `:basic` is supported. Defaults to `:basic` (future enhancement for `:gfm`, `:commonmark`) * `:normalize_whitespace` - Whether to normalize whitespace. When enabled, multiple spaces/tabs are converted to single spaces and leading/trailing whitespace is trimmed. Whitespace in code blocks and inline code is always preserved. Defaults to `true` ## __Examples ``` iex> Html2Markdown.convert("Hello
", %{navigation_classes: ["custom-nav"]}) "Hello" ``` ``` -------------------------------- ### Add html2markdown to dependencies Source: https://hexdocs.pm/html2markdown/index.html Add the html2markdown library to your project's dependencies in the mix.exs file. ```elixir def deps do [ {:html2markdown, "~> 0.3.1"} ] end ``` -------------------------------- ### Configurable HTML to Markdown Conversion Source: https://hexdocs.pm/html2markdown/Html2Markdown.html Customize the conversion process by providing a map of options. This allows fine-grained control over filtering and formatting. ```elixir Html2Markdown.convert(html, %{ navigation_classes: ["nav", "menu", "sidebar"], non_content_tags: ["script", "style", "iframe"], markdown_flavor: :basic, normalize_whitespace: true }) ``` -------------------------------- ### Convert HTML to Markdown Source: https://hexdocs.pm/html2markdown/readme.html Perform basic conversion or use custom options to filter content and normalize whitespace. ```elixir # Basic conversion Html2Markdown.convert("Welcome to Elixir!
") # => "\n# Hello World\n\n\n\nWelcome to **Elixir**!\n" # With custom options Html2Markdown.convert(html, %{ navigation_classes: ["nav", "menu", "custom-nav"], normalize_whitespace: true }) ``` -------------------------------- ### Convert HTML table to Markdown Source: https://hexdocs.pm/html2markdown/Html2Markdown.TableConverter.html Demonstrates the transformation of a basic HTML table structure into GitHub Flavored Markdown. ```markdown # Simple table| Name | Age |
|---|---|
| Alice | 30 |
Welcome to Elixir!
") # => "\n# Hello World\n\n\n\nWelcome to **Elixir**!\n" ``` -------------------------------- ### Html2Markdown Library Functions Source: https://hexdocs.pm/html2markdown/Html2Markdown.html Documentation for the core functions of the Html2Markdown Elixir library. ```APIDOC ## Html2Markdown Module ### Description Provides functions to convert HTML documents to clean, readable Markdown. ### Functions #### `convert(html_content)` ##### Description Converts the content from an HTML document to Markdown using default options. It intelligently extracts content while filtering out non-content elements. ##### Parameters - **html_content** (html_content()) - Required - The HTML string to convert. ##### Returns - **markdown_content()** - The converted Markdown string. ##### Example ```elixir MixedCase = Html2Markdown.convert("World
") # => "# Hello\n\nWorld" ``` #### `convert(html_content, options)` ##### Description Converts the content from an HTML document to Markdown with custom configuration options. ##### Parameters - **html_content** (html_content()) - Required - The HTML string to convert. - **options** (conversion_options()) - Optional - A map of options to customize the conversion process. - `navigation_classes`: List of CSS classes to identify navigation elements. - `non_content_tags`: List of HTML tags to exclude. - `markdown_flavor`: Specifies the Markdown flavor (:basic or :gfm). - `normalize_whitespace`: Boolean to enable or disable whitespace normalization. ##### Returns - **markdown_content()** - The converted Markdown string. - **{:error, String.t()}** - An error tuple if conversion fails. ##### Example ```elixir options = %{ navigation_classes: ["nav", "menu", "sidebar"], non_content_tags: ["script", "style", "iframe"], markdown_flavor: :basic, normalize_whitespace: true } converted_content = Html2Markdown.convert("Content
", options) # => "Content" ``` ### Types #### `html_content()` ```elixir @type html_content() :: String.t() ``` #### `markdown_content()` ```elixir @type markdown_content() :: String.t() ``` #### `conversion_options()` ```elixir @type conversion_options() :: %{ optional(:navigation_classes) => [String.t()], optional(:non_content_tags) => [String.t()], optional(:markdown_flavor) => :basic | :gfm, optional(:normalize_whitespace) => boolean() } ``` ``` -------------------------------- ### process_node/2 Source: https://hexdocs.pm/html2markdown/Html2Markdown.Converter.html Processes a single HTML node and converts it to Markdown. ```APIDOC ## process_node(node, opts) ### Description Processes a single HTML node and converts it to its Markdown equivalent. ### Parameters - **node** (Floki.html_node()) - Required - The HTML node to process. - **opts** (Html2Markdown.Options.t()) - Required - Configuration options for the conversion process. ### Response - **Returns** (String.t()) - The resulting Markdown formatted string. ``` -------------------------------- ### convert_to_markdown/2 Source: https://hexdocs.pm/html2markdown/Html2Markdown.Converter.html Converts a list of Floki HTML nodes into a Markdown string. ```APIDOC ## convert_to_markdown(document, opts) ### Description Converts a list of Floki HTML nodes into a Markdown string based on provided options. ### Parameters - **document** ([Floki.html_node()]) - Required - The list of parsed HTML nodes to convert. - **opts** (Html2Markdown.Options.t()) - Required - Configuration options for the conversion process. ### Response - **Returns** (String.t()) - The resulting Markdown formatted string. ``` -------------------------------- ### Html2Markdown Options API Source: https://hexdocs.pm/html2markdown/Html2Markdown.Options.html This section covers the Html2Markdown.Options module, detailing its available configuration settings and functions for managing these options. ```APIDOC ## Html2Markdown.Options (html2markdown v0.3.1) Handles configuration options for HTML to Markdown conversion. ### Available Options * `:navigation_classes` - List of CSS classes that identify navigation elements to remove. Default: `["footer", "menu", "nav", "sidebar", "aside"]` * `:non_content_tags` - List of HTML tags to filter out completely. Default includes tags like `script`, `style`, `iframe`, etc. * `:markdown_flavor` - The markdown variant to generate. Currently only `:basic` is supported. Future versions may support `:gfm` and `:commonmark`. * `:normalize_whitespace` - Whether to collapse multiple spaces and trim whitespace. Default: `true`. Code blocks always preserve whitespace regardless of this setting. ### Functions #### `defaults()` Returns the default options map. ```elixir @spec defaults() :: t() ``` #### `merge(user_options)` Merges user options with defaults. ```elixir @spec merge(map()) :: t() ``` ### Types #### `t()` Represents the structure of the options map. ```elixir @type t() :: %{ navigation_classes: [String.t()], non_content_tags: [String.t], markdown_flavor: :basic | :gfm, normalize_whitespace: boolean() } ``` ### Examples ```elixir # Use all defaults options = Options.defaults() # Merge custom options with defaults custom = Options.merge(%{ navigation_classes: ["custom-nav", "advertisement"], normalize_whitespace: false }) ``` ``` -------------------------------- ### Web Scraping with Html2Markdown Source: https://hexdocs.pm/html2markdown/Html2Markdown.html Extract article content from a web page using HTTPoison to fetch the HTML and Html2Markdown to convert it. Configure options to filter navigation and normalize whitespace. ```elixir # Extract article content from a web page {:ok, %{body: html}} = HTTPoison.get("https://example.com/article") content = Html2Markdown.convert(html, %{ navigation_classes: ["header", "footer", "nav", "sidebar"], normalize_whitespace: true }) ``` -------------------------------- ### HTML to Markdown Conversion with Custom Options Source: https://hexdocs.pm/html2markdown/index.html Convert HTML to Markdown with custom options, such as specifying navigation classes and normalizing whitespace. ```elixir # With custom options Html2Markdown.convert(html, %{ navigation_classes: ["nav", "menu", "custom-nav"], normalize_whitespace: true }) ``` -------------------------------- ### Html2Markdown Convert Function Signature (Default Options) Source: https://hexdocs.pm/html2markdown/Html2Markdown.html Specifies the function signature for converting HTML content to Markdown using default options. ```elixir @spec convert(html_content()) :: markdown_content() ``` -------------------------------- ### Html2Markdown.Parser Source: https://hexdocs.pm/html2markdown/api-reference.html Responsible for preprocessing and parsing HTML input before conversion. ```APIDOC ## Html2Markdown.Parser ### Description Handles HTML preprocessing and parsing operations. ``` -------------------------------- ### Convert HTML to Markdown Source: https://hexdocs.pm/html2markdown/Html2Markdown.Converter.html Converts a list of Floki HTML nodes into a Markdown string using the provided options. ```elixir @spec convert_to_markdown([Floki.html_node()], Html2Markdown.Options.t()) :: String.t() ``` -------------------------------- ### Web Scraping with Html2Markdown Source: https://hexdocs.pm/html2markdown/index.html Extract readable content from web pages by fetching HTML using Req and then converting it to Markdown with Html2Markdown. ```elixir {:ok, %{body: html}} = Req.get!(url) markdown = Html2Markdown.convert(html) ``` -------------------------------- ### Define preprocess_content specification Source: https://hexdocs.pm/html2markdown/Html2Markdown.Parser.html Specifies the function signature for preprocessing HTML content. ```elixir @spec preprocess_content(String.t(), Html2Markdown.Options.t()) :: html_tree() ``` -------------------------------- ### Html2Markdown Module Source: https://hexdocs.pm/html2markdown/api-reference.html The main module for converting HTML documents to clean, readable Markdown. ```APIDOC ## Html2Markdown Module ### Description Convert HTML documents to clean, readable Markdown. ### Modules - Html2Markdown.Converter - Html2Markdown.ElementTypes - Html2Markdown.Options - Html2Markdown.Parser - Html2Markdown.TableConverter ``` -------------------------------- ### Html2Markdown Convert Function Signature (Custom Options) Source: https://hexdocs.pm/html2markdown/Html2Markdown.html Specifies the function signature for converting HTML content to Markdown with custom conversion options. ```elixir @spec convert(html_content(), conversion_options()) :: markdown_content() ``` -------------------------------- ### Html2Markdown.Converter Source: https://hexdocs.pm/html2markdown/api-reference.html Handles the core logic for converting HTML nodes into Markdown format. ```APIDOC ## Html2Markdown.Converter ### Description Handles the conversion of HTML nodes to Markdown format. ``` -------------------------------- ### Html2Markdown Parser API Source: https://hexdocs.pm/html2markdown/Html2Markdown.Parser.html API documentation for the Html2Markdown.Parser module, including its types and functions. ```APIDOC ## Html2Markdown.Parser (html2markdown v0.3.1) Handles HTML preprocessing and parsing operations. This module is responsible for: 1. Parsing HTML content using Floki 2. Filtering out non-content elements 3. Preparing the document tree for conversion ### Filtering Strategy The parser removes elements in two ways: * **Tag-based filtering** : Removes elements like `